Hello flexlm reversers,

I read Nolan Blender latest essay about flexlm - about user crypt filters.
I happen to know of an eda product which uses this crypt filter method. I followed Nolan's essay but came to a snag...I couldn't continue at the point of observing the crypt filter routines.

I already have valid encryption seeds for this product but requires lmkeyfil.c to generate the appopriate lmcrypt.

So, please can anyone (especially Nolan) provide me this lmkeyfil.c for this particular product.

Not to sound obvious : this product last uses vendor defined encryption but now uses crypt filters instead. I hope this is enough.

- kittykat
ps: I'm using gdb on Solaris and Linux. I've never reversed anything on the windows platform.

To reverse engineer this routine, you have
to take a dead listing approach to it rather
than try and step trace through the function -
it's designed to be difficult.
It's a bit more difficult on the MIPS processor,
but still doable. One way is to
build your program with lmappfil.c and
reverse engineer that first - once you do
that you'll see what sort of code is generated
and how the variables map to the generated code.
The key to this protection is to understand
what the program is actually doing to the data.
The lmkeyfil you need will be similar to the
one provided in the essay - all you need to
do is reverse the program and extract the
tables from the target. You'll see that
the first part of the program holds the XOR
operations, and the second, larger part does
the permutations.

Thanks Nolan, I will try again...but I still feel if you can provide me the lmkeyfil.c for this product - I will be more than grateful

Thanks again

- kittykat

Sheesh. Some people want everything!

Hi,

Where can I find Nolan's newest essay?

Thanks!

Hello Nolan,

I've mananged to identify the user crypt routine and I believe I can extract num0-num19 and its corresponding x_0-x_19, however the permutation operation still puzzles me.

In your alternate lmkeyfil.c, how did you get the permute_t table ?? Put it in another way, how did you get values for idx 00 to idx 19 ? It's not so clear in your essay as the xor operation.

Also, I tried changing the values in your alternate lmkeyfil.c for example x_19 - from 0x1a to 0x2a, build lmcrypt and still the code generated by lmcrypt is same as the previous(with your original altered lmkeyfil.c). Why is this so ??

- kittykat

> In your alternate lmkeyfil.c, how did you get the permute_t table ?? Put it in
> another way, how did you get values for idx 00 to idx 19 ? It's not so clear in
> your essay as the xor operation.

Well, the permutation operation isn't really all that clear without some careful
analysis.

04011C6 loc_4011C6: ; CODE XREF: sub_401000+1A4j
004011C6 cmp edi, myoffset_0d ; Is this index 0x0d?
004011CC jnz short loc_4011FD ; No, skip to next one
004011CE mov cl, byte ptr mybit_4 ; Move mask consisting of bit 4 (0x10) in place
004011D4 test byte ptr [ebp+arg_4+3], cl ; Is that bit set?
004011D7 jz short loc_4011E2
004011D9 mov cl, byte ptr mybit_6 ; Set bit 6 in cl , so bit 6 in cl
004011D9 ; corresponds to bit 4 in input
004011DF or [ebp+var_1], cl ; Turn bit 6 on if bit 4 on input was set
004011E2
004011E2 loc_4011E2: ; CODE XREF: sub_401000+1D7j
004011E2 mov dl, [ebp+var_1] ; Grab result so far
004011E5 mov ecx, mybit_6 ; Load ecx with bit 6 pattern (0x40)
004011EB mov bl, [ebp+arg_C] ; Grab the "expected char" fed into algorithm
004011EE and edx, ecx ; Mask off everything except bit 6
004011F0 and ebx, ecx
004011F2 cmp dl, bl ; Check to see if bit is the same
004011F4 jz short loc_4011FD ; Same - good continue on.
004011F6 xor al, 0DEh ; Otherwise, mix in garbage
004011F8 jmp loc_403217 ; And bail out

By looking at the above code fragment (this one is from realviz.exe), it shows that
bit 4 is mapped to bit 6 in the permutation. so 0x10 becomes 0x40 if passed through
the function.

I wrote the permute program, and it's operation isn't as clear as it could be. The
The position (from left to right) correspond to the bit position in the input word -
thus, the first value corresponds to bit0, second corresponds to bit1, and the last
corresponds to bit7. The value corresponds to the bit that the input bit maps to.

From realviz, we can take this example:

{ 2, 5, 1, 3, 6, 7, 0, 4}, /* idx 13 */

bit0 maps to bit2
bit1 maps to bit5
bit2 maps to bit1
bit3 maps to bit3
bit4 maps to bit6 **this is the mapping for the code fragment above **
bit5 maps to bit7
bit6 maps to bit0
bit7 maps to bit4



> Also, I tried changing the values in your alternate lmkeyfil.c for example x_19
> - from 0x1a to 0x2a, build lmcrypt and still the code generated by lmcrypt is
> same as the previous(with your original altered lmkeyfil.c). Why is this so ??

Well, the index is also used as the index into the key, so if the keylength is less
than 20, you won't see this algorithm being used. Try changing x_0 and see what
happens.

Thanks for all your help Nolan - everything's okay now -

- raif