Log in

View Full Version : Reversing a algorithm: 4 bytes read - 8 bytes written

DakienDX
September 16th, 2001, 05:32
Hello anybody !

New messageboard, new reversing problems...
Well, we need to register now. Why not?

I've a little problem. I want to reverse an encryption algorithm, but don't know how this should be done in this chase. I've a feeling that it's an easy one, but my mind is out of order for this algo. I've tried two days to reverse it, but without any results.
Below is the decryption loop. The first 8 bytes of the encrypted data are unencrypted. They're used as start values for the other data blocks in a kind of CBC mode. My problem is: Only 4 bytes are read, but 8 bytes are written. In reversing, I should look like this
Code:


          Mov     EAX, [ESI]

          Mov     EDX, [ESI+4]

          Xor     EAX, [ESI+4]         ; I don't know the old values

          Xor     EAX, [ESI]           ; as above

But I don't have the old values of the memory locations.
Here's the Decryption Loop:
Code:


          Mov     ESI, CryptedData

          Mov     EDI, LengthOfCryptedData

          Shr     EDI, 3

Crypt1:

          Mov     EBX, [ESI+4]

          Add     EBX, [32BitPassword]

          Xor     EBX, [ESI]

          Add     ESI, 8

          Mov     ECX, 0Eh

Crypt2:

          Mov     EAX, [ESI]

          Mov     EDX, EAX

          Ror     EBX, 4

          Xor     EAX, EBX

          Add     EAX, 0A235832Ch

          Add     EAX, ECX

          Rol     EAX, CL

          Sub     EAX, EBX

          Xor     EAX, [ESI]

          Xor     EAX, [ESI+4]

          Mov     [ESI+4], EDX

          Mov     [ESI], EAX

          Dec     ECX

          Dec     ECX

          Jnl     Crypt2

          Dec     EDI

          Jne     Crypt1

I would be thankfull if somebody could give me a clue.
DakienDX
September 18th, 2001, 12:39
Hello everbody !

Seems I'm not the only one who has trouble with this algorithm. Or is it just to easy for your?

Well, to make it easier, here's a pseudo-code listing in chase you don't understand my ASM
Code:


  C  = 1

  PW = (DATA[C+1]+PASSWORD) XOR DATA[C]

1:

  C  = C+2

  L  = 0Eh

2:

  X  = DATA[C]

  Y  = DATA[C]

  PW = ROR(PW, 4)

  X  = (X XOR PW) + A235832Ch + L

  X  = ROL(X, L)

  X  = X - PW

  X  = X XOR DATA[C] XOR DATA[C+1]

  DATA[C+1] = Y

  DATA[C] = X

  L  = L-2

  REPEAT UNTIL (L < 0) 2:

  REPEAT UNTIL ENDOFDATA 1:
DakienDX
September 18th, 2001, 15:05
Hello Fake51 !

The target is HL-Server. The decryption algorithm is the one used in the target. I want to add some more network executions to HL-Server.
Well, my problem is not unpacking the target but repacking it. (Since I've already patched it!)
Because of this I'm trying to reverse the algorithm, since I want to add 2 patches to the protected version. I know the CBC-like code might make some trouble, but my first goal is to reverse it.
mike
September 18th, 2001, 16:48
What's your goal? Do you want to recover the 32-bit password?
DakienDX
September 19th, 2001, 10:59
Hello mike !

I don't want to recover the password. I have all passwords. (each PE-section is encrypted with it's own)
As I've written I want to repack the target to see which bytes have changed from the old one to the patched one.
Because of this I need to have the reversed algorithm to re-encrypt it.
This is no stream algorithm, it's a block algorithm, this means you have one proc for encrypting and one for decrypting.
mike
September 19th, 2001, 12:48
OK, the code is a Feistel cipher with 14 rounds (L in your pseudocode).

A feistel cipher works like this:

Split data into two halves, A and B

FOR R = 0 TO L
SWAP (A,B)
B = B XOR F (A, R PASSWORD)
NEXT R

Note that the code you posted goes the other way (this is the inner loop):

FOR R = L TO 0
B = B XOR F (A, R, PASSWORD)
SWAP (A,B)
NEXT R

Rewrite the decryption code you posted like this and it'll be a snap to change it into the encryption code.
mike
September 19th, 2001, 13:01
By the way, F in the code above is this:

F (A, R, PW) =

(ROR(A XOR PW + A23532Ch + R, R) - PW) XOR A

You have to also toss in those extra instructions for maniplulating PW (the initialization before and the rotates during) before calculating F.

So the whole thing looks like this

PW = (B + PASSWORD) XOR A
For R = 0 to L
SWAP(A,B)
PW = ROR(PW,4)
B=B XOR F(A,R,PW)
NEXT R
DakienDX
September 19th, 2001, 14:19
Hello mike !

I'm going to test the code. I've searched about 70 different crypting algorithms for this code, but like fate is, I haven't found it.

It looks like a cipher with 8 rounds to me. It starts at 14, decrements each round by 2 and ends at -2. Is this right or have I understood something wrong?

Is Feistel one special algorithm or just a name for a group of similar ones?
mike
September 19th, 2001, 19:52
You're right, it's an 8-round cipher. Sorry about that. You won't find it described anywhere because it's insecure. Differential cryptanalysis would make short work of this. Feistel ciphers are a class of ciphers. DES is probably the most famous Feistel cipher; Blowfish is another. Any cipher that splits the block into halves, xors a function of one half into the other & swaps halves is a Feistel cipher. They make them that way because they're trivially reversible.
DakienDX
September 20th, 2001, 13:27
Hello mike !

Thanks to you the code works now.
My problem was that I hadn't realised the SWAP(A, B) code and put it in the first line in encryption and decryption. So it had to fail.

Thanks again