Target: MFVdasm hxxp://redirect.to/MFVDasm Packed by Softlocx
Tools: IDA pro, Softice, dumpbin, regmon, filemon
Difficulty: intermediate

Initial analysis of the target yeilds the following findings:

1. The executable is packed by Softlocx and decompresses an executable
image insNNN.tmp in SYSTEM\TMP directory. The user registration is
handled by this executable which itself is packed by softlocx
protection by examing its source listing (around 244kB). The unpacker
uses unaligned jmp to fool disassembler but IDA was able to correctly
identify these commonly used techniques:

jmp near $+6
0xE9 <--- false MOV EAX, 4_bytes_after
push EBP <--- jmp destination
...

The unpacker allocates 2x3 pages of memory and does the unpacking, then
use rep movsb to 0x482c6 which is the next EIP address:

ebx = ebp+ (ds&4)? 17ac : 176f <-- a clever scheme is used to determine the base load address
<-- of this code entry point page address, i.e. ebp=0x00482000
<-- on w2k, the ds selector always ends up as 011y (gdt, ring3)
<-- so ebx = ebp+176f
call ebx
chunk1 = VirtualAlloc(0, 0x3000, MEM_COMMIT, PAGE_READWRITE)
eax = 1926+ebp
call eax
unpack(chunk1)
chunk2 = VirtualAlloc(0, 0x3000, MEM_COMMIT, PAGE_READWRITE)
size=unpack(chunk2, chunk1)

esi=chunk2
edi=0c6h <-- hardcoded
edi=edi+ebp
ecx=size
004820c4: rep movsb
004820c6: Start of self modifying code, dest of rep movsb from chunk2

An unpacker could be built based on the above analysis.

2. The installation image insNNN.tmp modify registry settings and
create a file called SYSTEM_ROOT\netdet.ini. This file is critical in
softlocx to determine if the protection scheme has been tempered with,
working together with the registry. After I manually edited the file
content, the software displays expired license even though it still
has 6 days left in demo mode (obviously I made bad changes to the
netdet.ini file). So it might be interesting to have a bpx CreateFile
if @@(esp+8)='...\netdet.ini', and examine the code flow afterwards.
filemon and regmon are highly recommended to examine insNNN.tmp IO and
registry activities.

3. The softlocx license manager displays a temporary license number
that changes with the number of days left. I haven't studied how these
two numbers relate to each other.

4. Now onto the key verification part, this part of code uses a few
tricks too. The key encoding data and a couple of pointers, [ebp-1c],
[ebp-2c], [ebp-28] are different in each softice session, even though
they seem to point to user_key, enc_key1, enc_key2. But there are a
few invariants. One invariant is the base pointer register ebp. It
always equals to 0012EAAC. If you examine the heaps insNNN uses,
0012XXXX is an additional process heap insNNN allocated. Another
invariant is in [ebp-3c] = 02f04b60, the 8 digit user registration
code. The encoding code segment can be tracked down by bpm 02f04b60
and is around 00415459 on my computer. There is one global variable
that maybe related to softice presence 2 levels down to the key
encoding subroutine.

Following the encoding subroutine, a seemingly simple key verification
routine takes user_key (4 digit) and enc_key1 and makes a comparison.

compare_keys(user_key, enc_key1, 8)
if(eax != 0) goto out_bad
do_somemore_encoding_comparison, goto out_bad whenver possible
compare_keys(user_key, enc_key2, 8)
do_somemore_encoding_comparison, goto out_bad whenver possible

It seems strange why user_key is only first half of the key but the
compare_keys subroutine actually compares 8 bytes from both
address. What is even more strange is that the enc_key1 and enc_key2
almost always ends up different and contains alphabatics. A little
more on the encoding part, the insNNN keeps a couple encoding magic
number arrays around [ebp-24], and an index shift array around
[ebp-60]. These arrays are used to produce enc_key1, and enc_key2,
both are 4 bytes long. I haven't figured out a way to even produce
numeric only enc_keys.

5. The "thanks for registration" message is kept at 0044D7b8
(invariant?), and is referenced at 0044e4c7 (by s 0 L -1 b8,d7,44,00).
Backtracing to this address may be impossible because the program is
in a constant state of self-modifying.