I have a dll file which implement flexlm 11 and many checksum algorithms which recognized by peid such as md5 and crc32 beside ecc and flexlm crypto stuff.
The target dll is statically linked and i put breakpoints at both entry point of exe and dll.
Checksum algorithm is triggered from dll main function even before the exe entry point executes and dll terminates itself if i patch anything.
Funny thing is that the error message comes from exe .rsc section.
Any comments??

-Try on Olly break on new module.
-Trace from the dll entry point (dllmain function) or even think that yet another dll may be the one that does the integrity check.
- Some crc32 structures and md5 poly conserved enough that are findable. Assuming the integrity check involves those hashing algos, which is a safe bet, try placing bp on read or execution around the algo code, hoping to break in the middle of the integrity check.

That is what I can think off the gate

Always when stuff seems to happen before the entry point of a certain exe/module, TLS callbacks are highly suspected, but of course also other DLLs statically linked from the original module (or even from DLLs that those in turn statically link to etc) could be the culprit.

If the checksumming code is good it will wipe any breakpoints before calculating the checksum, so in that case you must pinpoint the checksum code without using memory read breakpoints on the checksummed areas, i.e. binary search for it by manual tracing, like naides suggests.

Thanks guys for posting replies in such a short notice.

1- Ollydbg break on new module is not useful when target dll is statically linked)

2-It doesn't seem that another module is responsible for checksum

3-I have checked all possible references for CRC32, MD5, SHA1,…, and I put Breakpoints on the suspected functions and Memory addresses.

4-TLS callbacks seem interesting but I don't know how it can be investigated.

I have tried tracing but after a few instruction in dll main, the MSVCR71.dll!_initterm function is called and program terminates it self.

I would have post the dll function but since it is against the forum policy, I just hope that the problem can be solved with more help from you guys.

http://www.hexblog.com/2005/10/tls_callbacks.html

Ok, I checked and TLS callbacks are not the case for the target exe.

Anyway dll main function executes before target exe file and performs the checksum.

Is there a mechanism that calculates crc32 of dll before loading it and performs a comparison with the hard coded checksum in PE Header??

If I'm not mistaken, OllyDbg even has this functionality built in for statically linked DLLs, and otherwise there is 100% a plugin for it.


Exactly what makes you say that?


Such crypto scanners are always extremely approximative at best, and absolutely no proof of anything. Also, as I said above, good anti-debug code detects and/or removes breakpoints before they can be used.


Errr, if the whole thing only executes a few instructions before reaching the exit code, it should be extremely easy to analyze the logic that leads to the execution reaching the exit call?!?


No, this is only done for drivers.


And finally one last tip: The "Conditional Branch Logger" OllyDbg plugin is an extremely useful tool for situations like this...

As I suspected the before program terminates after call to msvcr71.dll _initterm function in dll main routine.

This function receives a list of function pointers and executes them one by one.

One of these function calls [imagehlp.dll!MapFileAndCheckSumA] and compares the checksum and if anything is wrong it terminates the dll instance.

I have gone the wrong way by focusing on the dll file, but I couldn’t find the problem with out tracing anyway.

Thanks for all the information guys