Exocist
September 4th, 2005, 20:04
I've been trying to get some help on debugging (and eventually unpacking) a PACE protected target. This is not an iLok protected application but it utilises the PACE wrapper based around the paceilui.dll and TPKD.sys device driver.
I made some progress into the heart of TPKD.sys (many many reboots and crashes etc) but I could use the help of someone with experience in this protection. Information is hard to find as the tutorials on this go back to the .vxd and were published in 2000. The posts regarding this protection on this forum are quite old also and nobody has posted anything conclusive.
The first trick in Windows 2K/XP/2K3 is the check for ntice.sys in the system32/drivers folder. Nothing new here and easily defeated by returning an INVALID_FILE_HANDLE condition back from FindFirstFileA.
I've then caught the first call into the device by stopping on deviceiocontrol. By tracing into this I have (eventually) found where I believe the anti-debug protection resides.
It seems to me there are MANY anti-debug features in PACE. This is what I think is happening:
(1) the TPKD device sets up an exception handler, it then sets the protected page bit and attempts to write into this memory area thus delibrately causing an exception. SoftICE captures this exception however and breaks so this immediately prevents the TPKD exception handler from executing correctly as SoftICE doesnt parse on this condition.
.text:00011E88 push ebp
.text:00011E89 mov ebp, esp
.text:00011E8B sub esp, 20h
.text:00011E8E push ebx
.text:00011E8F push esi
.text:00011E90 push edi
.text:00011E91 mov [ebp+var_C], 88h
.text:00011E95 pushf
.text:00011E96 cli
.text:00011E97 mov ebx, cr0 <--- start of enable page prot.
.text:00011E9A and ebx, 0FFFEFFFFh
.text:00011EA0 mov cr0, ebx
.text:00011EA3 popf
.text:00011EA4 mov byte ptr [ebp+var_8]
.text:00011EA8 pushf
.text:00011EA9 cli
.text:00011EAA mov ebx, cr0 <--- restore page prot.
.text:00011EAD or ebx, 10000h
.text:00011EB3 mov cr0, ebx
.text:00011EB6 popf
.text:00011EB7 mov [ebp+var_B], 3
(2) I can clearly see that the debug registers are being screwed around with which isnt good for SoftICE.
.text:000113FF mov [ebp+var_1C], eax
.text:00011402 mov eax, dr7
.text:00011405 and eax, [ebp+var_10]
.text:00011408 or eax, [ebp+var_1C]
.text:0001140B mov dr7, eax
.text:0001140E mov [ebp+var_1C], eax
.text:00011411 push [ebp+var_1C]
.text:00011414 push 0Ah
.text:00011416 mov ecx, [ebp+var_20]
.text:00011419 call sub_10876
.text:0001141E pop edi
.text:0001141F pop esi
.text:00011420 pop ebx
.text:00011421 leave
.text:00011422 retn 0Ch
(3) I think, PACE obtains the address to the IDT and redirects some of the table entries to it's own code
Now, I'm hardly an expert on this stuff as it is all very new to me. In fact, I have played around in ring0 much until looking at this stuff. I need some advice from more experienced people here please?
The issue for me is getting to the point where I can debug the application, finding the OEP, dumping and rebuilding the target will not be a problem, it's just getting past these anti-debug problems that has me scratching my head.
If anybody can help it would be appreciated. Thanks.
I made some progress into the heart of TPKD.sys (many many reboots and crashes etc) but I could use the help of someone with experience in this protection. Information is hard to find as the tutorials on this go back to the .vxd and were published in 2000. The posts regarding this protection on this forum are quite old also and nobody has posted anything conclusive.
The first trick in Windows 2K/XP/2K3 is the check for ntice.sys in the system32/drivers folder. Nothing new here and easily defeated by returning an INVALID_FILE_HANDLE condition back from FindFirstFileA.
I've then caught the first call into the device by stopping on deviceiocontrol. By tracing into this I have (eventually) found where I believe the anti-debug protection resides.
It seems to me there are MANY anti-debug features in PACE. This is what I think is happening:
(1) the TPKD device sets up an exception handler, it then sets the protected page bit and attempts to write into this memory area thus delibrately causing an exception. SoftICE captures this exception however and breaks so this immediately prevents the TPKD exception handler from executing correctly as SoftICE doesnt parse on this condition.
.text:00011E88 push ebp
.text:00011E89 mov ebp, esp
.text:00011E8B sub esp, 20h
.text:00011E8E push ebx
.text:00011E8F push esi
.text:00011E90 push edi
.text:00011E91 mov [ebp+var_C], 88h
.text:00011E95 pushf
.text:00011E96 cli
.text:00011E97 mov ebx, cr0 <--- start of enable page prot.
.text:00011E9A and ebx, 0FFFEFFFFh
.text:00011EA0 mov cr0, ebx
.text:00011EA3 popf
.text:00011EA4 mov byte ptr [ebp+var_8]
.text:00011EA8 pushf
.text:00011EA9 cli
.text:00011EAA mov ebx, cr0 <--- restore page prot.
.text:00011EAD or ebx, 10000h
.text:00011EB3 mov cr0, ebx
.text:00011EB6 popf
.text:00011EB7 mov [ebp+var_B], 3
(2) I can clearly see that the debug registers are being screwed around with which isnt good for SoftICE.
.text:000113FF mov [ebp+var_1C], eax
.text:00011402 mov eax, dr7
.text:00011405 and eax, [ebp+var_10]
.text:00011408 or eax, [ebp+var_1C]
.text:0001140B mov dr7, eax
.text:0001140E mov [ebp+var_1C], eax
.text:00011411 push [ebp+var_1C]
.text:00011414 push 0Ah
.text:00011416 mov ecx, [ebp+var_20]
.text:00011419 call sub_10876
.text:0001141E pop edi
.text:0001141F pop esi
.text:00011420 pop ebx
.text:00011421 leave
.text:00011422 retn 0Ch
(3) I think, PACE obtains the address to the IDT and redirects some of the table entries to it's own code
Now, I'm hardly an expert on this stuff as it is all very new to me. In fact, I have played around in ring0 much until looking at this stuff. I need some advice from more experienced people here please?
The issue for me is getting to the point where I can debug the application, finding the OEP, dumping and rebuilding the target will not be a problem, it's just getting past these anti-debug problems that has me scratching my head.
If anybody can help it would be appreciated. Thanks.