Log in

View Full Version : game screen crashes computer


WaxfordSqueers
October 24th, 2006, 01:20
I have not looked in the archives because I have no idea where to begin. I think you'll see why in a minute.

I have a game looking for a CD in the drive. I have it on decent information that the game has no protection other than CD check. I was going to try the old GetDriveType and play with that. But a dead listing in IDA does not show that function. So, I fired up Ice with IceExt.

The app loads OK with the loader, and if I start stepping through code, eventually I hit a point where an intro screen appears, asking me to pick a player. The minute I do, the computer crashes. It only does that with Ice running. I tried turning Faults Off with no success.

There is a variation. If I press ctrl-D, to access Ice, I crash to a reboot (both with Ice hidden by IceExt, and unprotected). If I select the player button with the mouse, I get a BSOD with params (stop 7F, 8,80042000,0,0). I can play with this data without bugging anyone here for help, but my question is, "what am I trying to accomplish"? There's no point playing in the dark, if you know what I mean.

OK...I've single stepped into the code, and there's a call before Winmain in which this intro screen is called. I'm closing in on the code where it is called. What then?? Once it's called, the intro takes up the entire screen and I can't access Ice when it does. I'm thinking I should get as close as I can to the code and look for a break point I can set after the call to it.

One problem there is that the CD check comes while the intro screen is up, after player selection, and as I've said, any mouse action causes a BSOD. The CD check is prompted by a flying horizontal window, with text in it, that falls down the screen. It's pretty fancy, but that might be the Achilles heel. They ask for a keypress (any key) to acknowledge the CD is in the drive. AHA...something to work on. (BPX with WM_Command on a keystroke???)

BTW...there's an Exit icon on the intro window that can be pressed with the mouse cursor that does not cause a BSOD or outright crash. That might indicate the BSOD is intentional. The question then, is how does one cause an intentional BSOD. What code am I looking for?

Another approach I've considered is bypassing the intro screen. Somehow, I still have to bypass the player selection, which seems trivial. There's only one player playing. It would seem, however, that if I can't work around the BSOD, that I'll have to work through a dead-listing. Obviously the app calls a movie in the intro and that might be a giveaway too, if I can ID the movie.

One last solution might be to convert the intro window from full screen to a window. I have no idea how to do that, if it can be done. In most games, you can use Alt-Tab to move back and forth between the game and the desktop, or another app. Is there a way to force that?

Does anyone have any suggestions? TIA.

evlncrn8
October 24th, 2006, 01:37
scan game with something like protectionid (pid.gamecopyworld.com), to make sure its 'just a cdcheck', if so, then disassemble the exe and look for api calls relating to cdchecks -> GetDriveTypeA, GetVolumeInformationA etc etc, also check any dlls it loads for other 'cdcheck' type code.. not all cdchecks are as simplistic as GetDriveTypeA

WaxfordSqueers
October 24th, 2006, 03:51
Quote:
[Originally Posted by evlncrn8]scan game with something like protectionid

scanned it with PEID...came back as only C++. Also, read in a forum that it has only CD protection.

Quote:
[Originally Posted by evlncrn8]also check any dlls it loads for other 'cdcheck' type code.. not all cdchecks are as simplistic as GetDriveTypeA


it only has 2 dll's. Looked at them in IDA and no obvious CD check functions show up. There are some mods loaded at run time (like is msvcrt, ole32 and oleaut) that I need to check, but the point is still that I can't get around the intro screen.

I'm tracing code right now and running into heavy math co-processor usage. They seem to be converting a hex number to a real number set to 3 decimal points. That is going on around a function called Dinput, which may indicate a DirectX check. Maybe they're just checking the version of DirectX. It may also be a screen to make it hard to get to the CD check code.

I can set any bpx I want, but once the intro screen comes up, any LB mouse activity on the player select button causes a BSOD. So, my BPX's are no good if the screen crashes the system before they set off. I need to capture the input to the player select button before the BSOD hits.

I could use a wm_leftbutton down if I had a hwnd. Does anyone know of a BPX I could set that affects the mouse and wouldn't trigger easily with other Windows stuff. I can single step close to the point where the intro window is triggered. Maybe the hwnd is set before the intro comes up. I'll check.

I'm also having problems with the Ice window resizing (getting smaller) after I bypass the intro screen. I can get past it using the exit button to an exit screen (credits) and I have to ESC out of that. Ice gets erratic after that and it's normally very stable.

dELTA
October 24th, 2006, 05:00
I have summoned the local DirectX wizard, in the meantime until he appears, you'd do good in reading the following two threads, in their entirety:

http://www.woodmann.com/forum/showthread.php?t=6556

http://www.woodmann.com/forum/showthread.php?t=6310

Silver
October 24th, 2006, 06:24
The local DirectX wizard is unfortunately busy this week so he sent me instead. I know, poor imitation and all that

Quote:
That is going on around a function called Dinput, which may indicate a DirectX check. Maybe they're just checking the version of DirectX


Here's your problem. As everyone knows, DirectX itself doesn't really exist, it's a bunch of components including D3D (the 3d graphics bit), DInput (user input etc). It's like referring to MS Word, Outlook etc as "Office". I'm going to make some assumptions here, mostly based around the fact that an app using DInput will also use D3D as otherwise it's a bit pointless.

You've found your app references DInput objects. This means you can immediately forget about tracking input through the WM_ pump. For all intents and purposes, when DInput is used in a DX app it takes complete control of the target device. For example, if I code a DX app that uses a DInput mouse I no longer expect to receive WM_MOUSExxx messages in my pump. Instead I will have to continuously query the physical state of the hardware. It's now my responsibility to manage the hardware and its state - DInput is designed for games where immediacy is important (as opposed to WM_ which pretty much arrives whenever it wants).

Here's a quick primer, by all means ask me to drill down on any bit I haven't made clear.

DirectX apps are made on a game loop principle. In other words whereas normal Windows apps start up, do some stuff then (generally) do nothing else until some input is received, DX apps are ALWAYS doing something. This is a DX app compressed into a few pseudocode lines:

Code:
int WinMain(...)
{
hwnd = CreateWindowEx(.......);
InitializeDirectX();

// assume these are global vars
g_d3d = AttachDirect3DDeviceToTheHWND(hwnd);
g_dinputmouse = AttachDirectInputDeviceToMouse();

// Message loop needed for other WM_, eg: WM_DESTROY.
while(messageloop)
{
// deal with messages
translate/dispatch();

// update mouse
UpdateDInputMouseStuff(g_dinputmouse);

// Draw all D3D output
Render();
}
}


See how the mouse and drawing is done constantly in the while()? That's really important, always bear this in mind with DX apps because if you try and use WM_ to follow DX related behaviour you'll get nowhere fast. Similarly remember that DX apps still have to obey Windows and thus WILL respond to some WM_. One notable quirk is if a DX app uses a real Windows messagebox inside the app itself.

Part of the difficulty with your specific situation is that the behaviour you're seeing could be intentional to protect the app, intentional for normal usage but an unexpected side effect, a side effect of how DX works etc etc etc. I'll throw some initial thoughts out there.

Quote:
I could use a wm_leftbutton down if I had a hwnd


You can't do this if the app uses DInput for mouse input. Even more importantly is what you're trying to click on. You have absolutely no guarantee it "exists" in the Windows sense of things. What you're looking at, although it might even look like a normal Windows UI, could quite easily be a manually D3D blitted bitmap that gets changes when the DInput mouse clicks on it. In that case there would be a check in the app to see if there was a mouse click through the DInput device, the click position would then be hit tested and, if a hit was found, the click would be passed on to the object that got clicked.

So your first question shouldn't be "how do I break on this UI element", it should be "what is this UI element really, and how does the user input process work".

Once you can confidently answer that question you can begin looking for IDirect3DInput8::GetDeviceState() calls (note that even though DX9 is released, the interface iteration is still ...Input8). This is a general purpose accessor that retrieves the (you got it) device state for DInput devices. In your case for the mouse you'll be looking for a (param order, not stack order) sizeof(DIMOUSESTATE), NULL, and a DIMOUSESTATE* pushed to the stack. Google the struct and you'll see what states are returned. These are a snapshot of the device at the exact moment GetDeviceState() was called. Normally an app will store this in a wrapper class for a DI mouse until the next iteration of the WinMain while() loop, so that other parts of the app can query the mouse state (think CMyDIMouseWrapper::IsLMBDown() etc).

*edit* One other thing, DInput works for ALL input devices, not just the mouse. So don't assume the first set of DInput calls you find are always going to be mouse related.

Quote:
I can set any bpx I want, but once the intro screen comes up, any LB mouse activity on the player select button causes a BSOD


This could be anything, unfortunately. I suggest you forget about this one for a second. The BSOD could be caused by both sice and D3D trying to gain exclusive access to the display device at the same time. Remember, D3D is working at the hardware level, there's no GDI abstraction in the way. This is very important when working with fullscreen DX apps. Any DX coder will tell you that it's basically impossible to debug a fullscreen DX app on a single monitor system.

So your first task should be converting the app from fullscreen to windowed mode. You need to make sure this is a REVERSIBLE change, because it's going to be far easier to convert it to work on your system with your hardware than patch it back to the fullscreen mode (which hopefully is compatible with most hardware thanks to the original coders doing good caps test work) when your reversing exercise is done.

To do this you're going to need to track down the IDirect3DDevice(8/9)::CreateDevice() call. Prior to this you'll see lots of mov to fill out a D3DPRESENT_PARAMETERS struct. This struct has a param for windowed mode, which you'll need to set (boolean). However you will also need to set the .BackBufferFormat member to something compatible with your Windows surface (remember, D3D still works at the hardware level even in windowed mode, so your surface format must match Windows' - imagine trying to save a BMP file with half of the image in 24bit and the other half in 32bit). I suggest you use decimal 22 for this param.

Right, I think I'll end there. That should give you something to think about Don't worry if you figure it out, follow what I've said and it still doesn't work. There are about a thousand factors that can stop a DX app working properly (one that comes to mind is lost devices, which will happen as soon as you switch away from the app to debug, so you'll need to deal with this for both the mouse and the display probably).

I'm not trying to put you off, it's a great challenge and I'll help you as much as I can. But to put it in context & give you some feel of what you're aiming at, what you're trying to do is a little like a Linux reverser switching to Windows. All the reversing skills and experience will translate and help you but you're dealing with an API, code approach and management you've got no experience of. Learning DX has defeated many good Windows coders, and that's just coding it - you're trying to reverse it

WaxfordSqueers
October 24th, 2006, 06:52
Quote:
[Originally Posted by dELTA]I have summoned the local DirectX wizard, in the meantime until he appears, you'd do good in reading the following two threads, in their entirety:


Delta...how's it going? Thank you kindly for your input. I just read Silver's reply, and whereas it's quite daunting, it makes sense generally. I'll carry on with my reply to him. I read the other two articles as well, although it was more of a skim at this point. Thanks.

Maximus
October 24th, 2006, 08:27
Quote:
[Originally Posted by Silver]Learning DX has defeated many good Windows coders, and that's just coding it


My first suggestion is: you need a bit of 3d knowledge to catch what's getting on. A good starting point is the 'Red Book' of OpenGL programming, a free PDF.
Supposed you have 3d knowledge, take the DX7 chm from microsoft, or wether it is already stored. It is very well written and by far much easier for learning DX principles than the others (really! not joking).
There you will learn quickly and easily the basis of dx.
From the programming side... you can play with a free commercial&nice software, mmh... TV3d library, DX based.

Also, remember that you can break most of DX in kernel mode, as m$ moved dx to kernel for speed and ...boh? they said it is for better platform portability or so. Don't know well, anyway it's on DDK.

By the way, you might try to place the game in VMWare (if it runs) and use such net for remote debugging.

I tried once with syser, which supposedly should save/restore dx state, but it didn't work fine for me. I'll give another shot in a (far) future, however.
Keep in mind that you might find a workaround by using a r3 debugger as long as you code a driver that does the magic ALT-TAB switch for you...
...share the code if you do

WaxfordSqueers
October 24th, 2006, 08:29
Sorry for the length of this reply. It's late and I'm too tired to edit effectively. Also, there's a lot to cover.
Quote:
[Originally Posted by Silver]The local DirectX wizard is unfortunately busy this week so he sent me instead. I know, poor imitation and all that
I must say, Silver, that if you're the stand-in, I'd hate to think how up there the wizard is. Brilliant reply!! Thank you.

Since I posted, I've been poking throught the code. A quick tip to anyone working through BSOD's: go into device manager under the drives and turn of the write caching under 'Properties/Policies'. I've been caught with that twice now, losing my logon profile due to data being unwritten by the crash. Mind you, checkdisk (XP) recovered it both times.

I also noticed that the RDSTC (real time stamp counter) instruction appears in the same code as the directx stuff. If I'm correct, it can be used to test for single-stepping. The co-processor instructions may be related to RDTSC, but I'll have to verify. There are definitely a couple of calls to Dinput, however.

I understand what you say later about the BSODs being related to a race condition between Ice and the 3D software. That's the more likely explanation, but there are exception blocks in use in the code, and that makes me suspicious that they are detecting Ice. On one mini-crash IceExt posted a MeltIce message.
Quote:
[Originally Posted by Silver]DirectX apps are made on a game loop principle....snip... remember that DX apps still have to obey Windows and thus WILL respond to some WM_. One notable quirk is if a DX app uses a real Windows messagebox inside the app itself.
On the first point, I'm thinking of the main message loop in Windows. It's circular. It has to keep running no matter what, as I understand. If DX is running in it's own loop there must be liason between them. I'm theorizing. If you consider your while loop, there must be a way to break out of it, or into it...no???
Quote:
[Originally Posted by Silver]Part of the difficulty with your specific situation is that the behaviour you're seeing could be intentional to protect the app, intentional for normal usage but an unexpected side effect, a side effect of how DX works etc etc etc. I'll throw some initial thoughts out there.
I may be a bit of both, I'm thinking. When the intro begins, there's definitely a 3-D effect with a rotating face, that gives way to fireworks moving in loops, then to banner-type, full-screen window. That gives way to another full screen window with a still drawing on it that contains the buttons for choosing up to 5 players. There's a fancy mouse cursor and another button for exiting.

All the buttons can change colours as the mouse moves over them. I would say the entire intro is 3-D rendered. I should mention that the DX requirement for the game is DirectX 7 or better. I'm using a DX 9c card. Now that you have raised my interest, I'll have to do some reading and see if i can identify parts of the code, either in Ice or IDA. My bag is audio, but I have an interest in video as well.
Quote:
[Originally Posted by Silver]In that case there would be a check in the app to see if there was a mouse click through the DInput device, the click position would then be hit tested and, if a hit was found, the click would be passed on to the object that got clicked.
About the Dinput function, is it re-entrant? I've been jumping over it, and I try to avoid Ntdll like the plague. Would it help to trace into the Dinput function, or would observing it's input/output be sufficient?

Also, in the WM_pump, as you call it, if I put a bmsg on it with a hwnd, I get an indication of that when it hits in Ice. Would it be similar with a DX function?

Note:to anyone still struggling with Ice in XP, it doesn't seem to like the format bmsg 'hwnd' 'wm_whatever'. It likes bmsg 'hwnd' 'msg#' (eg. bmsg '12345' '111' for wm_command).
Quote:
[Originally Posted by Silver]So your first question shouldn't be "how do I break on this UI element", it should be "what is this UI element really, and how does the user input process work".
you're not only a guru, you're a mind-reader.
Quote:
[Originally Posted by Silver]Once you can confidently answer that question you can begin looking for IDirect3DInput8::GetDeviceState() calls
any code with functions like that is absent in the code I've checked so far. Something just occured to me. They are using floating point, which could be related to transforms. I'm thinking the math I've seen so far is probably related to the RDTSC instruction. As you say, there are things to be verified.
Quote:
[Originally Posted by Silver]Normally an app will store this in a wrapper class for a DI mouse until the next iteration of the WinMain while() loop, so that other parts of the app can query the mouse state (think CMyDIMouseWrapper::IsLMBDown() etc).
I've already found a message loop that was not where I expected to find it. As I said in an earlier post, this code is all happening before winmain is called. The Windows message loop for the app hasn't even been set up yet. Something more to explore.
Quote:
[Originally Posted by Silver]*edit* One other thing, DInput works for ALL input devices, not just the mouse. So don't assume the first set of DInput calls you find are always going to be mouse related.
I'll keep that in mind. I'm also trying to keep in mind that the code is referencing a small 3-D application before any of the app's main windows are created. This is a mini app within itself. It's also a heck of a novel protection system for the CD check. If you and Delta hadn't come along I'd have gotten nowhere.
Quote:
[Originally Posted by Silver]So your first task should be converting the app from fullscreen to windowed mode. You need to make sure this is a REVERSIBLE change. To do this you're going to need to track down the IDirect3DDevice(8/9)::CreateDevice() call. Prior to this you'll see lots of mov to fill out a D3DPRESENT_PARAMETERS struct.
I have noticed bunched mov instructions on my way by, but they were initializing with zero's. There was also a point where I noted about 25 Calls one after the other. I mean, they were stacked back to back with no intervening functions.
Quote:
[Originally Posted by Silver]This struct has a param for windowed mode, which you'll need to set (boolean).
OK...cool...thanks. I'm editing your reply heavily, but I'll print it all out and study it closely.
Quote:
[Originally Posted by Silver]Right, I think I'll end there. That should give you something to think about
no kidding.
Quote:
[Originally Posted by Silver]I'm not trying to put you off, it's a great challenge and I'll help you as much as I can.
I appreciate the help...thanks. I post in the newbie section, but I'm an advanced noob in certain facets. Nothing you've said has gone over my head, but when I read those articles that Delta posted, I'm a bit out of my league. Mind you, I can follow what they're getting at albeit peripherally.

WaxfordSqueers
October 24th, 2006, 08:47
Quote:
[Originally Posted by Maximus]My first suggestion is: you need a bit of 3d knowledge to catch what's getting on. A good starting point is the 'Red Book' of OpenGL programming, a free PDF...snip... DX7 chm from microsoft
thanks for tips on books.
Quote:
[Originally Posted by Maximus]Also, remember that you can break most of DX in kernel mode
can you elaborate a bit??

Quote:
[Originally Posted by Maximus]By the way, you might try to place the game in VMWare (if it runs) and use such net for remote debugging.
that's a good tip. I wondered about that for using visual softice.

Maximus
October 24th, 2006, 19:37
mmh... browser crashed before I posted. bah.
most of DX is in r0: take DDK and you will see the r0 layer of DX.
Silver sugestion is the best way, and surely he's by far more competent than me on the matter. I stopped at DX7 and 'discovered' shaders existance last year or so... incredibly cool and powerful toys, I must say.
There is also a very cool ICE-like win2k debugger somewhere, with a russinovich comment on the top, but I lost the link anyone can recover? it uses (like syser) dx to draw the visual layer. worth a try, who knows.

To quickly understand what you are facing, google for ALT+TAB problem and DX. You will understand what is your problem that way: it is a common problem for dx programmers

Kayaker
October 24th, 2006, 20:58
Quote:
[Originally Posted by Maximus]There is also a very cool ICE-like win2k debugger somewhere, with a russinovich comment on the top, but I lost the link


I recognize the reference to BugChecker. I think I tried it once on Win2Ksp2, don't believe it worked with the combination of updates I had. Worth a look at least.

http://www.vitoplantamura.com/index.aspx?page=bugchecker

LLXX
October 25th, 2006, 02:38
Would it be too much trouble to put another video card and monitor in your system, and use that instead? That would be optimal.

(Most of the reversers I know have huge piles of hardware lying around, so...)

Silver
October 25th, 2006, 05:39
I'll keep this reply short and just answer the pertinent points otherwise I think this might get a bit hard to follow. By all means repost questions as needed if I missed them.

Quote:
If DX is running in it's own loop there must be liason between them. I'm theorizing. If you consider your while loop, there must be a way to break out of it, or into it...no???


You're correct, but I think you're still approaching this like a pure Windows app. Have you ever coded anything that wasn't UI toolkit dependant? Say a DOS game or some console homebrew? In those you will have had a loop that simply kept executing until the user told the app they wanted to quit.

I've written this paragraph 5 times now and I can't figure out how to explain it clearly.

In a Windows app, when you want to quit, what do you do? You click the X button, or select File - Exit, right? Those actions generate WM_ which your messageloop can then act upon. But in a DX app you can't use any Windows UI elements - no buttons, no menus, nothing. And no Windows UI means no WM_ when an action is taken. So DX apps have to draw buttons themselves, hit test mouse clicks themselves and then act on the result themselves. The most likely way would be to set a global bool that controls whether the message pump keeps iterating (the "while(messageloop)" in my 1st post pseudo code).

As far as DX apps go, Windows, the Win32API, the message loop and all that other stuff is just an annoying inconvenience - something you have to comply with otherwise your app won't load, but ultimately has zero affect on your DX code.

Quote:
the DX requirement for the game is DirectX 7 or better


Ah, which would imply the app uses DX7 and not 8 or 9, which is a bit different. I'm not that experienced with DX7 unfortunately, but it will explain some of the other things you said.


Quote:
About the Dinput function, is it re-entrant?


Not sure what you mean by that. GetDeviceState() is a snapshot of the current hardware state "right now". Just like the Win32 GetAsynchKeyState(). With DInput, your app's knowledge of the hardware state is only as accurate as the last time you called GetDeviceState().


Quote:
any code with functions like that is absent in the code I've checked so far


I believe DInput7 uses GetDeviceData(), not GetDeviceState(). Sorry.


For what it's worth, here's how I'd approach this. Assuming there's no way to directly jump into or find the validation code (what gets called after you click the DX non-button), I'd break on the calls to GetDeviceData() and then look around it to find where the state of the mouse buttons are pushed to. I'd then set a read breakpoint on that location (making the reasonable assumption it's a global var or class member) then go back into the app. I'd put an easily identifiable string into the protection code fields then click the "Go" button or whatever. This should, with any luck, break you at the hit test code that checks the mouse state and what (if anything) was clicked on - hopefully something obvious like a bunch of jne's.

At that point you know that your easily identifiable fake protection code is in memory somewhere (because remember - there's no GetWindowText etc, every time you press a key for each character of the code the app will have push'd it somewhere) so you could either find it with sice memory search then set a read bp on it, or you could do it the longer way and trace the hit test code until you arrive at the button handler code.

When you're doing this, keep thinking to yourself "If I had to code a UI and API from the ground up, with no framework to help me, how would I do it". Hopefully if the coder of the app is as smart as you he'll have done it in the same way. That idea kind of falls down if you get a less than competant coder (or a demoscener who is using every hack in the book to finish their product as quickly as possible ).

Quote:
To quickly understand what you are facing, google for ALT+TAB problem and DX. You will understand what is your problem that way: it is a common problem for dx programmers


Maximus, and by "common problem" you mean hair-rippingly, pant-wettingly, sanity-destroying hellish nightmare, right? (for anyone curious, google "directx lost devices", the utter frustration of newsgroup posts is great!)

Um, yeah, that was supposed to be a short post. Never mind. Oh and also I'd question whether this really does qualify as a "newbie" question. It's way beyond the "how do I find a time trial check"...

*last edit, promise* - If anyone wants me to attach the DX8/DX9 CHM reference (14mb) (awful for learning to code DX, great for a to-the-point API reference) let me know. It'll save you downloading hundreds of megs of SDK.

dELTA
October 25th, 2006, 09:34
Great information as usual Silver.

About "re-entrant", the short explanation of a "re-entrant function" is a function that the operating system can context-switch while the execution point is inside it. A more generalized explanation is a function that does not modify the state of any outside resource or data (i.e. except it's own stack variables) in a way that can cause race conditions.

About the DX8/DX9 CHM reference, sure, please post it. You cannot attach files of that size directly to the forum though, but a rapidshare/zupload link should work just fine.

WaxfordSqueers
October 25th, 2006, 17:33
Quote:
[Originally Posted by dELTA]Great information as usual Silver.


Yeah...it's awesome. I'll reply to Silver later when I get some sleep and my eyes open again.

Quote:
[Originally Posted by dELTA]About "re-entrant", the short explanation of a "re-entrant function" is a function that the operating system can context-switch while the execution point is inside it.


I'm probably using re-entrant in the wrong manner. I was thinking of something like kernel32 where it gets called, then it can turn around and call the calling app, which returns to k32 before it in turns returns to the app. If you jump over it while tracing, you sometimes miss some good, or critical action while k32 sidetracks into the calling program before returning officially. I asked the question with regard to Dinput because I was wondering if it was safe to jump over it, or should I snoop around in it to see if it calls back into the main app.

WaxfordSqueers
October 25th, 2006, 18:44
Quote:
[Originally Posted by Silver]I'll keep this reply short and just answer the pertinent points otherwise I think this might get a bit hard to follow. By all means repost questions as needed if I missed them.
Silver...thanks for going to so much trouble. I'll only answer in part because i'm really tired and I want to do justice to your efforts. You're probably right about the newbie thing, but I'm kind of a jack of all reversing and a master of none. I tend to post in the newbie section because I'm awful at some basic reversing.

For example, I said there was no GetDriveType function in the app. It was right under my nose. They are calling a lot of functions with GetProcAddress. Even so, it's not a trivial solution with a good guy-bad guy jump instruction handy. I squeezed a lot of information out of the app using Boundschecker and Smartcheck.

The intro video sequence is still proving to be a hurdle because the drive check comes after it. I am zeroing in on it, however, and must be close because they have thrown an RDTSC trap in front of me right after the sound, text and video were set up. BTW...I found a humungous series of MOV instructions in the video setup, as you predicted, and they weren't initializing with ones or zeros. Each MOV had a number with it that was way too small to be a pointer.

I have identified some DX functions (Ddraw.dll, DirectInputCreateA, DirectDrawCreate, but I haven't pieced together how and where they are applying them. I have a couple of decent books on DirectX and OpenGL which I am skimming. I need to sit down with them to do some justice to the caper.
Quote:
[Originally Posted by Silver]I think you're still approaching this like a pure Windows app. Have you ever coded anything that wasn't UI toolkit dependant? Say a DOS game or some console homebrew? In those you will have had a loop that simply kept executing until the user told the app they wanted to quit.
My programming skills are like my reversing skills: I know a lot about nothing. Most of my programming experience is in the old format, where a program started out and ran in a staight line. I still use the word 'subroutine' unashamedly. So, I'm not really hung up on the Windows model, although it is pretty ingrained. I'm keeping an open mind avout DX.
Quote:
[Originally Posted by Silver]In a Windows app, when you want to quit, what do you do? You click the X button, or select File - Exit, right? Those actions generate WM_ which your messageloop can then act upon. But in a DX app you can't use any Windows UI elements - no buttons, no menus, nothing.
that would make more sense to me if the DX app was written from the ground up, like say a Python app. But in this app, Dinput is called directly from Windows, and it returns to Windows. I can skip over it, trace into it, or F11 out of it.

That's why I asked if these modules talk to each other in a common language. I think the message loop I found is in Dinput, but seeing both are from Microsoft, and Cakewalk, to a large extent, you'd think there would be functions that spoke to each other.

A book I was skimming the other night talked about the use of COM objects. I take that to be an extension of the old object linking and embedding. You used Office as an example, with Word, Excel, etc., being different apps but able to communicate.

You're right though, it will take a bit of time for me to drop the Windows model and see DX in it's own environment. This app is peculiar in itself in that it doesn't use much of a resource section, if one at all. It has a large amount of embedded code to unpack sound and video files, along the line of Bink files. It has an extensive sound, video and text library in external directories.
Quote:
[Originally Posted by Silver]And no Windows UI means no WM_ when an action is taken. So DX apps have to draw buttons themselves, hit test mouse clicks themselves and then act on the result themselves.
I do understand what you're getting at, I just need some time and some reading to get the picture. I'm actually curious as to how that is accomplished.
Quote:
[Originally Posted by Silver]For what it's worth, here's how I'd approach this. Assuming there's no way to directly jump into or find the validation code (what gets called after you click the DX non-button), I'd break on the calls to GetDeviceData() and then look around it to find where the state of the mouse buttons are pushed to.
I need a bit of time to digest this. I'll try the function above in Ice ASAP.
Quote:
[Originally Posted by Silver]At that point you know that your easily identifiable fake protection code is in memory somewhere (because remember - there's no GetWindowText etc, every time you press a key for each character of the code the app will have push'd it somewhere) so you could either find it with sice memory search then set a read bp on it, or you could do it the longer way and trace the hit test code until you arrive at the button handler code.
good idea. I've already found a reference to the Player 1 button in Boundschecker. As you know, BC gives handy data to nail down the location with a pointer. At least, it's given me an offset into the app, which will make it easier to trace.

Silver
October 26th, 2006, 05:08
Great, good luck and let me know how you're getting on! For reference, all those mov's are most likely setup options depending on the caps (ie: making sure the app's graphical output is compatible with your hardware), so for now you can probably ignore them.

If I have time I might code up a little crackme along these lines...

DX9 SDK docs: http://www.savefile.com/files/194261 (Rapidshare says all its diskspace is full, hmm!)

WaxfordSqueers
October 26th, 2006, 07:01
Quote:
[Originally Posted by Silver]Great, good luck and let me know how you're getting on!
THanks again for help and the chm on DX9c. I haven't opened it up yet but there are some things I discovered last night that I'd like to comment on.

BTW...noted your tag line about trying to reverse D3D.

I traced into the Dinput call. It was a long drawn out affair involving ntdll, ntoskernel, HAL, rpcrt, and even usbport and usbuhc. That's the first time I've been able to step through ntdll without a blue screen. Mind you, I automated a lot of the single-stepping with t 100, t 500 and even t 3000 at one point. All in all, there were at least 20,000 single trace steps and I still had to F11 out of it. That was good in itself, since I've found ntdll to be very touchy about F11. The app carried on as normal into the intro sequence despite my intrusion at that low level. It would be very handy to be able to trace through that low level code and recognizing short cuts to extricate onesself.

It became re-entrant during the usbport/usbuhc phase, meaning it called back to the calling app. The fact that USB was involved, however, suggests mouse action, since it's the only USB device I have except for my printer. That may be a plan as well for capturing the mouse in the intro screen.

I realize that's not what you're getting at when you claim DX is an autonomous system with respect to the Windows messaging system. Once it's at a low level like that, I don't imagine the user interface comes into play much, if at all. Still, the interaction is impressive. I need to focus, as you said, on what the Dinput call is trying to accomplish and how it interfaces with the inner system.

WaxfordSqueers
October 27th, 2006, 10:57
Quote:
[Originally Posted by Silver]Great, good luck and let me know how you're getting on!


Silver...is this the DX mouse handler you were talking about?
Please excuse the code formatting, I edited it pretty heavily, but you'll get the idea.

If it is, should I be able to break in Ice with a BPX on one of these addresses while pressing the button in the DX window? I'm not too confident about that because the intro screen is pretty touchy. Sometimes it lets ne get away with pressing the Player button, and other times it crashes if Ice is running the code.

I'm remembering what you said about DX and Windows messaging being independent, but if this is the DX mouse handler, shouldn't Ice break on it? I may still have to reduce the full screen intro to a Window.

4D6EC7 push eax
4D6EC8 push offset aDxdispatchMous ; "DXDispatch, mouse handling"
4D6ECD call sub_4DA690
4D6ED2 add esp, 8
4D6ED5 mov esi, ds:PeekMessageA
4D6EDB push 0
4D6EDD push 0
4D6EDF push 0
4D6EE1 lea edx, [esp+3Ch]
4D6EE5 push edx
4D6EE8 call esi ; PeekMessageA
4D6EEA test eax, eax
4D6EEC jz short loc_4D6F3F
4D6EEE mov edi, ds:GetMessageA
4D6EF4 mov ebx, ds:TranslateMessage
4D6EFA mov ebp, ds: DispatchMessageA
4D6F00 push 0
4D6F02 push 0
4D6F04 lea eax, [esp+38h]
4D6F08 push 0
4D6F0A push eax
4D6F0B call edi ; GetMessageA ;
4D6F0D test eax, eax
4D6F0F jz short loc_4D6F21
4D6F11 lea ecx, [esp+30h]
4D6F15 push ecx
4D6F16 call ebx ; TranslateMessage
4D6F18 lea edx, [esp+30h]
4D6F1C push edx
4D6F1D call ebp ; DispatchMessageA
4D6F1F jmp short loc_4D6F2C
4D6F21 ; --------------------------------------------------
4D6F21 push 0
4D6F23 lea ecx, [esp+1Bh]
4D6F27 call sub_4DCAF0
4D6F2C push 0
4D6F2E push 0
4D6F30 push 0
4D6F32 lea eax, [esp+3Ch]
4D6F36 push 0
4D6F38 push eax
4D6F39 call esi ; PeekMessageA
4D6F3B test eax, eax
4D6F3D jnz short loc_4D6F00
4D6F3F pop edi
4D6F40 pop esi
4D6F41 pop ebp
4D6F42 pop ebx
4D6F43 add esp, 103Ch
4D6F49 retn

Silver
October 28th, 2006, 07:58
At cursory glance, no, I don't believe this is it. I'm not the worlds greatest reverser but following that through by eye it looks like a normal WinMain loop to check and dispatch messages, with perhaps a call to set up the DInput device but nothing else (that call at 4D6EE8 just doesn't feel right to me).

Let me clarify something with you, just to make sure I didn't skip over it. With vanilla win32 you're going to get a WM_MOUSEMOVE, WM_LBUTTONDOWN etc for every mouse event. Are you trying to find the equivalent for a DInput mouse handler?

If so, that's the misunderstanding. There is absolutely no handler of this type for DInput devices. In fact the odds are you'll find a structure pushed to the stack then a single call to GetDeviceState in one area of code, and nothing else. Then in a totally unrelated area you'll find some basic manipulation of that structure, for example the gfx render code might check it to know where to render a cursor. There won't be any apparent direct link between the 2, which is what will make this difficult. When I said "DInput mouse handler", I meant "generalized piece of code that deals with initiating actions based on hardware state", not "piece of code that responds to events". Sorry if that wasn't clear.

Here's the kicker. Assuming you correctly set the breakpoint on the piece of code that responds to a mouse click on the button, there is absolutely no reason to expect it will break the "instant" you press down on the mouse button. In processing terms it could be ages before any response happens. Also remember that DInput hardware queries are snapshots, so if you set a breakpoint on the chunk of code that checks the mouse state, click the mouse, the debugger pops up and you release the mousebutton so you can continue working, when you step over the actual state check the mousebutton will physically be up - that's what the app will see. This is in contrast to WM_, where if you click the mouse and get a WM_LBUTTONDOWN, when it's in the message queue it's there "for life". If that wasn't clear tell me and I'll explain further.

I'm wondering if this is a non-exclusive dinput device. Do you get any WM_ mouse messages in the normal pump itself?

Oh and also I recommend you concentrate on getting the app into windowed mode first, it will make your life much easier.

WaxfordSqueers
October 28th, 2006, 12:37
Quote:
[Originally Posted by Silver]At cursory glance, no, I don't believe this is it. I'm not the worlds greatest reverser but following that through by eye it looks like a normal WinMain loop to check and dispatch messages, with perhaps a call to set up the DInput device but nothing else (that call at 4D6EE8 just doesn't feel right to me).


thanks again for your time. The message loop appeared from a place I did not expect it to be. I was tracing through some timer code involving an rdtsc instruction. I figure it was put there to detect single stepping. One branch leads to an 'on terminate' reference and the entire area is thick with math coprocessor instructions. They are spending a lot of time manipulating and compairing the values from the RDTSC instruction.

I found something interesting, If I put a bpx on the 'ret' instruction at 4D6F49, and run the app with F5, Ice pops up, first on a completely black background. As I continue hitting F5, the app keeps breaking on the bpx, with more of the intro graphics appearing, and Ice sitting right on top of them. Eventually, I get the mouse cursor, and when I hit the button for Player 1, the cursor disappears, leaving me with the player selection panel. Then, the flying horizontal window appears asking for the DVD.

I found out why I couldn't get Ice with the ctrl-D. The input is switched from mouse to keyboard, so any keyboard hit brings back the window asking for the DVD. However, if I eject the DVD, and insert the drawer again, the window asking for the DVD disappears momentarily, giving me the chance to ctrl-D into Ice.

At first, I thought I'd lost the game context, but when I used 'addr game.exe' it flashed that screen. The app is stuck in a system dll called 'computer' and I can step out of it using F11. Eventually, that takes me back to the screen asking for the DVD. It's all very stable.

I'm stymied as to how I can use Ice to break elsewhere, but I don't understand enough about the DX input. It seems to me that if the code I posted is the main message loop, it should look a bit different. This one has two sets of GetMessage, TranslateMessage and DispatchMessage. Also, a couple of hundred lines of code before this code is the DX keyboard code.

Quote:
Are you trying to find the equivalent for a DInput mouse handler?
no...I've dismissed that approach from my mind. I am not looking for a typical wm_ messages to break on. What I'm trying is to bpx on a code location that is in the path of the DX code.

Quote:
When I said "DInput mouse handler", I meant "generalized piece of code that deals with initiating actions based on hardware state", not "piece of code that responds to events". Sorry if that wasn't clear.
what you said was pretty clear. It's odd that the reference to the mouse handler is where it is. Why have it right in the main mesage loop, if that's what it is? And why does the bpx at the end of that code bring up Ice right over the intro video?

The fact that they switch from a mouse input to a keyboard input seems significant as well. When I left off last night I was looking to trap the keyboard code somehow.

Quote:
Here's the kicker. Assuming you correctly set the breakpoint on the piece of code that responds to a mouse click on the button, there is absolutely no reason to expect it will break the "instant" you press down on the mouse button.
I was reading a bit on that. It seems the Dinput queries the mouse driver directly. I was looking for a way to incorporate BPIO, BPINT or BPM. The mouse is a PS/2 mouse on IRQ 12 that's supposed to plug into a USB port. My mobo is an Intel D845 and has two ports side by side for the keyboard and the mouse.

Does that seem a reasonable approach...to capture the mouse driver with a BPIO or BPINT and follow it to Dinput? Or do the same with the keyboard. Around the keyboard driver are numbers like 'cmp eax, 8007001Eh'. I have no idea what those numbers mean.

Quote:
In processing terms it could be ages before any response happens. Also remember that DInput hardware queries are snapshots, so if you set a breakpoint on the chunk of code that checks the mouse state, click the mouse, the debugger pops up and you release the mousebutton so you can continue working, when you step over the actual state check the mousebutton will physically be up - that's what the app will see.


This is not all that clear. If I set a bmsg hwnd msg# and press on the correct button, etc., the wm_ message goes into queue and the system has to decode it. I've traced through many of those decodes, watching it compare say wm_command = 111 to other numbers till it gets a match.

I don't get what you're saying about how Dinput processes the mouse button press. Usually, when something like a mouse click is performed, a software equivalent of a latch is set. In fact, there are software debouncers that go of on the first Press of a button and latch it. Are you saying that Dinput might miss my mouse press?

Quote:
I'm wondering if this is a non-exclusive dinput device. Do you get any WM_ mouse messages in the normal pump itself?
I'll check that out.

Quote:
Oh and also I recommend you concentrate on getting the app into windowed mode first, it will make your life much easier.
I'm looking for the windowing code. I've found something refering to 'vueport' that is sitting around the DX mouse and keboard stuff.

Silver
October 29th, 2006, 11:24
Quote:
hey are spending a lot of time manipulating and compairing the values from the RDTSC instruction.


That in itself is not surprising and not necessarily a definite indicator of debug checking. DX apps continually render to the screen as quickly as possible (limited by framerate etc because tearing is handled by swap method). So for any time based action such as character/mesh animation, camera movement etc a frame delta has to be calculated. In a sentence, if object 1 must move from position A to position B, the delta between frame renders must be used to calculate its position per frame. So repeated RDTSC (or timGetTime() as a fallback) checks are not unexpected.

Quote:
I found something interesting, If I put a bpx on the 'ret' instruction at 4D6F49....Then, the flying horizontal window appears asking for the DVD.


That is curious. I'll put my hands up here and say I have no idea how DInput/D3D/DDraw and sice might interact, not least because DX initialization is usually done on a hardware dependant basis (this is the caps thing I keep mentioning). However if I'm understanding you right you set a bp on the ret which does break, and you see the game screen being built behind sice? So something external is calling that chunk of code repeatedly. Perhaps then this is just one message pump implementation and your target has several. On pseudo-code level you might have a top level loop like this:

Code:
while(keepgoing)
{
switch(what_part_of_app_are_we_in)
{
case INTRO:
TopLevelIntroFunc();
case SOMETHINGELSE:
SomethingElseFunc();
}
}


Then inside both TopLevelIntroFunc() and SomethingElseFunc() is a pump - one for each part of the app. Odd but perfectly possible.

Quote:
And why does the bpx at the end of that code bring up Ice right over the intro video?


The more I think about this the stranger that seems. D3D apps simply don't build their display like Windows apps, where it's conceiveable to bp the code after window creation but before ui control creation. D3D apps render everything to an offscreen surface then flip it to the front buffer, so you should never see a D3D app building its display (unless the coder's intention was for you to see it).

Quote:
I don't get what you're saying about how Dinput processes the mouse button press. Usually, when something like a mouse click is performed, a software equivalent of a latch is set. In fact, there are software debouncers that go of on the first Press of a button and latch it. Are you saying that Dinput might miss my mouse press?


DInput won't miss your mouse press because DInput won't look for it. At a certain point in time you have to say to DInput "please go and get the current state of this hardware device right now". Later on you can then say "ok, when I told DInput to go and get the state of the device, the state was saved to a standard variable (on stack or heap), now let me go and see whats in that variable".

Think of it like this:

Code:
User: presses left mouse button
Time: wait 5 seconds
User: releases LMB
Code: DInput->GetDeviceState(mystatevar);
Code: Check mystatevar, finds LMB is NOT pressed

User: presses left mouse button
Code: DInput->GetDeviceState(mystatevar);
Time: wait 5 seconds
User: releases LMB and goes home
Time: wait 14 hours
Code: Check mystatevar, finds LMB IS pressed


Does that make more sense? In Windows, for a UI system, repeated state messages about hardware are mostly useless. All a UI cares about is when button N is pressed and when it's released. A game however will need to continually check the state of button N, perhaps for a rapid-fire option in a first person shooter. Remember that DInput is not just designed for mouse and keyboard input, it's abstracted input from any device - gloves, VR glasses, bodysuits, rumble chairs etc etc.

I was going to do some more to my crackme today but I think I'll leave it and release it ASAP. Perhaps you might want to switch to working on my crackme to get a better feel for working with DX in a controlled environment (where I can at least tell you exactly why you're seeing what you're seeing, in contrast to your current target where I can only make best guesses).

WaxfordSqueers
November 1st, 2006, 13:17
Quote:
[Originally Posted by Silver]I was going to do some more to my crackme today but I think I'll leave it and release it ASAP. Perhaps you might want to switch to working on my crackme to get a better feel for working with DX in a controlled environment (where I can at least tell you exactly why you're seeing what you're seeing, in contrast to your current target where I can only make best guesses).


Hey Silver...this is kind of a good news/bad news kind of reply. You made a comment about this topic being in the newbie section, and I tried to explain that I'm still kinda new to a lot of things. In fact, a while ago, when I was doing an apprenticeship, the journeyman used to prod me with, "What are ya...new??"

I pulled one of my bonehead maneuvers. I had this app on my hard disk and copied it to a blank DVD. Normally, I'd have it in an ISO image, but this time it was in a directory/sub-directory format. When I mounted it to a blank DVD, not thinking very well, I grabbed the entire directory icon and pasted it into my DVD writing software. As you know, what you see on the DVD writing software's file window is the root directory. If you drag a root directory into it, the former root is now a sub-directory. Of course, what should have been the root directory was now blank. No wonder it kept asking me to insert the DVD.

That's the bad news. The good news has to do with a blunder by the software authors, who are not in business anymore. Surprise!! Even after I'd resolved my error, the app began to crash. Dr. Watson revealed the address of the crash and it was in decompression code, which I think was using MMX. I tracked the reason down to problems they had encounterd with the new NVidia cards of that generation. Also, they had problems with processors running at a speed greater than 800 Mhz. There was a patch issued, but it was for general freezes. They kind of kept the other patch hush, hush.

I managed to find the correct patch, on some remote FTP. I applied it, and wouldn't you know it asked for the DVD again. This time, the solution was at hand. I've never seen this before, but the app came in a two DVD set with almost identical files. I have the feeling one was aimed at machines with 512 meg RAM and the other at 1024 M RAM. Anyway, when I got the right one in there, away it went...no protection.

Meanwhile, I went on tracking the flying window with the 'insert DVD' on it. I haven't isolated it yet, but I'm close. There's a CreateFile call looking for a zero-length file on each DVD. One has 512 in it's filename and the other 1024. That's why I think they are memory related. I did most of my tracking with Boundschecker, with reference to IDA.

CreateFile returns a -1, and I think that's because the files are zero length. I think it's main function is to return the filename. There are a few GetLastError/SetLastError's after the create file and eventually a GetFileType after it has looked for either or both files. Right in the vicinity of the GetFileType in Boundschecker, there's an ExtTextOutA with the message to insert the proper DVD.

I haven't traced into that yet. As far as Dinput is concerned, it's using SetupApi and HID.DLL, which is for processing USB ports. I saw reference to both the keyboard and mouse in the Boundschecker printout.

BTW, I highly recommend that any newbie check out Boundschecker and it's VB counterpart Smartcheck. I used Smartcheck once to work through a complex math coprocessor routine using ole32 and oleaut32. Smartcheck showed the math routines and their results, which were building a serial. If you try to trace through code like that with a debugger, it looks like a rat's nest. With either of the tools, however, the exection is serialized for you.

The problem with both of these tools is the long cumbersome amount of data, but you can prune it significantly by suppressing certain functions that crowd the readout. For example, with this app, I suppressed, EnterCriticalSection, LeaveCriticalSection, IsBadReadPtr, InteLockedDecrement/Increment, HeapAlloc and HeapFree. That means those functions did not show up in the read out, and since the readout was thick with them, it made finding the functions I wanted a lot easier.

I also suppressed PeakMessageA, since there was a gazillion of them. All I needed was the PeakMessageW that was left. Also, you can use the 'Find' function on both tools to zero in on certain API functions. When you find these functions, the tools will tell you exactly where the functions are being called in the app and in memory. They also give tou invaluable information as to where pointers to structures, windows, etc., can be found.

Following the order of execution on these tools also gives you a better idea of how to trace through code with a debugger. As anyone knows, who has traced blindly through the codewoods (was that +Ork??), one is always faced with decisions as to which functions to trace and which are a waste of time. Those devious programmers actually include code to send the good guys on wild goose chases. The tools mentioned above can reveal a lot of what's going on with the actual code execution.

Finally, on a deadlisting, as in IDA, sometimes you encounter a function that has no calling code reference. Sometimes that is done on purpose. With 'The Tools', you can work out where those kinds of functions are called from, by going back a few steps in Boundschecker to see where the code was operating before the call was made. Then it's a matter of going back to the deadlisting and carrying on.

With both tools, the right hand window reveals the function or app in which the left hand window code is being executed, PLUS the offset into the app/function. All you have to do is add 400000 to it if it's the main app, and you're off to the races.