Hi guys,

A question and a thought... I've been playing with global hooks recently, leading me to my question. A global hookproc must be coded in a DLL which is then loaded into every process' address space. The normal DLL rules apply, so even though it's your DLL being loaded the data segments are unique to each application. One way around this is to create a DLL with a shared segment - that way, all the DLL instances and your "parent" process (the one that installed the hook) can communicate with each other and retrieve data.

All well and good, except where it comes to debugging. First of all, you can't directly debug a hookproc. My VS6 won't break on any breakpoints set in the hookproc (which is somewhat understandable), so I have to debug via PostMessage(). Second, if you did break to debug in the hookproc you're going to immediately screw the Windows messagepump, which will stall until your debug is finished and the hookproc returns. Third, there are always going to be multiple instances of the hookproc, all of which have access to the shared segment. So if data in the shared segment changes, you have no way to detect which instance did the changing. You can implement a mutex/csection or some other protective mechanism, but then you still have to somehow register which instance of the DLL (attached to a specific process) made the change. To clarify that a bit - you, as the coder, are not in control of where, when and how a new instance of the DLL is loaded. It's like thread synchronization, but with a moving target.

So the question: what's the best way to debug a hookproc and get data into it from the main application (my app) that loaded the hook?

The data question is a chicken-and-egg problem. Once you load a hook you have no way to get data into it other than via the shared segment, but you have to load the hook to have somewhere to send the data. You can't load the hook then call an exported func from the DLL because that's still the instance attached to your process. You can only include basic type data in the shared segment.

I've read up on this a bit and I can't see a way round it. All anyone seems to do with hooks is to Postmessage something to another app and that's it. I'm trying to do something more complex.

The thought: this could be an interesting way to make reversing more difficult. Adding some critical code into a hook would cause problems. The reverser would see multiple instances of the DLL, the data in the segment changing, but no way to know which DLL made the change and under what conditions. And of course breaking into the hookproc brings the whole thing crashing down...

Perhaps someone can see a large flaw in my thinking

I guess named pipes could be an option, letting the "main process" run a multithreaded named pipe server, and having all the hooks report in and identify themselves upon loading, and then maintain an established pipe to the server on which they report anything you like from that point on (and receive commands from the main process if desired). TCP/IP would work just the same practically, but it would just be a bit more big and bulky in this case, also adding possible problems with personal firewalls and such, also finally being "unnecessarily far away from the operating system" in general, if you know what I mean.

This is an instance where I use Event objects. Since it's interprocess communication you need.

For example, if an event occurs in your debug hook ( for example, perhaps you've redirected some code so yours gets notified ) , you simply set a data variable in your shared segment with an identifier - for example, the DLL can just output it's processes name into a fixed length string var in the shared memory. It could of course be any type of data you are looking for. Then the DLL triggers a global system event object ( SetEvent() ). Your app, which is watching, has a thread which is waiting on the event object. When the event object gets signalled, that thread reads the shared memory area to get the data, and then resets the event object.

I've used this technique quite a bit and it works quite well. Since the "hook proc" simply grabs the data it wants and triggers the event, it is fast. And a good way to synchronize if you need to be absolutely sure that no process edits the data until you are done, you simply add a WaitFOrSingleObject in the debug hook itself, that waits for the event it just set. Then it works like this: If not one is watching the event, the event gets set in the debug hook and then the debug hook calls WaitForSingleObject on the event, which of course is now set, so it continues fine. IF someone is watching the event, when they receive it, the first thing they do is call ResetEvent() before anything else. This causes the debug hook to now wait for the external app since it's now stuck on a WaitForSingleObject. Then the processing app calls SetEvent on the same event object itself to let the hook continue. It seems to work fine since I've found that threads that are waiting get serviced before the thread that set the event.

Anyway yes that's confusing. Suffice it to say I tend to lean toward using Event objects mostly for interprocess (especially if you already have a DLL with a shared memory segment - so now you already have a way to share data, you just need a way to coordinate it - some sort of trigger - which is what event objects exist for).

-nt20

Hmm, in principle I can understand/agree with both your ideas. 2 things:



Surely that would bring the entire pump grinding to a halt, because the hook has to pass the message before it gets given to the app and taken out the queue. Thus the target app is going to stall, and if it's a time critical app (such as a game, or network app) then there will be ongoing synchronization problems.

Here's a hypothetical situation. Let's say I code a hook that removes certain messages from the queue. My hookproc would look like this: (pseudocode)



No problems here. Now imagine I want this to be a bit more dynamic, for example we'll knock out messages based on the contents of a file. New hookproc:



No problems. Now the question is how to fill out the knock-these-messages-out array. We can code a function to load it:



(Ignore the hybrid code, you get the idea hopefully). Now here's the question. If I hard-code the filename into the file.open() statement, we have no problems. Here's the DLLMain:



LoadKnockoutMessages() is called when the DLL attaches to a process. The function loads the data into the process local, DLL global var called g_knock-these-messages-out. The hookproc is run, it can access the data loaded into the var in this process and everything works.

Now consider if I change the file open code to file.open(g_szFilename);. Let's say I add an exported function to the DLL as follows:



So my main app (the one that installs the hook) can call GetProcAddress() for SetFilename(), then call SetFilename() with the knockout file. We have to remove LoadKnockoutMessages() from DLLMain because the filename isn't in the global var when the DLL is attached to a process. We can put it in SetFilename instead.

Except here's the problem. My app is only calling SetFilename() for the instance of the DLL attached to itself.

When the DLL attaches to another process, only the code in DLLMain is executed before the hook is installed. But the hook must have called LoadKnockoutMessages() before the hook is active, otherwise the config will be wrong. So we have a chicken and the egg situation.

The DLL is loaded by Windows and auto-attached to all the necessary processes. Those processes don't know they have to call SetFilename() when the DLL attaches, and even if they did they wouldn't know what filename to pass it.

The data in the knockout array is too "complex" to store in a shared segment. I could store the filename in the shared segment, but I've still got exactly the same problem of how to get the DLL instances to do the loading with the filename.

This is my logic problem. I can't logically see a way around this, because you can't force a DLL attached to another process to call a function on demand. I can see where you guys are coming from with the interprocess comms, pipes etc, but that principle only works if you can force a function to be called in the local context of each DLL instance as it attaches to a process, and in the case of hooks it's windows that controls the calling...

I hope that made sense, it's easier to explain what I don't understand on a piece of paper

I don't understand - do you want each app to have it's own knock-out-messages file ? Because if you have one global file, then use the global solution : In this case you would simply set the filename once in shared mem - when your app loaded the DLL - and before you set the hook. If you have multiple files, just use the same filename (just use one hardcoded filename) but put the config file into the same directory as the exe so it loads it from there..so no need to pass arguments...

-nt20

Really a "hook dll" in a global state overall is not a good solution depending on what you are trying to accomplish. If you need to get data from certain apps back to a controller app, you should be in control of the injection and just use CreateRemoteThread trick to do so. If the Hook DLL is just used to alter behavior it should be designed to not require communication with an outside program since global hooks are such a system bottleneck. In this case if you need a global hook then just simply place the <filename> file as a config file which goes into the hooked EXE's directory, this way the hook will do nothing to an app unless that file is found. (Then just set the DLL to CreateFile("filename" without a full path, so it just looks in local directory. Windows uses EXE dir by default first).



-nt20

It is also not very hard to get the full path of the main executable of the process you're executing inside, if that would be needed for some reason.

And I agree about placing the needed data in a file in the directory of each targeted process' main executable. Or will there be multiple process instances of each executable too? Damn it's a pain to only get to know a tenth of a problem at a time, and then have to refine the solution for each new "clue" you get about what the problem really is. Wouldn't it be much easier if we just got the entire problem specification from the start?

Btw, in case the file-in-exe-dir solution won't work for some yet-to-be-disclosed reason , I don't quite understand why the DLLMain can't do a one-time interprocess communication with the "mother process", which will in turn point out a suitable file for it to load the info from, which it will then do, into its current process' memory space, and then the hook will be activated with all the needed data loaded and ready?

Now, please tell us the next part of the problem, which will rule out our current generation of solutions too.

Think of it like cracking Christmas. Each post, you get to unwrap one layer of paper around your present

We can't place a file in each targetted process' directory because every process is targetted.

Partially the problem is I can't be more specific. The following assumptions are valid: this MUST be implemented as a hook, the hook MUST read data from a file, the hook MUST receive the filename at runtime (rather than being hard coded). Annoyingly it's easy to explain on paper and apparently hard for me to communicate properly in a forum.



.......

Hm, you know what... I just typed a reply explaining why that won't work, and in the process I figured out how to make it work . I'll need to implement some control variables to make sure the right actions are being taken in DLLMain()...

Thanks guys

Ok, I was just about to suggest writing the problem down on a paper, and then scanning the paper and uploading it here as an attachment.

I'm also still curious about why my solution with a short initial IPC in the DLLMain won't work? But maybe that's what you actually mean with these "control variables"?

What about.. debugging with SoftIce?

Just so anyone reading this thread in the future knows...

The problem is quite simply forcing all instances of a hook DLL to call an instance-local function to update some instance-local data based on the contents of a instance-global (shared segment) variable. This forced call must happen at any arbitrary point in time.

The way I've got round this in the meantime is by flooding all top level windows with a very fake WM_ message. When the hookproc receives the message it calls the instance-local function. The reason this is awkward is because hooks actually don't operate inside the "normal" messagepump, despite what MS says. They also only hook on messages for their specific type (obviously) - keyboard, mouse etc... It's messy, but it works.

Thanks guys, bouncing the idea around led me to the solution...

From your last post I am guessing you are using SetWindowsHook(Ex)()? These functions are really limited, I played around with them and found them to be inadequate for most purposes. Please describe your problem in more detail so we can suggest a proper solution

BTW, there is no need for a "very fake WM_Message", you can register unique ones with RegisterMessag() The first instance of your DLL that gets loaded will register the message and all others will get the initially registered value when calling RegisterMessage.


P.S. About that xmas gift stuff: I am not allowed to post my thoughts about that here but they contain things like a dark room, ropes and a water hose.

And btw, there is a very simple solution to the reload issue: Take a single byte in your shared segment. Whenever you require a reload, increment it by one.

The DLLs initialize their side of the counter (local var) with zero. Then a thread kicks in which reloads the filter table whenever the counter in the shared segment does not equal the internal counter. Thus it will reload the table instantly (initializing it for the first time), then set the internal counter to the value it picked up from the shared segment moments ago, then sleeps for a certain amount of time, goes to check the counter, and da capo.

I tried to code quite a few iterations to solve this problem, but I'm not a hooking expert so these comments are based on what I experienced myself. That could be either due to the behaviour of win32, or my buggy code



I am indeed. As far as I'm aware I have no choice in this, I need to filter all messages going to all processes through my app(/hook)



I've not heard of RegisterMessage() before, MSDN doesn't find any results.



That still doesn't solve the problem. You can't put a hookproc to sleep for any period of time or thread-block it. If you do, the whole messagepump comes to a grinding halt whilst everything waits for the hookproc to process the message. I tried this, and if a hookproc doesn't return "quickly" the app that's been hooked becomes a non-responsive task.

So if you can't block, sleep or loop it, how to do force a reload - or, in your solution, force the process local instance of the DLL to check a var in the shared segment?

Answer is, you can't. The only way to absolutely guarantee that a hookproc will do something at a given time is by posting a message to it via another process (for some reason internal posting of messages bypasses the hookproc). Your solution is the same as mine in principle - get the hookproc to do the updating. The difference is I use a WM_ posted to the process and you use a signalling byte in the shared seg. IMO neither of these solutions are particularly robust or adequate.

The entire problem is based around the DLL instance being process local. You can have a code snippet that says "if counter in shared seg > counter locally, call this function", but you have no way to initiate the execution of that code. All you can do is put the code in DLLMain (one time execution only) or in the hookproc, which is unreliable and slow.

I don't think I'm explaining this particularly well, unfortunately.

I think he means RegisterWindowMessage(). I did not suggest this due to the possible mentioned problems with the hook not actually receiving all messages, but rather only the ones in the given category when calling SetWindowsHookEx(). I haven't tried it myself though, so there might be a way to get it to work, although possibly messy.

I think he might be suggesting spawning a separate thread for this from DLLMain(). This is perfectly possible, but for global hooks you'd rather not spawn a new thread in every application in the system, due to the possible compatibility problems that may result from this in different applications. Also for performance reasons, even if this is not really an issue if implemented correctly.

Aah, feels nice to dust off the good old mind reading skills a bit.

*sigh*

Okay, I admit I was a bit overtired and too lazy to check the correct name (the downside of auto-completion in your IDE), RegisterWindowMessage() is correct.

And by "A thread kicks in" I actually meant "a new thread is spawned".

However, it is not even necessary to put this into a new thread. The hook function can take care of all the required stuff perfectly by itself. The overhead for checking a byte change is almost zero. And reloading a small file should be no problem as well.

When working with MFC, I often override PreTranslateMessage() or OnCommand() to do all kinds of debug stuff, depending on the particular case.

As I have stated before, please present the bigger picture. We can't help you if you don't describe the whole problem, that is the kind of thing you want to do and what you want to achieve with this - including all the details that are missing as of now.

If you need to have filter lists for each process, just create one master table file in a fixed location and provide the names of the particular data files for a given process. This way, each dll instance can load their specific filter list.


A completely different approach: Reserve some space in each process by the DLL with VirtualAlloc(), then put the filter list into this memory location. Use a named pipe or whatever to communicate the process-local memory address and process ID to your controller application. Each instance of the DLL goes to reserve some memory block, and sends the process ID and the process-local address to your app.
The page needs to have a suspend flag that makes your hook function idle (this is just a safety measure so that no wrong filtering happens) for a very short moment. You can implement a counter to avoid deadlocks here.

You can use three states for the flag: 0 means no list here, bypass filtering. 1 means go ahead, list is okay, and 2 means please wait, updating. (choose whatever values you like...)
Initially, the filter list will be empty. Now write the table and set the flag to "Go!". The hook function will start filtering from then on. Now when updating the list for a particular process, set the flag to "suspend", then write the new table after a short delay to avoid writing to the table while the hook function is looking up a filter value. Afterwards set the flag back to "Go!".

This will remedy your worries about delayed filter list updating.

BTW, the delay seems to be minimal, I just coded a raw sample and it seems to be working fine.

BTW, the most unreliable part of your approach is the use of SetWindowsHookEx as there are tons of messages that will not be caught there.

Really Silver now you're using a WM to do the same thing a global event object would do! - which is what I tend to use , so I'm biased. But really if you want load notification just have a local thread waiting for an event object, and then DLLMain simply does a SetEvent() call....

Just look more into the functions that allow cross-process..

But that's my final hype about Event Objects LOL - I just think they get overlooked a lot.

-nt20

Nope nikolatesla20, he actually isn't. The difference lies in this:

And to quote Silver again:

Hence, the difference is that with your method (i.e. using event objects) you need to inject a new thread into all the hooked processes that lies waiting for the event, while the WM-method doesn't. This is quite a big difference indeed.

Just a last message to say thanks to everyone for their thoughts & input, it got me to a working solution for now.

And apologies for not giving more info - I know it's frustrating but if I was to be any more explicit in what I'm trying to do I would effectively identify the target.

So at least you could outline your working solution now :P
(Sorry, but I am pretty curious!)

Ok, I agree with the arbitrary time thing. However, you could still use a SetEvent and a global shared data segment variable to move the data. I've done it that way very much.

The second point is incorrect however. It was my understanding that Silver wanted to collect data from the hooked app and go out - not to send data into the hooked app (he was just trying to figure out the best way to limit the hooked data)..so maybe I misunderstood.

As far as getting data out...in my understanding - The thread that waits is in the monitoring program. The DLL itself simply does a SetEvent call on DLLMain , for example, to inform the external program that it has loaded, or in whatever function is has the collects the data. There is no other thread in the DLL. SetEvent is a non blocking call. The DLL only has to fill in the shared memory variable (which he's already doing) and then call SetEvent (which is a non blocking call, and which he is also doing but now with a WM message)- these two items will occur nicely quick.

But I think I must have gotten it backwards eh... maybe he really meant to send data in. In that case yeah if you're hooking WM then the obvious way of course would be to use a WM

All case in point of the problem not being defined clearly enough.

-nt20

According to Silver's example with the "LoadKnockoutMessages()" function earlier in this thread, he did indeed mean that the data goes in the direction to the hook DLL, not from it.

So, it was not only my first point that was correct, but they were indeed both correct.

Consider me spanked.

But instead of taking responsibility, I'll blame it on "forum rules"... Or "George Bush"... or "New World Order"

Laola, my solution was to post a fake WM_ to all processes. My main app can obviously issue the fake WM_ at will, which meets the "update at arbitrary point" criteria. My hook sees the fake WM_ and does the updating etc. It has to be a fake WM_ because messages are specific to type (as in, a mouse hook won't receive WM_USER + n). To clarify by fake I mean, for example, WM_MOUSEMOVE with normally "invalid" parameters sent to a mouse hook.

It seems a whole lot easier to make a KMD, and hook the functions through the system service table.
An example can be find on http://www.windowsitlibrary.com/Content/356/06/2.html