I know that we're not supposed to mention specific targets, but this isn't exactly illegal I don't think. I'm also not sure if this belongs in the advanced section, but it is programming related so....

I use Procomm Plus (symantec app) at work. It's a terminal emulator with it's own scripting language for advanced (and not so advanced) macros and such.

It has a very limited set of dialog controls that you can use, but it does let you call functions in specifically coded dll's. So I wrote a small example that lets me use a listview (which I've grown to really like recently). Everything works fine, except I'm stuck at passing data back to the script from the library. Of course, I can pass a return value back, but I mean at arbitrary times, like in response to a window message and the like.

I can use dde, but really really really don't want to cause I really really really dislike dde. I can't use window messages the normal way, since any exernal function calls have to reside in specifically coded dll's. I also can't use atoms for the same reason.

One of my coworkers pointed me to a website with (non-official) support for Aspect (the scripting language in question). I sent the webmaster an email asking what the best method for doing this would be. He promptly replied that he'd never written any custom dll's for use with Aspect, but if I wanted, he would forward my question to a friend.

His friend turns out to be one of the original developers of Aspect which is great. That guy promptly replied that he no longer works with Symantec, but does do contracting work developing custom scripting solutions. He also said that he doesn't mind answering the occasional question. However he hasn't replied back (after the 1st email) yet so I'm not sure how willing he is to answer my question. He did hint that there are a few undocumented variables for use with Aspect that can be used with one of the included libraries that allow you to create and respond to custom messages. He wasn't specific, but the exported functions in that dll are named fairly explicitly so I can narrow down those functions that sound relevant.

My question is this:

I know that if I call one or more functions within a specific library from my aspect script, I will be able to do some interprocess communication via window messages. I have no examples of doing this and there is no documentation. This is going to require a ton of trial and error if this guy doesn't provide an example snippet.

So is this something that is feasible for me to reverse engineer? ....or will I likely be spending a ton of time on this with no reasonable likelihood of accomplishing anything?

If it is feasible, are there any similar tuts or essays floating around that will be of help?


thanks,
will

(sorry for the long post)

Also, in the email from the developer guy, he stated that variable data is passed by ref to the dll functions, so I could modify that data from the dll which could be useful. However, I couldn't seem to get that to work from my dll. It might be because of my poor masm programming skills, but I'm certain that it can't be done.

This may be useful for you. Take a look.

It is very hard to answer the general question if it's possible to do this in reasonable time or not. It very much depends on exactly how much knowledge you have about what these undocumented functions are supposed to do, how good you can read assemly code, how big the DLL is, how the DLL is written and so on.

My approach would be to find good "pinpointing targets" based on the knowledge I already might have about the functions. E.g. if you know that one of the functions might send a window message when called with the message number and target window handles as parameter, I would set my debugger to break on a certain message, and then try to call all functions with parameters including the number of this specific message and so on. If the debugger breaks, you can go from there. I would also start by taking a quick glance at the dead-listing of all the dll-exports in IDA, to see if there are any apparent hints. E.g. is the SendMessage/PostMessage functions referenced at many points in the DLL? If there are only a few references, you can concentrate on reversing the exported functions that contains these references, and go from there.

In any way, you can often reverse much more complex things than you initially think, if you just use some clever techniques that make the most out of your initial knowledge.


dELTA

Thanks to both of you for the quick replies!


squidge:
That example may be an alternate method that I hadn't even considered. It looks at first glance that it does pretty much exactly what I want to do, although it does it by using predefined global variables. Those variables are documented, although the techniques used in the example that you posted are not. Very nice!

delta:
I didn't have the first real intelligent clue on where to start with this whole mess, but you've definitely provided me a good zen starting point. If squidge's method doesn't get me where I need to be and that aspect guy doesn't have the time, then I'll have no real alternative then diving in head first with the real reversing.


Oh, and squidge? I didn't come across that example in my googling. Did you write that? If not, and you found it on the web, would you care to share the address? There may be other goodies.


cheers,
will

Doh! If I had read the comments then I'd know that you didn't code it.

I found it on google some time ago, and so just attached it here straight from my downloads dir. Hopefully it'll help you out.

So, what is the address to your downloads dir then?


dELTA

FILE:\\F:\www-ftp-downloads

Does not share very well across the internet though unfortunately

Well thanks for sharing.

So do you work with procomm much? The only reversing I've done so far with it is patching aspect32.dll to get rid of that damn 'are you sure you want to close with active scripts' messagebox. Then I found a script command to turn off that prompt. It was still fun though.

I'd appreciate it if someone could verify my code. The aspect guy emailed me back confirming the script does pass variables byref so they can be changed in the dll.



;the varArgs param is a far array of far pointers to arguments passed
;for this example the other params are irrelevant

;this moves the letter 'A' into the 1st byte of that string
;but when I print that string from the script after
;returning from the dll function call
;that string doesn't change

Could someone tell me what's wrong with my code?


thanks,
will

Have you verified that the pointer you pick out of the parameter array is really the right one?

Are you sure that it's a C-style string pointer? (otherwise you could e.g. be writing into the size field of a Pascal type string, or whatever)

Anyway, it should be very easy to verify in a debugger that the address you are writing to is actually inside the correct string, right?


dELTA

Rather than trying to change strings via a dll, when not pass a string to the DLL and see how it is represented first? Borland's Ansistring gives you a pointer, but the first dword at the position is another pointer which gives you the actual string. Make sure your's is not the same kind of thing.

I finally figured it out. In the script's event handler code, under one button I put the dll call code, and under another button I put the code to print the variable again to show that it'd been changed. When I cut/pasted the single line that printed the variable's contents to the screen from the other button to the line immediately following the dll call it worked fine. It's really quite strange.

i think you're missing the point, i've had the similar problem with my c++ code, but i figured out that since its reference you're not modifying the actual thing because you're returning a pointer. i believe the easiest way is to declare some type of that same object but with an updated string like you're doing, instead of adding the output to the bottom of the call, that should be the easiest way.

notice that with your way, your string is still unchanged.

The reason that it wasn't working is because in the aspect script's message loop I was leaving the stack, so the contents of that variable was getting reset. Once I realized that, the script was fixed.

It does pass variables by ref to the dll, so I can just change it and return back to the script. There are also some memory functions availabe in the scripting lang, so I've been using those as well.

thanks,
will