PDA

View Full Version : VM reversing


b3n
July 28th, 2007, 12:31
hello,

after i read 0rps post about his vm i developed my own little vm + compiler. i also used coco/r for the compiler part and a simple switch(opcode) structure to perform the interpretation of the byte code. then i tried to reverse engineer my own application which was not too hard. i wrote the whole thing in c++ and also included some opcodes to check for a debugger or encrypt a few byte code instructions. since the whole thing seems very static to me (the switch structure) i was wondering what i could do to make reverse engineering of my vm harder. it would be great if someone could drop me some ideas on what is possible and what would be useful in respect of using it in a vm.

thanks,

b3n

rendari
July 28th, 2007, 13:07
Themida VM is one of the best VMs out there for frustrating crackers You might want to look at it by downloading Themida unpackmes off of tuts4you, or you could search for a certain tutorial on Code Virtualizer that analyzes its features. I won't post it here because its got instructions on how to patch Code Virtualizer itself, and from what I understand this website is going legit But reading it is a great learning experience (I think).

Pretty much what you want to have are:
-Debug checks (small stuff, like antitrace)
-Extensive obfuscation in the VM itself. Few hundred thousand instructions for every 'real' instruction is what you should aim for. Even better is to let the user decide how much obfuscation they'd like.
-Non-static opcodes (this means VM opcodes for each instruction change from program to program)
-Obfuscation of the actual VM'd code (this means you take the original code during the protection process, insert obfuscation, and then only after that do you convert it to VM opcodes. Pretty wicked thing to do if implemented properly.)
-When a program jumps to the VM, the VM usually stores all the registers that it will be modifying in one place. This is stupid. Scatter the registers all over memory. Have fake registers, that your code will randomly access and change as if they were the real registers. This prevents crackers from setting memory breakpoints on them and observing how they change, and from that guessing what instructions the VM is emulating.

I had other ideas, but this should give you something to chew on for now.

JMI
July 28th, 2007, 14:22
And heck, "nobody" could find the tutorial by putting "tutorial on Code Virtualizer" (without the quotes) in their favorite search engine. Duh!

Regards,

OHPen
July 28th, 2007, 14:25
Hey,

actually I'm also working on an own virtual machine. I decided to kick common techniques in order to try something new. I only can recommand to take beer, a piece of paper and then note down everything you have in mind. You will soon discover that certain of your ideas are not possible to implement. I would suggest then think about modifying your former ideas until you got a working concept. Tatata, there we go.

Hehe, i formerly was in a creativity seminar and they told us that it is sometimes usefull to concentrate on the impossible

Think about it

Cheers.

PAPi

0rp
July 29th, 2007, 04:10
the most important thing is to make the executionflow unfollowable. starforce does this by maintaining some descramble-register which is hard to track offline (the descramble reg is used to decrypt params)

another important thing is crcing (obvious)

nice, but not really required are antitricks. again starforce has used a pretty hardcore one in earlier version (before they became weak
their driver replaces the int1 handler. each time the usermodevm throws an int1, they switch to the kernel vm and execute some important steps. an int1 in kernel is used to switch back to usermode. suffice it to say they check in their kernel vm the int1 handler and are able to detect changes.

another way to make your vm stronger is obfuscation. but using obfuscation is pretty noobish :]
better obfuscation is done on vm-level through messing up load/store instructions a bit.

OHPen
July 29th, 2007, 12:43
I'm also the opinion of that it is a good idea to use a driver within a protection. Less to fuck in system internals than just to make is more difficult to trace. Only a few reverser are able to debug and trace code in ring-0 what is a big advantage for a protectionist.

The one and only problem is that you have to pay for driver certification which is required for vista, which will be the future for any protection. I know that it is possible to load a driver into vista on your own risk but this is not an satisfying alternative in my eyes.

OHPen

rendari
July 29th, 2007, 14:24
Problem with drivers is after Starforce and Sony fiasco's, your average Joe hates them. Use drivers in your protection, and I can more or less guarantee that it will hurt your sales. However, when security is the only thing in question, and not user satisfaction, then go for it.

b3n
July 30th, 2007, 05:27
thank you very much for all the good ideas, this is really helping me a lot to figure out whats worth having a look at. thanks!

b3n
August 5th, 2007, 05:54
hello,

i got another question about reverse engineering a virtual machine. when i reverse engineered my own virtual machine i analyzed it from beginning to end to understand what is going on. now i was wondering, if someone was interested in just a mapping of opcodes to instructions performed by the vm, if there are other ways of creating this mapping than reversing every single case of a switch structure (in my case its a switch structure). it would be nice if someone could point me to other ways of achieving this goal.

thank you,
b3n

blabberer
August 5th, 2007, 11:33
i dont know if you have read rolf rolles article in openrce it dealt with unpacking ?? unravelling ?? some kind of unwhatever using ida writing procressor modules and such

check it out may be you get a few ideas

https://www.openrce.org/articles/full_view/28

b3n
August 6th, 2007, 09:06
thanks for the link blabberer,

but as far as i understand the article it is still necessary to fully reverse engineer the vm instruction set before you can develop an ida plugin. what i was thinking about is if there are ways to analyze what the byte code of the vm does without having to analyze the instruction set before. i thought of something as a program logging what the vm does (if that is possible).

cheers,
b3n

OHPen
August 6th, 2007, 20:01
hey,

@b3n: this sounds a bit like : "is there a way to examine algorithm xyz without reversing the validation sheme inside the binary ? "

I dont think that there is another possibility than reverse engineering the vm structure + handling + instructions. Thats why vms are that pain in the ass as they are if implemented well.

OHPen

rendari
August 7th, 2007, 12:01
Only thing I could think of is the lazy mans way out:
observe registers and stack before execution of instruction, and after. Also set a lot of memory breakpoints and see which ones get triggered during the execution of that one VM instruction. From that, guess what the instruction it is.

scherzo
August 7th, 2007, 16:03
Quote:
[Originally Posted by JMI;67457]And heck, "nobody" could find the tutorial by putting "tutorial on Code Virtualizer" (without the quotes) in their favorite search engine. Duh!

Regards,




Quote:
[Originally Posted by rendari]I won't post it here because its got instructions on how to patch Code Virtualizer itself, and from what I understand this website is going legit But reading it is a great learning experience (I think).


That part was only for fun
You can find a legit copy of it here http://rapidshare.com/files/16968098/Inside_Code_Virtualizer.rar

scherzo

rendari
August 7th, 2007, 17:40
Yeah, great work scherzo

disavowed
August 9th, 2007, 13:16
Quote:
[Originally Posted by rendari;67476]Problem with drivers is after Starforce and Sony fiasco's, your average Joe hates them.

Your average Joe doesn't know what a driver is, nor does he understand the difference between user-mode and kernel-mode, etc.

rendari
August 9th, 2007, 13:26
Exactly, so when everyone who has no clue what they are talking about start yelling: "OMG that game has drivers that will fuxx0r your comput0r" who's gonna buy the game?

disavowed
August 13th, 2007, 18:16
Yes, but that argument could be made about anything; it has nothing to do with drivers. For example, "OMG that game uses floating-point operations that will fuxx0r your comput0r"... same deal.

OHPen
August 14th, 2007, 08:56
i agree with disavowed. most persons which are not interested in computers and similar stuff consider each unknown technical detail or fact to be a potential virus, problem, bad thing and so on. Those people tend to predominantly see the "dark side" of technologie.

But it seems that the topic is goin' to be a little bit offtopic, doesn't ?



OHPen aka PAPiLLiON

disavowed
August 16th, 2007, 00:25
Quote:
[Originally Posted by OHPen;67783]But it seems that the topic is goin' to be a little bit offtopic, doesn't ?

I ran out of good advice to give about 8 months ago. My only purpose here now is to troll and to force threads off-topic

OHPen
August 16th, 2007, 09:51
lol