Log in

View Full Version : Can't recompile hex-rays decompiler output


abracadabra
November 26th, 2011, 11:05
What is the problem....
I have a piece of freeware, closed-source software which has stopped working as of an operating system update that occurred a year or so ago. The software developer is MIA and will not answer user emails, etc, and is no longer developing. The software throws a bad access exception. I would like to fix the big by decompiling or disassembling the software, fixing the issue, and recompiling/reassembling. I am a fairly experienced C/C++ developer so I'm hoping to be able to get some kind of compileable source code from Hex Rays, then recompile the program, debug it using my usual toolchain, and recompile a fixed version. Unfortunately, I am having a good deal of trouble compiling the decompiler output. Specifically, it seems like it is using classes/namespaces/structs (eg Foo::something()) before they are declared. I would think that the "generate C Header file" command would create a header file to declare all of these classes/member functions would work, but the resulting header file only has a handful of class declarations which are not really relevant. When compiling the file (just gcc outputFile.c) it raises errors about all of these declarations.

What is the protection.....
There is no protection per se, just a bug I'd like to fix.

What tools are you using....
IDA 6.1 with Hex Rays decompiler 1.5.

What tutorials have you read....
Skimmed inAbOol's IDA tutorial (didn't mention the decompiler), watched Hex Ray's videos

Show your output listing WITH comments....
There are many function declarations like
Code:

int __cdecl Foo::PreDestructor(int a1);

but Foo is not declared as a namespace, class, or struct anywhere, so the compiler barfs.

NOW ask your question....
Am I missing a step to generate a declaration of all of these namespace/class/structs? Or is trying to recompile the decompiler output a trivial pursuit, and not the intended use of the software? My reason for wanting to do it this way is that I can't figure out a good way to debug the binary in IDA itself - the binary is a plugin, so it runs as part of another program, and the host operating system doesn't run IDA/Hex Rays Decompiler, so I'd have to set up a debugging server... etc.

Thanks in advance for any advice!

Aimless
November 26th, 2011, 13:03
But you are looking at something that is not possible.

As it stands today, you cannot decompile a program, with the exception of the most basic of 'toy' programs, and re compile them again.

This decompilation and recompilation, is the holy grail --- being searched by researchers, doctorates, reversers and people of all variables and make.

And don't tell me you fell for the oldest marketing trick in the book. Truthful information, but slightly less than you would have liked to make the correct decision. The 'decompiler' in the Hex-Rays decompiler does not work like a proper decompiler. It's simply to add up your assembly lines into a more concise C/C++ like format.

When reading assembly, sometimes, hundreds of lines can be converted into a simple 3-5 line C/C++ statements. The decompiler is to help you VIEW those assembly programs where you need to see what it's doing in C/C++ 'like' statements. Caveat here being C/C++ LIKE.

If you got the decompiler thinking 'Aha! NOW I can decompile, get source code like files, add my own stuff, and VOILA!! Instant new functionality!!' then sorry to dash your hopes but its not going to be like that. DECOMPILER had different connotations. You seemed to have taken this literally. Won't do.

Sorry to burst your bubble, but there is no program, at the moment, that decompiles, allows you to modify and successfully recompiles.

Have Phun

abracadabra
November 26th, 2011, 13:50
Thanks Aimless - that's what I figured, but I was a bit confused about the terminology used. This is, after all, "the newbie forum", right?

Do you, or does anyone else, have any tips on how I might solve the problem using IDA without decompiling? The issue is that this is a Mac OS X binary file; I need to be able to modify the assembly (after tracking down the problem, which will be nontrivial but I at least have found the assembly for the offending function) and regenerate the binary. I also need to do all of the debugging from a Windows machine, while running the binary on a Mac. When I go to "produce EXE file" (I believe this can create binaries of various types), I get "This type of file is not supported". Would it be possible for me to export the asm and assemble it on the Mac as a Mac binary? Seems very roundabout but I would like to at least try to solve the issue.

Aimless
November 27th, 2011, 00:48
If you look at Ilfak (IDA Pro's creator) commenting, you would realize that he himself mentions that OUTPUT AS EXE is simply for 'toy' programs. It cannot be done for real world programs.

The only way I know to modify a binary is:

1. Understand the structure of the binary (for ex: Windows EXE have a PE/NE format)

2. If the modification you want, works with simple byte sized modifications, and without increasing the length of the binary, then use a hex-editor (Ultraedit, in hex mode. Quite good). Simply overwrite the offending chunks with your own bytes.

3. If the modification you want does not work in simple byte sized modifications, but requires additional space, then you will have to modify your executable (assuming you are aware of a MAC binary executable structure). Then, it gets complicated in a hurry.

(Of course, for points 2 and 3, you need to understand the functionality you wish to have, in binary! If you wish to use a hex editor, that is)

4. Prepare your own DLL with its own functionality. Then, load it into the executable space. After, of course, modifying the executable to call the function in the DLL when the errant function in the executable actually executes (Am I even making sense to you here?)

Unfortunately, this is how I do it in the Windows world. Am completely unawares of how it happens in the Mac world.

Though, I realize that Macs work on FreeBSD internally (I mean, they have unix underlings), I don't think they use the ELF format.

Also, the way to work it out (creation of binaries on mac) is usually via Objective-C and XCode (Check if GNU utilities offer you an Objective-C program --- I doubt it personally). Effectively, this means you'll have to check if XCode can assemble programs written in pure assembly (not that I think they will).

Another issue here is all Mac programs (especially ones for Snow and Lion) are predominantly 64-bit. I have no ideas if GNU utils, if they offer an objective-C compiler (very rare in itself) also offer a 64-bit version. If that is the case you have an additional problem on hand.

Either way, my suggestion is that you need to go to Fravia's archives, and figure out how to modify binaries (not for reversing, but for ADDING code) and then come back here. While that is not going to get to you help you in what you want, at least you'll understand what's really going on behind that hood.

Meanwhile, we are not going anywhere. We will be waiting when you come back.

Have Phun

abracadabra
November 27th, 2011, 16:54
So it sounds like the process is

1) Find the offending function
2) Try to fix the function by making byte-size changes to the hex representation of the binary
3) If this won't work, write a working version of the function and compile it to a separate library
4) Make byte-size changes to the binary using a hex editor so that rather than calling the offending function, it calls the function in the library

I am not hopeful that I will be able to fix the problem using step 2, but I will give it a try, because it makes sense. I don't have the skills to carry out steps 3 and 4, but I may try to learn.

I believe this binary is pure C++, because I know the framework which was used to create it (based on what the author said and funciton names). Do you have any tips on specific articles in Fravia's archive to become familiar with? I had not known of their existence until now.

Maximus
November 30th, 2011, 10:23
you do not understand...

it is not an 'offending instruction'. It is (almost) never that case. Do you ever write code like *(void*)null = 5; ? because this is the only case where you'd have an 'offending instruction'.

Find the reason of an AV means discovering what went wrong and why, and then look for a fix somewhere.
If you wish to do it by yourself, you'd better get a strong grasp of asm, reversing, and patching..
deep C/C++ knowledge is almost useless, unless i.e. your C++ knowledge extends to know how the polymorphic pointers are stored and accessed i memory..

I'm not saying you CANT do it - I'm just saying that you need to learn the proper skills for doing it.

1) learn what the whole function where the error is doing.
2) understand which/what is causing the crash, and backtrack the malfunction.
3A) backtrack the origin, if possible, and fix it.
3B) edit the code to add a workaround for when the bad condition arise, letting it either continue its work or 'return gracefully'.

Good luck.

evaluator
December 26th, 2011, 04:07
you should not care about "Foo::PreDestructor"
those mostly are RTL-functions.
for recompiling you need to find & rip only author-made functions, then put into new project..

is it newYer gift??