Log in

View Full Version : Good old fashioned overlaying


Peres
November 13th, 2006, 08:39
I'm trying to reverse engineer an old dos program which uses overlays. I've read some interesting articles about overlaying techniques and managed to reconstruct the overlay tree from the overlay table contained in the main program, so I actually know the layout of the overlay files and the segments they contain.

The overlay tree yields 22 mutually exclusive combinations of running segments [overlay paths], so here is the question I need to answer: how do I efficiently reverse engineer each leaf of the tree?

There are 2 approaches I can think of:

A) disassemble the main program (using IDA), then load the overlays as additional binary files. The big drawback here is manually fixing the segments to be able to properly navigate the 22 versions of the database.

B) create 22 different executables by merging the main program and the overlay files. This would remove the segment fixing needed in the previous solution, but would leave me with 22 files to be analyzed from scratch.

I'm hoping for someone to suggest a third feasible solution.

Thanks
Peres

naides
November 13th, 2006, 09:42
Question:

What exactly do you mean by saying Reverse engineer each leaf of the tree?

Why Do you HAVE to trace each overlay pathway?

Are you trying to find/modify a key behavior of the program?
Then the name of the game would be to locate zero in the ovl that contains the code for that behavior.

Are you trying to rewrite the prog without overlays?

I think the meaning of the term reverse engineer is too vague in this case and needs more qualifications.

Peres
November 13th, 2006, 10:14
Quote:
[Originally Posted by naides;62340]Question:

Are you trying to rewrite the prog without overlays?



Yessir

I grew bored of protections, so here is my new activity.

Nobody will be hurt if I can't do that, but I would like to be able to play that old game again nevertheless.

Peres

LLXX
November 13th, 2006, 20:41
Perhaps you should think about why overlays were used in the original design.

Programmers don't (well, most of the time) make things more complicated than they should be.

reverser
November 14th, 2006, 07:05
Overlays were the DOS version of Windows' page file, used to work around the DOS memory limitations. Necessary code was loaded on demand and unloaded when not needed. While you could create an EXE with all overlays linked in, it will probably not fit in DOS memory.

Zest
November 22nd, 2006, 23:48
Hi Peres,
May I ask you to share your program with me?
I want to take a look at it.

Regards,
Zest.

Peres
November 23rd, 2006, 04:10
Quote:
[Originally Posted by reverser;62378]While you could create an EXE with all overlays linked in...


As stupid as it may sound, I simply hadn't thought of this simple solution.

Many thanks
Peres