Log in

View Full Version : Custom calling conventions in IDA


Tramb
May 13th, 2004, 16:55
Hello everybody,

I am a newbie at reverse engineering (because I want to RE a good old game, Cruise for a Corpse, to reimplement it portably, for those who care ) but an experimented programmer in C++ and assembly (not much x86 though) (and some interesting academic languages )

I noticed in IDA that when I press Y on a function, I can specify parameters for _cdecl functions and then IDA would infer some information about parameters and comment the "push" with parameters name at caller's location. Which is good

But there are some functions in the .EXE that take their parameters as AX, BX or even different non standard conventions. Is there a way I can tell IDA how to map registers and input parameters so that I have comments at caller's site just like

mov bx, -1 ; fileHandle
?

That would save me a bit of typing


Thanks in advance and sorry for any English mistake!

Bertrand

Hopcode
May 14th, 2004, 16:35
Quote:
[Originally Posted by Tramb]Hello everybody,

I am a newbie at reverse engineering (because I want to RE a good old game, Cruise for a Corpse, to reimplement it portably, for those who care ) but an experimented programmer in C++ and assembly (not much x86 though) (and some interesting academic languages )

I noticed in IDA that when I press Y on a function, I can specify parameters for _cdecl functions and then IDA would infer some information about parameters and comment the "push" with parameters name at caller's location. Which is good

But there are some functions in the .EXE that take their parameters as AX, BX or even different non standard conventions. Is there a way I can tell IDA how to map registers and input parameters so that I have comments at caller's site just like

mov bx, -1 ; fileHandle
?

That would save me a bit of typing


Thanks in advance and sorry for any English mistake!

Bertrand


convention using registers could be fastcall calling convention

Tramb
May 15th, 2004, 05:02
Quote:
[Originally Posted by Hopcode]convention using registers could be fastcall calling convention


I thought this, too.
But no
fastcall, if I'm correct, would pass parameters in DX, CX and in this program, they use AX, BX...
I suppose it was pure assembly code linked, not compiler generated one.
Yet I'd LOVE to be able to tell IDA the mapping between registers and formal parameters

Polaris
May 15th, 2004, 09:23
Quote:
[Originally Posted by Tramb]I thought this, too.
But no
fastcall, if I'm correct, would pass parameters in DX, CX and in this program, they use AX, BX...
I suppose it was pure assembly code linked, not compiler generated one.
Yet I'd LOVE to be able to tell IDA the mapping between registers and formal parameters


I think that the only way for propagating type informations using custom/strange calling conventions is writing an IDA plugin. It should not be anything really big nor difficult...