PDA

View Full Version : weird msvc++ compiler behavior

roxaz
July 29th, 2008, 14:41
i noticed something weird while reversing one x64 app, look:
Code:
mov     [rsp+8], rcx

mov     rax, rsp

sub     rsp, 98h

mov     qword ptr [rsp+50h], 0FFFFFFFFFFFFFFFEh

mov     [rax+18h], rbx

mov     [rax+20h], rbp


why the hell compiler stores registers in stack space that does not belong to the function? now i cant push anything on to the stack before calling this function cause my data is overwritten. whats worse is that some of my own compiled functions act like that, and this is not good because of reason i mentioned before. what makes compiler to act so weirdly? and is it possible to disable this? btw this code does not write data to arguments that are passed to the function, cause only one argument passed in rcx.
Camus SoNiCo
July 29th, 2008, 16:29
Because it's part of the calling convention:

http://en.wikipedia.org/wiki/X86_calling_conventions#Microsoft_x64_calling_convention
TiGa
July 29th, 2008, 17:55
In my video #5, I show exactly that.
In it, I compare a same program compiled in x86 and x64.

TiGa
roxaz
July 30th, 2008, 01:47
Oo, MS never stops amazing me, ill check video and wiki out, thanks.
Arcane
July 30th, 2008, 02:41
it makes perfect sense..why use slow stack when you can use fast registers ..but makes it hellish to read
roxaz
July 30th, 2008, 02:48
it makes no sense that caller must allocate 32 bytes of the stack and then deallocate it. well, its no big deal as stack is allocated in beginning of the function and callees use that, but why the hell they couldnt allocate 32 more bytes in callee itself? this somehow doesnt make sense to me
naides
July 30th, 2008, 06:20
Hey, roxaz. . .
I also have a problem with nature, because she made biology too complex and redundant, and gave us an appendix, just for the surgeons to earn more money

Without going into technical details, this convention is a compromise between universality (most functions can accommodate their arguments in that 32 byte buffer), and efficiency in terms of cpu cycles; believe me: The software architects know this code style is not easily readable or elegant at the ASM level, but was not MEANT to be read or understood at the ASM level.
FrankRizzo
August 18th, 2008, 22:29
Just wait until you start to see the weird shortcuts that the compiler writers use sometimes. Like "Doing Multiplication by dividing by a fixed constant", or using the LEA instruction to do math, all those good optimizations.
Maximus
August 21st, 2008, 16:57
http://board.flatassembler.net/topic.php?t=4155

Old points, still good points I think.