omega_red
June 22nd, 2005, 07:14
I was playing with my ever-unfinished crackme and found that it crashes for no apparent reason in about 10% of runs. After some investigations I found that rep movsb can be dangerous 
(FASM format)
This innocent piece of code can crash randomly. Why? It's easy to see if we have JIT debugger assigned and analyze program's state upon crash:
As can be seen, EIP "slipped" by 1 byte, hence the error.
Interestings things pop up while we are debugging this with Olly. Tracing by F7 and F8 gives different results. F8 on REP = program exits, code is incorrect:
And with F7 all is OK.
I observed those effects only with movsb - ...w and ...d seem to not behave like this (f8 on movsw causes program halt too, but code is copied OK).
As a bonus, Olly seems to have a bug in handling such SMC. You can see it by F7-ing, after MOVSB selecting lines looks weird
And for last, something that made me really confused - EXE in CONSOLE format crashes (EIP "slip"
much more often than the GUI one.
My CPU: Intel P4 3.2 HT (enabled). OS: Win XP SP2.
[edit] Link for compiled exe: http://212.33.90.248/~omega/p.exe

Code:
format PE CONSOLE
entry start
include '%fasminc%\win32a.inc'
;------------------------------------------------
use = ExitProcess
section '' code data readable writeable executable
;------------------------------------------------
start:
mov ecx, 0x90909090 ; nops
mov [a_1], ecx
mov [a_1+4], ecx
mov [a_1+8], ecx
mov [a_1+12], ecx
mov ecx, (data_end-data_start)
mov esi, data_start
mov edi, a_1
rep movsb
; nop ; uncomment this to prevent crashes
a_1:
times 0x10 db 0xcc ; fake int3, will be replaced by NOPs
nop
ret
;------------------------------------------------
data_start:
; nop
jmp $+2
; nop
jmp $+2
data_end:
;------------------------------------------------
align 4
data import
library kernel,'kernel32.dll'
import kernel,\
ExitProcess,'ExitProcess'
end data
(FASM format)
This innocent piece of code can crash randomly. Why? It's easy to see if we have JIT debugger assigned and analyze program's state upon crash:
Code:
00401031 0090 90909090 ADD BYTE PTR DS:[EAX+90909090],DL
00401037 90 NOP
00401038 90 NOP
00401039 90 NOP
As can be seen, EIP "slipped" by 1 byte, hence the error.
Interestings things pop up while we are debugging this with Olly. Tracing by F7 and F8 gives different results. F8 on REP = program exits, code is incorrect:
Code:
0040102C |. F3:A4 REP MOVS BYTE PTR ES:[EDI],BYTE PTR DS:[>
0040102E |. 90 NOP
0040102F |. 00EB ADD BL,CH
00401031 |. 0090 90909090 ADD BYTE PTR DS:[EAX+90909090],DL
00401037 |. 90 NOP
00401038 |. 90 NOP
And with F7 all is OK.
I observed those effects only with movsb - ...w and ...d seem to not behave like this (f8 on movsw causes program halt too, but code is copied OK).
As a bonus, Olly seems to have a bug in handling such SMC. You can see it by F7-ing, after MOVSB selecting lines looks weird

And for last, something that made me really confused - EXE in CONSOLE format crashes (EIP "slip"

My CPU: Intel P4 3.2 HT (enabled). OS: Win XP SP2.
[edit] Link for compiled exe: http://212.33.90.248/~omega/p.exe