OpenRCE_EliCZ
April 25th, 2008, 12:41
How to write integer(32) division in i386 assembly language using idiv instruction so that it doesn't raise an exception?
mov eax, Dividend
mov ecx, Divisor
idiv ecx
It can throw #DE, add Divisor == 0 check:
mov eax, Dividend
mov ecx, Divisor
jecxz DontIdiv
idiv ecx
It can still throw #DE, edx must be initialized for (i)div.
AMD64 manual : "To avoid overflow problems, precede this instruction (idiv) with a CBW, CWD, CDQ, or CQO instruction to sign-extend the dividend."
mov eax, Dividend
mov ecx, Divisor
jecxz DontIdiv
cdq
idiv ecx
It looks fixed (C compilers think so), can it throw #DE now?
Yes, it can - I didn't avoid overflow problems.
There is still one combination that will raise #DE:
Dividend = INT_MIN and Divisor = -1.
mov eax, Dividend
mov ecx, Divisor
jecxz DontIdiv
cmp eax, INT_MIN
sete dl
cmp ecx, -1
sete dh
test dl, dh
jne DontIdiv
cdq
idiv ecx
Other integer arithmetic instructions set OF only and the result for 0-INT_MIN is INT_MIN.
How do C compilers (CPUs, OSes) handle the integer overflow?
neg32.c:
int neg32(int x) {
return(-x);
}
neg64.c:
long long neg64(long long x) {
return(-x);
}
idiv32.c:
int idiv32(int x, int y) {
return(x/y);
}
idiv64.c:
#define __USE_ISOC99
#include <limits.h>
#ifndef LLONG_MIN
#ifdef LONG_LONG_MIN
#define LLONG_MIN LONG_LONG_MIN
#else
#define LLONG_MIN -9223372036854775808LL
#endif
#endif
#include <stdio.h>
long long idiv64(long long x, long long y) {
return(x/y);
}
int main(void) {
long long x = LLONG_MIN;
long long y = -1;
long long r = 1234;
r = idiv64(x, y);
return(printf("%lli\n", r));
}
Sometimes a (CRT) function is used instead of division performing instruction,
especially for integer64 division in 32bit builds.
(int)-INT_MIN == INT_MIN always.
architecture/os/compilers/builds neg32 | neg64 | idiv32 | idiv64
i386/winnt/msc|dmc/32 INT_MIN | LLONG_MIN | crash | LLONG_MIN
i386/winnt/wc/32 INT_MIN | LLONG_MIN | print exception info and exit | LLONG_MIN
i386/linux/gcc/32 INT_MIN | LLONG_MIN | "Floating point exception" | LLONG_MIN
amd64/winnt/msc/64 INT_MIN | LLONG_MIN | crash | crash
amd64/linux/gcc/64 INT_MIN | LLONG_MIN | "Floating point exception" | "Floating point exception"
ia64/winnt/msc/32 INT_MIN | LLONG_MIN | long pause and exit | LLONG_MIN
ia64/winnt/msc/64 INT_MIN | LLONG_MIN | INT_MIN | LLONG_MIN
power/aix/gcc|xlc/32 INT_MIN | LLONG_MIN | 0 | LLONG_MIN
power/aix/gcc|xlc/64 INT_MIN | LLONG_MIN | 0 | 0
sparc/solaris/gcc|suncc/32|64 INT_MIN | LLONG_MIN | INT_MIN | LLONG_MIN
See you at caro workshop.
https://www.openrce.org/blog/view/1129/Integer_overflow
mov eax, Dividend
mov ecx, Divisor
idiv ecx
It can throw #DE, add Divisor == 0 check:
mov eax, Dividend
mov ecx, Divisor
jecxz DontIdiv
idiv ecx
It can still throw #DE, edx must be initialized for (i)div.
AMD64 manual : "To avoid overflow problems, precede this instruction (idiv) with a CBW, CWD, CDQ, or CQO instruction to sign-extend the dividend."
mov eax, Dividend
mov ecx, Divisor
jecxz DontIdiv
cdq
idiv ecx
It looks fixed (C compilers think so), can it throw #DE now?
Yes, it can - I didn't avoid overflow problems.
There is still one combination that will raise #DE:
Dividend = INT_MIN and Divisor = -1.
mov eax, Dividend
mov ecx, Divisor
jecxz DontIdiv
cmp eax, INT_MIN
sete dl
cmp ecx, -1
sete dh
test dl, dh
jne DontIdiv
cdq
idiv ecx
Other integer arithmetic instructions set OF only and the result for 0-INT_MIN is INT_MIN.
How do C compilers (CPUs, OSes) handle the integer overflow?
neg32.c:
int neg32(int x) {
return(-x);
}
neg64.c:
long long neg64(long long x) {
return(-x);
}
idiv32.c:
int idiv32(int x, int y) {
return(x/y);
}
idiv64.c:
#define __USE_ISOC99
#include <limits.h>
#ifndef LLONG_MIN
#ifdef LONG_LONG_MIN
#define LLONG_MIN LONG_LONG_MIN
#else
#define LLONG_MIN -9223372036854775808LL
#endif
#endif
#include <stdio.h>
long long idiv64(long long x, long long y) {
return(x/y);
}
int main(void) {
long long x = LLONG_MIN;
long long y = -1;
long long r = 1234;
r = idiv64(x, y);
return(printf("%lli\n", r));
}
Sometimes a (CRT) function is used instead of division performing instruction,
especially for integer64 division in 32bit builds.
(int)-INT_MIN == INT_MIN always.
architecture/os/compilers/builds neg32 | neg64 | idiv32 | idiv64
i386/winnt/msc|dmc/32 INT_MIN | LLONG_MIN | crash | LLONG_MIN
i386/winnt/wc/32 INT_MIN | LLONG_MIN | print exception info and exit | LLONG_MIN
i386/linux/gcc/32 INT_MIN | LLONG_MIN | "Floating point exception" | LLONG_MIN
amd64/winnt/msc/64 INT_MIN | LLONG_MIN | crash | crash
amd64/linux/gcc/64 INT_MIN | LLONG_MIN | "Floating point exception" | "Floating point exception"
ia64/winnt/msc/32 INT_MIN | LLONG_MIN | long pause and exit | LLONG_MIN
ia64/winnt/msc/64 INT_MIN | LLONG_MIN | INT_MIN | LLONG_MIN
power/aix/gcc|xlc/32 INT_MIN | LLONG_MIN | 0 | LLONG_MIN
power/aix/gcc|xlc/64 INT_MIN | LLONG_MIN | 0 | 0
sparc/solaris/gcc|suncc/32|64 INT_MIN | LLONG_MIN | INT_MIN | LLONG_MIN
See you at caro workshop.
https://www.openrce.org/blog/view/1129/Integer_overflow