http://wolfshade.home.ro/int_overflow.asm

at line comented "; modulo" (pointed by debugger)

i can't solve it, if needed i can include the rest of the code.
Posted on 2006-09-02 23:25:26 by w0lfshad3
You mean the line saying : div temp ?

Div tries to divide eax:edx. Do you know what values are in both of those registers?
Posted on 2006-09-02 23:30:34 by JimmyClif
hmmm true edx has garbage, alltough it should have been reset by the rand function?

How do i call rand from msvcrt.lib in VC environment? PROTO it then simply invoke it?
Posted on 2006-09-02 23:40:32 by w0lfshad3
"proto" it, "invoke it" and "includelib" it. or "loadlibrary-getprocaddress" it. don't forget that crt functions are 'cdecl' - you're the one to balance the stack after the function returns.
Posted on 2006-09-03 01:13:39 by ti_mo_n
No, I don't believe it's supposed to preserve edx. Eax, edx and ecx are trash registers which are commonly assumed to be overwritten by procedures.

Just clear edx before you divide, that's all ;)
Posted on 2006-09-03 10:06:21 by JimmyClif
thank you i keep forgetting about trash registers

"proto" it, "invoke it" and "includelib" it. or "loadlibrary-getprocaddress" it. don't forget that crt functions are 'cdecl' - you're the one to balance the stack after the function returns.


thanks about saving me another thread  :lol:
Posted on 2006-09-03 11:43:45 by w0lfshad3
bump, by balancing the stack you mean if a function pushes n parameters then i would have to go:

invoke function, param1, param2, ..., paramN
add esp, 4*N

for cdecl ones?
Posted on 2006-09-04 19:31:54 by w0lfshad3
yes :)

and know that some parameters (64-bit ones, like "double", etc.) get split into two 32-bit parameters, so they count as 2 parameters, not 1.
Posted on 2006-09-04 21:10:39 by ti_mo_n