I have got this working to an extent. It finds the GCD for 5 and 20 fine and goes along its merry way  :D. But when I run the second batch of number it kills the program  :cry:. I have been over this and I just cant figure out what I am doing wrong here. It might be a registry or I am not pushing or poping a reg right and that is messing up the code or the pointer is not pointing in the right variable or I am not cleaning up the stack right I really dont know.
Thanks for any help or advice you can give.
TITLE MASM 	GCD					(GCD.asm)


INCLUDE Irvine32.inc
.data
myMessage BYTE "GCD ",0dh,0ah,0
myMess2  BYTE "The GCD is = " ,0dh,0ah,0

;first set of nums
val1 DWORD  5
val2 DWORD  20

;second set of nums
val3 DWORD  24
val4 DWORD  18

.code
main PROC
call Clrscr 

mov edx,offset myMessage
call WriteString ;write message
call Crlf ;new line
push val1 ;push 1st set on the stack
push val2
call GCD
pop val1
pop val2
mov edx,offset myMess2
call WriteString
call WriteInt          ;Display GCD WriteInt uses EAX = qutent
call Crlf ;new line

;next set of values
push val4 ;push 1st set on the stack
push val3
call GCD
pop val4
pop val3
mov edx,offset myMess2
call WriteString
call WriteInt          ;Display GCD WriteInt uses EAX = qutent

exit
main ENDP

;------------------------------------------------
GCD PROC
; This finds GCD
; Gets values from stored values
;returns NA

;------------------------------------------------



mov  edx,0 ;zero out edx for remainder
mov  eax,dword ptr  ;dividend
        mov  ebx,dword ptr  ;divisor
div  ebx                    ;eax/ebx
cmp  edx,0                  ;remainder in edx

je  L1                 ;yes: quit
call GCD             ;no: call GCD agian
L1:
mov eax,ebx         ;move the divisor into eax for printing i.e GCD
    call DumpRegs ;show what is in the registry
 
    ret 8                ;clean up the stack
GCD ENDP

END main


now if I return 12 from the GCD proc it does not kill the program, but I does not return the right answer either so I am missing something here. I think its with the stack but not quit sure on what to do.
Posted on 2011-10-26 09:37:54 by lordgus
You seem to be inventing your own calling convention with the first call you push arguments and then inside the proc you don't. I can't blame you though because Kip invented his own calling convention too.

GCD_IR proc
mov ecx,edx
xor edx,edx
div ecx
or edx,edx
jz L1
mov eax,ecx
call GCD_IR
ret
L1:
mov eax,ecx
ret
GCD_IR endp


GCD_STD proc STDCALL A,B
mov eax,A
xor edx,edx
div B
or edx,edx
jz L1
invoke GCD_STD,B,edx
ret
L1:
mov eax,B
ret
GCD_STD endp


GCD_C proc C A,B
mov eax,A
xor edx,edx
div B
or edx,edx
jz L1
invoke GCD_C,B,edx
ret
L1:
mov eax,B
ret
GCD_C endp


;IRVINE-CALLING-CONVENTION
mov eax,24
mov edx,18
call GCD_IR

;STDCALL-CALLING-CONVENTION
invoke GCD_STD,24,18

;C-CALLING-CONVENTION
invoke GCD_C,24,18
 


As you can see GCD_STD and GCD_C are almost indentical but not if you look at the generated instructions either by disasm. debug or generating a listing.

Note, GCD_IR is not a real Irvine-Calling-Convention  proc as it doesn't preserve edx and ecx.

Please read about calling conventions and decide what you want to use!

Your errors are:
1)
ret 8                 ;clean up the stack

and
	pop val1
pop val2

are mutually exclusive, either callee frees the stack or caller frees the stack, not both.

2)
       mov  eax,dword ptr  ;dividend
        mov  ebx,dword ptr  ;divisor

if stack pointer is not modified in the proc prologue then
is the return address
is the FIRST argument
is the SECOND argument

3)
je   L1		                ;yes: quit
call GCD             ;no: call GCD agian
L1:
No RET after call means this falls through, I guess this is ok when debugging.
Posted on 2011-10-26 12:37:39 by drizz
As drizz points out, first you've got to figure out your calling convention. That's why it's crashing the second time. The reason it works the first time is that you picked some "lucky" values. There's been a lot written about GCD algorithms (some of which is more confusing than not reading it!). I recalled reading a very clever one someplace... ah, here it is!

http://www.azillionmonkeys.com/qed/asmexample.html

That might be too much of a clue. You're probably "supposed" to do it with a recursive algorithm. But you can take a look anyway...

Best,
Frank

Posted on 2011-10-26 13:35:28 by fbkotler
Thank you both very much that set me straight on a few things of why this program was not working and what I was doing wrong. I still just cant get the hang of assembly over java. But with practice come more practice.
Posted on 2011-10-26 14:48:42 by lordgus