I've started to write a DLL in ASM and it work probably well (I've tested it with asm, vb and delphi yet), but my question now ist: if my functions needs some parameters is a simple RET enough to return to the calling program or do I have to 'clean up' the stack?
I've read about ENTER and LEAVE but never use it yet. About RET I also read, to take some bytes from the stack.
Is this necessary? Or is a simple RET okay?

Thanks in advance,
Posted on 2002-03-02 14:54:41 by Marilyn

The common assemblers, MASM & TASM can both do the stack cleanup for you as long as you use the PROC capacity and specify your calling convention.

Returning one value is simple enough, the EAX register is used for the return value but if you need a number of results, you can do it by passing the address of a structure or an array to the DLL, place your return values into it in the DLL code and they are already available to your calling program when the DLL returns.

With most dialects of basic, if you pass BY REFERENCE the DLL will get thew address sent to it.


Posted on 2002-03-02 16:37:25 by hutch--
Just be aware of the traps of passing strings ByRef, especially in VB. For most of the other variable types, you can pass them by reference, and change them in the dll, and your HLL app will see the changed variable when you return to it. Also be aware that VB has a couple of other variables that or not as straight-forward as they seem, and require special handling for the dll to access them (or just avoid using them).
Posted on 2002-03-02 18:28:11 by sluggy
I personally use NASM since I'm not comfortable with PROC style directives, so returning values and restoring the stack frame is
a common task for me.
Randel Hyde's Art of Assembly (http://webster.com) provides a great description of dealing with stack frames, so if my explaination doesn't help, check him out.

Okay, first. RET returns to the address following the calling instruction by popping an address off the stack. If the current stack content isn't a valid address, you program will generally crash. So simply make sure that the stack points to the same place it did when your procedure was called.

The RET immediate form of the instruction (which in NASM is retn)
also increments the stack pointer by a specific number of bytes. This is useful for cleaning up after parameters passed on the stack.

For example, lets say you want to pass a 32 bit value to a function and return a 32 value.

;calling coding (in NASM syntax..you should be able to translate)
push eax ; or any other 32 bit value such as DWORD
call functionname

;fuction code
%define PARAMETERNAME EBP + 8 ; will be used to reference the
;setup the stack frame
;our parameter will be offset from the current stack index
;by 8 byte, because the return address is already on the stack
;and we're going to save ebp on the stack
push ebp ;save ebp
mov ebp, esp ;set ebp to the stack pointer
sub esp, 16 ;give myself 16 bytes of local memory
; when we retore the stack frame these 16 bytes
; will be lost.
; if I wanted to use this memory i could reference it
; . Ofcourse, you may not need local mem
;function code

;restore stack frame
mov esp, ebp ;set stack pointer back to were it was
pop ebp ; restore ebp. This serves two purposes
;first, it set the stack pointer to were the return address
;is stored. secondly, the code your returning to probably
; uses ebp to reference it's parameters and local variabl
mov eax, (some return value)
retn 4 ;this returns to the calling code and removes
;parameter from the stack

Why did I bother with esp? Because if I had wanted to allocate space for local variables, I could have lowered the value of esp
and referenced them from ebp.

the enter and leave instructions accomplish the stack frame manipulation for you, but I find that doing it manually forces you to think about what you're actually doing.

Now, different HLLs require specific parameter passing and stack clean up methods. I've merely demonstrated a basic approach.
I hope it helped some.
Posted on 2002-03-02 23:28:37 by Canite
Thanks a lot for the informations (quite interesting).

But now I've got a new problem. As I said, I tested my dll with other languages sucessfully, but I cannot get my dll-function work with Visual C++ 6.
I created a header-file:
__declspec (dllexport) CheckOnline();
included this header-file in my source and also included the lib-file for the linker.
It seems that my function is not called probably and I have no clue at all, why not.

#include <Windows.h>
#include "myHeaderFile"
int APIENTRY WinMain(HINSTANCE hInstance,
HINSTANCE hPrevInstance,
LPSTR lpCmdLine,
int nShowCmd)

return 0;

I've try to get this run for almost 5 hours now, searched this board and also google but I cannot find anything useful (or at least something that work :( )


PS: my function creates a thread which checks the online connection every few seconds and display the status in a messagebox

PPS: I've also tried to use __stdcall but that also seems not to work
Posted on 2002-03-03 11:41:01 by Marilyn
I VC++ you need to go into your Project->Settings Then Click on the C/C++ Tab. Then pull down the dropbox to Code Generation. In the Calling Convention dropbox select __cdecl* from the options. You also need to do this if you want to use a C++ Dll with vb. Also doing this means you need a def file. Hope this helps. Jag
Posted on 2002-03-03 11:58:03 by Jag
I've looked up the calling conventions and it was already set to __cdecl* :(
But what do you mean about the def-file? I don't have to include a def-file to the C-program, or have I??? I just use the def-file only to build my dll.
Posted on 2002-03-03 12:21:39 by Marilyn


that's a do-nothing statement. It evaluates to the address of the
function, which is a constant... and amazingly enough, a legal
expression in C :).

You'll want to change the call to


btw, do you have a pascal background? =)
Posted on 2002-03-03 12:53:27 by f0dder
as long as you have a .def file defined and you have it in your options you should be ok, jag
Posted on 2002-03-03 12:56:37 by Jag
.....select __cdecl* from the options. You also need to do this if you want to use a C++ Dll with vb
That is incorrect. VB can only call stdcall C functions, because it cannot clean up the stack when it returns from a cdecl call.
Posted on 2002-03-03 16:57:34 by sluggy
I could emberasse the whole world! Thanks a lot guys. f0dder pointed me in the right direction. (yes I have a pascal background ;-) )
After I included the brackets for the function I've got an unresolved external, but now it works!
Sometimes its a little bit difficult to write a program if you don't know the language well ;-)

(for those who are interested in it: I need this for a kind of essay and I'm running out of time. You know how this things work. An inner voice say: "You've got enough time". Until you get a "little" bit under pressure)

Again, thanks a lot guys! I'll go to bed now, I'm a little bit tired now.

Posted on 2002-03-03 17:34:48 by Marilyn