Ok, I have a question for all of you com guys. You seem to get it, and I don't :)

So anyways, here's my problem:



; To create a Direct3D8 device, we must first create a Direct3D8 object.
invoke Direct3DCreate8, D3D_SDK_VERSION
mov g_pD3D, eax
.if eax == NULL
return 0

; With this Direct3D8 object, we can obtain the current desktop display mode
mcall [g_pD3D],IDirect3D8_GetAdapterDisplayMode,D3DADAPTER_DEFAULT,ADDR d3ddm
.if eax != D3D_OK
return 0

; Next, we initialize the parameters for our Direct3D8 device so that the backbuffer
; has the same format as the current desktop display.
ZeroMemory &d3dpp,sizeof(D3DPRESENT_PARAMETERS)
mov d3dpp.Windowed, TRUE
mov d3dpp.SwapEffect, D3DSWAPEFFECT_DISCARD
mov eax, d3ddm.Format
mov d3dpp.BackBufferFormat, eax

; Finally, we create the Direct3D8 device.
mcall [g_pD3D],IDirect3D8_CreateDevice, D3DADAPTER_DEFAULT, D3DDEVTYPE_HAL, hWnd,\
ADDR d3dpp, ADDR g_pd3dDevice
.if eax != D3D_OK
return 0

; Device states would normally be set here

return S_OK

adds "leave | ret 4" after the proc... why?

Render PROC

; Make sure that the Direct3D8 device exists.
mov eax, g_pd3dDevice
.if eax == NULL

; Clear the backbuffer to a blue color (the 4th parameter).
mcall [g_pd3dDevice],IDirect3DDevice8_Clear,0, NULL, D3DCLEAR_TARGET, 000000ffh, tmpfloat, 0

; Begin the scene.
mcall [g_pd3dDevice],IDirect3DDevice8_BeginScene

; Rendering of scene objects can happen here

; End the scene.
mcall [g_pd3dDevice],IDirect3DDevice8_EndScene

; Present the backbuffer contents to the display.
mcall [g_pd3dDevice],IDirect3DDevice8_Present, NULL, NULL, NULL, NULL

Render ENDP

this adds "leave | ret 16" why do I need to kill 16 bytes here, and only 4 above??? Seems as if there is some sort of link somewhere because I simply don't get it. How does masm know how many bytes to kill??? I see NOTHING that will hint it, but maybe in the mcall. However, I looked over the source, and I see nothing...

PLEASE SOMEONE EXPLAIN!!!!!!!!!!!!!!!!! lol this is driving me nuts.
Posted on 2001-09-27 15:25:52 by Kenny
Ok, I discovered something:

For every D3D function with CONST something* I have to kill 4 bytes of the stack. Why???
Posted on 2001-09-27 15:47:26 by Kenny
Hello Kenny !

I wonder about the second question, I'll look at it later ...

The answere for the first question is as follows:

A "MyProc: proc ParaA, ParaB, ParaC : dword .............. ret" - sequence produce the following code:

MyProc: enter xxx
ret 12

By calling MyProc, you push three dwords (ParaA, ParaB, ParaC = 12 Bytes onto the stack !) which have to be freed again after having proceeded MyProc. This is done by ret 12 !

The enter xxx / leave instructions are for handling the local-memory onto the stack ... (Well I've never liked them ... ;) )

I do not believe that there is a counter for how much mcalls are done (the macro itself doesn't have a machanism like that !) and then do a ret x = 4 * number of calls. So the 16 Bytes are suspicious ! (therefore the first proc will have to release 12 Bytes instead of 4 !)

Well are you able to link the disassembled file here ?

Greetings, CALEB !
Posted on 2001-09-27 17:49:13 by Caleb
Ok, this is the best I can come up with right now:

In the D3D declarations there are these strange things like this:

HRESULT GetAdapterDisplayMode(
UINT Adapter,

Now, what happens is this: Every time there is a pointer to something, that pointer is returned after the call is done, and the .lib files know that, so MASM says: OK! Let's kill the stack at the end. (4 bytes for every pointer) So, That's why I have plain ol' code and masm adds the leave and the ret xx at the end of it.

I just did a test and after every com call, I would count the pointers and add ebp 4*number and it seemed that I generated no exceptions, but I had to set ret to normal or else I would generate an exception.

The other thing I learned is this: When a proc is established, it would be the equivalant of enter, but slightly different: "push ebp | mov ebp esp" and leave is just the same exact thing as saying: "mov ebp esp | pop ebp" So, I can understand the leave at the end because it's just returning the stack to the origional place because before it had reserved stack for that proc.

The disassembled file is attached
Posted on 2001-09-27 18:28:15 by Kenny

your problem has nothing to do with COM.

For calling convention "stdcall" (which is "standard" in WIN32 ) the called proc is responsible to clean up the stack. So, if your proc has 4 dword parameters when leaving the proc has to pop 4*sizeof dword = 16 bytes from the stack. Masm does this calculation for you at every "ret" it founds inside the proc.

In your code the "Render" proc has no parameters, so Masm shouldn't translate "ret" to "ret 16" but to "ret 0".

Posted on 2001-09-28 05:02:35 by japheth
Yeah, that's what got me... The thing is when I have ret 0, the program generates an exception. When I have ret 16 or add esp, 12 after present and add esp, 4 after clear, it works fine...

Explain that one :alright: hehe
Posted on 2001-09-28 11:48:37 by Kenny
First a quick disassebly of a very simply program:

00000000 .code
00000000 start:

invoke ExitProcess, eax
00000000 2 50 * push eax
00000001 7m E8 00000000 E * call ExitProcess

00000006 Func PROC blah:DWORD
00000006 2 55 * push ebp
00000007 2 8B EC * mov ebp, esp
00000009 4 8B 45 08 mov eax, blah
0000000C 4 C9 * leave
0000000D 10m C2 0004 * ret 00004h
00000010 Func endp
end start

I will refer to the Func procedure.

If we look at the first two instructions here

push ebp
mov ebp, esp

These are the prologue of the function, and are placed automatically by MASM. It is these functions that allow us to reference local variables (if there were any the code would look slightly different), as ebp is now a static reference point.
The leave instruction undoes this.

Second part of your question, the ret command is effectivly:

pop eip ;restore the instruction pointer
add esp, (no_of_function_args)*SIZEOF(args)

This is because a call is a push and a jump in one (it pushes the return address onto the stack, then jumps to the point specified in the call instruction).

By not specifying the correct 'xx' in your ret xx you will leave things on the stack, and this will eventually cause a stack overflow. By not specifying leave you don't clean up the stack frame you created at the start of the function, and hence leave ebp on the stack, which will then be popped off when you hit ret so your return address will be the stack rather than your code (not what most people intend!).

Posted on 2001-10-09 08:37:42 by Mirno
Helps a little :)

I just need to find out WHERE to put it now... Like this:

I have a PLAIN ol' function with no saving of ebp and no enter, and then when I use DX in it, DX makes an exception. However if I save ebp and then call DX, I don't get the exception until I try and ret. This is where I have to clear the stack using ret xx. So, I guess MASM knows because of the .lib files, and it knows the function will be returning a bunch of stuff on the stack, and since I can't read lib files, I'm stuck to figuring it out on my own huh :) OR I can use MASM for my coding :)
Posted on 2001-10-09 09:15:54 by Kenny
The pascal calling convention - used by most of windows - has the called function correcting the stack - you don't need to do anything, no matter what assembler you use - just push the args and call the function. The C calling convention has the calling code correct the stack - this is usually used when the number of passed params are variable: wsprintf.
    push pARM1

push pARM2
call pPascal
push pARM1
push pARM2
call pC
add esp,8
ret 8

Usually, nothing is left on the stack - COM or no COM, windows wasn't designed that way. Parameters are passed for temporary use, and then they are trashed. All returned information will be in a passed structure or eax - that is by design.
Posted on 2001-10-09 17:34:22 by bitRAKE
Yeah, but when I miss guess how much to trash, I end up with an exception :)
Posted on 2001-10-10 00:31:02 by Kenny
OF COURSE you end up with an exception.

What's on the stack just after the passed parameters?

That would be the return address, and that's very critical. Change the ret addr and you essentially jump to some random address and start executing code there.

No wonder it promptly GPFs.

MASM has some very concise built in macros to handle this, those being the PROC and ENDP block. Any 'ret' inside such a block will also automatically generate the correct stack clean up code.

It's a wheel, it is round, and you cannot generate code any more efficient, so why not use the existing tool?
Posted on 2001-10-10 01:54:02 by Ernie
You're not getting what I'm saying...

I don't want to code in MASM :) I hate the compile times! If I had it my way, I'd program in SpAsm or FASM, but the problem is when I try and use DX8 code that has the stack killing thing is I don't know when to kill it, and when not, so how would I go about finding out, or writing a macro that would do it for me? I don't understand how MASM knows, but I don't.
Posted on 2001-10-10 12:59:54 by Kenny
Maybe in order for MASM to know, it need a longer compile time :grin:

...hardcore MASM fan here!
Posted on 2001-10-10 14:07:28 by NaN
Ernie, I have a better wheel, but who cares. We are not going to rewrite windows for me. :)

Kenny, I didn't know DX8 had C-calling type functions?
(i.e. You shouldn't need to fix the stack?)
The return is a macro that puts a value into the EAX register, and that is what is confusing you? Or maybe I am wrong? :)

NaN, a level of understanding is lost in just using the tools without the understanding, and we see that time and again here on the board. Using a bare bones assembler is one way of knowing what's really going on. FASM is a great tool for such a small piece of code. :)
Posted on 2001-10-10 17:37:33 by bitRAKE
OK, when to use:
1) When to use "add esp, blah"
Use this after a call to a C calling convention function.

Because the C calling convention doesn't clean up after itself, so you must do it instead!

2) When to use "leave"
Use leave when you wish to get rid of the stack frame created by "enter", or "push ebp"; "mov ebp, esp".

Because if you don't all the locals will be left on the stack, so when you come to "ret" it will pop one of these local vars instead of the return address (and a whole lot of badness ensues)!

3) When to use "ret xx"
Use "ret xx" when you wish to leave a function. The value of xx is decided as follows:
- Is the function C style? If yes xx = 0
- Else xx = (no of arguments passed to the function) * SIZEOF(arguments).

Because the ret function will add xx to esp for you, hence removing the need for stack clean-up (except in C calling convention functions which don't want the stack cleaned up in order to fit in with the convention).


P.S. I agree with bitRAKE, we come to depend on the tools to such a degree that we don't understand them anymore.
The whole reason I like assembly is because I get away from that, and learn a bit more about why things work the way they do.
Posted on 2001-10-11 08:58:33 by Mirno
Point taken... :)

.. but from an engineering standpoint... "if it aint broke, dont fix it", this is why i like MASM so much.

Posted on 2001-10-11 13:27:47 by NaN

The PROC and ENDP-macros makes it very easy to handle all the local stuff you probably need in a subprocedure ... but it cost you one of 7 useable register ... so I will never like this hll-mechanism for assembly invoked by mnemonics like enter/leave (I don't have liked it on MC-68xxx already ...)

What I have did is to define some little macros which handles my locals with direct esp-addressing. The only difficult is that if you use invokes in your procedures you have to recalculate the offsets if locals are used as parameters ... some exercising and you will do this without thinking about it !

Greetings, CALEB
Posted on 2001-10-11 18:20:57 by Caleb
BitRAKE: There are MULTIPLE returns, and they are returned in the stack. I have no clue why, but I would like to know.

NaN: I don't care so much as the compile time is longer, I'm more just curious now. I want to know why :)

I'll show you guys the code, but I DON'T put anythign on the stack accept to call the function, but when the function is done, I need to add esp,xx or to ret xx. I'll post SpAsm code, but I can whip up some quick FASM code for ya if you want me too.

Lemme explain what I mean by "return"

HRESULT GetIndices(
IDirect3DIndexBuffer8** ppIndexData,
UINT* pBaseVertexIndex

Address of a pointer to an IDirect3DIndexBuffer8 interface, representing the returned index data.
Pointer to a UINT value, holding the returned base value for vertex indices. This value is added to all indices prior to referencing vertex data, defining a starting position in the vertex streams.

So what happens is I call GetIndices, and then it goes like this:

eax = returnfail/succede
esp = ppIndexData
esp+4 = pBaseVertexIndex

See now kinda?
Posted on 2001-10-11 19:00:04 by Kenny
NaN: I don't care so much as the compile time is longer, I'm more just curious now. I want to know why

This i can truely respect. Sorry, if I was coming across as opposed to your inquest.

Best of Luck.. (cause i have no answers :tongue: )

Posted on 2001-10-11 21:48:20 by NaN
Let's starrt here:

HRESULT GetIndices( IDirect3DIndexBuffer8** ppIndexData,
UINT* pBaseVertexIndex );

That means when calling GetIndices you are pushing 3 dwords onto the stack, two pointer addresses, and the object reference. If you are pushing any more, well, that's why you have to adjust the stack yourself.

Also, no way will this function adjust register values for you. It wants memory pointers. And yeah, figuring out these pointers is a royal mind-boink. It works like this:

pSomeBuffer dword ?

invoke HeapAlloc hHeap, ByteCount ; get some memory
mov pSomeData, eax ; save pointer to memory
invoke SomeFunct ADDR pSomeData

(don't look too hard at the HeapAlloc line, I didn't check the funct proto)

Note we pass WHERE we store the pointer, not the pointer itself. This has some interesting advantages, mostly it allows the called function not only to access our entire structure, but it also allows the function to free the struct we passed in, and create an entirely new one, and by knowing where we store our pointer, change that pointer to this new struct.

Knowing WHY things are passed "** ppSomeData" may help you see how to accomplish it.
Posted on 2001-10-11 23:10:52 by Ernie