I've been watching this place on and off from a distance for quite a while now and I must say that I'm very impressed with the knowledge here. You guys are very good at what you do!

Now, my first question here. It's probably something simple, I just can't seem to figure it out. I have created a DLL for use in another language just to play around and see if I could get it to work. I got it to work, but the DLL seems too large for something so simple. I'll try to give as much info as I can here.

Here is the code in the DLL:
.586

.model flat, stdcall
option casemap:none

include \masm32\include\windows.inc
include \masm32\include\user32.inc
include \masm32\include\kernel32.inc
includelib \masm32\lib\user32.lib
includelib \masm32\lib\kernel32.lib

.data
DLLName db "ASM-DLL", 0
hInstance dd 0

.code
DllEntry proc hInst:HINSTANCE, dwReason:DWORD, reserved1:DWORD
push hInst
pop hInstance

mov eax, TRUE
ret
DllEntry Endp

TestProc proc
invoke MessageBox, NULL, addr DLLName, addr DLLName, MB_OK
ret
TestProc endp

End DllEntry

And here is the DEF file:
LIBRARY ASM

EXPORTS TestProc

This DLL will assemble fine with no errors and when called form the other language it works exactly as expected. The problem though is that the DLL is 20k when assembled.

I'm using:
Masm 6.15.8803
Link 6.00.8447

The DLL is being assembled with the following in a batch file:
@echo off

\masm32\bin\ml /c /coff /Cp ASM.asm
\masm32\bin\link /DLL /DEF:ASM.def /SUBSYSTEM:WINDOWS /LIBPATH:\masm32\lib ASM.obj
pause


Any ideas why this DLL is so large when it hardly does anything?
Posted on 2004-03-23 10:06:40 by mastermind
I don't know what's wrong with your system, but on my system I get a 3.072 byte DLL with copy&paste of your code. Different masm+link versions (7.10.3077), but it shouldn't really matter that much I guess.

Perhaps the older version of masm does something funky, perhaps you got a virus. Please zip up the generated .obj and .dll, and post here.

Tested with ml 6.15.8803 and link 5.12.8078 , still 3072 byte dll.
Posted on 2004-03-23 10:38:33 by f0dder
Ok, here they are.

Thanks!
Posted on 2004-03-23 10:46:24 by mastermind
Ah, simple and harmless. Your DLL is linked with 4096-byte file section alignment (which should make things a bit more efficient, and generally don't matter too much (cluster size etc), especially not when your DLL grows a bit and has some actual functionality).

If you insist on have a smaaaaaaaaaall DLL, add /filealign:512 to the linker commandline.
Posted on 2004-03-23 10:55:48 by f0dder
Down to 3k now. Thanks very much.

It isn't so much that I absolutely have to have the smallest possible dll, I just din't understand why it was so big when using ASM and very little actual code.

It's nice to know why this was happening. Thanks again! :alright:
Posted on 2004-03-23 10:59:26 by mastermind
A beer an hour huh ? :tongue:

f0dder is right here, if you are using one of the VC linkers, you will need to set the file alignment but if you use the linker from MASM32 that is from the WIN98DDK, it was set up for device drivers where size matters and it has the 512 byte alignment as default.
Posted on 2004-03-23 19:47:33 by hutch--

from the WIN98DDK, it was set up for device drivers where size matters and it has the 512 byte alignment as default.

This is somewhat wrong :). The linker settings /OPT:WIN98 and /OPT:NOWIN98 translates directly to /FILEALIGN:4096 and /FILEALIGN:512 - this in no way changes the memory requirements of the loaded PE file, FILEALIGN:4096 is just (allegedly) a bit faster wrt. mapping of the executable - haven't ever felt much of a difference, though.

/MEMALIGN, on the other hand, would change the runtime loaded size of the PE. I don't think it's a good idea to set it below 4096 though, as this would mean section flags can't be enforced directly by hardware (at least for adjacent sections with different protection), besides memalign<4096 causes a lot of trouble for relocation items (which drivers ought to have).

Besides, 4k of memory alignment isn't really a problem for NT Kernel-mode drivers. I don't think it was a problem even for the win9x VXD's in LE format, but it was probably a genuine problem with win3.x (what did 3.x use for drivers? LE? NE?)
Posted on 2004-03-24 13:19:47 by f0dder
Yes,

The WIN98DDK linker can handle other file alignments but its default is 512. The setting that does need to be turned off is the /OPT:REF set by default as it produces a warning if one of the syatem DLLs in not called. /OPT:NOREF solves this problem.

VXDs were generally written to keep the size down and this is why it uses the lookup table at the end rather than the direct calls usually done in VC. You can do the direct calls using a different prototype that uses TYPEDEF which allows you the alternate use.

If you write very large files, the 4k alignment is supposed to improve the loading speed of the program but if you are buildng files of a meg or less, its very hard to justify the alignment in load terms where 512 byte alignment certainly produces smaller files which load faster anyway.
Posted on 2004-03-24 23:45:46 by hutch--

The WIN98DDK linker can handle other file alignments but its default is 512. The setting that does need to be turned off is the /OPT:REF set by default as it produces a warning if one of the syatem DLLs in not called. /OPT:NOREF solves this problem.

I dunno which was the first linker to support the /FILEALIGN setting, but every MS linker I've tried (which means 98DDK and upwards) have supported it, just with different defaults.

/OPT:REF shouldn't turned off, it's not an error, but a "I suppose you know what you're doing, so I'm just letting you know I disabled references to this DLL because you didn't actually use it" kind of helpful message. If anything needs to be "fixed", it's not adding /OPT:NOREF, it's to fix up your includelib statements.

The whole deal with 4k section alignment, I guess, is that (FILEALIGN=MEMALIGN)>=4096 will allow the PE loader to use a single filemapping with a single section view mapped in, because there's a 1:1 correspondence between file and memory layout. With FILEALIGN<MEMALIGN, the PE loader will have a slightly more complicated job. Whether the speed difference is noticable I do not know, but I'd prefer any dealings done in ring0 code, and especially with something as crucial as drivers, to happen as fast as possible.

As for the file 'bloat', it will be next to unnoticable once you have real functionality in your driver, whether you write it in assembly or not. And with the filemapping approach that windows uses, only needed parts of the PE will be pulled into memory (no, the PE loader doesn't FileRead the entire file to memory, it uses file mapping which uses demand-loading).
Posted on 2004-03-25 00:47:10 by f0dder
Maybe I have missed something here.

it's to fix up your includelib statements

MASM only has one format to load libraries in code and that is the INCLUDELIB statement.

Obviously the linker was set up for the tasks in writing device drivers but unless you need particular libraries treated differently to others, the /OPT:REF delivers an error for no reason so using the /OPT:NOREF is the best way to deal with it so you don't get an error that has no content.

Regards,
http://www.asmcommunity.net/board/cryptmail.php?tauntspiders=in.your.face@nomail.for.you&id=2f46ed9f24413347f14439b64bdc03fd
Posted on 2004-03-25 07:40:03 by hutch--

MASM only has one format to load libraries in code and that is the INCLUDELIB statement.

Yes, using INCLUDELIB or passing library names on the linker commandline.


Obviously the linker was set up for the tasks in writing device drivers but unless you need particular libraries treated differently to others, the /OPT:REF delivers an error for no reason so using the /OPT:NOREF is the best way to deal with it so you don't get an error that has no content.

It's not an error, it's a warning.
The reason it pops up, is because you have used includelib for some DLL, but aren't referencing anything from the DLL. Thus, you are importing the DLL for no reason. /OPT:REF makes the linker skip importing from the unused DLLs, but (friendly as it is) will inform you that it's doing this.

/OPT:NOREF is not the solution to get rid of this warning message, the fix is to remove the includelib statement for the unused DLLs.
Posted on 2004-03-25 10:56:18 by f0dder
What you look for in programming is consistency and the use of /OPT:REF outside of device driver development is inconsistent in that it only delivers the warning for one or two system libraries, not all of them.

To use the linker for general purpose programming means using a whole range of different libraries so when you have warnings for one and not the other, the loss is consistency. /OPT:NOREF fixes that consistency problem in that it treats the system libraries in the same way as it treats the rest.

Yes, using INCLUDELIB or passing library names on the linker commandline.

You must not have used MASM for a while, the includelib statement is used in code and is processed by ML first. MASM normally build by using ML seperately from link with the /c option and then you link it seperately after the object module is created.

Now of course link reads the data from the object module but it has nothing to do with passed command lines.
Posted on 2004-03-25 17:33:05 by hutch--

What you look for in programming is consistency and the use of /OPT:REF outside of device driver development is inconsistent in that it only delivers the warning for one or two system libraries, not all of them.


There isn't any inconsistency on the linker side. When it sees "includelib" records (which get translated into the commandline-argument /DEFAULTLIB, have a look at the .drectve section in an assembled object file), all the linker does is to add this .lib file to it's "search these libraries to resolve extern references" list.

Being added to this list doesn't, by itself, cause a library to be included in the executable in any way - it only means the exports of the library will be scanned when resolving external references (if you have a really really slow machine, there might be some sense in not specifying too many libraries as this would slow things down. Then again, searches are done with hash tables, so it's rather fast).

Why do you get the unreferenced warnings with *some* libraries when doing masm projects, then? Because of generated EXTERN records. You have probably only seen it on user32.dll, because of _wsprintfA. However, have a look at the attached example asm file. It's a *very* constructed example, but it shows what is happening - EXTERN causes an EXTERN record to be generated even if the symbol isn't referenced.

With /OPT:REF, the linker will refrain from linking in the library, because it can see the EXTERN symbol isn't referenced at all. With /OPT:NOREF it assumes that just because it got a EXTERN record, it should include the symbol, whether used or not, and will pull in the library. There might be a few times when somebody wants this to happen, but I can't really think of any examples. Using EXTERNDEF instead of EXTERN also solves this - haven't looked into how EXTERNDEF works, but I assume it tells masm to only emit the EXTERN record if the symbol is referenced.

Now, why does this happen with wsprintfA, then? It's not defined with EXTERN but rather a PROTO, and all the other PROTOs from the masm32 includes doesn't cause this? I don't know exactly what is happening with masm, but the cause is that wsprintf is declared both in windows.inc and user32.inc. If you remove the definition from windows.inc, you don't get the warning.

Adding /OPT:NOREF is symptom- rather than problem-fixing. And it's a plain silly thing to do just to make harmless warning messages go away. While it won't matter too much for simplistic little assembly projects, it can mean a lot when linking to "real world" static libraries - you can end up with including a lot more code than necessary. I wouldn't expect an assembly coder to turn off an optimization, especially when it's almost free and has no side effects.

Anything else about masm & link that you don't understand? :tongue:
Posted on 2004-03-26 14:03:42 by f0dder
Hi f0dder,


With /OPT:REF, the linker will refrain from linking in the library, because it can see the EXTERN symbol isn't referenced at all. With /OPT:NOREF it assumes that just because it got a EXTERN record, it should include the symbol, whether used or not, and will pull in the library. There might be a few times when somebody wants this to happen, but I can't really think of any examples. Using EXTERNDEF instead of EXTERN also solves this - haven't looked into how EXTERNDEF works, but I assume it tells masm to only emit the EXTERN record if the symbol is referenced


One example for EXTERN was line:

extern _acrtused:ABS

in many versions of VC, which forced the c runtime startup code to be included in the executable.
A similiar switch exist(ed) forcing the linker to include the floating point support.
Posted on 2004-03-27 01:13:58 by japheth
With /OPT:NOREF it assumes that just because it got a EXTERN record, it should include the symbol, whether used or not, and will pull in the library. There might be a few times when somebody wants this to happen, but I can't really think of any examples


Could be useful for rich edit controls where you have to load the DLL but don't really call any functions in your app, just use the class and send messages to the control.
Posted on 2004-03-27 01:19:07 by donkey

A similiar switch exist(ed) forcing the linker to include the floating point support.

_fltused, iirc. And I think that symbol reference only gets generated when floating-point code is used?


Could be useful for rich edit controls where you have to load the DLL but don't really call any functions in your app, just use the class and send messages to the control.

Ah yes, that's one of the few examples, and this would probably be optimized away with /OPT:REF. The soultion here would be either to /OPT:NOREF, or something like putting a "dummy dd offset IID_ITextHost" somewhere in your program. It might seem stupid to add a dword to your app instead of just /OPT:NOREF, but if you use static libraries that use COMDAT function-level splitting, well... then it certainly isn't stupid :)
Posted on 2004-03-27 02:52:34 by f0dder
It seems that a little knowledge is dangerous. Microsoft linker handle benign redefinitions with no problems and no errors so even though there is a redefinition of wsprintfA in 2 seperate include files, there is no error that occurs because there is no change that could be called a redefinition.

It is a tolerance based on the method of storing the data in either a hash table or a dynamic tree where if there is no change, there is no error. All you have to do is error check the the data stored for the name and if its the same, ignore it, if its not, drop a redefinition error.

Here is how you test it in MASM. The prototype is directly from kernel32.inc from MASM32.


AddAtomA PROTO :DWORD
AddAtom equ <AddAtomA>
AddAtomA PROTO :DWORD
AddAtom equ <AddAtomA>
AddAtomA PROTO :DWORD
AddAtom equ <AddAtomA>
AddAtomA PROTO :DWORD
AddAtom equ <AddAtomA>
AddAtomA PROTO :DWORD
AddAtom equ <AddAtomA>
AddAtomA PROTO :DWORD
AddAtom equ <AddAtomA>
AddAtomA PROTO :DWORD
AddAtom equ <AddAtomA>
AddAtomA PROTO :DWORD
AddAtom equ <AddAtomA>
AddAtomA PROTO :DWORD
AddAtom equ <AddAtomA>
AddAtomA PROTO :DWORD
AddAtom equ <AddAtomA>

The exe that I tested it with builds with no errors.

Pull the other leg. :tongue:

Regards,
http://www.asmcommunity.net/board/cryptmail.php?tauntspiders=in.your.face@nomail.for.you&id=2f46ed9f24413347f14439b64bdc03fd
Posted on 2004-03-27 02:53:50 by hutch--
It's a thing with masm, not the linker. Appearantly the symbol redefinition (or perhaps some other thing) causes the EXTERN record to be generated. My guess is that masm doesn't emit a EXTERN record for PROTOs unless the symbol is used, and that a symbol lookup (causes by symbol redefinition) is enough that the symbol is considered 'used'.

So there's no difference to the linker with this, except whether it sees an EXTERN or not. What you might be thinking about is if multiple symbols with the same name are found in different compilation modules, this will be handled by COMDAT folding.

Again, the "all references discarded" is a warning, not an error. The attached sample builds fine, and demonstrates the thing pretty well.

I suggest you remove the wsprintf definition from windows.inc as it should really only be in user32.inc, and that you don't propose using /OPT:NOREF to "fix things" if you don't really know what's going on.

PS
Microsoft (R) Macro Assembler Version 7.10.3077
Microsoft (R) Incremental Linker Version 7.10.3077
but the behaviour should be the same for older versions as well, including:
Microsoft (R) Macro Assembler Version 6.15.8803
Microsoft (R) Incremental Linker Version 5.12.8078


Try doing the redefinition thing, but without issuing any "includelib" statements and not passing any lib names on the linker commandline...
Posted on 2004-03-27 03:15:54 by f0dder

The exe that I tested it with builds with no errors.

It build with no warnings either, various version of Microsoft LINK.EXE from the XP DDK versions you have mentioned down to the version from the win98ddk all handle benign redefinitions without either warnings or errors.

The /OPT:REF was for a special purpose in the win98ddk which is no use in general purpose programming. Its something like having you warning level set to 4 on release code, no point in general purpose programming.

It is an option that was preset to ON rather than a default OFF so unless you need that specific characteristic, using the /OPT:NOREF just resets it to default.

Regards,
http://www.asmcommunity.net/board/cryptmail.php?tauntspiders=in.your.face@nomail.for.you&id=2f46ed9f24413347f14439b64bdc03fd
Posted on 2004-03-27 04:45:43 by hutch--

It build with no warnings either

Have a look at the test.asm I posted, and use the same commandline arguments.


The /OPT:REF was for a special purpose in the win98ddk which is no use in general purpose programming. Its something like having you warning level set to 4 on release code, no point in general purpose programming.

Erm, that's plain wrong. /OPT:REF was introduced to remove unused (unreferenced) code and data, which is very important with the COMDAT method of handling "function-level linking". It's preset to ON because it's an optimization (almost) without any side effects. Setting it to OFF means you will get an unused DLL reference in your code, which is silly.

Also, do go ahead and remove the wsprintf definition from windows.inc, otherwise you're forcing everybody using masm32 to link with user32.dll even if they don't use any functions from it (well, when combined with /OPT:NOREF. /OPT:REF will give the innocent warning and discard user32.dll).
Posted on 2004-03-27 05:00:36 by f0dder