Hi All,
What does IMAGE_SCN_MEM_NOT_PAGED do?

Is there any way to force the OS EXE loader to allocate physical
(locked, not swappable) memory for my program, just as if it was
a VxD?

Thanks,
Maverick
Posted on 2002-01-16 17:16:47 by Maverick
There aren't any way to do it that I know of. If you really *need*
to lock the memory, you should probably use a VXD/KMD. If you
don't *need* to lock the memory... don't :). If your program is small
enough it will not be swapped out. It it's large enough to be swapped
out, it's rude to use locked memory.

Just my thoughts.

Why do you need locked memory?
Posted on 2002-01-16 17:47:00 by f0dder
flag 00040000 also states "LOCKED".. dunno if it's supported for real anyway.

Anyway, why do I need locked memory? Because I think that virtual memory sucks, that's why. I hope you will forgive me about having such a strong opinion on the subject (relative to the use I do with a computer, of course, I do NOT say that virtual memory ALWAYS AND FOR ALL APPLICATIONS sucks).

I write *realtime* games/demos: ok, Windows is not a realtime OS but that's what we were all enforced to use and to code for, expecially at the commercial level. To die of hunger is not an option for me, so I don't think that it would be fair to tell me "you're free to code for UnknownNoMarketOS if you think Windows is not perfect".
So I have to make it as much realtime as possible.. this means using high priority to get as much CPU as possible (or even more advanced solutions), this also means to get physical memory instead of virtual memory, when I have almost all the CPU for me anyway. Why? Because I hate to get the action in my frantic/realtime game interrupted by those silly and often useless vmem swaps.

That's my view on the subject, you may have a radically different view, so don't insult me if I replied to your question.

Take care,
Maverick
Posted on 2002-01-16 18:06:29 by Maverick
Well... if you lock your game memory, windows will see it's available
pool of memory decrease, and might choose to do swapping it might
not have done otherwise. Also... *why* lock, really? I've seen countless
games that don't lock, and they have all run fine. No swapping done in
critical parts. And no, this is not on some superleet machine with tons
of ram (well, of course it hasn't swapped with my machine with tons of
ram either), but machines that have met the recommended game requirements.

Windows doesn't swap for no reason. It swaps if it's low on memory. It will
go for least recently used pages first. Thus, critical parts of your executable
shouldn't be swapped out, nor should data you use much. With "proper" use of
memory (ie, not allocating a big 128meg block or more, like some games do
today :-( ), you shouldn't really need locked memory.

Be careful when messing with thread priorities. Especially realtime ;).
Realtime thread priority isn't needed in games. It might be justifiable
to set it to "above normal" or "high" though. Hell, if games run fluidly on
my old 200mhz machine without any of these tricks... why would you need them
at all? I can understand why you might desire these things, coming from an
amiga background where you could write much lower level code, since you didn't
have to think about multitasking, and didn't have to support a very broad
range of hardware.

Come to think of it, I have never experienced paging in any well-designed
game unless some background task has kicked in - and locked memory would
hardly help on that ;).
.
Posted on 2002-01-16 18:24:24 by f0dder
Hi again f0dder,

Well... if you lock your game memory, windows will see it's available
pool of memory decrease, and might choose to do swapping it might
not have done otherwise.


Not if I disable "multitasking" :grin:


Also... *why* lock, really? I've seen countless
games that don't lock, and they have all run fine. No swapping done in
critical parts. And no, this is not on some superleet machine with tons
of ram (well, of course it hasn't swapped with my machine with tons of
ram either), but machines that have met the recommended game requirements.


Hmm.. permit me to disagree. In my experience those swaps happen, and more than I'd allow. Anyway, since I had 256MB (now 512MB) I've always disabled virtual memory (Win95/98/ME), and am happy so. I'm not a fan of virtual memory (as we know it from Windows), can't help it.
In my "Game OS" (let's call it this way for simplicity, ok?) I have implemented the heap functions in a way that will limit fragmentation and physical usage as much as possible.. but still my OS doesn't take too much decisional power from my application. Of course all of this is useless if the host OS supports only virtual memory. Not that I don't know how to allocate physical memory (or lock virtual) without even using a VxD in 9x.. it's just that I'd prefer a more general method, that doesn't use semi-undocumented functions, and that runs on NT/2000/XP as well. All of this is optional, of course, but I know how I keep that option set when I run my apps..

Windows doesn't swap for no reason.


Last time I really tested this (I admit, it was Windows95 OSR2.1 times) this was *absolutely* untrue (I tested it throughfully, don't doubt). I hope things have evolved with the newer Windows versions.

Also, please don't assume that Windows is perfect. There was a thread time ago on Usenet (which then became a big flamewar) because I was a bit pissed that a function (don't really recall anymore if it was related to high resolution timers or to DX's VBL support) the OS returned "this function IS implemented", while it was NOT.. with all the consequences you may imagine. Really, such things may piss me off a lot, while to others it's just "a bug", or even a reason to blame me as an anti-Microsoft troll, as if I was happy to waste my development time hunting down Windows bugs instead of my own bugs. Too many times it happened this way.. there was a nasty, strange bug.. I looked for it for days and days, until I found it.. and it wasn't in my code. I would have preferred it was.

Hell, if games run fluidly on
my old 200mhz machine without any of these tricks


May I ask you something? All of DX 8.1's samples contained in the SDK, on my Athlon-XP 1800+ with a 3DFX Voodoo5 V5500 run with "marvellous", hundreds of fps framerate for 0.9 sec, then stop for 0.1 sec, and so on.. cyclically. Do you call this "fluid"?

Have you ever seen an Amiga mouse pointer (mouse and video are perfectly frame rate synched)? How does it compare to the one of any PC? That's what I call "fluid", personally. Another reason why I don't like triple-buffering (it makes you claim a higher framerate, but it makes the fps unconstant. With double-buffering you have a bigger possibility to make it constant).
I have a certain experience with video oriented machines, I'm quite sensitive to what is fluid and what seems to be.

I can understand why you might desire these things, coming from an
amiga background where you could write much lower level code, since you didn't
have to think about multitasking


Let's exclude memory protection, the Amiga has a much better multitasking than Windows. The Amiga is not a console, it's designed more like a workstation.

The lack of memory protection didn't make the Amiga crash more than Windows9x anyway, and by far.

In 12 years that I've had an HD in my Amiga, I've NEVER reinstalled the OS. I really don't have a clue how installing AmigaOS is.. I never did it. I bought my A1200 with the OS already installed on the HD. Now tell me how many times the average user reinstalls Windows9x?

Ok, although I don't think it's fair, you'll tell me that if 9x sucks, NT is a good product instead. Well, at what price? A task switch for every OS function call, as a norm? Ain't that ridicolous? I can't crash a machine neither if I code in Java or even better in Logo.. for me it's the same thing.

NT doesn't crash?

Was just studying this (quite famous) document on the PE file format, when I found: (document is hellowmb.txt)


3. This article was based on a 'trial and error' method. Some solutions exist
only because they work. So don't ask why... (Actually the trial and error
resulted in two BSODs, thus proving that a program can crash windows NT without
even running it's own code)



Yes, you can call me an anti-Microsoft person (don't call me a troll though :tongue: ). But I didn't born and say "Microsoft sucks", actually I learnt it during the years..

IF I'm biased, I became biased anyway.. I wasn't born this way (nor anybody tryed to convince me to have these anti-MS opinions.. rather the opposite in some cases).

For honesty of thought, I've to add I'm not a fan of Linux too (although I certainly prefer it to Windows under many aspects.. starting from the OpenSource philosophy and the "community" way of doing things).

Anyway, I wish you g'nite.. I better go to sleep (a couple of hours per day is better than nothing ;) )

Maverick
Posted on 2002-01-16 19:54:49 by Maverick
I don't call you anti-microsoft. And I'm not a blind devotee myself.
There's a lot of microsoft stuff that sucks. Win9x. I'm not too fond
of .net. The happyfluffy UI of WinXP sucks. Office sucks. And win2k
isn't perfect - it's just the best solution there is at the moment.
(The kernel is quite fine and dandy, the problems lie with the other
parts of the OS, that have been left to less competent programmers.)


Not if I disable "multitasking"

What you want is to code for the X-Box then :-). If you just mean
"task switching" as in alt+tab... then don't disable that either.
What if I have to adjust my volume settings because I've changed
from headphones to speakers and forgot to adjust? Or what if I
need to shut down my ICQ client, or perhaps quickly respond to a
message? Terminating the game and starting it again is annoying,
especially with large games that take a long time to load.


I'm not a fan of virtual memory (as we know it from Windows), can't help it.

I am. I couldn't run stuff like 3d studio max without it. Ok,
sometimes the strategies might feel a bit weird... but it's all
quite tweakable really. Btw, you don't want to run games on the
server editions of win2k, as they feel the filesystem cache is
more important than keeping all applications in memory... which
makes sense for a server, I guess.


Not that I don't know how to allocate physical memory (or lock virtual)
without even using a VxD in 9x..

Kernel32 ordinal 0? Oh well, if you make physical locking optional
I guess it's okay. Just don't stuff it down anybodys throath :-).


hundreds of fps framerate for 0.9 sec, then stop for 0.1 sec, and so
on.. cyclically. Do you call this "fluid"?

No, I call it weird. I've never had problems like that on my puny
little GeForce2, nor on my kid brothers' even punier voodoo3.
My friend who recently bought a GeForce3 (top model, Ti-500 deluxe)
had similar problems though. And lots of crashes. And stalls. So now
he's ditching his 1ghz athlon and getting a pentium4... you see,
VIA (the guys who make the chipsets for athlons) didn't feel like
licensing the AGP specs, so the reverse engineered them. That's really
oh so great for system stability. I have an athlon myself, but it's
one of the lower frequencies (700mhz)... if I turn on fast writes or
set AGP rate to > 2x... I get crashes 5 minutes after I enter windows.
But at least with these options off... it's perfectly stable and everything
is fluid.


Have you ever seen an Amiga mouse pointer (mouse and video are perfectly
frame rate synched)?

Yes, I owned an amiga600, and had several friends with amiga500's.


How does it compare to the one of any PC?

I don't really know what you're getting at here... with hardware mouse
cursors, the cursor should be perfectly synced with the video rate on
a PC... thing is, if you use serial mice or untweaked PS/2 mice, the
update rate of the mouse is quite sucky. With a tweaked PS/2 mouse, or
my wonderful optical mouse, all the mouse business feels a lot more
smooth than it ever did back in the amiga days.


The lack of memory protection didn't make the Amiga crash more than
Windows9x anyway, and by far.

I've had my share of guru meditations ;). BUt face it, when you run a
game on an amiga, this is almost always done from a floppy with a custom
bootloader, which means there's hardly anything going on in the background.
There's just much less chance of a crash happening. And if it happens?
Not too much of a deal, it's only the game that goes down.

As for operating system reinstalls... the only reason I've really had to
do this has been because of third-party software clogging up the system.
Or faulty drivers. Which is hardly microsoft's fault, really. It's just
the average joe windows-programmer that's a stupid twat, likes to dump
various files in various places, modify registry keys without restoring
them on uninstall (et cetera). After a few months of testing various
shareware applications on a 9x box... time to reinstall.


NT is a good product instead. Well, at what price? A task switch for
every OS function call, as a norm? Ain't that ridicolous?

Task switch? Don't think so. Ring3->Ring0? Probably, yes. When we talk
core OS functions. A call to kernel32 or user32 doesn't necessarily end
in ring0 land. I don't think this is too bad, considering the protection
the NT kernel offers. In 9x, you end up taking ring3->ring0 transitions,
*as well* as 32->16bit transitions... now tell me what is worst ;).
Also, remember that NT was built with stability in mind, not necessarily
multimedia performance. It still has better performance (for me) than
9x had. Don't tell me we all have to go code in sandboxed languages
because the core OS could be a few percentages faster that way.

NT *can* crash. If you're seriously messing around, or perhaps even trying
deliberately to make it crash, you might just get lucky :-). Interesting
that some PE stuff should be able to crash NT. Perhaps this is old 3.x or 4.x
kernels? I have thrown some *quite* messy PE's at the loader, and I've never
had anything but a GPF. I've never had any of my ring3 code BSOD nt. And
trust me, I have done some pretty dirty stuff ;). Under 9x I would have a
development cycle of build, run, crash, (hard)reboot, goto 10. Win2k? I just
press the "OK" button in the GPF dialog, and goto 10 :).
Posted on 2002-01-17 06:17:25 by f0dder