Hello everybody,


I got the following code from MicroSoft support site:


**************************************
32-bit Windows Operating Systems
In 32-bit versions of Windows, the operating system has built-in disk caching. The only way to force a file to be flushed to disk is by linking to Commode.obj.

Commode.obj is designed to affect the way the C Runtime handles files. When you link to this .obj file, a call to the C runtime function fflush also forces the operating system to flush its cache to disk, making the call to _commit unnecessary.
Sample Code
/* Compile options needed: none

This sample code is designed to show some of the more
common ways to flush both the C runtime buffers and the
operating system cache
*/

#include <stdio.h>
#include <ofstream.h>

void DoSomeCFileIO();
void DoSomeCXXFileIO();

void main()
{
DoSomeCFileIO()
DoSomeCXXFileIO();
}

void DoSomeCFileIO()
{
FILE* CFileBuf;

// Open CFileBuf for output & perform some writes

fflush(CFileBuf);
_commit(_fileno(CFileBuf));

// The call to fflush will cause the C Runtime to flush
// the buffers associated with CFileBuf to the
// Operating system.
// The call to _commit will tell Smartdrv.exe to flush
// its cache to the disk.
// The _commit function requires a file handle, hence the
// call to _fileno
// _commit will only function on 16-bit operating systems.
// On 32-bit operating systems, you need to link to Commode.obj
}

void DoSomeCXXFileIO()
{
ofstream CXXFileBuf;

// Open CXXFileBuf & perform some writes

CXXFileBuf.flush();
_commit(CXXFileBuf.rdbuf()->fd());

// The call to flush causes the C Runtime to flush
// the buffer associated with CXXFileBuf to the operating system.
//
// The call to _commit tells Smartdrv.exe to flush
// its cache to the disk.
//
// The _commit function requires a file handle, hence the
// call to ofstream.rdbuf()->fd()
// _commit will only function on 16-bit operating systesm.
// On 32-bit operating systems, you will need to link to Commode.obj.
}
**************************************

Can anybody help me understand this code?
What do these two functions mean in assembly?

fflush(CFileBuf);
and
_commit(_fileno(CFileBuf));

How do I link to Commode.obj ?



Any help suggestions appreciated.

best regards,

czDrillard
Posted on 2002-03-03 12:27:38 by czDrillard
fflush flushes the FILE stream cache. You can't say "what does this
mean in assembly"... except perhaps "invoke fflush, offset myFile" ;).
It's a library function. You should look up C FILE I/O if you need a
reference.

As for flushing the filesystem cache... dunno how to do it programatically,
but you can get "sync.exe" from http://www.systeminternals.com/ .
You shouldn't be flushing the fs cache without very good reason
anyway.
Posted on 2002-03-03 12:49:37 by f0dder
From my understanding (decription from ms doc :)) you only need to call fflush(CFileBuf) on anything win95 and up.
To "link" to Commode.obj you need a Commode.h and add it to your project settings (assuming you are using VC++) But if you are using VC++ I would recommend using the following:

#include <fstream>
using namespace std;

int main()
{
fstream file("filename.dat");

//some file stuff (write to, manipulate...)

file.flush();
file.close();
}

agian, that is just a personal choice. If you would like for me to explain why you need to flush :) just ask, or anyother questions. Hope this helps, Jag
Posted on 2002-03-03 12:53:06 by Jag
you don't need flush, it's done automatically on close. One of the
few uses of flush these days is when you need to display a partial
line, do some calculations, and then write done... because printf
doesn't flush until it sees a '\n', which means you wouldn't see the
"performing mindboggling computations..." until the "done\n" is
printed.
Posted on 2002-03-03 14:29:48 by f0dder
sorry forgot the comment between file.flush(); and file.close();

...
file.flush();
// do more file modifications
file.close();
...

flush is good inside of loops, where you still want the file open but you need the data writen out eg:

while(SomeCondition)
{
file << NewRecord << endl;
file.flush();
}
file.close();

Jag
Posted on 2002-03-03 15:28:29 by Jag
One of the things you do use a HLL flush for is if you are writing large amounts of data to disk sequentially over a long period. VCACHE can die and take the program doing the disk writes out if it overflows. You also get this if you try single large writes to disk with very large files so its worth making the effort to avoid the odd crash.

Regards,

hutch@movsd.com
Posted on 2002-03-03 15:43:44 by hutch--
vcache and fflush have nothing to do with eachother. fflush flushes
the internal (libc) file buffer for that specific file, but does not guarantee
a windows cache subsystem (vcache in case of 9x, dunno what they
call it on NT) flush. Linking with commode.obj might do this...

fflush should only be used when you know you need it... as in the printf
situation. Incorrect usage of fflush can give major speed slowdowns.

As for manually doing filesystem flushes, I can thing of two situations
where it can be useful. The first is if you want to "sync" your filesystems
if you for instance need to run a driver that might fault - in this case
it's not only a good idea to flush the cache, it's insane not doing this.
Again, this has nothing to do with fflush.

The second would be mission-critical applications where data loss is
not tolerable, but I don't know if you'd use typical operating systems
there ;).

As for vcache b0rking... I've never seen it do that. It's written to
flush at intervals (lazy write), to write data as linearly as possible
(to avoid head seek overhead), and to collect data in chunks before
writing (for speed purposes). If vcache runs low on buffer space, it
will flush write cache data, and perhaps start discarding read cache
data... automatically. So there's not much point in manually flushing
it, as you will end up discarding a lot of read cache data.
Posted on 2002-03-03 16:01:27 by f0dder
=========
fflush flushes the internal (libc) file buffer
=========

Fortunately there are other languages that do not use C runtimes. Cache saturation is a well understood phenomenon and that is why many HLLs have a flush capacity.

Regards,

hutch@movsd.com
Posted on 2002-03-03 16:11:11 by hutch--
Well, we were discussing C fflush here, so... :).
For buffered file I/O using your language libraries (whichever that
may be), it is correct not flushing the filesystem cache on a single
file flush. After all, fs cache flush flushes everything, where you
only want to flush a single file when calling the fflush function.
As for fs flushing, it should be done by using the OS API, if your OS
has one.

Yep, flushing file buffers has its uses, just know the effects before
you start sprinkling flushes all over your code.

And don't flush the fs cache unless you know you have to ;).
Posted on 2002-03-03 16:23:44 by f0dder
This is not an idle question that I'm asking to satisfy curiousity but I'm still trying to find a way to write the index.dat files from memory to disk, if that's is the correct way of expressing it, when I want to. Right now this happens only on system shut down. It seems to me that there should be a method to emulate this behaviour without shutdown the system. I have searched everywhere and read any info from Google that seemed appropriate but have learned nothing only that I know what I want to do but without any real understanding of how to do it. I was grasping at straws and hoping maybe fflush was the answer.

I will do as f0dder suggested and search for C FILE I/O to learn more. Thanks to everyone who has replied to this post and if any other ideas come up please let me know. Thanks for your code Jag, but how would it look in assembly?

Btw, thanks for the link f0dder. I tried the utility and it flushes the buffer but does not write the index.dat files to disk. So that saves me the trouble of looking at fflush any longer.

best regards,

czDrillard
Posted on 2002-03-04 09:47:25 by czDrillard
czDrillard, fflush would have nothing to do with what you want.
Too bad that a filesystem flush doesn't flush the files... perhaps
some system DLL, or explorer.exe, is responsible for the index.dat
files, and don't flush until the app is terminated? Just an idea.
Posted on 2002-03-04 10:52:39 by f0dder
Hey

Seems to me that you're not really looking to flush the cache, but looking for when the system writes the file to disk AND closes the handle to the file (as f0dder is suggesting) - I'm guessing you want to change the files or perhaps delete them. Seems to me, the thing to do is to figure what programs holds the handles and closes the files at system shutdown, and then perhaps toy a bit with those programs. If you're unlucky (as I would expect you are) it's explorer that uses the file, and you'll have to either close it, or patch it before you can do what you want (i.e. access the file).
Patching explorer is probably not the best way to go about things, I would instead suggest hooking a preferred api, and then sending a wm_close or the likes to explorer - the system will open a new version of it, instead of shutting down, and in the space between the old version closing and the new version starting, you may be able to do what you want.

Fake
Posted on 2002-03-04 11:03:04 by Fake51
cz,

There is an API for flushing cached file data.

BOOL FlushFileBuffers( HANDLE hFile );

// open handle to file whose buffers are to be flushed

This should in part do what you are after.

Regards,

hutch@movsd.com
Posted on 2002-03-04 17:28:53 by hutch--
Hutch-- I have no problem getting handles to all the index.dat files and I tried FlushFileBuffers with no success. GetLastError shows the function call executed successfully but index.dat files were not forced to write to disk :(

Fake51 You guess correctly. I want to alter the data in the index.dat files, not delete it. I don't want to patch internet explorer. but your idea of closing explorer might have possibilities.

f0dder I will look at some of the api's in wininet.dll Maybe one of them can do what I want.
Thanks kindly to all replies,

best regards,

czDrillard
Posted on 2002-03-04 23:32:39 by czDrillard
cz,

You have probably done this but check this,

The file handle must have GENERIC_WRITE access to the file.

Regards,

hutch@movsd.com
Posted on 2002-03-05 04:24:51 by hutch--
Hi hutch--

Unfortunately I did use GENERIC_WRITE. Here's the code:

***********************
invoke CreateFile,IndexPath,GENERIC_READ or GENERIC_WRITE,
FILE_SHARE_READ or FILE_SHARE_WRITE,NULL,\
OPEN_EXISTING,NULL,NULL
mov hIndexFile,eax
cmp eax,-1
jz @OpenError

invoke FlushFileBuffers,hIndexFile
invoke CloseHandle,hIndexFile
***********************

When I first saw this api ( FlushFileBuffers )I read all the documentation I could find. I've been working on this off and on for weeks and naturally I got pretty excited. It sounds like it should work, but it doesn't.

best regards,

czDrillard
Posted on 2002-03-05 09:42:14 by czDrillard
If <whatever> hasn't written to the index files yet, there's no filesystem
cache data to flush =)
Posted on 2002-03-05 10:08:52 by f0dder
I recommend checking if there are any write protections on the file, if not get the address of the file (Actual harddrive address) and then hard write it out, a byte at a time. of course I would make a copy of the original (exept in this case doesnt sound importatant). Dont know what else to tell you, Jag
Posted on 2002-03-05 21:23:01 by Jag
f0dder, I may be thinking about this all wrong, but my assumption is or has been the index files are used (among other things) to display the web addresses by certain CLSID files. Evidence of this is if I delete an index entry it disappears from the windows explorer view. Specifically History.IE5 index.dat and MSHist~1 index.dat. After an internet session, if I open windows explorer and look in the History folder I see a listing of all the web sites and under 'Today' I see a listing of all the pages. However, if I open the relevant index files they may have some data, or may not, written in them. It appears that only the complete data from the previous web site is being written, not the last web site until shut down. Yet it's visible from windows explorer. The only thing is it must be displayed from a memory location whether or not it has been written to the index file. Question is: how to either read it from memory or display it from memory or write the complete memory location to the index.dat file in my program?

Jag , I tried that but if the original, (the index.dat file) has incomplete data it doesn't help. The index files are not read only. But an access error is generated (share violation) if try to open large files in windows explorer. I don't have any problem opening or writing to them with my program.

Thanks for the answers and I keep trying. Although I don't pretend to be an expert on index files I have a good understanding of the inner working of them. But maybe I'm looking totally in the wrong direction. If any solution or light comes to this problem I will post it for interested people.

best regards,

czDrillard
Posted on 2002-03-06 09:56:52 by czDrillard