Some of you might know my current project (see sig), well i needed a function to securely delete a file if users wants to.
I've implemented the function in TableCrypt, but also decided to write another program around it.
This is the idea:
1. run installer
2. right-click file and select "Shredder".
3. Confirm that you want to delete it.
4. file is deleted securely (total times overwritten = 10).
finally you can run installer once more to uninstall the program.
Source is included in the archive: Shredder.
PS. I'm always happy to hear any comments on my programs ;)
I've implemented the function in TableCrypt, but also decided to write another program around it.
This is the idea:
1. run installer
2. right-click file and select "Shredder".
3. Confirm that you want to delete it.
4. file is deleted securely (total times overwritten = 10).
finally you can run installer once more to uninstall the program.
Source is included in the archive: Shredder.
PS. I'm always happy to hear any comments on my programs ;)
Nice Work!
Although i have to confess the installer will be 10x more useful to me than the program!
Thanks for sharing. :)
Although i have to confess the installer will be 10x more useful to me than the program!
Thanks for sharing. :)
Have you tried to recover the trashed file, just to test how effective the shredder is? (just out of curiocity), and man, you pump out code like crazy, lol.
I guess that's what I get for being a hobbyist coder who can't seem to get anything right.
I guess that's what I get for being a hobbyist coder who can't seem to get anything right.
Suggestions:
Don't use filemapping, it's a bit slower than "raw" I/O with WriteFile. Actually in your case, it might be plenty slower, since you get a read-modify-writeback cycle when using MMF.
Instead I'd advice you to open the file in FILE_FLAG_NO_BUFFERING mode - this ensures that writes don't go through the disk cache system. If you don't do this, wiping a small file could theoretically end with only a single wipe, if windows for some reason decides that FlushFileBuffers doesn't have to flush the FS cache.
As for the security aspects of the patterns to use (a new random byte for each byte in the file, alternating patterns of some form, bla bla) I'm not really sure what's most effective - reading off magnetic residual data is not something I've dealt with :)
Don't use filemapping, it's a bit slower than "raw" I/O with WriteFile. Actually in your case, it might be plenty slower, since you get a read-modify-writeback cycle when using MMF.
Instead I'd advice you to open the file in FILE_FLAG_NO_BUFFERING mode - this ensures that writes don't go through the disk cache system. If you don't do this, wiping a small file could theoretically end with only a single wipe, if windows for some reason decides that FlushFileBuffers doesn't have to flush the FS cache.
As for the security aspects of the patterns to use (a new random byte for each byte in the file, alternating patterns of some form, bla bla) I'm not really sure what's most effective - reading off magnetic residual data is not something I've dealt with :)
If you REALLY wanted to be safe, you could write the section over with random data (coming from whatever) untill you basically break that section of disk :P but that's pretty abusive, and would probably take quite some time...
Have you tried to recover the trashed file, just to test how effective the shredder is? (just out of curiocity), and man, you pump out code like crazy, lol.
Not with this release, but i've already written this program in 2005, but never released it.
In that program i've tested it.
And although you can retrieve the file without any problems, it only has random content and so rendering the file useless.
The only better way will probably be to rename the file first to a random name.
Don't use filemapping, it's a bit slower than "raw" I/O with WriteFile. Actually in your case, it might be plenty slower, since you get a read-modify-writeback cycle when using MMF.
Instead I'd advice you to open the file in FILE_FLAG_NO_BUFFERING mode - this ensures that writes don't go through the disk cache system. If you don't do this, wiping a small file could theoretically end with only a single wipe, if windows for some reason decides that FlushFileBuffers doesn't have to flush the FS cache.
I will look into this this evening. You're probably right. I've used this approach since the original program where this program is based on, encrypted the file with a singlekey XOR encryption. There it was a lot faster to memorymap. But since i do not have to read here, WriteFile will probably be a lot faster.
Thanks for the feedback! I really appreciate it!
Hm, sounds weird that filemapping was faster - if you're doing "linear" processing of a file, filemapping tends to be one of the slower alternatives, because it incurs a pagefault for each 4kb you process. With almost all I/O the (clock) time to process a file will be the same, but the CPU load while processing will be higher for mmapped files.
well theres that, and the possibility that mapped view of a file might not flush the buffers to the file, it might be kept in the cache... realistically for security the readfile/writefile method should be done, with a flushfilebuffers on each write, to ensure the data is purged to the disk, otherwise if a cache hit occours then the flush is not done, and the security goes out the window, as the data wont be written to the disk, thus the security is weakened... the multiple writes are 'standard' security (dod etc) for overwriting data, done usually in 7 stages, 0, 1, checkerboard, inverse checkerboard, random byte, xor'd random byte and so on... each stage attempting to permanently destroy any residual magnetic / binary pattern remaining on the disk sector......
I've updated the program using the writefile method. Although its much slower now, i feel it's more secure.
I've held open taskmgr during shredding a 66mb file.
It took 11 minutes, but you could see that 660MB's were written to disk.
With the FlushFile method not even 1 byte is visibly written to disk (although the data has changed.....).
I've held open taskmgr during shredding a 66mb file.
It took 11 minutes, but you could see that 660MB's were written to disk.
With the FlushFile method not even 1 byte is visibly written to disk (although the data has changed.....).
Scorpion, what buffer size do you use, and are you doing FILE_FLAG_NO_BUFFERING? It really should NOT be slower!
Unless the other technique does not actually write that many bytes to disk.
here's the code for the function:
Since it takes 11 minutes to write 660mb's to disk it isn't that slow. It's just a lot slower then the previous function.
it writes about 1mb/s so perhaps i could improve speed, but i'm not quite sure how.
here's the code for the function:
;##################################################
;procedure which removes the file. returns -1 on error
;##################################################
RemoveFile proc file_to_delete:DWORD,number_of_times:DWORD
LOCAL hFile:DWORD
LOCAL inputfilesize:DWORD
LOCAL backup:DWORD
LOCAL BytesWr:DWORD
LOCAL mybyte:BYTE
cmp number_of_times,0
jnz @F
xor eax,eax
dec eax
ret
@@:
invoke CreateFile,file_to_delete,GENERIC_WRITE, 0, NULL, OPEN_EXISTING,\
FILE_ATTRIBUTE_NORMAL, NULL
.if eax==NULL
@@:
invoke CloseHandle,hFile
xor eax,eax
dec eax
ret
.endif
mov hFile,eax
invoke GetFileSize,eax,NULL
mov inputfilesize,eax
;----Rounds Start -----
invoke GetTickCount
mov backup,eax
mov edx,number_of_times
times_loop:
push edx
invoke SetFilePointer,hFile,0,0,FILE_BEGIN
mov ecx,inputfilesize
@@:
push ecx
invoke GenerateRandomByte
mov mybyte,al
invoke WriteFile,hFile,addr mybyte,1,addr BytesWr,NULL
pop ecx
dec ecx
jnz @B
invoke SetFilePointer,hFile,0,0,FILE_BEGIN
mov eax,backup
mov myseed,eax
mov ecx,inputfilesize
@@:
push ecx
invoke GenerateRandomByte
not eax
mov mybyte,al
invoke WriteFile,hFile,addr mybyte,1,addr BytesWr,NULL
pop ecx
dec ecx
jnz @B
pop edx
dec edx
jnz times_loop
;----Rounds End -------
invoke CloseHandle,hFile
invoke DeleteFile,file_to_delete
xor eax,eax
ret
RemoveFile endp
Since it takes 11 minutes to write 660mb's to disk it isn't that slow. It's just a lot slower then the previous function.
it writes about 1mb/s so perhaps i could improve speed, but i'm not quite sure how.
Omg, you're writing one byte at a time - eek! I said you should do a decent buffersize :)
(and yes, there actually is a chance that the old routine didn't write just as much to disk, because of the "lazy writeback" nature of MMF).
(and yes, there actually is a chance that the old routine didn't write just as much to disk, because of the "lazy writeback" nature of MMF).
Scorp,
It's normal that it's slower as your calling Writefile for each byte your writing.
Try writing 4kb chuncks instead of just 1 byte, it should alot be faster.
It's normal that it's slower as your calling Writefile for each byte your writing.
Try writing 4kb chuncks instead of just 1 byte, it should alot be faster.
Scorp,
It's normal that it's slower as your calling Writefile for each byte your writing.
Try writing 4kb chuncks instead of just 1 byte, it should alot be faster.
I would even suggest 64kb or larger buffers... with 4kb buffers you're getting around the same amount of the expensive user->kernel->usermode transitions as the MMF approach, even though you don't get the expensive #PF.
I know, i know... I was at work and a bit busy... :lol:
here's the updated code.
However, i am not able to use the no buffer flag since it requires writes which might overwrite other files or not the complete file.
I had 10kb buffer, but changed to 64.
Any ideas on the flag though?
here's the updated code.
However, i am not able to use the no buffer flag since it requires writes which might overwrite other files or not the complete file.
RemoveFile proc file_to_delete:DWORD,number_of_times:DWORD
LOCAL hFile:DWORD
LOCAL inputfilesize:DWORD
LOCAL backup:DWORD
LOCAL BytesWr:DWORD
LOCAL shredbuf[1024*64]:BYTE
LOCAL buffer_times:DWORD
LOCAL my_remainder:DWORD
cmp number_of_times,0
jnz @F
xor eax,eax
dec eax
ret
@@:
invoke CreateFile,file_to_delete,GENERIC_WRITE, 0, NULL, OPEN_EXISTING,\
FILE_ATTRIBUTE_NORMAL, NULL
.if eax==NULL
@@:
invoke CloseHandle,hFile
xor eax,eax
dec eax
ret
.endif
mov hFile,eax
invoke GetFileSize,eax,NULL
mov inputfilesize,eax
;----Rounds Start -----
invoke GetTickCount
mov backup,eax
mov myseed,eax
.if inputfilesize<=sizeof shredbuf
mov edx,number_of_times
times_loop_small:
push edx
mov ecx,inputfilesize
lea esi,shredbuf
@@:
invoke GenerateRandomByte
mov byte ptr ,al
inc esi
dec ecx
jnz @B
mov eax,backup
mov myseed,eax
invoke SetFilePointer,hFile,0,0,FILE_BEGIN
invoke WriteFile,hFile,addr shredbuf,inputfilesize,addr BytesWr,NULL
mov ecx,inputfilesize
lea esi,shredbuf
@@:
invoke GenerateRandomByte
not eax
mov byte ptr ,al
inc esi
dec ecx
jnz @B
invoke SetFilePointer,hFile,0,0,FILE_BEGIN
invoke WriteFile,hFile,addr shredbuf,inputfilesize,addr BytesWr,NULL
pop edx
dec edx
jnz times_loop_small
;----Rounds End -------
.else
mov eax,inputfilesize
mov ebx,sizeof shredbuf
xor edx,edx
div ebx
mov buffer_times,eax
mov my_remainder,edx
mov edx,number_of_times
times_loop_big:
push edx
invoke SetFilePointer,hFile,0,0,FILE_BEGIN
mov eax,buffer_times
bufloop1:
push eax
mov ecx,sizeof shredbuf
lea esi,shredbuf
@@:
invoke GenerateRandomByte
mov byte ptr ,al
inc esi
dec ecx
jnz @B
invoke WriteFile,hFile,addr shredbuf,sizeof shredbuf,addr BytesWr,NULL
pop eax
dec eax
jnz bufloop1
mov ecx,my_remainder
.if ecx==0
jmp second_round
.endif
lea esi,shredbuf
@@:
invoke GenerateRandomByte
mov byte ptr ,al
inc esi
dec ecx
jnz @B
invoke WriteFile,hFile,addr shredbuf,my_remainder,addr BytesWr,NULL
second_round:
invoke SetFilePointer,hFile,0,0,FILE_BEGIN
mov eax,buffer_times
bufloop2:
push eax
mov ecx,sizeof shredbuf
lea esi,shredbuf
@@:
invoke GenerateRandomByte
not eax
mov byte ptr ,al
inc esi
dec ecx
jnz @B
invoke WriteFile,hFile,addr shredbuf,sizeof shredbuf,addr BytesWr,NULL
pop eax
dec eax
jnz bufloop2
mov ecx,my_remainder
.if ecx==0
jmp finish_round
.endif
lea esi,shredbuf
@@:
invoke GenerateRandomByte
not eax
mov byte ptr ,al
inc esi
dec ecx
jnz @B
invoke WriteFile,hFile,addr shredbuf,my_remainder,addr BytesWr,NULL
finish_round:
pop edx
dec edx
jnz times_loop_big
.endif
invoke CloseHandle,hFile
invoke DeleteFile,file_to_delete
xor eax,eax
ret
RemoveFile endp
I had 10kb buffer, but changed to 64.
Any ideas on the flag though?
However, i am not able to use the no buffer flag since it requires writes which might overwrite other files or not the complete file.
Huh?
All the nobuffer flag requires is that your reads/writes are sector aligned. What this means is that you should write a bit more than the real length of the file, but that's not a problem.
Your new code looks weird.
Your new code looks weird.
thanks ;)
It doesn't really matter that it looks weird, but it works a lot faster (10kb buffer btw, 64kb makes the program crash... it's probably too much to push on stack ;) ).
total time for 40mb file was around 10 seconds with over 400mb's written.
Will take a look at the no buffer story once more.
this should be it.
44MB file takes around 18 seconds to overwrite 10 times with flag no buffer.
RemoveFile proc file_to_delete:DWORD,number_of_times:DWORD
LOCAL hFile:DWORD
LOCAL inputfilesize:DWORD
LOCAL backup:DWORD
LOCAL BytesWr:DWORD
LOCAL shredbuf[1024*10]:BYTE
LOCAL buffer_times:DWORD
LOCAL my_remainder:DWORD
cmp number_of_times,0
jnz @F
xor eax,eax
dec eax
ret
@@:
invoke CreateFile,file_to_delete,GENERIC_WRITE, 0, NULL, OPEN_EXISTING,\
FILE_ATTRIBUTE_NORMAL or FILE_FLAG_NO_BUFFERING, NULL
.if eax==NULL
@@:
invoke CloseHandle,hFile
xor eax,eax
dec eax
ret
.endif
mov hFile,eax
invoke GetFileSize,eax,NULL
mov ebx,512
xor edx,edx
div ebx
.if edx==0
mul ebx
.else
mul ebx
add eax,512
.endif
mov inputfilesize,eax
;----Rounds Start -----
invoke GetTickCount
mov backup,eax
mov myseed,eax
.if inputfilesize<=sizeof shredbuf
mov edx,number_of_times
times_loop_small:
push edx
mov ecx,inputfilesize
lea esi,shredbuf
@@:
invoke GenerateRandomByte
mov byte ptr ,al
inc esi
dec ecx
jnz @B
mov eax,backup
mov myseed,eax
invoke SetFilePointer,hFile,0,0,FILE_BEGIN
invoke WriteFile,hFile,addr shredbuf,inputfilesize,addr BytesWr,NULL
mov ecx,inputfilesize
lea esi,shredbuf
@@:
invoke GenerateRandomByte
not eax
mov byte ptr ,al
inc esi
dec ecx
jnz @B
invoke SetFilePointer,hFile,0,0,FILE_BEGIN
invoke WriteFile,hFile,addr shredbuf,inputfilesize,addr BytesWr,NULL
pop edx
dec edx
jnz times_loop_small
;----Rounds End -------
.else
mov eax,inputfilesize
mov ebx,sizeof shredbuf
xor edx,edx
div ebx
mov buffer_times,eax
mov my_remainder,edx
mov edx,number_of_times
times_loop_big:
push edx
invoke SetFilePointer,hFile,0,0,FILE_BEGIN
mov eax,buffer_times
bufloop1:
push eax
mov ecx,sizeof shredbuf
lea esi,shredbuf
@@:
invoke GenerateRandomByte
mov byte ptr ,al
inc esi
dec ecx
jnz @B
invoke WriteFile,hFile,addr shredbuf,sizeof shredbuf,addr BytesWr,NULL
pop eax
dec eax
jnz bufloop1
mov ecx,my_remainder
.if ecx==0
jmp second_round
.endif
lea esi,shredbuf
@@:
invoke GenerateRandomByte
mov byte ptr ,al
inc esi
dec ecx
jnz @B
invoke WriteFile,hFile,addr shredbuf,my_remainder,addr BytesWr,NULL
second_round:
invoke SetFilePointer,hFile,0,0,FILE_BEGIN
mov eax,buffer_times
bufloop2:
push eax
mov ecx,sizeof shredbuf
lea esi,shredbuf
@@:
invoke GenerateRandomByte
not eax
mov byte ptr ,al
inc esi
dec ecx
jnz @B
invoke WriteFile,hFile,addr shredbuf,sizeof shredbuf,addr BytesWr,NULL
pop eax
dec eax
jnz bufloop2
mov ecx,my_remainder
.if ecx==0
jmp finish_round
.endif
lea esi,shredbuf
@@:
invoke GenerateRandomByte
not eax
mov byte ptr ,al
inc esi
dec ecx
jnz @B
invoke WriteFile,hFile,addr shredbuf,my_remainder,addr BytesWr,NULL
finish_round:
pop edx
dec edx
jnz times_loop_big
.endif
invoke CloseHandle,hFile
invoke DeleteFile,file_to_delete
xor eax,eax
ret
RemoveFile endp
Any other suggestions?
using above code the average deletion speed will be around 2,4MB's per second.
This is on my 1600mhz system (running at 50% cpu) with my 4200rpm harddisk.
so if i had a faster harddisk it might come up to 5mb's per second.
One final question though: how can i retrieve the sector size on a harddisk?
i now used 512bytes, but i'm not sure if this will be effective on all harddisks....
This is on my 1600mhz system (running at 50% cpu) with my 4200rpm harddisk.
so if i had a faster harddisk it might come up to 5mb's per second.
One final question though: how can i retrieve the sector size on a harddisk?
i now used 512bytes, but i'm not sure if this will be effective on all harddisks....
White Scorpion,
The DISK_GEOMETRY structure describes the geometry of disk devices and media.
typedef struct _DISK_GEOMETRY {
LARGE_INTEGER Cylinders;
MEDIA_TYPE MediaType;
DWORD TracksPerCylinder;
DWORD SectorsPerTrack;
DWORD BytesPerSector;
} DISK_GEOMETRY ;
That is the structure. Look at DeviceIOControl function to see how to get it.
Paul
The DISK_GEOMETRY structure describes the geometry of disk devices and media.
typedef struct _DISK_GEOMETRY {
LARGE_INTEGER Cylinders;
MEDIA_TYPE MediaType;
DWORD TracksPerCylinder;
DWORD SectorsPerTrack;
DWORD BytesPerSector;
} DISK_GEOMETRY ;
That is the structure. Look at DeviceIOControl function to see how to get it.
Paul
Thanks a lot.
That is exactly what i'm looking for!
That is exactly what i'm looking for!