Some of you might know my current project (see sig), well i needed a function to securely delete a file if users wants to.
I've implemented the function in TableCrypt, but also decided to write another program around it.
This is the idea:
1. run installer
2. right-click file and select "Shredder".
3. Confirm that you want to delete it.
4. file is deleted securely (total times overwritten = 10).

finally you can run installer once more to uninstall the program.

Source is included in the archive: Shredder.

PS. I'm always happy to hear any comments on my programs ;)
Posted on 2006-07-04 17:11:31 by white scorpion
Nice Work!
Although i have to confess the installer will be 10x more useful to me than the program!
Thanks for sharing. :)
Posted on 2006-07-04 17:16:29 by asmrixstar
Have you tried to recover the trashed file, just to test how effective the shredder is? (just out of curiocity), and man, you pump out code like crazy, lol.

I guess that's what I get for being a hobbyist coder who can't seem to get anything right.
Posted on 2006-07-05 03:06:00 by Bobbias
Suggestions:

Don't use filemapping, it's a bit slower than "raw" I/O with WriteFile. Actually in your case, it might be plenty slower, since you get a read-modify-writeback cycle when using MMF.

Instead I'd advice you to open the file in FILE_FLAG_NO_BUFFERING mode - this ensures that writes don't go through the disk cache system. If you don't do this, wiping a small file could theoretically end with only a single wipe, if windows for some reason decides that FlushFileBuffers doesn't have to flush the FS cache.

As for the security aspects of the patterns to use (a new random byte for each byte in the file, alternating patterns of some form, bla bla) I'm not really sure what's most effective - reading off magnetic residual data is not something I've dealt with :)
Posted on 2006-07-05 03:17:23 by f0dder
If you REALLY wanted to be safe, you could write the section over with random data (coming from whatever) untill you basically break that section of disk :P but that's pretty abusive, and would probably take quite some time...
Posted on 2006-07-05 03:22:31 by Bobbias
Have you tried to recover the trashed file, just to test how effective the shredder is? (just out of curiocity), and man, you pump out code like crazy, lol.

Not with this release, but i've already written this program in 2005, but never released it.
In that program i've tested it.
And although you can retrieve the file without any problems, it only has random content and so rendering the file useless.
The only better way will probably be to rename the file first to a random name.


Don't use filemapping, it's a bit slower than "raw" I/O with WriteFile. Actually in your case, it might be plenty slower, since you get a read-modify-writeback cycle when using MMF.

Instead I'd advice you to open the file in FILE_FLAG_NO_BUFFERING mode - this ensures that writes don't go through the disk cache system. If you don't do this, wiping a small file could theoretically end with only a single wipe, if windows for some reason decides that FlushFileBuffers doesn't have to flush the FS cache.

I will look into this this evening. You're probably right. I've used this approach since the original program where this program is based on, encrypted the file with a singlekey XOR encryption. There it was a lot faster to memorymap. But since i do not have to read here, WriteFile will probably be a lot faster.

Thanks for the feedback! I really appreciate it!
Posted on 2006-07-05 05:32:46 by white scorpion
Hm, sounds weird that filemapping was faster - if you're doing "linear" processing of a file, filemapping tends to be one of the slower alternatives, because it incurs a pagefault for each 4kb you process. With almost all I/O the (clock) time to process a file will be the same, but the CPU load while processing will be higher for mmapped files.
Posted on 2006-07-05 05:38:38 by f0dder
well theres that, and the possibility that mapped view of a file might not flush the buffers to the file, it might be kept in the cache... realistically for security the readfile/writefile method should be done, with a flushfilebuffers on each write, to ensure the data is purged to the disk, otherwise if a cache hit occours then the flush is not done, and the security goes out the window, as the data wont be written to the disk, thus the security is weakened... the multiple writes are 'standard' security (dod etc) for overwriting data, done usually in 7 stages, 0, 1, checkerboard, inverse checkerboard, random byte, xor'd random byte and so on... each stage attempting to permanently destroy any residual magnetic / binary pattern remaining on the disk sector......
Posted on 2006-07-05 06:47:53 by evlncrn8
I've updated the program using the writefile method. Although its much slower now, i feel it's more secure.
I've held open taskmgr during shredding a 66mb file.
It took 11 minutes, but you could see that 660MB's were written to disk.
With the FlushFile method not even 1 byte is visibly written to disk (although the data has changed.....).
Posted on 2006-07-05 07:24:42 by white scorpion
Scorpion, what buffer size do you use, and are you doing FILE_FLAG_NO_BUFFERING? It really should NOT be slower!
Posted on 2006-07-05 08:49:02 by f0dder
Unless the other technique does not actually write that many bytes to disk.
here's the code for the function:

;##################################################
;procedure which removes the file. returns -1 on error
;##################################################
RemoveFile proc file_to_delete:DWORD,number_of_times:DWORD
    LOCAL hFile:DWORD
    LOCAL inputfilesize:DWORD
    LOCAL backup:DWORD
    LOCAL BytesWr:DWORD
    LOCAL mybyte:BYTE

    cmp number_of_times,0
    jnz @F 
    xor eax,eax
    dec eax
    ret
@@:
    invoke CreateFile,file_to_delete,GENERIC_WRITE, 0, NULL, OPEN_EXISTING,\
    FILE_ATTRIBUTE_NORMAL, NULL
    .if eax==NULL
@@:
        invoke CloseHandle,hFile
        xor eax,eax
        dec eax
    ret
    .endif 
    mov hFile,eax
    invoke GetFileSize,eax,NULL
    mov inputfilesize,eax
   
;----Rounds Start -----
        invoke GetTickCount
        mov backup,eax
        mov edx,number_of_times
times_loop:
        push edx
        invoke SetFilePointer,hFile,0,0,FILE_BEGIN
        mov ecx,inputfilesize
@@:
        push ecx
        invoke GenerateRandomByte
        mov mybyte,al
        invoke WriteFile,hFile,addr mybyte,1,addr BytesWr,NULL
        pop ecx
        dec ecx
        jnz @B
        invoke SetFilePointer,hFile,0,0,FILE_BEGIN
        mov eax,backup
        mov myseed,eax
        mov ecx,inputfilesize
@@:
        push ecx
        invoke GenerateRandomByte
        not eax
        mov mybyte,al
        invoke WriteFile,hFile,addr mybyte,1,addr BytesWr,NULL
        pop ecx
        dec ecx
        jnz @B
       
        pop edx
        dec edx
        jnz times_loop   
;----Rounds End ------- 
    invoke CloseHandle,hFile
    invoke DeleteFile,file_to_delete
    xor eax,eax
    ret
RemoveFile endp

Since it takes 11 minutes to write 660mb's to disk it isn't that slow. It's just a lot slower then the previous function.
it writes about 1mb/s so perhaps i could improve speed, but i'm not quite sure how.
Posted on 2006-07-05 09:38:07 by white scorpion
Omg, you're writing one byte at a time - eek! I said you should do a decent buffersize :)

(and yes, there actually is a chance that the old routine didn't write just as much to disk, because of the "lazy writeback" nature of MMF).
Posted on 2006-07-05 09:42:23 by f0dder
Scorp,
It's normal that it's slower as your calling Writefile for each byte your writing.
Try writing 4kb chuncks instead of just 1 byte, it should alot be faster.
Posted on 2006-07-05 09:51:20 by Axial

Scorp,
It's normal that it's slower as your calling Writefile for each byte your writing.
Try writing 4kb chuncks instead of just 1 byte, it should alot be faster.


I would even suggest 64kb or larger buffers... with 4kb buffers you're getting around the same amount of the expensive user->kernel->usermode transitions as the MMF approach, even though you don't get the expensive #PF.
Posted on 2006-07-05 10:03:47 by f0dder
I know, i know... I was at work and a bit busy... :lol:
here's the updated code.
However, i am not able to use the no buffer flag since it requires writes which might overwrite other files or not the complete file.

RemoveFile proc file_to_delete:DWORD,number_of_times:DWORD
    LOCAL hFile:DWORD
    LOCAL inputfilesize:DWORD
    LOCAL backup:DWORD
    LOCAL BytesWr:DWORD
    LOCAL shredbuf[1024*64]:BYTE
    LOCAL buffer_times:DWORD
    LOCAL my_remainder:DWORD

    cmp number_of_times,0
    jnz @F 
    xor eax,eax
    dec eax
    ret
@@:
    invoke CreateFile,file_to_delete,GENERIC_WRITE, 0, NULL, OPEN_EXISTING,\
    FILE_ATTRIBUTE_NORMAL, NULL
    .if eax==NULL
@@:
        invoke CloseHandle,hFile
        xor eax,eax
        dec eax
    ret
    .endif 
    mov hFile,eax
    invoke GetFileSize,eax,NULL
    mov inputfilesize,eax
   
;----Rounds Start -----
    invoke GetTickCount
    mov backup,eax
    mov myseed,eax
  .if inputfilesize<=sizeof shredbuf
        mov edx,number_of_times
times_loop_small:
        push edx     
        mov ecx,inputfilesize 
        lea esi,shredbuf     
@@:
        invoke GenerateRandomByte
        mov byte ptr ,al
        inc esi
        dec ecx
        jnz @B
        mov eax,backup
        mov myseed,eax
        invoke SetFilePointer,hFile,0,0,FILE_BEGIN
        invoke WriteFile,hFile,addr shredbuf,inputfilesize,addr BytesWr,NULL
        mov ecx,inputfilesize
        lea esi,shredbuf
@@:
        invoke GenerateRandomByte
        not eax
        mov byte ptr ,al
        inc esi
        dec ecx
        jnz @B
        invoke SetFilePointer,hFile,0,0,FILE_BEGIN
        invoke WriteFile,hFile,addr shredbuf,inputfilesize,addr BytesWr,NULL
        pop edx
        dec edx
        jnz times_loop_small   
;----Rounds End ------- 
    .else
        mov eax,inputfilesize
        mov ebx,sizeof shredbuf
        xor edx,edx
        div ebx
        mov buffer_times,eax
        mov my_remainder,edx
        mov edx,number_of_times
times_loop_big:
        push edx
        invoke SetFilePointer,hFile,0,0,FILE_BEGIN
        mov eax,buffer_times
bufloop1: 
        push eax     
        mov ecx,sizeof shredbuf
        lea esi,shredbuf       
@@:       
        invoke GenerateRandomByte
        mov byte ptr ,al
        inc esi
        dec ecx
        jnz @B
        invoke WriteFile,hFile,addr shredbuf,sizeof shredbuf,addr BytesWr,NULL
        pop eax
        dec eax
        jnz bufloop1
        mov ecx,my_remainder
        .if ecx==0
            jmp second_round
        .endif
        lea esi,shredbuf
@@:
        invoke GenerateRandomByte
        mov byte ptr ,al
        inc esi
        dec ecx
        jnz @B
        invoke WriteFile,hFile,addr shredbuf,my_remainder,addr BytesWr,NULL 
second_round:             
        invoke SetFilePointer,hFile,0,0,FILE_BEGIN
        mov eax,buffer_times
bufloop2:   
        push eax   
        mov ecx,sizeof shredbuf
        lea esi,shredbuf
@@:       
        invoke GenerateRandomByte
        not eax
        mov byte ptr ,al
        inc esi
        dec ecx
        jnz @B
        invoke WriteFile,hFile,addr shredbuf,sizeof shredbuf,addr BytesWr,NULL
        pop eax
        dec eax
        jnz bufloop2
        mov ecx,my_remainder
        .if ecx==0
            jmp finish_round
        .endif
        lea esi,shredbuf
@@:
        invoke GenerateRandomByte
        not eax
        mov byte ptr ,al
        inc esi
        dec ecx
        jnz @B
        invoke WriteFile,hFile,addr shredbuf,my_remainder,addr BytesWr,NULL       
finish_round:
        pop edx
        dec edx
        jnz times_loop_big
    .endif       
    invoke CloseHandle,hFile
    invoke DeleteFile,file_to_delete
    xor eax,eax
    ret
RemoveFile endp

I had 10kb buffer, but changed to 64.

Any ideas on the flag though?
Posted on 2006-07-05 12:36:27 by white scorpion

However, i am not able to use the no buffer flag since it requires writes which might overwrite other files or not the complete file.

Huh?

All the nobuffer flag requires is that your reads/writes are sector aligned. What this means is that you should write a bit more than the real length of the file, but that's not a problem.

Your new code looks weird.
Posted on 2006-07-05 12:45:40 by f0dder
Your new code looks weird.

thanks ;)
It doesn't really matter that it looks weird, but it works a lot faster (10kb buffer btw, 64kb makes the program crash... it's probably too much to push on stack ;) ).
total time for 40mb file was around 10 seconds with over 400mb's written.

Will take a look at the no buffer story once more.


this should be it.
44MB file takes around 18 seconds to overwrite 10 times with flag no buffer.


RemoveFile proc file_to_delete:DWORD,number_of_times:DWORD
    LOCAL hFile:DWORD
    LOCAL inputfilesize:DWORD
    LOCAL backup:DWORD
    LOCAL BytesWr:DWORD
    LOCAL shredbuf[1024*10]:BYTE
    LOCAL buffer_times:DWORD
    LOCAL my_remainder:DWORD

    cmp number_of_times,0
    jnz @F 
    xor eax,eax
    dec eax
    ret
@@:
    invoke CreateFile,file_to_delete,GENERIC_WRITE, 0, NULL, OPEN_EXISTING,\
    FILE_ATTRIBUTE_NORMAL or FILE_FLAG_NO_BUFFERING, NULL
    .if eax==NULL
@@:
        invoke CloseHandle,hFile
        xor eax,eax
        dec eax
    ret
    .endif 
    mov hFile,eax
    invoke GetFileSize,eax,NULL
    mov ebx,512
    xor edx,edx
    div ebx
    .if edx==0
        mul ebx
    .else
        mul ebx
        add eax,512
    .endif       
    mov inputfilesize,eax
   
;----Rounds Start -----
    invoke GetTickCount
    mov backup,eax
    mov myseed,eax
  .if inputfilesize<=sizeof shredbuf
        mov edx,number_of_times
times_loop_small:
        push edx     
        mov ecx,inputfilesize 
        lea esi,shredbuf     
@@:
        invoke GenerateRandomByte
        mov byte ptr ,al
        inc esi
        dec ecx
        jnz @B
        mov eax,backup
        mov myseed,eax
        invoke SetFilePointer,hFile,0,0,FILE_BEGIN
        invoke WriteFile,hFile,addr shredbuf,inputfilesize,addr BytesWr,NULL
        mov ecx,inputfilesize
        lea esi,shredbuf
@@:
        invoke GenerateRandomByte
        not eax
        mov byte ptr ,al
        inc esi
        dec ecx
        jnz @B
        invoke SetFilePointer,hFile,0,0,FILE_BEGIN
        invoke WriteFile,hFile,addr shredbuf,inputfilesize,addr BytesWr,NULL
        pop edx
        dec edx
        jnz times_loop_small   
;----Rounds End ------- 
    .else
        mov eax,inputfilesize
        mov ebx,sizeof shredbuf
        xor edx,edx
        div ebx
        mov buffer_times,eax
        mov my_remainder,edx
        mov edx,number_of_times
times_loop_big:
        push edx
        invoke SetFilePointer,hFile,0,0,FILE_BEGIN
        mov eax,buffer_times
bufloop1: 
        push eax     
        mov ecx,sizeof shredbuf
        lea esi,shredbuf       
@@:       
        invoke GenerateRandomByte
        mov byte ptr ,al
        inc esi
        dec ecx
        jnz @B
        invoke WriteFile,hFile,addr shredbuf,sizeof shredbuf,addr BytesWr,NULL
        pop eax
        dec eax
        jnz bufloop1
        mov ecx,my_remainder
        .if ecx==0
            jmp second_round
        .endif
        lea esi,shredbuf
@@:
        invoke GenerateRandomByte
        mov byte ptr ,al
        inc esi
        dec ecx
        jnz @B
        invoke WriteFile,hFile,addr shredbuf,my_remainder,addr BytesWr,NULL 
second_round:             
        invoke SetFilePointer,hFile,0,0,FILE_BEGIN
        mov eax,buffer_times
bufloop2:   
        push eax   
        mov ecx,sizeof shredbuf
        lea esi,shredbuf
@@:       
        invoke GenerateRandomByte
        not eax
        mov byte ptr ,al
        inc esi
        dec ecx
        jnz @B
        invoke WriteFile,hFile,addr shredbuf,sizeof shredbuf,addr BytesWr,NULL
        pop eax
        dec eax
        jnz bufloop2
        mov ecx,my_remainder
        .if ecx==0
            jmp finish_round
        .endif
        lea esi,shredbuf
@@:
        invoke GenerateRandomByte
        not eax
        mov byte ptr ,al
        inc esi
        dec ecx
        jnz @B
        invoke WriteFile,hFile,addr shredbuf,my_remainder,addr BytesWr,NULL       
finish_round:
        pop edx
        dec edx
        jnz times_loop_big
    .endif       
    invoke CloseHandle,hFile
    invoke DeleteFile,file_to_delete
    xor eax,eax
    ret
RemoveFile endp


Any other suggestions?
Posted on 2006-07-05 12:52:28 by white scorpion
using above code the average deletion speed will be around 2,4MB's per second.
This is on my 1600mhz system (running at 50% cpu) with my 4200rpm harddisk.
so if i had a faster harddisk it might come up to 5mb's per second.

One final question though: how can i retrieve the sector size on a harddisk?
i now used 512bytes, but i'm not sure if this will be effective on all harddisks....

Posted on 2006-07-05 14:21:03 by white scorpion
White Scorpion,

The DISK_GEOMETRY structure describes the geometry of disk devices and media.

typedef struct _DISK_GEOMETRY { 
  LARGE_INTEGER  Cylinders;
  MEDIA_TYPE  MediaType;
  DWORD  TracksPerCylinder;
  DWORD  SectorsPerTrack;
  DWORD  BytesPerSector;
} DISK_GEOMETRY ;

That is the structure.  Look at DeviceIOControl function to see how to get it.

Paul
Posted on 2006-07-05 18:11:38 by PBrennick
Thanks a lot.
That is exactly what i'm looking for!

Posted on 2006-07-06 01:16:42 by white scorpion