So now I'm wondering what compression algo to use in my Installer Maker plug-in. I tried aPLib 0.42, but in some cases it gives me negative compression - yesterday I played with a bunch of 20 files with size about 241 MBs. On the other hand in most of the other cases it gives excellent ratios. I know that the version of LZMA implemented in 7zip is the best compression I've ever seen, so I'm wondering if anybody made an assembler implementation of it? Other ideas for other compression algos are welcome - indeed this is the main aim of this thread - some kind of poll on compression algos :) So, I'm waiting for your pros and cons and your creative ideas!

Posted on 2004-12-16 01:57:33 by siddhartha
You don't need an assembly implementation in order to be able to use it - a static library written in C will be just fine. aPLib is C code too, except for the asm decompressors.
Posted on 2004-12-16 07:01:58 by f0dder
hi siddhartha,

You may give a try to use Jeremy Collake's compression library. Check this SFX archive demo.
Posted on 2004-12-16 12:27:20 by Vortex
I really didn't know that aPLib is written in C, f0dder. Interesting information. I thought that if I had a library written in assembler, it would be easier for me to tweak its code as/if needed. I'll try LZMA from 7zip then. I suppose that the only thing that I'll have to do is to create my own .inc for it, but that's not a disaster :)

Vortex, I'll check this out!

Posted on 2004-12-20 05:13:24 by siddhartha
Jeremy Collake's JCALG uses the same encoding as aPLib, so you could view it as an "assembly implementation" of aPLib... the encoder is somewhat different, though.

Anyway, you'll probably want to use LZMA since you're writing an installer - it's one of the best general-purpose algorithms size-wise. The purpose of aPLib and JCALG is to have a simple encoding, reasonably fast compression (but somewhat sub-optimal, compared to "heavier" algorithms, compression ratio), and *fast* decompression (and small decompressor size).

So for an installer, I'd choose LZMA, while for an exe-protection system with compression, I'd choose aPLib.
Posted on 2004-12-20 07:40:17 by f0dder
OK. I've tested jcalg and aPLib - i think aPLib outperforms jcalg in most cases. I downloaded the LZMA SDK, but as I know nothing of C/C++ I can't figure out for myself how to compile a library from these sources. Anyone with knowledge in this area is welcome with his ideas. Merry Christmas!

Posted on 2004-12-24 19:18:58 by siddhartha
siddhartha, Reason Refills are heavily compressed + heavily encrypted files, so do not wonder why the best algorithms will produce negative compression - it's just like if you zip a file several times: :)
Posted on 2004-12-25 05:04:21 by Ultrano
Yeah, I know that, but I wonder if it wouldn't be easier if the compressor looks at the data it is going to compress and after seeing it can't be compressed - just leave it as it was. It's crazy thing... Any ideas about compiling LZMA to a .lib?

Posted on 2004-12-26 22:56:50 by siddhartha
Siddharta, if compressed size >= original size, indeed leave as-is. The unpacker can check if rawsize==packedsize, and if so, copy directly. You might even want to leave uncompressed if "only a few bytes are gained" if the compression routine is expensive.

I'm at my gf's place right now, but I can have a look at compiling to .lib for you in a couple of days :)
Posted on 2004-12-27 11:03:13 by f0dder
f0dder, I'll be very greatful if you can do this for me. And I'll be even more greatful if you tell me how you did it, because I don't have the time for reading and I want to know :)

Posted on 2004-12-29 00:55:23 by siddhartha
siddharta, I've taken a cursory look at the LZMA SDK. It looks pretty good and flexible, but this comes at the price of a bit of complexity. I'll have to write some C wrapper code around C++ objects to make it (easily) callable from assembly... will take some hours of work to design and implement. I should have the time to do so a couple of days after new years eve. I'll probably want to play around with LZMA myself, so I should definitely get the work done :)
Posted on 2004-12-29 06:50:58 by f0dder
Thanks for the patience and the great help, f0dder! I greatly appreciate it!

Posted on 2004-12-29 08:57:04 by siddhartha
I personally would also be interested in your results, if it can be shared.

If not, oh well :-/

PM me if you need any special information.

Posted on 2005-01-01 01:14:37 by NaN
OK. I'll test more accurately these days and post a comparison table her. Now I'm in hurry and don't have time :)

Posted on 2005-01-02 01:09:47 by siddhartha
Sorry this has taken so long - been having fun out in that weird thing called "real life" :). I'm looking further at the LZMA SDK right now and boy is it messy! It looks fairly trivial to use the decoder, but the encoder is some of the nastier C++ code I've seen.

There's good ideas - like not allocating memory itself and requiring you to pass it a memory block, using "streams" instead of direct file access etc... all this allows better portability and makes it more flexible to use.

It's the (lack of) documentation, weird directory structure, and messy source that makes it weird. I'm going to spend some time and see if I can extract the necessary parts from the soup of code :)
Posted on 2005-01-08 14:52:14 by f0dder

It's the (lack of) documentation, weird directory structure, and messy source that makes it weird. I'm going to spend some time and see if I can extract the necessary parts from the soup of code :)

I have to agree with you - after looking a couple of hours at the SDK and trying to decypher the logic of its directory structure I gave up. Of course, my lack of experience with C/C++ helped me a lot for my desicion to stop gazing at it :) I hope you'll be able to work it out soon, while in the meantime I'm wholly recoding my Installer Maker add-in for RadASM. Oops, I almost forgot - does anybody know why my add-in does not react when pressing tab? I've enabled Tab stop for all edits, but it won't do it. I just don't know why...

Posted on 2005-01-13 01:42:34 by siddhartha