how does upx work? i can't figure out how it can compress the bitmaps associated with my .exe so that they are still understandable without any special coding by me... is it just a general compression on the bitmaps such that they are still understandable anyway? ( i.e. they were bloated in the first place ) or is it something more complex ....
Posted on 2003-04-27 03:21:13 by abc123
From my "research" I found out that upx compress the .rsrc, thus of course your bitmap would be compressed. It seemed that the decrypting routine is found in the section upx02. Maybe as what qweerty suggest, they may be hooking the functions that call the resources such as loadresource, findresource or maybe they overwrite the .rsrc when loaded into memory. Too bad there is no source code of upx to wade through, thus I seriosuly have no idea how they decrypt the resources, just some wild guesses.
Posted on 2003-04-27 03:47:48 by roticv
abc123,

There are a couple of things to consider when you mention an EXE compressor like UPX and others. There is the PE file manipulation to make it possible and the actual compression algorithm that does the compression.

With the executable file that is compressed, most systems contain a small stub that uncompresses it as it runs.

Its something like this in sequence. The compressed PE file must be able to start so it must have enough code for the operating system loader to place it in memory. Once the control is placed with the EXE file, it executes the stub decompressor that expands the file in memory into a running form that will execute.

If you can get an answer out of him, the guy I know who has the expertise in compression algorithms is "Jibz" who has been writing successful compression software for years.

You can get the info about PE file structures from a couple of documents, an Intel one and a later Microsoft one and there is a simplified introduction by Luvelmeyer that is helpful as well.

Regards,

hutch@movsd.com
Posted on 2003-04-27 04:20:57 by hutch--
Hi Hutch, :)

I have a question:There are some articles considering that exe compressing is bad.
What's your opinion?
Posted on 2003-04-27 04:48:33 by Vortex
Vortex,

It has a lot to do with what the application or in particular the DLL is going to be used for. With something like a web browser where youy usually only use one instance it has the advantage of smaller size and usually loads faster but you can have a down side as well, with a shared DLL that is compressed, it must load a new copy every time which is not efficient in terms of memory.

For example you would never compress a system DLL as your memory usage would end up very high but at the other end, if you manly write small apps where the complete memory image is not that big anyway, you have faster loading and a full in memory image so there is no paging within the application.

I rarely write anything large these days so I routinely compress EXE files and most of the DLLs I write are only loaded on demand so there is no problem with those either. What you keep in mind is if you are writing a large DLL that will be called by many other applcations at the same time, you probably should not compress it as it will increase your memory usage.

A factor that may reduce this as a problem is that not all apps can load a DLL at the preferred address because of other DLLs that are loaded first so in this case which is common enough, there is no great loss in the DLL being compressed.

Regards,

hutch@movsd.com
Posted on 2003-04-27 07:21:40 by hutch--

From my "research" I found out that upx compress the .rsrc, thus of course your bitmap would be compressed. It seemed that the decrypting routine is found in the section upx02. Maybe as what qweerty suggest, they may be hooking the functions that call the resources such as loadresource, findresource or maybe they overwrite the .rsrc when loaded into memory. Too bad there is no source code of upx to wade through, thus I seriosuly have no idea how they decrypt the resources, just some wild guesses.

http://upx.sourceforge.net/download/upx-1.24-src.tar.gz
Posted on 2003-04-27 08:09:17 by Tola
Hutch,

Thanks for the infos :alright:
Posted on 2003-04-28 03:30:35 by Vortex
A web browser will typically run multiple instances - IE has a smart optimization, though, where it appears like multiple instances but only launch a new thread for each open window. Iirc, this can be disabled (since it means that one crashing IE window brings down all the other windows).

Multiple instances is not the only thing to consider, though.
If you run an app through an EXE compressor, it means that the decompression has to be done on each program load. This is obvious. An uncompressed app has the advantage that a second load will require very little work by the OS. Code pages and data pages that have not been modified are very likely to already be in cache.

Even more important is the event where windows has to swap out (parts of) your application in a low-memory situation. With compressed executables, all pages are "dirty". Thus windows must write out your entire application to swap space, and load it back in when your program becomes active. With an uncompressed application, windows can simply discard "non-dirty" pages, and read them back from the original executable when they're needed.

Needless to say, exe compression has it's places. But it's a bad think to do generically without understanding the implications ("Gee whiz, this makes my exe smaller, it must be good")

As for resource handling (UPX source is a mess to read ;-)), they require a bit more handling than the rest. You must have a valid resource tree in the compressed executable, and it is all pretty messy.
Posted on 2003-04-28 03:50:39 by f0dder

("Gee whiz, this makes my exe smaller, it must be good")


oh.. i'm guilty of that :)

i was under the impression that it cleaned out my .exe rather than actually compressing ...

thanks for the info guys ...
Posted on 2003-04-28 05:49:26 by abc123