Since I am the happy owner of a AMD 64 that supports this feature in SP2, I have data-execution prevention turned on for system processes. What I'd like to know is how this technology is implemented, and specifically if it possible for it to give false positives.
The reason I'm asking is this: I recently downloaded a .avi file from a file-sharing network, and whenever I highlight it in windows Explorer (not opening it), a "Data Execution Prevention" box pops up. I have been keeping up with patches. Does this mean I've stumbled upon a trojan that exploits a new security hole?
I'd send the file to some other people, but I don't think the MPAA would approve. So let's not talk about that here :)
Posted on 2005-02-27 09:56:05 by Qweerdy
It doesn't have 'false positives' as such, but some applications do things which are not permitted, which basically is executing code from the heap (several anti-cracking products do this, of course).

Usually, programs that do this will use VirtualAlloc() to alloc the memory, and VirtualProtect() to enable executable access, in order to be compatible with DEP. If they are just calling malloc or HeapAlloc, decrypting some data, and then jumping to it, dep will pop up a warning like it did for you.

MSDN has some information on DEP as well...

Since I am the happy owner of a AMD 64 that supports this feature in SP2, I have data-execution prevention turned on for system processes. What I'd like to know is how this technology is implemented, and specifically if it possible for it to give false positives.
The reason I'm asking is this: I recently downloaded a .avi file from a file-sharing network, and whenever I highlight it in windows Explorer (not opening it), a "Data Execution Prevention" box pops up. I have been keeping up with patches. Does this mean I've stumbled upon a trojan that exploits a new security hole?
I'd send the file to some other people, but I don't think the MPAA would approve. So let's not talk about that here :)
Posted on 2005-02-27 10:34:54 by tiocsti
It could be that the .avi is corrupt, containing some invalid header values causing media player or whatever to end up trying to execute from the stack.
Posted on 2005-02-27 12:13:31 by f0dder
Dosen't C++ put a lot of code in the heap? This could break a lot of applications. Maybe it would be better if only memory used by a stack wasn't executable, there are still heap overflows but I think these are not as common.
Posted on 2005-02-27 18:30:14 by QuantumMatrix1024

Dosen't C++ put a lot of code in the heap?

There's nothing in C++ itself that does this, and I don't know of any implementation that does it. Heap memory is used for storing object member data when you're using dynamic objects (ie, new/delete rather than a local heap object), but this is not code.

Some programs do use the heap for code, most often software protection or JITting for scripting. This software should be rewritten to use VirtualAlloc with the proper flags - while x86 has only recently gotten the NX bit, the win32 API has had the required support flags for ages.
Posted on 2005-02-27 18:37:06 by f0dder
I was under the impresion that member functions were inlined into each instance of an object. A little research shows me this is possible but not default behavior.
Posted on 2005-02-27 20:09:27 by QuantumMatrix1024
I was under the impresion that member functions were inlined into each instance of an object. A little research shows me this is possible but not default behavior.

I never saw an implementation that does this, and it doesn't make sense anyway.
Posted on 2005-02-28 03:44:53 by death
No, it doesn't make sense.

Member functions are ordinary functions at the machine code level. They can either be shared (single instance) or inlined (like macros). Inlining occurs at the call point, not in the data space of an object.
Posted on 2005-03-01 02:28:45 by tenkey
Each time we access a class variable from a member function the this pointer is used. If the function code was part of the object we would have direct access to variables so it would be a an optimization. This would give a slight speed increase for a massive size increase.
Posted on 2005-03-01 11:02:18 by QuantumMatrix1024
Each time we access a class variable from a member function the this pointer is used. If the function code was part of the object we would have direct access to variables so it would be a an optimization. This would give a slight speed increase for a massive size increase.

In most implementations you do have "direct access". For example, using MS's thiscall calling convention, ECX contains the 'this' pointer, so you access a member variable using .

Your approach is total nonsense. The size of an object is determined using the class definition, which doesn't necessarily include the member function definitions.
Posted on 2005-03-01 13:00:04 by death