i have a question :P where is the stack actually located? in the datasegment? also what about the heap? sorry if this is a dumb question
Posted on 2010-02-24 19:20:38 by maybnxtseasn
The stack is a contiguous block of memory which is allocated for each thread in your process (when a thread is created).
It's a fixed size, determined when we assembled the executable, usually 8kb per thread (I think), and if we ever use all of it up, it will 'wrap' (like a circular buffer), overwriting potentially valid data (buffer overflow) and usually leading to a crash.

The heap is a little different - each process is allocated a Heap ('process heap') but can create more heaps.
Unlike stacks, heaps can grow on demand... think of them as a 'pile of memory chunks that we allocated' (actually stored internally as a linked list).
When we call HeapAlloc, the operating system 'heap manager' will attempt to allocate a chunk of memory on whichever heap we want - this memory may not be contiguous, and in fact usually is not... the result is that heap memory is often 'fragmented' - and another api exists that we can try to 'defragment' a heap, should it become a problem.

I hope that description helped you get a better idea of what they are.
As for where they are located, well they are located in general system memory, and as usual, their addresses have been 'mapped' by the operating system as 'virtual address space' - not actual physical addresses - so two different processes can appear to be sharing the same memory addresses, but in fact are not.
Posted on 2010-02-24 22:18:55 by Homer

i have a question :P where is the stack actually located? in the datasegment? also what about the heap? sorry if this is a dumb question


I suppose you are talking about 16-bit mode? In 32-bit and 64-bit mode you use flat addressing, so technically all segments are the same, the segment registers are 'descriptors' instead, mainly affecting the access priviliges of the memory (eg stack is not executable memory, code is read-only).
In 16-bit mode, you have a separate stack segment, ss.
The heap depends on the OS used, but in general your heap will be larger than a single segment, so when you allocate some memory, you will get a long pointer, containing both the segment and the offset.

In certain cases (eg a .com executable in DOS), multiple segments can overlap. So it is possible that the stack is in the data segment, then ss is just set to the same value as ds.
Posted on 2010-02-25 02:16:47 by Scali
This is misleading, but ok. I won't take this one up. It's at least as correct as my previous post, heh.

Posted on 2010-02-25 04:26:48 by Homer

... and if we ever use all of it up, it will 'wrap' (like a circular buffer), overwriting potentially valid data (buffer overflow) and usually leading to a crash.


not so sure about that m8, the stack top and bottom are stored in fs:[4] and fs:[8]
once the limit is reached it creates an exception, where (if the system handles it) the stack is 'grown', otherwise a stack exception occours usually resulting in termination
Posted on 2010-02-25 04:40:09 by evlncrn8


... and if we ever use all of it up, it will 'wrap' (like a circular buffer), overwriting potentially valid data (buffer overflow) and usually leading to a crash.


not so sure about that m8, the stack top and bottom are stored in fs:[4] and fs:[8]
once the limit is reached it creates an exception, where (if the system handles it) the stack is 'grown', otherwise a stack exception occours usually resulting in termination


I think we should discern two possible scenario's here:
1) The stack runs out of the currently mapped/allocated region.
2) The stack has grown to its maximum allowed size.

In a virtual memory system, the OS will reserve a certain address range for the stack, but it might not actually commit physical memory to it. This is what Windows does, for example.
The first uncommited page is used as a 'guard page'. If you reach that page, an exception is triggered, and Windows will map a new page (or set of pages) of memory for the stack.
This can cause a problem when you want to use large arrays on stack. The array may be so large that it stretches beyond the guard page area. And since stack normally grows from high address to low address, but arrays are accessed from low-to-high instead, as soon as you access the first element in the array, your application may crash (touching elements back-to-front at every page interval would avoid this scenario, because it allows Windows to grow the stack).

Once the stack has reached a predefined maximum, the application will also crash. The initial stack size (commit) and maximum allowed stack (reserve) are specified in the PE header in Windows, and you can set these values at link time (or use editbin to modify an existing exe).
Posted on 2010-02-25 06:56:37 by Scali


... and if we ever use all of it up, it will 'wrap' (like a circular buffer), overwriting potentially valid data (buffer overflow) and usually leading to a crash.

not so sure about that m8, the stack top and bottom are stored in fs:[4] and fs:[8]
once the limit is reached it creates an exception, where (if the system handles it) the stack is 'grown', otherwise a stack exception occours usually resulting in termination
Not entirely correct.

Those FS:xx pointers are the "safe region for stack" - if ESP goes outside the range defined there, your program will be silently & forcefully terminated. Automatic stack growing does indeed work like Scali describes.


This is misleading, but ok. I won't take this one up. It's at least as correct as my previous post, heh.
What is misleading about Scalis post? Only things I see is that
1) stack no-execute is done via pagetable, not selector/descriptor
2) 32/64bit modes do have SS selector as well, but (on "normal" OSes) it's the same as the data selector.
Posted on 2010-02-25 07:13:52 by f0dder

What is misleading about Scalis post? Only things I see is that
1) stack no-execute is done via pagetable, not selector/descriptor
2) 32/64bit modes do have SS selector as well, but (on "normal" OSes) it's the same as the data selector.


Mea culpa... I just meant to express that the segment registers work slightly differently in protected mode than in real mode. Real mode doesn't have a concept of paging or access privileges.
In 32/64-bit mode, it's all 'one flat pool of memory', but the OS divides it up into different sections with different privileges and behaviour.
Also, I didn't say there is no ss selector in 32/64-bit... I just said that in 16-bit, ss points directly to your stack segment.
Since 32/64-bit mode are flat rather than segmented... well, see above.
Posted on 2010-02-25 07:50:31 by Scali
Also, I didn't say there is no ss selector in 32/64-bit... I just said that in 16-bit, ss points directly to your stack segment.
Yup, I just clarified it since your post could be misread :)
Posted on 2010-02-25 07:54:03 by f0dder

Also, I didn't say there is no ss selector in 32/64-bit... I just said that in 16-bit, ss points directly to your stack segment.
Yup, I just clarified it since your post could be misread :)


Anything can be misread, it mostly depends on how hard one tries...
Posted on 2010-02-25 08:19:09 by Scali

I suppose you are talking about 16-bit mode? In 32-bit and 64-bit mode you use flat addressing, so technically all segments are the same, the segment registers are 'descriptors' instead, mainly affecting the access priviliges of the memory (eg stack is not executable memory, code is read-only).


Hi, I'd like to use this thread to clarify myself the concept of segments. Say I crank up a remote kernel debugger and connect to Win XP. I dump some segment descriptors and the segment register:

kd> dg 0 25
                                  P Si Gr Pr Lo
Sel    Base    Limit    Type    l ze an es ng Flags
---- -------- -------- ---------- - -- -- -- -- --------
0000 00000000 00000000 <Reserved> 0 Nb By Np Nl 00000000
0008 00000000 ffffffff Code RE Ac 0 Bg Pg P  Nl 00000c9b
0010 00000000 ffffffff Data RW Ac 0 Bg Pg P  Nl 00000c93
0018 00000000 ffffffff Code RE Ac 3 Bg Pg P  Nl 00000cfb
0020 00000000 ffffffff Data RW Ac 3 Bg Pg P  Nl 00000cf3
kd> r cs
cs=00000008


Okay, segment register points to the ring 0 code segment. But then I open a user-mode debugger on the debuggee and attach to calc.exe:

0:001> r cs
cs=0000001b
0:001> .formats 0000001b
Evaluate expression:
  Hex:    0000001b
  Decimal: 27
  Octal:  00000000033
  Binary:  00000000 00000000 00000000 00011011
  Chars:  ....
  Time:    Thu Jan 01 02:00:27 1970
  Float:  low 3.78351e-044 high 0
  Double:  1.33398e-322


The cs=0000001b is still a segment selector? If I dump all the selectors in GDT, I can't find a direct match for 1b. Is it just because of different RPL?

And while on the topic of segments, I'd also like to ask about far calls. Previously I thought they operate on segment selector and IP offset. But say I observe the following far call in Windbg:

ff155011807c call    dword ptr


Should this still compose a segment selector that is then resolved to PDE, PTE etc?

ps. I just registered but I've read the board a little and you guys definitely have the knowledge to answer my newbie question.
Posted on 2010-03-04 02:30:31 by qpok
qpok: you mask off the RPL from the selector before looking up the descriptor :)

As for FF15 it's not a far but a near call, so no selector involved. You don't often see far pointers when dealing with a flat address space.
Posted on 2010-03-04 02:38:18 by f0dder
f0dder,

cs & 3 is CPL, not RPL. And there is TI to mask off too (though LDT is rara avis to see one). ;-)
Posted on 2010-03-04 12:57:03 by baldr
Thanks for the answers. I knew about the TI and CPL (well, I thought it was RPL) but apparently miscalculated the selector when trying to figure out it in my head. But a follow-up question: Am I to find any far calls when disassembling the Windows native api or user-mode components? ANd please bear with my simple questions }:]
Posted on 2010-03-04 14:53:26 by qpok
qpok,

CPL is RPL too when you're trying to access data via cs: segment override prefix.

About far calls: there is no hard evidence of their usage, GDT doesn't contain call gates; yet there is one conforming DPL-0 readable 16-bit code segment that appears to be mapped to BIOS (I'm talking about my XP SP2 installation; Sven Schreiber's w2k_mem tool is invaluable, his "Undocumented Windows 2000 Secrets: A Programmers Cookbook" too).

Probably a program that searches %SystemRoot%\system32 for PEs with direct/indirect far jmp/call in code sections will shed some light on this subject.
Posted on 2010-03-05 01:18:37 by baldr
id also like to add on to this thread :P
my question is if the stack is LIFO ...isn't it possible to use ESP+offset to get to certain places within the stack? is the LIFO concept just for the computer...we have the control around this unlike the cpu does?
Posted on 2010-03-06 20:51:51 by dougfunny
Correct, Correct, Correct.
Posted on 2010-03-06 21:31:11 by Homer