Okay so I finally made my little 32-bit kernel with a simple GDT for code, data and stack, but I am really curious about the structure of memory once the processor is switched to Protected Mode. These are the questions that I have:

1. What will happen to the Interrupt Vector Table?
2. Is there a special place in memory that you should load your kernel?
3. Why should you use 0x1D00 as the stack segment while still in Real Mode?
4. Why should the GDT be saved in 0x0500? Is it Intel?s standard?
5. Can?t we have a bare memory and then organize it how we like?
6. I use LMSW and SMSW to load and store the machine status word in CR0. Is it okay to use CR0 directly while in real-mode after having made sure that the CPU is 32-bit?
7. What happens when all three segment (code, data and stack) descriptors have the base address of 0x00000000 and the limit of 0xFFFFFFFF (Granularity bit = 1). Where in the physical memory are these located? Do they overlap on such conditions? Who am I? What are you doing here?

I?d appreciate it if somebody could give me some pointers related to above questions. Thanks in advance guys.
Posted on 2007-01-17 02:09:50 by XCHG
#1: you set up a new one, with a different format (look at LIDT).

#2: there's all sorts of different strategies. I load mine at just above the 1meg mark, and map it linearly (paging) at the 3GB mark - YMMV.

#3: *shrug* - you just want to have SS:SP somewhere it doesn't interfere with anything.

#4: you can put the GDT wherever you want - 0x0500 sounds like "whatever". You can either use a temporary GDT and build a new one when in pmode, or put the GDT at a fixed spot in realmode and just "extend it" when you get to pmode.

#5: more or less, you'll have to respect the e820 memory map though, if you want to avoid trouble.

#6: these days, you don't really need to check for 32bit CPU - save those bytes in the bootloader for something else. If anybody tries to run something modern on less than a 80386, they really do deserve a crash :). And just use access to CR0 instead of L/SMSW, that's what everybody else does (do keep in mind you can only load/store the CRx registers since they're special, you need a detour via EAX or whatever to modify bits).

#7: you get flat mode, which most operating systems use (though almost always combined with paging to isolate processes from eachother for safety).
Posted on 2007-01-17 02:58:16 by f0dder
f0dder's answers should be enough for you to start happily, but let me dwelve a bit more into the topic:

1. It's no longer Iterrupt Vector Table. It gets changed into a completely new monster (Interrupt Descriptor Table) - it's not only the name that changes. Actually, exception and interrupt handling is one of the most complicated stuff in the whole OS development. Especially if you want to switch into ACPI mode and turn the IO APIC(s) on. Please read a lot about this stuff because the CPU can't really do anything productive without properly configured IDTs, ACPI, LAPICs and IOAPICs. An alternative is to run in the so called 'legacy mode', which is recommended only for testing purposes, because it doesn't support a lot of stuff and is being kept just for compatibility.

2. Wherever you wish, but there are some pros and cons for every place you choose. First 16MB (physical) are special (you'll learn why, when the time comes ;) ), so try to keep them free.

3. o_O ?

4. o_O ?

5. When the computer starts, it starts in 'legacy mode' with all devices configured by the BIOS. Once you reconfigure the CPU, it's time to reconfigure the host bridge, PCI devices, Super IO chip, etc. As f0dder said: Just make sure you don't mess with E820 map, because this map tells you not only how much memory you have, but also marks all unusable places. Once you are in ACPI mode, you can reconfigure a lot of device mappings, so you'll be able to get more continuous memory, which is actually NOT very required thanks to the paging mechanism.

6. cr0 is just a register. If it is present, you can read from and write to it whatever you wish (actually: make sure that you write something useful). Be sure, NOT to alter any 'reserved' bits (use ORing and ANDing). A good practice is to alter only those bits you want to alter and preserve the rest (not just the reserved ones). And you don't have to make sure that the CPU is 32-bit. If someone has a 16-bit CPU, then "they really do deserve a crash :)"

7. They overlap each other. There is no problem with it: EIP points to the instruction being executed, ESP points to the last dword pushed, and all memory references reference this same memory region. Flat mode is really what you want to have. But before you go on and enable the paging, make sure that you play enough with ACPI, LAPICs and IOAPICs. Paging, IMHO, is not very useful without it. If you want your OS easily adaptable to 64-bit, then make sure to use the PAE (this is not to be mistaken with PSE). PAE is not very useful itself, but if your OS is designed to run in PAE mode (additionally: with flat memory and _NO_ hardware task switching), then it is somewhat easier to make it run in 64-bit mode, because 64-bit mode requires PAE, flat memory and doesn't support hardware task switching.
Posted on 2007-01-17 09:50:54 by ti_mo_n

2. Wherever you wish, but there are some pros and cons for every place you choose. First 16MB (physical) are special (you'll learn why, when the time comes ;) ), so try to keep them free.


Is that only important if you're using the legacy DMA controller?  I think that only sound cards still use it, and everything else uses the PCI-configurable DMA controller.  I haven't done any PCI DMA yet, but I got the impression that it would support any 32-bit or even 64-bit physical address.  Are there any good references on how to do PCI stuff?  I haven't been able to find much, and I'd like to get that info soon.

With regard to the rest of the stuff, I put my GDT at physical address 0, each processor's stack is 1KB all just below the MBR load point, my temporary IDT is wherever it gets loaded on booting, my bootloader is loaded right after the MBR, I load my kernel code from a file on NTFS to virtual address 80100000.  Keep in mind that your bootloader is not your kernel; it sounds like you don't have a kernel yet, but keep working at it.  ;)
Posted on 2007-01-17 10:14:48 by hackulous
IMHO it doesn't really matter if you place your kernel in the low 16meg - even considering legacy DMA. It's limited how large DMA buffers you're going to use, and your core kernel hopefully has a limited size as well; probably makes sense to load additional drivers above 16meg, though.
Posted on 2007-01-17 15:12:53 by f0dder
If you are concerned about "bare minimums", then simply make sure your "initial" kernel image can load within the first 256KB of Conventional Memory. Detect the amount of RAM installed and relocate the kernel if desired. Please note that this would be easier if the kernel was in a constant binary format (i.e. PE or ELF) to ease the headache of address "fix-ups" ;)

Personally, I would not change/touch anything in the first 4KB of memory (0000:0000 to 0000:1000) as that is where your IVT and BIOS Data Area is. You might never need to use it, but then again... you just might for some reason.

I have a basic layout of how I achieve 32-bit Protected Mode... it comes from years of fiddling around with the initialization code... but hey... it works well. If anyone wishes to see a general outline of it, let me know :)
Posted on 2007-01-17 16:42:09 by SpooK
Okay let me start by what I am doing with the boot loader and the kernel itself.

1. Set the DS, FS and ES to the code segment and set up my stack.
2. I enable the A20 pin with the on-board and the keyboard microcontroller.
3. I read my data from the second sector in the floppy drive and the kernel which for now is only 2 sectors long and place them at the physical address of 0x00100000.
4. Copy my GDT to 0x0500.
5. Switch to protected mode.
6. Load the Global Descriptor Table.
7. Jump to a label to flush Real Mode instructions and set the new CS:EIP.
8. Set DS, ES, FS and GS to the Data Segment selector in my GDT and the stack segment to another selector.
9. I then jump to my kernel?s code with has only two procedures. One for clearing the screen and another for writing to the video memory at 0x000B8000.


After reading your posts (by the way, thank you all so much) I decided to relocate the kernel to 0x01000000 as to skip the first 16 megabytes of the memory but I guess I am still lost in the whole world of paging, segment overlapping, ACPI, BBC, NBC, CNN, Dr. Phil and etc.

These are my new questions if you guys can still be patient and care enough to help me out with them:

1. What is my Boot-loader?s code segment just when the boot sector is read?
2. If the first 16 megabytes of memory should be passed over and the kernel should be loaded after that boundary, how would I be able to load my GDT? Am I supposed to copy it into the physical address greater than 16 megabytes?
3. I have not set an IDT for the video memory but I have set the GS segment register to its segmented address and yet I can use video memory with the physical address of 0x00B80000. Is this a good way to do this or am I doing something dangerous? Because I got all these ideas by looking at some source codes.

This is going to be the longest virtual journey of my life with paging enabled, probably. Lol. I still am really confused about the physical, virtual, linear and etc addresses so I guess I should get back to Intel manuals.

Once again, I?d like to thank you all for your time and concern.
Posted on 2007-01-18 02:07:32 by XCHG

1. What is my Boot-loader?s code segment just when the boot sector is read?


It could be 0000:7C00 or 07C0:0000 (mind you this is 16-bit segment:offset format), depending on the BIOS manufacturer. Best thing to do is use position independent code (short/near jumps only) or make an initial "far-jump" which will set CS:IP to a known value. Personally, I move my MBR to a different location, jump to it and the rest of the MBR loads the kernel at the standard MBR address (0000:7C00) just to keep some consistency.


2. If the first 16 megabytes of memory should be passed over and the kernel should be loaded after that boundary, how would I be able to load my GDT? Am I supposed to copy it into the physical address greater than 16 megabytes?


First, by "relocated my kernel", do you mean you modified the BIOS loading routine to reflect the address, or do you actually load the data in Real Mode and move the data in Protected/Unreal Mode?

As I said before, I wouldn't lose sleep over using-up the first 256KB of memory.

Do me a huge favor and erase the whole "legacy/16MB" conversation from your head right now. It is insignificant and will hinder your progress before you even come close to approaching the need to mess with legacy architecture and DMA. You have much more to learn before those subjects ;)


3. I have not set an IDT for the video memory but I have set the GS segment register to its segmented address and yet I can use video memory with the physical address of 0x00B80000. Is this a good way to do this or am I doing something dangerous? Because I got all these ideas by looking at some source codes.


You are doing OK, as that is a basic "beginner" route to learning OS Dev (i.e. successfully get to PMODE, stop and show proof.)

Eventually, you will realize that writing to video memory has lost its appeal and you will want to complete things like the reprogramming the PIC along with making an IDT and related Interrupt Handlers.

If you are interested in Paging, you may wish to learn it earlier as it can very well influence everything else that you learn.

Good luck ;)
Posted on 2007-01-18 12:01:03 by SpooK
1. What is my Boot-loader?s code segment just when the boot sector is read?

As SpooK suggested: Just do a jump to "07c0:some_label" and you'll be safe :)

2. If the first 16 megabytes of memory should be passed over and the kernel should be loaded after that boundary, how would I be able to load my GDT? Am I supposed to copy it into the physical address greater than 16 megabytes?

Do me a huge favor and erase the whole "legacy/16MB" conversation from your head right now. It is insignificant and will hinder your progress before you even come close to approaching the need to mess with legacy architecture and DMA. You have much more to learn before those subjects ;)

Looks like SpooK is right. Sorry for giving you too many headaches so early. Just forget about it for now and concentrate on what is important: reconfiguring the CPU :)

3. I have not set an IDT for the video memory but I have set the GS segment register to its segmented address and yet I can use video memory with the physical address of 0x00B80000.

(almost) Every gfx card has a BIOS of its own (you can see gfx card's BIOS starting just before the main BIOS starts when you turn on your computer). This VGA BIOS puts the GFX card into legacy mode (again, sorry for saying this word :P ) with B80000 set as the start of legacy gfx memory. Please run the Windows device manager ["C:\windows\system32\mmc.exe" c:\windows\system32\devmgmt.msc]. Select "View -> Resources by type" (or something similiar - my Windows is not English). Now look under "memory". You can see that A0000-BFFFF is occupied by your GFX card. It means that writing to this area will affect this particular device in some way. Use Windows Device manager when you need some clues about device config because they are quite similiar for every PC. You can also look at the IO memory and see that port 64h is being used by your keyboard. etc. etc. But to the point: You can use B80000 to display some stuff on the screen until you yourself reconfigure the gfx card. B80000 is the default setting for virtually all VGA-type cards and virtually all modern cards are VGA-type.

I still am really confused about the physical, virtual, linear and etc addresses so I guess I should get back to Intel manuals.

Look at the images in this topic. They come from Intel manuals and explain a lot of stuff plainlessly :P
Posted on 2007-01-18 14:29:04 by ti_mo_n
Thank you both for all this information and being tolerant with me. I learned a lot of things from these posts and I really appreciate it. The biggest problem for me is to categorize all this information in my head because as you know, I am coming from the world of ASSE:MBLY and there are still a lot of confusing matters in my head related to the protected mode architecture of CPU.

I read the Intel manuals today and at least figured the difference between paging, segmentation, linear and physical addresses. I also tried to fix some mistakes in my boot-loader code such as a separate segment descriptor that I had made for my stack. I also did a little jump at the beginning and as Spook said, tried not to think about the 16-megabyte boundary.

About the address 0x0500 that I was storing the GDT into, I was doing it to skip over the BDA because I read in OSDEV.org that BDA starts from 0x0400 and ends at 0x04FF. I?m starting to feel like a dummy about this whole Operating System development thing. Is it because I have not even done a simple OS in 16-bit real mode and trying to jump right off to protected mode or is it really because it is hard? Once again, I?d like to appreciate your time for answering my questions.
Posted on 2007-01-19 02:53:22 by XCHG
You will continue to "feel like a dummy", like the rest of us did/do, because you must tackle the odd number of years of backward compatible hardware muck to get anywhere useful and more related to 32-bit programming. The best examples are ones that you have to learn just to use once... and flush it from your memory... like almost everything 16-bit Real Mode.

IMHO, 16-bit Real Mode OS development is pointless and shouldn't be sought unless trying to learn how to use the standard BIOS routines without the interference/hand-holding of DOS and related TSRs.

All-in-all, you are on your way. Just be patient and keep reading ;)
Posted on 2007-01-19 03:03:55 by SpooK
Thank you so much everyone. Spook, thanks for all the information but will you kill me if I ask one more question? I am setting the base address of my code and data segment in the GDT to 0x00000000. Does this mean that these segments are located at the physical address 0x00000000?
Posted on 2007-01-20 01:06:33 by XCHG
Yes, it means that they start at physical address: 0. Now you SET (to 1) thet "G" (granularity) flag, and set their sizes to FFFFFF. This way you get full 4GB segments. This configuration is called "flat memory model" and is what you really want to have :)
Posted on 2007-01-20 09:47:04 by ti_mo_n
Okay thank you so much. Appreciations.
Posted on 2007-01-20 20:52:39 by XCHG
I frequently "feel like a dummy" because I've been at it for about 5 years and all that I have to show my friends is a whole lot of code and an image displayed on the screen, hehe.  :lol:

I remember when I first read something from a harddrive without using the BIOS calls.  It took a lot of work and I was so proud.  Too bad that I don't have much more to show people yet.  Memory and thread management just aren't as visual.  Oh well, once I'm done this really busy term I can get back to development again and then I'm not too far from a first release of the kernel.  Then I can write up those tutorials people have requested too.  :)

By the way, has anyone here ever tried to rewrite their BIOS to switch to protected mode (and stay in protected mode) before loading the MBR and stuff?  I might do it in the fall term for a project in the operating systems course.  (I'm saving the OS for my honours project.)
Posted on 2007-01-21 01:35:53 by hackulous

By the way, has anyone here ever tried to rewrite their BIOS to switch to protected mode (and stay in protected mode) before loading the MBR and stuff?  I might do it in the fall term for a project in the operating systems course.  (I'm saving the OS for my honours project.)


If you mean fixing up the 16-bit code to be 32-bit compatible, I've started on something similar but stopped quite some time ago. I decided to be less dependent on the BIOS and didn't continue any further.

It is feasible though, you would just be limited in a similar fashion to that of DMA. 
Posted on 2007-01-21 03:41:44 by SpooK
Perhaps take a look at http://www.linuxbios.org ...
Posted on 2007-01-21 04:59:37 by f0dder

Perhaps take a look at http://www.linuxbios.org ...



By the time they try to support *every* motherboard/chipset, their BIOS will be another glorious example of generic bloat.

I think the motherboard BIOS manufacturers know what they are doing and there is a lot of motherboard-specific programming involved... so I'd rather depend on them than some generic Linux-based BIOS ;)
Posted on 2007-01-21 14:17:09 by SpooK


Perhaps take a look at http://www.linuxbios.org ...

By the time they try to support *every* motherboard/chipset, their BIOS will be another glorious example of generic bloat.

Thanks f0dder.  SpooK makes a good point though, so maybe I shouldn't bother and just wait until EFI becomes standard.
Posted on 2007-01-21 15:05:59 by hackulous



Perhaps take a look at http://www.linuxbios.org ...

By the time they try to support *every* motherboard/chipset, their BIOS will be another glorious example of generic bloat.

Thanks f0dder.  SpooK makes a good point though, so maybe I shouldn't bother and just wait until EFI becomes standard.


I wouldn't invest too much in EFI either. It is not advancing as fast as originally expected. BIOS will be around for a long time ;)
Posted on 2007-01-21 18:20:26 by SpooK