to f0dder

I am not aiming at speed issues here, as you might notice in SOLAR OS source code i am curently rdrawing screen every frame :)

I aim at simple, i aim at my own way/concepts i dislike walking other paths instead of my own...esp when their path is determined by comercialism and wrong technical assumptions

I do not see why i should check every jump/syscall i make ... IMHO they are :
1) Wrong and then the program needs to be reworked
2) OK and then the check is stupid and done every time for NOTHING ..

Come on maybe first time to see if it is ok but doing it agin and again is like me redrawing screen everytime :P at least i have a reason i am lazy :P

I will not trash cache at task switch because i use cooperative multitasking... works perfect
I will look into that "carefull task scheduler" when the time comes for preemtive multitasking... i am just happy as it is now

_bugs_always show up? hehehe i know

Maybe people should be more carefull about OOP design and C++ and huge teams?
Maybe they should release source code so more people take a look at it and many eyes will find the bug faster.

Or mybe the will just have to deal with it them selfes... if they want closed propietary software.

... ring0 dangerous :)) hahahaha make me laough today...

You mean that all those applications i use to run on old DOS days like 3DStudio, Wordperfet, Ventura etc etc that owned the whole memory like in ring0 .... where all dangerous? ... what did they do? make fun of me?

I do not see how ring0 is dangerouse... really ... if done properly i mean :)

Design the API decently...OK

But to reduce API calls...hmmm?

Is that not the ideea of an API functions to be usefull and to be called many many times... at least more than once in a while?

Well i know my OS is stable... I can let it run for centurys and it will not GPF or need a restart :)
(yes i know i do not have many applications now) but we will see in the future...

Besides why sould a OS be "unstable" wtf can happen?


It's nice that you're playing around and all, but if you think you're building an "OS" you're delusional - this is at best a toy kernel with some pretty wrong assumptions.


My friend... i carefylly considered ALL assumptions with PROs and CONS and in fully understanding them and the consequences I HAVE CHOOSEEN to be diferent and go my own way .

I consider OTHERS are wrong but i will fight and die for their right to be so :) trust me

I just want to implement my own thinking about an OS and i also stated that I will use time to implement OTHERS points of view also...so we cand have a FAIR set of OPTIONS and test acordingly

This is indeed my own OS and you can call it "a toy"...

but an OS is just a set of base software functions that interfaces you with the hardarw and offers you a way to design applications for that computer ...

I have interfaces for :
-mouse,keyboard,memory,kernel,State Machines,GUI,Events,FDD,HDD... and you can do applications in my OS... it is not yet finished and there is much work to be done indeed

but i still like to think of it as on OS ... thank you
Posted on 2003-04-28 15:19:28 by BogdanOntanu

Paging makes memory mapping (files, IO) possible.

Paging, because of fixed size and native CPU support, is easy to handle when doing virtual memory / OS paging.

Of course bogdan will claim that VM is bad, that apps should handle VM themselves if they need it, that all the protection is slow and unnecessary etc.

Why mapp memory out of RAM, I can't see the use, using eg a file on a HDD as RAM can't be an titz-ripper. Isn't it better to use RAM as RAM and HDD as storage memory not working memory?
I'm not sure about the virtual memory thing, have I gotten it wrong if virtual memory can be memory that doesn't exist, which would enable usage of memory that you don't have? (I don't like the idea of living/using resurces one doesn't have)

My definition of an OS is basically "late" at school day night hours:
Operating System, deals with the basic stuff like supplying memory manager, task swithing, basic gfx calls, hardware interface functions (like BeepSysSummer, just beep the F* sys speaker for me, or PrintText print the text at "this loaction" to "that IO device" (may it be a printer, a terminal, the screen or whatever). The kernel is just the core that glues it all together, and keeps the weels spinning.

(I think I better get some sleep before school starts, I'll check the forum tomorrow, so god night friends (or be we foes after these posts ;)))
Posted on 2003-04-28 15:35:01 by scientica
Yes fodder VirtualMemory is bad...

It is alzy solution and trust me i use it a lot because i am lazy...esp in comercial applications i wrote...
But i dosent mean i have to also THINK is is good. I know it is bad.

Basicaly an application that needs huge ammounts ot memory should be aware of just home many RAM is there available on a machine and act acordingly ...yes eventualy use HDD for a private swap if it needs to...

To let the OS guess your needs for you is convenient and lazy...wastes time eh... etc etc

But so very much used today... because it means you do less work

Indeed the "big " advantages of pagins over segmentation is the auto VM feature of x86 CPU that work only with paging.

That is because when a page is not actually present in memory...lets say at adress 2.5Gigabytes
(anybody has 3Gigabytes here ? i guess yes)...

Well IF you do not have 2.5G RAM ... then the CPU will generate a "page fault" exception... the handler to this exception could just download that page (and its next few in set) to RAM from HDD and of course send some other pages to safe storage on HDD to make a few free space...

Imagine what happens IF the application that just loosed "a few" pages wants them back at the next task switch and if the 2-3 applications that swap eachother will actually also need eachothers to function properly... :) i have seen that huge HDD activity from nowhere on Linux and Windows so many times.... :(

Well the big advantage is that AFTER the exception code the program that generated it can be RESTARTED just where it was interupted and it will just belive it has the needed memory...


Segmentation also make it possible for many applications to "belive" that they are actually running at address 0x40000000 when they are in fact at diferent physical locations in RAM... but it does not offer a simple convenient...

There are way to many tings abut paging and segmentation to be explained here but maybe we sould...hehe

Like TLB, and the fact that paging takes up huge ammounts of system memory to keep track of every 4K page in status OS RAM... also the sad gfact that segmentation is indeed limited to 8192 segments...and each program needs at leas 2 of them (code and data) but maybe more
...

Well novices... feel free to ask me anytime ...
Posted on 2003-04-28 15:37:32 by BogdanOntanu
BTW f0dder,

There are not some many tons of ring0 <--> ring3 switches in modern OS like Win9x and WinNT/2K/XP as one might think. (yes WinNT/2K also)

They knowed it is going to be slow so they use a trick: Many many components of the OS (vital ones like the GDI and kernel parts for example) are runned like simple ring3 applications ...this helps reduce the "tons" of ring switches... and compromites the system stblility :(...

Because the OS might work ... but with a damaged GDI or memory alocator wtf can one still use it for? maybe conect from another computer via terminal and restart it :D (aka Linux style)

There would not be any need of copy on write if those important system components would have been protected via "protection" in ring0 ... but it would be awfull slow...hmmm at least they think so...

I am not so sure myself, i will check it out when i implement it myself
Posted on 2003-04-28 15:52:54 by BogdanOntanu
ok, reducing the harshness a bit (sorry).


I do not see why i should check every jump/syscall i make ... IMHO they are :
1) Wrong and then the program needs to be reworked
2) OK and then the check is stupid and done every time for NOTHING ..

Of course this depends on what you're aiming at. For a "real OS", you *need* checking - lots of people are trying to be malicious and you want to stop them. Even on your own OS, which won't be targetted, it's still nice not having too simple software faults resulting in reboots or kernel panics. Even with extensive testing, you wont catch every bug. If you know of a way to get 100% test coverage, let me know.


I will not trash cache at task switch because i use cooperative multitasking... works perfect
I will look into that "carefull task scheduler" when the time comes for preemtive multitasking... i am just happy as it is now

Cache trashing will only happen once you actually switch task. (Sure, scheduler will use some cache, but so will all IRQs. It's your responsible to program those very low-level aspects well). With a decent algorithm for the scheduler, cache trashing effects shouldn't be too bad.

As for the DOS days... I'm glad we're past those days. Single tasking, yuck. A lot of other bad things not related to the current thread. True, you didn't see crashing applications that much back then. True, applications have grown too huge, and there's a lot of careless developers out there. I also remember badly programmed software from back then, though. And being annoyed at rebooting. And I remember cooperative multitasking fron win3.x and being annoyed how one bad app could make the entire system run sluggish (at best) or even worse, lock it up. I also remember software development in unprotected environments, and weeh am I happy about ring3, paging, separated process spaces, etc.


Besides why sould a OS be "unstable" wtf can happen?

Your kernel is probably "stable" enough - but the whole environment widely open to be trashed by any app that has a little flaw here or there.


This is indeed my own OS and you can call it "a toy"...

Nah, it's a kernel - a OS is a kernel + "all those extra things".

Well, not much point in discussing most of these things, they're higly subjective.

scientica,

Why mapp memory out of RAM, I can't see the use, using eg a file on a HDD as RAM can't be an titz-ripper. Isn't it better to use RAM as RAM and HDD as storage memory not working memory?

memory mapped files are a convenient way of handling file access in some situations. It's generally slower than a block based approach, but in a lot of situations the ease of use means more than the speed. I wouldn't use it for stuff like file copy or file checksumming or anything similar requiring hi-speed though (well, not the implementations in the environments I work with - on some architectures it's probably possible to implement very well-working MMF).


I'm not sure about the virtual memory thing, have I gotten it wrong if virtual memory can be memory that doesn't exist, which would enable usage of memory that you don't have? (I don't like the idea of living/using resurces one doesn't have)

Tjaaah, it's somewhat vague. When done correctly, you dont "live off resources you don't have" - the swap to disk aspect handles situations where something goes wrong, situations where you have a temporary need for a large amount of memory, etc. Basically most of the time you won't be using more memory than what is physically available, and when you do - you wont run into trouble.

There's other parts handled by paging that I see as part of VM too. Virtual Memory space... keep processes separated from eachother (no accidental overwrites), keep memory fragmentation localized to processeses (because linear memory is mapped to physical, memory fragmentation is much less of an issue). Being able to remap physical addresses (mapping linear framebuffers, device buffers, . . .).


Basicaly an application that needs huge ammounts ot memory should be aware of just home many RAM is there available on a machine and act acordingly ...yes eventualy use HDD for a private swap if it needs to...

Nobody says you can't do that in a virtual memory OS. For some/many applications, using the OS defined disk based memory will be enough (especially if enough care is taken to design this, of course). In other situations, you might need to do your own virtual memory thingamajig (audio editors are a good example of this), and you do. Applications with very high memory demands tend to be run without much else going on in the system, so they should be able to query available memory and work from those assumptions (ie, there wont be another app grapping a huge chunk of memory). And _if_ that happens, well, it's the users fault, and he pays the penalty of possible disk swapping. And you avoid out of memory crashes.


Segmentation also make it possible for many applications to "belive" that they are actually running at address 0x40000000 when they are in fact at diferent physical locations in RAM... but it does not offer a simple convenient...

Apart from the protection features of separate process spaces, it offers the convenience that relocs do not have to be applied, and thus you can page in the executable directly from disk without (m)any further fixes. Some of the same can be achieved with segmentation, but I dislike segmentation.

Hum, wasteful keeping track of pages. You don't need much more than 4meg to map the entire 4GB address space of x86 with 4k pages (well, some internal OS memory tables too). Let's assume extreme++ waste and say 32megs are allocated to handle 4GB of memory... what's the problem here? One could also use 4meg pages for large memory allocations.

Sure, there's overhead for paging. There'd overhead for security. You have to weigh the overhead against the benefits. For a consumer OS (and especially a server OS), I'd say a lot of overhead can be acceptable.

Reducing r3<>r0 switches... make kernel functions do more ("everything in the kernel" - I don't like this general attitude, but there's sense in having kernel functions that do _some_ processing), or you can do more operations in ring3. There's security tradeoffs - and you wouldn't want to keep kernel mode objects in ring3 acessible memory.


They knowed it is going to be slow so they use a trick: Many many components of the OS (vital ones like the GDI and kernel parts for example) are runned like simple ring3 applications ...this helps reduce the "tons" of ring switches... and compromites the system stblility ... :(

NT5 runs well. I haven't gotten NT5 to crash by accident (except when doing ring0 coding, but what can you expect there). There's a few issues here and there (the kernel crash from ring3 by doing console output is very stupid), but generally it's a pretty stable system.

Well, enough.
If we shall discuss very subjective things, at least we need to define a context. The context I've been arguing from is "multiprocess multiuser multipurpose real-world assume-human-errors".
Posted on 2003-04-28 16:28:40 by f0dder

NT5 runs well. I haven't gotten NT5 to crash by accident (except when doing ring0 coding, but what can you expect there). There's a few issues here and there (the kernel crash from ring3 by doing console output is very stupid), but generally it's a pretty stable system.

It' not tha hard from ring3 either... (I think I've been albe to to crash every system I've tried, including 2k, haven't tested them all though)

f0dder, "all those extra things" doeas that mean stuff like samll games, simple text editor, video players, audio player, GUI/shell? (btw, do you know some good docs on paging, cant seem to find any simple docs, only docs assuming you already know paging to learn paging %-/ )
Posted on 2003-04-28 23:45:17 by scientica
"all those extra things"... well, at least a GUI. A shell, perhaps. Administration tools. With linux distros, linux is the kernel, and GNU/Linux is the OS. With windows... well, it's somewhat hard to separate the kernel, but there's still the kernel and the rest. Most OSes are like that (I don't really consider DOS an OS ;-)

As for docs on paging... the intel systems programming manual has extensive documentation about the mechanisms.
Posted on 2003-04-29 02:30:43 by f0dder

btw, do you know some good docs on paging, cant seem to find any simple docs, only docs assuming you already know paging to learn paging %-/
Here is one link, but I wish I could find the other one I had:
http://courses.ece.uiuc.edu/ece291/archive/archive-f2000/class-information/class-outline.html
Look at lectures for Oct.12.
Posted on 2003-04-29 02:39:11 by bitRAKE
Take a look at the attachment...,
Can I addressing the whole 4 GB, by only depending on prefix?
Posted on 2003-04-29 04:12:27 by realvampire
if your assembler can't handle "mov , al" in realmode (ie, automatically put the prefix), I suggest finding a different assembler.

Acessing memory that way isn't legal in a normal environment... but you can switch temporarily to pmode and do a few hacks to make it work... voodoo mode, unreal pmode, real big mode, whatever those various hacks have been called. I doubt Emu8086 would handle that though - it only handles 16bit mode, doesn't it? If so, have a look at http://bochs.sf.net
Posted on 2003-04-29 04:18:53 by f0dder
So accessing mem that way is illegal?. Mov ,al using prefix?
Posted on 2003-04-29 04:26:12 by realvampire
Should be, I'm rather confident I did it in the past :grin:
Just note that unless you "fiddle around", this will give problems when you cross segment boundaries.

The thing to do is switch to pmode, and change fs+gs to base0, limit4g, then change back.

have a look here: http://www.nondot.org/sabre/os/files/ProtectedMode/segments.html#flat
...
I prefer real 32bit mode.
Posted on 2003-04-29 04:30:52 by f0dder
How to set FS and GS? just simply do this step?



Mov ax,0FFFFh
mov FS,ax
mov GS,ax


....? :confused:



GDT dw GDT_length - 1
dd GDT
dw 0



What value should I put ? There?




mov ax,DS
shl eax,4
mov [GDT+2],eax ;where is it? dw 0? or dw GDT_length -1 ?

Posted on 2003-04-29 04:41:05 by realvampire
Another routine to do it...
http://www.df.lth.se/~john_e/gems/gem0022.html

And, quoting some other site:

For those who do not know: Flat Real Mode is done by starting in Real Mode DOS, switching to 32 bit Protected Mode, loading the segment registers with selectors which point to descriptors set to a segment size of 4GB and switching back to Real Mode while keeping the segment register values set in Protected Mode. One may expect what you are back in the standard Real Mode, but - due to the fact that the 386 does not reset the segment limits while switching back to Real Mode - you have full 32 bit access to the whole memory while using 16 bit code.

This sounds great (it is), but there is one big problem: You must start in Real Mode rather than in V86. Most people today are using PM OSses, and the few still having DOS are often using EMM386. In both cases you cannot switch to PM because the CPU is already in it, your code runs at PL 3 and the switch will not work.

If you want that more than one or two people out of 100 are able to start the program without problems you should forget this method and use another, like DPMI instead.
Posted on 2003-04-29 04:51:45 by f0dder
not with your help, but finally i got a completely working gfx-mode. the problem was A20. with it disabled i read that there are memory holes everywhere in size of 1 mb - exactly matching to the black bar in the middle of the screen. furthermore i couldn't really access memory beyond 1mb (that caused another weired problem). just by enabling it with 4 lines of code everything worked fine!

but i'm still riddeling about the mouse, i don't get it working. as i already mentioned i can enable it and it partially works. but when i move the mouse nothing happens (i.e no interrupt is called), but after doing that the keyboard also seems to be disabled (i.e. no react on keystrokes). i've tried several code-variants, but nothing worked. i digged through the information you gave me, but i do everything according to it.

any help for me? some further infos?
Posted on 2003-05-11 13:32:58 by hartyl
I have enable it too. But Im not sure I can access it. How did you access it. And what is the mark it was succeed. Can I see your code ?

:alright:

Can I running my Switch mode from DOS(Because it was .Com), and put the GDT value at PSP ?




Where GDT is DS value shifted left by 4.
GDT dw 0xF, GDT, 0, 0, 0xFFFF, 0, 0x9200, 0x8F


Posted on 2003-05-11 17:42:30 by realvampire

I have enable it too. But Im not sure I can access it. How did you access it. And what is the mark it was succeed. Can I see your code ?

here is the code i use:


;enable A20
mov al,0xd1
out 0x64,al ;get statusbyte to port 0x60
mov al,0xdf ;set A20-bit and cpu-bit not to reset it
out 0x60,al ;write it back


Originally posted by realvampire
Can I running my Switch mode from DOS(Because it was .Com), and put the GDT value at PSP ?

h??? what do you mean by "switch mode" and whats up with dos and .com? and finally: what's PSP?
understanding "railwaystation" here...


i'd like to point you out to my problem, again. yesterday i tried the complete code from SolarOS just for testing. in SolarOS itself the mouse completely works, but in my code... same problem. do you initialize more than i do, BogdanOntanu, or have i overseen something?
Posted on 2003-05-12 12:50:33 by hartyl
Well there are many steeps in PS/2 initializations:
-enable device in keyboard controller, yeah mouse and keyboard share the same controller, hence the starnge keyboard behavoiur if mouse works heratic,

-but indeed the Interupt is the most important stuff there, have you enabled it in the PIC? ---IRQ12 i mean

-also you can not start and output data to mouse/controller, esp on today fast CPU's one must wait for ACK from controller and also wait for the FA=>ACK from PS/2 mouse

Well i must say SOLAR OS also has some problems with mouse initializations esp on a HP kayak PC...but rarely and a new resetup usually clears them


But Keyboard and Mouse must both use working IRQ's handlers for everything to be ok.
Posted on 2003-05-12 14:26:40 by BogdanOntanu
Line A20 enabled are used for memory is not it? If I have enable it, how to access the memory? Its above 1MB and DS:SI cannot do that.
Posted on 2003-05-12 21:44:25 by realvampire
Realvampire,

A20 line is an artifact left over from the 80286 era.

At that time the 80286 CPU was able to acces memory up to 16M. A mistake in CPU made it able to acces 64K ABOVE 1M even in standard real mode 8086 like mode of operation.

The bug was related to making a segment base to FFFF that shifted left one digit inside CPU and adding the offset made it possible to acces memory just a little above 1M.

Need to say that this "feature" was crashing the old 8086 applications that assumed the memory will WRAP arround and all memory above 1M will actually be above zero.

BUt this feature was also nice because after all one could have 1M+64Kb at hiw dispossal. To make everybody happy the PC hardware motherboards proviced a way to ENABLE/DISABLE this feature via a few OUTs to a free port....and guess ot the keyboard controller had a few bits free :P

The disable was default mode and it actually hold A20 line at ZERO. This way memory above 1M was translate to 0.


So enabling A20 does nothing but allow CPU to not bee crippled by forcefully keeping memory line A20 at ZERO

It is NOT allowing you byt itself to acces memory ABOVE 2Mega for example.
Normal Real Mode can not acces that memory (but Unreal can)...

And trying to use DS:DI is useless unless you want to use that little 64K above 1M.

Should use DS:EDI to try and access 32 bit memory but the CPU in real mode has some internal LIMITS setup and will trim EDI to DI ...

The Unreal mode (aka go fast in Protected mode, and load GDT with new limits then return in real mode could help as f0dder told you...but is somehow useless if already in protected mode or V86
Posted on 2003-05-13 01:48:41 by BogdanOntanu