I was wondering if anybody on this board has experience directly interfacing with USB devices... ie, no BIOS or OS support. I'm toying around with a hobby OS project, and was wondering if it's feasible to add support for USB pendrives.

Booting is easy enough because my BIOS supports it, but once the kernel is loaded I'd prefer to use native drivers (v86 might be an option, but eeky-pooh in the long run).

So, anybody with experience... how much coding would such USB support take? Any good documentation available? I don't want to look at linux source, GPL taint and all that :)
Posted on 2005-01-06 05:30:30 by f0dder
I am also intrested in this issue for SolOS so in case i will find anything before you do i will share it with you ;)

There are pdf's about the USB interfaces here:
http://www.beyondlogic.org/
additional info:
http://www.nondot.org/sabre/os/articles/InterconnectBuses/

I have very simple and clear ASM code for enumerating PCI devices -- including USB in SolOS.

This is the first phase for finding devices type/vendor and IRQ+ I/O + memory resources in use by USB; after that you will have to study/implement USB standards.

I know that my OS is GPL but i am willing to grant you any licence you need, just ask :D . So you will not get "tainted" or "infected" by it :P

Unfortunately my test macine is an Aquanta P1/166 and aparently it can not boot from an USB stick. But other machines might.

So how exactly do you make a stick bootable?
I just have to run c:\sys X: ?
Posted on 2005-01-06 06:18:23 by BogdanOntanu
Hm, I can't actually remember what I did to make the stick bootable... but formatting with FAT16 and getting a DOS bootsector (+ io.sys etc) is enough. I don't have Win9x around, so I couldn't "sys X:" - but I guess it should be good enough.

If your machine is a P1, it probably doesn't support booting from USB :-(

My main P4 (SiS 645DX chipset) doesn't even support USB booting :(, but my brothers' p4 celeron with intel 845GE chipset does. I had to enable "usb legacy device support" in the BIOS. Then attach the pendrive, reboot, and I can select the pendrive as a boot device.

Apart from BIOS support, I think pendrive might have to support it as well - I don't think my old pendrive showed up as a boot device, but that *might* have been because I didn't have legacy usb support enabled.



I know that my OS is GPL but i am willing to grant you any licence you need, just ask :D . So you will not get "tainted" or "infected" by it :tongue:

Hehe, thanks :). I'd prefer writing the code myself because I'm doing this for fun, so looking at other source would mainly be if there's no good documentation around. Thanks for the offer though, I might use it some time :)

My kernel is just a hobby project, and I haven't worked on it for a couple of years... but I would like to play around with it again sometime soon, if I can find the time.
Posted on 2005-01-06 06:32:56 by f0dder
You may want to search for UHCI (USB 1), OHCI (USB 1), and EHCI (USB 2). These are the specs for the high performance USB root controllers used in PCs.

You will need to understand Chapter 9 of the USB specs, which can be found at http://usb.org. And you will need the specs for the "Mass Storage" device class.

If you want to use hubs to connect to several USB devices, then you will also need to understand the hub commands listed in Chapter 11.

Once you've figured out how to control the hardware, I can help you understand the basics of the data protocol. I've never written any host code, but I do know what to expect from a host, as I have worked on the device side of USB. At present, I can't help very much with the added functionality of USB 2.

The amount of required work depends on how universal you want your USB host to be. If you restrict yourself to supporting exactly one device, no hubs, the work may be very easy. The mass storage class looks relatively simple. But full HID support is a nightmare because it's hard to understand the usage of report descriptor items. The BIOS gets around the HID nightmare because HID defines a simplified keyboard and mouse interface just for "legacy" use.
Posted on 2005-01-06 21:21:21 by tenkey
Thanks tenkey, this sounds promising :)

So the USB specs are public... very nice, I shouldn't have to look at other people's source then. At first I'll just support one device, my pendrive (I guess this means support for all pendrives, aren't they just mass storage devices?), but I might do more late on. I don't intend to write "the next linux" or anything, I'm just playing around, so apart from booting from a pendrive, USB support isn't that high on my wishlist.

As for USB2... is there anything I need to do to get the higher transfer rate, or does that happen automagically even with OHCI as long as the device is plugged into a usb2 port?

.........

As for my kernel, I've sorta restarted it from scratch. I learned a lot from my previous version, so the next version is going to be better. It might even end up doing something useful :P.

I've switched from the DJGPP version of GCC and NASM, to Microsoft Visual C++ and FASM. Since VC++ is now free (vc toolkit) and is *not* restricted to windows apps, I think this is fine. Making the bootloader boot a PE file rather than raw binary was pretty easy, but it's not like I'm loading the PE - I just get e_lfanew from the MZ header, then jump to the entrypoint. It works well, as long as memalign==filealign :)
Posted on 2005-01-07 10:25:47 by f0dder
I would also be happy to help as well. I wrote the Dell USB legacy BIOS. You will probably need to also support hubs. That is because what happens if someone attaches your usb storage device to a hub? You can't configure it? I also came across quite a few keyboards that had built in hubs. Just by looking at the keyboard you couldn't tell. But when you started talking to it, the hub responded first. And the keyboard replied second. You don't have to do EHCI ( USB 2.0). Every device has to be backwards compabtible with USB 1.1. Currently in the Dell BIOS we do not support USB 2.0.

... But full HID support is a nightmare because it's hard to understand the usage of report descriptor items. The BIOS gets around the HID nightmare because HID defines a simplified keyboard and mouse interface just for "legacy" use.



You can also use the simplified keyboard and mouse interface for your OS. No sense in making things harder then they need to be.
Posted on 2005-01-07 10:33:45 by mark_larson
I had hoped you would kick in sooner or later, Mark, as I figured you might have some experience with this :)


Every device has to be backwards compabtible with USB 1.1.

Does this mean reverting to USB 1 speed too?
How much more complicated is EHCI compared to OHCI?
(Guess I should start reading the specs pretty soon ;) ).

USB1 speed would be acceptable I guess, the main purpose of USB support will be to boot off the pendrive and access modules and such - and those won't be very large.
Posted on 2005-01-07 10:41:36 by f0dder
I had hoped you would kick in sooner or later, Mark, as I figured you might have some experience with this :)


Every device has to be backwards compabtible with USB 1.1.

Does this mean reverting to USB 1 speed too?


Yes, but for some things that's not a problem. A keyboard and mouse does not need to run at USB 2.0 speeds.


How much more complicated is EHCI compared to OHCI?
(Guess I should start reading the specs pretty soon ;) ).


They are all pretty similar. You have to implement both UHCI and OHCI for USB 1.1 support. And then if you want USB 2.0 you will do EHCI. The reason for having to do both UHCI and OHCI is two differnet groups wanted to come up with their own way of doing transactions ( bastards;). So you have to support both methods. As a general rule, Intel systems all come with UHCI. So the vast bulk of the systems are UHCI. So if you had to pick only one to do, do UHCI.

USB1 speed would be acceptable I guess, the main purpose of USB support will be to boot off the pendrive and access modules and such - and those won't be very large.


That's the same for our BIOS. We support bootable drives via 1.1. A lot of devices aren't USB 2.0 anyways. And the ones that are don't exhibit a lot of speed for being USB 2.0. I have seen a few USB 2.0 storage devices that are really fast. But for the most part they aren't.

When I wrote the code, there were no tools for testing USB or doing USB development. So I wrote my own tool that displays the I/O ports ( it continually reads the I/O ports so you can watch them get updated), and let's you run different transactions in memory, and shows you the results. The tool runs in DOS, and uses a mouse. You can hold the mouse over different bits in the transactions in memory and it will tell you what the bits mean. I wrote it in C. It uses UHCI and doesn't do interrupts. I'll have to see if I can still find it.
Posted on 2005-01-07 16:21:35 by mark_larson
Indeed a mouse or keyboard doesn't need USB2 speed... I guess that's mostly for storage devices and cameras. Perhaps some printers and scanners (though I suspect most scanners are too slow to really take advantage of USB2?).

There's quite a speed difference between usb1 and usb2 with my pendrive - I thought the bastards had cheated and written the speed in mbits rather than mbytes, then I realized that I had my usb front ports connected to the usb1.1 pins on the motherboard - doh!. Suddenly transfers to my pendrive and MuVo^2 went a *lot* faster ;-).

I think I get about 12mbyte/s reading the pendrive, and somewhat slower writing speed, but this is quite acceptable to me, even if it's way below the 480mbit that USB2 can theoretically deliver :)

For my kernel, USB1 speed should be acceptable as well - I don't think the "OS" will ever become very large. But it would be nifty adding USB2 support anyway. But first things first, and there's a lot of coding that needs to be (re)done first.

I would very much appreciate if you can find the tool, it sounds interesting and useful! And thanks for the clarification of UHCI+OHCI... I'll start with UHCI then, since my boxes are mostly intel. Any idea if an AMD64 based system would be UHCI or OHCI?
Posted on 2005-01-07 16:38:32 by f0dder
Indeed a mouse or keyboard doesn't need USB2 speed... I guess that's mostly for storage devices and cameras. Perhaps some printers and scanners (though I suspect most scanners are too slow to really take advantage of USB2?).


Never used a USB2 scanner. I have a USB2 camera ( 7.2 megapixel). The USB 2.0 makes a big difference since the pictures compressed run around 7 megs per picture.



There's quite a speed difference between usb1 and usb2 with my pendrive - I thought the bastards had cheated and written the speed in mbits rather than mbytes, then I realized that I had my usb front ports connected to the usb1.1 pins on the motherboard - doh!. Suddenly transfers to my pendrive and MuVo^2 went a *lot* faster ;-).

I think I get about 12mbyte/s reading the pendrive, and somewhat slower writing speed, but this is quite acceptable to me, even if it's way below the 480mbit that USB2 can theoretically deliver :)



The UBS 2.0 bus has a much bigger bandwidth. However most USB 2.0 devices I've played with run at 8megabytes/second or slower. So the 12 you are getting is really good. The 480mbit was designed not so one device can use all that bandwidth, but so that multiple devices can all run at 10+ megabytes/sec. If that makes sense.


For my kernel, USB1 speed should be acceptable as well - I don't think the "OS" will ever become very large. But it would be nifty adding USB2 support anyway. But first things first, and there's a lot of coding that needs to be (re)done first.

I would very much appreciate if you can find the tool, it sounds interesting and useful! And thanks for the clarification of UHCI+OHCI... I'll start with UHCI then, since my boxes are mostly intel. Any idea if an AMD64 based system would be UHCI or OHCI?


For AMD it's gonna vary. Intel built their USB controllers into their southbridges ( called ICH and PIIX). So all the chipsets got them. For AMD it varies because they don't always have them built into their chipset. The last piece of data I read from AMD's website is they don't have USB 2.0 built into their chipset for AMD64, but that you can add it via an external chip. So you still get USB 1.1, and if you want USB 2.0 you add a chip to your motherboard that does USB 2.0. Make sense?

I'm half asleep, so not sure if this makes sense. I played WOW all weekend. Till 4am on Saturday and 1am on Sunday.


I'll see if I can find that utility. I archive everything since I never know when I"ll need something again.
Posted on 2005-01-10 16:12:52 by mark_larson

The 480mbit was designed not so one device can use all that bandwidth, but so that multiple devices can all run at 10+ megabytes/sec. If that makes sense.

Indeed it does, and I knew this already. It's not even possible for a single device to use the full bandwidth, is it? And I really hate how many companies are labelling their stuff "USB2 hi-speed" when it only runs somewhat faster than usb1.1. I guess with pendrives there's some physical limitation because of the type of flash used?


and if you want USB 2.0 you add a chip to your motherboard that does USB 2.0. Make sense?

Yup. SiS chose the same strategy for their 645DX chipset, and my MSI-645E max2 has a NEC (I think) USB2 controller. Means there are separate USB1 and USB2 ports, and the nice little 18-in-1 cardreader + 3port UBS2 hub can only run usb1.1 speed on my box (yes, plugged in usb2 port). Nice. My mothers laptop and younger brothers' celeron (both with intel chipset) works just fine and dandy.

Hehe, world of warcraft has bitten you too? There's one of my friends I rarely get to talk to anymore because of it ;)

PS: My upcoming AMD64 box will have a nvidia nForce4 chipset, hope they're doing USB2 properly.
Posted on 2005-01-10 16:23:37 by f0dder

Indeed it does, and I knew this already. It's not even possible for a single device to use the full bandwidth, is it? And I really hate how many companies are labelling their stuff "USB2 hi-speed" when it only runs somewhat faster than usb1.1. I guess with pendrives there's some physical limitation because of the type of flash used?



Not that I know of. I just figured the manufacturer's were morons. ;) I read a comparison on Anandtech of different pendrives. Their was actually quite a wide range of transfer speeds. They even as a joke got two pendrives working in a software RAID configuration and benchmarked that as well.


Yup. SiS chose the same strategy for their 645DX chipset, and my MSI-645E max2 has a NEC (I think) USB2 controller. Means there are separate USB1 and USB2 ports, and the nice little 18-in-1 cardreader + 3port UBS2 hub can only run usb1.1 speed on my box (yes, plugged in usb2 port). Nice. My mothers laptop and younger brothers' celeron (both with intel chipset) works just fine and dandy.


AH cool. I tend to stay with more of the standard stuff unless I am building my own system.



Hehe, world of warcraft has bitten you too? There's one of my friends I rarely get to talk to anymore because of it ;)


Yep, and I fit in REAL good to. A 35 year old playing with all the 15 year olds. ;)



PS: My upcoming AMD64 box will have a nvidia nForce4 chipset, hope they're doing USB2 properly.


Yep. I am going to get AMD64 for my next system to. I am going to try and con hutch into giving us a forum for it ;) Might not generate enough traffic, so not sure if he'll do it or not. But I'm dying to get an AMD64 processor. I did get to play with one for a few months at work. Ran Fedora linux on it and used YASM and 64-bit GCC to compile 64-bit APPS :)


I think I found the code. The bad news is I had 10 or 12 differnet versions of it, and I'm not sure what is the latest. I haven't had a chance to look at it closely to see. Also it is set to compile with Borland C for DOS. It also checks for an ICH2 ( remember those in a previous post?). Because it uses some hardcoded I/O ports for the ICH2 that it uses for USB. You'll have to fix that to work with what the USB I/O ports are on your machine ( you could modify it to just pass in the base address). The code is ugly because it was never designed to be distributed.

Second thing. I found a cool tool that came out after I wrote my tool. It is called usbcomp. It let's you single step through transactions under windows. That's useful for getting a feel for how transactions work.

I just realized I can't post ZIP files here. Let me add it to masmforum.

I posted it in the laboratory, with the title EWW ESS BEE ( what I called it frequently when coding it ;)
Posted on 2005-01-11 00:39:44 by mark_larson

AH cool. I tend to stay with more of the standard stuff unless I am building my own system.

I was a fool to go for the SiS chipset - I was lured by DDR333 support, where the available boards with intel chipsets only supported DDR266. Turns out that an 845PE (I think it was) based board with DDR266 performed just as well as my DDR333 SiS board... and of course has less trouble. *sigh*. I hope nvidia is doing a good job with their nForce4 - I haven't used AMD hardware since my Athlon700, but it seems AMD64 is a good choice nowadays.

I think I have a machine with ICH2 or ICH3 - can't remember which version it was, but it's in the celeron box with intel chipset. Ugly code and BCC isn't that much of a problem, I'm currently looking at the LZMA SDK by the 7-zip author... and I doubt you can beat him ;)

I'll have a look at the masmforum when I get home :)

Btw, as for my current new-box shopping list: http://f0dder.mine.nu/newbox_specs.txt . The CPU isn't available around here until mid february (being 0.09 instead of 0.13 micron technology), so the list might change yet... but this seems like a nice config to me.
Posted on 2005-01-11 02:41:53 by f0dder

I was a fool to go for the SiS chipset - I was lured by DDR333 support, where the available boards with intel chipsets only supported DDR266. Turns out that an 845PE (I think it was) based board with DDR266 performed just as well as my DDR333 SiS board... and of course has less trouble. *sigh*. I hope nvidia is doing a good job with their nForce4 - I haven't used AMD hardware since my Athlon700, but it seems AMD64 is a good choice nowadays.


I usually stay with the big vendors for my chipsets since it's a very important piece.


I think I have a machine with ICH2 or ICH3 - can't remember which version it was, but it's in the celeron box with intel chipset. Ugly code and BCC isn't that much of a problem, I'm currently looking at the LZMA SDK by the 7-zip author... and I doubt you can beat him ;)


You just modify the program to look for whatever I/O controller you have instead. That way you don't have to worry about having an ICH2 system. You can also just look at the base address of the UBS in a PCI viewer. Write it down, and pass it into the program instead. I think it is at offset 20h in the PCI space of the ICH2.

Yea, well he made his for public distribution. I didn't. I wasn't planning on ever giving it out. It was a tool to help me better understand how USB works both by writing it, and by visually seeing how transcations work.



I'll have a look at the masmforum when I get home :)

Btw, as for my current new-box shopping list: http://f0dder.mine.nu/newbox_specs.txt . The CPU isn't available around here until mid february (being 0.09 instead of 0.13 micron technology), so the list might change yet... but this seems like a nice config to me.




USD 2.018


You're going to pay 2 dollars for your system? Shoot where do you get those things? I want one! ;) heheee



couple points if you are going to run the program

1) Requires borland C to compile

2) Has to run from a bootable floppy because it directly talks to the I/O for the USB host controller

3) You have to run MODE from the floppy and set it to 50 rows, that's needed because the program does text graphics to display transactions visually. The QueueHeads and Transfer Descriptors get drawn.

4) You have to have a PS/2 mouse and keyboard hooked up ( not USB). USB won't work because we will be using the USB controller to run our own transactions.

5) I can't remember if I turn off USB legacy support in my program or not. But to be safe you should probably turn it off. We will be changing the base address of the USB transactions, and unless it is properly turned off, the BIOS might have problems.

6) The program uses the PS/2 mouse. You can select transactions and it will pop up a box with all the bits defined for that dword you clicked on in the transaction.

7) hit 'r' to run a transaction.

8) The program uses UHCI. If you hook it up to a system that uses OHCI it won't work.

9) Be a cool dude and hot plug a USB device. You can watch the bits change in the I/O ports for the USB host controller ( way cool!)
Posted on 2005-01-11 09:50:34 by mark_larson

I usually stay with the big vendors for my chipsets since it's a very important piece.

I thought SiS was an okay vendor - silly me. I knew I definitely wanted to stay away from VIA, as their chipsets suck.


You just modify the program to look for whatever I/O controller you have instead. That way you don't have to worry about having an ICH2 system. You can also just look at the base address of the UBS in a PCI viewer. Write it down, and pass it into the program instead. I think it is at offset 20h in the PCI space of the ICH2.

Hope I can figure this out :P


Yea, well he made his for public distribution. I didn't. I wasn't planning on ever giving it out.

I meant beating him in terms of crap unreadable spaghetti code ;)


You're going to pay 2 dollars for your system? Shoot where do you get those things? I want one! ;)

Hehe, in Denmark the thousands separator is '.' and the decimal point is ','. I usually use the english (ie, reverse) notation myself (because of C, duh), but the numbers were copied directly from shg.dk and valutakurser.dk , so... :)

1) no problem, got BCC3.0 lying around
2) a bootable pendrive is probably a bad idea? ;) Good thing I still have a few boxes with floppy drives around.
3) ok, http://f0dder.mine.nu/80x50.exe - pretty nice, and can work as a "device driver" as well.
9) I have a few :) - MuVo, printer, pendrive, usbhub/cardreader, camera.

Looking forward to play around with this :)
Posted on 2005-01-11 13:46:38 by f0dder


You just modify the program to look for whatever I/O controller you have instead. That way you don't have to worry about having an ICH2 system. You can also just look at the base address of the UBS in a PCI viewer. Write it down, and pass it into the program instead. I think it is at offset 20h in the PCI space of the ICH2.

Hope I can figure this out :P


If the chipset has an integrated USB controller it will be in the south bridge. For most Intel systems that will be ICH, ICH2, ICH3, etc. Or for older systems PIIX, PIIX2, etc. Do you have any programs to read PCI space for devices? Alternatively you should be able to do it from Windows. I would be surprised if the I/O address used in Windows is different from DOS.

For windows go into Control Panel->System->Hardware->Device Manager

scroll down in the list until you see "Universal Serial Bus Controllers". Click on the plus sign to expand the list. You should have one or more items that should have the words:
"USB Unversal Host Controller". It'll have other text on the same line. You just want to look for those 4 words. I have 3 on my system. 2 for 1.1 USB and 1 for USB 2.0. I'd pick the first one in the list since it's probably USB 1.1.

Pull up it's properties. Go to resources. Find the I/O range. Write it down. That's it. In my program go and remove the check for ICH2. I can't remember if I hardcoded the I/O port or used what I found in the ICH2. In either case use the one that windows reported in the program that I wrote.












Yea, well he made his for public distribution. I didn't. I wasn't planning on ever giving it out.

I meant beating him in terms of crap unreadable spaghetti code ;)


Yea, I have a lot less, because my program is smaller ;)


1) no problem, got BCC3.0 lying around
2) a bootable pendrive is probably a bad idea? ;) Good thing I still have a few boxes with floppy drives around.
3) ok, http://f0dder.mine.nu/80x50.exe - pretty nice, and can work as a "device driver" as well.
9) I have a few :) - MuVo, printer, pendrive, usbhub/cardreader, camera.

Looking forward to play around with this :)


If you want to be a cool dude, you can always modify the program to support the legacy USB BIOS and also allow you to run transactions. You'd have to find the base address for the transactions in memory, and add one to the list. Basically their is a big linked list in memory of transactions to run. It's a bit more complicated than that, but that's the easy way to view it. It's more complicated because it's not a linearly linked list. It looks more like this.




Each "box" represents a dword ( 4bytes). Extending that logic a Queue Head is 2 dwords, and a Transfer Descriptor is 4 dwords. The first dword of a queuehead points to another queuehead. The second dword points to a transfer descriptor. Most boxes are pointers to other boxes. An arrow indicates the box is a pointer and points to another box. Each box that is a pointer also has a bit you can set that says it doesn't point to anything. That is called the terminate bit.

------------ <----array of pointers to transactions
| |
------------
| |
------------
| |
------------
| |
------------
| |----------> ------------ ------------
------------ | Queue |--------->| Queue |
------------ ------------
| Head | | Head |
------------ ------------
| |
| |
V V
--------------- ---------------
| Transfer | | Transfer |
-------------- ---------------
| Descriptor | | Descriptor |
--------------- --------------
| | | |
--------------- --------------
| | | |
--------------- --------------
| |
| |
V V
--------------- ---------------
| Transfer | | Transfer |
-------------- ---------------
| Descriptor | | Descriptor |
--------------- --------------
| | | |
--------------- --------------
| | | |
--------------- --------------
| |
| |
V V
--------------- ---------------
| Transfer | | Transfer |
-------------- ---------------
| Descriptor | | Descriptor |
--------------- --------------
| | | |
--------------- --------------
| | | |
--------------- --------------

Posted on 2005-01-11 15:48:10 by mark_larson
Hold on there Babalouie. Fodder is always way ahead of me. It's like starting a new race and he's starting at the 60 yard line. Now Mark can we start with some basics here. Does Icz have a tutorial I'm not aware of? What is the port number. How about a pinconfiguration. Does one of the pins turn on VCC? I'm an old RS-232 guy and mainly hardware but thanks to 4F I can get to hardware in win32. Any links or suggestions?
                                    Thanks
Posted on 2006-07-13 15:32:04 by mrgone