I finally finished my neural network object (with the masm32 OOP model), it's working great...
For those who don't know, neural networks are used to create artificial intelligence.. Inspired by the brain, a network of interconnected neurons is set up, with a set of inputs and outputs.
The network is then trained with a training set, and learns which outputs correspond to which inputs. Then the trained network can be stored an reused anytime for making decisions.
Neural networks are pariculary good at pattern recognision, as you will see in the example.
I've worked out all the dirty maths and put everything into an easy to use class (classes are excellent for these kind of things), which you can for any AI you need.
The class has very simple methods like 'train(inputs, desired outputs)' and 'run(inputs)', as well as save & load functions for storing the learned data in a file or memory.
I will release the class code soon, the example in the attachment demonstrates it's use. It can recognize the first four characters of the alphabet, drawn with the mouse the way most handhelds do. It's a relatively simple network (20 neurons, 108 connections) but I was surprised at how well it works.
Just draw an A, B, C or D on the drawing tablet, the program speaks for itself.
I will try to post the class code this week, for now you'll have to do it with this sneak peek..

Thomas
Posted on 2002-01-13 11:55:49 by Thomas
Thomas, this tool is absolutelly impressive. I have found only a little difficulty to draw a 'D', I draw an arc (right side) from top to bottom and the program parse it as a 'B'. All other letters are well recognized. Again, good work.
Posted on 2002-01-13 12:12:44 by LuHa
That's pretty neat :). It has *major* trouble recognizing "D", though.
And if I draw the letters too small, it doesn't recognize them either.
But all in all, pretty cool. And I like the look of the program as well.
Posted on 2002-01-13 12:15:26 by f0dder
Thats incredble Thomas. Will you be releasing the source, or perhaps just an explanation of how the networks works. I was always intrested in AI, but could never figure out how to connect neurons properly.

Great Work.
Posted on 2002-01-13 12:18:20 by Eóin
Thanks,
yeah I also noticed it has troubles recognizing the 'D', but I think it's due to bad training :). I've drawn each character about 8 times to train the network. I'll train it some more and upload it again.

Thomas
Posted on 2002-01-13 12:19:03 by Thomas
Will you be releasing the source, or perhaps just an explanation of how the networks works. I was always intrested in AI, but could never figure out how to connect neurons properly.


Yes I'll release the full source but I think I will need to write an accompanying tutor to make it understandable. The algorithms are quite complex to program, and very hard to understand.. I've spend quite some time reading info about neural networks (especially back-propagation) but it's still hard to get a firm grasp of it.. Right now I exactly know how the code works but I need to document it well for myself to understand it when some time has passed...
I site that really helped me was this one:
http://www.generation5.org/essays.shtml#nn

It doesn't show the full mathematical derrivations of the formulas, which most books and documents do (very confusing...). It also had a simple example (fixed number of neurons, only 5), which I used as reference to see if the program produced the right output (I've spend hours to trace a bug that was caused by using ecx instead of ebx...aaargh).

Thomas
Posted on 2002-01-13 12:24:00 by Thomas
What about releasing source first, tutorial later? That way people
who want to get their hands dirty, can :).
Posted on 2002-01-13 12:28:30 by f0dder
Great work! :alright:
But my 'D' looks strange :grin:
Posted on 2002-01-13 12:34:44 by bazik
Attachment updated, if downloaded please download again

It works much better for the D now :)

And if I draw the letters too small, it doesn't recognize them either.

It splits your drawn character into pieces with a length of 15 pixels, and it needs at least 8 points to be recognized. But besides that, size doesn't matter.

What about releasing source first, tutorial later? That way people who want to get their hands dirty, can


Hmm I can do that, however the source is really hard to follow without knowing what is happening... besides the class misses some parts (easy parts though, like returning error values, not the actual network code)..

Thomas
Posted on 2002-01-13 12:37:09 by Thomas
Here's the current source for the network object, as well as a simple example that learns the XOR operation. It needs masm32 v7 and outputs in vkim's debug window. Inputs and outputs are REAL4 values between 0.0 and 1.0. Inputs are given by an array of real4 values, the 'run' method returns a pointer to an array of real4 values that represent the output.
Note that this is still a beta, it isn't finished yet.

Thomas
Posted on 2002-01-13 13:27:49 by Thomas
coooooooool!!! :)
Posted on 2002-01-13 13:43:41 by stryker
Very awesome Thomas! :alright:
Posted on 2002-01-13 14:08:30 by bitRAKE
Extremely Impresive Thomas! Great Work!

:alright:
NaN
Posted on 2002-01-13 14:38:59 by NaN
Thomas,

Thank you. I also am intrested in this area of computers. I wish I could express my admiration better, but I'm lost for words. Seeing your progress in AI gives me a great hope that someday (hopefully in my lifetime) we will acheive our ultimate goal. Please keep it coming, feed my hungry mind, and once again, thanks.
Posted on 2002-01-13 15:06:30 by lackluster
object MACRO nm:REQ,ty:REQ

_BSS SEGMENT
lp&nm dd ?
_BSS ENDS

&nm MACRO mth:REQ,vars:VARARG
IFB <&vars>
method lp&nm,ty,mth
ELSE
method lp&nm,ty,mth,vars
ENDIF
ENDM
ENDM

object XorNet, NNet

...

XorNet initRandom

.WHILE ebx<2000 ;train 2000 times
rdtsc
mov ecx,eax
xor eax,edx
ror eax,cl
and eax,3*4
XorNet train, inputs[eax], outputs[eax]
inc ebx
.ENDW
Maybe, this could be merged into the newobject macro? Okay, maybe not - this really only works when dealing with single objects.
Posted on 2002-01-13 15:21:32 by bitRAKE
great work! thomas,

i have some question.

what is the data u use to train the network? just a raw bitmap or u have some preprocessing? how many inputs to the network? and the last one what is network u r using? backpropagate?
Posted on 2002-01-13 17:29:02 by doby
:alright:

Wow. . .
Posted on 2002-01-13 19:24:02 by -T-
Great work Thomas, and that is a good looking UI as well.

How difficult is it to take your concept one step further, and build a OCR type engine, for recognising text on a scanned image, or for ripping text out of a screen-scrape image?
Posted on 2002-01-13 21:07:03 by sluggy
I hope that teacher, that was looking for REAL Assembly examples, is still around... :grin:
Posted on 2002-01-13 23:58:30 by S/390
Awesome ! :eek:
Too bad I don't have the time to look at it more carefully. :(
Posted on 2002-01-14 00:57:38 by JCP