Neural networks object: digital brain

[Thomas]

I finally finished my neural network object (with the masm32 OOP model), it's working great...
For those who don't know, neural networks are used to create artificial intelligence.. Inspired by the brain, a network of interconnected neurons is set up, with a set of inputs and outputs.
The network is then trained with a training set, and learns which outputs correspond to which inputs. Then the trained network can be stored an reused anytime for making decisions.
Neural networks are pariculary good at pattern recognision, as you will see in the example.
I've worked out all the dirty maths and put everything into an easy to use class (classes are excellent for these kind of things), which you can for any AI you need.
The class has very simple methods like 'train(inputs, desired outputs)' and 'run(inputs)', as well as save & load functions for storing the learned data in a file or memory.
I will release the class code soon, the example in the attachment demonstrates it's use. It can recognize the first four characters of the alphabet, drawn with the mouse the way most handhelds do. It's a relatively simple network (20 neurons, 108 connections) but I was surprised at how well it works.
Just draw an A, B, C or D on the drawing tablet, the program speaks for itself.
I will try to post the class code this week, for now you'll have to do it with this sneak peek..

Thomas

[LuHa]

Thomas, this tool is absolutelly impressive. I have found only a little difficulty to draw a 'D', I draw an arc (right side) from top to bottom and the program parse it as a 'B'. All other letters are well recognized. Again, good work.

[f0dder]

That's pretty neat . It has *major* trouble recognizing "D", though.
And if I draw the letters too small, it doesn't recognize them either.
But all in all, pretty cool. And I like the look of the program as well.

[Eóin]

Thats incredble Thomas. Will you be releasing the source, or perhaps just an explanation of how the networks works. I was always intrested in AI, but could never figure out how to connect neurons properly.

Great Work.

[Thomas]

Thanks,
yeah I also noticed it has troubles recognizing the 'D', but I think it's due to bad training . I've drawn each character about 8 times to train the network. I'll train it some more and upload it again.

Thomas

[Thomas]

Will you be releasing the source, or perhaps just an explanation of how the networks works. I was always intrested in AI, but could never figure out how to connect neurons properly.

Yes I'll release the full source but I think I will need to write an accompanying tutor to make it understandable. The algorithms are quite complex to program, and very hard to understand.. I've spend quite some time reading info about neural networks (especially back-propagation) but it's still hard to get a firm grasp of it.. Right now I exactly know how the code works but I need to document it well for myself to understand it when some time has passed...
I site that really helped me was this one:
http://www.generation5.org/essays.shtml#nn

It doesn't show the full mathematical derrivations of the formulas, which most books and documents do (very confusing...). It also had a simple example (fixed number of neurons, only 5), which I used as reference to see if the program produced the right output (I've spend hours to trace a bug that was caused by using ecx instead of ebx...aaargh).

Thomas

[f0dder]

What about releasing source first, tutorial later? That way people
who want to get their hands dirty, can .

[bazik]

Great work! :alright:
But my 'D' looks strange :grin:

[Thomas]

Attachment updated, if downloaded please download again

It works much better for the D now

And if I draw the letters too small, it doesn't recognize them either.

It splits your drawn character into pieces with a length of 15 pixels, and it needs at least 8 points to be recognized. But besides that, size doesn't matter.

What about releasing source first, tutorial later? That way people who want to get their hands dirty, can

Hmm I can do that, however the source is really hard to follow without knowing what is happening... besides the class misses some parts (easy parts though, like returning error values, not the actual network code)..

Thomas

[Thomas]

Here's the current source for the network object, as well as a simple example that learns the XOR operation. It needs masm32 v7 and outputs in vkim's debug window. Inputs and outputs are REAL4 values between 0.0 and 1.0. Inputs are given by an array of real4 values, the 'run' method returns a pointer to an array of real4 values that represent the output.
Note that this is still a beta, it isn't finished yet.

Thomas

[stryker]

coooooooool!!!

[bitRAKE]

Very awesome Thomas!  :alright:

[NaN]

Extremely Impresive Thomas!  Great Work!

:alright:
NaN

[lackluster]

Thomas,

Thank you. I also am intrested in this area of computers.  I wish I could express my admiration better, but I'm lost for words. Seeing your progress in AI gives me a great hope that someday (hopefully in my lifetime) we will acheive our ultimate goal. Please keep it coming, feed my hungry mind, and once again, thanks.

[bitRAKE]

Code:

object MACRO nm:REQ,ty:REQ
_BSS SEGMENT
lp&nm dd ?
_BSS ENDS

&nm MACRO mth:REQ,vars:VARARG
IFB <&vars>
method lp&nm,ty,mth
ELSE
method lp&nm,ty,mth,vars
ENDIF
ENDM
ENDM

object XorNet, NNet

...

XorNet initRandom

.WHILE ebx<2000 ;train 2000 times
rdtsc
mov ecx,eax
xor eax,edx
ror eax,cl
and eax,3*4
XorNet train, inputs[eax], outputs[eax]
inc ebx
.ENDW

Maybe, this could be merged into the newobject macro?  Okay, maybe not - this really only works when dealing with single objects.