Are you randomizing the inputs when you train your NN's?


Yes, it seems to give better results than initializing with a constant. The weights are initialized with values between -1 and 1 though, so not completely random FP values, because those values can change faster during the training (iirc).
Posted on 2002-01-19 06:32:58 by Thomas
I wasn't referring to the initial conditions, but your training set. To prevent the NN from developing cohesion between training items (ex. more apt to detect a C after D, than a C after A).
Posted on 2002-01-19 06:45:02 by bitRAKE
I don't randomize the inputs but I train the network with a set: i.e. not first training A 300 times, then B 300 times, but 300 times. I didn't need to randomize them as the results where good but it could give some improvement I guess.

Thomas
Posted on 2002-01-19 06:51:26 by Thomas
It will take longer to train, too. But the results should be better, IMHO.
Posted on 2002-01-19 07:03:54 by bitRAKE
This version is slightly optimized: mainly moving the fdiv's outside the loop, and getting rid of one of the fmul's (horray!). No need to optimize further, IMO.
; arrDest    Pointer to buffer of size lenDest*4.  This buffer will hold an

; array of REAL4 values returned from proceedure.
; arrSrc Pointer to buffer of size lenSrc*4. This buffer holds lenSrc REAL4
; values to be averaged by the proceedure.
;
NonUniformAverages PROC uses esi edi, arrDest:DWORD, arrSrc:DWORD, lenDest:DWORD, lenSrc:DWORD
LOCAL fTemp:QWORD

mov edx,lenDest
mov esi,arrSrc
mov edi,arrDest
cmp edx,lenSrc
jge Error

fild lenDest
fidiv lenSrc

xor eax,eax
and DWORD PTR [fTemp+4],0

fld st
; 1/2^32
fdivr fpc(0.000000000232830643653869628906250)

div lenSrc

fldz ; init

Main:
add edx,eax
jc @F

fadd REAL4 PTR [esi]
add esi,4
jmp Main
@@:
fadd REAL4 PTR [esi]

mov DWORD PTR [fTemp],edx
fild fTemp
fmul st(0),st(2)
fmul REAL4 PTR [esi]
fxch st(1)

fsub st(0),st(1)
fmul st(0),st(3)
fstp REAL4 PTR [edi]

add esi,4
add edi,4
dec lenDest
jne Main

fstp st(0)
fstp st(0)
fstp st(0)
ret

Error:
jne @F
; this isn't exactly an error ;)
mov ecx,edx
rep movsd
or eax,-1
ret

@@: xor eax,eax ; error checking isn't a bad thing
ret
NonUniformAverages ENDP
Posted on 2002-01-21 22:39:21 by bitRAKE