Neural nets was: Re[2]: Info on Smalltalk DSLs or Metaprogramming...

Rich Warren rwmlist at gmail.com
Thu Sep 7 10:27:27 UTC 2006


On Sep 6, 2006, at 11:40 PM, Herbert König wrote:

> Hello Rich,
>
>
>>> This (reinforcement learning) is said to be slow. What number of
>>> inputs and how many neurons would such a brain have? How many  
>>> agents?
>
> RW> Regarding reinforcement learning, I've seen this complaint form  
> other
> RW> EE people, and I never really understood it. Perhaps you could  
> give
> RW> me an example. In my opinion, it really depends.
>
> regarding slowness, I'm an absolute beginner on neural nets and only
> read about how much rounds of training it needs. Currently I'm in the
> stage of: "Is my data representation suitable for processing by a
> (which type of?) neural net?"

I was actually asking about reinforcement learning. I've heard other  
EE people complain about reinforcement learning, saying it was too  
slow. However, I've used it to good effect on a few projects. It  
makes me wonder if they're using it for the wrong purpose, but I  
never got a chance to ask anyone for details (or rather, the one time  
I did ask, the EE professor just scoffed at me and said "they're just  
to slow!" without explaining anything).

Reinforcement learning  can, however, be really fussy when it comes  
to training neural nets. Some of the early research showed great  
results training neural nets, but the researchers were already  
experts at implementing neural nets for the given domain, so they  
knew how to represent the data effectively. I think the issue here  
has been the neural nets, not the reinforcement learning.

Neural nets are really fussy. You have to grey-code all incoming  
numerical data. There's no good a priori way to determine how many  
nodes or levels you should have. It's hard to know how much you  
should train it, since they are subject to overfitting.  
Experimentally trying to determine these settings can take a lot of  
time (alter...train...test...iterate).

That's one of the reasons I'm interested in trying out genetic  
algorithms for building neural nets.
If it's possible to have neural nets that automatically grow to  
reasonably fit the problem space, then that would be a big help.

So far, however, I've only played with static neural nets, training  
them using either backpropogation or reinforcement learning.

-Rich- 


More information about the Squeak-dev mailing list