<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
Hi TTY,<br>
<br>
haven't yet found the proper way to addres you :-)<br>
<br>
Back when I got all my knowledge from: Ulrich Nehmzow, Mobile
Robotics (I found it fun) and experiments.<br>
<br>
What is left from that and I maybe wrong:<br>
<ul>
<li>The base element is a neuron (named but unlike the billion
tiny things in the brain). It is simply multiply, add what any
DSP or graphics card can do much better than a PC. Audio guys
like me call it FIR filter. outputOfEachNeuron :=
(lotsOfCoefficients * sameNumberOfInputs) sum. FloatArray does
that quickly.</li>
<li>To get a limited output of the neuron you use a so-called
activation function so output := optput * activationFunction.
Tons of implementations but you could use output := min( 1.0,
(max (output, -1.0))) sorry for python syntax.</li>
<li>Then you take e.g. as many neurons as you have inputs, connect
each neuron to all inputs and get as many outputs as you have
inputs (or any number of outputs you desire) and call this a
layer. Start with random coefficients. This is the one layerd
perceptron which cannot learn the xor function.</li>
<li>Then you start teaching this thingi by putting a gazillion
samples with known desired outputs to the network, compare the
desired output to the actual output and change the coefficients
(all those numberOfInputs * NumberOfNeurons) so that you get
closer to the desired output by gradient descent via a so called
learn rate. Repeat with the next trainings sample. Call this an
epoch. When done start over with the first sample with
diminished learn rate. (next epoch) Use a ton of knowledge on
how to train in batches etc as to make the network find a
general solution vs. learning your samples by heart and produce
utterly stupid outputs on the first unknown sample. Needs inputs
* Neurons * samples * epochs multiplications. Each >>
1000.<br>
</li>
<li>Ooops, so much work for an AI that even cannot learn a
simple XOR?</li>
<li>More helps more so after the first layer put another layer.
All inputs of each neuron of the second layer connected to all
outputs of the first layer. Throw in much more computing power
and use even more layers. Be clever and vary the number of
neurons per layer, choose more complicated connection paths etc.<br>
</li>
<li>All the above is called supervised learning.</li>
</ul>
<p><br>
</p>
<ul>
<li>Here comes the SOFM suggested by Stef:</li>
</ul>
<p><br>
</p>
<ul>
<li>Take Neurons as above which you may organize linearly, as a
torus, as ring or 3D. Take samples w/o knowing the desired
output for the samples.</li>
<li>Put the first sample to all neurons and find out which one has
the strongest output. Define a neighbourhood depending on
topology chosen above. <br>
</li>
<li>Train the neuron with the strongest output and its neighbours
on this sample.</li>
<li>Repeat for all samples. <br>
</li>
<li>Lower the learn rate and take a smaller neighbourhood.</li>
<li>Again train all samples (this is the second epoch) <br>
</li>
<li>Repeat for many epochs.</li>
<li>Make it more complicated like using samples with desired
outputs and put layers of other networks around it. I did.</li>
<li>In my perception the SOFM has fallen out of favour.<br>
</li>
</ul>
<p>This is (a) unsupervised learning (no desired outputs known for
the samples) and (b) similar new samples fire neurons in the same
neighbourhood --> clustering.</p>
<p>Anybody feel free to correct me without significantly
complicating it. I'm no AI expert.<br>
</p>
<p>That is the so called AI. A gigantic gigo machine (garbage in,
garbage out). You don't know what the network learns. So in image
recognition they (Google?) made a heatmap for the pixels of an
image which contributed most to the decision. Come tons of tagged
images scraped on the net. Most horses came from one page, showing
the text 'Copyright horsephotgrapher.com' at the bottom. This was
what the net learned to be the image of a horse. That guy also had
a photo of a cat. Must be a horse. His children .... a horse.
Horses w/o copyright notice ... cats :-)</p>
<p>Huskies were mostly photographed in snow which got to be the main
criterion for huskies. <br>
</p>
<p>Recruiting decisions from biased people were use to train
networks for pre-selecting job applicants. --> Guess who got no
interview.<br>
</p>
<p>So make sure to understand why your network learns what, I have
implemented a SOFM learn watcher to help me with that.</p>
<p><br>
</p>
<p>Cheers,</p>
<p><br>
</p>
<p>Herbert<br>
</p>
<br>
<div class="moz-cite-prefix">Am 08.04.2020 um 17:41 schrieb
gettimothy via Squeak-dev:<br>
</div>
<blockquote type="cite"
cite="mid:1715a741808.fa7362084240.2935624650456320090@zoho.com">
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
<div style="font-family: Verdana, Arial, Helvetica, sans-serif;
font-size: 10pt;">
<div>Hi Herbert,<br>
</div>
<div><br>
</div>
<div>I will get back to you after I read up on Neural Networks. <br>
</div>
<div><br>
</div>
<div>I found this on the web, and it looks interesting and
challenging.<br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><a
href="https://dai.fmph.uniba.sk/courses/NN/haykin.neural-networks.3ed.2009.pdf"
target="_blank" moz-do-not-send="true">https://dai.fmph.uniba.sk/courses/NN/haykin.neural-networks.3ed.2009.pdf</a><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>cheers,<br>
</div>
<div><br>
</div>
<div>t <br>
</div>
<br>
<div data-zbluepencil-ignore="true" style="" class="zmail_extra"><br>
<div id="Zm-_Id_-Sgn1">---- On Sat, 04 Apr 2020 05:53:13 -0400
<b>Herbert König <a class="moz-txt-link-rfc2396E" href="mailto:herbertkoenig@gmx.net"><herbertkoenig@gmx.net></a></b> wrote
----<br>
</div>
<br>
<blockquote style="border-left: 1px solid rgb(204, 204, 204);
padding-left: 6px; margin: 0px 0px 0px 5px;">
<div>Hi,<br>
<br>
I used this for mapping comments from testers to error
causes (soldering, supplier.....) in production of
electronics. I achieved a recognition rate 80% which was
good because the computer looked at 100% of the comments,
humans (= me :-) at 1%. <br>
<br>
It's shaky, and slow and depends a lot on deciding of the
similarity criterion and mapping the text input to floats.
1250 trainings samples, 1200 Neurons 280 inputs (known
words) each take 15 minutes to train. The code is from
2007 and I remember running it over night because hardware
was slower back when. <br>
<br>
Also it was not pure SOFM. Lots of fun thou.<br>
<br>
Anyway gettimothy if you want to give it a try, we can
talk and I can share code.<br>
<br>
Cheers,<br>
<br>
Herbert<br>
<br>
<div class="x_1731772791moz-cite-prefix">Am 03.04.2020 um
23:23 schrieb gettimothy via Squeak-dev:<br>
</div>
<div><br>
</div>
<br>
</div>
<br>
<blockquote>
<div style="font-family: Verdana, Arial, Helvetica,
sans-serif;font-size: 10.0pt;">
<div>That's interesting and I will read up on it as I
get time.<br>
</div>
<div><br>
</div>
<div>It is an interesting problem isn't it?<br>
</div>
<div><br>
</div>
<div>What I might try is having each LatinRoot object
visit each other one and by some heuristic, have them
determine if they are "close" to each other.<br>
</div>
<div><br>
</div>
<div>For giggles, I can randomly assign an integer
weight to each one, and if the absolute value of their
difference is within X, then they are close.<br>
</div>
<div><br>
</div>
<div>That in itself is an interesting problem in itself.
How to efficiently (or not) have 800 objects visit
the other 799 objects.<br>
</div>
<div>How to store the set of "other close objects" in an
object.<br>
</div>
<div><br>
</div>
<div>Then, as other heuristics of "close" are developed,
I can re-use that <br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>Thanks for your reply!<br>
</div>
<br>
<div class="x_1731772791zmail_extra"><br>
<div>---- On Fri, 03 Apr 2020 17:12:31 -0400 <b>Stéphane
Rollandin <a target="_blank"
href="mailto:lecteur@zogotounga.net"
class="x_1731772791moz-txt-link-rfc2396E"
moz-do-not-send="true"><lecteur@zogotounga.net></a></b>
wrote ----<br>
</div>
<br>
<blockquote style="border-left: 1.0px solid
rgb(204,204,204);padding-left: 6.0px;margin: 0.0px
0.0px 0.0px 5.0px;">
<div>Just a wild guess: self-organizing maps?<br>
<br>
<a
href="https://en.wikipedia.org/wiki/Self-organizing_map"
target="_blank" moz-do-not-send="true">https://en.wikipedia.org/wiki/Self-organizing_map</a><br>
<br>
Stef<br>
<br>
<br>
</div>
</blockquote>
</div>
<div><br>
</div>
</div>
<br>
<br>
<pre class="x_1731772791moz-quote-pre">
</pre>
</blockquote>
</blockquote>
</div>
<div><br>
</div>
</div>
<br>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<pre class="moz-quote-pre" wrap="">
</pre>
</blockquote>
<br>
</body>
</html>