Chris Muller wrote on Fri, 13 Jan 2023 15:41:02 -0600
> > My biggest surprise was that it knew to treat '12345' as a Smalltalk from-1 string.
> > 
> > And then proceeded to return 2 characters even after stating the to: term is inclusive.
> > Something in it's KB was really insistent on the to: term being exclusive, even though
> > the reasoning engine got the correct answer while showing its work.
> 
> 
> Yes, exactly!  And I was trying to see if it could "realize" that glitch only by asking it to
> scrutinize the inconsistent part of its own responses.  It didn't work.  Maybe I gave up
> too quickly or didn't ask quite the right questions (I'm no AI researcher), but my take-away
> is that it doesn't grok what it's saying.

These engines have to very different phases: training and inference.

For training you use pairs of inputs / desired outputs and adjust the
numbers in these huge matrices. You do that for an absurd number of
pairs and all this knowledge gets encoded somehow in the contents of
these matrices.

For inference the user puts in a novel input and after passing through
the matrices you get the output. The cool thing about artificial neural
networks is that the output tends to make sense even for inputs that
were not in the training set.

We have been doing this since the 1950s, but the level of hardware
advances has reached a threashold in the last decade to allow absurdly
large "models". In addition, the Internet has given us access to vast
amounts to training data. The third change was the development of new
ways to connect together these matrices in the past half decade to
overcome many of the limitations of simple neural networks.

Back to ChatGPT - its Smalltalk knowledge is encoded in the contents of
the matrices I mentioned during the training phase. Thanks to the novel
architecture I mentioned, when you use it in the inference phase it can
hold "session information" to allow its responses to be coherent with
what you and it have said before. But it can't learn during the
inference phase - the matrices are not changed by your chat.

Humans learn from very small training sets - you don't have to show a
child thousands of pictures of cats before they understand what a cat
is. Humans also don't have separate training and inference phases.
Having an AI with these features is a simple matter of programming. We
might see real progress in the next few years. But we are not there yet.

-- Jecel