Jecel was explaining how the core of the system (i.e. the 'GPT' part, which
is the main neural net) worked.  There's also the 'Chat' part, which is the
front end where they'd handle things like incorporating session state etc.
So it has a short-term memory of sorts but has no way to persist it beyond
an individual session currently, AFAIK.


On Sat, Jan 21, 2023 at 7:56 PM Yoshiki Ohshima <Yoshiki.Ohshima at acm.org>
wrote:

> ChatGPT
>
> On Fri, Jan 13, 2023 at 5:52 PM Jecel Assumpcao Jr <jecel at merlintec.com>
> wrote:
>
>> Chris Muller wrote on Fri, 13 Jan 2023 15:41:02 -0600
>> > > My biggest surprise was that it knew to treat '12345' as a Smalltalk
>> from-1 string.
>> > >
>> > > And then proceeded to return 2 characters even after stating the to:
>> term is inclusive.
>> > > Something in it's KB was really insistent on the to: term being
>> exclusive, even though
>> > > the reasoning engine got the correct answer while showing its work.
>> >
>> >
>> > Yes, exactly!  And I was trying to see if it could "realize" that
>> glitch only by asking it to
>> > scrutinize the inconsistent part of its own responses.  It didn't
>> work.  Maybe I gave up
>> > too quickly or didn't ask quite the right questions (I'm no AI
>> researcher), but my take-away
>> > is that it doesn't grok what it's saying.
>>
>> These engines have to very different phases: training and inference.
>>
>> For training you use pairs of inputs / desired outputs and adjust the
>> numbers in these huge matrices. You do that for an absurd number of
>> pairs and all this knowledge gets encoded somehow in the contents of
>> these matrices.
>>
>> For inference the user puts in a novel input and after passing through
>> the matrices you get the output. The cool thing about artificial neural
>> networks is that the output tends to make sense even for inputs that
>> were not in the training set.
>
>
>> We have been doing this since the 1950s, but the level of hardware
>> advances has reached a threashold in the last decade to allow absurdly
>> large "models". In addition, the Internet has given us access to vast
>> amounts to training data. The third change was the development of new
>> ways to connect together these matrices in the past half decade to
>> overcome many of the limitations of simple neural networks.
>>
>>
>
> I agree with the basic part of this but ChatGPT being a chat engine, it
> has a bit more interesting features.
>
> First I asked: "I'd like to you to act like that a word gegehek means a
> male person who has a sister who is two years older"
>
> then ChatGPT said: "Sure, I can understand your use of the word "gegehek"
> in that context, but please note that it is not a widely recognized or
> accepted term in the English language. In general conversation, it would be
> more common to say "a man with an older sister" rather than using a made-up
> word like "gegehek.""
>
> and then I said: "When I was in the first year student at a high school,
> my sister was in the third year at the same high school."
>
> and ChatGPT said: I see, so in that context you would be considered a
> "gegehek" as you are a male person with a sister who is two years older
> than you and both of you were in the same high school. Again, please note
> that this is not a widely recognized or accepted term in the English
> language.
>
> -----
> When I say "i'd like to remember that..." or such to effectively mean the
> samething, it pretends that it cannot remember, but there are prompts you
> can make it remember things, according to some reports, in the last 4096
> characters. On top of something that might be a very static network, it has
> this layer. Handling neologisms was something considered hard, and I'd say
> that it still is not understanding it but on the surface it appears to be
> doing something.
>
>
> --
> -- Yoshiki
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20230122/1eac2cc0/attachment.html>