You're making the mistake of thinking it understands any of it... it
doesn't.  There isn't a KB in the sense you're probably expecting to be
there, only the meaning it's extracted in terms of the relationships of the
text (in the form of encoded symbols) ingested.  It only appeared to
understand how Smalltalk strings work because its training data included
plenty of Smalltalk code, in which string manipulation is abundant.  These
models are trained on hundreds of gigabytes of text from the usual sources
(wikipedia, stackoverflow, open source books, reddit etc.) around the
Internet.

Sure it's doing far more than simply regurgitating the text it's seen
before back to you, but it's a whole lot less than any actual
understanding.  Think of it as ELIZA NG++.  Its output often has a sort of
fractals with text quality in that it's spitting out something that looks
nice (i.e. reasonable) since that's what it was rewarded for doing during
training.  I must admit I find the dynamic style transfer endlessly
entertaining:

me: 'hello' copyFrom: 3 to: 4 would return the string 'l'... answer in the
style of shakespeare

gpt: Aye, 'tis true, for the 'hello' string hath a length of five
characters, and from index three to index four is the specified range, thus
the resulting substring shall contain only the character at index three,
which is the letter 'l'.
-------

me: 'hello' copyFrom: 3 to: 4 would return the string 'l'... answer in the
style of yoda

gpt: 'l', the string returning, 'hello' copyFrom: 3 to: 4 would.
-------

me: 'hello' copyFrom: 3 to: 4 would return the string 'l'... answer in the
style of edgar allan poe

gpt: Ah, 'tis a question of such simplicity, yet a riddle of such intrigue.
'Tis true that a single letter may be copied from one point to another.
'Tis that single letter which stands between two points, with the answer to
this conundrum standing between them. Aye, 'twould be precisely that single
letter 'l' that would be returned.

On Thu, Jan 12, 2023 at 11:07 AM Lauren P <drurowin at gmail.com> wrote:

> Chris, all,
>
> Very interesting.  Even while looking for the inconsistencies you pointed
> out, it was still very hard to follow since each answer was so verbose.
>
> My biggest surprise was that it knew to treat '12345' as a Smalltalk
> from-1 string.
>
> And then proceeded to return 2 characters even after stating the to: term
> is inclusive.  Something in it's KB was really insistent on the to: term
> being exclusive, even though the reasoning engine got the correct answer
> while showing its work.
>
> I had a similar experience with the art AI.  My dad said he got some
> really cool results from it, so I tried asking it for a lemur playing chess
> with a penguin and could not get a penguin at all.  All of my queries were
> so disappointingly met, and I made a lot.
>
> I guess people have forgotten AI winter is still going strong.
>
> On Mon, Jan 9, 2023, 13:46 tim Rowledge <tim at rowledge.org> wrote:
>
>> Hey, maybe I should ask it about my limits problem :-)
>>
>> > On 2023-01-09, at 12:09 PM, Craig Latta <craig at blackpagedigital.com>
>> wrote:
>> >
>> >
>> >     Tim writes:
>> >
>> > > Unfortunately I suspect we are going to see a lot of 'content'
>> > > produced by this system that will be treated as Serious Information To
>> > > Base Policy On, whilst actually being as valuable as "this bloke down
>> > > the pub told me..."
>> >
>> >     I think there's a huge market opportunity for signing content and
>> verifying its provenance, in ways which are usable and well-known. We
>> haven't been able to trust mediated experiences for a very long time; now
>> that more is digital, we can use tools to identify those who will vouch for
>> it.
>> >
>> >     Also, the current stuff being passed off as "AI" is far from it.
>> There's no comprehension or understanding, only glorified brute-force
>> auto-completion, and social engineering via human-written code
>> (backpedaling caveats, mostly). I enjoyed Ezra Klein's recent interview
>> with Gary Marcus about it:
>> >
>> >     https://overcast.fm/+oiPWgxJVM
>> >
>> >
>> > -C
>> >
>> > --
>> > Craig Latta        :: research computer scientist ::
>> > Black Page Digital ::     Berkeley, California    ::
>> > 663137D7940BF5C0AF ::    C1349FB2ADA32C4D5314CE   ::
>> >
>> >
>> >
>>
>>
>> tim
>> --
>> tim Rowledge; tim at rowledge.org; http://www.rowledge.org/tim
>> Strange OpCodes: SLTMDL: Shift Left, Test Mask and Dim the Lights
>>
>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20230114/0ce502cd/attachment.html>