[Vm-dev] [Spur] endless recursion in forward become

[Eliot Miranda]

Hi Igor,

On Thu, Jun 23, 2016 at 8:11 AM, Igor Stasenko <siguctua at gmail.com> wrote:

>
>
>
> On 23 June 2016 at 18:00, Eliot Miranda <eliot.miranda at gmail.com> wrote:
>
>>
>> Hi Igor,
>>
>> On Jun 23, 2016, at 7:34 AM, Igor Stasenko <siguctua at gmail.com> wrote:
>>
>> Hi, Eliot,
>>
>> why not trimming the object (if its size allows it) to mark rest of
>> now-forwader object to be a freespace immediately?
>> Like that you can avoid stressing memory too much by doubling amount of
>> memory needed per each forwaded object involved in become operation.
>> And that, of course, if direct contents swap is not possible.
>>
>>
>> Agreed.  Would you like to have a go and I'll review it?  The problem
>> with truncating short objects is that Spur's memory manager cannot
>> represent a free chunk of only 8 bytes in length.  The minimum sized free
>> chunk (& minimum sized object) is 16 bytes.  And any size higher (24,
>> 32, 40 etc) is ok.  So if you want to shorten a 24 byte object to 16 bytes
>> because it's been turned into a forwarder, you can't because that would
>> leave an 8-byte sliver.  So be sure to check that the object is > 24 bytes
>> long before trying to truncate it.
>>
>
> That's why i said - if its size allows it.
> I could give it try.. Of course i will need some more details, to do it
> fast, like updating a first-free object pointer, if needed and what
> limitations there, and things like if it safe to throw this newly-free
> object space into a soup of free chunks regardless of their location in
> memory (old heap/eden etc)
>

See SpurMemoryManager>>shorten:toIndexableSize: for a utility that is close
to what you need.


>
> I would be glad, if you can give me quick brief of these potential
> pitfall, since i don't know much about new spur memory model innards.
>

I think the sliver problem is the only pitfall.  Basically follow
shorten:toIndexableSize: to see what to do with the remnant.  Good luck!

On 22 June 2016 at 16:44, Eliot Miranda <eliot.miranda at gmail.com> wrote:
>>
>>>
>>> Hi Bert,
>>>
>>>
>>> On Jun 22, 2016, at 5:57 AM, Bert Freudenberg <bert at freudenbergs.de>
>>> wrote:
>>>
>>> Hi Eliot,
>>>
>>> the become-forward works now, indeed.
>>>
>>> But why is it different with the swapping become? This still gets into
>>> an endless recursion:
>>>
>>> (ByteString new: 20000000) become: (WideString new: 20000000)
>>>
>>> Why is it creating copies at all?
>>>
>>>
>>> because that's how Spur works.  Become is lazy; the two objects are
>>> turned into forwarders to copies of each other.  If the two objects have
>>> the same byte size as heap objects (rounded up to 8 byte units) their
>>> contents can be exchanged.  If they are not, two copies must be created and
>>> the two originals turned into forwarders to the copies.  Remember the
>>> slides from my Cambridge talk.
>>>
>>> Spur chooses to trade memory (creating the copies) over time (searching
>>> the entire heap looking fir references).  This is the right trade off until
>>> you start becoming objects that are substantial fractions of the entire
>>> heap in size.
>>>
>>> In your case you could avoid the pain simply by making sure the steam
>>> had a wide String as contents in the first place.  I understand that may
>>> not be possible.
>>>
>>>
>>> - Bert -
>>>
>>> On Mon, Jun 20, 2016 at 10:36 PM, Eliot Miranda <eliot.miranda at gmail.com
>>> > wrote:
>>>
>>>>
>>>> Hi Bert,
>>>>
>>>>     it was a regression in argument validation for become introduced in
>>>> fixing argument validation for the one-way copy hash case.  The code was
>>>> erroneously checking that it had space to create copies of the input
>>>> arguments, even though it was a one-way become.  It's a copyHash become and
>>>> that confused it.  It's now fixed.  I'll generate sources and push and new
>>>> builds should appear shortly ;-)
>>>>
>>>> On Thu, Jun 16, 2016 at 5:11 AM, Bert Freudenberg <bert at freudenbergs.de
>>>> > wrote:
>>>>
>>>>>
>>>>> I'm reading a 20MB text file. At some point it tries to convert the 20
>>>>> MB ByteString into an 80 MB WideString using forward become. This fails
>>>>> resulting in an endless loop. The primitive failure code for
>>>>> elementsForwardIdentityTo: (primitive 72) is  #'insufficient object memory'
>>>>> and it tries again after growing:
>>>>>
>>>>> ec == #'insufficient object memory' ifTrue:
>>>>> [Smalltalk garbageCollect < 1048576 ifTrue:
>>>>> [Smalltalk growMemoryByAtLeast: 1048576].
>>>>> ^self elementsForwardIdentityTo: otherArray].
>>>>>
>>>>> but the growMemoryByAtLeast: does not actually grow the memory:
>>>>>
>>>>> {Smalltalk garbageCollect.
>>>>> Smalltalk growMemoryByAtLeast: 1048576.
>>>>> Smalltalk garbageCollect.}
>>>>>
>>>>> ==>  #(58576608 16777216 58576608)
>>>>>
>>>>> 58 MB are not enough, but it doesn't grow any further.
>>>>>
>>>>> The question is, why does it try to allocate a new object at all? The
>>>>> WideString exists already. I thought Spur would simply install a forwarder?
>>>>>
>>>>> To reproduce, execute
>>>>>
>>>>> (ByteString new: 20000000) writeStream nextPut: (Character value:
>>>>> 128169)
>>>>>
>>>>> You probably want to follow that with a user interrupt (Cmd-period).
>>>>>
>>>>> This happens using the latest Spur VM from github (r201606160944)
>>>>>
>>>>> - Bert -
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> _,,,^..^,,,_
>>>> best, Eliot
>>>>
>>>
>>>
>>>
>>
>>
>> --
>> Best regards,
>> Igor Stasenko.
>>
>>
>>
>
>
> --
> Best regards,
> Igor Stasenko.
>
>


-- 
_,,,^..^,,,_
best, Eliot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20160623/43713b88/attachment.htm