... and that has appeared to work fine (except for Character DNU #null).

So Collections-nice.527 is back, and the newly generated mcd works (I deleted the older mcds).

Great job, Nicolas! We're back in business.

- Bert -

On 2013-08-06, at 00:02, Bert Freudenberg <bert at freudenbergs.de> wrote:

> I applied Nicolas' changes to the squeaksource server image and will temporarily un-delete the mcz for testing.
> 
> - Bert - (back from vacation now)
> 
> 
> On 2013-08-05, at 19:48, Frank Shearar <frank.shearar at gmail.com> wrote:
> 
>> I've deleted this commit from the update stream, just so that we
>> unbreak the update stream.
>> 
>> Then we can test the change, and reapply it at a later point.
>> 
>> frank
>> 
>> On 2 August 2013 22:26, Nicolas Cellier
>> <nicolas.cellier.aka.nice at gmail.com> wrote:
>>> Bert, I prepared a patch for a 3.10.2 image in two folds:
>>> - 1) decode UTF8 from mcz sources
>>> - 2) encode mcd and mcz sources in UTF8
>>> I've rapidly tested, and the image seems OK after applying 1) or both.
>>> If only first one is applied, then old images can continue to read new Mcd
>>> from source.squeak.org
>>> I don't know if this is a real requirement
>>> If not, then you can load 2) too. In doubt, a bit of conservatism don't
>>> hurt.
>>> 
>>> As for loading that in a non official 3.11, then you'll have to test a
>>> bit...
>>> 
>>> Cheers
>>> 
>>> Nicolas
>>> 
>>> 
>>> 
>>> 
>>> 2013/8/1 Nicolas Cellier <nicolas.cellier.aka.nice at gmail.com>
>>>> 
>>>> But Bert, where to find a Squeak 3.11alpha image #8824 ?
>>>> The 3.11 stuff on http://ftp.squeak.org/ is declared obsolete, and I did
>>>> find anything but Level Playing Field there...
>>>> The updates (stream) stop at 7159 (3.10) with Monticello-edc.312
>>>> Then squeak4.1 starts at #9957 with Monticello-bp-387...
>>>> 
>>>> 
>>>> 2013/7/31 Nicolas Cellier <nicolas.cellier.aka.nice at gmail.com>
>>>>> 
>>>>> Yes, it's a small change
>>>>> 
>>>>> 
>>>>> 2013/7/30 Bert Freudenberg <bert at freudenbergs.de>
>>>>>> 
>>>>>> FWIW, the server image is a Squeak 3.11alpha image #8824 with
>>>>>> Monticello-bf.356. I'm afraid simply loading a current MC version is not
>>>>>> going to work, but maybe the utf8 changes could be backported?
>>>>>> 
>>>>>> - Bert -
>>>>>> 
>>>>>> On 30.07.2013, at 22:26, Nicolas Cellier
>>>>>> <nicolas.cellier.aka.nice at gmail.com> wrote:
>>>>>> 
>>>>>> To me, the server should try and read in utf8, then fallback to old
>>>>>> encoding like current trunk does.
>>>>>> Then, whether it should continue to write with old encoding or utf8 is a
>>>>>> question of how we want to support serving older images.
>>>>>> Would it be possible to provide a hackish transition: create mcd with
>>>>>> same encoding as mcz?
>>>>>> 
>>>>>> 
>>>>>> 2013/7/30 Nicolas Cellier <nicolas.cellier.aka.nice at gmail.com>
>>>>>>> 
>>>>>>> A possible scenario:
>>>>>>> For some reason (?) the snapshot.bin fails. In that case, the server
>>>>>>> used the sources in .st format.
>>>>>>> But newly produced sources are in utf8 and the server is probably not
>>>>>>> utf8 aware...
>>>>>>> In which case the server re-interprets bad characters for building the
>>>>>>> .mcd and put them in patch.bin stone
>>>>>>> Makes sense?
>>>>>>> 
>>>>>>> Is it possible to backport the utf8 correction to the server ?
>>>>>>> I mean, is it going to cause forward compatibility problems ?
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 2013/7/30 Bert Freudenberg <bert at freudenbergs.de>
>>>>>>>> 
>>>>>>>> If the MCZ works and the MCD does not, then whatever fix we make needs
>>>>>>>> to be applied to our squeaksource server image, which generates the MCDs.
>>>>>>>> 
>>>>>>>> - Bert -
>>>>>>>> 
>>>>>>>> On 2013-07-30, at 14:59, Nicolas Cellier
>>>>>>>> <nicolas.cellier.aka.nice at gmail.com> wrote:
>>>>>>>> 
>>>>>>>> I tried to debug late at night and it seems to come from the patch.bin
>>>>>>>> member of a .mcd
>>>>>>>> I'm not an expert of MC and I do not really know when a .mcd is used
>>>>>>>> vs a .mcz, nor how this binary member is created.
>>>>>>>> But presumably, there is an extra or missing utf8 conversion in some
>>>>>>>> path...
>>>>>>>> I had to stop to get some sleep...
>>>>>>>> I suggest we keep trying a couple of days and solve the problem while
>>>>>>>> under our hands.
>>>>>>>> 
>>>>>>>> Nicolas
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 2013/7/30 Frank Shearar <frank.shearar at gmail.com>
>>>>>>>>> 
>>>>>>>>> On 30 July 2013 02:17, Nicolas Cellier
>>>>>>>>> <nicolas.cellier.aka.nice at gmail.com> wrote:
>>>>>>>>>> Hmm it looks like some UTF8 mess
>>>>>>>>>> I did only change Base64MimeConverter... Did I break it?
>>>>>>>>> 
>>>>>>>>> Yep, I'm afraid so. When it loads it throws a SyntaxError:
>>>>>>>>> 
>>>>>>>>> asCharacter
>>>>>>>>> "Answer the receiver's first character, or '*' if none.
>>>>>>>>> Idiosyncratic, provisional."
>>>>>>>>> 
>>>>>>>>> ^ self size > 0 ifTrue: [self first] ifFalse:[$·Argument expected
>>>>>>>>> ->]
>>>>>>>>> 
>>>>>>>>> Is that supposed to be a middle dot just before "Argument" ? Note how
>>>>>>>>> the original source doesn't show the A-with-caret.
>>>>>>>>> 
>>>>>>>>> frank
>>>>>>>>> 
>>>>>>>>>> 2013/7/30 <commits at source.squeak.org>
>>>>>>>>>> 
>>>>>>>>>>> Nicolas Cellier uploaded a new version of Collections to project
>>>>>>>>>>> The
>>>>>>>>>>> Trunk:
>>>>>>>>>>> http://source.squeak.org/trunk/Collections-nice.527.mcz
>>>>>>>>>>> 
>>>>>>>>>>> ==================== Summary ====================
>>>>>>>>>>> 
>>>>>>>>>>> Name: Collections-nice.527
>>>>>>>>>>> Author: nice
>>>>>>>>>>> Time: 30 July 2013, 2:13:07.506 am
>>>>>>>>>>> UUID: 17e38fb5-76dd-499b-aa6d-bfe07045c28d
>>>>>>>>>>> Ancestors: Collections-fbs.526
>>>>>>>>>>> 
>>>>>>>>>>> Simplify Base64MimeConverter class initialization with
>>>>>>>>>>> Character>>to:
>>>>>>>>>>> 
>>>>>>>>>>> =============== Diff against Collections-fbs.526 ===============
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>>> 
>>> 
>>> 
>>> 
>> 
> 
> - Bert -
> 
> 
> 

- Bert -