From eliot.miranda at gmail.com Tue Dec 1 00:39:22 2015 From: eliot.miranda at gmail.com (Eliot Miranda) Date: Tue Dec 1 00:39:25 2015 Subject: [squeak-dev] Core developers (was: The Inbox: Traits-pre.307.mcz) In-Reply-To: <20151129195931.GA92720@shell.msen.com> References: <20151120135830.295FD60613@mail2.hpi.uni-potsdam.de> <1448028452837.42806@hpi.de> <4A03255F-A50C-4DE4-98DC-4F4332508800@rowledge.org> <8C3DFFEB-F8F4-45ED-B754-44597C00FBA2@gmail.com> <56543AD3.1020503@zogotounga.net> <56581F12.7040303@zogotounga.net> <5658213C.8060805@zogotounga.net> <20151129195931.GA92720@shell.msen.com> Message-ID: On Sun, Nov 29, 2015 at 11:59 AM, David T. Lewis wrote: > On Fri, Nov 27, 2015 at 10:24:12AM +0100, St??phane Rollandin wrote: > > > > On Thu, Nov 26, 2015 at 01:04:39PM +0100, karl ramberg wrote: > > > > > > On Tue, Nov 24, 2015 at 11:24 AM, St??phane Rollandin < > lecteur@zogotounga.net wrote: > > > > > > > > I'm so convinced nobody cares about what I do that stopped long ago > > > > sending fixes to bug I encounters: I just fix them in my code. For > example, > > > > the Saucers game I did has a much faster way of handling morphs, > down to > > > > modified #addMorph: logic. This could be leveraged, if someone > looked at > > > > the code. But you cannot command interest. > > > > > > > > > > Are you not interested becoming a core developer so you can contribute > > > directly to trunk ? > > > I would guess you are qualified, based on the stuff you have > produced... > > > > > > > Why not ? > > > > By which I mean: "yes, I'd be honored". Which is more proper I guess... > > > > > Hi Stef, > > It would be great to have you contributing to trunk. Can I suggest that > you start by creating an account for yourself on source.squeak.org, and > maybe commit a few of your fixes to the inbox? You have done some > remarkable > work with Squeak and muO, and it would be really nice to see you involved > as a core developer also. > +1 !! > > Dave > > > -- _,,,^..^,,,_ best, Eliot -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151130/998284d7/attachment.htm From lecteur at zogotounga.net Tue Dec 1 09:49:09 2015 From: lecteur at zogotounga.net (=?UTF-8?Q?St=c3=a9phane_Rollandin?=) Date: Tue Dec 1 09:49:07 2015 Subject: [squeak-dev] Re: Core developers In-Reply-To: <20151129195931.GA92720@shell.msen.com> References: <20151120135830.295FD60613@mail2.hpi.uni-potsdam.de> <1448028452837.42806@hpi.de> <4A03255F-A50C-4DE4-98DC-4F4332508800@rowledge.org> <8C3DFFEB-F8F4-45ED-B754-44597C00FBA2@gmail.com> <56543AD3.1020503@zogotounga.net> <56581F12.7040303@zogotounga.net> <5658213C.8060805@zogotounga.net> <20151129195931.GA92720@shell.msen.com> Message-ID: <565D6D15.107@zogotounga.net> > It would be great to have you contributing to trunk. Can I suggest that > you start by creating an account for yourself on source.squeak.org, and > maybe commit a few of your fixes to the inbox? Ok, I now have an account at source.squeak.org (initials spfa) Stef From bert at freudenbergs.de Tue Dec 1 11:21:27 2015 From: bert at freudenbergs.de (Bert Freudenberg) Date: Tue Dec 1 11:21:31 2015 Subject: [squeak-dev] Update Squeak on iOS In-Reply-To: References: <6B9404C7-3652-4235-AD85-49E42429087F@mac.com> Message-ID: <7375F536-2C3B-4FBA-AFB7-738E621D86BB@freudenbergs.de> Skipped content of type multipart/alternative-------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4115 bytes Desc: not available Url : http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151201/e2872105/smime.bin From robert.w.withers at gmail.com Tue Dec 1 12:37:44 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Tue Dec 1 12:37:53 2015 Subject: [squeak-dev] evolutions of squeakelib & crypto (Reed Solomon) Message-ID: <565D9498.3030104@gmail.com> This version of the Cryptography package has beginnings of ReedSolomon encoding. The rendezvous is separated into an OperationProtocol and a StartupProtocol and the message pipeline of the session layer of SqueakElib has been completely rewritten and LayeredProtocols have been folded in. Please look at ElibVatTPDataTestCase for an example. Cryptography: https://www.dropbox.com/s/z9dux2goimej79e/Cryptography-rww.40.mcz?dl=0 SqueakElib: https://www.dropbox.com/s/5yvkl3df89uyjnh/SqueakElib-rww.200.mcz?dl=0 In the spirit of #GivingTuesday, Robert From robert.w.withers at gmail.com Tue Dec 1 16:44:03 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Tue Dec 1 16:44:09 2015 Subject: [squeak-dev] Re: evolutions of squeakelib & crypto (Reed Solomon) In-Reply-To: <565D9498.3030104@gmail.com> References: <565D9498.3030104@gmail.com> Message-ID: <565DCE53.90501@gmail.com> Here is a newer version (201) with a protocol change to remove all tcpIds from the rendezvous. https://www.dropbox.com/s/7mgfglwu3e4bf4j/SqueakElib-rww.201.mcz?dl=0 - Robert On 12/01/2015 07:37 AM, Robert Withers wrote: > This version of the Cryptography package has beginnings of ReedSolomon > encoding. > > The rendezvous is separated into an OperationProtocol and a > StartupProtocol and the message pipeline of the session layer of > SqueakElib has been completely rewritten and LayeredProtocols have > been folded in. Please look at ElibVatTPDataTestCase for an example. > > Cryptography: > https://www.dropbox.com/s/z9dux2goimej79e/Cryptography-rww.40.mcz?dl=0 > SqueakElib: > https://www.dropbox.com/s/5yvkl3df89uyjnh/SqueakElib-rww.200.mcz?dl=0 > > In the spirit of #GivingTuesday, > Robert From robert.w.withers at gmail.com Tue Dec 1 19:23:32 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Tue Dec 1 19:23:39 2015 Subject: [squeak-dev] Re: evolutions of squeakelib & crypto (Reed Solomon) In-Reply-To: <565DCE53.90501@gmail.com> References: <565D9498.3030104@gmail.com> <565DCE53.90501@gmail.com> Message-ID: <565DF3B4.8020100@gmail.com> Another version fixing empty payloads... https://www.dropbox.com/s/wjhbopwabbrszjn/SqueakElib-rww.202.mcz?dl=0 thank you, - Robert On 12/01/2015 11:44 AM, Robert Withers wrote: > Here is a newer version (201) with a protocol change to remove all > tcpIds from the rendezvous. > > https://www.dropbox.com/s/7mgfglwu3e4bf4j/SqueakElib-rww.201.mcz?dl=0 > > - Robert > > On 12/01/2015 07:37 AM, Robert Withers wrote: >> This version of the Cryptography package has beginnings of >> ReedSolomon encoding. >> >> The rendezvous is separated into an OperationProtocol and a >> StartupProtocol and the message pipeline of the session layer of >> SqueakElib has been completely rewritten and LayeredProtocols have >> been folded in. Please look at ElibVatTPDataTestCase for an example. >> >> Cryptography: >> https://www.dropbox.com/s/z9dux2goimej79e/Cryptography-rww.40.mcz?dl=0 >> SqueakElib: >> https://www.dropbox.com/s/5yvkl3df89uyjnh/SqueakElib-rww.200.mcz?dl=0 >> >> In the spirit of #GivingTuesday, >> Robert > From tim at rowledge.org Wed Dec 2 02:30:21 2015 From: tim at rowledge.org (tim Rowledge) Date: Wed Dec 2 02:30:16 2015 Subject: [squeak-dev] mouse enter/move/leave event confusion Message-ID: <4973AB59-FF96-41B1-AE15-904AAE8752D9@rowledge.org> I?m trying to provide the appearance of a cursor that is gridded in a paint program; the idea is that the tool cursor should jump only to positions on the grid and when the bitmap is scaled to say 4X that would be every 4th pixel position on the screen. Since making the actual HandMorph & real cursor do gridding seemed a bit of a heavyweight thing to impose on everything else, I?m attempting to do it by handling mousemove events within my painting canvas morph. It *almost* works but there is something very odd going on at far too frequent occasions. So far as I can currently tell the event handling code up in Morph & HandMorph is sometimes deciding that the mouse has left my morph and thus sends me a mouseleave event instead of a a move. Then as soon as the cursor moves, I get a new mouseenter and more moves until whatever it is upsets things again. There?s some unpleasantly complex code up at MouseOverHandler>processMouseOver: for example that appears to be in the middle of my problem - best I can tell is that ?leftMorphs? is somehow including my morph when the mouse is still right there in the middle of it. My basic scheme is to create a suitable ImageMorph on the mouseEnter, change its position with each mouseMove and then delete it on mouseLeave. I have implemented handlesMouseMove: to always return true, so I seem to get all the move events. Any idea about why I get the dud mouseLeave events? Has anyone else done a scaled-up-grid-cursor thingy? tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim "How many Pak Protectors does it take to change a lightbulb?? "Only one, but the lightbulb has to smell right." From karlramberg at gmail.com Wed Dec 2 07:59:10 2015 From: karlramberg at gmail.com (karl ramberg) Date: Wed Dec 2 07:59:13 2015 Subject: [squeak-dev] mouse enter/move/leave event confusion In-Reply-To: <4973AB59-FF96-41B1-AE15-904AAE8752D9@rowledge.org> References: <4973AB59-FF96-41B1-AE15-904AAE8752D9@rowledge.org> Message-ID: Do you have the code available so I can look at it? Best, Karl On Wed, Dec 2, 2015 at 3:30 AM, tim Rowledge wrote: > I?m trying to provide the appearance of a cursor that is gridded in a > paint program; the idea is that the tool cursor should jump only to > positions on the grid and when the bitmap is scaled to say 4X that would be > every 4th pixel position on the screen. > > Since making the actual HandMorph & real cursor do gridding seemed a bit > of a heavyweight thing to impose on everything else, I?m attempting to do > it by handling mousemove events within my painting canvas morph. It > *almost* works but there is something very odd going on at far too frequent > occasions. > > So far as I can currently tell the event handling code up in Morph & > HandMorph is sometimes deciding that the mouse has left my morph and thus > sends me a mouseleave event instead of a a move. Then as soon as the cursor > moves, I get a new mouseenter and more moves until whatever it is upsets > things again. There?s some unpleasantly complex code up at > MouseOverHandler>processMouseOver: for example that appears to be in the > middle of my problem - best I can tell is that ?leftMorphs? is somehow > including my morph when the mouse is still right there in the middle of it. > > My basic scheme is to create a suitable ImageMorph on the mouseEnter, > change its position with each mouseMove and then delete it on mouseLeave. > I have implemented handlesMouseMove: to always return true, so I seem to > get all the move events. > > Any idea about why I get the dud mouseLeave events? Has anyone else done a > scaled-up-grid-cursor thingy? > > tim > -- > tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim > "How many Pak Protectors does it take to change a lightbulb?? > "Only one, but the lightbulb has to smell right." > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151202/c6c2f33f/attachment.htm From Marcel.Taeumel at hpi.de Wed Dec 2 10:21:02 2015 From: Marcel.Taeumel at hpi.de (marcel.taeumel) Date: Wed Dec 2 10:37:30 2015 Subject: [squeak-dev] Re: mouse enter/move/leave event confusion In-Reply-To: <4973AB59-FF96-41B1-AE15-904AAE8752D9@rowledge.org> References: <4973AB59-FF96-41B1-AE15-904AAE8752D9@rowledge.org> Message-ID: <1449051662880-4864703.post@n4.nabble.com> Hi Tim, yes, mouse-leave and mouse-enter events occur more than what a user would expect. Also, when you speed up your mouse movements there is a chance that you will not get a mouse-leave event at all. Find attached some code that works fine. :) Without gridding but that could easily be added. Sandbox.st Best, Marcel -- View this message in context: http://forum.world.st/mouse-enter-move-leave-event-confusion-tp4864651p4864703.html Sent from the Squeak - Dev mailing list archive at Nabble.com. From karlramberg at gmail.com Wed Dec 2 13:32:43 2015 From: karlramberg at gmail.com (karl ramberg) Date: Wed Dec 2 13:32:45 2015 Subject: [squeak-dev] The Trunk: Universes-kfr.49.mcz In-Reply-To: References: <213BA91F-07E4-4889-9E97-630A6A10493F@gmx.de> Message-ID: On Sun, Nov 29, 2015 at 4:03 AM, Levente Uzonyi wrote: > Sadly the update process will break if your image has Universes-mt.48 in > it, because it'll try to load Universes-kfr.49 after it has been removed by > Squeak-Version. I think removing Universes from the update map will help. Hi, what can I do to fix this ? I'm not sure how to make a update map Best, Karl > > > Levente > > > On Fri, 27 Nov 2015, Edgar De Cleene wrote: > > >> On Nov 27, 2015, at 10:22, H. Hirzel >> wrote: >> >> Update through the SqueakMenu up to update number 15555 is fine. Seems >> to work fine at this end. >> >> Universes are no longer in the image. Only and Installer class is left. >> >> ?Hannes >> >> >> Very good news. >> Thanks Karl >> >> >> > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151202/2d4ec96d/attachment.htm From tim at rowledge.org Wed Dec 2 19:11:40 2015 From: tim at rowledge.org (tim Rowledge) Date: Wed Dec 2 19:11:33 2015 Subject: [squeak-dev] mouse enter/move/leave event confusion In-Reply-To: <1449051662880-4864703.post@n4.nabble.com> References: <4973AB59-FF96-41B1-AE15-904AAE8752D9@rowledge.org> <1449051662880-4864703.post@n4.nabble.com> Message-ID: <655B52F2-8736-476F-9B62-4A279AF46ECE@rowledge.org> > On 02-12-2015, at 2:21 AM, marcel.taeumel wrote: > yes, mouse-leave and mouse-enter events occur more than what a user would > expect. Also, when you speed up your mouse movements there is a chance that > you will not get a mouse-leave event at all. > Oh dear. That really shouldn?t happen. > Find attached some code that works fine. :) Without gridding but that could > easily be added. Yah, that?s almost exactly what my code looks like wrt the events etc. The only difference is that my ?cursor? isn?t a submorph of the ?playfield?. I wonder if that would help at all... tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Performance is easier to add than clarity. From commits at source.squeak.org Wed Dec 2 23:38:26 2015 From: commits at source.squeak.org (commits@source.squeak.org) Date: Wed Dec 2 23:38:28 2015 Subject: [squeak-dev] The Trunk: Tools-eem.657.mcz Message-ID: Eliot Miranda uploaded a new version of Tools to project The Trunk: http://source.squeak.org/trunk/Tools-eem.657.mcz ==================== Summary ==================== Name: Tools-eem.657 Author: eem Time: 2 December 2015, 3:38:00.319 pm UUID: 8ffda57b-575c-4398-8067-5e310d2fde82 Ancestors: Tools-kfr.656 Provide local senders... in the system browser which will browse methods that reference a selector or a literal within the selected class's hierarchy. Modify selection in MessageNames so that if one selects a different selector and an implementation exists in the same class as the current method, select that method. Provide a canonical message name for CompiledMethod for creating methods from exceptions in the debugger. =============== Diff against Tools-kfr.656 =============== Item was changed: ----- Method: Browser>>shiftedClassListMenu: (in category 'class functions') ----- shiftedClassListMenu: aMenu "Set up the menu to apply to the receiver's class list when the shift key is down" + ^ aMenu + addList: #( - - ^ aMenu addList: #( - + ('local senders...' browseLocalSenders 'browse senders local to this class') ('unsent methods' browseUnusedMethods 'browse all methods defined by this class that have no senders') ('unreferenced inst vars' showUnreferencedInstVars 'show a list of all instance variables that are not referenced in methods') ('unreferenced class vars' showUnreferencedClassVars 'show a list of all class variables that are not referenced in methods') ('subclass template' makeNewSubclass 'put a template into the code pane for defining of a subclass of this class') - ('sample instance' makeSampleInstance 'give me a sample instance of this class, if possible') ('inspect instances' inspectInstances 'open an inspector on all the extant instances of this class') ('inspect subinstances' inspectSubInstances 'open an inspector on all the extant instances of this class and of all of its subclasses') - ('add all meths to current chgs' addAllMethodsToCurrentChangeSet 'place all the methods defined by this class into the current change set') ('create inst var accessors' createInstVarAccessors 'compile instance-variable access methods for any instance variables that do not yet have them')); + yourself! - yourself! Item was added: + ----- Method: CompiledMethod>>canonicalArgumentName (in category '*Tools-Debugger') ----- + canonicalArgumentName + ^ 'CompiledMethod'! Item was changed: ----- Method: MessageNames>>selectorListIndex: (in category 'selector list') ----- selectorListIndex: anInteger "Set the selectorListIndex as specified, and propagate consequences" + | methodClass index | + methodClass := currentCompiledMethod ifNotNil: [currentCompiledMethod methodClass]. - selectorListIndex := anInteger. self changed: #selectorListIndex. messageList := self computeMessageList. self changed: #messageList. + methodClass ifNotNil: [index := messageList findFirst: [:methodRef| methodRef actualClass = methodClass]]. + + "If a method of the same class exists, select that, otherwise select the first message if any." + self messageListIndex: (index ifNil: [1 min: messageList size])! - - "Select the first message if any." - self messageListIndex: (1 min: messageList size).! Item was added: + ----- Method: StringHolder>>browseLocalSenders (in category '*Tools') ----- + browseLocalSenders + self selectedClass ifNotNil: + [:cls| | token | + token := UIManager default request: 'browse for selector or literal'. + token isEmpty ifTrue: [^self]. + Scanner new + typedScan: token + do: [:scannedToken :tokenType| + token := tokenType = #word + ifTrue: + [(Symbol findInterned: token) + ifNil: [^UIManager default inform: 'no such selector: ', token] + ifNotNil: [:symbol| symbol]] + ifFalse: + [scannedToken]. + self systemNavigation browseAllCallsOn: token localTo: cls]]! From huw.softdesigns at gmail.com Thu Dec 3 01:13:49 2015 From: huw.softdesigns at gmail.com (Huw Lloyd) Date: Thu Dec 3 01:13:52 2015 Subject: [squeak-dev] Mac release broken? ClassBinding>>objectForDataStream Message-ID: For both 4.6 & 5.0 on the mac, saving a project generates the following error: "NotImplemented: ClassBinding or a superclass should implement objectForDataStream." The comments in the method suggest this shouldn't have been released to a stable version: "It's not yet clear how serialization should work in the presence of environments". Problem 2 is that if I then "abandon" the error messge, the workspace is then flooded with "MessageNotUnderstood" error boxes. Is "environments" something experimental that should be switched off? Best, Huw -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151203/380656c7/attachment.htm From huw.softdesigns at gmail.com Thu Dec 3 01:25:37 2015 From: huw.softdesigns at gmail.com (Huw Lloyd) Date: Thu Dec 3 01:25:39 2015 Subject: [squeak-dev] Re: Mac release broken? ClassBinding>>objectForDataStream In-Reply-To: References: Message-ID: Arbitrarily picking an earlier version ... 4.1 does not have this issue/bug. Any quick fixes for this? I'm interested in using this in conjunction with aida. Best, Huw On 3 December 2015 at 01:13, Huw Lloyd wrote: > For both 4.6 & 5.0 on the mac, saving a project generates the following > error: "NotImplemented: ClassBinding or a superclass should implement > objectForDataStream." > > The comments in the method suggest this shouldn't have been released to a > stable version: "It's not yet clear how serialization should work in the > presence of environments". > > Problem 2 is that if I then "abandon" the error messge, the workspace is > then flooded with "MessageNotUnderstood" error boxes. > > Is "environments" something experimental that should be switched off? > > Best, > Huw > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151203/c917102b/attachment.htm From lewis at mail.msen.com Thu Dec 3 02:00:34 2015 From: lewis at mail.msen.com (David T. Lewis) Date: Thu Dec 3 02:00:36 2015 Subject: [squeak-dev] Re: Mac release broken? ClassBinding>>objectForDataStream In-Reply-To: References: Message-ID: <20151203020034.GA32593@shell.msen.com> Huw, The timing of your question is perfect :-) There are some very recent updates (in the last week or so) to the Squeak trunk development stream that address this problem. This is work in progress, but in a nutshell, the latest trunk fixes should resolve the problem for 4.6 images, but there is work still to be done in order to make it work in 5.0 Spur. We still need to get these latest updates into the standard update stream for squeak46, but as soon as that is complete it should work for those images. I am less certain of the timing for 5.0 images. Dave On Thu, Dec 03, 2015 at 01:25:37AM +0000, Huw Lloyd wrote: > Arbitrarily picking an earlier version ... 4.1 does not have this issue/bug. > > Any quick fixes for this? I'm interested in using this in conjunction with > aida. > > Best, > Huw > > > On 3 December 2015 at 01:13, Huw Lloyd wrote: > > > For both 4.6 & 5.0 on the mac, saving a project generates the following > > error: "NotImplemented: ClassBinding or a superclass should implement > > objectForDataStream." > > > > The comments in the method suggest this shouldn't have been released to a > > stable version: "It's not yet clear how serialization should work in the > > presence of environments". > > > > Problem 2 is that if I then "abandon" the error messge, the workspace is > > then flooded with "MessageNotUnderstood" error boxes. > > > > Is "environments" something experimental that should be switched off? > > > > Best, > > Huw > > > > > > > From huw.softdesigns at gmail.com Thu Dec 3 02:59:42 2015 From: huw.softdesigns at gmail.com (Huw Lloyd) Date: Thu Dec 3 02:59:44 2015 Subject: [squeak-dev] Re: Mac release broken? ClassBinding>>objectForDataStream In-Reply-To: <20151203020034.GA32593@shell.msen.com> References: <20151203020034.GA32593@shell.msen.com> Message-ID: Thanks Dave. How do I achieve that -- do I swap in a trunk image to a 4.6 application? What about the VM? Best, Huw On 3 December 2015 at 02:00, David T. Lewis wrote: > Huw, > > The timing of your question is perfect :-) > > There are some very recent updates (in the last week or so) to the Squeak > trunk development stream that address this problem. This is work in > progress, > but in a nutshell, the latest trunk fixes should resolve the problem for > 4.6 images, but there is work still to be done in order to make it work > in 5.0 Spur. We still need to get these latest updates into the standard > update stream for squeak46, but as soon as that is complete it should work > for those images. I am less certain of the timing for 5.0 images. > > Dave > > > > On Thu, Dec 03, 2015 at 01:25:37AM +0000, Huw Lloyd wrote: > > Arbitrarily picking an earlier version ... 4.1 does not have this > issue/bug. > > > > Any quick fixes for this? I'm interested in using this in conjunction > with > > aida. > > > > Best, > > Huw > > > > > > On 3 December 2015 at 01:13, Huw Lloyd > wrote: > > > > > For both 4.6 & 5.0 on the mac, saving a project generates the following > > > error: "NotImplemented: ClassBinding or a superclass should implement > > > objectForDataStream." > > > > > > The comments in the method suggest this shouldn't have been released > to a > > > stable version: "It's not yet clear how serialization should work in > the > > > presence of environments". > > > > > > Problem 2 is that if I then "abandon" the error messge, the workspace > is > > > then flooded with "MessageNotUnderstood" error boxes. > > > > > > Is "environments" something experimental that should be switched off? > > > > > > Best, > > > Huw > > > > > > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151203/1b8873c8/attachment.htm From tim at rowledge.org Thu Dec 3 03:38:57 2015 From: tim at rowledge.org (tim Rowledge) Date: Thu Dec 3 03:38:48 2015 Subject: [squeak-dev] mouse enter/move/leave event confusion In-Reply-To: <655B52F2-8736-476F-9B62-4A279AF46ECE@rowledge.org> References: <4973AB59-FF96-41B1-AE15-904AAE8752D9@rowledge.org> <1449051662880-4864703.post@n4.nabble.com> <655B52F2-8736-476F-9B62-4A279AF46ECE@rowledge.org> Message-ID: <7D3C4F24-F59F-4C93-BE45-A454D91EB853@rowledge.org> > On 02-12-2015, at 11:11 AM, tim Rowledge wrote: > >> Find attached some code that works fine. :) Without gridding but that could >> easily be added. > > Yah, that?s almost exactly what my code looks like wrt the events etc. The only difference is that my ?cursor? isn?t a submorph of the ?playfield?. I wonder if that would help at all? Well, it sort of did and sort of didn?t. It certainly removed the appearance of the fake-cursor jumping around due to spurious mouse-leaves. But unfortunately it *crushed* performance and made painting horribly slow. A Tally implied some horrible volume of draw related messages that don?t happen normally. Sigh. So right now I?m wondering if I can make the HandMorph?s temporaryCursor be some variety of form/morph that does the gridding while displaying. The issue there is that all the relevant code expects the temporaryCursor to be an image, so a lot would need fiddling with. I guess another option might be to subclass HandMorph to try out adding more instvars and add gridding that way. At least that might avoid breaking everyone else?s Hands. ;-) tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Useful random insult:- His head whistles in a cross wind. From hannes.hirzel at gmail.com Thu Dec 3 08:57:57 2015 From: hannes.hirzel at gmail.com (H. Hirzel) Date: Thu Dec 3 08:58:00 2015 Subject: [squeak-dev] Re: Mac release broken? ClassBinding>>objectForDataStream In-Reply-To: <20151203020034.GA32593@shell.msen.com> References: <20151203020034.GA32593@shell.msen.com> Message-ID: Huw, See a proposal for a quick fix below using the work of Colin Putney last week. --Hannes On 12/3/15, David T. Lewis wrote: > Huw, > > The timing of your question is perfect :-) > > There are some very recent updates (in the last week or so) to the Squeak > trunk development stream that address this problem. A copy of the mail describing these updates is here http://wiki.squeak.org/squeak/6218 This is work in > progress, > but in a nutshell, the latest trunk fixes should resolve the problem for > 4.6 images, but there is work still to be done in order to make it work > in 5.0 Spur. We still need to get these latest updates into the standard > update stream for squeak46, but as soon as that is complete it should work > for those images. The updates Colin Putney committed to the trunk on Friday, Saturday 27/28th Nov are Name: Environments-cwp.60 Author: cwp Time: 27 November 2015, 5:14:36.376 pm UUID: c8836cde-f21e-4652-a2e9-73f8de6f00e0 Ancestors: Environments-kfr.59 Always serialize Bindings as DiskProxies. Name: System-cwp.781 Author: cwp Time: 27 November 2015, 5:22:38.591 pm UUID: c0ffbab7-f7ee-4966-ba07-d57436ac42f2 Ancestors: System-mt.780 Remove specialized serialization of Associations. Since associations are no longer used as globals, we never need to serialize them using DiskProxy. Name: System-cwp.782 Author: cwp Time: 27 November 2015, 9:00:42.904 pm UUID: ae1f1166-0438-4e78-8ded-46c59c212210 Ancestors: System-cwp.781 Reload DiskProxies using the current environment When looking up objects from the destination image, DiskProxies should use the current environment, rather than referring to to Smalltalk. This lets objects on disk be loaded into arbitrary environments. These updates are committed to the trunk (i.e. the ongoing development after the release of 5.0). However in a trunk image http://build.squeak.org/job/Trunk/default/lastSuccessfulBuild/artifact/target/TrunkImage.zip they do NOT work yet. What you could do as of now ----------------------------------------- - File in the updates above into a 4.6 image. - It should fix the problem (I have not tried yet, so please report if it is successful). > I am less certain of the timing for 5.0 images. The reason is that the new VM crashes, and Colin has reported this to the Virtual Machine List. > Dave > > > > On Thu, Dec 03, 2015 at 01:25:37AM +0000, Huw Lloyd wrote: >> Arbitrarily picking an earlier version ... 4.1 does not have this >> issue/bug. >> >> Any quick fixes for this? I'm interested in using this in conjunction >> with >> aida. >> >> Best, >> Huw >> >> >> On 3 December 2015 at 01:13, Huw Lloyd wrote: >> >> > For both 4.6 & 5.0 on the mac, saving a project generates the following >> > error: "NotImplemented: ClassBinding or a superclass should implement >> > objectForDataStream." >> > >> > The comments in the method suggest this shouldn't have been released to >> > a >> > stable version: "It's not yet clear how serialization should work in >> > the >> > presence of environments". >> > >> > Problem 2 is that if I then "abandon" the error messge, the workspace >> > is >> > then flooded with "MessageNotUnderstood" error boxes. >> > >> > Is "environments" something experimental that should be switched off? >> > >> > Best, >> > Huw >> > >> > >> > > >> > > > From hannes.hirzel at gmail.com Thu Dec 3 09:08:47 2015 From: hannes.hirzel at gmail.com (H. Hirzel) Date: Thu Dec 3 09:08:50 2015 Subject: [squeak-dev] Re: Mac release broken? ClassBinding>>objectForDataStream In-Reply-To: References: <20151203020034.GA32593@shell.msen.com> Message-ID: P.S. The report for this issue is here http://bugs.squeak.org/view.php?id=7814 (not yet updated with the new development of last week) On 12/3/15, H. Hirzel wrote: > Huw, > > See a proposal for a quick fix below using the work of Colin Putney last > week. > > --Hannes > > On 12/3/15, David T. Lewis wrote: >> Huw, >> >> The timing of your question is perfect :-) >> >> There are some very recent updates (in the last week or so) to the Squeak >> trunk development stream that address this problem. > > A copy of the mail describing these updates is here > http://wiki.squeak.org/squeak/6218 > > This is work in >> progress, >> but in a nutshell, the latest trunk fixes should resolve the problem for >> 4.6 images, but there is work still to be done in order to make it work >> in 5.0 Spur. We still need to get these latest updates into the standard >> update stream for squeak46, but as soon as that is complete it should >> work >> for those images. > > The updates Colin Putney committed to the trunk on Friday, Saturday > 27/28th Nov are > > Name: Environments-cwp.60 > Author: cwp > Time: 27 November 2015, 5:14:36.376 pm > UUID: c8836cde-f21e-4652-a2e9-73f8de6f00e0 > Ancestors: Environments-kfr.59 > > Always serialize Bindings as DiskProxies. > > > Name: System-cwp.781 > Author: cwp > Time: 27 November 2015, 5:22:38.591 pm > UUID: c0ffbab7-f7ee-4966-ba07-d57436ac42f2 > Ancestors: System-mt.780 > > Remove specialized serialization of Associations. > Since associations are no longer used as globals, we never need to > serialize them using DiskProxy. > > > Name: System-cwp.782 > Author: cwp > Time: 27 November 2015, 9:00:42.904 pm > UUID: ae1f1166-0438-4e78-8ded-46c59c212210 > Ancestors: System-cwp.781 > > Reload DiskProxies using the current environment > > When looking up objects from the destination image, DiskProxies should > use the current environment, rather than referring to to Smalltalk. > This lets objects on disk be loaded into arbitrary environments. > > These updates are committed to the trunk (i.e. the ongoing development > after the release of 5.0). However in a trunk image > http://build.squeak.org/job/Trunk/default/lastSuccessfulBuild/artifact/target/TrunkImage.zip > they do NOT work yet. > > > What you could do as of now > ----------------------------------------- > > - File in the updates above into a 4.6 image. > - It should fix the problem (I have not tried yet, so please report if > it is successful). > > > > >> I am less certain of the timing for 5.0 images. > > The reason is that the new VM crashes, and Colin has reported this to > the Virtual Machine List. > > > >> Dave >> >> >> >> On Thu, Dec 03, 2015 at 01:25:37AM +0000, Huw Lloyd wrote: >>> Arbitrarily picking an earlier version ... 4.1 does not have this >>> issue/bug. >>> >>> Any quick fixes for this? I'm interested in using this in conjunction >>> with >>> aida. >>> >>> Best, >>> Huw >>> >>> >>> On 3 December 2015 at 01:13, Huw Lloyd >>> wrote: >>> >>> > For both 4.6 & 5.0 on the mac, saving a project generates the >>> > following >>> > error: "NotImplemented: ClassBinding or a superclass should implement >>> > objectForDataStream." >>> > >>> > The comments in the method suggest this shouldn't have been released >>> > to >>> > a >>> > stable version: "It's not yet clear how serialization should work in >>> > the >>> > presence of environments". >>> > >>> > Problem 2 is that if I then "abandon" the error messge, the workspace >>> > is >>> > then flooded with "MessageNotUnderstood" error boxes. >>> > >>> > Is "environments" something experimental that should be switched off? >>> > >>> > Best, >>> > Huw >>> > >>> > >>> > >> >>> >> >> >> > From robert.w.withers at gmail.com Thu Dec 3 11:00:20 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Thu Dec 3 11:00:27 2015 Subject: [squeak-dev] Re: evolutions of squeakelib & crypto (Reed Solomon) In-Reply-To: <565DF3B4.8020100@gmail.com> References: <565D9498.3030104@gmail.com> <565DCE53.90501@gmail.com> <565DF3B4.8020100@gmail.com> Message-ID: <566020C4.8080703@gmail.com> Here is an update with some refactorings and extensions. I added 'domain' to the IWant/IAm messages to support subdomains. https://www.dropbox.com/s/tvpxfuixzxrj3op/SqueakElib-rww.204.mcz?dl=0 thanks for trying it, Robert On 12/01/2015 02:23 PM, Robert Withers wrote: > Another version fixing empty payloads... > > https://www.dropbox.com/s/wjhbopwabbrszjn/SqueakElib-rww.202.mcz?dl=0 > > thank you, > - Robert > > On 12/01/2015 11:44 AM, Robert Withers wrote: >> Here is a newer version (201) with a protocol change to remove all >> tcpIds from the rendezvous. >> >> https://www.dropbox.com/s/7mgfglwu3e4bf4j/SqueakElib-rww.201.mcz?dl=0 >> >> - Robert >> >> On 12/01/2015 07:37 AM, Robert Withers wrote: >>> This version of the Cryptography package has beginnings of >>> ReedSolomon encoding. >>> >>> The rendezvous is separated into an OperationProtocol and a >>> StartupProtocol and the message pipeline of the session layer of >>> SqueakElib has been completely rewritten and LayeredProtocols have >>> been folded in. Please look at ElibVatTPDataTestCase for an example. >>> >>> Cryptography: >>> https://www.dropbox.com/s/z9dux2goimej79e/Cryptography-rww.40.mcz?dl=0 >>> SqueakElib: >>> https://www.dropbox.com/s/5yvkl3df89uyjnh/SqueakElib-rww.200.mcz?dl=0 >>> >>> In the spirit of #GivingTuesday, >>> Robert >> > From robert.w.withers at gmail.com Thu Dec 3 11:20:30 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Thu Dec 3 11:20:37 2015 Subject: [squeak-dev] Re: evolutions of squeakelib & crypto (Reed Solomon) In-Reply-To: <566020C4.8080703@gmail.com> References: <565D9498.3030104@gmail.com> <565DCE53.90501@gmail.com> <565DF3B4.8020100@gmail.com> <566020C4.8080703@gmail.com> Message-ID: <5660257E.7070502@gmail.com> ReedSolomon updates: https://www.dropbox.com/s/f5unbbdetv4c5xs/Cryptography-rww.41.mcz?dl=0 more refactoring of the pipeline: https://www.dropbox.com/s/g45himklwa66zqb/SqueakElib-rww.205.mcz?dl=0 - Robert On 12/03/2015 06:00 AM, Robert Withers wrote: > Here is an update with some refactorings and extensions. I added > 'domain' to the IWant/IAm messages to support subdomains. > > https://www.dropbox.com/s/tvpxfuixzxrj3op/SqueakElib-rww.204.mcz?dl=0 > > thanks for trying it, > Robert > > On 12/01/2015 02:23 PM, Robert Withers wrote: >> Another version fixing empty payloads... >> >> https://www.dropbox.com/s/wjhbopwabbrszjn/SqueakElib-rww.202.mcz?dl=0 >> >> thank you, >> - Robert >> >> On 12/01/2015 11:44 AM, Robert Withers wrote: >>> Here is a newer version (201) with a protocol change to remove all >>> tcpIds from the rendezvous. >>> >>> https://www.dropbox.com/s/7mgfglwu3e4bf4j/SqueakElib-rww.201.mcz?dl=0 >>> >>> - Robert >>> >>> On 12/01/2015 07:37 AM, Robert Withers wrote: >>>> This version of the Cryptography package has beginnings of >>>> ReedSolomon encoding. >>>> >>>> The rendezvous is separated into an OperationProtocol and a >>>> StartupProtocol and the message pipeline of the session layer of >>>> SqueakElib has been completely rewritten and LayeredProtocols have >>>> been folded in. Please look at ElibVatTPDataTestCase for an example. >>>> >>>> Cryptography: >>>> https://www.dropbox.com/s/z9dux2goimej79e/Cryptography-rww.40.mcz?dl=0 >>>> SqueakElib: >>>> https://www.dropbox.com/s/5yvkl3df89uyjnh/SqueakElib-rww.200.mcz?dl=0 >>>> >>>> In the spirit of #GivingTuesday, >>>> Robert >>> >> > From Marcel.Taeumel at hpi.de Thu Dec 3 11:33:30 2015 From: Marcel.Taeumel at hpi.de (marcel.taeumel) Date: Thu Dec 3 11:50:04 2015 Subject: [squeak-dev] Re: mouse enter/move/leave event confusion In-Reply-To: <7D3C4F24-F59F-4C93-BE45-A454D91EB853@rowledge.org> References: <4973AB59-FF96-41B1-AE15-904AAE8752D9@rowledge.org> <1449051662880-4864703.post@n4.nabble.com> <655B52F2-8736-476F-9B62-4A279AF46ECE@rowledge.org> <7D3C4F24-F59F-4C93-BE45-A454D91EB853@rowledge.org> Message-ID: <1449142410592-4864917.post@n4.nabble.com> Unfortunately, it is not possible anymore to modify the mouse position programmatically. You approach with an additional cursor seems reasonable. That slow-down can occur because your paint area is a single morph? How does it draw? If you want efficient redraw with the damage-recorder working, you should use morph composition. Every stroke should be a morph. This can speed-up drawing routine. Can you show some code? Best, Marcel -- View this message in context: http://forum.world.st/mouse-enter-move-leave-event-confusion-tp4864651p4864917.html Sent from the Squeak - Dev mailing list archive at Nabble.com. From commits at source.squeak.org Thu Dec 3 12:59:04 2015 From: commits at source.squeak.org (commits@source.squeak.org) Date: Thu Dec 3 12:59:05 2015 Subject: [squeak-dev] The Trunk: Environments-kfr.61.mcz Message-ID: Karl Ramberg uploaded a new version of Environments to project The Trunk: http://source.squeak.org/trunk/Environments-kfr.61.mcz ==================== Summary ==================== Name: Environments-kfr.61 Author: kfr Time: 3 December 2015, 1:58:55.138 pm UUID: 2fd02cbd-d78c-4672-be7a-627a58cf0adf Ancestors: Environments-cwp.60 Adds help interface so one can read all class and method comments in a HelpBrowser =============== Diff against Environments-cwp.60 =============== Item was changed: SystemOrganization addCategory: #'Environments-Core'! - SystemOrganization addCategory: #'Environments-Policies'! SystemOrganization addCategory: #'Environments-Loading'! + SystemOrganization addCategory: #'Environments-Policies'! + SystemOrganization addCategory: #'Environments-Help'! Item was added: + Object subclass: #EnvironmentsAPIDocumentation + instanceVariableNames: '' + classVariableNames: '' + poolDictionaries: '' + category: 'Environments-Help'! Item was added: + ----- Method: EnvironmentsAPIDocumentation class>>asHelpTopic (in category 'as yet unclassified') ----- + asHelpTopic + + ^ (HelpTopic named: self bookName) + subtopics: (self packages collect: [:pkgName | PackageAPIHelpTopic new packageName: pkgName]); + yourself! Item was added: + ----- Method: EnvironmentsAPIDocumentation class>>bookName (in category 'as yet unclassified') ----- + bookName + ^'API Documentation'! Item was added: + ----- Method: EnvironmentsAPIDocumentation class>>packages (in category 'as yet unclassified') ----- + packages + ^#('Environments-Core' 'Environments-Loading' 'Environments-Policies')! From commits at source.squeak.org Thu Dec 3 13:13:18 2015 From: commits at source.squeak.org (commits@source.squeak.org) Date: Thu Dec 3 13:13:19 2015 Subject: [squeak-dev] The Trunk: WebClient-Help-kfr.11.mcz Message-ID: Karl Ramberg uploaded a new version of WebClient-Help to project The Trunk: http://source.squeak.org/trunk/WebClient-Help-kfr.11.mcz ==================== Summary ==================== Name: WebClient-Help-kfr.11 Author: kfr Time: 3 December 2015, 2:13:10.555 pm UUID: b1183115-c3aa-440f-aef6-5bc942f12212 Ancestors: WebClient-Help-ar.10 Fix for help referece so API documentation show in HelpBrowser =============== Diff against WebClient-Help-ar.10 =============== Item was added: + ----- Method: WebClientReference class>>asHelpTopic (in category 'accessing') ----- + asHelpTopic + + ^ (HelpTopic named: self bookName) + subtopics: (self packages collect: [:pkgName | PackageAPIHelpTopic new packageName: pkgName]); + yourself! Item was added: + ----- Method: WebServerReference class>>asHelpTopic (in category 'accessing') ----- + asHelpTopic + + ^ (HelpTopic named: self bookName) + subtopics: (self packages collect: [:pkgName | PackageAPIHelpTopic new packageName: pkgName]); + yourself! From huw.softdesigns at gmail.com Thu Dec 3 14:43:33 2015 From: huw.softdesigns at gmail.com (Huw Lloyd) Date: Thu Dec 3 14:43:37 2015 Subject: [squeak-dev] Re: Mac release broken? ClassBinding>>objectForDataStream In-Reply-To: References: <20151203020034.GA32593@shell.msen.com> Message-ID: Hannes, I got as far as a failure to load System-cwp.781 with missing dependency on class UnhandledWarning: "Warning: This package depends on the following classes: UnhandledWarning You must resolve these dependencies before you will be able to load these definitions: UnhandledWarning>>defaultAction" The version of package system for 4.6 is System-topa.753 Best, Huw On 3 December 2015 at 08:57, H. Hirzel wrote: > Huw, > > See a proposal for a quick fix below using the work of Colin Putney last > week. > > --Hannes > > On 12/3/15, David T. Lewis wrote: > > Huw, > > > > The timing of your question is perfect :-) > > > > There are some very recent updates (in the last week or so) to the Squeak > > trunk development stream that address this problem. > > A copy of the mail describing these updates is here > http://wiki.squeak.org/squeak/6218 > > This is work in > > progress, > > but in a nutshell, the latest trunk fixes should resolve the problem for > > 4.6 images, but there is work still to be done in order to make it work > > in 5.0 Spur. We still need to get these latest updates into the standard > > update stream for squeak46, but as soon as that is complete it should > work > > for those images. > > The updates Colin Putney committed to the trunk on Friday, Saturday > 27/28th Nov are > > Name: Environments-cwp.60 > Author: cwp > Time: 27 November 2015, 5:14:36.376 pm > UUID: c8836cde-f21e-4652-a2e9-73f8de6f00e0 > Ancestors: Environments-kfr.59 > > Always serialize Bindings as DiskProxies. > > > Name: System-cwp.781 > Author: cwp > Time: 27 November 2015, 5:22:38.591 pm > UUID: c0ffbab7-f7ee-4966-ba07-d57436ac42f2 > Ancestors: System-mt.780 > > Remove specialized serialization of Associations. > Since associations are no longer used as globals, we never need to > serialize them using DiskProxy. > > > Name: System-cwp.782 > Author: cwp > Time: 27 November 2015, 9:00:42.904 pm > UUID: ae1f1166-0438-4e78-8ded-46c59c212210 > Ancestors: System-cwp.781 > > Reload DiskProxies using the current environment > > When looking up objects from the destination image, DiskProxies should > use the current environment, rather than referring to to Smalltalk. > This lets objects on disk be loaded into arbitrary environments. > > These updates are committed to the trunk (i.e. the ongoing development > after the release of 5.0). However in a trunk image > > http://build.squeak.org/job/Trunk/default/lastSuccessfulBuild/artifact/target/TrunkImage.zip > they do NOT work yet. > > > What you could do as of now > ----------------------------------------- > > - File in the updates above into a 4.6 image. > - It should fix the problem (I have not tried yet, so please report if > it is successful). > > > > > > I am less certain of the timing for 5.0 images. > > The reason is that the new VM crashes, and Colin has reported this to > the Virtual Machine List. > > > > > Dave > > > > > > > > On Thu, Dec 03, 2015 at 01:25:37AM +0000, Huw Lloyd wrote: > >> Arbitrarily picking an earlier version ... 4.1 does not have this > >> issue/bug. > >> > >> Any quick fixes for this? I'm interested in using this in conjunction > >> with > >> aida. > >> > >> Best, > >> Huw > >> > >> > >> On 3 December 2015 at 01:13, Huw Lloyd > wrote: > >> > >> > For both 4.6 & 5.0 on the mac, saving a project generates the > following > >> > error: "NotImplemented: ClassBinding or a superclass should implement > >> > objectForDataStream." > >> > > >> > The comments in the method suggest this shouldn't have been released > to > >> > a > >> > stable version: "It's not yet clear how serialization should work in > >> > the > >> > presence of environments". > >> > > >> > Problem 2 is that if I then "abandon" the error messge, the workspace > >> > is > >> > then flooded with "MessageNotUnderstood" error boxes. > >> > > >> > Is "environments" something experimental that should be switched off? > >> > > >> > Best, > >> > Huw > >> > > >> > > >> > > > > >> > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151203/ab0962d0/attachment.htm From huw.softdesigns at gmail.com Thu Dec 3 14:58:30 2015 From: huw.softdesigns at gmail.com (Huw Lloyd) Date: Thu Dec 3 14:58:33 2015 Subject: [squeak-dev] Re: Mac release broken? ClassBinding>>objectForDataStream In-Reply-To: References: <20151203020034.GA32593@shell.msen.com> Message-ID: Looks earliest System package to not load to 4.6 is: Name: System-ul.757 Author: ul Time: 14 August 2015, 7:19:00.474 pm UUID: fca51b99-736e-4666-acac-e5c5b6a1d397 Ancestors: System-cmm.756 #includesSubString: -> #includesSubstring: #subStrings -> #substrings Best, Huw On 3 December 2015 at 14:43, Huw Lloyd wrote: > Hannes, > > I got as far as a failure to load System-cwp.781 with missing dependency > on class UnhandledWarning: > > "Warning: This package depends on the following classes: > UnhandledWarning > You must resolve these dependencies before you will be able to load these > definitions: > UnhandledWarning>>defaultAction" > > The version of package system for 4.6 is System-topa.753 > > Best, > Huw > > > On 3 December 2015 at 08:57, H. Hirzel wrote: > >> Huw, >> >> See a proposal for a quick fix below using the work of Colin Putney last >> week. >> >> --Hannes >> >> On 12/3/15, David T. Lewis wrote: >> > Huw, >> > >> > The timing of your question is perfect :-) >> > >> > There are some very recent updates (in the last week or so) to the >> Squeak >> > trunk development stream that address this problem. >> >> A copy of the mail describing these updates is here >> http://wiki.squeak.org/squeak/6218 >> >> This is work in >> > progress, >> > but in a nutshell, the latest trunk fixes should resolve the problem for >> > 4.6 images, but there is work still to be done in order to make it work >> > in 5.0 Spur. We still need to get these latest updates into the standard >> > update stream for squeak46, but as soon as that is complete it should >> work >> > for those images. >> >> The updates Colin Putney committed to the trunk on Friday, Saturday >> 27/28th Nov are >> >> Name: Environments-cwp.60 >> Author: cwp >> Time: 27 November 2015, 5:14:36.376 pm >> UUID: c8836cde-f21e-4652-a2e9-73f8de6f00e0 >> Ancestors: Environments-kfr.59 >> >> Always serialize Bindings as DiskProxies. >> >> >> Name: System-cwp.781 >> Author: cwp >> Time: 27 November 2015, 5:22:38.591 pm >> UUID: c0ffbab7-f7ee-4966-ba07-d57436ac42f2 >> Ancestors: System-mt.780 >> >> Remove specialized serialization of Associations. >> Since associations are no longer used as globals, we never need to >> serialize them using DiskProxy. >> >> >> Name: System-cwp.782 >> Author: cwp >> Time: 27 November 2015, 9:00:42.904 pm >> UUID: ae1f1166-0438-4e78-8ded-46c59c212210 >> Ancestors: System-cwp.781 >> >> Reload DiskProxies using the current environment >> >> When looking up objects from the destination image, DiskProxies should >> use the current environment, rather than referring to to Smalltalk. >> This lets objects on disk be loaded into arbitrary environments. >> >> These updates are committed to the trunk (i.e. the ongoing development >> after the release of 5.0). However in a trunk image >> >> http://build.squeak.org/job/Trunk/default/lastSuccessfulBuild/artifact/target/TrunkImage.zip >> they do NOT work yet. >> >> >> What you could do as of now >> ----------------------------------------- >> >> - File in the updates above into a 4.6 image. >> - It should fix the problem (I have not tried yet, so please report if >> it is successful). >> >> >> >> >> > I am less certain of the timing for 5.0 images. >> >> The reason is that the new VM crashes, and Colin has reported this to >> the Virtual Machine List. >> >> >> >> > Dave >> > >> > >> > >> > On Thu, Dec 03, 2015 at 01:25:37AM +0000, Huw Lloyd wrote: >> >> Arbitrarily picking an earlier version ... 4.1 does not have this >> >> issue/bug. >> >> >> >> Any quick fixes for this? I'm interested in using this in conjunction >> >> with >> >> aida. >> >> >> >> Best, >> >> Huw >> >> >> >> >> >> On 3 December 2015 at 01:13, Huw Lloyd >> wrote: >> >> >> >> > For both 4.6 & 5.0 on the mac, saving a project generates the >> following >> >> > error: "NotImplemented: ClassBinding or a superclass should implement >> >> > objectForDataStream." >> >> > >> >> > The comments in the method suggest this shouldn't have been released >> to >> >> > a >> >> > stable version: "It's not yet clear how serialization should work in >> >> > the >> >> > presence of environments". >> >> > >> >> > Problem 2 is that if I then "abandon" the error messge, the workspace >> >> > is >> >> > then flooded with "MessageNotUnderstood" error boxes. >> >> > >> >> > Is "environments" something experimental that should be switched off? >> >> > >> >> > Best, >> >> > Huw >> >> > >> >> > >> >> > >> > >> >> >> > >> > >> > >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151203/26f59cbc/attachment.htm From karlramberg at gmail.com Thu Dec 3 15:01:01 2015 From: karlramberg at gmail.com (karl ramberg) Date: Thu Dec 3 15:01:04 2015 Subject: [squeak-dev] Re: Mac release broken? ClassBinding>>objectForDataStream In-Reply-To: References: <20151203020034.GA32593@shell.msen.com> Message-ID: UnhandledWarning was added in http://source.squeak.org/trunk/Kernel-mt.945.mcz http://source.squeak.org/trunk/System-mt.759.mcz Best, Karl On Thu, Dec 3, 2015 at 3:58 PM, Huw Lloyd wrote: > Looks earliest System package to not load to 4.6 is: > > Name: System-ul.757 > Author: ul > Time: 14 August 2015, 7:19:00.474 pm > UUID: fca51b99-736e-4666-acac-e5c5b6a1d397 > Ancestors: System-cmm.756 > > #includesSubString: -> #includesSubstring: > #subStrings -> #substrings > > Best, > Huw > > > On 3 December 2015 at 14:43, Huw Lloyd wrote: > >> Hannes, >> >> I got as far as a failure to load System-cwp.781 with missing dependency >> on class UnhandledWarning: >> >> "Warning: This package depends on the following classes: >> UnhandledWarning >> You must resolve these dependencies before you will be able to load these >> definitions: >> UnhandledWarning>>defaultAction" >> >> The version of package system for 4.6 is System-topa.753 >> >> Best, >> Huw >> >> >> On 3 December 2015 at 08:57, H. Hirzel wrote: >> >>> Huw, >>> >>> See a proposal for a quick fix below using the work of Colin Putney last >>> week. >>> >>> --Hannes >>> >>> On 12/3/15, David T. Lewis wrote: >>> > Huw, >>> > >>> > The timing of your question is perfect :-) >>> > >>> > There are some very recent updates (in the last week or so) to the >>> Squeak >>> > trunk development stream that address this problem. >>> >>> A copy of the mail describing these updates is here >>> http://wiki.squeak.org/squeak/6218 >>> >>> This is work in >>> > progress, >>> > but in a nutshell, the latest trunk fixes should resolve the problem >>> for >>> > 4.6 images, but there is work still to be done in order to make it work >>> > in 5.0 Spur. We still need to get these latest updates into the >>> standard >>> > update stream for squeak46, but as soon as that is complete it should >>> work >>> > for those images. >>> >>> The updates Colin Putney committed to the trunk on Friday, Saturday >>> 27/28th Nov are >>> >>> Name: Environments-cwp.60 >>> Author: cwp >>> Time: 27 November 2015, 5:14:36.376 pm >>> UUID: c8836cde-f21e-4652-a2e9-73f8de6f00e0 >>> Ancestors: Environments-kfr.59 >>> >>> Always serialize Bindings as DiskProxies. >>> >>> >>> Name: System-cwp.781 >>> Author: cwp >>> Time: 27 November 2015, 5:22:38.591 pm >>> UUID: c0ffbab7-f7ee-4966-ba07-d57436ac42f2 >>> Ancestors: System-mt.780 >>> >>> Remove specialized serialization of Associations. >>> Since associations are no longer used as globals, we never need to >>> serialize them using DiskProxy. >>> >>> >>> Name: System-cwp.782 >>> Author: cwp >>> Time: 27 November 2015, 9:00:42.904 pm >>> UUID: ae1f1166-0438-4e78-8ded-46c59c212210 >>> Ancestors: System-cwp.781 >>> >>> Reload DiskProxies using the current environment >>> >>> When looking up objects from the destination image, DiskProxies should >>> use the current environment, rather than referring to to Smalltalk. >>> This lets objects on disk be loaded into arbitrary environments. >>> >>> These updates are committed to the trunk (i.e. the ongoing development >>> after the release of 5.0). However in a trunk image >>> >>> http://build.squeak.org/job/Trunk/default/lastSuccessfulBuild/artifact/target/TrunkImage.zip >>> they do NOT work yet. >>> >>> >>> What you could do as of now >>> ----------------------------------------- >>> >>> - File in the updates above into a 4.6 image. >>> - It should fix the problem (I have not tried yet, so please report if >>> it is successful). >>> >>> >>> >>> >>> > I am less certain of the timing for 5.0 images. >>> >>> The reason is that the new VM crashes, and Colin has reported this to >>> the Virtual Machine List. >>> >>> >>> >>> > Dave >>> > >>> > >>> > >>> > On Thu, Dec 03, 2015 at 01:25:37AM +0000, Huw Lloyd wrote: >>> >> Arbitrarily picking an earlier version ... 4.1 does not have this >>> >> issue/bug. >>> >> >>> >> Any quick fixes for this? I'm interested in using this in conjunction >>> >> with >>> >> aida. >>> >> >>> >> Best, >>> >> Huw >>> >> >>> >> >>> >> On 3 December 2015 at 01:13, Huw Lloyd >>> wrote: >>> >> >>> >> > For both 4.6 & 5.0 on the mac, saving a project generates the >>> following >>> >> > error: "NotImplemented: ClassBinding or a superclass should >>> implement >>> >> > objectForDataStream." >>> >> > >>> >> > The comments in the method suggest this shouldn't have been >>> released to >>> >> > a >>> >> > stable version: "It's not yet clear how serialization should work in >>> >> > the >>> >> > presence of environments". >>> >> > >>> >> > Problem 2 is that if I then "abandon" the error messge, the >>> workspace >>> >> > is >>> >> > then flooded with "MessageNotUnderstood" error boxes. >>> >> > >>> >> > Is "environments" something experimental that should be switched >>> off? >>> >> > >>> >> > Best, >>> >> > Huw >>> >> > >>> >> > >>> >> > >>> > >>> >> >>> > >>> > >>> > >>> >>> >> > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151203/c9591b0f/attachment.htm From hannes.hirzel at gmail.com Thu Dec 3 15:06:28 2015 From: hannes.hirzel at gmail.com (H. Hirzel) Date: Thu Dec 3 15:06:31 2015 Subject: [squeak-dev] Re: Mac release broken? ClassBinding>>objectForDataStream In-Reply-To: References: <20151203020034.GA32593@shell.msen.com> Message-ID: As direct loading does not work I suggest something like copy/paste updated methods from a Squeak 5 trunk image one by one. back to the 4.6 image. I.e. open the mcz file in a code browser but do not load it. If necessary the code has to be adapted to cater for updates which happened elsewhere in the meantime as you write #includesSubString: -> #includesSubstring: #subStrings -> #substrings It seems that there not all that many fixes. --Hannes On 12/3/15, Huw Lloyd wrote: > Looks earliest System package to not load to 4.6 is: > > Name: System-ul.757 > Author: ul > Time: 14 August 2015, 7:19:00.474 pm > UUID: fca51b99-736e-4666-acac-e5c5b6a1d397 > Ancestors: System-cmm.756 > > #includesSubString: -> #includesSubstring: > #subStrings -> #substrings > > Best, > Huw > > > On 3 December 2015 at 14:43, Huw Lloyd wrote: > >> Hannes, >> >> I got as far as a failure to load System-cwp.781 with missing dependency >> on class UnhandledWarning: >> >> "Warning: This package depends on the following classes: >> UnhandledWarning >> You must resolve these dependencies before you will be able to load these >> definitions: >> UnhandledWarning>>defaultAction" >> >> The version of package system for 4.6 is System-topa.753 >> >> Best, >> Huw >> >> >> On 3 December 2015 at 08:57, H. Hirzel wrote: >> >>> Huw, >>> >>> See a proposal for a quick fix below using the work of Colin Putney last >>> week. >>> >>> --Hannes >>> >>> On 12/3/15, David T. Lewis wrote: >>> > Huw, >>> > >>> > The timing of your question is perfect :-) >>> > >>> > There are some very recent updates (in the last week or so) to the >>> Squeak >>> > trunk development stream that address this problem. >>> >>> A copy of the mail describing these updates is here >>> http://wiki.squeak.org/squeak/6218 >>> >>> This is work in >>> > progress, >>> > but in a nutshell, the latest trunk fixes should resolve the problem >>> > for >>> > 4.6 images, but there is work still to be done in order to make it >>> > work >>> > in 5.0 Spur. We still need to get these latest updates into the >>> > standard >>> > update stream for squeak46, but as soon as that is complete it should >>> work >>> > for those images. >>> >>> The updates Colin Putney committed to the trunk on Friday, Saturday >>> 27/28th Nov are >>> >>> Name: Environments-cwp.60 >>> Author: cwp >>> Time: 27 November 2015, 5:14:36.376 pm >>> UUID: c8836cde-f21e-4652-a2e9-73f8de6f00e0 >>> Ancestors: Environments-kfr.59 >>> >>> Always serialize Bindings as DiskProxies. >>> >>> >>> Name: System-cwp.781 >>> Author: cwp >>> Time: 27 November 2015, 5:22:38.591 pm >>> UUID: c0ffbab7-f7ee-4966-ba07-d57436ac42f2 >>> Ancestors: System-mt.780 >>> >>> Remove specialized serialization of Associations. >>> Since associations are no longer used as globals, we never need to >>> serialize them using DiskProxy. >>> >>> >>> Name: System-cwp.782 >>> Author: cwp >>> Time: 27 November 2015, 9:00:42.904 pm >>> UUID: ae1f1166-0438-4e78-8ded-46c59c212210 >>> Ancestors: System-cwp.781 >>> >>> Reload DiskProxies using the current environment >>> >>> When looking up objects from the destination image, DiskProxies should >>> use the current environment, rather than referring to to Smalltalk. >>> This lets objects on disk be loaded into arbitrary environments. >>> >>> These updates are committed to the trunk (i.e. the ongoing development >>> after the release of 5.0). However in a trunk image >>> >>> http://build.squeak.org/job/Trunk/default/lastSuccessfulBuild/artifact/target/TrunkImage.zip >>> they do NOT work yet. >>> >>> >>> What you could do as of now >>> ----------------------------------------- >>> >>> - File in the updates above into a 4.6 image. >>> - It should fix the problem (I have not tried yet, so please report if >>> it is successful). >>> >>> >>> >>> >>> > I am less certain of the timing for 5.0 images. >>> >>> The reason is that the new VM crashes, and Colin has reported this to >>> the Virtual Machine List. >>> >>> >>> >>> > Dave >>> > >>> > >>> > >>> > On Thu, Dec 03, 2015 at 01:25:37AM +0000, Huw Lloyd wrote: >>> >> Arbitrarily picking an earlier version ... 4.1 does not have this >>> >> issue/bug. >>> >> >>> >> Any quick fixes for this? I'm interested in using this in >>> >> conjunction >>> >> with >>> >> aida. >>> >> >>> >> Best, >>> >> Huw >>> >> >>> >> >>> >> On 3 December 2015 at 01:13, Huw Lloyd >>> wrote: >>> >> >>> >> > For both 4.6 & 5.0 on the mac, saving a project generates the >>> following >>> >> > error: "NotImplemented: ClassBinding or a superclass should >>> >> > implement >>> >> > objectForDataStream." >>> >> > >>> >> > The comments in the method suggest this shouldn't have been >>> >> > released >>> to >>> >> > a >>> >> > stable version: "It's not yet clear how serialization should work >>> >> > in >>> >> > the >>> >> > presence of environments". >>> >> > >>> >> > Problem 2 is that if I then "abandon" the error messge, the >>> >> > workspace >>> >> > is >>> >> > then flooded with "MessageNotUnderstood" error boxes. >>> >> > >>> >> > Is "environments" something experimental that should be switched >>> >> > off? >>> >> > >>> >> > Best, >>> >> > Huw >>> >> > >>> >> > >>> >> > >>> > >>> >> >>> > >>> > >>> > >>> >>> >> > From colin at wiresong.com Thu Dec 3 17:08:38 2015 From: colin at wiresong.com (Colin Putney) Date: Thu Dec 3 17:08:41 2015 Subject: [squeak-dev] Re: Mac release broken? ClassBinding>>objectForDataStream In-Reply-To: References: <20151203020034.GA32593@shell.msen.com> Message-ID: I'll backport the trunk changes to 4.6 tonight. On Thu, Dec 3, 2015 at 7:06 AM, H. Hirzel wrote: > As direct loading does not work I suggest something like copy/paste > updated methods from a Squeak 5 trunk image one by one. > > back to the 4.6 image. > > I.e. open the mcz file in a code browser but do not load it. > > If necessary the code has to be adapted to cater for updates which > happened elsewhere in the meantime as you write > > #includesSubString: -> #includesSubstring: > #subStrings -> #substrings > > It seems that there not all that many fixes. > > > --Hannes > > On 12/3/15, Huw Lloyd wrote: > > Looks earliest System package to not load to 4.6 is: > > > > Name: System-ul.757 > > Author: ul > > Time: 14 August 2015, 7:19:00.474 pm > > UUID: fca51b99-736e-4666-acac-e5c5b6a1d397 > > Ancestors: System-cmm.756 > > > > #includesSubString: -> #includesSubstring: > > #subStrings -> #substrings > > > > Best, > > Huw > > > > > > On 3 December 2015 at 14:43, Huw Lloyd > wrote: > > > >> Hannes, > >> > >> I got as far as a failure to load System-cwp.781 with missing dependency > >> on class UnhandledWarning: > >> > >> "Warning: This package depends on the following classes: > >> UnhandledWarning > >> You must resolve these dependencies before you will be able to load > these > >> definitions: > >> UnhandledWarning>>defaultAction" > >> > >> The version of package system for 4.6 is System-topa.753 > >> > >> Best, > >> Huw > >> > >> > >> On 3 December 2015 at 08:57, H. Hirzel wrote: > >> > >>> Huw, > >>> > >>> See a proposal for a quick fix below using the work of Colin Putney > last > >>> week. > >>> > >>> --Hannes > >>> > >>> On 12/3/15, David T. Lewis wrote: > >>> > Huw, > >>> > > >>> > The timing of your question is perfect :-) > >>> > > >>> > There are some very recent updates (in the last week or so) to the > >>> Squeak > >>> > trunk development stream that address this problem. > >>> > >>> A copy of the mail describing these updates is here > >>> http://wiki.squeak.org/squeak/6218 > >>> > >>> This is work in > >>> > progress, > >>> > but in a nutshell, the latest trunk fixes should resolve the problem > >>> > for > >>> > 4.6 images, but there is work still to be done in order to make it > >>> > work > >>> > in 5.0 Spur. We still need to get these latest updates into the > >>> > standard > >>> > update stream for squeak46, but as soon as that is complete it should > >>> work > >>> > for those images. > >>> > >>> The updates Colin Putney committed to the trunk on Friday, Saturday > >>> 27/28th Nov are > >>> > >>> Name: Environments-cwp.60 > >>> Author: cwp > >>> Time: 27 November 2015, 5:14:36.376 pm > >>> UUID: c8836cde-f21e-4652-a2e9-73f8de6f00e0 > >>> Ancestors: Environments-kfr.59 > >>> > >>> Always serialize Bindings as DiskProxies. > >>> > >>> > >>> Name: System-cwp.781 > >>> Author: cwp > >>> Time: 27 November 2015, 5:22:38.591 pm > >>> UUID: c0ffbab7-f7ee-4966-ba07-d57436ac42f2 > >>> Ancestors: System-mt.780 > >>> > >>> Remove specialized serialization of Associations. > >>> Since associations are no longer used as globals, we never need to > >>> serialize them using DiskProxy. > >>> > >>> > >>> Name: System-cwp.782 > >>> Author: cwp > >>> Time: 27 November 2015, 9:00:42.904 pm > >>> UUID: ae1f1166-0438-4e78-8ded-46c59c212210 > >>> Ancestors: System-cwp.781 > >>> > >>> Reload DiskProxies using the current environment > >>> > >>> When looking up objects from the destination image, DiskProxies should > >>> use the current environment, rather than referring to to Smalltalk. > >>> This lets objects on disk be loaded into arbitrary environments. > >>> > >>> These updates are committed to the trunk (i.e. the ongoing development > >>> after the release of 5.0). However in a trunk image > >>> > >>> > http://build.squeak.org/job/Trunk/default/lastSuccessfulBuild/artifact/target/TrunkImage.zip > >>> they do NOT work yet. > >>> > >>> > >>> What you could do as of now > >>> ----------------------------------------- > >>> > >>> - File in the updates above into a 4.6 image. > >>> - It should fix the problem (I have not tried yet, so please report if > >>> it is successful). > >>> > >>> > >>> > >>> > >>> > I am less certain of the timing for 5.0 images. > >>> > >>> The reason is that the new VM crashes, and Colin has reported this to > >>> the Virtual Machine List. > >>> > >>> > >>> > >>> > Dave > >>> > > >>> > > >>> > > >>> > On Thu, Dec 03, 2015 at 01:25:37AM +0000, Huw Lloyd wrote: > >>> >> Arbitrarily picking an earlier version ... 4.1 does not have this > >>> >> issue/bug. > >>> >> > >>> >> Any quick fixes for this? I'm interested in using this in > >>> >> conjunction > >>> >> with > >>> >> aida. > >>> >> > >>> >> Best, > >>> >> Huw > >>> >> > >>> >> > >>> >> On 3 December 2015 at 01:13, Huw Lloyd > >>> wrote: > >>> >> > >>> >> > For both 4.6 & 5.0 on the mac, saving a project generates the > >>> following > >>> >> > error: "NotImplemented: ClassBinding or a superclass should > >>> >> > implement > >>> >> > objectForDataStream." > >>> >> > > >>> >> > The comments in the method suggest this shouldn't have been > >>> >> > released > >>> to > >>> >> > a > >>> >> > stable version: "It's not yet clear how serialization should work > >>> >> > in > >>> >> > the > >>> >> > presence of environments". > >>> >> > > >>> >> > Problem 2 is that if I then "abandon" the error messge, the > >>> >> > workspace > >>> >> > is > >>> >> > then flooded with "MessageNotUnderstood" error boxes. > >>> >> > > >>> >> > Is "environments" something experimental that should be switched > >>> >> > off? > >>> >> > > >>> >> > Best, > >>> >> > Huw > >>> >> > > >>> >> > > >>> >> > > >>> > > >>> >> > >>> > > >>> > > >>> > > >>> > >>> > >> > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151203/b13a34a8/attachment-0001.htm From huw.softdesigns at gmail.com Thu Dec 3 17:19:55 2015 From: huw.softdesigns at gmail.com (Huw Lloyd) Date: Thu Dec 3 17:19:57 2015 Subject: [squeak-dev] Re: Mac release broken? ClassBinding>>objectForDataStream In-Reply-To: References: <20151203020034.GA32593@shell.msen.com> Message-ID: I don't know enough about squeak to determine the length of that task (2nd day of squeaking for me). However, the modified 4.6 does now save (after loading the newer environment package and loading the classes from the system package). Best, Huw On 3 December 2015 at 15:06, H. Hirzel wrote: > As direct loading does not work I suggest something like copy/paste > updated methods from a Squeak 5 trunk image one by one. > > back to the 4.6 image. > > I.e. open the mcz file in a code browser but do not load it. > > If necessary the code has to be adapted to cater for updates which > happened elsewhere in the meantime as you write > > #includesSubString: -> #includesSubstring: > #subStrings -> #substrings > > It seems that there not all that many fixes. > > > --Hannes > > On 12/3/15, Huw Lloyd wrote: > > Looks earliest System package to not load to 4.6 is: > > > > Name: System-ul.757 > > Author: ul > > Time: 14 August 2015, 7:19:00.474 pm > > UUID: fca51b99-736e-4666-acac-e5c5b6a1d397 > > Ancestors: System-cmm.756 > > > > #includesSubString: -> #includesSubstring: > > #subStrings -> #substrings > > > > Best, > > Huw > > > > > > On 3 December 2015 at 14:43, Huw Lloyd > wrote: > > > >> Hannes, > >> > >> I got as far as a failure to load System-cwp.781 with missing dependency > >> on class UnhandledWarning: > >> > >> "Warning: This package depends on the following classes: > >> UnhandledWarning > >> You must resolve these dependencies before you will be able to load > these > >> definitions: > >> UnhandledWarning>>defaultAction" > >> > >> The version of package system for 4.6 is System-topa.753 > >> > >> Best, > >> Huw > >> > >> > >> On 3 December 2015 at 08:57, H. Hirzel wrote: > >> > >>> Huw, > >>> > >>> See a proposal for a quick fix below using the work of Colin Putney > last > >>> week. > >>> > >>> --Hannes > >>> > >>> On 12/3/15, David T. Lewis wrote: > >>> > Huw, > >>> > > >>> > The timing of your question is perfect :-) > >>> > > >>> > There are some very recent updates (in the last week or so) to the > >>> Squeak > >>> > trunk development stream that address this problem. > >>> > >>> A copy of the mail describing these updates is here > >>> http://wiki.squeak.org/squeak/6218 > >>> > >>> This is work in > >>> > progress, > >>> > but in a nutshell, the latest trunk fixes should resolve the problem > >>> > for > >>> > 4.6 images, but there is work still to be done in order to make it > >>> > work > >>> > in 5.0 Spur. We still need to get these latest updates into the > >>> > standard > >>> > update stream for squeak46, but as soon as that is complete it should > >>> work > >>> > for those images. > >>> > >>> The updates Colin Putney committed to the trunk on Friday, Saturday > >>> 27/28th Nov are > >>> > >>> Name: Environments-cwp.60 > >>> Author: cwp > >>> Time: 27 November 2015, 5:14:36.376 pm > >>> UUID: c8836cde-f21e-4652-a2e9-73f8de6f00e0 > >>> Ancestors: Environments-kfr.59 > >>> > >>> Always serialize Bindings as DiskProxies. > >>> > >>> > >>> Name: System-cwp.781 > >>> Author: cwp > >>> Time: 27 November 2015, 5:22:38.591 pm > >>> UUID: c0ffbab7-f7ee-4966-ba07-d57436ac42f2 > >>> Ancestors: System-mt.780 > >>> > >>> Remove specialized serialization of Associations. > >>> Since associations are no longer used as globals, we never need to > >>> serialize them using DiskProxy. > >>> > >>> > >>> Name: System-cwp.782 > >>> Author: cwp > >>> Time: 27 November 2015, 9:00:42.904 pm > >>> UUID: ae1f1166-0438-4e78-8ded-46c59c212210 > >>> Ancestors: System-cwp.781 > >>> > >>> Reload DiskProxies using the current environment > >>> > >>> When looking up objects from the destination image, DiskProxies should > >>> use the current environment, rather than referring to to Smalltalk. > >>> This lets objects on disk be loaded into arbitrary environments. > >>> > >>> These updates are committed to the trunk (i.e. the ongoing development > >>> after the release of 5.0). However in a trunk image > >>> > >>> > http://build.squeak.org/job/Trunk/default/lastSuccessfulBuild/artifact/target/TrunkImage.zip > >>> they do NOT work yet. > >>> > >>> > >>> What you could do as of now > >>> ----------------------------------------- > >>> > >>> - File in the updates above into a 4.6 image. > >>> - It should fix the problem (I have not tried yet, so please report if > >>> it is successful). > >>> > >>> > >>> > >>> > >>> > I am less certain of the timing for 5.0 images. > >>> > >>> The reason is that the new VM crashes, and Colin has reported this to > >>> the Virtual Machine List. > >>> > >>> > >>> > >>> > Dave > >>> > > >>> > > >>> > > >>> > On Thu, Dec 03, 2015 at 01:25:37AM +0000, Huw Lloyd wrote: > >>> >> Arbitrarily picking an earlier version ... 4.1 does not have this > >>> >> issue/bug. > >>> >> > >>> >> Any quick fixes for this? I'm interested in using this in > >>> >> conjunction > >>> >> with > >>> >> aida. > >>> >> > >>> >> Best, > >>> >> Huw > >>> >> > >>> >> > >>> >> On 3 December 2015 at 01:13, Huw Lloyd > >>> wrote: > >>> >> > >>> >> > For both 4.6 & 5.0 on the mac, saving a project generates the > >>> following > >>> >> > error: "NotImplemented: ClassBinding or a superclass should > >>> >> > implement > >>> >> > objectForDataStream." > >>> >> > > >>> >> > The comments in the method suggest this shouldn't have been > >>> >> > released > >>> to > >>> >> > a > >>> >> > stable version: "It's not yet clear how serialization should work > >>> >> > in > >>> >> > the > >>> >> > presence of environments". > >>> >> > > >>> >> > Problem 2 is that if I then "abandon" the error messge, the > >>> >> > workspace > >>> >> > is > >>> >> > then flooded with "MessageNotUnderstood" error boxes. > >>> >> > > >>> >> > Is "environments" something experimental that should be switched > >>> >> > off? > >>> >> > > >>> >> > Best, > >>> >> > Huw > >>> >> > > >>> >> > > >>> >> > > >>> > > >>> >> > >>> > > >>> > > >>> > > >>> > >>> > >> > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151203/7222421d/attachment.htm From huw.softdesigns at gmail.com Thu Dec 3 17:22:08 2015 From: huw.softdesigns at gmail.com (Huw Lloyd) Date: Thu Dec 3 17:22:12 2015 Subject: [squeak-dev] Re: Mac release broken? ClassBinding>>objectForDataStream In-Reply-To: References: <20151203020034.GA32593@shell.msen.com> Message-ID: On 3 December 2015 at 17:08, Colin Putney wrote: > I'll backport the trunk changes to 4.6 tonight. > > Thanks, Colin. That will also enable new users to save projects. :) Best, Huw > On Thu, Dec 3, 2015 at 7:06 AM, H. Hirzel wrote: > >> As direct loading does not work I suggest something like copy/paste >> updated methods from a Squeak 5 trunk image one by one. >> >> back to the 4.6 image. >> >> I.e. open the mcz file in a code browser but do not load it. >> >> If necessary the code has to be adapted to cater for updates which >> happened elsewhere in the meantime as you write >> >> #includesSubString: -> #includesSubstring: >> #subStrings -> #substrings >> >> It seems that there not all that many fixes. >> >> >> --Hannes >> >> On 12/3/15, Huw Lloyd wrote: >> > Looks earliest System package to not load to 4.6 is: >> > >> > Name: System-ul.757 >> > Author: ul >> > Time: 14 August 2015, 7:19:00.474 pm >> > UUID: fca51b99-736e-4666-acac-e5c5b6a1d397 >> > Ancestors: System-cmm.756 >> > >> > #includesSubString: -> #includesSubstring: >> > #subStrings -> #substrings >> > >> > Best, >> > Huw >> > >> > >> > On 3 December 2015 at 14:43, Huw Lloyd >> wrote: >> > >> >> Hannes, >> >> >> >> I got as far as a failure to load System-cwp.781 with missing >> dependency >> >> on class UnhandledWarning: >> >> >> >> "Warning: This package depends on the following classes: >> >> UnhandledWarning >> >> You must resolve these dependencies before you will be able to load >> these >> >> definitions: >> >> UnhandledWarning>>defaultAction" >> >> >> >> The version of package system for 4.6 is System-topa.753 >> >> >> >> Best, >> >> Huw >> >> >> >> >> >> On 3 December 2015 at 08:57, H. Hirzel >> wrote: >> >> >> >>> Huw, >> >>> >> >>> See a proposal for a quick fix below using the work of Colin Putney >> last >> >>> week. >> >>> >> >>> --Hannes >> >>> >> >>> On 12/3/15, David T. Lewis wrote: >> >>> > Huw, >> >>> > >> >>> > The timing of your question is perfect :-) >> >>> > >> >>> > There are some very recent updates (in the last week or so) to the >> >>> Squeak >> >>> > trunk development stream that address this problem. >> >>> >> >>> A copy of the mail describing these updates is here >> >>> http://wiki.squeak.org/squeak/6218 >> >>> >> >>> This is work in >> >>> > progress, >> >>> > but in a nutshell, the latest trunk fixes should resolve the problem >> >>> > for >> >>> > 4.6 images, but there is work still to be done in order to make it >> >>> > work >> >>> > in 5.0 Spur. We still need to get these latest updates into the >> >>> > standard >> >>> > update stream for squeak46, but as soon as that is complete it >> should >> >>> work >> >>> > for those images. >> >>> >> >>> The updates Colin Putney committed to the trunk on Friday, Saturday >> >>> 27/28th Nov are >> >>> >> >>> Name: Environments-cwp.60 >> >>> Author: cwp >> >>> Time: 27 November 2015, 5:14:36.376 pm >> >>> UUID: c8836cde-f21e-4652-a2e9-73f8de6f00e0 >> >>> Ancestors: Environments-kfr.59 >> >>> >> >>> Always serialize Bindings as DiskProxies. >> >>> >> >>> >> >>> Name: System-cwp.781 >> >>> Author: cwp >> >>> Time: 27 November 2015, 5:22:38.591 pm >> >>> UUID: c0ffbab7-f7ee-4966-ba07-d57436ac42f2 >> >>> Ancestors: System-mt.780 >> >>> >> >>> Remove specialized serialization of Associations. >> >>> Since associations are no longer used as globals, we never need to >> >>> serialize them using DiskProxy. >> >>> >> >>> >> >>> Name: System-cwp.782 >> >>> Author: cwp >> >>> Time: 27 November 2015, 9:00:42.904 pm >> >>> UUID: ae1f1166-0438-4e78-8ded-46c59c212210 >> >>> Ancestors: System-cwp.781 >> >>> >> >>> Reload DiskProxies using the current environment >> >>> >> >>> When looking up objects from the destination image, DiskProxies should >> >>> use the current environment, rather than referring to to Smalltalk. >> >>> This lets objects on disk be loaded into arbitrary environments. >> >>> >> >>> These updates are committed to the trunk (i.e. the ongoing development >> >>> after the release of 5.0). However in a trunk image >> >>> >> >>> >> http://build.squeak.org/job/Trunk/default/lastSuccessfulBuild/artifact/target/TrunkImage.zip >> >>> they do NOT work yet. >> >>> >> >>> >> >>> What you could do as of now >> >>> ----------------------------------------- >> >>> >> >>> - File in the updates above into a 4.6 image. >> >>> - It should fix the problem (I have not tried yet, so please report if >> >>> it is successful). >> >>> >> >>> >> >>> >> >>> >> >>> > I am less certain of the timing for 5.0 images. >> >>> >> >>> The reason is that the new VM crashes, and Colin has reported this to >> >>> the Virtual Machine List. >> >>> >> >>> >> >>> >> >>> > Dave >> >>> > >> >>> > >> >>> > >> >>> > On Thu, Dec 03, 2015 at 01:25:37AM +0000, Huw Lloyd wrote: >> >>> >> Arbitrarily picking an earlier version ... 4.1 does not have this >> >>> >> issue/bug. >> >>> >> >> >>> >> Any quick fixes for this? I'm interested in using this in >> >>> >> conjunction >> >>> >> with >> >>> >> aida. >> >>> >> >> >>> >> Best, >> >>> >> Huw >> >>> >> >> >>> >> >> >>> >> On 3 December 2015 at 01:13, Huw Lloyd >> >>> wrote: >> >>> >> >> >>> >> > For both 4.6 & 5.0 on the mac, saving a project generates the >> >>> following >> >>> >> > error: "NotImplemented: ClassBinding or a superclass should >> >>> >> > implement >> >>> >> > objectForDataStream." >> >>> >> > >> >>> >> > The comments in the method suggest this shouldn't have been >> >>> >> > released >> >>> to >> >>> >> > a >> >>> >> > stable version: "It's not yet clear how serialization should work >> >>> >> > in >> >>> >> > the >> >>> >> > presence of environments". >> >>> >> > >> >>> >> > Problem 2 is that if I then "abandon" the error messge, the >> >>> >> > workspace >> >>> >> > is >> >>> >> > then flooded with "MessageNotUnderstood" error boxes. >> >>> >> > >> >>> >> > Is "environments" something experimental that should be switched >> >>> >> > off? >> >>> >> > >> >>> >> > Best, >> >>> >> > Huw >> >>> >> > >> >>> >> > >> >>> >> > >> >>> > >> >>> >> >> >>> > >> >>> > >> >>> > >> >>> >> >>> >> >> >> > >> >> > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151203/fe6ced75/attachment.htm From asqueaker at gmail.com Thu Dec 3 17:35:54 2015 From: asqueaker at gmail.com (Chris Muller) Date: Thu Dec 3 17:36:38 2015 Subject: [squeak-dev] Re: Mac release broken? ClassBinding>>objectForDataStream In-Reply-To: <20151203020034.GA32593@shell.msen.com> References: <20151203020034.GA32593@shell.msen.com> Message-ID: > There are some very recent updates (in the last week or so) to the Squeak > trunk development stream that address this problem. This is work in progress, > but in a nutshell, the latest trunk fixes should resolve the problem for > 4.6 images, but there is work still to be done in order to make it work > in 5.0 Spur. We still need to get these latest updates into the standard > update stream for squeak46, but as soon as that is complete it should work > for those images. I am less certain of the timing for 5.0 images. I must've missed the conversation; what is the problem with 5.0 that Colin's Environments fix won't work? From hannes.hirzel at gmail.com Thu Dec 3 17:51:15 2015 From: hannes.hirzel at gmail.com (H. Hirzel) Date: Thu Dec 3 17:51:18 2015 Subject: [squeak-dev] Re: Mac release broken? ClassBinding>>objectForDataStream In-Reply-To: References: <20151203020034.GA32593@shell.msen.com> Message-ID: Here is Colin's report of last Saturday http://wiki.squeak.org/squeak/6218 The new VM with the trunk images crashes with the environment fixes of Colin whereas the fixes work in 4.6 --Hannes On 12/3/15, Chris Muller wrote: >> There are some very recent updates (in the last week or so) to the Squeak >> trunk development stream that address this problem. This is work in >> progress, >> but in a nutshell, the latest trunk fixes should resolve the problem for >> 4.6 images, but there is work still to be done in order to make it work >> in 5.0 Spur. We still need to get these latest updates into the standard >> update stream for squeak46, but as soon as that is complete it should >> work >> for those images. I am less certain of the timing for 5.0 images. > > I must've missed the conversation; what is the problem with 5.0 that > Colin's Environments fix won't work? > > From colin at wiresong.com Thu Dec 3 18:16:11 2015 From: colin at wiresong.com (Colin Putney) Date: Thu Dec 3 18:16:13 2015 Subject: [squeak-dev] Re: Mac release broken? ClassBinding>>objectForDataStream In-Reply-To: References: <20151203020034.GA32593@shell.msen.com> Message-ID: On Thu, Dec 3, 2015 at 9:35 AM, Chris Muller wrote: > I must've missed the conversation; what is the problem with 5.0 that > Colin's Environments fix won't work? > It's basically that Spur doesn't support ImageSegments yet. It crashes on when saving a segment, and fails to load one properly (but doesn't crash). -Colin -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151203/b3118c16/attachment.htm From robert.w.withers at gmail.com Thu Dec 3 22:09:04 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Thu Dec 3 22:09:10 2015 Subject: [squeak-dev] passwd reset for squeaksource Message-ID: <5660BD80.6000007@gmail.com> Could somebody help reset my passwd for squeaksource? I can't use the process to do so. My user_id is rww. thank you, Robert From lewis at mail.msen.com Thu Dec 3 22:34:30 2015 From: lewis at mail.msen.com (David T. Lewis) Date: Thu Dec 3 22:34:32 2015 Subject: [squeak-dev] passwd reset for squeaksource In-Reply-To: <5660BD80.6000007@gmail.com> References: <5660BD80.6000007@gmail.com> Message-ID: <43331.136.2.1.104.1449182070.squirrel@webmail.msen.com> > Could somebody help reset my passwd for squeaksource? I can't use the > process to do so. My user_id is rww. > > thank you, > Robert > Done (see private email) Dave From commits at source.squeak.org Thu Dec 3 22:55:02 2015 From: commits at source.squeak.org (commits@source.squeak.org) Date: Thu Dec 3 22:55:03 2015 Subject: [squeak-dev] Daily Commit Log Message-ID: <20151203225502.594.qmail@box4.squeakfoundation.org> Changes to Trunk (http://source.squeak.org/trunk.html) in the last 24 hours: http://lists.squeakfoundation.org/pipermail/packages/2015-December/009203.html Name: Tools-eem.657 Ancestors: Tools-kfr.656 Provide local senders... in the system browser which will browse methods that reference a selector or a literal within the selected class's hierarchy. Modify selection in MessageNames so that if one selects a different selector and an implementation exists in the same class as the current method, select that method. Provide a canonical message name for CompiledMethod for creating methods from exceptions in the debugger. ============================================= http://lists.squeakfoundation.org/pipermail/packages/2015-December/009204.html Name: Environments-kfr.61 Ancestors: Environments-cwp.60 Adds help interface so one can read all class and method comments in a HelpBrowser ============================================= http://lists.squeakfoundation.org/pipermail/packages/2015-December/009205.html Name: WebClient-Help-kfr.11 Ancestors: WebClient-Help-ar.10 Fix for help referece so API documentation show in HelpBrowser ============================================= From robert.w.withers at gmail.com Thu Dec 3 23:55:13 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Thu Dec 3 23:55:18 2015 Subject: [squeak-dev] passwd reset for squeaksource In-Reply-To: <43331.136.2.1.104.1449182070.squirrel@webmail.msen.com> References: <5660BD80.6000007@gmail.com> <43331.136.2.1.104.1449182070.squirrel@webmail.msen.com> Message-ID: <5660D661.2050104@gmail.com> Thank you kindly, I've gotten established there, robert On 12/03/2015 05:34 PM, David T. Lewis wrote: >> Could somebody help reset my passwd for squeaksource? I can't use the >> process to do so. My user_id is rww. >> >> thank you, >> Robert >> > Done (see private email) > > Dave > > > From commits at source.squeak.org Fri Dec 4 00:28:31 2015 From: commits at source.squeak.org (commits@source.squeak.org) Date: Fri Dec 4 00:28:32 2015 Subject: [squeak-dev] The Trunk: Morphic-jmg.1055.mcz Message-ID: David T. Lewis uploaded a new version of Morphic to project The Trunk: http://source.squeak.org/trunk/Morphic-jmg.1055.mcz ==================== Summary ==================== Name: Morphic-jmg.1055 Author: jmg Time: 27 November 2015, 11:34:49.151 pm UUID: 2866bfd1-711c-4f03-b944-b8d4adb0f6cb Ancestors: Morphic-mt.1054 If you're like me and have the swapMouseButtons perference unselected because you are on windows, then right-clicking on a morph and picking the 'select' menu item raises an exception because popUpFor:at:hand: is expecting a position for the at: argument, and currently this sends in a MouseEvent transformed by the relevant amount. We need to take one additional step and extract the position from this newly translated event, which I do in this change. =============== Diff against Morphic-mt.1054 =============== Item was changed: ----- Method: SimpleHaloMorph>>popUpFor:hand: (in category 'pop up') ----- popUpFor: morph hand: hand self popUpFor: morph + at: (hand lastEvent transformedBy: (morph transformedFrom: nil)) position - at: (hand lastEvent transformedBy: (morph transformedFrom: nil)) hand: hand! From commits at source.squeak.org Fri Dec 4 00:29:11 2015 From: commits at source.squeak.org (commits@source.squeak.org) Date: Fri Dec 4 00:29:12 2015 Subject: [squeak-dev] The Trunk: Morphic-cwp.1055.mcz Message-ID: David T. Lewis uploaded a new version of Morphic to project The Trunk: http://source.squeak.org/trunk/Morphic-cwp.1055.mcz ==================== Summary ==================== Name: Morphic-cwp.1055 Author: cwp Time: 27 November 2015, 9:46:27.986 pm UUID: 75600228-b4f9-4739-b9c8-0318e1ef1d3a Ancestors: Morphic-mt.1054 Implement PasteUpMorph>>referencePool Loading a project seems to require it. =============== Diff against Morphic-mt.1054 =============== Item was added: + ----- Method: PasteUpMorph>>referencePool (in category 'objects from disk') ----- + referencePool + | pool | + pool := OrderedCollection new. + self setProperty: #References toValue: pool. + ^ pool! From commits at source.squeak.org Fri Dec 4 00:34:47 2015 From: commits at source.squeak.org (commits@source.squeak.org) Date: Fri Dec 4 00:34:49 2015 Subject: [squeak-dev] The Trunk: Morphic-dtl.1056.mcz Message-ID: David T. Lewis uploaded a new version of Morphic to project The Trunk: http://source.squeak.org/trunk/Morphic-dtl.1056.mcz ==================== Summary ==================== Name: Morphic-dtl.1056 Author: dtl Time: 3 December 2015, 7:33:58.442 pm UUID: fd009a5d-4910-45be-bd72-6121f05689f1 Ancestors: Morphic-jmg.1055, Morphic-cwp.1055 Merge Morphic-jmg.1055, Morphic-cwp.1055 =============== Diff against Morphic-jmg.1055 =============== Item was added: + ----- Method: PasteUpMorph>>referencePool (in category 'objects from disk') ----- + referencePool + | pool | + pool := OrderedCollection new. + self setProperty: #References toValue: pool. + ^ pool! From tim at rowledge.org Fri Dec 4 00:54:01 2015 From: tim at rowledge.org (tim Rowledge) Date: Fri Dec 4 00:53:51 2015 Subject: [squeak-dev] mouse enter/move/leave event confusion In-Reply-To: <1449142410592-4864917.post@n4.nabble.com> References: <4973AB59-FF96-41B1-AE15-904AAE8752D9@rowledge.org> <1449051662880-4864703.post@n4.nabble.com> <655B52F2-8736-476F-9B62-4A279AF46ECE@rowledge.org> <7D3C4F24-F59F-4C93-BE45-A454D91EB853@rowledge.org> <1449142410592-4864917.post@n4.nabble.com> Message-ID: Hi, so now I know why I can get so many spurious enter/leave events. It?s really embarrassingly dumb? When you are dragging a morph around and have the morph positioned at the event cursor point - ie not centred under it as you eventually want - then moving even moderately quickly can result in the cursor point being over the dragged morph. Which is not the canvas morph. So you?ve left the canvas morph; and you get a mouse leave event. D?oh! Then the fake-cursor-morph gets deleted, you are now over the canvas morph so you get a mouse enter event, and so on. I worked this out eventually because I added the centring offset negated and so the fake-cursor was well out of the way and hardly any enter/leave events came in. When I ?corrected' the offset, everything was flashing madly as gazillions of enter/leaves flooded the place since the morph was directly under the cursor and thus always causing a leave event. Thus my conclusion is that we can?t drag a morph around like this. Sigh. It looks like adding some code to grid-step in HandMorph>position: is the only way to proceed now. I?m going to work on the assumption that only the temporary cursor needs to worry about it, though if people want the cursor in general to grid it wouldn?t be hard to expand that. I?m not hugely keen on messing with HandMorph since it is so central to the UI so I?ll stick with a couple of globals or properties for the gridding info for now. One question for you all - if I added a subclass GriddingHandMorph, how tricky is it to swap the current hand around. Could I reasonably swap to a gridded hand when entering my canvas morph and back to an ordinary one on leave? Would it be saner to just swap to a gridded hand in my product image and leave it at that? tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Strange OpCodes: TVO: Type Various Obscenities From tim at rowledge.org Fri Dec 4 02:14:34 2015 From: tim at rowledge.org (tim Rowledge) Date: Fri Dec 4 02:14:23 2015 Subject: [squeak-dev] mouse enter/move/leave event confusion In-Reply-To: References: <4973AB59-FF96-41B1-AE15-904AAE8752D9@rowledge.org> <1449051662880-4864703.post@n4.nabble.com> <655B52F2-8736-476F-9B62-4A279AF46ECE@rowledge.org> <7D3C4F24-F59F-4C93-BE45-A454D91EB853@rowledge.org> <1449142410592-4864917.post@n4.nabble.com> Message-ID: > On 03-12-2015, at 4:54 PM, tim Rowledge wrote: > > It looks like adding some code to grid-step in HandMorph>position: is the only way to proceed now. > I?m going to work on the assumption that only the temporary cursor needs to worry about it, though if people want the cursor in general to grid it wouldn?t be hard to expand that. I?m not hugely keen on messing with HandMorph since it is so central to the UI so I?ll stick with a couple of globals or properties for the gridding info for now. Using a couple of properties for now I can get pretty good results. There are a couple of places in my painting code that need mild hacks because the events sent are*not* gridded right now, only the hand position is and only when using a temporary cursor as well. IF everyone likes the idea of having the ability to grid the cursor and mouse event positions as a general thing we could look into all the places that would need changes. tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Useful Latin Phrases:- Mellita, domi adsum. = Honey, I'm home. From lewis at mail.msen.com Fri Dec 4 04:10:18 2015 From: lewis at mail.msen.com (David T. Lewis) Date: Fri Dec 4 04:10:20 2015 Subject: [squeak-dev] The Trunk: Morphic-dtl.1056.mcz In-Reply-To: <201512040034.tB40YnE3009549@shell.msen.com> References: <201512040034.tB40YnE3009549@shell.msen.com> Message-ID: <20151204041018.GA42787@shell.msen.com> On Fri, Dec 04, 2015 at 12:34:07AM +0000, commits@source.squeak.org wrote: > David T. Lewis uploaded a new version of Morphic to project The Trunk: > http://source.squeak.org/trunk/Morphic-dtl.1056.mcz > > ==================== Summary ==================== > > Name: Morphic-dtl.1056 > Author: dtl > Time: 3 December 2015, 7:33:58.442 pm > UUID: fd009a5d-4910-45be-bd72-6121f05689f1 > Ancestors: Morphic-jmg.1055, Morphic-cwp.1055 > > Merge Morphic-jmg.1055, Morphic-cwp.1055 These two have been in the inbox long enough, so I moved them to trunk. Morphic-jmg.1055 seems straightforward, and Morphic-cwp.1055 is needed for image segments but would still benefit from review and comment as per Colin's earlier request. Dave From Marcel.Taeumel at hpi.de Fri Dec 4 08:16:34 2015 From: Marcel.Taeumel at hpi.de (marcel.taeumel) Date: Fri Dec 4 08:33:13 2015 Subject: [squeak-dev] Re: mouse enter/move/leave event confusion In-Reply-To: References: <4973AB59-FF96-41B1-AE15-904AAE8752D9@rowledge.org> <1449051662880-4864703.post@n4.nabble.com> <655B52F2-8736-476F-9B62-4A279AF46ECE@rowledge.org> <7D3C4F24-F59F-4C93-BE45-A454D91EB853@rowledge.org> <1449142410592-4864917.post@n4.nabble.com> Message-ID: <1449216994126-4865058.post@n4.nabble.com> You can try optimizing hat mouse enter/leave thing with HandMorph >> #newMouseFocus:. If your "canvas" morph receives all the mouse events, you can drop additional mouse enter/leave events. Do you have some code for us? ;) Best, Marcel -- View this message in context: http://forum.world.st/mouse-enter-move-leave-event-confusion-tp4864651p4865058.html Sent from the Squeak - Dev mailing list archive at Nabble.com. From stephan at stack.nl Fri Dec 4 10:33:19 2015 From: stephan at stack.nl (Stephan Eggermont) Date: Fri Dec 4 10:33:35 2015 Subject: [squeak-dev] Re: mouse enter/move/leave event confusion In-Reply-To: References: <4973AB59-FF96-41B1-AE15-904AAE8752D9@rowledge.org> <1449051662880-4864703.post@n4.nabble.com> <655B52F2-8736-476F-9B62-4A279AF46ECE@rowledge.org> <7D3C4F24-F59F-4C93-BE45-A454D91EB853@rowledge.org> <1449142410592-4864917.post@n4.nabble.com> Message-ID: On 04-12-15 01:54, tim Rowledge wrote: > Hi, > so now I know why I can get so many spurious enter/leave events. It?s really embarrassingly dumb? > > When you are dragging a morph around and have the morph positioned at the event cursor point - ie not centred under it as you eventually want - then moving even moderately quickly can result in the cursor point being over the dragged morph. Which is not the canvas morph. So you?ve left the canvas morph; and you get a mouse leave event. D?oh! Then the fake-cursor-morph gets deleted, you are now over the canvas morph so you get a mouse enter event, and so on. Hmm. conceptually I would expect to get no spurious enter/leaves when the morph you're dragging around is either a submorph of the canvas, or grabbed by the hand. Stephan From robert.w.withers at gmail.com Fri Dec 4 10:44:44 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Fri Dec 4 10:44:49 2015 Subject: [squeak-dev] Re: [Pharo-users] evolutions of squeakelib & crypto (Reed Solomon) In-Reply-To: References: <565DCE53.90501@gmail.com> <565DD073.1060409@gmail.com> <565FF8ED.1030606@free.fr> <5660BCFA.4050107@gmail.com> Message-ID: <56616E9C.6040201@gmail.com> After my password reset on squeaksource, I committed to both Cryptography and SqueakElib, project links below. In the case of Cryptography, I had a version ported to Pharo ... I will organize Pharo ports of both after Reed Solomon is stable, and announce them to the Pharo list. This way both environments can be supported through this one repository. Robert On 12/04/2015 04:35 AM, Stephan Eggermont wrote: > On 03-12-15 23:06, Robert Withers wrote: >> Are any of these used by both squeak and Pharo? That would be the right >> move I think. I will ask about getting my password reset for >> squeaksource, since that is where the old code resides. > > All of them. Mostly timing of project start/high activity and who are > maintaining it decided on platforms. There was a time when > squeaksource was not so stable and then many projects migrated, and I > currently hear least about stability issues from ss3, but the load on > smalltalkhub is much higher, I assume, as that is used for the pharo ci. > > In the not so far future Pharo is likely to move to a git based > infrastructure, using libgit2. Early adopters are already using it. > I haven't heard the squeak ideas about that. > > Stephan > > > From robert.w.withers at gmail.com Fri Dec 4 10:47:03 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Fri Dec 4 10:47:08 2015 Subject: [squeak-dev] Re: [Pharo-users] evolutions of squeakelib & crypto (Reed Solomon) In-Reply-To: <56616E9C.6040201@gmail.com> References: <565DCE53.90501@gmail.com> <565DD073.1060409@gmail.com> <565FF8ED.1030606@free.fr> <5660BCFA.4050107@gmail.com> <56616E9C.6040201@gmail.com> Message-ID: <56616F27.4070002@gmail.com> Best Regards http://www.squeaksource.com/Cryptography.html http://www.squeaksource.com/squeakelib.html On 12/04/2015 05:44 AM, Robert Withers wrote: > After my password reset on squeaksource, I committed to both > Cryptography and SqueakElib, project links below. > > In the case of Cryptography, I had a version ported to Pharo ... I > will organize Pharo ports of both after Reed Solomon is stable, and > announce them to the Pharo list. This way both environments can be > supported through this one repository. > > Robert > > On 12/04/2015 04:35 AM, Stephan Eggermont wrote: >> On 03-12-15 23:06, Robert Withers wrote: >>> Are any of these used by both squeak and Pharo? That would be the right >>> move I think. I will ask about getting my password reset for >>> squeaksource, since that is where the old code resides. >> >> All of them. Mostly timing of project start/high activity and who are >> maintaining it decided on platforms. There was a time when >> squeaksource was not so stable and then many projects migrated, and I >> currently hear least about stability issues from ss3, but the load on >> smalltalkhub is much higher, I assume, as that is used for the pharo ci. >> >> In the not so far future Pharo is likely to move to a git based >> infrastructure, using libgit2. Early adopters are already using it. >> I haven't heard the squeak ideas about that. >> >> Stephan >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151204/67b1fd9f/attachment.htm From robert.w.withers at gmail.com Fri Dec 4 11:15:54 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Fri Dec 4 11:15:59 2015 Subject: [squeak-dev] Re: [Pharo-users] evolutions of squeakelib & crypto (Reed Solomon) In-Reply-To: <566174E9.9020001@gmail.com> References: <565DCE53.90501@gmail.com> <565DD073.1060409@gmail.com> <565FF8ED.1030606@free.fr> <5660BCFA.4050107@gmail.com> <56616E9C.6040201@gmail.com> <56616F27.4070002@gmail.com> <566174E9.9020001@gmail.com> Message-ID: <566175EA.2090107@gmail.com> I just realized that the squeak version uses ReferenceStream while the Pharo version uses Fuel, so the binary serializations are different and they won't speak to each other. Any chance that Fuel is ported to Squeak? Regards, Robert On 12/04/2015 06:11 AM, Robert Withers wrote: > I am unable to import these files into SqueakSource, so it may be best > done from inside Pharo with Monticello. Here are working Crypto and > SqueakElib in Pharo, prior to SecureSession refactoring and Reed > Solomon. I include the correct version of LayeredProtocol. > > Regards, > Robert > > > On 12/04/2015 05:47 AM, Robert Withers wrote: >> Best Regards >> >> http://www.squeaksource.com/Cryptography.html >> http://www.squeaksource.com/squeakelib.html >> >> >> On 12/04/2015 05:44 AM, Robert Withers wrote: >>> After my password reset on squeaksource, I committed to both >>> Cryptography and SqueakElib, project links below. >>> >>> In the case of Cryptography, I had a version ported to Pharo ... I >>> will organize Pharo ports of both after Reed Solomon is stable, and >>> announce them to the Pharo list. This way both environments can be >>> supported through this one repository. >>> >>> Robert >>> >>> On 12/04/2015 04:35 AM, Stephan Eggermont wrote: >>>> On 03-12-15 23:06, Robert Withers wrote: >>>>> Are any of these used by both squeak and Pharo? That would be the >>>>> right >>>>> move I think. I will ask about getting my password reset for >>>>> squeaksource, since that is where the old code resides. >>>> >>>> All of them. Mostly timing of project start/high activity and who are >>>> maintaining it decided on platforms. There was a time when >>>> squeaksource was not so stable and then many projects migrated, and >>>> I currently hear least about stability issues from ss3, but the >>>> load on smalltalkhub is much higher, I assume, as that is used for >>>> the pharo ci. >>>> >>>> In the not so far future Pharo is likely to move to a git based >>>> infrastructure, using libgit2. Early adopters are already using it. >>>> I haven't heard the squeak ideas about that. >>>> >>>> Stephan >>>> >>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151204/3ceb48e3/attachment-0001.htm From euanmee at gmail.com Fri Dec 4 11:42:11 2015 From: euanmee at gmail.com (EuanM) Date: Fri Dec 4 11:42:14 2015 Subject: [squeak-dev] Unicode Support Message-ID: I'm currently groping my way to seeing how feature-complete our Unicode support is. I am doing this to establish what still needs to be done to provide full Unicode support. This seems to me to be an area where it would be best to write it once, and then have the same codebase incorporated into the Smalltalks that most share a common ancestry. I am keen to get: equality-testing for strings; sortability for strings which have ligatures and diacritic characters; and correct round-tripping of data. Call to action: ========== If you have comments on these proposals - such as "but we already have that facility" or "the reason we do not have these facilities is because they are dog-slow" - please let me know them. If you would like to help out, please let me know. If you have Unicode experience and expertise, and would like to be, or would be willing to be, in the 'council of experts' for this project, please let me know. If you have comments or ideas on anything mentioned in this email In the first instance, the initiative's website will be: http://smalltalk.uk.to/unicode.html I have created a SqueakSource.com project called UnicodeSupport I want to avoid re-inventing any facilities which already exist. Except where they prevent us reaching the goals of: - sortable UTF8 strings - sortable UTF16 strings - equivalence testing of 2 UTF8 strings - equivalence testing of 2 UTF16 strings - round-tripping UTF8 strings through Smalltalk - roundtripping UTF16 strings through Smalltalk. As I understand it, we have limited Unicode support atm. Current state of play =============== ByteString gets converted to WideString when need is automagically detected. Is there anything else that currently exists? Definition of Terms ============== A quick definition of terms before I go any further: Standard terms from the Unicode standard =============================== a compatibility character : an additional encoding of a *normal* character, for compatibility and round-trip conversion purposes. For instance, a 1-byte encoding of a Latin character with a diacritic. Made-up terms ============ a convenience codepoint : a single codepoint which represents an item that is also encoded as a string of codepoints. (I tend to use the terms compatibility character and compatibility codepoint interchangably. The standard only refers to them as compatibility characters. However, the standard is determined to emphasise that characters are abstract and that codepoints are concrete. So I think it is often more useful and productive to think of compatibility or convenience codepoints). a composed character : a character made up of several codepoints Unicode encoding explained ===================== A convenience codepoint can therefore be thought of as a code point used for a character which also has a composed form. The way Unicode works is that sometimes you can encode a character in one byte, sometimes not. Sometimes you can encode it in two bytes, sometimes not. You can therefore have a long stream of ASCII which is single-byte Unicode. If there is an occasional Cyrillic or Greek character in the stream, it would be represented either by a compatibility character or by a multi-byte combination. Using compatibility characters can prevent proper sorting and equivalence testing. Using "pure" Unicode, ie. "normal encodings", can cause compatibility and round-tripping probelms. Although avoiding them can *also* cause compatibility issues and round-tripping problems. Currently my thinking is: a Utf8String class an Ordered collection, with 1 byte characters as the modal element, but short arrays of wider strings where necessary a Utf16String class an Ordered collection, with 2 byte characters as the modal element, but short arrays of wider strings beginning with a 2-byte endianness indicator. Utf8Strings sometimes need to be sortable, and sometimes need to be compatible. So my thinking is that Utf8String will contain convenience codepoints, for round-tripping. And where there are multiple convenience codepoints for a character, that it standardises on one. And that there is a Utf8SortableString which uses *only* normal characters. We then need methods to convert between the two. aUtf8String asUtf8SortableString and aUtf8SortableString asUtf8String Sort orders are culture and context dependent - Sweden and Germany have different sort orders for the same diacritic-ed characters. Some countries have one order in general usage, and another for specific usages, such as phone directories (e.g. UK and France) Similarly for Utf16 : Utf16String and Utf16SortableString and conversion methods A list of sorted words would be a SortedCollection, and there could be pre-prepared sortBlocks for them, e.g. frPhoneBookOrder, deOrder, seOrder, ukOrder, etc along the lines of aListOfWords := SortedCollection sortBlock: deOrder If a word is either a Utf8SortableString, or a well-formed Utf8String, then we can perform equivalence testing on them trivially. To make sure a Utf8String is well formed, we would need to have a way of cleaning up any convenience codepoints which were valid, but which were for a character which has multiple equally-valid alternative convenience codepoints, and for which the string currently had the "wrong" convenience codepoint. (i.e for any character with valid alternative convenience codepoints, we would choose one to be in the well-formed Utf8String, and we would need a method for cleaning the alternative convenience codepoints out of the string, and replacing them with the chosen approved convenience codepoint. aUtf8String cleanUtf8String With WideString, a lot of the issues disappear - except round-tripping(although I'm sure I have seen something recently about 4-byte strings that also have an additional bit. Which would make some Unicode characters 5-bytes long.) (I'm starting to zone out now - if I've overlooked anything - obvious, subtle, or somewhere in between, please let me know) Cheers, Euan From robert.w.withers at gmail.com Fri Dec 4 13:31:07 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Fri Dec 4 13:31:11 2015 Subject: [squeak-dev] SqueakElib project Message-ID: <5661959B.3060804@gmail.com> Good morning, Here is a brief writeup of the SqueakElib project and its direction. https://github.com/SqueakCryptographySquad/Cryptography-Common/wiki/SqueakElib. All new squad members welcome. Regards, Robert From leves at caesar.elte.hu Fri Dec 4 14:46:59 2015 From: leves at caesar.elte.hu (Levente Uzonyi) Date: Fri Dec 4 14:47:06 2015 Subject: [squeak-dev] Unicode Support In-Reply-To: References: Message-ID: Why would you want to have strings with UTF-8 or UTF-16 encoding in the image? What's wrong with the current UTF-32 representation? Levente On Fri, 4 Dec 2015, EuanM wrote: > I'm currently groping my way to seeing how feature-complete our > Unicode support is. I am doing this to establish what still needs to > be done to provide full Unicode support. > > This seems to me to be an area where it would be best to write it > once, and then have the same codebase incorporated into the Smalltalks > that most share a common ancestry. > > I am keen to get: equality-testing for strings; sortability for > strings which have ligatures and diacritic characters; and correct > round-tripping of data. > > Call to action: > ========== > > If you have comments on these proposals - such as "but we already have > that facility" or "the reason we do not have these facilities is > because they are dog-slow" - please let me know them. > > If you would like to help out, please let me know. > > If you have Unicode experience and expertise, and would like to be, or > would be willing to be, in the 'council of experts' for this project, > please let me know. > > If you have comments or ideas on anything mentioned in this email > > In the first instance, the initiative's website will be: > http://smalltalk.uk.to/unicode.html > > I have created a SqueakSource.com project called UnicodeSupport > > I want to avoid re-inventing any facilities which already exist. > Except where they prevent us reaching the goals of: > - sortable UTF8 strings > - sortable UTF16 strings > - equivalence testing of 2 UTF8 strings > - equivalence testing of 2 UTF16 strings > - round-tripping UTF8 strings through Smalltalk > - roundtripping UTF16 strings through Smalltalk. > As I understand it, we have limited Unicode support atm. > > Current state of play > =============== > ByteString gets converted to WideString when need is automagically detected. > > Is there anything else that currently exists? > > Definition of Terms > ============== > A quick definition of terms before I go any further: > > Standard terms from the Unicode standard > =============================== > a compatibility character : an additional encoding of a *normal* > character, for compatibility and round-trip conversion purposes. For > instance, a 1-byte encoding of a Latin character with a diacritic. > > Made-up terms > ============ > a convenience codepoint : a single codepoint which represents an item > that is also encoded as a string of codepoints. > > (I tend to use the terms compatibility character and compatibility > codepoint interchangably. The standard only refers to them as > compatibility characters. However, the standard is determined to > emphasise that characters are abstract and that codepoints are > concrete. So I think it is often more useful and productive to think > of compatibility or convenience codepoints). > > a composed character : a character made up of several codepoints > > Unicode encoding explained > ===================== > A convenience codepoint can therefore be thought of as a code point > used for a character which also has a composed form. > > The way Unicode works is that sometimes you can encode a character in > one byte, sometimes not. Sometimes you can encode it in two bytes, > sometimes not. > > You can therefore have a long stream of ASCII which is single-byte > Unicode. If there is an occasional Cyrillic or Greek character in the > stream, it would be represented either by a compatibility character or > by a multi-byte combination. > > Using compatibility characters can prevent proper sorting and > equivalence testing. > > Using "pure" Unicode, ie. "normal encodings", can cause compatibility > and round-tripping probelms. Although avoiding them can *also* cause > compatibility issues and round-tripping problems. > > Currently my thinking is: > > a Utf8String class > an Ordered collection, with 1 byte characters as the modal element, > but short arrays of wider strings where necessary > a Utf16String class > an Ordered collection, with 2 byte characters as the modal element, > but short arrays of wider strings > beginning with a 2-byte endianness indicator. > > Utf8Strings sometimes need to be sortable, and sometimes need to be compatible. > > So my thinking is that Utf8String will contain convenience codepoints, > for round-tripping. And where there are multiple convenience > codepoints for a character, that it standardises on one. > > And that there is a Utf8SortableString which uses *only* normal characters. > > We then need methods to convert between the two. > > aUtf8String asUtf8SortableString > > and > > aUtf8SortableString asUtf8String > > > Sort orders are culture and context dependent - Sweden and Germany > have different sort orders for the same diacritic-ed characters. Some > countries have one order in general usage, and another for specific > usages, such as phone directories (e.g. UK and France) > > Similarly for Utf16 : Utf16String and Utf16SortableString and > conversion methods > > A list of sorted words would be a SortedCollection, and there could be > pre-prepared sortBlocks for them, e.g. frPhoneBookOrder, deOrder, > seOrder, ukOrder, etc > > along the lines of > aListOfWords := SortedCollection sortBlock: deOrder > > If a word is either a Utf8SortableString, or a well-formed Utf8String, > then we can perform equivalence testing on them trivially. > > To make sure a Utf8String is well formed, we would need to have a way > of cleaning up any convenience codepoints which were valid, but which > were for a character which has multiple equally-valid alternative > convenience codepoints, and for which the string currently had the > "wrong" convenience codepoint. (i.e for any character with valid > alternative convenience codepoints, we would choose one to be in the > well-formed Utf8String, and we would need a method for cleaning the > alternative convenience codepoints out of the string, and replacing > them with the chosen approved convenience codepoint. > > aUtf8String cleanUtf8String > > With WideString, a lot of the issues disappear - except > round-tripping(although I'm sure I have seen something recently about > 4-byte strings that also have an additional bit. Which would make > some Unicode characters 5-bytes long.) > > > (I'm starting to zone out now - if I've overlooked anything - obvious, > subtle, or somewhere in between, please let me know) > > Cheers, > Euan > > From bert at freudenbergs.de Fri Dec 4 15:03:51 2015 From: bert at freudenbergs.de (Bert Freudenberg) Date: Fri Dec 4 15:03:55 2015 Subject: [squeak-dev] Re: mouse enter/move/leave event confusion In-Reply-To: References: <4973AB59-FF96-41B1-AE15-904AAE8752D9@rowledge.org> <1449051662880-4864703.post@n4.nabble.com> <655B52F2-8736-476F-9B62-4A279AF46ECE@rowledge.org> <7D3C4F24-F59F-4C93-BE45-A454D91EB853@rowledge.org> <1449142410592-4864917.post@n4.nabble.com> Message-ID: <3E57CAF7-55F0-448E-B781-DF133A606B53@freudenbergs.de> > On 04.12.2015, at 11:33, Stephan Eggermont wrote: > > On 04-12-15 01:54, tim Rowledge wrote: >> Hi, >> so now I know why I can get so many spurious enter/leave events. It?s really embarrassingly dumb? >> >> When you are dragging a morph around and have the morph positioned at the event cursor point - ie not centred under it as you eventually want - then moving even moderately quickly can result in the cursor point being over the dragged morph. Which is not the canvas morph. So you?ve left the canvas morph; and you get a mouse leave event. D?oh! Then the fake-cursor-morph gets deleted, you are now over the canvas morph so you get a mouse enter event, and so on. > > Hmm. conceptually I would expect to get no spurious enter/leaves when the morph you're dragging around is either a submorph of the canvas, or grabbed by the hand. This. A grabbed morph is not in the world so should not generate events. - Bert - -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4115 bytes Desc: not available Url : http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151204/7f5ee3c7/smime.bin From bert at freudenbergs.de Fri Dec 4 15:11:55 2015 From: bert at freudenbergs.de (Bert Freudenberg) Date: Fri Dec 4 15:12:00 2015 Subject: [squeak-dev] Re: [Pharo-users] evolutions of squeakelib & crypto (Reed Solomon) In-Reply-To: <566175EA.2090107@gmail.com> References: <565DCE53.90501@gmail.com> <565DD073.1060409@gmail.com> <565FF8ED.1030606@free.fr> <5660BCFA.4050107@gmail.com> <56616E9C.6040201@gmail.com> <56616F27.4070002@gmail.com> <566174E9.9020001@gmail.com> <566175EA.2090107@gmail.com> Message-ID: <814F9CD2-9631-4674-A986-FD0C0C84E2AB@freudenbergs.de> Skipped content of type multipart/alternative-------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4115 bytes Desc: not available Url : http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151204/d78f4d2b/smime.bin From bert at freudenbergs.de Fri Dec 4 15:12:45 2015 From: bert at freudenbergs.de (Bert Freudenberg) Date: Fri Dec 4 15:12:49 2015 Subject: [squeak-dev] Re: mouse enter/move/leave event confusion In-Reply-To: <3E57CAF7-55F0-448E-B781-DF133A606B53@freudenbergs.de> References: <4973AB59-FF96-41B1-AE15-904AAE8752D9@rowledge.org> <1449051662880-4864703.post@n4.nabble.com> <655B52F2-8736-476F-9B62-4A279AF46ECE@rowledge.org> <7D3C4F24-F59F-4C93-BE45-A454D91EB853@rowledge.org> <1449142410592-4864917.post@n4.nabble.com> <3E57CAF7-55F0-448E-B781-DF133A606B53@freudenbergs.de> Message-ID: <089B43C4-C681-44F4-8A4B-54C98CB20B9A@freudenbergs.de> > On 04.12.2015, at 16:03, Bert Freudenberg wrote: > > >> On 04.12.2015, at 11:33, Stephan Eggermont wrote: >> >> On 04-12-15 01:54, tim Rowledge wrote: >>> Hi, >>> so now I know why I can get so many spurious enter/leave events. It?s really embarrassingly dumb? >>> >>> When you are dragging a morph around and have the morph positioned at the event cursor point - ie not centred under it as you eventually want - then moving even moderately quickly can result in the cursor point being over the dragged morph. Which is not the canvas morph. So you?ve left the canvas morph; and you get a mouse leave event. D?oh! Then the fake-cursor-morph gets deleted, you are now over the canvas morph so you get a mouse enter event, and so on. >> >> Hmm. conceptually I would expect to get no spurious enter/leaves when the morph you're dragging around is either a submorph of the canvas, or grabbed by the hand. > > This. A grabbed morph is not in the world so should not generate events. > > - Bert - Oh, and I thought you were going to use the temp cursor? Why wouldn?t that work? - Bert - -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4115 bytes Desc: not available Url : http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151204/921967e6/smime.bin From leves at caesar.elte.hu Fri Dec 4 15:19:33 2015 From: leves at caesar.elte.hu (Levente Uzonyi) Date: Fri Dec 4 15:19:39 2015 Subject: [squeak-dev] Re: [Pharo-users] evolutions of squeakelib & crypto (Reed Solomon) In-Reply-To: <566175EA.2090107@gmail.com> References: <565DCE53.90501@gmail.com> <565DD073.1060409@gmail.com> <565FF8ED.1030606@free.fr> <5660BCFA.4050107@gmail.com> <56616E9C.6040201@gmail.com> <56616F27.4070002@gmail.com> <566174E9.9020001@gmail.com> <566175EA.2090107@gmail.com> Message-ID: On Fri, 4 Dec 2015, Robert Withers wrote: > I just realized that the squeak version uses ReferenceStream while the Pharo version uses Fuel, so the binary serializations are different and they won't speak to each other. Any chance that Fuel is ported to Squeak? Fuel should work in Squeak. Probably not in 5.0 due to the changes introduced by Spur, but you should ask the devs about those. Levente > > Regards, > Robert > > On 12/04/2015 06:11 AM, Robert Withers wrote: > I am unable to import these files into SqueakSource, so it may be best done from inside Pharo with Monticello. Here are working Crypto and SqueakElib in Pharo, prior to SecureSession refactoring and Reed Solomon. I include the correct version of > LayeredProtocol. > > Regards, > Robert > > > On 12/04/2015 05:47 AM, Robert Withers wrote: > Best Regards > > http://www.squeaksource.com/Cryptography.html > http://www.squeaksource.com/squeakelib.html > > > On 12/04/2015 05:44 AM, Robert Withers wrote: > After my password reset on squeaksource, I committed to both Cryptography and SqueakElib, project links below. > > In the case of Cryptography, I had a version ported to Pharo ... I will organize Pharo ports of both after Reed Solomon is stable, and announce them to the Pharo list. This way both environments can be supported through this one repository. > > Robert > > On 12/04/2015 04:35 AM, Stephan Eggermont wrote: > On 03-12-15 23:06, Robert Withers wrote: > Are any of these used by both squeak and Pharo? That would be the right > move I think. I will ask about getting my password reset for > squeaksource, since that is where the old code resides. > > > All of them. Mostly timing of project start/high activity and who are > maintaining it decided on platforms. There was a time when squeaksource was not so stable and then many projects migrated, and I currently hear least about stability issues from ss3, but the load on smalltalkhub is much higher, I > assume, as that is used for the pharo ci. > > In the not so far future Pharo is likely to move to a git based infrastructure, using libgit2. Early adopters are already using it. > I haven't heard the squeak ideas about that. > > Stephan > > > > > > > > > From timfelgentreff at gmail.com Fri Dec 4 15:28:16 2015 From: timfelgentreff at gmail.com (timfelgentreff) Date: Fri Dec 4 15:44:58 2015 Subject: [squeak-dev] Call for Papers: Constrained and Reactive Objects Workshop at Modularity 2016 Message-ID: <1449242896847-4865240.post@n4.nabble.com> CROW 2016 - Constrained and Reactive Objects Workshop The Constrained and Reactive Objects Workshop (CROW) invites papers that present research results related to using constraints and reactive programming in combination or integrated with imperative and object-oriented systems. This research area has recently received increasing interest again. Object-constraint, constraint-reactive, constraint-imperative systems and others were proposed in response to challenges posed by, for example, the requirements for applications to work across different devices and form-factors, to provide best-effort computing in the face of unreliable devices and networks, and to more fully utilise HPC technology. However, many questions remain. For example, the interactions among constraints, reactive or event-based programming, and mainstream imperative and object-oriented languages are still not well-understood, and different implementation techniques to combine these paradigms are still being explored. The lack of large applications written with combinations of these paradigms means that there is little experience with patterns and tools for such applications, and modularity mechanisms required for such large applications are in their infancy or completely absent. This workshop is intended to gather researchers working on using constraints or reactive programming in combination with mainstream imperative and object-oriented languages. The goal of this workshop is to present technical research results related to such systems, discuss applications, tools, and patterns used in these systems, and to define better the directions into which this field may evolve by providing overviews of existing work. Call for Papers Even though constraint-based and reactive programming are not new concepts, their combination and interaction with more mainstream imperative and object-oriented languages are receiving more and more attention. This workshop is an attempt to gather researchers working on models, languages, and implementations relating these paradigms. We welcome all submissions on language design, implementation, runtime systems, program analysis, software metrics, patterns, modularity principles, case-studies and benchmarks of systems that combine constraint-based or reactive programming with imperative paradigms. Of particular interest are topics from the following areas: ?Study of the interactions of constraints and reactive programming with imperative language features such as object-oriented programming, mutable state, encapsulation, modules, or concurrency ?Applications and case-studies that showcase examples in which the combination of constraints and reactive programming facilitates systems development ?Empirical studies that motivate further research in the field ?Patterns, modularity mechanisms, and best-practices for larger applications ?IDEs and Tools ?Language runtimes, virtual machine support, or compilers ?Related fields that constraints and reactive programming can help facilitate, such as self-healing systems, best-effort computing, acceptability-oriented computing, or end-user programming Authors are invited to submit short position papers or long technical papers. using the ACM sigplanconf style at 9pt font size (see http://www.acm.org/publications/article-templates/proceedings-template.html). Short papers are imited to a maximum of 4 pages, excluding references. Long papers are limited to a maximum of 10 pages, including references. Submissions should not have been previously published nor be under review at other events. All submitted papers will be peer reviewed. Accepted papers will be published in the ACM Digital Library (unless the authors wish otherwise). Papers should be submitted via EasyChair: https://easychair.org/conferences/?conf=crow16. Workshop Format The format of the workshop will be that of a mini-conference. Participants can present their work in slots of 30 minutes including Q&A. Because it is often hard to understand the semantics of novel combinations of multiple paradigms as in this workshop just by looking at the code, we also encourage authors to present live demos. The goal of the workshop is to come to a better understanding of the the field. We hope to use the late afternoon session to draw conclusions from the presented work, and to identify fundamental paradigmatic problems that still need to be resolved to reconcile these different technologies and paradigms. Important Dates ?Submission deadline: January 15, 2016 ?Notification: February 6, 2016 ?Camera ready: February 13, 2016 ?Workshop: March 14-15, 2016 -- View this message in context: http://forum.world.st/Call-for-Papers-Constrained-and-Reactive-Objects-Workshop-at-Modularity-2016-tp4865240.html Sent from the Squeak - Dev mailing list archive at Nabble.com. From Marcel.Taeumel at hpi.de Fri Dec 4 15:28:33 2015 From: Marcel.Taeumel at hpi.de (marcel.taeumel) Date: Fri Dec 4 15:45:15 2015 Subject: [squeak-dev] Re: mouse enter/move/leave event confusion In-Reply-To: <3E57CAF7-55F0-448E-B781-DF133A606B53@freudenbergs.de> References: <4973AB59-FF96-41B1-AE15-904AAE8752D9@rowledge.org> <1449051662880-4864703.post@n4.nabble.com> <655B52F2-8736-476F-9B62-4A279AF46ECE@rowledge.org> <7D3C4F24-F59F-4C93-BE45-A454D91EB853@rowledge.org> <1449142410592-4864917.post@n4.nabble.com> <3E57CAF7-55F0-448E-B781-DF133A606B53@freudenbergs.de> Message-ID: <1449242913113-4865241.post@n4.nabble.com> Why grabbing if you just need an indication (for the gridding) where the next stroke will go on a mouse down/click? Best, Marcel -- View this message in context: http://forum.world.st/mouse-enter-move-leave-event-confusion-tp4864651p4865241.html Sent from the Squeak - Dev mailing list archive at Nabble.com. From robert.w.withers at gmail.com Fri Dec 4 15:49:27 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Fri Dec 4 15:49:32 2015 Subject: [squeak-dev] Re: [Pharo-users] evolutions of squeakelib & crypto (Reed Solomon) In-Reply-To: <814F9CD2-9631-4674-A986-FD0C0C84E2AB@freudenbergs.de> References: <565DCE53.90501@gmail.com> <565DD073.1060409@gmail.com> <565FF8ED.1030606@free.fr> <5660BCFA.4050107@gmail.com> <56616E9C.6040201@gmail.com> <56616F27.4070002@gmail.com> <566174E9.9020001@gmail.com> <566175EA.2090107@gmail.com> <814F9CD2-9631-4674-A986-FD0C0C84E2AB@freudenbergs.de> Message-ID: <5661B607.80608@gmail.com> On 12/04/2015 10:11 AM, Bert Freudenberg wrote: > >> On 04.12.2015, at 12:15, Robert Withers > > wrote: >> >> I just realized that the squeak version uses ReferenceStream while >> the Pharo version uses Fuel, so the binary serializations are >> different and they won't speak to each other. > > What is this used for? SqueakElib (name change forthcoming!... :) ) uses binary serialization to communicate over the wire between vats (an execution container). As this framework is to provide the substrate for distributed secure objects, there is a critical need to do substitutions, such that PassByProxy objects are shareable and resolutions can home back to the right pending context (a distributed continuation). This requirement for substitutions require a robust serialization stream framework, of which ReferenceStream in Squeak and Fuel in Pharo meet this need. >> Any chance that Fuel is ported to Squeak? > > That would be nice, yes. Yes, that would indeed. I hope for it to magically appear! Robert > > - Bert - > >> Robert >> >> On 12/04/2015 06:11 AM, Robert Withers wrote: >>> I am unable to import these files into SqueakSource, so it may be >>> best done from inside Pharo with Monticello. Here are working Crypto >>> and SqueakElib in Pharo, prior to SecureSession refactoring and Reed >>> Solomon. I include the correct version of LayeredProtocol. >>> >>> Regards, >>> Robert >>> >>> >>> On 12/04/2015 05:47 AM, Robert Withers wrote: >>>> Best Regards >>>> >>>> http://www.squeaksource.com/Cryptography.html >>>> http://www.squeaksource.com/squeakelib.html >>>> >>>> >>>> On 12/04/2015 05:44 AM, Robert Withers wrote: >>>>> After my password reset on squeaksource, I committed to both >>>>> Cryptography and SqueakElib, project links below. >>>>> >>>>> In the case of Cryptography, I had a version ported to Pharo ... I >>>>> will organize Pharo ports of both after Reed Solomon is stable, >>>>> and announce them to the Pharo list. This way both environments >>>>> can be supported through this one repository. >>>>> >>>>> Robert >>>>> >>>>> On 12/04/2015 04:35 AM, Stephan Eggermont wrote: >>>>>> On 03-12-15 23:06, Robert Withers wrote: >>>>>>> Are any of these used by both squeak and Pharo? That would be >>>>>>> the right >>>>>>> move I think. I will ask about getting my password reset for >>>>>>> squeaksource, since that is where the old code resides. >>>>>> >>>>>> All of them. Mostly timing of project start/high activity and who >>>>>> are >>>>>> maintaining it decided on platforms. There was a time when >>>>>> squeaksource was not so stable and then many projects migrated, >>>>>> and I currently hear least about stability issues from ss3, but >>>>>> the load on smalltalkhub is much higher, I assume, as that is >>>>>> used for the pharo ci. >>>>>> >>>>>> In the not so far future Pharo is likely to move to a git based >>>>>> infrastructure, using libgit2. Early adopters are already using it. >>>>>> I haven't heard the squeak ideas about that. >>>>>> >>>>>> Stephan >>>>>> >>>>>> >>>>>> >>>>> >>>> >>> >> >> > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151204/2a72c1e7/attachment.htm From robert.w.withers at gmail.com Fri Dec 4 15:57:37 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Fri Dec 4 15:57:41 2015 Subject: [squeak-dev] Re: [Pharo-users] evolutions of squeakelib & crypto (Reed Solomon) In-Reply-To: <224A3FD6-A8DA-45E3-8EDE-AA52133940CC@stfx.eu> References: <565DCE53.90501@gmail.com> <565DD073.1060409@gmail.com> <565FF8ED.1030606@free.fr> <5660BCFA.4050107@gmail.com> <56616E9C.6040201@gmail.com> <56616F27.4070002@gmail.com> <566174E9.9020001@gmail.com> <566175EA.2090107@gmail.com> <60B913A3-565C-4924-9F82-4E0E96553B72@gmail.com> <5661A6A5.9030902@gmail.com> <224A3FD6-A8DA-45E3-8EDE-AA52133940CC@stfx.eu> Message-ID: <5661B7F1.1080701@gmail.com> Now I think you are right on with your observation. Additionally, the number of dialects could increase further with Fuel serialization, just port SecureSession and bits. Alright, I came up with a name and it may border on the egregious ... presenting ... "Maelstrom" A new distributed metric will be established in Maelstrom to measure intra- & inter-vat activity: "Sanguinity". Robert PS. thanks for pointing out Fuel support for older squeaks. On 12/04/2015 10:25 AM, Sven Van Caekenberghe wrote: > Another suggestion: if you intend to make something cross dialect and hope to get traction, I would not use a project name which contains one dialect, but something more general like Cryptography, or some cool project name, like Seaside, Fuel, ... > >> On 04 Dec 2015, at 15:43, Robert Withers wrote: >> >> I use squeak 5.0 and would want the Fuel support to customize for wire serializations and substitutions, such that Squeak and Pharo could talk to each other, and all other Fuel environments. The Fuel changes I made are in the Pharo port of SqueakElib in the SqueakElib-CapTP-Serialization category and consist of a Decoder, Materializer and a Materialization. >> >> Robert >> >> On 12/04/2015 09:35 AM, Esteban Lorenzano wrote: >>> which squeak version? >>> of what? >>> >>>> On 04 Dec 2015, at 12:15, Robert Withers wrote: >>>> >>>> I just realized that the squeak version uses ReferenceStream while the Pharo version uses Fuel, so the binary serializations are different and they won't speak to each other. Any chance that Fuel is ported to Squeak? >>>> >>>> Regards, >>>> Robert >>>> >>>> On 12/04/2015 06:11 AM, Robert Withers wrote: >>>>> I am unable to import these files into SqueakSource, so it may be best done from inside Pharo with Monticello. Here are working Crypto and SqueakElib in Pharo, prior to SecureSession refactoring and Reed Solomon. I include the correct version of LayeredProtocol. >>>>> >>>>> Regards, >>>>> Robert >>>>> >>>>> >>>>> On 12/04/2015 05:47 AM, Robert Withers wrote: >>>>>> Best Regards >>>>>> >>>>>> http://www.squeaksource.com/Cryptography.html >>>>>> http://www.squeaksource.com/squeakelib.html >>>>>> >>>>>> >>>>>> On 12/04/2015 05:44 AM, Robert Withers wrote: >>>>>>> After my password reset on squeaksource, I committed to both Cryptography and SqueakElib, project links below. >>>>>>> >>>>>>> In the case of Cryptography, I had a version ported to Pharo ... I will organize Pharo ports of both after Reed Solomon is stable, and announce them to the Pharo list. This way both environments can be supported through this one repository. >>>>>>> >>>>>>> Robert >>>>>>> >>>>>>> On 12/04/2015 04:35 AM, Stephan Eggermont wrote: >>>>>>>> On 03-12-15 23:06, Robert Withers wrote: >>>>>>>>> Are any of these used by both squeak and Pharo? That would be the right >>>>>>>>> move I think. I will ask about getting my password reset for >>>>>>>>> squeaksource, since that is where the old code resides. >>>>>>>> All of them. Mostly timing of project start/high activity and who are >>>>>>>> maintaining it decided on platforms. There was a time when squeaksource was not so stable and then many projects migrated, and I currently hear least about stability issues from ss3, but the load on smalltalkhub is much higher, I assume, as that is used for the pharo ci. >>>>>>>> >>>>>>>> In the not so far future Pharo is likely to move to a git based infrastructure, using libgit2. Early adopters are already using it. >>>>>>>> I haven't heard the squeak ideas about that. >>>>>>>> >>>>>>>> Stephan >>>>>>>> >>>>>>>> >>>>>>>> > From marianopeck at gmail.com Fri Dec 4 16:41:19 2015 From: marianopeck at gmail.com (Mariano Martinez Peck) Date: Fri Dec 4 16:41:22 2015 Subject: [squeak-dev] Re: [Pharo-users] evolutions of squeakelib & crypto (Reed Solomon) In-Reply-To: <5661B7F1.1080701@gmail.com> References: <565DCE53.90501@gmail.com> <565DD073.1060409@gmail.com> <565FF8ED.1030606@free.fr> <5660BCFA.4050107@gmail.com> <56616E9C.6040201@gmail.com> <56616F27.4070002@gmail.com> <566174E9.9020001@gmail.com> <566175EA.2090107@gmail.com> <60B913A3-565C-4924-9F82-4E0E96553B72@gmail.com> <5661A6A5.9030902@gmail.com> <224A3FD6-A8DA-45E3-8EDE-AA52133940CC@stfx.eu> <5661B7F1.1080701@gmail.com> Message-ID: I think that either Max or Martin did the Spur needed changes for Fuel. So likely it should be easy to make it work in latest Squeak. On Fri, Dec 4, 2015 at 12:57 PM, Robert Withers wrote: > Now I think you are right on with your observation. Additionally, the > number of dialects could increase further with Fuel serialization, just > port SecureSession and bits. > > Alright, I came up with a name and it may border on the egregious ... > presenting ... > > "Maelstrom" > > A new distributed metric will be established in Maelstrom to measure > intra- & inter-vat activity: "Sanguinity". > > Robert > > PS. thanks for pointing out Fuel support for older squeaks. > > > On 12/04/2015 10:25 AM, Sven Van Caekenberghe wrote: > >> Another suggestion: if you intend to make something cross dialect and >> hope to get traction, I would not use a project name which contains one >> dialect, but something more general like Cryptography, or some cool project >> name, like Seaside, Fuel, ... >> >> On 04 Dec 2015, at 15:43, Robert Withers >>> wrote: >>> >>> I use squeak 5.0 and would want the Fuel support to customize for wire >>> serializations and substitutions, such that Squeak and Pharo could talk to >>> each other, and all other Fuel environments. The Fuel changes I made are in >>> the Pharo port of SqueakElib in the SqueakElib-CapTP-Serialization category >>> and consist of a Decoder, Materializer and a Materialization. >>> >>> Robert >>> >>> On 12/04/2015 09:35 AM, Esteban Lorenzano wrote: >>> >>>> which squeak version? >>>> of what? >>>> >>>> On 04 Dec 2015, at 12:15, Robert Withers >>>>> wrote: >>>>> >>>>> I just realized that the squeak version uses ReferenceStream while the >>>>> Pharo version uses Fuel, so the binary serializations are different and >>>>> they won't speak to each other. Any chance that Fuel is ported to Squeak? >>>>> >>>>> Regards, >>>>> Robert >>>>> >>>>> On 12/04/2015 06:11 AM, Robert Withers wrote: >>>>> >>>>>> I am unable to import these files into SqueakSource, so it may be >>>>>> best done from inside Pharo with Monticello. Here are working Crypto and >>>>>> SqueakElib in Pharo, prior to SecureSession refactoring and Reed Solomon. I >>>>>> include the correct version of LayeredProtocol. >>>>>> >>>>>> Regards, >>>>>> Robert >>>>>> >>>>>> >>>>>> On 12/04/2015 05:47 AM, Robert Withers wrote: >>>>>> >>>>>>> Best Regards >>>>>>> >>>>>>> http://www.squeaksource.com/Cryptography.html >>>>>>> http://www.squeaksource.com/squeakelib.html >>>>>>> >>>>>>> >>>>>>> On 12/04/2015 05:44 AM, Robert Withers wrote: >>>>>>> >>>>>>>> After my password reset on squeaksource, I committed to both >>>>>>>> Cryptography and SqueakElib, project links below. >>>>>>>> >>>>>>>> In the case of Cryptography, I had a version ported to Pharo ... I >>>>>>>> will organize Pharo ports of both after Reed Solomon is stable, and >>>>>>>> announce them to the Pharo list. This way both environments can be >>>>>>>> supported through this one repository. >>>>>>>> >>>>>>>> Robert >>>>>>>> >>>>>>>> On 12/04/2015 04:35 AM, Stephan Eggermont wrote: >>>>>>>> >>>>>>>>> On 03-12-15 23:06, Robert Withers wrote: >>>>>>>>> >>>>>>>>>> Are any of these used by both squeak and Pharo? That would be the >>>>>>>>>> right >>>>>>>>>> move I think. I will ask about getting my password reset for >>>>>>>>>> squeaksource, since that is where the old code resides. >>>>>>>>>> >>>>>>>>> All of them. Mostly timing of project start/high activity and who >>>>>>>>> are >>>>>>>>> maintaining it decided on platforms. There was a time when >>>>>>>>> squeaksource was not so stable and then many projects migrated, and I >>>>>>>>> currently hear least about stability issues from ss3, but the load on >>>>>>>>> smalltalkhub is much higher, I assume, as that is used for the pharo ci. >>>>>>>>> >>>>>>>>> In the not so far future Pharo is likely to move to a git based >>>>>>>>> infrastructure, using libgit2. Early adopters are already using it. >>>>>>>>> I haven't heard the squeak ideas about that. >>>>>>>>> >>>>>>>>> Stephan >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >> > > -- Mariano http://marianopeck.wordpress.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151204/d593f9b0/attachment-0001.htm From robert.w.withers at gmail.com Fri Dec 4 17:07:18 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Fri Dec 4 17:07:24 2015 Subject: [squeak-dev] Re: [Pharo-users] evolutions of squeakelib & crypto (Reed Solomon) In-Reply-To: References: <565DCE53.90501@gmail.com> <565DD073.1060409@gmail.com> <565FF8ED.1030606@free.fr> <5660BCFA.4050107@gmail.com> <56616E9C.6040201@gmail.com> <56616F27.4070002@gmail.com> <566174E9.9020001@gmail.com> <566175EA.2090107@gmail.com> <60B913A3-565C-4924-9F82-4E0E96553B72@gmail.com> <5661A6A5.9030902@gmail.com> <224A3FD6-A8DA-45E3-8EDE-AA52133940CC@stfx.eu> <5661B7F1.1080701@gmail.com> Message-ID: <5661C846.4010208@gmail.com> I had to reconceptualize how substitutions worked in Pharo, while learning Fuel, as it has been awhile since I played with that serialization. I recently came back to smalltalk after years. My real goal is mobile code so it can run where it has most affinity, even though that's not yet known. Enter sanguinity. This is great news and I am excited by the potential to see Squeak and Pharo talking to each other. cheers, Robert On 12/04/2015 11:41 AM, Mariano Martinez Peck wrote: > I think that either Max or Martin did the Spur needed changes for Fuel. > So likely it should be easy to make it work in latest Squeak. > > On Fri, Dec 4, 2015 at 12:57 PM, Robert Withers > > wrote: > > Now I think you are right on with your observation. Additionally, > the number of dialects could increase further with Fuel > serialization, just port SecureSession and bits. > > Alright, I came up with a name and it may border on the egregious > ... presenting ... > > "Maelstrom" > > A new distributed metric will be established in Maelstrom to > measure intra- & inter-vat activity: "Sanguinity". > > Robert > > PS. thanks for pointing out Fuel support for older squeaks. > > > On 12/04/2015 10:25 AM, Sven Van Caekenberghe wrote: > > Another suggestion: if you intend to make something cross > dialect and hope to get traction, I would not use a project > name which contains one dialect, but something more general > like Cryptography, or some cool project name, like Seaside, > Fuel, ... > > On 04 Dec 2015, at 15:43, Robert Withers > > wrote: > > I use squeak 5.0 and would want the Fuel support to > customize for wire serializations and substitutions, such > that Squeak and Pharo could talk to each other, and all > other Fuel environments. The Fuel changes I made are in > the Pharo port of SqueakElib in the > SqueakElib-CapTP-Serialization category and consist of a > Decoder, Materializer and a Materialization. > > Robert > > On 12/04/2015 09:35 AM, Esteban Lorenzano wrote: > > which squeak version? > of what? > > On 04 Dec 2015, at 12:15, Robert Withers > > wrote: > > I just realized that the squeak version uses > ReferenceStream while the Pharo version uses Fuel, > so the binary serializations are different and > they won't speak to each other. Any chance that > Fuel is ported to Squeak? > > Regards, > Robert > > On 12/04/2015 06:11 AM, Robert Withers wrote: > > I am unable to import these files into > SqueakSource, so it may be best done from > inside Pharo with Monticello. Here are working > Crypto and SqueakElib in Pharo, prior to > SecureSession refactoring and Reed Solomon. I > include the correct version of LayeredProtocol. > > Regards, > Robert > > > On 12/04/2015 05:47 AM, Robert Withers wrote: > > Best Regards > > http://www.squeaksource.com/Cryptography.html > http://www.squeaksource.com/squeakelib.html > > > On 12/04/2015 05:44 AM, Robert Withers wrote: > > After my password reset on > squeaksource, I committed to both > Cryptography and SqueakElib, project > links below. > > In the case of Cryptography, I had a > version ported to Pharo ... I will > organize Pharo ports of both after > Reed Solomon is stable, and announce > them to the Pharo list. This way both > environments can be supported through > this one repository. > > Robert > > On 12/04/2015 04:35 AM, Stephan > Eggermont wrote: > > On 03-12-15 23:06, Robert Withers > wrote: > > Are any of these used by both > squeak and Pharo? That would > be the right > move I think. I will ask about > getting my password reset for > squeaksource, since that is > where the old code resides. > > All of them. Mostly timing of > project start/high activity and > who are > maintaining it decided on > platforms. There was a time when > squeaksource was not so stable and > then many projects migrated, and I > currently hear least about > stability issues from ss3, but the > load on smalltalkhub is much > higher, I assume, as that is used > for the pharo ci. > > In the not so far future Pharo is > likely to move to a git based > infrastructure, using libgit2. > Early adopters are already using it. > I haven't heard the squeak ideas > about that. > > Stephan > > > > > > > > > > -- > Mariano > http://marianopeck.wordpress.com > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151204/6071db6e/attachment.htm From robert.w.withers at gmail.com Fri Dec 4 17:29:45 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Fri Dec 4 17:29:56 2015 Subject: [squeak-dev] Re: [Pharo-users] evolutions of squeakelib & crypto (Reed Solomon) In-Reply-To: <5661B7F1.1080701@gmail.com> References: <565DCE53.90501@gmail.com> <565DD073.1060409@gmail.com> <565FF8ED.1030606@free.fr> <5660BCFA.4050107@gmail.com> <56616E9C.6040201@gmail.com> <56616F27.4070002@gmail.com> <566174E9.9020001@gmail.com> <566175EA.2090107@gmail.com> <60B913A3-565C-4924-9F82-4E0E96553B72@gmail.com> <5661A6A5.9030902@gmail.com> <224A3FD6-A8DA-45E3-8EDE-AA52133940CC@stfx.eu> <5661B7F1.1080701@gmail.com> Message-ID: <5661CD89.6090504@gmail.com> I've created a new SqueakSource project for this: http://www.squeaksource.com/maelstrom.html. All future improvements will go to this repository. Here's where wiki documentation can go: https://github.com/SqueakCryptographySquad/maelstrom/wiki Thank you for prompting this name change. Robert On 12/04/2015 10:57 AM, Robert Withers wrote: > Now I think you are right on with your observation. Additionally, the > number of dialects could increase further with Fuel serialization, > just port SecureSession and bits. > > Alright, I came up with a name and it may border on the egregious ... > presenting ... > > "Maelstrom" > > A new distributed metric will be established in Maelstrom to measure > intra- & inter-vat activity: "Sanguinity". > > Robert > > PS. thanks for pointing out Fuel support for older squeaks. > > On 12/04/2015 10:25 AM, Sven Van Caekenberghe wrote: >> Another suggestion: if you intend to make something cross dialect and >> hope to get traction, I would not use a project name which contains >> one dialect, but something more general like Cryptography, or some >> cool project name, like Seaside, Fuel, ... >> >>> On 04 Dec 2015, at 15:43, Robert Withers >>> wrote: >>> >>> I use squeak 5.0 and would want the Fuel support to customize for >>> wire serializations and substitutions, such that Squeak and Pharo >>> could talk to each other, and all other Fuel environments. The Fuel >>> changes I made are in the Pharo port of SqueakElib in the >>> SqueakElib-CapTP-Serialization category and consist of a Decoder, >>> Materializer and a Materialization. >>> >>> Robert >>> >>> On 12/04/2015 09:35 AM, Esteban Lorenzano wrote: >>>> which squeak version? >>>> of what? >>>> >>>>> On 04 Dec 2015, at 12:15, Robert Withers >>>>> wrote: >>>>> >>>>> I just realized that the squeak version uses ReferenceStream while >>>>> the Pharo version uses Fuel, so the binary serializations are >>>>> different and they won't speak to each other. Any chance that Fuel >>>>> is ported to Squeak? >>>>> >>>>> Regards, >>>>> Robert >>>>> >>>>> On 12/04/2015 06:11 AM, Robert Withers wrote: >>>>>> I am unable to import these files into SqueakSource, so it may be >>>>>> best done from inside Pharo with Monticello. Here are working >>>>>> Crypto and SqueakElib in Pharo, prior to SecureSession >>>>>> refactoring and Reed Solomon. I include the correct version of >>>>>> LayeredProtocol. >>>>>> >>>>>> Regards, >>>>>> Robert >>>>>> >>>>>> >>>>>> On 12/04/2015 05:47 AM, Robert Withers wrote: >>>>>>> Best Regards >>>>>>> >>>>>>> http://www.squeaksource.com/Cryptography.html >>>>>>> http://www.squeaksource.com/squeakelib.html >>>>>>> >>>>>>> >>>>>>> On 12/04/2015 05:44 AM, Robert Withers wrote: >>>>>>>> After my password reset on squeaksource, I committed to both >>>>>>>> Cryptography and SqueakElib, project links below. >>>>>>>> >>>>>>>> In the case of Cryptography, I had a version ported to Pharo >>>>>>>> ... I will organize Pharo ports of both after Reed Solomon is >>>>>>>> stable, and announce them to the Pharo list. This way both >>>>>>>> environments can be supported through this one repository. >>>>>>>> >>>>>>>> Robert >>>>>>>> >>>>>>>> On 12/04/2015 04:35 AM, Stephan Eggermont wrote: >>>>>>>>> On 03-12-15 23:06, Robert Withers wrote: >>>>>>>>>> Are any of these used by both squeak and Pharo? That would be >>>>>>>>>> the right >>>>>>>>>> move I think. I will ask about getting my password reset for >>>>>>>>>> squeaksource, since that is where the old code resides. >>>>>>>>> All of them. Mostly timing of project start/high activity and >>>>>>>>> who are >>>>>>>>> maintaining it decided on platforms. There was a time when >>>>>>>>> squeaksource was not so stable and then many projects >>>>>>>>> migrated, and I currently hear least about stability issues >>>>>>>>> from ss3, but the load on smalltalkhub is much higher, I >>>>>>>>> assume, as that is used for the pharo ci. >>>>>>>>> >>>>>>>>> In the not so far future Pharo is likely to move to a git >>>>>>>>> based infrastructure, using libgit2. Early adopters are >>>>>>>>> already using it. >>>>>>>>> I haven't heard the squeak ideas about that. >>>>>>>>> >>>>>>>>> Stephan >>>>>>>>> >>>>>>>>> >>>>>>>>> >> > From dale.henrichs at gemtalksystems.com Fri Dec 4 18:02:51 2015 From: dale.henrichs at gemtalksystems.com (Dale Henrichs) Date: Fri Dec 4 18:02:55 2015 Subject: [squeak-dev] Unicode Support In-Reply-To: References: Message-ID: <5661D54B.2090409@gemtalksystems.com> Euan, When you encode a Unicode*String into UTF8, it should be in the form of a ByteArray -- in GemStone we have a Utf8 class that is a subclass of ByteArray that stores the encoded bytes -- If you use a String to store a Utf8 encoded String, then you run the risk of getting confused as to whether or not the String is encoded ... a separate class eliminates this potential confusion... My $0.02... Dale On 12/04/2015 03:42 AM, EuanM wrote: > I'm currently groping my way to seeing how feature-complete our > Unicode support is. I am doing this to establish what still needs to > be done to provide full Unicode support. > > This seems to me to be an area where it would be best to write it > once, and then have the same codebase incorporated into the Smalltalks > that most share a common ancestry. > > I am keen to get: equality-testing for strings; sortability for > strings which have ligatures and diacritic characters; and correct > round-tripping of data. > > Call to action: > ========== > > If you have comments on these proposals - such as "but we already have > that facility" or "the reason we do not have these facilities is > because they are dog-slow" - please let me know them. > > If you would like to help out, please let me know. > > If you have Unicode experience and expertise, and would like to be, or > would be willing to be, in the 'council of experts' for this project, > please let me know. > > If you have comments or ideas on anything mentioned in this email > > In the first instance, the initiative's website will be: > http://smalltalk.uk.to/unicode.html > > I have created a SqueakSource.com project called UnicodeSupport > > I want to avoid re-inventing any facilities which already exist. > Except where they prevent us reaching the goals of: > - sortable UTF8 strings > - sortable UTF16 strings > - equivalence testing of 2 UTF8 strings > - equivalence testing of 2 UTF16 strings > - round-tripping UTF8 strings through Smalltalk > - roundtripping UTF16 strings through Smalltalk. > As I understand it, we have limited Unicode support atm. > > Current state of play > =============== > ByteString gets converted to WideString when need is automagically detected. > > Is there anything else that currently exists? > > Definition of Terms > ============== > A quick definition of terms before I go any further: > > Standard terms from the Unicode standard > =============================== > a compatibility character : an additional encoding of a *normal* > character, for compatibility and round-trip conversion purposes. For > instance, a 1-byte encoding of a Latin character with a diacritic. > > Made-up terms > ============ > a convenience codepoint : a single codepoint which represents an item > that is also encoded as a string of codepoints. > > (I tend to use the terms compatibility character and compatibility > codepoint interchangably. The standard only refers to them as > compatibility characters. However, the standard is determined to > emphasise that characters are abstract and that codepoints are > concrete. So I think it is often more useful and productive to think > of compatibility or convenience codepoints). > > a composed character : a character made up of several codepoints > > Unicode encoding explained > ===================== > A convenience codepoint can therefore be thought of as a code point > used for a character which also has a composed form. > > The way Unicode works is that sometimes you can encode a character in > one byte, sometimes not. Sometimes you can encode it in two bytes, > sometimes not. > > You can therefore have a long stream of ASCII which is single-byte > Unicode. If there is an occasional Cyrillic or Greek character in the > stream, it would be represented either by a compatibility character or > by a multi-byte combination. > > Using compatibility characters can prevent proper sorting and > equivalence testing. > > Using "pure" Unicode, ie. "normal encodings", can cause compatibility > and round-tripping probelms. Although avoiding them can *also* cause > compatibility issues and round-tripping problems. > > Currently my thinking is: > > a Utf8String class > an Ordered collection, with 1 byte characters as the modal element, > but short arrays of wider strings where necessary > a Utf16String class > an Ordered collection, with 2 byte characters as the modal element, > but short arrays of wider strings > beginning with a 2-byte endianness indicator. > > Utf8Strings sometimes need to be sortable, and sometimes need to be compatible. > > So my thinking is that Utf8String will contain convenience codepoints, > for round-tripping. And where there are multiple convenience > codepoints for a character, that it standardises on one. > > And that there is a Utf8SortableString which uses *only* normal characters. > > We then need methods to convert between the two. > > aUtf8String asUtf8SortableString > > and > > aUtf8SortableString asUtf8String > > > Sort orders are culture and context dependent - Sweden and Germany > have different sort orders for the same diacritic-ed characters. Some > countries have one order in general usage, and another for specific > usages, such as phone directories (e.g. UK and France) > > Similarly for Utf16 : Utf16String and Utf16SortableString and > conversion methods > > A list of sorted words would be a SortedCollection, and there could be > pre-prepared sortBlocks for them, e.g. frPhoneBookOrder, deOrder, > seOrder, ukOrder, etc > > along the lines of > aListOfWords := SortedCollection sortBlock: deOrder > > If a word is either a Utf8SortableString, or a well-formed Utf8String, > then we can perform equivalence testing on them trivially. > > To make sure a Utf8String is well formed, we would need to have a way > of cleaning up any convenience codepoints which were valid, but which > were for a character which has multiple equally-valid alternative > convenience codepoints, and for which the string currently had the > "wrong" convenience codepoint. (i.e for any character with valid > alternative convenience codepoints, we would choose one to be in the > well-formed Utf8String, and we would need a method for cleaning the > alternative convenience codepoints out of the string, and replacing > them with the chosen approved convenience codepoint. > > aUtf8String cleanUtf8String > > With WideString, a lot of the issues disappear - except > round-tripping(although I'm sure I have seen something recently about > 4-byte strings that also have an additional bit. Which would make > some Unicode characters 5-bytes long.) > > > (I'm starting to zone out now - if I've overlooked anything - obvious, > subtle, or somewhere in between, please let me know) > > Cheers, > Euan > From commits at source.squeak.org Fri Dec 4 22:55:01 2015 From: commits at source.squeak.org (commits@source.squeak.org) Date: Fri Dec 4 22:55:03 2015 Subject: [squeak-dev] Daily Commit Log Message-ID: <20151204225501.26969.qmail@box4.squeakfoundation.org> Changes to Trunk (http://source.squeak.org/trunk.html) in the last 24 hours: http://lists.squeakfoundation.org/pipermail/packages/2015-December/009206.html Name: Morphic-jmg.1055 Ancestors: Morphic-mt.1054 If you're like me and have the swapMouseButtons perference unselected because you are on windows, then right-clicking on a morph and picking the 'select' menu item raises an exception because popUpFor:at:hand: is expecting a position for the at: argument, and currently this sends in a MouseEvent transformed by the relevant amount. We need to take one additional step and extract the position from this newly translated event, which I do in this change. ============================================= http://lists.squeakfoundation.org/pipermail/packages/2015-December/009207.html Name: Morphic-cwp.1055 Ancestors: Morphic-mt.1054 Implement PasteUpMorph>>referencePool Loading a project seems to require it. ============================================= http://lists.squeakfoundation.org/pipermail/packages/2015-December/009208.html Name: Morphic-dtl.1056 Ancestors: Morphic-jmg.1055, Morphic-cwp.1055 Merge Morphic-jmg.1055, Morphic-cwp.1055 ============================================= From jeffgonis at fastmail.com Sat Dec 5 00:04:19 2015 From: jeffgonis at fastmail.com (Jeff Gonis) Date: Sat Dec 5 00:04:22 2015 Subject: [squeak-dev] The Trunk: Morphic-dtl.1056.mcz In-Reply-To: <20151204041018.GA42787@shell.msen.com> References: <201512040034.tB40YnE3009549@shell.msen.com> <20151204041018.GA42787@shell.msen.com> Message-ID: <20151205000419.5386319.44036.41848@fastmail.com> Hi David,? Thank you very much for merging my change in! Hopefully I can submit some more things in the coming days. I appreciate your efforts!? Jeff From robert.w.withers at gmail.com Sat Dec 5 02:57:02 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Sat Dec 5 02:57:07 2015 Subject: [squeak-dev] Crypto plugins for ubuntu 32-bits Message-ID: <5662527E.8010509@gmail.com> I loaded the CryptographyPlugins package, from the Cryptography repository into a VMMaker image, and generated the crypto plugins externally. Although I have not measured it rigorously, there is a noticeable performance improvement with SqueakElib tests. Untar these Ubuntu 32-bit crypto plugins into /usr/local/lib/squeak/5.0... https://www.dropbox.com/s/v502zudwsbtr243/CryptoPlugins.ubuntu32.12.4.15.tgz?dl=0 Let me know any issues. Regards, Robert From tim at rowledge.org Sat Dec 5 04:37:10 2015 From: tim at rowledge.org (tim Rowledge) Date: Sat Dec 5 04:36:57 2015 Subject: [squeak-dev] Re: mouse enter/move/leave event confusion In-Reply-To: References: <4973AB59-FF96-41B1-AE15-904AAE8752D9@rowledge.org> <1449051662880-4864703.post@n4.nabble.com> <655B52F2-8736-476F-9B62-4A279AF46ECE@rowledge.org> <7D3C4F24-F59F-4C93-BE45-A454D91EB853@rowledge.org> <1449142410592-4864917.post@n4.nabble.com> Message-ID: > On 04-12-2015, at 2:33 AM, Stephan Eggermont wrote: > > Hmm. conceptually I would expect to get no spurious enter/leaves when the morph you're dragging around is either a submorph of the canvas, or grabbed by the hand. Well, I?ve now done this I think four ways, so roughly speaking - a) making a morph as a fake-cursor, opening it in the World and dragging it around via the canvas? mouseEnter/Move/Leave messages doesn?t work well. I got many leave/enter messages and so the fake-cursor would get deleted and recreated very often. Also, since my mouse enter code made the morph and opened it it would appear in the top-left before flickering to the ?proper? place and then back and forth with each leave/enter. b) the next try added the fake-cursor as a submorph of the canvas, which caused some interesting layout issues until I noticed that it really ought to get position before adding it. It worked relatively well except for ruining performance. c) another attempt at a) along with centring the morph on the cursor position showed me the error of my ways and explained the spurious leave/enter flickering. d) the latest code adds gridding to the HandMorph if and only if there is a temporaryCursor set. Using the ?attach? protocol simply didn?t occur to me. Which is crazy since I?ve used it enough to drag blocks and sprites around. Perhaps I?ll try it out. tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim A sad tale that brings a lump to the eye and a tear to the throat. From tim at rowledge.org Sat Dec 5 04:49:21 2015 From: tim at rowledge.org (tim Rowledge) Date: Sat Dec 5 04:49:08 2015 Subject: [squeak-dev] Unicode Support In-Reply-To: References: Message-ID: <6432DE4A-DEC1-432D-A950-9257A452C175@rowledge.org> > On 04-12-2015, at 6:46 AM, Levente Uzonyi wrote: > > Why would you want to have strings with UTF-8 or UTF-16 encoding in the image? > What's wrong with the current UTF-32 representation? WideStrings are perfectly ok most of the time, as are plain old byte String. Where things get a bit awkward is when interfacing to code that requires UTF8, such as Cairo/Pango and some OS interfaces. Currently we can have simple byte String and edit in or append a wide character and all works properly; a WideString is made, everything gets sorted out. Well, everything I?ve had to try out for the Pi Scratch project. The problem is in having to convert too often; for example every rendering operation requires a conversion from Squeak format to UTF8. Some file reading operations require conversion from utf8 to squeak. One idea I had but haven?t done anything with yet is to make a class that keeps both formats around to effectively cache the utf8. It isn?t needed for anywhere near all Strings. All editing/sorting would work on the squeak format part and after each edit the conversion would be done, or possibly the utf version flushed to cause a new conversion if it were ever requested. tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Useful Latin Phrases:- Sentio aliquos togatos contra me conspirare = I think some people in togas are plotting against me. From Marcel.Taeumel at hpi.de Sat Dec 5 09:13:10 2015 From: Marcel.Taeumel at hpi.de (marcel.taeumel) Date: Sat Dec 5 09:29:57 2015 Subject: [squeak-dev] Re: mouse enter/move/leave event confusion In-Reply-To: References: <4973AB59-FF96-41B1-AE15-904AAE8752D9@rowledge.org> <1449051662880-4864703.post@n4.nabble.com> <655B52F2-8736-476F-9B62-4A279AF46ECE@rowledge.org> <7D3C4F24-F59F-4C93-BE45-A454D91EB853@rowledge.org> <1449142410592-4864917.post@n4.nabble.com> Message-ID: <1449306790720-4865384.post@n4.nabble.com> Just a thought: If users only see a cursor that jumps along some grid, users might err too often. Especially if that grid is rather big. Best, Marcel -- View this message in context: http://forum.world.st/mouse-enter-move-leave-event-confusion-tp4864651p4865384.html Sent from the Squeak - Dev mailing list archive at Nabble.com. From rmacnak at gmail.com Sat Dec 5 19:10:08 2015 From: rmacnak at gmail.com (Ryan Macnak) Date: Sat Dec 5 19:10:11 2015 Subject: [squeak-dev] Unicode Support In-Reply-To: References: Message-ID: This may be out of the scope of your project, but there is also the issue that Squeak/Pharo don't display most characters. Copy the following in a workspace and only the first line is rendered properly. At a minimum there should be font substitution happening when the current font doesn't contain the necessary glyphs. Welcome ??????????? ????? ?????????? ????? ???? ?? ?????? ???????????? ???? ?? ?? ? ? On Fri, Dec 4, 2015 at 3:42 AM, EuanM wrote: > I'm currently groping my way to seeing how feature-complete our > Unicode support is. I am doing this to establish what still needs to > be done to provide full Unicode support. > > This seems to me to be an area where it would be best to write it > once, and then have the same codebase incorporated into the Smalltalks > that most share a common ancestry. > > I am keen to get: equality-testing for strings; sortability for > strings which have ligatures and diacritic characters; and correct > round-tripping of data. > > Call to action: > ========== > > If you have comments on these proposals - such as "but we already have > that facility" or "the reason we do not have these facilities is > because they are dog-slow" - please let me know them. > > If you would like to help out, please let me know. > > If you have Unicode experience and expertise, and would like to be, or > would be willing to be, in the 'council of experts' for this project, > please let me know. > > If you have comments or ideas on anything mentioned in this email > > In the first instance, the initiative's website will be: > http://smalltalk.uk.to/unicode.html > > I have created a SqueakSource.com project called UnicodeSupport > > I want to avoid re-inventing any facilities which already exist. > Except where they prevent us reaching the goals of: > - sortable UTF8 strings > - sortable UTF16 strings > - equivalence testing of 2 UTF8 strings > - equivalence testing of 2 UTF16 strings > - round-tripping UTF8 strings through Smalltalk > - roundtripping UTF16 strings through Smalltalk. > As I understand it, we have limited Unicode support atm. > > Current state of play > =============== > ByteString gets converted to WideString when need is automagically > detected. > > Is there anything else that currently exists? > > Definition of Terms > ============== > A quick definition of terms before I go any further: > > Standard terms from the Unicode standard > =============================== > a compatibility character : an additional encoding of a *normal* > character, for compatibility and round-trip conversion purposes. For > instance, a 1-byte encoding of a Latin character with a diacritic. > > Made-up terms > ============ > a convenience codepoint : a single codepoint which represents an item > that is also encoded as a string of codepoints. > > (I tend to use the terms compatibility character and compatibility > codepoint interchangably. The standard only refers to them as > compatibility characters. However, the standard is determined to > emphasise that characters are abstract and that codepoints are > concrete. So I think it is often more useful and productive to think > of compatibility or convenience codepoints). > > a composed character : a character made up of several codepoints > > Unicode encoding explained > ===================== > A convenience codepoint can therefore be thought of as a code point > used for a character which also has a composed form. > > The way Unicode works is that sometimes you can encode a character in > one byte, sometimes not. Sometimes you can encode it in two bytes, > sometimes not. > > You can therefore have a long stream of ASCII which is single-byte > Unicode. If there is an occasional Cyrillic or Greek character in the > stream, it would be represented either by a compatibility character or > by a multi-byte combination. > > Using compatibility characters can prevent proper sorting and > equivalence testing. > > Using "pure" Unicode, ie. "normal encodings", can cause compatibility > and round-tripping probelms. Although avoiding them can *also* cause > compatibility issues and round-tripping problems. > > Currently my thinking is: > > a Utf8String class > an Ordered collection, with 1 byte characters as the modal element, > but short arrays of wider strings where necessary > a Utf16String class > an Ordered collection, with 2 byte characters as the modal element, > but short arrays of wider strings > beginning with a 2-byte endianness indicator. > > Utf8Strings sometimes need to be sortable, and sometimes need to be > compatible. > > So my thinking is that Utf8String will contain convenience codepoints, > for round-tripping. And where there are multiple convenience > codepoints for a character, that it standardises on one. > > And that there is a Utf8SortableString which uses *only* normal characters. > > We then need methods to convert between the two. > > aUtf8String asUtf8SortableString > > and > > aUtf8SortableString asUtf8String > > > Sort orders are culture and context dependent - Sweden and Germany > have different sort orders for the same diacritic-ed characters. Some > countries have one order in general usage, and another for specific > usages, such as phone directories (e.g. UK and France) > > Similarly for Utf16 : Utf16String and Utf16SortableString and > conversion methods > > A list of sorted words would be a SortedCollection, and there could be > pre-prepared sortBlocks for them, e.g. frPhoneBookOrder, deOrder, > seOrder, ukOrder, etc > > along the lines of > aListOfWords := SortedCollection sortBlock: deOrder > > If a word is either a Utf8SortableString, or a well-formed Utf8String, > then we can perform equivalence testing on them trivially. > > To make sure a Utf8String is well formed, we would need to have a way > of cleaning up any convenience codepoints which were valid, but which > were for a character which has multiple equally-valid alternative > convenience codepoints, and for which the string currently had the > "wrong" convenience codepoint. (i.e for any character with valid > alternative convenience codepoints, we would choose one to be in the > well-formed Utf8String, and we would need a method for cleaning the > alternative convenience codepoints out of the string, and replacing > them with the chosen approved convenience codepoint. > > aUtf8String cleanUtf8String > > With WideString, a lot of the issues disappear - except > round-tripping(although I'm sure I have seen something recently about > 4-byte strings that also have an additional bit. Which would make > some Unicode characters 5-bytes long.) > > > (I'm starting to zone out now - if I've overlooked anything - obvious, > subtle, or somewhere in between, please let me know) > > Cheers, > Euan > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151205/bb26a3b8/attachment.htm From tim at rowledge.org Sat Dec 5 19:15:11 2015 From: tim at rowledge.org (tim Rowledge) Date: Sat Dec 5 19:15:15 2015 Subject: [squeak-dev] Unicode Support In-Reply-To: References: Message-ID: > On 05-12-2015, at 11:10 AM, Ryan Macnak wrote: > > This may be out of the scope of your project, but there is also the issue that Squeak/Pharo don't display most characters. Copy the following in a workspace and only the first line is rendered properly. At a minimum there should be font substitution happening when the current font doesn't contain the necessary glyphs. We?ll need to properly complete an interface to Cairo/Pango or perhaps even better, incorporate Nile. In Pi Scratch I have moderately tacky-hack to make all text rendering go via Pango if the relevant plugin is present. It is surprisingly effective, with no noticeable impact on the text rendering performance *within Scratch*. I don?t know what it might do to performance for general dev ui tool text rendering; clearly large volumes of text in multi-paragraph chunks is not an issue in Scratch. tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Useful random insult:- IQ = dx / (1 + dx), where x = age. From lewis at mail.msen.com Sat Dec 5 22:28:11 2015 From: lewis at mail.msen.com (David T. Lewis) Date: Sat Dec 5 22:29:05 2015 Subject: [squeak-dev] Crypto plugins for ubuntu 32-bits In-Reply-To: <5662527E.8010509@gmail.com> References: <5662527E.8010509@gmail.com> Message-ID: <20151205222811.GA48751@shell.msen.com> On Fri, Dec 04, 2015 at 09:57:02PM -0500, Robert Withers wrote: > I loaded the CryptographyPlugins package, from the Cryptography > repository into a VMMaker image, and generated the crypto plugins > externally. Although I have not measured it rigorously, there is a > noticeable performance improvement with SqueakElib tests. > > Untar these Ubuntu 32-bit crypto plugins into /usr/local/lib/squeak/5.0... > > https://www.dropbox.com/s/v502zudwsbtr243/CryptoPlugins.ubuntu32.12.4.15.tgz?dl=0 > > Let me know any issues. > > Regards, > Robert Yikes, it seems that we have had this issue open for nearly 10 years now: http://bugs.squeak.org/view.php?id=5228 "0005228: Include Cryptographic Primitives in Base VM - DESPlugin" Robert, I think you are on the vm-dev list, may I ask some follow up questions there? Dave From robert.w.withers at gmail.com Sat Dec 5 22:43:15 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Sat Dec 5 22:43:19 2015 Subject: [squeak-dev] Crypto plugins for ubuntu 32-bits In-Reply-To: <20151205222811.GA48751@shell.msen.com> References: <5662527E.8010509@gmail.com> <20151205222811.GA48751@shell.msen.com> Message-ID: <56636883.50004@gmail.com> 10 years is long enough. :) How can I help? On 12/05/2015 05:28 PM, David T. Lewis wrote: > On Fri, Dec 04, 2015 at 09:57:02PM -0500, Robert Withers wrote: >> I loaded the CryptographyPlugins package, from the Cryptography >> repository into a VMMaker image, and generated the crypto plugins >> externally. Although I have not measured it rigorously, there is a >> noticeable performance improvement with SqueakElib tests. >> >> Untar these Ubuntu 32-bit crypto plugins into /usr/local/lib/squeak/5.0... >> >> https://www.dropbox.com/s/v502zudwsbtr243/CryptoPlugins.ubuntu32.12.4.15.tgz?dl=0 >> >> Let me know any issues. >> >> Regards, >> Robert > Yikes, it seems that we have had this issue open for nearly 10 years now: > > http://bugs.squeak.org/view.php?id=5228 "0005228: Include Cryptographic Primitives in Base VM - DESPlugin" > > Robert, I think you are on the vm-dev list, may I ask some follow up questions there? > > Dave > From btc at openinworld.com Sun Dec 6 01:34:39 2015 From: btc at openinworld.com (Ben Coman) Date: Sun Dec 6 01:35:01 2015 Subject: [squeak-dev] SqueakElib project In-Reply-To: <5661959B.3060804@gmail.com> References: <5661959B.3060804@gmail.com> Message-ID: This is interesting, "error correction through Reed Solomon encoding" but its not clear how its used. Maybe a sketch of the architecture would be useful. cheers -ben On Fri, Dec 4, 2015 at 9:31 PM, Robert Withers wrote: > Good morning, > > Here is a brief writeup of the SqueakElib project and its direction. > > https://github.com/SqueakCryptographySquad/Cryptography-Common/wiki/SqueakElib. > > All new squad members welcome. > > Regards, > Robert > From btc at openinworld.com Sun Dec 6 02:24:13 2015 From: btc at openinworld.com (Ben Coman) Date: Sun Dec 6 02:24:35 2015 Subject: [squeak-dev] Re: [Pharo-users] evolutions of squeakelib & crypto (Reed Solomon) In-Reply-To: <5661B7F1.1080701@gmail.com> References: <565DCE53.90501@gmail.com> <565DD073.1060409@gmail.com> <565FF8ED.1030606@free.fr> <5660BCFA.4050107@gmail.com> <56616E9C.6040201@gmail.com> <56616F27.4070002@gmail.com> <566174E9.9020001@gmail.com> <566175EA.2090107@gmail.com> <60B913A3-565C-4924-9F82-4E0E96553B72@gmail.com> <5661A6A5.9030902@gmail.com> <224A3FD6-A8DA-45E3-8EDE-AA52133940CC@stfx.eu> <5661B7F1.1080701@gmail.com> Message-ID: On Fri, Dec 4, 2015 at 11:57 PM, Robert Withers wrote: > Now I think you are right on with your observation. Additionally, the number > of dialects could increase further with Fuel serialization, just port > SecureSession and bits. > > Alright, I came up with a name and it may border on the egregious ... > presenting ... > > "Maelstrom" Great sounding name. However some general advice for the community, since I see a lot of great sounding project names drowned out in the noise of our web-search-centric universe. A litmus test for project naming is using google search to find which return low search results. Today, its more important to be unique than any other attribute of a name. So in general, *dictionary* english words are not the best. One technique is to intentionally mispell the word you like. Here are some comparative examples (note, the surrounding quotes are required to avoid google trying to be helpful and correct the spelling)... "maelstrom" --> 7,480,000 "maelstroom" --> 6,200 "maelstrum" --> 2,280 "maelstruum" --> 7 Lots of interesting other techniques can be found by searching on: techniques to generate brand names or domain names. cheers -ben > > A new distributed metric will be established in Maelstrom to measure intra- > & inter-vat activity: "Sanguinity". > > Robert > > PS. thanks for pointing out Fuel support for older squeaks. > > > On 12/04/2015 10:25 AM, Sven Van Caekenberghe wrote: >> >> Another suggestion: if you intend to make something cross dialect and hope >> to get traction, I would not use a project name which contains one dialect, >> but something more general like Cryptography, or some cool project name, >> like Seaside, Fuel, ... >> >>> On 04 Dec 2015, at 15:43, Robert Withers >>> wrote: >>> >>> I use squeak 5.0 and would want the Fuel support to customize for wire >>> serializations and substitutions, such that Squeak and Pharo could talk to >>> each other, and all other Fuel environments. The Fuel changes I made are in >>> the Pharo port of SqueakElib in the SqueakElib-CapTP-Serialization category >>> and consist of a Decoder, Materializer and a Materialization. >>> >>> Robert >>> >>> On 12/04/2015 09:35 AM, Esteban Lorenzano wrote: >>>> >>>> which squeak version? >>>> of what? >>>> >>>>> On 04 Dec 2015, at 12:15, Robert Withers >>>>> wrote: >>>>> >>>>> I just realized that the squeak version uses ReferenceStream while the >>>>> Pharo version uses Fuel, so the binary serializations are different and they >>>>> won't speak to each other. Any chance that Fuel is ported to Squeak? >>>>> >>>>> Regards, >>>>> Robert >>>>> >>>>> On 12/04/2015 06:11 AM, Robert Withers wrote: >>>>>> >>>>>> I am unable to import these files into SqueakSource, so it may be best >>>>>> done from inside Pharo with Monticello. Here are working Crypto and >>>>>> SqueakElib in Pharo, prior to SecureSession refactoring and Reed Solomon. I >>>>>> include the correct version of LayeredProtocol. >>>>>> >>>>>> Regards, >>>>>> Robert >>>>>> >>>>>> >>>>>> On 12/04/2015 05:47 AM, Robert Withers wrote: >>>>>>> >>>>>>> Best Regards >>>>>>> >>>>>>> http://www.squeaksource.com/Cryptography.html >>>>>>> http://www.squeaksource.com/squeakelib.html >>>>>>> >>>>>>> >>>>>>> On 12/04/2015 05:44 AM, Robert Withers wrote: >>>>>>>> >>>>>>>> After my password reset on squeaksource, I committed to both >>>>>>>> Cryptography and SqueakElib, project links below. >>>>>>>> >>>>>>>> In the case of Cryptography, I had a version ported to Pharo ... I >>>>>>>> will organize Pharo ports of both after Reed Solomon is stable, and announce >>>>>>>> them to the Pharo list. This way both environments can be supported through >>>>>>>> this one repository. >>>>>>>> >>>>>>>> Robert >>>>>>>> >>>>>>>> On 12/04/2015 04:35 AM, Stephan Eggermont wrote: >>>>>>>>> >>>>>>>>> On 03-12-15 23:06, Robert Withers wrote: >>>>>>>>>> >>>>>>>>>> Are any of these used by both squeak and Pharo? That would be the >>>>>>>>>> right >>>>>>>>>> move I think. I will ask about getting my password reset for >>>>>>>>>> squeaksource, since that is where the old code resides. >>>>>>>>> >>>>>>>>> All of them. Mostly timing of project start/high activity and who >>>>>>>>> are >>>>>>>>> maintaining it decided on platforms. There was a time when >>>>>>>>> squeaksource was not so stable and then many projects migrated, and I >>>>>>>>> currently hear least about stability issues from ss3, but the load on >>>>>>>>> smalltalkhub is much higher, I assume, as that is used for the pharo ci. >>>>>>>>> >>>>>>>>> In the not so far future Pharo is likely to move to a git based >>>>>>>>> infrastructure, using libgit2. Early adopters are already using it. >>>>>>>>> I haven't heard the squeak ideas about that. >>>>>>>>> >>>>>>>>> Stephan >>>>>>>>> >>>>>>>>> >>>>>>>>> >> > > From robert.w.withers at gmail.com Sun Dec 6 02:30:59 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Sun Dec 6 02:31:03 2015 Subject: [squeak-dev] SqueakElib project In-Reply-To: References: <5661959B.3060804@gmail.com> Message-ID: <56639DE3.5020908@gmail.com> I could briefly explain and put creation of new SecureSession architecture sketch on my medium stack. I have a few diagrams already which I will add, but none of this new SecureSession design. This is done so please check that page. Reed Solomon encoding: Using Reed-Solomon encoding for Forward Error Correction is used is satellite communications. Other lossy communications probably uses this or something similar. By interleaving RS encoded blocks, you quadruple the contiguous bytes that can be corrupted and still recovered. My use here will specify 4 interleaved blocks. So the encoding is really an RS encoding of 4 blocks and then interleaving them. Note that QR Codes are using RS encoding. RS encoding uses a GaloisField, or a finite field. This is a system of polynomials that are multiplied, divided, inversed and added to transform the data into code (as in coded data). The Galois Field is initialized with a primitive polynomial that sets the symbol size of data, symbol size of code and the number of errors that is still recoverable. Thus there are many different primitive polynomials as there are uses of RS encoding. Note that the symbol size tends to be a nibble. I use 3 different sized primitive polynomials, so I have different sized blocks depending on the size of the payload to be fragmented and encoded, in groups of 4 interleaved blocks. The easiest RS encoding to look at has 9 data symbols, 15 code symbols and can handle 3 symbol errors. Considering that these 15 symbols/block will be interleaved with 3 other blocks, we are talking about 60 symbols, or 30 bytes coded. This is 18 bytes of data. As they are interleaved with 3 errors per block permissible, that is 12 contiguous bytes can be corrupted and the msg recovered. The interleaving takes the first byte from each of 4 blocks, then the second... [byteA1, byteB1, byteC1, byteD1, byteA2, byteB2, ...] I hope this helps! I'll add it to the project wiki. Robert On 12/05/2015 08:34 PM, Ben Coman wrote: > This is interesting, "error correction through Reed Solomon encoding" > but its not clear how its used. Maybe a sketch of the architecture > would be useful. > cheers -ben > > On Fri, Dec 4, 2015 at 9:31 PM, Robert Withers > wrote: >> Good morning, >> >> Here is a brief writeup of the SqueakElib project and its direction. >> >> https://github.com/SqueakCryptographySquad/Cryptography-Common/wiki/SqueakElib. >> >> All new squad members welcome. >> >> Regards, >> Robert >> From robert.w.withers at gmail.com Sun Dec 6 02:42:20 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Sun Dec 6 02:42:25 2015 Subject: [squeak-dev] Re: [Pharo-users] evolutions of squeakelib & crypto (Reed Solomon) In-Reply-To: References: <565DCE53.90501@gmail.com> <565DD073.1060409@gmail.com> <565FF8ED.1030606@free.fr> <5660BCFA.4050107@gmail.com> <56616E9C.6040201@gmail.com> <56616F27.4070002@gmail.com> <566174E9.9020001@gmail.com> <566175EA.2090107@gmail.com> <60B913A3-565C-4924-9F82-4E0E96553B72@gmail.com> <5661A6A5.9030902@gmail.com> <224A3FD6-A8DA-45E3-8EDE-AA52133940CC@stfx.eu> <5661B7F1.1080701@gmail.com> Message-ID: <5663A08C.8030602@gmail.com> On 12/05/2015 09:24 PM, Ben Coman wrote: > On Fri, Dec 4, 2015 at 11:57 PM, Robert Withers > wrote: >> Now I think you are right on with your observation. Additionally, the number >> of dialects could increase further with Fuel serialization, just port >> SecureSession and bits. >> >> Alright, I came up with a name and it may border on the egregious ... >> presenting ... >> >> "Maelstrom" > Great sounding name. However some general advice for the community, > since I see a lot of great sounding project names drowned out in the > noise of our web-search-centric universe. A litmus test for project > naming is using google search to find which return low search results. > Today, its more important to be unique than any other attribute of a > name. So in general, *dictionary* english words are not the best. > One technique is to intentionally mispell the word you like. Here are > some comparative examples (note, the surrounding quotes are required > to avoid google trying to be helpful and correct the spelling)... > > "maelstrom" --> 7,480,000 > "maelstroom" --> 6,200 > "maelstrum" --> 2,280 > "maelstruum" --> 7 > > Lots of interesting other techniques can be found by searching on: > techniques to generate brand names or domain names. > > cheers -ben I would be happy to change the names to something more unique, though it may take a few. Are you suggesting "maelstruum"? cheers, Robert > >> A new distributed metric will be established in Maelstrom to measure intra- >> & inter-vat activity: "Sanguinity". >> >> Robert >> >> PS. thanks for pointing out Fuel support for older squeaks. >> >> >> On 12/04/2015 10:25 AM, Sven Van Caekenberghe wrote: >>> Another suggestion: if you intend to make something cross dialect and hope >>> to get traction, I would not use a project name which contains one dialect, >>> but something more general like Cryptography, or some cool project name, >>> like Seaside, Fuel, ... >>> >>>> On 04 Dec 2015, at 15:43, Robert Withers >>>> wrote: >>>> >>>> I use squeak 5.0 and would want the Fuel support to customize for wire >>>> serializations and substitutions, such that Squeak and Pharo could talk to >>>> each other, and all other Fuel environments. The Fuel changes I made are in >>>> the Pharo port of SqueakElib in the SqueakElib-CapTP-Serialization category >>>> and consist of a Decoder, Materializer and a Materialization. >>>> >>>> Robert >>>> >>>> On 12/04/2015 09:35 AM, Esteban Lorenzano wrote: >>>>> which squeak version? >>>>> of what? >>>>> >>>>>> On 04 Dec 2015, at 12:15, Robert Withers >>>>>> wrote: >>>>>> >>>>>> I just realized that the squeak version uses ReferenceStream while the >>>>>> Pharo version uses Fuel, so the binary serializations are different and they >>>>>> won't speak to each other. Any chance that Fuel is ported to Squeak? >>>>>> >>>>>> Regards, >>>>>> Robert >>>>>> >>>>>> On 12/04/2015 06:11 AM, Robert Withers wrote: >>>>>>> I am unable to import these files into SqueakSource, so it may be best >>>>>>> done from inside Pharo with Monticello. Here are working Crypto and >>>>>>> SqueakElib in Pharo, prior to SecureSession refactoring and Reed Solomon. I >>>>>>> include the correct version of LayeredProtocol. >>>>>>> >>>>>>> Regards, >>>>>>> Robert >>>>>>> >>>>>>> >>>>>>> On 12/04/2015 05:47 AM, Robert Withers wrote: >>>>>>>> Best Regards >>>>>>>> >>>>>>>> http://www.squeaksource.com/Cryptography.html >>>>>>>> http://www.squeaksource.com/squeakelib.html >>>>>>>> >>>>>>>> >>>>>>>> On 12/04/2015 05:44 AM, Robert Withers wrote: >>>>>>>>> After my password reset on squeaksource, I committed to both >>>>>>>>> Cryptography and SqueakElib, project links below. >>>>>>>>> >>>>>>>>> In the case of Cryptography, I had a version ported to Pharo ... I >>>>>>>>> will organize Pharo ports of both after Reed Solomon is stable, and announce >>>>>>>>> them to the Pharo list. This way both environments can be supported through >>>>>>>>> this one repository. >>>>>>>>> >>>>>>>>> Robert >>>>>>>>> >>>>>>>>> On 12/04/2015 04:35 AM, Stephan Eggermont wrote: >>>>>>>>>> On 03-12-15 23:06, Robert Withers wrote: >>>>>>>>>>> Are any of these used by both squeak and Pharo? That would be the >>>>>>>>>>> right >>>>>>>>>>> move I think. I will ask about getting my password reset for >>>>>>>>>>> squeaksource, since that is where the old code resides. >>>>>>>>>> All of them. Mostly timing of project start/high activity and who >>>>>>>>>> are >>>>>>>>>> maintaining it decided on platforms. There was a time when >>>>>>>>>> squeaksource was not so stable and then many projects migrated, and I >>>>>>>>>> currently hear least about stability issues from ss3, but the load on >>>>>>>>>> smalltalkhub is much higher, I assume, as that is used for the pharo ci. >>>>>>>>>> >>>>>>>>>> In the not so far future Pharo is likely to move to a git based >>>>>>>>>> infrastructure, using libgit2. Early adopters are already using it. >>>>>>>>>> I haven't heard the squeak ideas about that. >>>>>>>>>> >>>>>>>>>> Stephan >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >> From robert.w.withers at gmail.com Sun Dec 6 02:59:07 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Sun Dec 6 02:59:13 2015 Subject: [squeak-dev] SqueakElib project In-Reply-To: <56639DE3.5020908@gmail.com> References: <5661959B.3060804@gmail.com> <56639DE3.5020908@gmail.com> Message-ID: <5663A47B.5040508@gmail.com> On 12/05/2015 09:30 PM, Robert Withers wrote: > I could briefly explain and put creation of new SecureSession > architecture sketch on my medium stack. I have a few diagrams already > which I will add, but none of this new SecureSession design. This is > done so please check that page. > > Reed Solomon encoding: > Using Reed-Solomon encoding for Forward Error Correction is used is > satellite communications. Other lossy communications probably uses > this or something similar. By interleaving RS encoded blocks, you > quadruple the contiguous bytes that can be corrupted and still > recovered. My use here will specify 4 interleaved blocks. So the > encoding is really an RS encoding of 4 blocks and then interleaving them. > > Note that QR Codes are using RS encoding. > > RS encoding uses a GaloisField, or a finite field. This is a system of > polynomials that are multiplied, divided, inversed and added to > transform the data into code (as in coded data). The Galois Field is > initialized with a primitive polynomial that sets the symbol size of > data, symbol size of code and the number of errors that is still > recoverable. Thus there are many different primitive polynomials as > there are uses of RS encoding. Note that the symbol size tends to be a > nibble. I use 3 different sized primitive polynomials, so I have > different sized blocks depending on the size of the payload to be > fragmented and encoded, in groups of 4 interleaved blocks. > > The easiest RS encoding to look at has 9 data symbols, 15 code symbols > and can handle 3 symbol errors. Considering that these 15 > symbols/block will be interleaved with 3 other blocks, we are talking > about 60 symbols, or 30 bytes coded. This is 18 bytes of data. As they > are interleaved with 3 errors per block permissible, that is 12 > contiguous bytes can be corrupted and the msg recovered. I need to make a correction to what I've said, that is 3 symbol errors per coded block, or 3 nibbles. So the contiguous space that can be recovered is 6 bytes coded, out of 30 bytes. Higher degree primitive polynomials have different sizes for data, code, errors and so a larger encoding with 4-block interleaving may result in hundreds of contiguous bytes being resilient. I do think it is in the neighborhood of 10-20% of the coded message is recoverable, generally. Robert > > The interleaving takes the first byte from each of 4 blocks, then the > second... > > [byteA1, byteB1, byteC1, byteD1, byteA2, byteB2, ...] > > I hope this helps! I'll add it to the project wiki. > > Robert > > > > On 12/05/2015 08:34 PM, Ben Coman wrote: >> This is interesting, "error correction through Reed Solomon encoding" >> but its not clear how its used. Maybe a sketch of the architecture >> would be useful. >> cheers -ben >> >> On Fri, Dec 4, 2015 at 9:31 PM, Robert Withers >> wrote: >>> Good morning, >>> >>> Here is a brief writeup of the SqueakElib project and its direction. >>> >>> https://github.com/SqueakCryptographySquad/Cryptography-Common/wiki/SqueakElib. >>> >>> >>> All new squad members welcome. >>> >>> Regards, >>> Robert >>> > From robert.w.withers at gmail.com Sun Dec 6 03:41:05 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Sun Dec 6 03:41:10 2015 Subject: [squeak-dev] java to squeak conversation Message-ID: <5663AE51.1040103@gmail.com> I still have some rot in my head, from too much java, and so I don't think consistently clearly, it's more cyclic I've noticed. As a result, my eyes are doing back-flips when looking at this code; perhaps I need eine kleine pause...does anyone else want to try it? int termPlus1 = ((term & 0x1) == 0) ? (term | 1) : (term & ~1); cheers, Robert ps. an example of Java conciseness, dear God. From colin at wiresong.com Sun Dec 6 03:55:50 2015 From: colin at wiresong.com (Colin Putney) Date: Sun Dec 6 03:55:55 2015 Subject: [squeak-dev] Unicode Support In-Reply-To: References: Message-ID: First, what's UTF-32? Second, we have the whole language tag thing that nobody else uses. Finally, UTF-8 is a great encoding that certain kinds of applications really ought to use. Web apps, in particular, benefit from using UTF-8 so the don't have to decode and then re-encode strings coming in from the network. In DabbleDB we used UTF-8 encoded string in the image, and just ignored the fact that they were displayed incorrectly by inspectors. Having a proper UTF-8 string class would be useful. - Colin > On Dec 4, 2015, at 6:46 AM, Levente Uzonyi wrote: > > Why would you want to have strings with UTF-8 or UTF-16 encoding in the image? > What's wrong with the current UTF-32 representation? > > Levente > >> On Fri, 4 Dec 2015, EuanM wrote: >> >> I'm currently groping my way to seeing how feature-complete our >> Unicode support is. I am doing this to establish what still needs to >> be done to provide full Unicode support. >> >> This seems to me to be an area where it would be best to write it >> once, and then have the same codebase incorporated into the Smalltalks >> that most share a common ancestry. >> >> I am keen to get: equality-testing for strings; sortability for >> strings which have ligatures and diacritic characters; and correct >> round-tripping of data. >> >> Call to action: >> ========== >> >> If you have comments on these proposals - such as "but we already have >> that facility" or "the reason we do not have these facilities is >> because they are dog-slow" - please let me know them. >> >> If you would like to help out, please let me know. >> >> If you have Unicode experience and expertise, and would like to be, or >> would be willing to be, in the 'council of experts' for this project, >> please let me know. >> >> If you have comments or ideas on anything mentioned in this email >> >> In the first instance, the initiative's website will be: >> http://smalltalk.uk.to/unicode.html >> >> I have created a SqueakSource.com project called UnicodeSupport >> >> I want to avoid re-inventing any facilities which already exist. >> Except where they prevent us reaching the goals of: >> - sortable UTF8 strings >> - sortable UTF16 strings >> - equivalence testing of 2 UTF8 strings >> - equivalence testing of 2 UTF16 strings >> - round-tripping UTF8 strings through Smalltalk >> - roundtripping UTF16 strings through Smalltalk. >> As I understand it, we have limited Unicode support atm. >> >> Current state of play >> =============== >> ByteString gets converted to WideString when need is automagically detected. >> >> Is there anything else that currently exists? >> >> Definition of Terms >> ============== >> A quick definition of terms before I go any further: >> >> Standard terms from the Unicode standard >> =============================== >> a compatibility character : an additional encoding of a *normal* >> character, for compatibility and round-trip conversion purposes. For >> instance, a 1-byte encoding of a Latin character with a diacritic. >> >> Made-up terms >> ============ >> a convenience codepoint : a single codepoint which represents an item >> that is also encoded as a string of codepoints. >> >> (I tend to use the terms compatibility character and compatibility >> codepoint interchangably. The standard only refers to them as >> compatibility characters. However, the standard is determined to >> emphasise that characters are abstract and that codepoints are >> concrete. So I think it is often more useful and productive to think >> of compatibility or convenience codepoints). >> >> a composed character : a character made up of several codepoints >> >> Unicode encoding explained >> ===================== >> A convenience codepoint can therefore be thought of as a code point >> used for a character which also has a composed form. >> >> The way Unicode works is that sometimes you can encode a character in >> one byte, sometimes not. Sometimes you can encode it in two bytes, >> sometimes not. >> >> You can therefore have a long stream of ASCII which is single-byte >> Unicode. If there is an occasional Cyrillic or Greek character in the >> stream, it would be represented either by a compatibility character or >> by a multi-byte combination. >> >> Using compatibility characters can prevent proper sorting and >> equivalence testing. >> >> Using "pure" Unicode, ie. "normal encodings", can cause compatibility >> and round-tripping probelms. Although avoiding them can *also* cause >> compatibility issues and round-tripping problems. >> >> Currently my thinking is: >> >> a Utf8String class >> an Ordered collection, with 1 byte characters as the modal element, >> but short arrays of wider strings where necessary >> a Utf16String class >> an Ordered collection, with 2 byte characters as the modal element, >> but short arrays of wider strings >> beginning with a 2-byte endianness indicator. >> >> Utf8Strings sometimes need to be sortable, and sometimes need to be compatible. >> >> So my thinking is that Utf8String will contain convenience codepoints, >> for round-tripping. And where there are multiple convenience >> codepoints for a character, that it standardises on one. >> >> And that there is a Utf8SortableString which uses *only* normal characters. >> >> We then need methods to convert between the two. >> >> aUtf8String asUtf8SortableString >> >> and >> >> aUtf8SortableString asUtf8String >> >> >> Sort orders are culture and context dependent - Sweden and Germany >> have different sort orders for the same diacritic-ed characters. Some >> countries have one order in general usage, and another for specific >> usages, such as phone directories (e.g. UK and France) >> >> Similarly for Utf16 : Utf16String and Utf16SortableString and >> conversion methods >> >> A list of sorted words would be a SortedCollection, and there could be >> pre-prepared sortBlocks for them, e.g. frPhoneBookOrder, deOrder, >> seOrder, ukOrder, etc >> >> along the lines of >> aListOfWords := SortedCollection sortBlock: deOrder >> >> If a word is either a Utf8SortableString, or a well-formed Utf8String, >> then we can perform equivalence testing on them trivially. >> >> To make sure a Utf8String is well formed, we would need to have a way >> of cleaning up any convenience codepoints which were valid, but which >> were for a character which has multiple equally-valid alternative >> convenience codepoints, and for which the string currently had the >> "wrong" convenience codepoint. (i.e for any character with valid >> alternative convenience codepoints, we would choose one to be in the >> well-formed Utf8String, and we would need a method for cleaning the >> alternative convenience codepoints out of the string, and replacing >> them with the chosen approved convenience codepoint. >> >> aUtf8String cleanUtf8String >> >> With WideString, a lot of the issues disappear - except >> round-tripping(although I'm sure I have seen something recently about >> 4-byte strings that also have an additional bit. Which would make >> some Unicode characters 5-bytes long.) >> >> >> (I'm starting to zone out now - if I've overlooked anything - obvious, >> subtle, or somewhere in between, please let me know) >> >> Cheers, >> Euan >> >> > From leves at caesar.elte.hu Sun Dec 6 04:41:22 2015 From: leves at caesar.elte.hu (Levente Uzonyi) Date: Sun Dec 6 04:41:29 2015 Subject: [squeak-dev] Unicode Support In-Reply-To: References: Message-ID: On Sat, 5 Dec 2015, Colin Putney wrote: > First, what's UTF-32? Second, we have the whole language tag thing that nobody else uses. In Squeak, Strings use UTF-32 encoding[1]. It's straightforward to see for WideString, but ByteString is just a subset of WideString, so it uses the same encoding. We also use language tags, but that's a different story. Language tags make it possible to work around the problems introduced by the Han unification[2]. We shouldn't really use them for non-CJKV languages. > > Finally, UTF-8 is a great encoding that certain kinds of applications really ought to use. Web apps, in particular, benefit from using UTF-8 so the don't have to decode and then re-encode strings coming in from the network. In DabbleDB we used UTF-8 encoded string in the image, and just ignored the fact that they were displayed incorrectly by inspectors. Having a proper UTF-8 string class would be useful. We do the same thing, but that doesn't mean it's a good idea to create a new String-like class having its content encoded in UTF-8, because UTF-8-encoded strings can't be modified like regular strings. While it would be possible to implement all operations, such implementation would become the next SortedCollection (bad performance due to misuse). Levente [1] https://en.wikipedia.org/wiki/UTF-32 [2] https://en.wikipedia.org/wiki/Han_unification > > - Colin > > >> On Dec 4, 2015, at 6:46 AM, Levente Uzonyi wrote: >> >> Why would you want to have strings with UTF-8 or UTF-16 encoding in the image? >> What's wrong with the current UTF-32 representation? >> >> Levente >> >>> On Fri, 4 Dec 2015, EuanM wrote: >>> >>> I'm currently groping my way to seeing how feature-complete our >>> Unicode support is. I am doing this to establish what still needs to >>> be done to provide full Unicode support. >>> >>> This seems to me to be an area where it would be best to write it >>> once, and then have the same codebase incorporated into the Smalltalks >>> that most share a common ancestry. >>> >>> I am keen to get: equality-testing for strings; sortability for >>> strings which have ligatures and diacritic characters; and correct >>> round-tripping of data. >>> >>> Call to action: >>> ========== >>> >>> If you have comments on these proposals - such as "but we already have >>> that facility" or "the reason we do not have these facilities is >>> because they are dog-slow" - please let me know them. >>> >>> If you would like to help out, please let me know. >>> >>> If you have Unicode experience and expertise, and would like to be, or >>> would be willing to be, in the 'council of experts' for this project, >>> please let me know. >>> >>> If you have comments or ideas on anything mentioned in this email >>> >>> In the first instance, the initiative's website will be: >>> http://smalltalk.uk.to/unicode.html >>> >>> I have created a SqueakSource.com project called UnicodeSupport >>> >>> I want to avoid re-inventing any facilities which already exist. >>> Except where they prevent us reaching the goals of: >>> - sortable UTF8 strings >>> - sortable UTF16 strings >>> - equivalence testing of 2 UTF8 strings >>> - equivalence testing of 2 UTF16 strings >>> - round-tripping UTF8 strings through Smalltalk >>> - roundtripping UTF16 strings through Smalltalk. >>> As I understand it, we have limited Unicode support atm. >>> >>> Current state of play >>> =============== >>> ByteString gets converted to WideString when need is automagically detected. >>> >>> Is there anything else that currently exists? >>> >>> Definition of Terms >>> ============== >>> A quick definition of terms before I go any further: >>> >>> Standard terms from the Unicode standard >>> =============================== >>> a compatibility character : an additional encoding of a *normal* >>> character, for compatibility and round-trip conversion purposes. For >>> instance, a 1-byte encoding of a Latin character with a diacritic. >>> >>> Made-up terms >>> ============ >>> a convenience codepoint : a single codepoint which represents an item >>> that is also encoded as a string of codepoints. >>> >>> (I tend to use the terms compatibility character and compatibility >>> codepoint interchangably. The standard only refers to them as >>> compatibility characters. However, the standard is determined to >>> emphasise that characters are abstract and that codepoints are >>> concrete. So I think it is often more useful and productive to think >>> of compatibility or convenience codepoints). >>> >>> a composed character : a character made up of several codepoints >>> >>> Unicode encoding explained >>> ===================== >>> A convenience codepoint can therefore be thought of as a code point >>> used for a character which also has a composed form. >>> >>> The way Unicode works is that sometimes you can encode a character in >>> one byte, sometimes not. Sometimes you can encode it in two bytes, >>> sometimes not. >>> >>> You can therefore have a long stream of ASCII which is single-byte >>> Unicode. If there is an occasional Cyrillic or Greek character in the >>> stream, it would be represented either by a compatibility character or >>> by a multi-byte combination. >>> >>> Using compatibility characters can prevent proper sorting and >>> equivalence testing. >>> >>> Using "pure" Unicode, ie. "normal encodings", can cause compatibility >>> and round-tripping probelms. Although avoiding them can *also* cause >>> compatibility issues and round-tripping problems. >>> >>> Currently my thinking is: >>> >>> a Utf8String class >>> an Ordered collection, with 1 byte characters as the modal element, >>> but short arrays of wider strings where necessary >>> a Utf16String class >>> an Ordered collection, with 2 byte characters as the modal element, >>> but short arrays of wider strings >>> beginning with a 2-byte endianness indicator. >>> >>> Utf8Strings sometimes need to be sortable, and sometimes need to be compatible. >>> >>> So my thinking is that Utf8String will contain convenience codepoints, >>> for round-tripping. And where there are multiple convenience >>> codepoints for a character, that it standardises on one. >>> >>> And that there is a Utf8SortableString which uses *only* normal characters. >>> >>> We then need methods to convert between the two. >>> >>> aUtf8String asUtf8SortableString >>> >>> and >>> >>> aUtf8SortableString asUtf8String >>> >>> >>> Sort orders are culture and context dependent - Sweden and Germany >>> have different sort orders for the same diacritic-ed characters. Some >>> countries have one order in general usage, and another for specific >>> usages, such as phone directories (e.g. UK and France) >>> >>> Similarly for Utf16 : Utf16String and Utf16SortableString and >>> conversion methods >>> >>> A list of sorted words would be a SortedCollection, and there could be >>> pre-prepared sortBlocks for them, e.g. frPhoneBookOrder, deOrder, >>> seOrder, ukOrder, etc >>> >>> along the lines of >>> aListOfWords := SortedCollection sortBlock: deOrder >>> >>> If a word is either a Utf8SortableString, or a well-formed Utf8String, >>> then we can perform equivalence testing on them trivially. >>> >>> To make sure a Utf8String is well formed, we would need to have a way >>> of cleaning up any convenience codepoints which were valid, but which >>> were for a character which has multiple equally-valid alternative >>> convenience codepoints, and for which the string currently had the >>> "wrong" convenience codepoint. (i.e for any character with valid >>> alternative convenience codepoints, we would choose one to be in the >>> well-formed Utf8String, and we would need a method for cleaning the >>> alternative convenience codepoints out of the string, and replacing >>> them with the chosen approved convenience codepoint. >>> >>> aUtf8String cleanUtf8String >>> >>> With WideString, a lot of the issues disappear - except >>> round-tripping(although I'm sure I have seen something recently about >>> 4-byte strings that also have an additional bit. Which would make >>> some Unicode characters 5-bytes long.) >>> >>> >>> (I'm starting to zone out now - if I've overlooked anything - obvious, >>> subtle, or somewhere in between, please let me know) >>> >>> Cheers, >>> Euan >>> >>> >> > > From colin at wiresong.com Sun Dec 6 05:19:36 2015 From: colin at wiresong.com (Colin Putney) Date: Sun Dec 6 05:19:38 2015 Subject: [squeak-dev] Unicode Support In-Reply-To: References: Message-ID: On Sat, Dec 5, 2015 at 8:41 PM, Levente Uzonyi wrote: > We do the same thing, but that doesn't mean it's a good idea to create a > new String-like class having its content encoded in UTF-8, because > UTF-8-encoded strings can't be modified like regular strings. While it > would be possible to implement all operations, such implementation would > become the next SortedCollection (bad performance due to misuse). > Well, UTF-8 strings would have different performance tradeoffs than our existing string classes. Random-access would be expensive, in-place modification would be sometimes expensive, memory usage for non-English strings would be lower, encoding/decoding for IO would be eliminated. I find that's a good fit to some of my uses of strings, and don't mind thinking about the tradeoffs. YMMV. One I idea I've wondered about in the past is having classes instead of language tags. EnglishString, RomainianString etc, with encodings that make sense for the language. That would do a lot for m17n, without going for the full complexity of Unicode. It could also co-exist well with Utf8String, Utf16String etc, since those coudl be considered pseudo-languages/encodings. The downside would be that multi-lingual strings would be more difficult - you'd need ropes or the like. Colin -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151205/bc583b59/attachment.htm From btc at openinworld.com Sun Dec 6 06:42:49 2015 From: btc at openinworld.com (Ben Coman) Date: Sun Dec 6 06:43:13 2015 Subject: [squeak-dev] Re: [Pharo-users] evolutions of squeakelib & crypto (Reed Solomon) In-Reply-To: <5663A08C.8030602@gmail.com> References: <565DCE53.90501@gmail.com> <565DD073.1060409@gmail.com> <565FF8ED.1030606@free.fr> <5660BCFA.4050107@gmail.com> <56616E9C.6040201@gmail.com> <56616F27.4070002@gmail.com> <566174E9.9020001@gmail.com> <566175EA.2090107@gmail.com> <60B913A3-565C-4924-9F82-4E0E96553B72@gmail.com> <5661A6A5.9030902@gmail.com> <224A3FD6-A8DA-45E3-8EDE-AA52133940CC@stfx.eu> <5661B7F1.1080701@gmail.com> <5663A08C.8030602@gmail.com> Message-ID: On Sun, Dec 6, 2015 at 10:42 AM, Robert Withers wrote: > > On 12/05/2015 09:24 PM, Ben Coman wrote: >> >> On Fri, Dec 4, 2015 at 11:57 PM, Robert Withers >> wrote: >>> >>> Now I think you are right on with your observation. Additionally, the >>> number >>> of dialects could increase further with Fuel serialization, just port >>> SecureSession and bits. >>> >>> Alright, I came up with a name and it may border on the egregious ... >>> presenting ... >>> >>> "Maelstrom" >> >> Great sounding name. However some general advice for the community, >> since I see a lot of great sounding project names drowned out in the >> noise of our web-search-centric universe. A litmus test for project >> naming is using google search to find which return low search results. >> Today, its more important to be unique than any other attribute of a >> name. So in general, *dictionary* english words are not the best. >> One technique is to intentionally mispell the word you like. Here are >> some comparative examples (note, the surrounding quotes are required >> to avoid google trying to be helpful and correct the spelling)... >> >> "maelstrom" --> 7,480,000 >> "maelstroom" --> 6,200 >> "maelstrum" --> 2,280 >> "maelstruum" --> 7 >> >> Lots of interesting other techniques can be found by searching on: >> techniques to generate brand names or domain names. >> >> cheers -ben > > > I would be happy to change the names to something more unique, though it may > take a few. Are you suggesting "maelstruum"? > > cheers, > Robert > > *Suggesting* yes, but the choice is yours ;) You need to own it. I think maelstruum is certainly memorable with the double "u", but maybe jarring next the the "m". I'm inclined to maelstroom, since I associate it with "zoom". I wouldn't necessarily go for the absolute lowest results. I have an entirely unsubstantiated belief that anything less than 10,000 gives a reasonable chance to compete once a user's browsing history is taken into account. Finally you need to check existing results don't return something abhorrent (I didn't do this). I'd encourage to play around testing on google search. Its quick and easy to generate and test alternatives. I've added a few more below. "maelstra" --> 3,560 "maelstram" --> 504 "maelstrim" --> 1200 "maelstroon" --> 58 "maelstroomi" --> 4 btw, I wouldn't swap the order of the "ae" since that would be susceptible to real typing errors. cheers -ben From robert.w.withers at gmail.com Sun Dec 6 06:55:22 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Sun Dec 6 06:55:27 2015 Subject: [squeak-dev] Re: [Pharo-users] evolutions of squeakelib & crypto (Reed Solomon) In-Reply-To: References: <565DCE53.90501@gmail.com> <565DD073.1060409@gmail.com> <565FF8ED.1030606@free.fr> <5660BCFA.4050107@gmail.com> <56616E9C.6040201@gmail.com> <56616F27.4070002@gmail.com> <566174E9.9020001@gmail.com> <566175EA.2090107@gmail.com> <60B913A3-565C-4924-9F82-4E0E96553B72@gmail.com> <5661A6A5.9030902@gmail.com> <224A3FD6-A8DA-45E3-8EDE-AA52133940CC@stfx.eu> <5661B7F1.1080701@gmail.com> <5663A08C.8030602@gmail.com> Message-ID: <5663DBDA.9080501@gmail.com> Why is it that naming turns out to be the most challenging and also rewarding of activities at times? Lot's of ROI in a good name. It's strange, I'll have to think more about this naming activity and arrive at a good metaname. I think another good possibility you brought to my mind is "mushroom"...it's growing on me? :) On 12/06/2015 01:42 AM, Ben Coman wrote: > On Sun, Dec 6, 2015 at 10:42 AM, Robert Withers > wrote: >> On 12/05/2015 09:24 PM, Ben Coman wrote: >>> On Fri, Dec 4, 2015 at 11:57 PM, Robert Withers >>> wrote: >>>> Now I think you are right on with your observation. Additionally, the >>>> number >>>> of dialects could increase further with Fuel serialization, just port >>>> SecureSession and bits. >>>> >>>> Alright, I came up with a name and it may border on the egregious ... >>>> presenting ... >>>> >>>> "Maelstrom" >>> Great sounding name. However some general advice for the community, >>> since I see a lot of great sounding project names drowned out in the >>> noise of our web-search-centric universe. A litmus test for project >>> naming is using google search to find which return low search results. >>> Today, its more important to be unique than any other attribute of a >>> name. So in general, *dictionary* english words are not the best. >>> One technique is to intentionally mispell the word you like. Here are >>> some comparative examples (note, the surrounding quotes are required >>> to avoid google trying to be helpful and correct the spelling)... >>> >>> "maelstrom" --> 7,480,000 >>> "maelstroom" --> 6,200 >>> "maelstrum" --> 2,280 >>> "maelstruum" --> 7 >>> >>> Lots of interesting other techniques can be found by searching on: >>> techniques to generate brand names or domain names. >>> >>> cheers -ben >> >> I would be happy to change the names to something more unique, though it may >> take a few. Are you suggesting "maelstruum"? >> >> cheers, >> Robert >> >> > *Suggesting* yes, but the choice is yours ;) You need to own it. > > I think maelstruum is certainly memorable with the double "u", but > maybe jarring next the the "m". I'm inclined to maelstroom, since I > associate it with "zoom". I wouldn't necessarily go for the absolute > lowest results. I have an entirely unsubstantiated belief that > anything less than 10,000 gives a reasonable chance to compete once a > user's browsing history is taken into account. Finally you need to > check existing results don't return something abhorrent (I didn't do > this). > > I'd encourage to play around testing on google search. Its quick and > easy to generate and test alternatives. I've added a few more below. > "maelstra" --> 3,560 > "maelstram" --> 504 > "maelstrim" --> 1200 > "maelstroon" --> 58 > "maelstroomi" --> 4 > > btw, I wouldn't swap the order of the "ae" since that would be > susceptible to real typing errors. > > cheers -ben > From nicolas.cellier.aka.nice at gmail.com Sun Dec 6 10:05:09 2015 From: nicolas.cellier.aka.nice at gmail.com (Nicolas Cellier) Date: Sun Dec 6 10:05:13 2015 Subject: [squeak-dev] java to squeak conversation In-Reply-To: <5663AE51.1040103@gmail.com> References: <5663AE51.1040103@gmail.com> Message-ID: Hi, ((term & 0x1) ==0) just test if least significant bit is unset ? (term | 1) will set the LSB to 1 if unset : (term & ~1) will unset the LSB if set so, all it does is (term bitXOr: 1), that is it changes the LSB of term... The name termPlus1 is misleading. 2015-12-06 4:41 GMT+01:00 Robert Withers : > I still have some rot in my head, from too much java, and so I don't think > consistently clearly, it's more cyclic I've noticed. As a result, my eyes > are doing back-flips when looking at this code; perhaps I need eine kleine > pause...does anyone else want to try it? > > int termPlus1 = ((term & 0x1) == 0) ? (term | 1) : (term & ~1); > > cheers, > Robert > > ps. an example of Java conciseness, dear God. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151206/88858fb8/attachment.htm From robert.w.withers at gmail.com Sun Dec 6 11:24:37 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Sun Dec 6 11:24:44 2015 Subject: [squeak-dev] SqueakElib project In-Reply-To: References: <5661959B.3060804@gmail.com> Message-ID: <56641AF5.8070602@gmail.com> Please ask for any details or clarifications you would like added to this diagram of the SecureSession protocol stack. Ty, Robert. On 12/05/2015 08:34 PM, Ben Coman wrote: > This is interesting, "error correction through Reed Solomon encoding" > but its not clear how its used. Maybe a sketch of the architecture > would be useful. > cheers -ben > > On Fri, Dec 4, 2015 at 9:31 PM, Robert Withers > wrote: >> Good morning, >> >> Here is a brief writeup of the SqueakElib project and its direction. >> >> https://github.com/SqueakCryptographySquad/Cryptography-Common/wiki/SqueakElib. >> >> All new squad members welcome. >> >> Regards, >> Robert >> -------------- next part -------------- Skipped content of type multipart/related From btc at openinworld.com Sun Dec 6 12:18:52 2015 From: btc at openinworld.com (Ben Coman) Date: Sun Dec 6 12:19:15 2015 Subject: [squeak-dev] SqueakElib project In-Reply-To: <56641AF5.8070602@gmail.com> References: <5661959B.3060804@gmail.com> <56641AF5.8070602@gmail.com> Message-ID: Which layer does the Reed Solomon encoding? cheers -ben On Sun, Dec 6, 2015 at 7:24 PM, Robert Withers wrote: > Please ask for any details or clarifications you would like added to this > diagram of the SecureSession protocol stack. Ty, Robert. > > > On 12/05/2015 08:34 PM, Ben Coman wrote: > > This is interesting, "error correction through Reed Solomon encoding" > but its not clear how its used. Maybe a sketch of the architecture > would be useful. > cheers -ben > > On Fri, Dec 4, 2015 at 9:31 PM, Robert Withers wrote: > > Good morning, > > Here is a brief writeup of the SqueakElib project and its direction. > https://github.com/SqueakCryptographySquad/Cryptography-Common/wiki/SqueakElib. > > All new squad members welcome. > > Regards, > Robert > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151206/01ab6c58/attachment.htm From robert.w.withers at gmail.com Sun Dec 6 13:20:51 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Sun Dec 6 13:20:57 2015 Subject: [squeak-dev] SqueakElib project In-Reply-To: References: <5661959B.3060804@gmail.com> <56641AF5.8070602@gmail.com> Message-ID: <56643633.7010806@gmail.com> That is the FEC for Forward Error Correction. Robert On 12/06/2015 07:18 AM, Ben Coman wrote: > Which layer does the Reed Solomon encoding? > cheers -ben > > On Sun, Dec 6, 2015 at 7:24 PM, Robert Withers > > wrote: > > Please ask for any details or clarifications you would like added > to this diagram of the SecureSession protocol stack. Ty, Robert. > > > On 12/05/2015 08:34 PM, Ben Coman wrote: >> This is interesting, "error correction through Reed Solomon encoding" >> but its not clear how its used. Maybe a sketch of the architecture >> would be useful. >> cheers -ben >> >> On Fri, Dec 4, 2015 at 9:31 PM, Robert Withers >> wrote: >>> Good morning, >>> >>> Here is a brief writeup of the SqueakElib project and its direction. >>> >>> https://github.com/SqueakCryptographySquad/Cryptography-Common/wiki/SqueakElib. >>> >>> All new squad members welcome. >>> >>> Regards, >>> Robert >>> > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151206/82f06130/attachment.htm From hannes.hirzel at gmail.com Sun Dec 6 15:23:58 2015 From: hannes.hirzel at gmail.com (H. Hirzel) Date: Sun Dec 6 15:24:01 2015 Subject: [squeak-dev] Unicode Support In-Reply-To: References: Message-ID: On 12/6/15, Levente Uzonyi wrote: > On Sat, 5 Dec 2015, Colin Putney wrote: > >> First, what's UTF-32? Second, we have the whole language tag thing that >> nobody else uses. > > In Squeak, Strings use UTF-32 encoding[1]. It's straightforward > to see for WideString, but ByteString is just a subset of WideString, so > it uses the same encoding. We also use language tags, but that's a > different story. > Language tags make it possible to work around the problems introduced by > the Han unification[2]. We shouldn't really use them for non-CJKV > languages. > >> >> Finally, UTF-8 is a great encoding that certain kinds of applications >> really ought to use. Web apps, in particular, benefit from using UTF-8 so >> the don't have to decode and then re-encode strings coming in from the >> network. In DabbleDB we used UTF-8 encoded string in the image, and just >> ignored the fact that they were displayed incorrectly by inspectors. >> Having a proper UTF-8 string class would be useful. > > We do the same thing, but that doesn't mean it's a good idea to create a > new String-like class having its content encoded in UTF-8, because > UTF-8-encoded strings can't be modified like regular strings. While it > would be possible to implement all operations, such implementation would > become the next SortedCollection (bad performance due to misuse). This is not the case if you go for ropes https://github.com/KenDickey/Cuis-Smalltalk-Ropes > > Levente > > [1] https://en.wikipedia.org/wiki/UTF-32 > [2] https://en.wikipedia.org/wiki/Han_unification > >> >> - Colin >> >> >>> On Dec 4, 2015, at 6:46 AM, Levente Uzonyi wrote: >>> >>> Why would you want to have strings with UTF-8 or UTF-16 encoding in the >>> image? >>> What's wrong with the current UTF-32 representation? >>> >>> Levente >>> >>>> On Fri, 4 Dec 2015, EuanM wrote: >>>> >>>> I'm currently groping my way to seeing how feature-complete our >>>> Unicode support is. I am doing this to establish what still needs to >>>> be done to provide full Unicode support. >>>> >>>> This seems to me to be an area where it would be best to write it >>>> once, and then have the same codebase incorporated into the Smalltalks >>>> that most share a common ancestry. >>>> >>>> I am keen to get: equality-testing for strings; sortability for >>>> strings which have ligatures and diacritic characters; and correct >>>> round-tripping of data. >>>> >>>> Call to action: >>>> ========== >>>> >>>> If you have comments on these proposals - such as "but we already have >>>> that facility" or "the reason we do not have these facilities is >>>> because they are dog-slow" - please let me know them. >>>> >>>> If you would like to help out, please let me know. >>>> >>>> If you have Unicode experience and expertise, and would like to be, or >>>> would be willing to be, in the 'council of experts' for this project, >>>> please let me know. >>>> >>>> If you have comments or ideas on anything mentioned in this email >>>> >>>> In the first instance, the initiative's website will be: >>>> http://smalltalk.uk.to/unicode.html >>>> >>>> I have created a SqueakSource.com project called UnicodeSupport >>>> >>>> I want to avoid re-inventing any facilities which already exist. >>>> Except where they prevent us reaching the goals of: >>>> - sortable UTF8 strings >>>> - sortable UTF16 strings >>>> - equivalence testing of 2 UTF8 strings >>>> - equivalence testing of 2 UTF16 strings >>>> - round-tripping UTF8 strings through Smalltalk >>>> - roundtripping UTF16 strings through Smalltalk. >>>> As I understand it, we have limited Unicode support atm. >>>> >>>> Current state of play >>>> =============== >>>> ByteString gets converted to WideString when need is automagically >>>> detected. >>>> >>>> Is there anything else that currently exists? >>>> >>>> Definition of Terms >>>> ============== >>>> A quick definition of terms before I go any further: >>>> >>>> Standard terms from the Unicode standard >>>> =============================== >>>> a compatibility character : an additional encoding of a *normal* >>>> character, for compatibility and round-trip conversion purposes. For >>>> instance, a 1-byte encoding of a Latin character with a diacritic. >>>> >>>> Made-up terms >>>> ============ >>>> a convenience codepoint : a single codepoint which represents an item >>>> that is also encoded as a string of codepoints. >>>> >>>> (I tend to use the terms compatibility character and compatibility >>>> codepoint interchangably. The standard only refers to them as >>>> compatibility characters. However, the standard is determined to >>>> emphasise that characters are abstract and that codepoints are >>>> concrete. So I think it is often more useful and productive to think >>>> of compatibility or convenience codepoints). >>>> >>>> a composed character : a character made up of several codepoints >>>> >>>> Unicode encoding explained >>>> ===================== >>>> A convenience codepoint can therefore be thought of as a code point >>>> used for a character which also has a composed form. >>>> >>>> The way Unicode works is that sometimes you can encode a character in >>>> one byte, sometimes not. Sometimes you can encode it in two bytes, >>>> sometimes not. >>>> >>>> You can therefore have a long stream of ASCII which is single-byte >>>> Unicode. If there is an occasional Cyrillic or Greek character in the >>>> stream, it would be represented either by a compatibility character or >>>> by a multi-byte combination. >>>> >>>> Using compatibility characters can prevent proper sorting and >>>> equivalence testing. >>>> >>>> Using "pure" Unicode, ie. "normal encodings", can cause compatibility >>>> and round-tripping probelms. Although avoiding them can *also* cause >>>> compatibility issues and round-tripping problems. >>>> >>>> Currently my thinking is: >>>> >>>> a Utf8String class >>>> an Ordered collection, with 1 byte characters as the modal element, >>>> but short arrays of wider strings where necessary >>>> a Utf16String class >>>> an Ordered collection, with 2 byte characters as the modal element, >>>> but short arrays of wider strings >>>> beginning with a 2-byte endianness indicator. >>>> >>>> Utf8Strings sometimes need to be sortable, and sometimes need to be >>>> compatible. >>>> >>>> So my thinking is that Utf8String will contain convenience codepoints, >>>> for round-tripping. And where there are multiple convenience >>>> codepoints for a character, that it standardises on one. >>>> >>>> And that there is a Utf8SortableString which uses *only* normal >>>> characters. >>>> >>>> We then need methods to convert between the two. >>>> >>>> aUtf8String asUtf8SortableString >>>> >>>> and >>>> >>>> aUtf8SortableString asUtf8String >>>> >>>> >>>> Sort orders are culture and context dependent - Sweden and Germany >>>> have different sort orders for the same diacritic-ed characters. Some >>>> countries have one order in general usage, and another for specific >>>> usages, such as phone directories (e.g. UK and France) >>>> >>>> Similarly for Utf16 : Utf16String and Utf16SortableString and >>>> conversion methods >>>> >>>> A list of sorted words would be a SortedCollection, and there could be >>>> pre-prepared sortBlocks for them, e.g. frPhoneBookOrder, deOrder, >>>> seOrder, ukOrder, etc >>>> >>>> along the lines of >>>> aListOfWords := SortedCollection sortBlock: deOrder >>>> >>>> If a word is either a Utf8SortableString, or a well-formed Utf8String, >>>> then we can perform equivalence testing on them trivially. >>>> >>>> To make sure a Utf8String is well formed, we would need to have a way >>>> of cleaning up any convenience codepoints which were valid, but which >>>> were for a character which has multiple equally-valid alternative >>>> convenience codepoints, and for which the string currently had the >>>> "wrong" convenience codepoint. (i.e for any character with valid >>>> alternative convenience codepoints, we would choose one to be in the >>>> well-formed Utf8String, and we would need a method for cleaning the >>>> alternative convenience codepoints out of the string, and replacing >>>> them with the chosen approved convenience codepoint. >>>> >>>> aUtf8String cleanUtf8String >>>> >>>> With WideString, a lot of the issues disappear - except >>>> round-tripping(although I'm sure I have seen something recently about >>>> 4-byte strings that also have an additional bit. Which would make >>>> some Unicode characters 5-bytes long.) >>>> >>>> >>>> (I'm starting to zone out now - if I've overlooked anything - obvious, >>>> subtle, or somewhere in between, please let me know) >>>> >>>> Cheers, >>>> Euan >>>> >>>> >>> >> >> > > From hannes.hirzel at gmail.com Sun Dec 6 17:04:47 2015 From: hannes.hirzel at gmail.com (H. Hirzel) Date: Sun Dec 6 17:04:50 2015 Subject: Fwd: [Pharo-dev] [squeak-dev] Unicode Support In-Reply-To: <4D7C739D-4554-4293-B9F5-37F5C79A613D@mac.com> References: <4D7C739D-4554-4293-B9F5-37F5C79A613D@mac.com> Message-ID: ---------- Forwarded message ---------- From: Todd Blanchard Date: Sun, 06 Dec 2015 08:37:12 -0800 Subject: Re: [Pharo-dev] [squeak-dev] Unicode Support To: Pharo Development List (Resent because of bounce notification (email handling in osx is really beginning to annoy me). Sorry if its a dup) I used to worry a lot about strings being indexable. And then I eventually let go of that and realized that it isn't a particularly important property for them to have. I think you will find that UTF8 is generally the most convenient for a lot of things but its a bit like light in that you treat it alternately as a wave or particle depending on what you are trying to do. So goes strings - they can be treated alternately as streams or byte arrays (not character arrays - stop thinking in characters). In practice, this tends to not be a problem since a lot of the times when you want to replace a character or pick out the nth one you are doing something very computerish and the characters you are working with are the single byte (ASCII legacy) variety. You generally know when you can get away with that and when you can't. Otherwise you are most likely doing things that are best dealt with in a streaming paradigm. For most computation, you come to realize you don't generally care how many characters but how much space (bytes) you need to store your chunk of text. Collation is tricky and complicated in unicode in general but it isn't any worse in UTF8 than any other encoding. You are still going to scan each sortable item from front to back to determine its order, regardless. Most of the outside world has settled on UTF8 and any ASCII file is already UTF8 - which is why it ends up being so convenient. Most of our old text handling infrastructure can still handle UTF8 while it tends to choke on wider encodings. -Todd Blanchard > On Dec 6, 2015, at 07:23, H. Hirzel wrote: > >> We do the same thing, but that doesn't mean it's a good idea to create a >> new String-like class having its content encoded in UTF-8, because >> UTF-8-encoded strings can't be modified like regular strings. While it >> would be possible to implement all operations, such implementation would >> become the next SortedCollection (bad performance due to misuse). From eliot.miranda at gmail.com Sun Dec 6 17:26:46 2015 From: eliot.miranda at gmail.com (Eliot Miranda) Date: Sun Dec 6 17:26:51 2015 Subject: [squeak-dev] Re: Mac release broken? ClassBinding>>objectForDataStream In-Reply-To: References: <20151203020034.GA32593@shell.msen.com> Message-ID: <0E950FD7-F024-4680-8BA7-EB83E82DF05D@gmail.com> Hi Colin, AFAIA the Spur VM does provide image segment support ;-). If the system is crashing there's a bug. But the image segment support does differ somewhat from the previous version since Sour does not support stable object enumeration and so relies on the segment load prim answering an array of the loaded objects and the image level code using this to post process the loaded objs rather than nextObject. There were a raft of changes about a month ago that made the image segment tests run successfully. Can you say more about what's broken? _,,,^..^,,,_ (phone) > On Dec 3, 2015, at 10:16 AM, Colin Putney wrote: > > > >> On Thu, Dec 3, 2015 at 9:35 AM, Chris Muller wrote: >> >> I must've missed the conversation; what is the problem with 5.0 that >> Colin's Environments fix won't work? > > It's basically that Spur doesn't support ImageSegments yet. It crashes on when saving a segment, and fails to load one properly (but doesn't crash). > > -Colin > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151206/f6b5d12e/attachment.htm From colin at wiresong.com Sun Dec 6 18:04:37 2015 From: colin at wiresong.com (Colin Putney) Date: Sun Dec 6 18:04:40 2015 Subject: [squeak-dev] Re: Mac release broken? ClassBinding>>objectForDataStream In-Reply-To: <0E950FD7-F024-4680-8BA7-EB83E82DF05D@gmail.com> References: <20151203020034.GA32593@shell.msen.com> <0E950FD7-F024-4680-8BA7-EB83E82DF05D@gmail.com> Message-ID: On Sun, Dec 6, 2015 at 9:26 AM, Eliot Miranda wrote: > Hi Colin, > > AFAIA the Spur VM does provide image segment support ;-). If the > system is crashing there's a bug. But the image segment support does > differ somewhat from the previous version since Sour does not support > stable object enumeration and so relies on the segment load prim answering > an array of the loaded objects and the image level code using this to post > process the loaded objs rather than nextObject. There were a raft of > changes about a month ago that made the image segment tests run > successfully. Can you say more about what's broken? > The crash occurs when saving a project. I don't know what the issue is, but there's a reproducible case here: http://squeakvm.org/~lewis/tmp/crash-image.zip (Thanks to David for rescuing my overly large attachment!) In an apparently separate issue, when loading a project, I get an error, 'Segment has not been becommed into the loaded objects', raised by ImageSegment>>checkAndReportLoadError. The segment inst var is still an instance of WordArrayForSegment, rather than an Array, so it appears the primitive failed. Colin -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151206/acd19c05/attachment.htm From asqueaker at gmail.com Sun Dec 6 19:03:06 2015 From: asqueaker at gmail.com (Chris Muller) Date: Sun Dec 6 19:03:49 2015 Subject: [squeak-dev] Re: [Pharo-users] evolutions of squeakelib & crypto (Reed Solomon) In-Reply-To: <566175EA.2090107@gmail.com> References: <565DCE53.90501@gmail.com> <565DD073.1060409@gmail.com> <565FF8ED.1030606@free.fr> <5660BCFA.4050107@gmail.com> <56616E9C.6040201@gmail.com> <56616F27.4070002@gmail.com> <566174E9.9020001@gmail.com> <566175EA.2090107@gmail.com> Message-ID: On Fri, Dec 4, 2015 at 5:15 AM, Robert Withers wrote: > I just realized that the squeak version uses ReferenceStream while the > Pharo version uses Fuel, so the binary serializations are different and > they won't speak to each other. Any chance that Fuel is ported to Squeak? > > Installer new merge: #fuel -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151206/28e8c967/attachment.htm From commits at source.squeak.org Sun Dec 6 20:21:03 2015 From: commits at source.squeak.org (commits@source.squeak.org) Date: Sun Dec 6 20:21:06 2015 Subject: [squeak-dev] The Trunk: Tools-eem.658.mcz Message-ID: Eliot Miranda uploaded a new version of Tools to project The Trunk: http://source.squeak.org/trunk/Tools-eem.658.mcz ==================== Summary ==================== Name: Tools-eem.658 Author: eem Time: 6 December 2015, 12:20:40.433 pm UUID: 67c801aa-b3d5-4dc7-a52f-c49bbc7e0022 Ancestors: Tools-eem.657 Fix the mssageList one-element-selection regression in Tools-eem.657. =============== Diff against Tools-eem.657 =============== Item was changed: ----- Method: MessageNames>>selectorListIndex: (in category 'selector list') ----- selectorListIndex: anInteger "Set the selectorListIndex as specified, and propagate consequences" | methodClass index | methodClass := currentCompiledMethod ifNotNil: [currentCompiledMethod methodClass]. selectorListIndex := anInteger. self changed: #selectorListIndex. messageList := self computeMessageList. self changed: #messageList. + messageList size > 1 ifTrue: + [methodClass ifNotNil: [index := messageList findFirst: [:methodRef| methodRef actualClass = methodClass]]]. - methodClass ifNotNil: [index := messageList findFirst: [:methodRef| methodRef actualClass = methodClass]]. "If a method of the same class exists, select that, otherwise select the first message if any." self messageListIndex: (index ifNil: [1 min: messageList size])! From commits at source.squeak.org Sun Dec 6 22:55:02 2015 From: commits at source.squeak.org (commits@source.squeak.org) Date: Sun Dec 6 22:55:03 2015 Subject: [squeak-dev] Daily Commit Log Message-ID: <20151206225502.8325.qmail@box4.squeakfoundation.org> Changes to Trunk (http://source.squeak.org/trunk.html) in the last 24 hours: http://lists.squeakfoundation.org/pipermail/packages/2015-December/009209.html Name: Tools-eem.658 Ancestors: Tools-eem.657 Fix the mssageList one-element-selection regression in Tools-eem.657. ============================================= From leves at caesar.elte.hu Mon Dec 7 01:25:03 2015 From: leves at caesar.elte.hu (Levente Uzonyi) Date: Mon Dec 7 01:25:08 2015 Subject: [squeak-dev] Unicode Support In-Reply-To: References: Message-ID: On Sun, 6 Dec 2015, H. Hirzel wrote: > On 12/6/15, Levente Uzonyi wrote: >> On Sat, 5 Dec 2015, Colin Putney wrote: >> >>> First, what's UTF-32? Second, we have the whole language tag thing that >>> nobody else uses. >> >> In Squeak, Strings use UTF-32 encoding[1]. It's straightforward >> to see for WideString, but ByteString is just a subset of WideString, so >> it uses the same encoding. We also use language tags, but that's a >> different story. >> Language tags make it possible to work around the problems introduced by >> the Han unification[2]. We shouldn't really use them for non-CJKV >> languages. >> >>> >>> Finally, UTF-8 is a great encoding that certain kinds of applications >>> really ought to use. Web apps, in particular, benefit from using UTF-8 so >>> the don't have to decode and then re-encode strings coming in from the >>> network. In DabbleDB we used UTF-8 encoded string in the image, and just >>> ignored the fact that they were displayed incorrectly by inspectors. >>> Having a proper UTF-8 string class would be useful. >> >> We do the same thing, but that doesn't mean it's a good idea to create a >> new String-like class having its content encoded in UTF-8, because >> UTF-8-encoded strings can't be modified like regular strings. While it >> would be possible to implement all operations, such implementation would >> become the next SortedCollection (bad performance due to misuse). > > > This is not the case if you go for ropes > > https://github.com/KenDickey/Cuis-Smalltalk-Ropes Ropes are nice, but they are not strings. Levente > >> >> Levente >> >> [1] https://en.wikipedia.org/wiki/UTF-32 >> [2] https://en.wikipedia.org/wiki/Han_unification >> >>> >>> - Colin >>> >>> >>>> On Dec 4, 2015, at 6:46 AM, Levente Uzonyi wrote: >>>> >>>> Why would you want to have strings with UTF-8 or UTF-16 encoding in the >>>> image? >>>> What's wrong with the current UTF-32 representation? >>>> >>>> Levente >>>> >>>>> On Fri, 4 Dec 2015, EuanM wrote: >>>>> >>>>> I'm currently groping my way to seeing how feature-complete our >>>>> Unicode support is. I am doing this to establish what still needs to >>>>> be done to provide full Unicode support. >>>>> >>>>> This seems to me to be an area where it would be best to write it >>>>> once, and then have the same codebase incorporated into the Smalltalks >>>>> that most share a common ancestry. >>>>> >>>>> I am keen to get: equality-testing for strings; sortability for >>>>> strings which have ligatures and diacritic characters; and correct >>>>> round-tripping of data. >>>>> >>>>> Call to action: >>>>> ========== >>>>> >>>>> If you have comments on these proposals - such as "but we already have >>>>> that facility" or "the reason we do not have these facilities is >>>>> because they are dog-slow" - please let me know them. >>>>> >>>>> If you would like to help out, please let me know. >>>>> >>>>> If you have Unicode experience and expertise, and would like to be, or >>>>> would be willing to be, in the 'council of experts' for this project, >>>>> please let me know. >>>>> >>>>> If you have comments or ideas on anything mentioned in this email >>>>> >>>>> In the first instance, the initiative's website will be: >>>>> http://smalltalk.uk.to/unicode.html >>>>> >>>>> I have created a SqueakSource.com project called UnicodeSupport >>>>> >>>>> I want to avoid re-inventing any facilities which already exist. >>>>> Except where they prevent us reaching the goals of: >>>>> - sortable UTF8 strings >>>>> - sortable UTF16 strings >>>>> - equivalence testing of 2 UTF8 strings >>>>> - equivalence testing of 2 UTF16 strings >>>>> - round-tripping UTF8 strings through Smalltalk >>>>> - roundtripping UTF16 strings through Smalltalk. >>>>> As I understand it, we have limited Unicode support atm. >>>>> >>>>> Current state of play >>>>> =============== >>>>> ByteString gets converted to WideString when need is automagically >>>>> detected. >>>>> >>>>> Is there anything else that currently exists? >>>>> >>>>> Definition of Terms >>>>> ============== >>>>> A quick definition of terms before I go any further: >>>>> >>>>> Standard terms from the Unicode standard >>>>> =============================== >>>>> a compatibility character : an additional encoding of a *normal* >>>>> character, for compatibility and round-trip conversion purposes. For >>>>> instance, a 1-byte encoding of a Latin character with a diacritic. >>>>> >>>>> Made-up terms >>>>> ============ >>>>> a convenience codepoint : a single codepoint which represents an item >>>>> that is also encoded as a string of codepoints. >>>>> >>>>> (I tend to use the terms compatibility character and compatibility >>>>> codepoint interchangably. The standard only refers to them as >>>>> compatibility characters. However, the standard is determined to >>>>> emphasise that characters are abstract and that codepoints are >>>>> concrete. So I think it is often more useful and productive to think >>>>> of compatibility or convenience codepoints). >>>>> >>>>> a composed character : a character made up of several codepoints >>>>> >>>>> Unicode encoding explained >>>>> ===================== >>>>> A convenience codepoint can therefore be thought of as a code point >>>>> used for a character which also has a composed form. >>>>> >>>>> The way Unicode works is that sometimes you can encode a character in >>>>> one byte, sometimes not. Sometimes you can encode it in two bytes, >>>>> sometimes not. >>>>> >>>>> You can therefore have a long stream of ASCII which is single-byte >>>>> Unicode. If there is an occasional Cyrillic or Greek character in the >>>>> stream, it would be represented either by a compatibility character or >>>>> by a multi-byte combination. >>>>> >>>>> Using compatibility characters can prevent proper sorting and >>>>> equivalence testing. >>>>> >>>>> Using "pure" Unicode, ie. "normal encodings", can cause compatibility >>>>> and round-tripping probelms. Although avoiding them can *also* cause >>>>> compatibility issues and round-tripping problems. >>>>> >>>>> Currently my thinking is: >>>>> >>>>> a Utf8String class >>>>> an Ordered collection, with 1 byte characters as the modal element, >>>>> but short arrays of wider strings where necessary >>>>> a Utf16String class >>>>> an Ordered collection, with 2 byte characters as the modal element, >>>>> but short arrays of wider strings >>>>> beginning with a 2-byte endianness indicator. >>>>> >>>>> Utf8Strings sometimes need to be sortable, and sometimes need to be >>>>> compatible. >>>>> >>>>> So my thinking is that Utf8String will contain convenience codepoints, >>>>> for round-tripping. And where there are multiple convenience >>>>> codepoints for a character, that it standardises on one. >>>>> >>>>> And that there is a Utf8SortableString which uses *only* normal >>>>> characters. >>>>> >>>>> We then need methods to convert between the two. >>>>> >>>>> aUtf8String asUtf8SortableString >>>>> >>>>> and >>>>> >>>>> aUtf8SortableString asUtf8String >>>>> >>>>> >>>>> Sort orders are culture and context dependent - Sweden and Germany >>>>> have different sort orders for the same diacritic-ed characters. Some >>>>> countries have one order in general usage, and another for specific >>>>> usages, such as phone directories (e.g. UK and France) >>>>> >>>>> Similarly for Utf16 : Utf16String and Utf16SortableString and >>>>> conversion methods >>>>> >>>>> A list of sorted words would be a SortedCollection, and there could be >>>>> pre-prepared sortBlocks for them, e.g. frPhoneBookOrder, deOrder, >>>>> seOrder, ukOrder, etc >>>>> >>>>> along the lines of >>>>> aListOfWords := SortedCollection sortBlock: deOrder >>>>> >>>>> If a word is either a Utf8SortableString, or a well-formed Utf8String, >>>>> then we can perform equivalence testing on them trivially. >>>>> >>>>> To make sure a Utf8String is well formed, we would need to have a way >>>>> of cleaning up any convenience codepoints which were valid, but which >>>>> were for a character which has multiple equally-valid alternative >>>>> convenience codepoints, and for which the string currently had the >>>>> "wrong" convenience codepoint. (i.e for any character with valid >>>>> alternative convenience codepoints, we would choose one to be in the >>>>> well-formed Utf8String, and we would need a method for cleaning the >>>>> alternative convenience codepoints out of the string, and replacing >>>>> them with the chosen approved convenience codepoint. >>>>> >>>>> aUtf8String cleanUtf8String >>>>> >>>>> With WideString, a lot of the issues disappear - except >>>>> round-tripping(although I'm sure I have seen something recently about >>>>> 4-byte strings that also have an additional bit. Which would make >>>>> some Unicode characters 5-bytes long.) >>>>> >>>>> >>>>> (I'm starting to zone out now - if I've overlooked anything - obvious, >>>>> subtle, or somewhere in between, please let me know) >>>>> >>>>> Cheers, >>>>> Euan >>>>> >>>>> >>>> >>> >>> >> >> > > From robert.w.withers at gmail.com Mon Dec 7 02:13:13 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Mon Dec 7 02:13:20 2015 Subject: [squeak-dev] Re: [Pharo-users] evolutions of squeakelib & crypto (Reed Solomon) In-Reply-To: References: <565DCE53.90501@gmail.com> <565DD073.1060409@gmail.com> <565FF8ED.1030606@free.fr> <5660BCFA.4050107@gmail.com> <56616E9C.6040201@gmail.com> <56616F27.4070002@gmail.com> <566174E9.9020001@gmail.com> <566175EA.2090107@gmail.com> Message-ID: <5664EB39.6030308@gmail.com> Thanks, Chris! I'll tuck this away until I get back to my presentation layer. On 12/06/2015 02:03 PM, Chris Muller wrote: > On Fri, Dec 4, 2015 at 5:15 AM, Robert Withers > > wrote: > > I just realized that the squeak version uses ReferenceStream while > the Pharo version uses Fuel, so the binary serializations are > different and they won't speak to each other. Any chance that Fuel > is ported to Squeak? > > > Installer new merge: #fuel > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151206/0e57b856/attachment.htm From euanmee at gmail.com Mon Dec 7 03:21:29 2015 From: euanmee at gmail.com (EuanM) Date: Mon Dec 7 03:21:31 2015 Subject: [squeak-dev] Re: [Pharo-dev] Unicode Support In-Reply-To: <5662E1D8.1070802@free.fr> References: <5662E1D8.1070802@free.fr> Message-ID: This a long email. A *lot* of it is encapsulated in the Venn diagram both: http://smalltalk.uk.to/unicode-utf8.html and my Smalltalk in Small Steps blog at: http://smalltalkinsmallsteps.blogspot.co.uk/2015/12/utf-8-for-cuis-pharo-and-squeak.html My current thinking, and understanding. ============================== 0) a) ASCII and ISO-8859-1 consist of characters encoded in 1 byte. b) UTF-8 can encode all of those characters in 1 byte, but can prefer some of them to be encoded as sequences of multiple bytes. And can encode additional characters as sequences of multiple bytes. 1) Smalltalk has long had multiple String classes. 2) Any UTF16 Unicode codepoint which has a codepoint of 00nn hex is encoded as a UTF-8 codepoint of nn hex. 3) All valid ISO-8859-1 characters have a character code between 20 hex and 7E hex, or between A0 hex and FF hex. https://en.wikipedia.org/wiki/ISO/IEC_8859-1 4) All valid ASCII characters have a character code between 00 hex and 7E hex. https://en.wikipedia.org/wiki/ASCII 5) a) All character codes which are defined within ISO-8859-1 and also defined within ASCII. (i.e. character codes 20 hex to 7E hex) are defined identically in both. b) All printable ASCII characters are defined identically in both ASCII and ISO-8859-1 6) All character codes defined in ASCII (00 hex to 7E hex) are defined identically in Unicode UTF-8. 7) All character codes defined in ISO-8859-1 (20 hex - 7E hex ; A0 hex - FF hex ) are defined identically in UTF-8. 8) => some Unicode codepoints map to both ASCII and ISO-8859-1. all ASCII maps 1:1 to Unicode UTF-8 all ISO-8859-1 maps 1:1 to Unicode UTF-8 9) All ByteStrings elements which are either a valid ISO-8859-1 character or a valid ASCII character are *also* a valid UTF-8 character. 10) ISO-8859-1 characters representing a character with a diacritic, or a two-character ligature, have no ASCII equivalent. In Unicode UTF-8, those character codes which are representing compound glyphs, are called "compatibility codepoints". 11) The preferred Unicode representation of the characters which have compatibility codepoints is as a a short set of codepoints representing the characters which are combined together to form the glyph of the convenience codepoint, as a sequence of bytes representing the component characters. 12) Some concrete examples: A - aka Upper Case A In ASCII, in ISO 8859-1 ASCII A - 41 hex ISO-8859-1 A - 41 hex UTF-8 A - 41 hex BEL (a bell sound, often invoked by a Ctrl-g keyboard chord) In ASCII, not in ISO 8859-1 ASCII : BEL - 07 hex ISO-8859-1 : 07 hex is not a valid character code UTF-8 : BEL - 07 hex ? (GBP currency symbol) In ISO-8859-1, not in ASCII ASCII : A3 hex is not a valid ASCII code UTF-8: ? - A3 hex ISO-8859-1: ? - A3 hex Upper Case C cedilla In ISO-8859-1, not in ASCII, in UTF-8 as a compatibility codepoint *and* a composed set of codepoints ASCII : C7 hex is not a valid ASCII character code ISO-8859-1 : Upper Case C cedilla - C7 hex UTF-8 : Upper Case C cedilla (compatibility codepoint) - C7 hex Unicode preferred Upper Case C cedilla (composed set of codepoints) Upper case C 0043 hex (Upper case C) followed by cedilla 00B8 hex (cedilla) 13) For any valid ASCII string *and* for any valid ISO-8859-1 string, aByteString is completely adequate for editing and display. 14) When sorting any valid ASCII string *or* any valid ISO-8859-1 string, upper and lower case versions of the same character will be treated differently. 15) When sorting any valid ISO-8859-1 string containing letter+diacritic combination glyphs or ligature combination glyphs, the glyphs in combination will treated differently to a "plain" glyph of the character i.e. "C" and "C cedilla" will be treated very differently. "?" and "fs" will be treated very differently. 16) Different nations have different rules about where diacritic-ed characted and ligature pairs should be placed when in alphabetical order. 17) Some nations even have multiple standards - e.g. surnames beginning either "M superscript-c" or "M superscript-a superscript-c" are treated as beginning equivalently in UK phone directories, but not in other situations. Some practical upshots ================== 1) Cuis and its ISO-8859-1 encoding is *exactly* the same as UTF-8, for any single character it considers valid, or any ByteString it has made up of characters it considers valid. 2) Any ByteString is valid UTF-8 in any of Squeak, Pharo, Cuis or any other Smalltalk with a single byte ByteString following ASCII or ISO-8859-1. 3) Any Smalltalk (or derivative language) using ByteString can immediately consider it's ByteString as valid UTF-8, as long as it also considers the ByteSring as valid ASCII and/or ISO-8859-1. 4) All of those can be successfully exported to any system using UTF-8 (e.g. HTML). 5) To successfully *accept* all UTF-8 we much be able to do either: a) accept UTF-8 strings with composed characters b) convert UTF-8 strings with composed characters into UTF-8 strings that use *only* compatibility codepoints. Class + protocol proposals a Utf8CompatibilityString class. asByteString - ensure only compatibility codepoints are used. Ensure it doews not encode characters above 00FF hex. asIso8859String - ensures only compatibility codepoints are used, and that the characters are each valid ISO 8859-1 asAsciiString - ensures only characters 00hex - 7F hex are used. asUtf8ComposedIso8859String - ensures all compatibility codepoints are expanded into small OrderedCollections of codepoints a Utf8ComposedIso8859String class - will provide sortable and comparable UTF8 strings of all ASCII and ISO 8859-1 strings. Then a Utf8SortableCollection class - a collection of Utf8ComposedIso8859Strings words and phrases. Custom sortBlocks will define the applicable sort order. We can create a collection... a Dictionary, thinking about it, of named, prefabricated sortBlocks. This will work for all UTF8 strings of ISO-8859-1 and ASCII strings. If anyone has better names for the classes, please let me know. If anyone else wants to help - build these, - create SUnit tests for these - write documentation for these Please let me know. n.b. I have had absolutely no experience of Ropes. My own background with this stuff: In the early 90's as a Project Manager implementing office automation systems across a global company, with offices in the Americas, Western, Eastern and Central Europe, (including Slavic and Cyrillic users) nations, Japan and China. The mission-critical application was word-processing. Our offices were spread around the globe, and we needed those offices to successfully exchange documents with their sister offices, and with the customers in each region the offices were in. Unicode was then new, and our platform supplier was the NeXT Corporation, who had been founder members in of the Unicode Consortium in 1990. So far: I've read the latest version of the Unicode Standard (v8.0). This is freely downloadable. I've purchased a paper copy of an earlier release. New releases typically consist additional codespaces (i.e. alphabets). So old copies are useful, as well as cheap. (Paper copies of version 4.0 are available second-hand for < $10 / ?10). The typical change with each release is the addition of further codespaces (i.e alphabets (more or less) ), so you don't lose a lot. (I'll be going through my V4.0 just to make sure) Cheers, Euan On 5 December 2015 at 13:08, stepharo wrote: > Hi EuanM > > Le 4/12/15 12:42, EuanM a ?crit : >> >> I'm currently groping my way to seeing how feature-complete our >> Unicode support is. I am doing this to establish what still needs to >> be done to provide full Unicode support. > > > this is great. Thanks for pushing this. I wrote and collected some roadmap > (analyses on different topics) > on the pharo github project feel free to add this one there. >> >> >> This seems to me to be an area where it would be best to write it >> once, and then have the same codebase incorporated into the Smalltalks >> that most share a common ancestry. >> >> I am keen to get: equality-testing for strings; sortability for >> strings which have ligatures and diacritic characters; and correct >> round-tripping of data. > > Go! > My suggestion is > start small > make steady progress > write tests > commit often :) > > Stef > > What is the french phoneBook ordering because this is the first time I hear > about it. > >> >> Call to action: >> ========== >> >> If you have comments on these proposals - such as "but we already have >> that facility" or "the reason we do not have these facilities is >> because they are dog-slow" - please let me know them. >> >> If you would like to help out, please let me know. >> >> If you have Unicode experience and expertise, and would like to be, or >> would be willing to be, in the 'council of experts' for this project, >> please let me know. >> >> If you have comments or ideas on anything mentioned in this email >> >> In the first instance, the initiative's website will be: >> http://smalltalk.uk.to/unicode.html >> >> I have created a SqueakSource.com project called UnicodeSupport >> >> I want to avoid re-inventing any facilities which already exist. >> Except where they prevent us reaching the goals of: >> - sortable UTF8 strings >> - sortable UTF16 strings >> - equivalence testing of 2 UTF8 strings >> - equivalence testing of 2 UTF16 strings >> - round-tripping UTF8 strings through Smalltalk >> - roundtripping UTF16 strings through Smalltalk. >> As I understand it, we have limited Unicode support atm. >> >> Current state of play >> =============== >> ByteString gets converted to WideString when need is automagically >> detected. >> >> Is there anything else that currently exists? >> >> Definition of Terms >> ============== >> A quick definition of terms before I go any further: >> >> Standard terms from the Unicode standard >> =============================== >> a compatibility character : an additional encoding of a *normal* >> character, for compatibility and round-trip conversion purposes. For >> instance, a 1-byte encoding of a Latin character with a diacritic. >> >> Made-up terms >> ============ >> a convenience codepoint : a single codepoint which represents an item >> that is also encoded as a string of codepoints. >> >> (I tend to use the terms compatibility character and compatibility >> codepoint interchangably. The standard only refers to them as >> compatibility characters. However, the standard is determined to >> emphasise that characters are abstract and that codepoints are >> concrete. So I think it is often more useful and productive to think >> of compatibility or convenience codepoints). >> >> a composed character : a character made up of several codepoints >> >> Unicode encoding explained >> ===================== >> A convenience codepoint can therefore be thought of as a code point >> used for a character which also has a composed form. >> >> The way Unicode works is that sometimes you can encode a character in >> one byte, sometimes not. Sometimes you can encode it in two bytes, >> sometimes not. >> >> You can therefore have a long stream of ASCII which is single-byte >> Unicode. If there is an occasional Cyrillic or Greek character in the >> stream, it would be represented either by a compatibility character or >> by a multi-byte combination. >> >> Using compatibility characters can prevent proper sorting and >> equivalence testing. >> >> Using "pure" Unicode, ie. "normal encodings", can cause compatibility >> and round-tripping probelms. Although avoiding them can *also* cause >> compatibility issues and round-tripping problems. >> >> Currently my thinking is: >> >> a Utf8String class >> an Ordered collection, with 1 byte characters as the modal element, >> but short arrays of wider strings where necessary >> a Utf16String class >> an Ordered collection, with 2 byte characters as the modal element, >> but short arrays of wider strings >> beginning with a 2-byte endianness indicator. >> >> Utf8Strings sometimes need to be sortable, and sometimes need to be >> compatible. >> >> So my thinking is that Utf8String will contain convenience codepoints, >> for round-tripping. And where there are multiple convenience >> codepoints for a character, that it standardises on one. >> >> And that there is a Utf8SortableString which uses *only* normal >> characters. >> >> We then need methods to convert between the two. >> >> aUtf8String asUtf8SortableString >> >> and >> >> aUtf8SortableString asUtf8String >> >> >> Sort orders are culture and context dependent - Sweden and Germany >> have different sort orders for the same diacritic-ed characters. Some >> countries have one order in general usage, and another for specific >> usages, such as phone directories (e.g. UK and France) >> >> Similarly for Utf16 : Utf16String and Utf16SortableString and >> conversion methods >> >> A list of sorted words would be a SortedCollection, and there could be >> pre-prepared sortBlocks for them, e.g. frPhoneBookOrder, deOrder, >> seOrder, ukOrder, etc >> >> along the lines of >> aListOfWords := SortedCollection sortBlock: deOrder >> >> If a word is either a Utf8SortableString, or a well-formed Utf8String, >> then we can perform equivalence testing on them trivially. >> >> To make sure a Utf8String is well formed, we would need to have a way >> of cleaning up any convenience codepoints which were valid, but which >> were for a character which has multiple equally-valid alternative >> convenience codepoints, and for which the string currently had the >> "wrong" convenience codepoint. (i.e for any character with valid >> alternative convenience codepoints, we would choose one to be in the >> well-formed Utf8String, and we would need a method for cleaning the >> alternative convenience codepoints out of the string, and replacing >> them with the chosen approved convenience codepoint. >> >> aUtf8String cleanUtf8String >> >> With WideString, a lot of the issues disappear - except >> round-tripping(although I'm sure I have seen something recently about >> 4-byte strings that also have an additional bit. Which would make >> some Unicode characters 5-bytes long.) >> >> >> (I'm starting to zone out now - if I've overlooked anything - obvious, >> subtle, or somewhere in between, please let me know) >> >> Cheers, >> Euan >> >> > > From euanmee at gmail.com Mon Dec 7 03:31:27 2015 From: euanmee at gmail.com (EuanM) Date: Mon Dec 7 03:31:30 2015 Subject: [squeak-dev] Re: [Pharo-dev] Unicode Support In-Reply-To: References: <5662E1D8.1070802@free.fr> <4014E1FA-47F6-4677-9118-869C415BCD74@mac.com> Message-ID: "Canonicalisation and sorting issues are hardly discussed. In one place, the fact that a lot of special characters can have multiple representations is a big argument, while it is not mentioned how just treating things like a byte sequence would solve this (it doesn't AFAIU). Like how do you search for $e or $? if you know that it is possible to represent $? as just $? and as $e + $? ?" This, for me, is one of the chief purposes of Unicode support. What you have it a convertor for "might contain compatibility codepoints" to "contains only composed sequences of codepoints, and no compatibility codepoints". As long as you're not using Strings where you should use Streams, it should be okay. And of course, for passing back to ISO Latin 1 or ASCII systems, you need to have a convertor to "contains only compatibility codepoints, and no composed sets of codepoints". As long as you can tell one type from the other, it's not a problem. Any string that mixes both can be converted in either direction by the same methods which I've just outlined. Once you have these, we can do this for all 1 byte characters. We can then expand this to have Classes and methods for character strings which contain the occasional character from other ISO character sets. Cheers, Euan On 6 December 2015 at 17:44, Sven Van Caekenberghe wrote: > >> On 05 Dec 2015, at 17:35, Todd Blanchard wrote: >> >> would suggest that the only worthwhile encoding is UTF8 - the rest are distractions except for being able to read and convert from other encodings to UTF8. UTF16 is a complete waste of time. >> >> Read http://utf8everywhere.org/ >> >> I have extensive Unicode chops from around 1999 to 2004 and my experience leads me to strongly agree with the views on that site. > > Well, I read the page/document/site as well. It was very interesting indeed, thanks for sharing it. > > In some sense it made me reconsider my aversion against in-image utf-8 encoding, maybe it could have some value. Absolute storage is more efficient, some processing might also be more efficient, i/o conversions to/from utf-8 become a no-op. What I found nice is the suggestion that most structured parsing (XML, JSON, CSV, STON, ...) could actually ignore the encoding for a large part and just assume its ASCII, which would/could be nice for performance. Also the fact that a lot of strings are (or should be) treated as opaque makes a lot of sense. > > What I did not like is that much of argumentation is based on issue in the Windows world, take all that away and the document shrinks in half. I would have liked a bit more fundamental CS arguments. > > Canonicalisation and sorting issues are hardly discussed. > > In one place, the fact that a lot of special characters can have multiple representations is a big argument, while it is not mentioned how just treating things like a byte sequence would solve this (it doesn't AFAIU). Like how do you search for $e or $? if you know that it is possible to represent $? as just $? and as $e + $? ? > > Sven > >> Sent from the road >> >> On Dec 5, 2015, at 05:08, stepharo wrote: >> >>> Hi EuanM >>> >>> Le 4/12/15 12:42, EuanM a ?crit : >>>> I'm currently groping my way to seeing how feature-complete our >>>> Unicode support is. I am doing this to establish what still needs to >>>> be done to provide full Unicode support. >>> >>> this is great. Thanks for pushing this. I wrote and collected some roadmap (analyses on different topics) >>> on the pharo github project feel free to add this one there. >>>> >>>> This seems to me to be an area where it would be best to write it >>>> once, and then have the same codebase incorporated into the Smalltalks >>>> that most share a common ancestry. >>>> >>>> I am keen to get: equality-testing for strings; sortability for >>>> strings which have ligatures and diacritic characters; and correct >>>> round-tripping of data. >>> Go! >>> My suggestion is >>> start small >>> make steady progress >>> write tests >>> commit often :) >>> >>> Stef >>> >>> What is the french phoneBook ordering because this is the first time I hear about it. >>>> >>>> Call to action: >>>> ========== >>>> >>>> If you have comments on these proposals - such as "but we already have >>>> that facility" or "the reason we do not have these facilities is >>>> because they are dog-slow" - please let me know them. >>>> >>>> If you would like to help out, please let me know. >>>> >>>> If you have Unicode experience and expertise, and would like to be, or >>>> would be willing to be, in the 'council of experts' for this project, >>>> please let me know. >>>> >>>> If you have comments or ideas on anything mentioned in this email >>>> >>>> In the first instance, the initiative's website will be: >>>> http://smalltalk.uk.to/unicode.html >>>> >>>> I have created a SqueakSource.com project called UnicodeSupport >>>> >>>> I want to avoid re-inventing any facilities which already exist. >>>> Except where they prevent us reaching the goals of: >>>> - sortable UTF8 strings >>>> - sortable UTF16 strings >>>> - equivalence testing of 2 UTF8 strings >>>> - equivalence testing of 2 UTF16 strings >>>> - round-tripping UTF8 strings through Smalltalk >>>> - roundtripping UTF16 strings through Smalltalk. >>>> As I understand it, we have limited Unicode support atm. >>>> >>>> Current state of play >>>> =============== >>>> ByteString gets converted to WideString when need is automagically detected. >>>> >>>> Is there anything else that currently exists? >>>> >>>> Definition of Terms >>>> ============== >>>> A quick definition of terms before I go any further: >>>> >>>> Standard terms from the Unicode standard >>>> =============================== >>>> a compatibility character : an additional encoding of a *normal* >>>> character, for compatibility and round-trip conversion purposes. For >>>> instance, a 1-byte encoding of a Latin character with a diacritic. >>>> >>>> Made-up terms >>>> ============ >>>> a convenience codepoint : a single codepoint which represents an item >>>> that is also encoded as a string of codepoints. >>>> >>>> (I tend to use the terms compatibility character and compatibility >>>> codepoint interchangably. The standard only refers to them as >>>> compatibility characters. However, the standard is determined to >>>> emphasise that characters are abstract and that codepoints are >>>> concrete. So I think it is often more useful and productive to think >>>> of compatibility or convenience codepoints). >>>> >>>> a composed character : a character made up of several codepoints >>>> >>>> Unicode encoding explained >>>> ===================== >>>> A convenience codepoint can therefore be thought of as a code point >>>> used for a character which also has a composed form. >>>> >>>> The way Unicode works is that sometimes you can encode a character in >>>> one byte, sometimes not. Sometimes you can encode it in two bytes, >>>> sometimes not. >>>> >>>> You can therefore have a long stream of ASCII which is single-byte >>>> Unicode. If there is an occasional Cyrillic or Greek character in the >>>> stream, it would be represented either by a compatibility character or >>>> by a multi-byte combination. >>>> >>>> Using compatibility characters can prevent proper sorting and >>>> equivalence testing. >>>> >>>> Using "pure" Unicode, ie. "normal encodings", can cause compatibility >>>> and round-tripping probelms. Although avoiding them can *also* cause >>>> compatibility issues and round-tripping problems. >>>> >>>> Currently my thinking is: >>>> >>>> a Utf8String class >>>> an Ordered collection, with 1 byte characters as the modal element, >>>> but short arrays of wider strings where necessary >>>> a Utf16String class >>>> an Ordered collection, with 2 byte characters as the modal element, >>>> but short arrays of wider strings >>>> beginning with a 2-byte endianness indicator. >>>> >>>> Utf8Strings sometimes need to be sortable, and sometimes need to be compatible. >>>> >>>> So my thinking is that Utf8String will contain convenience codepoints, >>>> for round-tripping. And where there are multiple convenience >>>> codepoints for a character, that it standardises on one. >>>> >>>> And that there is a Utf8SortableString which uses *only* normal characters. >>>> >>>> We then need methods to convert between the two. >>>> >>>> aUtf8String asUtf8SortableString >>>> >>>> and >>>> >>>> aUtf8SortableString asUtf8String >>>> >>>> >>>> Sort orders are culture and context dependent - Sweden and Germany >>>> have different sort orders for the same diacritic-ed characters. Some >>>> countries have one order in general usage, and another for specific >>>> usages, such as phone directories (e.g. UK and France) >>>> >>>> Similarly for Utf16 : Utf16String and Utf16SortableString and >>>> conversion methods >>>> >>>> A list of sorted words would be a SortedCollection, and there could be >>>> pre-prepared sortBlocks for them, e.g. frPhoneBookOrder, deOrder, >>>> seOrder, ukOrder, etc >>>> >>>> along the lines of >>>> aListOfWords := SortedCollection sortBlock: deOrder >>>> >>>> If a word is either a Utf8SortableString, or a well-formed Utf8String, >>>> then we can perform equivalence testing on them trivially. >>>> >>>> To make sure a Utf8String is well formed, we would need to have a way >>>> of cleaning up any convenience codepoints which were valid, but which >>>> were for a character which has multiple equally-valid alternative >>>> convenience codepoints, and for which the string currently had the >>>> "wrong" convenience codepoint. (i.e for any character with valid >>>> alternative convenience codepoints, we would choose one to be in the >>>> well-formed Utf8String, and we would need a method for cleaning the >>>> alternative convenience codepoints out of the string, and replacing >>>> them with the chosen approved convenience codepoint. >>>> >>>> aUtf8String cleanUtf8String >>>> >>>> With WideString, a lot of the issues disappear - except >>>> round-tripping(although I'm sure I have seen something recently about >>>> 4-byte strings that also have an additional bit. Which would make >>>> some Unicode characters 5-bytes long.) >>>> >>>> >>>> (I'm starting to zone out now - if I've overlooked anything - obvious, >>>> subtle, or somewhere in between, please let me know) >>>> >>>> Cheers, >>>> Euan >>>> >>>> >>> >>> > > From euanmee at gmail.com Mon Dec 7 10:53:08 2015 From: euanmee at gmail.com (EuanM) Date: Mon Dec 7 10:53:17 2015 Subject: [squeak-dev] Re: [Pharo-dev] Unicode Support Message-ID: Verifying assumptions is the key reason why you should documents like this out for review. Sven - Cuis is encoded with ISO 8859-15 (aka ISO Latin 9) Sven, this is *NOT* as you state, ISO 99591, (and not as I stated, 8859-1). We caught the right specification bug for the wrong reason. Juan: "Cuis: Chose not to use Squeak approach. Chose to make the base image include and use only 1-byte strings. Chose to use ISO-8859-15" I have double-checked - each character encoded in ISO Latin 15 (ISO 8859-15) is exactly the character represented by the corresponding 1-byte codepoint in Unicode 0000 to 00FF, with the following exceptions: codepoint 20ac - Euro symbol character code a4 (replaces codepoint 00a4 generic currency symbol) codepoint 0160 Latin Upper Case S with Caron character code a6 (replaces codepoint 00A6 was | Unix pipe character) codepoint 0161 Latin Lower Case s with Caron character code a8 (replaces codepoint 00A8 was dierisis) codepoint 017d Latin Upper Case Z with Caron character code b4 (replaces codepoint 00b4 was Acute accent) codepoint 017e Latin Lower Case Z with Caron character code b8 (replaces codepoint 00b8 was cedilla) codepoint 0152 Upper Case OE ligature = Ethel character code bc (replaces codepoint 00bc was 1/4 symbol) codepoint 0153 Lower Case oe ligature = ethel character code bd (replaces codepoint 00bd was 1/2 symbol) codepoint 0178 Upper Case Y diaeresis character code be (replaces codepoint 00be was 3/4 symbol) Juan - I don't suppose we could persuade you to change to ISO Latin-1 from ISO Latin-9 ? It means we could run the same 1 byte string encoding across Cuis, Squeak, Pharo, and, as far as I can make out so far, Dolphin Smalltalk and Gnu Smalltalk. The downside would be that French Y diaeresis would lose the ability to use that character, along with users of oe, OE, and s, S, z, Z with caron. Along with the Euro. https://en.wikipedia.org/wiki/ISO/IEC_8859-15. I'm confident I understand the use of UTF-8 in principle. On 7 December 2015 at 08:27, Sven Van Caekenberghe wrote: > I am sorry but one of your basic assumptions is completely wrong: > > 'Les ?l?ves Fran?ais' encodeWith: #iso99591. > > => #[76 101 115 32 233 108 232 118 101 115 32 70 114 97 110 231 97 105 115] > > 'Les ?l?ves Fran?ais' utf8Encoded. > > => #[76 101 115 32 195 169 108 195 168 118 101 115 32 70 114 97 110 195 167 97 105 115] > > ISO-9959-1 (~Latin1) is NOT AT ALL identical to UTF-8 in its upper, non-ACII part !! > > Or shorter, $? is encoded in ISO-9959-1 as #[233], but as #[195 169] in UTF-8. > > So more than half the points you make, or the facts that you state, are thus plain wrong. > > The only thing that is correct is that the code points are equal, but that is not the same as the encoding ! > > From this I am inclined to conclude that you do not fundamentally understand how UTF-8 works, which does not strike me as good basis to design something called a UTF8String. > > Sorry. > > PS: Note also that Cuis' choice to use ISO-9959-1 only is pretty limiting in a Unicode world. > >> On 07 Dec 2015, at 04:21, EuanM wrote: >> >> This a long email. A *lot* of it is encapsulated in the Venn diagram both: >> http://smalltalk.uk.to/unicode-utf8.html >> and my Smalltalk in Small Steps blog at: >> http://smalltalkinsmallsteps.blogspot.co.uk/2015/12/utf-8-for-cuis-pharo-and-squeak.html >> >> My current thinking, and understanding. >> ============================== >> >> 0) a) ASCII and ISO-8859-1 consist of characters encoded in 1 byte. >> b) UTF-8 can encode all of those characters in 1 byte, but can >> prefer some of them to be encoded as sequences of multiple bytes. And >> can encode additional characters as sequences of multiple bytes. >> >> 1) Smalltalk has long had multiple String classes. >> >> 2) Any UTF16 Unicode codepoint which has a codepoint of 00nn hex >> is encoded as a UTF-8 codepoint of nn hex. >> >> 3) All valid ISO-8859-1 characters have a character code between 20 >> hex and 7E hex, or between A0 hex and FF hex. >> https://en.wikipedia.org/wiki/ISO/IEC_8859-1 >> >> 4) All valid ASCII characters have a character code between 00 hex and 7E hex. >> https://en.wikipedia.org/wiki/ASCII >> >> >> 5) a) All character codes which are defined within ISO-8859-1 and also >> defined within ASCII. (i.e. character codes 20 hex to 7E hex) are >> defined identically in both. >> >> b) All printable ASCII characters are defined identically in both >> ASCII and ISO-8859-1 >> >> 6) All character codes defined in ASCII (00 hex to 7E hex) are >> defined identically in Unicode UTF-8. >> >> 7) All character codes defined in ISO-8859-1 (20 hex - 7E hex ; A0 hex >> - FF hex ) are defined identically in UTF-8. >> >> 8) => some Unicode codepoints map to both ASCII and ISO-8859-1. >> all ASCII maps 1:1 to Unicode UTF-8 >> all ISO-8859-1 maps 1:1 to Unicode UTF-8 >> >> 9) All ByteStrings elements which are either a valid ISO-8859-1 >> character or a valid ASCII character are *also* a valid UTF-8 >> character. >> >> 10) ISO-8859-1 characters representing a character with a diacritic, >> or a two-character ligature, have no ASCII equivalent. In Unicode >> UTF-8, those character codes which are representing compound glyphs, >> are called "compatibility codepoints". >> >> 11) The preferred Unicode representation of the characters which have >> compatibility codepoints is as a a short set of codepoints >> representing the characters which are combined together to form the >> glyph of the convenience codepoint, as a sequence of bytes >> representing the component characters. >> >> >> 12) Some concrete examples: >> >> A - aka Upper Case A >> In ASCII, in ISO 8859-1 >> ASCII A - 41 hex >> ISO-8859-1 A - 41 hex >> UTF-8 A - 41 hex >> >> BEL (a bell sound, often invoked by a Ctrl-g keyboard chord) >> In ASCII, not in ISO 8859-1 >> ASCII : BEL - 07 hex >> ISO-8859-1 : 07 hex is not a valid character code >> UTF-8 : BEL - 07 hex >> >> ? (GBP currency symbol) >> In ISO-8859-1, not in ASCII >> ASCII : A3 hex is not a valid ASCII code >> UTF-8: ? - A3 hex >> ISO-8859-1: ? - A3 hex >> >> Upper Case C cedilla >> In ISO-8859-1, not in ASCII, in UTF-8 as a compatibility codepoint >> *and* a composed set of codepoints >> ASCII : C7 hex is not a valid ASCII character code >> ISO-8859-1 : Upper Case C cedilla - C7 hex >> UTF-8 : Upper Case C cedilla (compatibility codepoint) - C7 hex >> Unicode preferred Upper Case C cedilla (composed set of codepoints) >> Upper case C 0043 hex (Upper case C) >> followed by >> cedilla 00B8 hex (cedilla) >> >> 13) For any valid ASCII string *and* for any valid ISO-8859-1 string, >> aByteString is completely adequate for editing and display. >> >> 14) When sorting any valid ASCII string *or* any valid ISO-8859-1 >> string, upper and lower case versions of the same character will be >> treated differently. >> >> 15) When sorting any valid ISO-8859-1 string containing >> letter+diacritic combination glyphs or ligature combination glyphs, >> the glyphs in combination will treated differently to a "plain" glyph >> of the character >> i.e. "C" and "C cedilla" will be treated very differently. "?" and >> "fs" will be treated very differently. >> >> 16) Different nations have different rules about where diacritic-ed >> characted and ligature pairs should be placed when in alphabetical >> order. >> >> 17) Some nations even have multiple standards - e.g. surnames >> beginning either "M superscript-c" or "M superscript-a superscript-c" >> are treated as beginning equivalently in UK phone directories, but not >> in other situations. >> >> >> Some practical upshots >> ================== >> >> 1) Cuis and its ISO-8859-1 encoding is *exactly* the same as UTF-8, >> for any single character it considers valid, or any ByteString it has >> made up of characters it considers valid. >> >> 2) Any ByteString is valid UTF-8 in any of Squeak, Pharo, Cuis or any >> other Smalltalk with a single byte ByteString following ASCII or >> ISO-8859-1. >> >> 3) Any Smalltalk (or derivative language) using ByteString can >> immediately consider it's ByteString as valid UTF-8, as long as it >> also considers the ByteSring as valid ASCII and/or ISO-8859-1. >> >> 4) All of those can be successfully exported to any system using UTF-8 >> (e.g. HTML). >> >> 5) To successfully *accept* all UTF-8 we much be able to do either: >> a) accept UTF-8 strings with composed characters >> b) convert UTF-8 strings with composed characters into UTF-8 strings >> that use *only* compatibility codepoints. >> >> >> Class + protocol proposals >> >> >> >> a Utf8CompatibilityString class. >> >> asByteString - ensure only compatibility codepoints are used. >> Ensure it doews not encode characters above 00FF hex. >> >> asIso8859String - ensures only compatibility codepoints are used, >> and that the characters are each valid ISO 8859-1 >> >> asAsciiString - ensures only characters 00hex - 7F hex are used. >> >> asUtf8ComposedIso8859String - ensures all compatibility codepoints >> are expanded into small OrderedCollections of codepoints >> >> a Utf8ComposedIso8859String class - will provide sortable and >> comparable UTF8 strings of all ASCII and ISO 8859-1 strings. >> >> Then a Utf8SortableCollection class - a collection of >> Utf8ComposedIso8859Strings words and phrases. >> >> Custom sortBlocks will define the applicable sort order. >> >> We can create a collection... a Dictionary, thinking about it, of >> named, prefabricated sortBlocks. >> >> This will work for all UTF8 strings of ISO-8859-1 and ASCII strings. >> >> If anyone has better names for the classes, please let me know. >> >> If anyone else wants to help >> - build these, >> - create SUnit tests for these >> - write documentation for these >> Please let me know. >> >> n.b. I have had absolutely no experience of Ropes. >> >> My own background with this stuff: In the early 90's as a Project >> Manager implementing office automation systems across a global >> company, with offices in the Americas, Western, Eastern and Central >> Europe, (including Slavic and Cyrillic users) nations, Japan and >> China. The mission-critical application was word-processing. >> >> Our offices were spread around the globe, and we needed those offices >> to successfully exchange documents with their sister offices, and with >> the customers in each region the offices were in. >> >> Unicode was then new, and our platform supplier was the NeXT >> Corporation, who had been founder members in of the Unicode Consortium >> in 1990. >> >> So far: I've read the latest version of the Unicode Standard (v8.0). >> This is freely downloadable. >> I've purchased a paper copy of an earlier release. New releases >> typically consist additional codespaces (i.e. alphabets). So old >> copies are useful, as well as cheap. (Paper copies of version 4.0 >> are available second-hand for < $10 / ?10). >> >> The typical change with each release is the addition of further >> codespaces (i.e alphabets (more or less) ), so you don't lose a lot. >> (I'll be going through my V4.0 just to make sure) >> >> Cheers, >> Euan >> >> >> >> >> On 5 December 2015 at 13:08, stepharo wrote: >>> Hi EuanM >>> >>> Le 4/12/15 12:42, EuanM a ?crit : >>>> >>>> I'm currently groping my way to seeing how feature-complete our >>>> Unicode support is. I am doing this to establish what still needs to >>>> be done to provide full Unicode support. >>> >>> >>> this is great. Thanks for pushing this. I wrote and collected some roadmap >>> (analyses on different topics) >>> on the pharo github project feel free to add this one there. >>>> >>>> >>>> This seems to me to be an area where it would be best to write it >>>> once, and then have the same codebase incorporated into the Smalltalks >>>> that most share a common ancestry. >>>> >>>> I am keen to get: equality-testing for strings; sortability for >>>> strings which have ligatures and diacritic characters; and correct >>>> round-tripping of data. >>> >>> Go! >>> My suggestion is >>> start small >>> make steady progress >>> write tests >>> commit often :) >>> >>> Stef >>> >>> What is the french phoneBook ordering because this is the first time I hear >>> about it. >>> >>>> >>>> Call to action: >>>> ========== >>>> >>>> If you have comments on these proposals - such as "but we already have >>>> that facility" or "the reason we do not have these facilities is >>>> because they are dog-slow" - please let me know them. >>>> >>>> If you would like to help out, please let me know. >>>> >>>> If you have Unicode experience and expertise, and would like to be, or >>>> would be willing to be, in the 'council of experts' for this project, >>>> please let me know. >>>> >>>> If you have comments or ideas on anything mentioned in this email >>>> >>>> In the first instance, the initiative's website will be: >>>> http://smalltalk.uk.to/unicode.html >>>> >>>> I have created a SqueakSource.com project called UnicodeSupport >>>> >>>> I want to avoid re-inventing any facilities which already exist. >>>> Except where they prevent us reaching the goals of: >>>> - sortable UTF8 strings >>>> - sortable UTF16 strings >>>> - equivalence testing of 2 UTF8 strings >>>> - equivalence testing of 2 UTF16 strings >>>> - round-tripping UTF8 strings through Smalltalk >>>> - roundtripping UTF16 strings through Smalltalk. >>>> As I understand it, we have limited Unicode support atm. >>>> >>>> Current state of play >>>> =============== >>>> ByteString gets converted to WideString when need is automagically >>>> detected. >>>> >>>> Is there anything else that currently exists? >>>> >>>> Definition of Terms >>>> ============== >>>> A quick definition of terms before I go any further: >>>> >>>> Standard terms from the Unicode standard >>>> =============================== >>>> a compatibility character : an additional encoding of a *normal* >>>> character, for compatibility and round-trip conversion purposes. For >>>> instance, a 1-byte encoding of a Latin character with a diacritic. >>>> >>>> Made-up terms >>>> ============ >>>> a convenience codepoint : a single codepoint which represents an item >>>> that is also encoded as a string of codepoints. >>>> >>>> (I tend to use the terms compatibility character and compatibility >>>> codepoint interchangably. The standard only refers to them as >>>> compatibility characters. However, the standard is determined to >>>> emphasise that characters are abstract and that codepoints are >>>> concrete. So I think it is often more useful and productive to think >>>> of compatibility or convenience codepoints). >>>> >>>> a composed character : a character made up of several codepoints >>>> >>>> Unicode encoding explained >>>> ===================== >>>> A convenience codepoint can therefore be thought of as a code point >>>> used for a character which also has a composed form. >>>> >>>> The way Unicode works is that sometimes you can encode a character in >>>> one byte, sometimes not. Sometimes you can encode it in two bytes, >>>> sometimes not. >>>> >>>> You can therefore have a long stream of ASCII which is single-byte >>>> Unicode. If there is an occasional Cyrillic or Greek character in the >>>> stream, it would be represented either by a compatibility character or >>>> by a multi-byte combination. >>>> >>>> Using compatibility characters can prevent proper sorting and >>>> equivalence testing. >>>> >>>> Using "pure" Unicode, ie. "normal encodings", can cause compatibility >>>> and round-tripping probelms. Although avoiding them can *also* cause >>>> compatibility issues and round-tripping problems. >>>> >>>> Currently my thinking is: >>>> >>>> a Utf8String class >>>> an Ordered collection, with 1 byte characters as the modal element, >>>> but short arrays of wider strings where necessary >>>> a Utf16String class >>>> an Ordered collection, with 2 byte characters as the modal element, >>>> but short arrays of wider strings >>>> beginning with a 2-byte endianness indicator. >>>> >>>> Utf8Strings sometimes need to be sortable, and sometimes need to be >>>> compatible. >>>> >>>> So my thinking is that Utf8String will contain convenience codepoints, >>>> for round-tripping. And where there are multiple convenience >>>> codepoints for a character, that it standardises on one. >>>> >>>> And that there is a Utf8SortableString which uses *only* normal >>>> characters. >>>> >>>> We then need methods to convert between the two. >>>> >>>> aUtf8String asUtf8SortableString >>>> >>>> and >>>> >>>> aUtf8SortableString asUtf8String >>>> >>>> >>>> Sort orders are culture and context dependent - Sweden and Germany >>>> have different sort orders for the same diacritic-ed characters. Some >>>> countries have one order in general usage, and another for specific >>>> usages, such as phone directories (e.g. UK and France) >>>> >>>> Similarly for Utf16 : Utf16String and Utf16SortableString and >>>> conversion methods >>>> >>>> A list of sorted words would be a SortedCollection, and there could be >>>> pre-prepared sortBlocks for them, e.g. frPhoneBookOrder, deOrder, >>>> seOrder, ukOrder, etc >>>> >>>> along the lines of >>>> aListOfWords := SortedCollection sortBlock: deOrder >>>> >>>> If a word is either a Utf8SortableString, or a well-formed Utf8String, >>>> then we can perform equivalence testing on them trivially. >>>> >>>> To make sure a Utf8String is well formed, we would need to have a way >>>> of cleaning up any convenience codepoints which were valid, but which >>>> were for a character which has multiple equally-valid alternative >>>> convenience codepoints, and for which the string currently had the >>>> "wrong" convenience codepoint. (i.e for any character with valid >>>> alternative convenience codepoints, we would choose one to be in the >>>> well-formed Utf8String, and we would need a method for cleaning the >>>> alternative convenience codepoints out of the string, and replacing >>>> them with the chosen approved convenience codepoint. >>>> >>>> aUtf8String cleanUtf8String >>>> >>>> With WideString, a lot of the issues disappear - except >>>> round-tripping(although I'm sure I have seen something recently about >>>> 4-byte strings that also have an additional bit. Which would make >>>> some Unicode characters 5-bytes long.) >>>> >>>> >>>> (I'm starting to zone out now - if I've overlooked anything - obvious, >>>> subtle, or somewhere in between, please let me know) >>>> >>>> Cheers, >>>> Euan >>>> >>>> >>> >>> > >> On 07 Dec 2015, at 04:21, EuanM wrote: >> >> This a long email. A *lot* of it is encapsulated in the Venn diagram both: >> http://smalltalk.uk.to/unicode-utf8.html >> and my Smalltalk in Small Steps blog at: >> http://smalltalkinsmallsteps.blogspot.co.uk/2015/12/utf-8-for-cuis-pharo-and-squeak.html >> >> My current thinking, and understanding. >> ============================== >> >> 0) a) ASCII and ISO-8859-1 consist of characters encoded in 1 byte. >> b) UTF-8 can encode all of those characters in 1 byte, but can >> prefer some of them to be encoded as sequences of multiple bytes. And >> can encode additional characters as sequences of multiple bytes. >> >> 1) Smalltalk has long had multiple String classes. >> >> 2) Any UTF16 Unicode codepoint which has a codepoint of 00nn hex >> is encoded as a UTF-8 codepoint of nn hex. >> >> 3) All valid ISO-8859-1 characters have a character code between 20 >> hex and 7E hex, or between A0 hex and FF hex. >> https://en.wikipedia.org/wiki/ISO/IEC_8859-1 >> >> 4) All valid ASCII characters have a character code between 00 hex and 7E hex. >> https://en.wikipedia.org/wiki/ASCII >> >> >> 5) a) All character codes which are defined within ISO-8859-1 and also >> defined within ASCII. (i.e. character codes 20 hex to 7E hex) are >> defined identically in both. >> >> b) All printable ASCII characters are defined identically in both >> ASCII and ISO-8859-1 >> >> 6) All character codes defined in ASCII (00 hex to 7E hex) are >> defined identically in Unicode UTF-8. >> >> 7) All character codes defined in ISO-8859-1 (20 hex - 7E hex ; A0 hex >> - FF hex ) are defined identically in UTF-8. >> >> 8) => some Unicode codepoints map to both ASCII and ISO-8859-1. >> all ASCII maps 1:1 to Unicode UTF-8 >> all ISO-8859-1 maps 1:1 to Unicode UTF-8 >> >> 9) All ByteStrings elements which are either a valid ISO-8859-1 >> character or a valid ASCII character are *also* a valid UTF-8 >> character. >> >> 10) ISO-8859-1 characters representing a character with a diacritic, >> or a two-character ligature, have no ASCII equivalent. In Unicode >> UTF-8, those character codes which are representing compound glyphs, >> are called "compatibility codepoints". >> >> 11) The preferred Unicode representation of the characters which have >> compatibility codepoints is as a a short set of codepoints >> representing the characters which are combined together to form the >> glyph of the convenience codepoint, as a sequence of bytes >> representing the component characters. >> >> >> 12) Some concrete examples: >> >> A - aka Upper Case A >> In ASCII, in ISO 8859-1 >> ASCII A - 41 hex >> ISO-8859-1 A - 41 hex >> UTF-8 A - 41 hex >> >> BEL (a bell sound, often invoked by a Ctrl-g keyboard chord) >> In ASCII, not in ISO 8859-1 >> ASCII : BEL - 07 hex >> ISO-8859-1 : 07 hex is not a valid character code >> UTF-8 : BEL - 07 hex >> >> ? (GBP currency symbol) >> In ISO-8859-1, not in ASCII >> ASCII : A3 hex is not a valid ASCII code >> UTF-8: ? - A3 hex >> ISO-8859-1: ? - A3 hex >> >> Upper Case C cedilla >> In ISO-8859-1, not in ASCII, in UTF-8 as a compatibility codepoint >> *and* a composed set of codepoints >> ASCII : C7 hex is not a valid ASCII character code >> ISO-8859-1 : Upper Case C cedilla - C7 hex >> UTF-8 : Upper Case C cedilla (compatibility codepoint) - C7 hex >> Unicode preferred Upper Case C cedilla (composed set of codepoints) >> Upper case C 0043 hex (Upper case C) >> followed by >> cedilla 00B8 hex (cedilla) >> >> 13) For any valid ASCII string *and* for any valid ISO-8859-1 string, >> aByteString is completely adequate for editing and display. >> >> 14) When sorting any valid ASCII string *or* any valid ISO-8859-1 >> string, upper and lower case versions of the same character will be >> treated differently. >> >> 15) When sorting any valid ISO-8859-1 string containing >> letter+diacritic combination glyphs or ligature combination glyphs, >> the glyphs in combination will treated differently to a "plain" glyph >> of the character >> i.e. "C" and "C cedilla" will be treated very differently. "?" and >> "fs" will be treated very differently. >> >> 16) Different nations have different rules about where diacritic-ed >> characted and ligature pairs should be placed when in alphabetical >> order. >> >> 17) Some nations even have multiple standards - e.g. surnames >> beginning either "M superscript-c" or "M superscript-a superscript-c" >> are treated as beginning equivalently in UK phone directories, but not >> in other situations. >> >> >> Some practical upshots >> ================== >> >> 1) Cuis and its ISO-8859-1 encoding is *exactly* the same as UTF-8, >> for any single character it considers valid, or any ByteString it has >> made up of characters it considers valid. >> >> 2) Any ByteString is valid UTF-8 in any of Squeak, Pharo, Cuis or any >> other Smalltalk with a single byte ByteString following ASCII or >> ISO-8859-1. >> >> 3) Any Smalltalk (or derivative language) using ByteString can >> immediately consider it's ByteString as valid UTF-8, as long as it >> also considers the ByteSring as valid ASCII and/or ISO-8859-1. >> >> 4) All of those can be successfully exported to any system using UTF-8 >> (e.g. HTML). >> >> 5) To successfully *accept* all UTF-8 we much be able to do either: >> a) accept UTF-8 strings with composed characters >> b) convert UTF-8 strings with composed characters into UTF-8 strings >> that use *only* compatibility codepoints. >> >> >> Class + protocol proposals >> >> >> >> a Utf8CompatibilityString class. >> >> asByteString - ensure only compatibility codepoints are used. >> Ensure it doews not encode characters above 00FF hex. >> >> asIso8859String - ensures only compatibility codepoints are used, >> and that the characters are each valid ISO 8859-1 >> >> asAsciiString - ensures only characters 00hex - 7F hex are used. >> >> asUtf8ComposedIso8859String - ensures all compatibility codepoints >> are expanded into small OrderedCollections of codepoints >> >> a Utf8ComposedIso8859String class - will provide sortable and >> comparable UTF8 strings of all ASCII and ISO 8859-1 strings. >> >> Then a Utf8SortableCollection class - a collection of >> Utf8ComposedIso8859Strings words and phrases. >> >> Custom sortBlocks will define the applicable sort order. >> >> We can create a collection... a Dictionary, thinking about it, of >> named, prefabricated sortBlocks. >> >> This will work for all UTF8 strings of ISO-8859-1 and ASCII strings. >> >> If anyone has better names for the classes, please let me know. >> >> If anyone else wants to help >> - build these, >> - create SUnit tests for these >> - write documentation for these >> Please let me know. >> >> n.b. I have had absolutely no experience of Ropes. >> >> My own background with this stuff: In the early 90's as a Project >> Manager implementing office automation systems across a global >> company, with offices in the Americas, Western, Eastern and Central >> Europe, (including Slavic and Cyrillic users) nations, Japan and >> China. The mission-critical application was word-processing. >> >> Our offices were spread around the globe, and we needed those offices >> to successfully exchange documents with their sister offices, and with >> the customers in each region the offices were in. >> >> Unicode was then new, and our platform supplier was the NeXT >> Corporation, who had been founder members in of the Unicode Consortium >> in 1990. >> >> So far: I've read the latest version of the Unicode Standard (v8.0). >> This is freely downloadable. >> I've purchased a paper copy of an earlier release. New releases >> typically consist additional codespaces (i.e. alphabets). So old >> copies are useful, as well as cheap. (Paper copies of version 4.0 >> are available second-hand for < $10 / ?10). >> >> The typical change with each release is the addition of further >> codespaces (i.e alphabets (more or less) ), so you don't lose a lot. >> (I'll be going through my V4.0 just to make sure) >> >> Cheers, >> Euan >> >> >> >> >> On 5 December 2015 at 13:08, stepharo wrote: >>> Hi EuanM >>> >>> Le 4/12/15 12:42, EuanM a ?crit : >>>> >>>> I'm currently groping my way to seeing how feature-complete our >>>> Unicode support is. I am doing this to establish what still needs to >>>> be done to provide full Unicode support. >>> >>> >>> this is great. Thanks for pushing this. I wrote and collected some roadmap >>> (analyses on different topics) >>> on the pharo github project feel free to add this one there. >>>> >>>> >>>> This seems to me to be an area where it would be best to write it >>>> once, and then have the same codebase incorporated into the Smalltalks >>>> that most share a common ancestry. >>>> >>>> I am keen to get: equality-testing for strings; sortability for >>>> strings which have ligatures and diacritic characters; and correct >>>> round-tripping of data. >>> >>> Go! >>> My suggestion is >>> start small >>> make steady progress >>> write tests >>> commit often :) >>> >>> Stef >>> >>> What is the french phoneBook ordering because this is the first time I hear >>> about it. >>> >>>> >>>> Call to action: >>>> ========== >>>> >>>> If you have comments on these proposals - such as "but we already have >>>> that facility" or "the reason we do not have these facilities is >>>> because they are dog-slow" - please let me know them. >>>> >>>> If you would like to help out, please let me know. >>>> >>>> If you have Unicode experience and expertise, and would like to be, or >>>> would be willing to be, in the 'council of experts' for this project, >>>> please let me know. >>>> >>>> If you have comments or ideas on anything mentioned in this email >>>> >>>> In the first instance, the initiative's website will be: >>>> http://smalltalk.uk.to/unicode.html >>>> >>>> I have created a SqueakSource.com project called UnicodeSupport >>>> >>>> I want to avoid re-inventing any facilities which already exist. >>>> Except where they prevent us reaching the goals of: >>>> - sortable UTF8 strings >>>> - sortable UTF16 strings >>>> - equivalence testing of 2 UTF8 strings >>>> - equivalence testing of 2 UTF16 strings >>>> - round-tripping UTF8 strings through Smalltalk >>>> - roundtripping UTF16 strings through Smalltalk. >>>> As I understand it, we have limited Unicode support atm. >>>> >>>> Current state of play >>>> =============== >>>> ByteString gets converted to WideString when need is automagically >>>> detected. >>>> >>>> Is there anything else that currently exists? >>>> >>>> Definition of Terms >>>> ============== >>>> A quick definition of terms before I go any further: >>>> >>>> Standard terms from the Unicode standard >>>> =============================== >>>> a compatibility character : an additional encoding of a *normal* >>>> character, for compatibility and round-trip conversion purposes. For >>>> instance, a 1-byte encoding of a Latin character with a diacritic. >>>> >>>> Made-up terms >>>> ============ >>>> a convenience codepoint : a single codepoint which represents an item >>>> that is also encoded as a string of codepoints. >>>> >>>> (I tend to use the terms compatibility character and compatibility >>>> codepoint interchangably. The standard only refers to them as >>>> compatibility characters. However, the standard is determined to >>>> emphasise that characters are abstract and that codepoints are >>>> concrete. So I think it is often more useful and productive to think >>>> of compatibility or convenience codepoints). >>>> >>>> a composed character : a character made up of several codepoints >>>> >>>> Unicode encoding explained >>>> ===================== >>>> A convenience codepoint can therefore be thought of as a code point >>>> used for a character which also has a composed form. >>>> >>>> The way Unicode works is that sometimes you can encode a character in >>>> one byte, sometimes not. Sometimes you can encode it in two bytes, >>>> sometimes not. >>>> >>>> You can therefore have a long stream of ASCII which is single-byte >>>> Unicode. If there is an occasional Cyrillic or Greek character in the >>>> stream, it would be represented either by a compatibility character or >>>> by a multi-byte combination. >>>> >>>> Using compatibility characters can prevent proper sorting and >>>> equivalence testing. >>>> >>>> Using "pure" Unicode, ie. "normal encodings", can cause compatibility >>>> and round-tripping probelms. Although avoiding them can *also* cause >>>> compatibility issues and round-tripping problems. >>>> >>>> Currently my thinking is: >>>> >>>> a Utf8String class >>>> an Ordered collection, with 1 byte characters as the modal element, >>>> but short arrays of wider strings where necessary >>>> a Utf16String class >>>> an Ordered collection, with 2 byte characters as the modal element, >>>> but short arrays of wider strings >>>> beginning with a 2-byte endianness indicator. >>>> >>>> Utf8Strings sometimes need to be sortable, and sometimes need to be >>>> compatible. >>>> >>>> So my thinking is that Utf8String will contain convenience codepoints, >>>> for round-tripping. And where there are multiple convenience >>>> codepoints for a character, that it standardises on one. >>>> >>>> And that there is a Utf8SortableString which uses *only* normal >>>> characters. >>>> >>>> We then need methods to convert between the two. >>>> >>>> aUtf8String asUtf8SortableString >>>> >>>> and >>>> >>>> aUtf8SortableString asUtf8String >>>> >>>> >>>> Sort orders are culture and context dependent - Sweden and Germany >>>> have different sort orders for the same diacritic-ed characters. Some >>>> countries have one order in general usage, and another for specific >>>> usages, such as phone directories (e.g. UK and France) >>>> >>>> Similarly for Utf16 : Utf16String and Utf16SortableString and >>>> conversion methods >>>> >>>> A list of sorted words would be a SortedCollection, and there could be >>>> pre-prepared sortBlocks for them, e.g. frPhoneBookOrder, deOrder, >>>> seOrder, ukOrder, etc >>>> >>>> along the lines of >>>> aListOfWords := SortedCollection sortBlock: deOrder >>>> >>>> If a word is either a Utf8SortableString, or a well-formed Utf8String, >>>> then we can perform equivalence testing on them trivially. >>>> >>>> To make sure a Utf8String is well formed, we would need to have a way >>>> of cleaning up any convenience codepoints which were valid, but which >>>> were for a character which has multiple equally-valid alternative >>>> convenience codepoints, and for which the string currently had the >>>> "wrong" convenience codepoint. (i.e for any character with valid >>>> alternative convenience codepoints, we would choose one to be in the >>>> well-formed Utf8String, and we would need a method for cleaning the >>>> alternative convenience codepoints out of the string, and replacing >>>> them with the chosen approved convenience codepoint. >>>> >>>> aUtf8String cleanUtf8String >>>> >>>> With WideString, a lot of the issues disappear - except >>>> round-tripping(although I'm sure I have seen something recently about >>>> 4-byte strings that also have an additional bit. Which would make >>>> some Unicode characters 5-bytes long.) >>>> >>>> >>>> (I'm starting to zone out now - if I've overlooked anything - obvious, >>>> subtle, or somewhere in between, please let me know) >>>> >>>> Cheers, >>>> Euan >>>> >>>> >>> >>> >> > > From euanmee at gmail.com Mon Dec 7 11:03:45 2015 From: euanmee at gmail.com (EuanM) Date: Mon Dec 7 11:03:49 2015 Subject: [squeak-dev] Re: [Pharo-dev] Unicode Support In-Reply-To: <96B16DCE-A9E7-49E6-B866-E5A846801AA0@stfx.eu> References: <5662E1D8.1070802@free.fr> <96B16DCE-A9E7-49E6-B866-E5A846801AA0@stfx.eu> Message-ID: As for the issue of lower case e acute, it is compatibility codepoint 00e9 hex and therefore encodable in UTF-8 as compatibility codepoint e9 hex and as the composed character #(0065 00b4) (all in hex) and as the same composed character as both #(feff 0065 00b4) and #(ffef 0065 00b4) when endianness markers are included and as I understand it, should also be legitimate to encode it in UTF8 as a composed character #(65 b4) (all hex) etc On 7 December 2015 at 08:27, Sven Van Caekenberghe wrote: > I am sorry but one of your basic assumptions is completely wrong: > > 'Les ?l?ves Fran?ais' encodeWith: #iso99591. > > => #[76 101 115 32 233 108 232 118 101 115 32 70 114 97 110 231 97 105 115] > > 'Les ?l?ves Fran?ais' utf8Encoded. > > => #[76 101 115 32 195 169 108 195 168 118 101 115 32 70 114 97 110 195 167 97 105 115] > > ISO-9959-1 (~Latin1) is NOT AT ALL identical to UTF-8 in its upper, non-ACII part !! > > Or shorter, $? is encoded in ISO-9959-1 as #[233], but as #[195 169] in UTF-8. > > So more than half the points you make, or the facts that you state, are thus plain wrong. > > The only thing that is correct is that the code points are equal, but that is not the same as the encoding ! > > From this I am inclined to conclude that you do not fundamentally understand how UTF-8 works, which does not strike me as good basis to design something called a UTF8String. > > Sorry. > > PS: Note also that Cuis' choice to use ISO-9959-1 only is pretty limiting in a Unicode world. > >> On 07 Dec 2015, at 04:21, EuanM wrote: >> >> This a long email. A *lot* of it is encapsulated in the Venn diagram both: >> http://smalltalk.uk.to/unicode-utf8.html >> and my Smalltalk in Small Steps blog at: >> http://smalltalkinsmallsteps.blogspot.co.uk/2015/12/utf-8-for-cuis-pharo-and-squeak.html >> >> My current thinking, and understanding. >> ============================== >> >> 0) a) ASCII and ISO-8859-1 consist of characters encoded in 1 byte. >> b) UTF-8 can encode all of those characters in 1 byte, but can >> prefer some of them to be encoded as sequences of multiple bytes. And >> can encode additional characters as sequences of multiple bytes. >> >> 1) Smalltalk has long had multiple String classes. >> >> 2) Any UTF16 Unicode codepoint which has a codepoint of 00nn hex >> is encoded as a UTF-8 codepoint of nn hex. >> >> 3) All valid ISO-8859-1 characters have a character code between 20 >> hex and 7E hex, or between A0 hex and FF hex. >> https://en.wikipedia.org/wiki/ISO/IEC_8859-1 >> >> 4) All valid ASCII characters have a character code between 00 hex and 7E hex. >> https://en.wikipedia.org/wiki/ASCII >> >> >> 5) a) All character codes which are defined within ISO-8859-1 and also >> defined within ASCII. (i.e. character codes 20 hex to 7E hex) are >> defined identically in both. >> >> b) All printable ASCII characters are defined identically in both >> ASCII and ISO-8859-1 >> >> 6) All character codes defined in ASCII (00 hex to 7E hex) are >> defined identically in Unicode UTF-8. >> >> 7) All character codes defined in ISO-8859-1 (20 hex - 7E hex ; A0 hex >> - FF hex ) are defined identically in UTF-8. >> >> 8) => some Unicode codepoints map to both ASCII and ISO-8859-1. >> all ASCII maps 1:1 to Unicode UTF-8 >> all ISO-8859-1 maps 1:1 to Unicode UTF-8 >> >> 9) All ByteStrings elements which are either a valid ISO-8859-1 >> character or a valid ASCII character are *also* a valid UTF-8 >> character. >> >> 10) ISO-8859-1 characters representing a character with a diacritic, >> or a two-character ligature, have no ASCII equivalent. In Unicode >> UTF-8, those character codes which are representing compound glyphs, >> are called "compatibility codepoints". >> >> 11) The preferred Unicode representation of the characters which have >> compatibility codepoints is as a a short set of codepoints >> representing the characters which are combined together to form the >> glyph of the convenience codepoint, as a sequence of bytes >> representing the component characters. >> >> >> 12) Some concrete examples: >> >> A - aka Upper Case A >> In ASCII, in ISO 8859-1 >> ASCII A - 41 hex >> ISO-8859-1 A - 41 hex >> UTF-8 A - 41 hex >> >> BEL (a bell sound, often invoked by a Ctrl-g keyboard chord) >> In ASCII, not in ISO 8859-1 >> ASCII : BEL - 07 hex >> ISO-8859-1 : 07 hex is not a valid character code >> UTF-8 : BEL - 07 hex >> >> ? (GBP currency symbol) >> In ISO-8859-1, not in ASCII >> ASCII : A3 hex is not a valid ASCII code >> UTF-8: ? - A3 hex >> ISO-8859-1: ? - A3 hex >> >> Upper Case C cedilla >> In ISO-8859-1, not in ASCII, in UTF-8 as a compatibility codepoint >> *and* a composed set of codepoints >> ASCII : C7 hex is not a valid ASCII character code >> ISO-8859-1 : Upper Case C cedilla - C7 hex >> UTF-8 : Upper Case C cedilla (compatibility codepoint) - C7 hex >> Unicode preferred Upper Case C cedilla (composed set of codepoints) >> Upper case C 0043 hex (Upper case C) >> followed by >> cedilla 00B8 hex (cedilla) >> >> 13) For any valid ASCII string *and* for any valid ISO-8859-1 string, >> aByteString is completely adequate for editing and display. >> >> 14) When sorting any valid ASCII string *or* any valid ISO-8859-1 >> string, upper and lower case versions of the same character will be >> treated differently. >> >> 15) When sorting any valid ISO-8859-1 string containing >> letter+diacritic combination glyphs or ligature combination glyphs, >> the glyphs in combination will treated differently to a "plain" glyph >> of the character >> i.e. "C" and "C cedilla" will be treated very differently. "?" and >> "fs" will be treated very differently. >> >> 16) Different nations have different rules about where diacritic-ed >> characted and ligature pairs should be placed when in alphabetical >> order. >> >> 17) Some nations even have multiple standards - e.g. surnames >> beginning either "M superscript-c" or "M superscript-a superscript-c" >> are treated as beginning equivalently in UK phone directories, but not >> in other situations. >> >> >> Some practical upshots >> ================== >> >> 1) Cuis and its ISO-8859-1 encoding is *exactly* the same as UTF-8, >> for any single character it considers valid, or any ByteString it has >> made up of characters it considers valid. >> >> 2) Any ByteString is valid UTF-8 in any of Squeak, Pharo, Cuis or any >> other Smalltalk with a single byte ByteString following ASCII or >> ISO-8859-1. >> >> 3) Any Smalltalk (or derivative language) using ByteString can >> immediately consider it's ByteString as valid UTF-8, as long as it >> also considers the ByteSring as valid ASCII and/or ISO-8859-1. >> >> 4) All of those can be successfully exported to any system using UTF-8 >> (e.g. HTML). >> >> 5) To successfully *accept* all UTF-8 we much be able to do either: >> a) accept UTF-8 strings with composed characters >> b) convert UTF-8 strings with composed characters into UTF-8 strings >> that use *only* compatibility codepoints. >> >> >> Class + protocol proposals >> >> >> >> a Utf8CompatibilityString class. >> >> asByteString - ensure only compatibility codepoints are used. >> Ensure it doews not encode characters above 00FF hex. >> >> asIso8859String - ensures only compatibility codepoints are used, >> and that the characters are each valid ISO 8859-1 >> >> asAsciiString - ensures only characters 00hex - 7F hex are used. >> >> asUtf8ComposedIso8859String - ensures all compatibility codepoints >> are expanded into small OrderedCollections of codepoints >> >> a Utf8ComposedIso8859String class - will provide sortable and >> comparable UTF8 strings of all ASCII and ISO 8859-1 strings. >> >> Then a Utf8SortableCollection class - a collection of >> Utf8ComposedIso8859Strings words and phrases. >> >> Custom sortBlocks will define the applicable sort order. >> >> We can create a collection... a Dictionary, thinking about it, of >> named, prefabricated sortBlocks. >> >> This will work for all UTF8 strings of ISO-8859-1 and ASCII strings. >> >> If anyone has better names for the classes, please let me know. >> >> If anyone else wants to help >> - build these, >> - create SUnit tests for these >> - write documentation for these >> Please let me know. >> >> n.b. I have had absolutely no experience of Ropes. >> >> My own background with this stuff: In the early 90's as a Project >> Manager implementing office automation systems across a global >> company, with offices in the Americas, Western, Eastern and Central >> Europe, (including Slavic and Cyrillic users) nations, Japan and >> China. The mission-critical application was word-processing. >> >> Our offices were spread around the globe, and we needed those offices >> to successfully exchange documents with their sister offices, and with >> the customers in each region the offices were in. >> >> Unicode was then new, and our platform supplier was the NeXT >> Corporation, who had been founder members in of the Unicode Consortium >> in 1990. >> >> So far: I've read the latest version of the Unicode Standard (v8.0). >> This is freely downloadable. >> I've purchased a paper copy of an earlier release. New releases >> typically consist additional codespaces (i.e. alphabets). So old >> copies are useful, as well as cheap. (Paper copies of version 4.0 >> are available second-hand for < $10 / ?10). >> >> The typical change with each release is the addition of further >> codespaces (i.e alphabets (more or less) ), so you don't lose a lot. >> (I'll be going through my V4.0 just to make sure) >> >> Cheers, >> Euan >> >> >> >> >> On 5 December 2015 at 13:08, stepharo wrote: >>> Hi EuanM >>> >>> Le 4/12/15 12:42, EuanM a ?crit : >>>> >>>> I'm currently groping my way to seeing how feature-complete our >>>> Unicode support is. I am doing this to establish what still needs to >>>> be done to provide full Unicode support. >>> >>> >>> this is great. Thanks for pushing this. I wrote and collected some roadmap >>> (analyses on different topics) >>> on the pharo github project feel free to add this one there. >>>> >>>> >>>> This seems to me to be an area where it would be best to write it >>>> once, and then have the same codebase incorporated into the Smalltalks >>>> that most share a common ancestry. >>>> >>>> I am keen to get: equality-testing for strings; sortability for >>>> strings which have ligatures and diacritic characters; and correct >>>> round-tripping of data. >>> >>> Go! >>> My suggestion is >>> start small >>> make steady progress >>> write tests >>> commit often :) >>> >>> Stef >>> >>> What is the french phoneBook ordering because this is the first time I hear >>> about it. >>> >>>> >>>> Call to action: >>>> ========== >>>> >>>> If you have comments on these proposals - such as "but we already have >>>> that facility" or "the reason we do not have these facilities is >>>> because they are dog-slow" - please let me know them. >>>> >>>> If you would like to help out, please let me know. >>>> >>>> If you have Unicode experience and expertise, and would like to be, or >>>> would be willing to be, in the 'council of experts' for this project, >>>> please let me know. >>>> >>>> If you have comments or ideas on anything mentioned in this email >>>> >>>> In the first instance, the initiative's website will be: >>>> http://smalltalk.uk.to/unicode.html >>>> >>>> I have created a SqueakSource.com project called UnicodeSupport >>>> >>>> I want to avoid re-inventing any facilities which already exist. >>>> Except where they prevent us reaching the goals of: >>>> - sortable UTF8 strings >>>> - sortable UTF16 strings >>>> - equivalence testing of 2 UTF8 strings >>>> - equivalence testing of 2 UTF16 strings >>>> - round-tripping UTF8 strings through Smalltalk >>>> - roundtripping UTF16 strings through Smalltalk. >>>> As I understand it, we have limited Unicode support atm. >>>> >>>> Current state of play >>>> =============== >>>> ByteString gets converted to WideString when need is automagically >>>> detected. >>>> >>>> Is there anything else that currently exists? >>>> >>>> Definition of Terms >>>> ============== >>>> A quick definition of terms before I go any further: >>>> >>>> Standard terms from the Unicode standard >>>> =============================== >>>> a compatibility character : an additional encoding of a *normal* >>>> character, for compatibility and round-trip conversion purposes. For >>>> instance, a 1-byte encoding of a Latin character with a diacritic. >>>> >>>> Made-up terms >>>> ============ >>>> a convenience codepoint : a single codepoint which represents an item >>>> that is also encoded as a string of codepoints. >>>> >>>> (I tend to use the terms compatibility character and compatibility >>>> codepoint interchangably. The standard only refers to them as >>>> compatibility characters. However, the standard is determined to >>>> emphasise that characters are abstract and that codepoints are >>>> concrete. So I think it is often more useful and productive to think >>>> of compatibility or convenience codepoints). >>>> >>>> a composed character : a character made up of several codepoints >>>> >>>> Unicode encoding explained >>>> ===================== >>>> A convenience codepoint can therefore be thought of as a code point >>>> used for a character which also has a composed form. >>>> >>>> The way Unicode works is that sometimes you can encode a character in >>>> one byte, sometimes not. Sometimes you can encode it in two bytes, >>>> sometimes not. >>>> >>>> You can therefore have a long stream of ASCII which is single-byte >>>> Unicode. If there is an occasional Cyrillic or Greek character in the >>>> stream, it would be represented either by a compatibility character or >>>> by a multi-byte combination. >>>> >>>> Using compatibility characters can prevent proper sorting and >>>> equivalence testing. >>>> >>>> Using "pure" Unicode, ie. "normal encodings", can cause compatibility >>>> and round-tripping probelms. Although avoiding them can *also* cause >>>> compatibility issues and round-tripping problems. >>>> >>>> Currently my thinking is: >>>> >>>> a Utf8String class >>>> an Ordered collection, with 1 byte characters as the modal element, >>>> but short arrays of wider strings where necessary >>>> a Utf16String class >>>> an Ordered collection, with 2 byte characters as the modal element, >>>> but short arrays of wider strings >>>> beginning with a 2-byte endianness indicator. >>>> >>>> Utf8Strings sometimes need to be sortable, and sometimes need to be >>>> compatible. >>>> >>>> So my thinking is that Utf8String will contain convenience codepoints, >>>> for round-tripping. And where there are multiple convenience >>>> codepoints for a character, that it standardises on one. >>>> >>>> And that there is a Utf8SortableString which uses *only* normal >>>> characters. >>>> >>>> We then need methods to convert between the two. >>>> >>>> aUtf8String asUtf8SortableString >>>> >>>> and >>>> >>>> aUtf8SortableString asUtf8String >>>> >>>> >>>> Sort orders are culture and context dependent - Sweden and Germany >>>> have different sort orders for the same diacritic-ed characters. Some >>>> countries have one order in general usage, and another for specific >>>> usages, such as phone directories (e.g. UK and France) >>>> >>>> Similarly for Utf16 : Utf16String and Utf16SortableString and >>>> conversion methods >>>> >>>> A list of sorted words would be a SortedCollection, and there could be >>>> pre-prepared sortBlocks for them, e.g. frPhoneBookOrder, deOrder, >>>> seOrder, ukOrder, etc >>>> >>>> along the lines of >>>> aListOfWords := SortedCollection sortBlock: deOrder >>>> >>>> If a word is either a Utf8SortableString, or a well-formed Utf8String, >>>> then we can perform equivalence testing on them trivially. >>>> >>>> To make sure a Utf8String is well formed, we would need to have a way >>>> of cleaning up any convenience codepoints which were valid, but which >>>> were for a character which has multiple equally-valid alternative >>>> convenience codepoints, and for which the string currently had the >>>> "wrong" convenience codepoint. (i.e for any character with valid >>>> alternative convenience codepoints, we would choose one to be in the >>>> well-formed Utf8String, and we would need a method for cleaning the >>>> alternative convenience codepoints out of the string, and replacing >>>> them with the chosen approved convenience codepoint. >>>> >>>> aUtf8String cleanUtf8String >>>> >>>> With WideString, a lot of the issues disappear - except >>>> round-tripping(although I'm sure I have seen something recently about >>>> 4-byte strings that also have an additional bit. Which would make >>>> some Unicode characters 5-bytes long.) >>>> >>>> >>>> (I'm starting to zone out now - if I've overlooked anything - obvious, >>>> subtle, or somewhere in between, please let me know) >>>> >>>> Cheers, >>>> Euan >>>> >>>> >>> >>> > >> On 07 Dec 2015, at 04:21, EuanM wrote: >> >> This a long email. A *lot* of it is encapsulated in the Venn diagram both: >> http://smalltalk.uk.to/unicode-utf8.html >> and my Smalltalk in Small Steps blog at: >> http://smalltalkinsmallsteps.blogspot.co.uk/2015/12/utf-8-for-cuis-pharo-and-squeak.html >> >> My current thinking, and understanding. >> ============================== >> >> 0) a) ASCII and ISO-8859-1 consist of characters encoded in 1 byte. >> b) UTF-8 can encode all of those characters in 1 byte, but can >> prefer some of them to be encoded as sequences of multiple bytes. And >> can encode additional characters as sequences of multiple bytes. >> >> 1) Smalltalk has long had multiple String classes. >> >> 2) Any UTF16 Unicode codepoint which has a codepoint of 00nn hex >> is encoded as a UTF-8 codepoint of nn hex. >> >> 3) All valid ISO-8859-1 characters have a character code between 20 >> hex and 7E hex, or between A0 hex and FF hex. >> https://en.wikipedia.org/wiki/ISO/IEC_8859-1 >> >> 4) All valid ASCII characters have a character code between 00 hex and 7E hex. >> https://en.wikipedia.org/wiki/ASCII >> >> >> 5) a) All character codes which are defined within ISO-8859-1 and also >> defined within ASCII. (i.e. character codes 20 hex to 7E hex) are >> defined identically in both. >> >> b) All printable ASCII characters are defined identically in both >> ASCII and ISO-8859-1 >> >> 6) All character codes defined in ASCII (00 hex to 7E hex) are >> defined identically in Unicode UTF-8. >> >> 7) All character codes defined in ISO-8859-1 (20 hex - 7E hex ; A0 hex >> - FF hex ) are defined identically in UTF-8. >> >> 8) => some Unicode codepoints map to both ASCII and ISO-8859-1. >> all ASCII maps 1:1 to Unicode UTF-8 >> all ISO-8859-1 maps 1:1 to Unicode UTF-8 >> >> 9) All ByteStrings elements which are either a valid ISO-8859-1 >> character or a valid ASCII character are *also* a valid UTF-8 >> character. >> >> 10) ISO-8859-1 characters representing a character with a diacritic, >> or a two-character ligature, have no ASCII equivalent. In Unicode >> UTF-8, those character codes which are representing compound glyphs, >> are called "compatibility codepoints". >> >> 11) The preferred Unicode representation of the characters which have >> compatibility codepoints is as a a short set of codepoints >> representing the characters which are combined together to form the >> glyph of the convenience codepoint, as a sequence of bytes >> representing the component characters. >> >> >> 12) Some concrete examples: >> >> A - aka Upper Case A >> In ASCII, in ISO 8859-1 >> ASCII A - 41 hex >> ISO-8859-1 A - 41 hex >> UTF-8 A - 41 hex >> >> BEL (a bell sound, often invoked by a Ctrl-g keyboard chord) >> In ASCII, not in ISO 8859-1 >> ASCII : BEL - 07 hex >> ISO-8859-1 : 07 hex is not a valid character code >> UTF-8 : BEL - 07 hex >> >> ? (GBP currency symbol) >> In ISO-8859-1, not in ASCII >> ASCII : A3 hex is not a valid ASCII code >> UTF-8: ? - A3 hex >> ISO-8859-1: ? - A3 hex >> >> Upper Case C cedilla >> In ISO-8859-1, not in ASCII, in UTF-8 as a compatibility codepoint >> *and* a composed set of codepoints >> ASCII : C7 hex is not a valid ASCII character code >> ISO-8859-1 : Upper Case C cedilla - C7 hex >> UTF-8 : Upper Case C cedilla (compatibility codepoint) - C7 hex >> Unicode preferred Upper Case C cedilla (composed set of codepoints) >> Upper case C 0043 hex (Upper case C) >> followed by >> cedilla 00B8 hex (cedilla) >> >> 13) For any valid ASCII string *and* for any valid ISO-8859-1 string, >> aByteString is completely adequate for editing and display. >> >> 14) When sorting any valid ASCII string *or* any valid ISO-8859-1 >> string, upper and lower case versions of the same character will be >> treated differently. >> >> 15) When sorting any valid ISO-8859-1 string containing >> letter+diacritic combination glyphs or ligature combination glyphs, >> the glyphs in combination will treated differently to a "plain" glyph >> of the character >> i.e. "C" and "C cedilla" will be treated very differently. "?" and >> "fs" will be treated very differently. >> >> 16) Different nations have different rules about where diacritic-ed >> characted and ligature pairs should be placed when in alphabetical >> order. >> >> 17) Some nations even have multiple standards - e.g. surnames >> beginning either "M superscript-c" or "M superscript-a superscript-c" >> are treated as beginning equivalently in UK phone directories, but not >> in other situations. >> >> >> Some practical upshots >> ================== >> >> 1) Cuis and its ISO-8859-1 encoding is *exactly* the same as UTF-8, >> for any single character it considers valid, or any ByteString it has >> made up of characters it considers valid. >> >> 2) Any ByteString is valid UTF-8 in any of Squeak, Pharo, Cuis or any >> other Smalltalk with a single byte ByteString following ASCII or >> ISO-8859-1. >> >> 3) Any Smalltalk (or derivative language) using ByteString can >> immediately consider it's ByteString as valid UTF-8, as long as it >> also considers the ByteSring as valid ASCII and/or ISO-8859-1. >> >> 4) All of those can be successfully exported to any system using UTF-8 >> (e.g. HTML). >> >> 5) To successfully *accept* all UTF-8 we much be able to do either: >> a) accept UTF-8 strings with composed characters >> b) convert UTF-8 strings with composed characters into UTF-8 strings >> that use *only* compatibility codepoints. >> >> >> Class + protocol proposals >> >> >> >> a Utf8CompatibilityString class. >> >> asByteString - ensure only compatibility codepoints are used. >> Ensure it doews not encode characters above 00FF hex. >> >> asIso8859String - ensures only compatibility codepoints are used, >> and that the characters are each valid ISO 8859-1 >> >> asAsciiString - ensures only characters 00hex - 7F hex are used. >> >> asUtf8ComposedIso8859String - ensures all compatibility codepoints >> are expanded into small OrderedCollections of codepoints >> >> a Utf8ComposedIso8859String class - will provide sortable and >> comparable UTF8 strings of all ASCII and ISO 8859-1 strings. >> >> Then a Utf8SortableCollection class - a collection of >> Utf8ComposedIso8859Strings words and phrases. >> >> Custom sortBlocks will define the applicable sort order. >> >> We can create a collection... a Dictionary, thinking about it, of >> named, prefabricated sortBlocks. >> >> This will work for all UTF8 strings of ISO-8859-1 and ASCII strings. >> >> If anyone has better names for the classes, please let me know. >> >> If anyone else wants to help >> - build these, >> - create SUnit tests for these >> - write documentation for these >> Please let me know. >> >> n.b. I have had absolutely no experience of Ropes. >> >> My own background with this stuff: In the early 90's as a Project >> Manager implementing office automation systems across a global >> company, with offices in the Americas, Western, Eastern and Central >> Europe, (including Slavic and Cyrillic users) nations, Japan and >> China. The mission-critical application was word-processing. >> >> Our offices were spread around the globe, and we needed those offices >> to successfully exchange documents with their sister offices, and with >> the customers in each region the offices were in. >> >> Unicode was then new, and our platform supplier was the NeXT >> Corporation, who had been founder members in of the Unicode Consortium >> in 1990. >> >> So far: I've read the latest version of the Unicode Standard (v8.0). >> This is freely downloadable. >> I've purchased a paper copy of an earlier release. New releases >> typically consist additional codespaces (i.e. alphabets). So old >> copies are useful, as well as cheap. (Paper copies of version 4.0 >> are available second-hand for < $10 / ?10). >> >> The typical change with each release is the addition of further >> codespaces (i.e alphabets (more or less) ), so you don't lose a lot. >> (I'll be going through my V4.0 just to make sure) >> >> Cheers, >> Euan >> >> >> >> >> On 5 December 2015 at 13:08, stepharo wrote: >>> Hi EuanM >>> >>> Le 4/12/15 12:42, EuanM a ?crit : >>>> >>>> I'm currently groping my way to seeing how feature-complete our >>>> Unicode support is. I am doing this to establish what still needs to >>>> be done to provide full Unicode support. >>> >>> >>> this is great. Thanks for pushing this. I wrote and collected some roadmap >>> (analyses on different topics) >>> on the pharo github project feel free to add this one there. >>>> >>>> >>>> This seems to me to be an area where it would be best to write it >>>> once, and then have the same codebase incorporated into the Smalltalks >>>> that most share a common ancestry. >>>> >>>> I am keen to get: equality-testing for strings; sortability for >>>> strings which have ligatures and diacritic characters; and correct >>>> round-tripping of data. >>> >>> Go! >>> My suggestion is >>> start small >>> make steady progress >>> write tests >>> commit often :) >>> >>> Stef >>> >>> What is the french phoneBook ordering because this is the first time I hear >>> about it. >>> >>>> >>>> Call to action: >>>> ========== >>>> >>>> If you have comments on these proposals - such as "but we already have >>>> that facility" or "the reason we do not have these facilities is >>>> because they are dog-slow" - please let me know them. >>>> >>>> If you would like to help out, please let me know. >>>> >>>> If you have Unicode experience and expertise, and would like to be, or >>>> would be willing to be, in the 'council of experts' for this project, >>>> please let me know. >>>> >>>> If you have comments or ideas on anything mentioned in this email >>>> >>>> In the first instance, the initiative's website will be: >>>> http://smalltalk.uk.to/unicode.html >>>> >>>> I have created a SqueakSource.com project called UnicodeSupport >>>> >>>> I want to avoid re-inventing any facilities which already exist. >>>> Except where they prevent us reaching the goals of: >>>> - sortable UTF8 strings >>>> - sortable UTF16 strings >>>> - equivalence testing of 2 UTF8 strings >>>> - equivalence testing of 2 UTF16 strings >>>> - round-tripping UTF8 strings through Smalltalk >>>> - roundtripping UTF16 strings through Smalltalk. >>>> As I understand it, we have limited Unicode support atm. >>>> >>>> Current state of play >>>> =============== >>>> ByteString gets converted to WideString when need is automagically >>>> detected. >>>> >>>> Is there anything else that currently exists? >>>> >>>> Definition of Terms >>>> ============== >>>> A quick definition of terms before I go any further: >>>> >>>> Standard terms from the Unicode standard >>>> =============================== >>>> a compatibility character : an additional encoding of a *normal* >>>> character, for compatibility and round-trip conversion purposes. For >>>> instance, a 1-byte encoding of a Latin character with a diacritic. >>>> >>>> Made-up terms >>>> ============ >>>> a convenience codepoint : a single codepoint which represents an item >>>> that is also encoded as a string of codepoints. >>>> >>>> (I tend to use the terms compatibility character and compatibility >>>> codepoint interchangably. The standard only refers to them as >>>> compatibility characters. However, the standard is determined to >>>> emphasise that characters are abstract and that codepoints are >>>> concrete. So I think it is often more useful and productive to think >>>> of compatibility or convenience codepoints). >>>> >>>> a composed character : a character made up of several codepoints >>>> >>>> Unicode encoding explained >>>> ===================== >>>> A convenience codepoint can therefore be thought of as a code point >>>> used for a character which also has a composed form. >>>> >>>> The way Unicode works is that sometimes you can encode a character in >>>> one byte, sometimes not. Sometimes you can encode it in two bytes, >>>> sometimes not. >>>> >>>> You can therefore have a long stream of ASCII which is single-byte >>>> Unicode. If there is an occasional Cyrillic or Greek character in the >>>> stream, it would be represented either by a compatibility character or >>>> by a multi-byte combination. >>>> >>>> Using compatibility characters can prevent proper sorting and >>>> equivalence testing. >>>> >>>> Using "pure" Unicode, ie. "normal encodings", can cause compatibility >>>> and round-tripping probelms. Although avoiding them can *also* cause >>>> compatibility issues and round-tripping problems. >>>> >>>> Currently my thinking is: >>>> >>>> a Utf8String class >>>> an Ordered collection, with 1 byte characters as the modal element, >>>> but short arrays of wider strings where necessary >>>> a Utf16String class >>>> an Ordered collection, with 2 byte characters as the modal element, >>>> but short arrays of wider strings >>>> beginning with a 2-byte endianness indicator. >>>> >>>> Utf8Strings sometimes need to be sortable, and sometimes need to be >>>> compatible. >>>> >>>> So my thinking is that Utf8String will contain convenience codepoints, >>>> for round-tripping. And where there are multiple convenience >>>> codepoints for a character, that it standardises on one. >>>> >>>> And that there is a Utf8SortableString which uses *only* normal >>>> characters. >>>> >>>> We then need methods to convert between the two. >>>> >>>> aUtf8String asUtf8SortableString >>>> >>>> and >>>> >>>> aUtf8SortableString asUtf8String >>>> >>>> >>>> Sort orders are culture and context dependent - Sweden and Germany >>>> have different sort orders for the same diacritic-ed characters. Some >>>> countries have one order in general usage, and another for specific >>>> usages, such as phone directories (e.g. UK and France) >>>> >>>> Similarly for Utf16 : Utf16String and Utf16SortableString and >>>> conversion methods >>>> >>>> A list of sorted words would be a SortedCollection, and there could be >>>> pre-prepared sortBlocks for them, e.g. frPhoneBookOrder, deOrder, >>>> seOrder, ukOrder, etc >>>> >>>> along the lines of >>>> aListOfWords := SortedCollection sortBlock: deOrder >>>> >>>> If a word is either a Utf8SortableString, or a well-formed Utf8String, >>>> then we can perform equivalence testing on them trivially. >>>> >>>> To make sure a Utf8String is well formed, we would need to have a way >>>> of cleaning up any convenience codepoints which were valid, but which >>>> were for a character which has multiple equally-valid alternative >>>> convenience codepoints, and for which the string currently had the >>>> "wrong" convenience codepoint. (i.e for any character with valid >>>> alternative convenience codepoints, we would choose one to be in the >>>> well-formed Utf8String, and we would need a method for cleaning the >>>> alternative convenience codepoints out of the string, and replacing >>>> them with the chosen approved convenience codepoint. >>>> >>>> aUtf8String cleanUtf8String >>>> >>>> With WideString, a lot of the issues disappear - except >>>> round-tripping(although I'm sure I have seen something recently about >>>> 4-byte strings that also have an additional bit. Which would make >>>> some Unicode characters 5-bytes long.) >>>> >>>> >>>> (I'm starting to zone out now - if I've overlooked anything - obvious, >>>> subtle, or somewhere in between, please let me know) >>>> >>>> Cheers, >>>> Euan >>>> >>>> >>> >>> >> > > From euanmee at gmail.com Mon Dec 7 11:10:45 2015 From: euanmee at gmail.com (EuanM) Date: Mon Dec 7 11:10:47 2015 Subject: [squeak-dev] Re: [Pharo-dev] Unicode Support In-Reply-To: References: <5662E1D8.1070802@free.fr> <96B16DCE-A9E7-49E6-B866-E5A846801AA0@stfx.eu> Message-ID: Hi Sven, okay I'm plodding my through https://tools.ietf.org/html/rfc3629 and https://en.wikipedia.org/wiki/UTF-8#Examples to see what's what. On 7 December 2015 at 11:01, Sven Van Caekenberghe wrote: > >> On 07 Dec 2015, at 11:51, EuanM wrote: >> >> Verifying assumptions is the key reason why you should documents like >> this out for review. > > Fair enough, discussion can only help. > >> Sven - >> >> Cuis is encoded with ISO 8859-15 (aka ISO Latin 9) >> >> Sven, this is *NOT* as you state, ISO 99591, (and not as I stated, 8859-1). > > Ah, that was a typo, I meant, of course (and sorry for the confusion): > > 'Les ?l?ves Fran?ais' encodeWith: #iso88591. > > "#[76 101 115 32 233 108 232 118 101 115 32 70 114 97 110 231 97 105 115]" > > 'Les ?l?ves Fran?ais' utf8Encoded > > "#[76 101 115 32 195 169 108 195 168 118 101 115 32 70 114 97 110 195 167 97 105 115]" > > Or shorter, $? is encoded in ISO-88591-1 as #[233], but as #[195 169] in UTF-8. > > That Cuis chose ISO-8859-15 makes no real difference. > > The thing is: you started talking about UTF-8 encoded strings in the image, and then the difference between code point and encoding is really important. > > Only in ASCII is the encoding identical, not for anything else. > >> We caught the right specification bug for the wrong reason. >> >> Juan: "Cuis: Chose not to use Squeak approach. Chose to make the base >> image include and use only 1-byte strings. Chose to use ISO-8859-15" >> >> I have double-checked - each character encoded in ISO Latin 15 (ISO >> 8859-15) is exactly the character represented by the corresponding >> 1-byte codepoint in Unicode 0000 to 00FF, >> >> with the following exceptions: >> >> codepoint 20ac - Euro symbol >> character code a4 (replaces codepoint 00a4 generic currency symbol) >> >> codepoint 0160 Latin Upper Case S with Caron >> character code a6 (replaces codepoint 00A6 was | Unix pipe character) >> >> codepoint 0161 Latin Lower Case s with Caron >> character code a8 (replaces codepoint 00A8 was dierisis) >> >> codepoint 017d Latin Upper Case Z with Caron >> character code b4 (replaces codepoint 00b4 was Acute accent) >> >> codepoint 017e Latin Lower Case Z with Caron >> character code b8 (replaces codepoint 00b8 was cedilla) >> >> codepoint 0152 Upper Case OE ligature = Ethel >> character code bc (replaces codepoint 00bc was 1/4 symbol) >> >> codepoint 0153 Lower Case oe ligature = ethel >> character code bd (replaces codepoint 00bd was 1/2 symbol) >> >> codepoint 0178 Upper Case Y diaeresis >> character code be (replaces codepoint 00be was 3/4 symbol) >> >> Juan - I don't suppose we could persuade you to change to ISO Latin-1 >> from ISO Latin-9 ? >> >> It means we could run the same 1 byte string encoding across Cuis, >> Squeak, Pharo, and, as far as I can make out so far, Dolphin Smalltalk >> and Gnu Smalltalk. >> >> The downside would be that French Y diaeresis would lose the ability >> to use that character, along with users of oe, OE, and s, S, z, Z with >> caron. Along with the Euro. >> >> https://en.wikipedia.org/wiki/ISO/IEC_8859-15. >> >> I'm confident I understand the use of UTF-8 in principal. >> >> >> On 7 December 2015 at 08:27, Sven Van Caekenberghe wrote: >>> I am sorry but one of your basic assumptions is completely wrong: >>> >>> 'Les ?l?ves Fran?ais' encodeWith: #iso99591. >>> >>> => #[76 101 115 32 233 108 232 118 101 115 32 70 114 97 110 231 97 105 115] >>> >>> 'Les ?l?ves Fran?ais' utf8Encoded. >>> >>> => #[76 101 115 32 195 169 108 195 168 118 101 115 32 70 114 97 110 195 167 97 105 115] >>> >>> ISO-9959-1 (~Latin1) is NOT AT ALL identical to UTF-8 in its upper, non-ACII part !! >>> >>> Or shorter, $? is encoded in ISO-9959-1 as #[233], but as #[195 169] in UTF-8. >>> >>> So more than half the points you make, or the facts that you state, are thus plain wrong. >>> >>> The only thing that is correct is that the code points are equal, but that is not the same as the encoding ! >>> >>> From this I am inclined to conclude that you do not fundamentally understand how UTF-8 works, which does not strike me as good basis to design something called a UTF8String. >>> >>> Sorry. >>> >>> PS: Note also that Cuis' choice to use ISO-9959-1 only is pretty limiting in a Unicode world. >>> >>>> On 07 Dec 2015, at 04:21, EuanM wrote: >>>> >>>> This a long email. A *lot* of it is encapsulated in the Venn diagram both: >>>> http://smalltalk.uk.to/unicode-utf8.html >>>> and my Smalltalk in Small Steps blog at: >>>> http://smalltalkinsmallsteps.blogspot.co.uk/2015/12/utf-8-for-cuis-pharo-and-squeak.html >>>> >>>> My current thinking, and understanding. >>>> ============================== >>>> >>>> 0) a) ASCII and ISO-8859-1 consist of characters encoded in 1 byte. >>>> b) UTF-8 can encode all of those characters in 1 byte, but can >>>> prefer some of them to be encoded as sequences of multiple bytes. And >>>> can encode additional characters as sequences of multiple bytes. >>>> >>>> 1) Smalltalk has long had multiple String classes. >>>> >>>> 2) Any UTF16 Unicode codepoint which has a codepoint of 00nn hex >>>> is encoded as a UTF-8 codepoint of nn hex. >>>> >>>> 3) All valid ISO-8859-1 characters have a character code between 20 >>>> hex and 7E hex, or between A0 hex and FF hex. >>>> https://en.wikipedia.org/wiki/ISO/IEC_8859-1 >>>> >>>> 4) All valid ASCII characters have a character code between 00 hex and 7E hex. >>>> https://en.wikipedia.org/wiki/ASCII >>>> >>>> >>>> 5) a) All character codes which are defined within ISO-8859-1 and also >>>> defined within ASCII. (i.e. character codes 20 hex to 7E hex) are >>>> defined identically in both. >>>> >>>> b) All printable ASCII characters are defined identically in both >>>> ASCII and ISO-8859-1 >>>> >>>> 6) All character codes defined in ASCII (00 hex to 7E hex) are >>>> defined identically in Unicode UTF-8. >>>> >>>> 7) All character codes defined in ISO-8859-1 (20 hex - 7E hex ; A0 hex >>>> - FF hex ) are defined identically in UTF-8. >>>> >>>> 8) => some Unicode codepoints map to both ASCII and ISO-8859-1. >>>> all ASCII maps 1:1 to Unicode UTF-8 >>>> all ISO-8859-1 maps 1:1 to Unicode UTF-8 >>>> >>>> 9) All ByteStrings elements which are either a valid ISO-8859-1 >>>> character or a valid ASCII character are *also* a valid UTF-8 >>>> character. >>>> >>>> 10) ISO-8859-1 characters representing a character with a diacritic, >>>> or a two-character ligature, have no ASCII equivalent. In Unicode >>>> UTF-8, those character codes which are representing compound glyphs, >>>> are called "compatibility codepoints". >>>> >>>> 11) The preferred Unicode representation of the characters which have >>>> compatibility codepoints is as a a short set of codepoints >>>> representing the characters which are combined together to form the >>>> glyph of the convenience codepoint, as a sequence of bytes >>>> representing the component characters. >>>> >>>> >>>> 12) Some concrete examples: >>>> >>>> A - aka Upper Case A >>>> In ASCII, in ISO 8859-1 >>>> ASCII A - 41 hex >>>> ISO-8859-1 A - 41 hex >>>> UTF-8 A - 41 hex >>>> >>>> BEL (a bell sound, often invoked by a Ctrl-g keyboard chord) >>>> In ASCII, not in ISO 8859-1 >>>> ASCII : BEL - 07 hex >>>> ISO-8859-1 : 07 hex is not a valid character code >>>> UTF-8 : BEL - 07 hex >>>> >>>> ? (GBP currency symbol) >>>> In ISO-8859-1, not in ASCII >>>> ASCII : A3 hex is not a valid ASCII code >>>> UTF-8: ? - A3 hex >>>> ISO-8859-1: ? - A3 hex >>>> >>>> Upper Case C cedilla >>>> In ISO-8859-1, not in ASCII, in UTF-8 as a compatibility codepoint >>>> *and* a composed set of codepoints >>>> ASCII : C7 hex is not a valid ASCII character code >>>> ISO-8859-1 : Upper Case C cedilla - C7 hex >>>> UTF-8 : Upper Case C cedilla (compatibility codepoint) - C7 hex >>>> Unicode preferred Upper Case C cedilla (composed set of codepoints) >>>> Upper case C 0043 hex (Upper case C) >>>> followed by >>>> cedilla 00B8 hex (cedilla) >>>> >>>> 13) For any valid ASCII string *and* for any valid ISO-8859-1 string, >>>> aByteString is completely adequate for editing and display. >>>> >>>> 14) When sorting any valid ASCII string *or* any valid ISO-8859-1 >>>> string, upper and lower case versions of the same character will be >>>> treated differently. >>>> >>>> 15) When sorting any valid ISO-8859-1 string containing >>>> letter+diacritic combination glyphs or ligature combination glyphs, >>>> the glyphs in combination will treated differently to a "plain" glyph >>>> of the character >>>> i.e. "C" and "C cedilla" will be treated very differently. "?" and >>>> "fs" will be treated very differently. >>>> >>>> 16) Different nations have different rules about where diacritic-ed >>>> characted and ligature pairs should be placed when in alphabetical >>>> order. >>>> >>>> 17) Some nations even have multiple standards - e.g. surnames >>>> beginning either "M superscript-c" or "M superscript-a superscript-c" >>>> are treated as beginning equivalently in UK phone directories, but not >>>> in other situations. >>>> >>>> >>>> Some practical upshots >>>> ================== >>>> >>>> 1) Cuis and its ISO-8859-1 encoding is *exactly* the same as UTF-8, >>>> for any single character it considers valid, or any ByteString it has >>>> made up of characters it considers valid. >>>> >>>> 2) Any ByteString is valid UTF-8 in any of Squeak, Pharo, Cuis or any >>>> other Smalltalk with a single byte ByteString following ASCII or >>>> ISO-8859-1. >>>> >>>> 3) Any Smalltalk (or derivative language) using ByteString can >>>> immediately consider it's ByteString as valid UTF-8, as long as it >>>> also considers the ByteSring as valid ASCII and/or ISO-8859-1. >>>> >>>> 4) All of those can be successfully exported to any system using UTF-8 >>>> (e.g. HTML). >>>> >>>> 5) To successfully *accept* all UTF-8 we much be able to do either: >>>> a) accept UTF-8 strings with composed characters >>>> b) convert UTF-8 strings with composed characters into UTF-8 strings >>>> that use *only* compatibility codepoints. >>>> >>>> >>>> Class + protocol proposals >>>> >>>> >>>> >>>> a Utf8CompatibilityString class. >>>> >>>> asByteString - ensure only compatibility codepoints are used. >>>> Ensure it doews not encode characters above 00FF hex. >>>> >>>> asIso8859String - ensures only compatibility codepoints are used, >>>> and that the characters are each valid ISO 8859-1 >>>> >>>> asAsciiString - ensures only characters 00hex - 7F hex are used. >>>> >>>> asUtf8ComposedIso8859String - ensures all compatibility codepoints >>>> are expanded into small OrderedCollections of codepoints >>>> >>>> a Utf8ComposedIso8859String class - will provide sortable and >>>> comparable UTF8 strings of all ASCII and ISO 8859-1 strings. >>>> >>>> Then a Utf8SortableCollection class - a collection of >>>> Utf8ComposedIso8859Strings words and phrases. >>>> >>>> Custom sortBlocks will define the applicable sort order. >>>> >>>> We can create a collection... a Dictionary, thinking about it, of >>>> named, prefabricated sortBlocks. >>>> >>>> This will work for all UTF8 strings of ISO-8859-1 and ASCII strings. >>>> >>>> If anyone has better names for the classes, please let me know. >>>> >>>> If anyone else wants to help >>>> - build these, >>>> - create SUnit tests for these >>>> - write documentation for these >>>> Please let me know. >>>> >>>> n.b. I have had absolutely no experience of Ropes. >>>> >>>> My own background with this stuff: In the early 90's as a Project >>>> Manager implementing office automation systems across a global >>>> company, with offices in the Americas, Western, Eastern and Central >>>> Europe, (including Slavic and Cyrillic users) nations, Japan and >>>> China. The mission-critical application was word-processing. >>>> >>>> Our offices were spread around the globe, and we needed those offices >>>> to successfully exchange documents with their sister offices, and with >>>> the customers in each region the offices were in. >>>> >>>> Unicode was then new, and our platform supplier was the NeXT >>>> Corporation, who had been founder members in of the Unicode Consortium >>>> in 1990. >>>> >>>> So far: I've read the latest version of the Unicode Standard (v8.0). >>>> This is freely downloadable. >>>> I've purchased a paper copy of an earlier release. New releases >>>> typically consist additional codespaces (i.e. alphabets). So old >>>> copies are useful, as well as cheap. (Paper copies of version 4.0 >>>> are available second-hand for < $10 / ?10). >>>> >>>> The typical change with each release is the addition of further >>>> codespaces (i.e alphabets (more or less) ), so you don't lose a lot. >>>> (I'll be going through my V4.0 just to make sure) >>>> >>>> Cheers, >>>> Euan >>>> >>>> >>>> >>>> >>>> On 5 December 2015 at 13:08, stepharo wrote: >>>>> Hi EuanM >>>>> >>>>> Le 4/12/15 12:42, EuanM a ?crit : >>>>>> >>>>>> I'm currently groping my way to seeing how feature-complete our >>>>>> Unicode support is. I am doing this to establish what still needs to >>>>>> be done to provide full Unicode support. >>>>> >>>>> >>>>> this is great. Thanks for pushing this. I wrote and collected some roadmap >>>>> (analyses on different topics) >>>>> on the pharo github project feel free to add this one there. >>>>>> >>>>>> >>>>>> This seems to me to be an area where it would be best to write it >>>>>> once, and then have the same codebase incorporated into the Smalltalks >>>>>> that most share a common ancestry. >>>>>> >>>>>> I am keen to get: equality-testing for strings; sortability for >>>>>> strings which have ligatures and diacritic characters; and correct >>>>>> round-tripping of data. >>>>> >>>>> Go! >>>>> My suggestion is >>>>> start small >>>>> make steady progress >>>>> write tests >>>>> commit often :) >>>>> >>>>> Stef >>>>> >>>>> What is the french phoneBook ordering because this is the first time I hear >>>>> about it. >>>>> >>>>>> >>>>>> Call to action: >>>>>> ========== >>>>>> >>>>>> If you have comments on these proposals - such as "but we already have >>>>>> that facility" or "the reason we do not have these facilities is >>>>>> because they are dog-slow" - please let me know them. >>>>>> >>>>>> If you would like to help out, please let me know. >>>>>> >>>>>> If you have Unicode experience and expertise, and would like to be, or >>>>>> would be willing to be, in the 'council of experts' for this project, >>>>>> please let me know. >>>>>> >>>>>> If you have comments or ideas on anything mentioned in this email >>>>>> >>>>>> In the first instance, the initiative's website will be: >>>>>> http://smalltalk.uk.to/unicode.html >>>>>> >>>>>> I have created a SqueakSource.com project called UnicodeSupport >>>>>> >>>>>> I want to avoid re-inventing any facilities which already exist. >>>>>> Except where they prevent us reaching the goals of: >>>>>> - sortable UTF8 strings >>>>>> - sortable UTF16 strings >>>>>> - equivalence testing of 2 UTF8 strings >>>>>> - equivalence testing of 2 UTF16 strings >>>>>> - round-tripping UTF8 strings through Smalltalk >>>>>> - roundtripping UTF16 strings through Smalltalk. >>>>>> As I understand it, we have limited Unicode support atm. >>>>>> >>>>>> Current state of play >>>>>> =============== >>>>>> ByteString gets converted to WideString when need is automagically >>>>>> detected. >>>>>> >>>>>> Is there anything else that currently exists? >>>>>> >>>>>> Definition of Terms >>>>>> ============== >>>>>> A quick definition of terms before I go any further: >>>>>> >>>>>> Standard terms from the Unicode standard >>>>>> =============================== >>>>>> a compatibility character : an additional encoding of a *normal* >>>>>> character, for compatibility and round-trip conversion purposes. For >>>>>> instance, a 1-byte encoding of a Latin character with a diacritic. >>>>>> >>>>>> Made-up terms >>>>>> ============ >>>>>> a convenience codepoint : a single codepoint which represents an item >>>>>> that is also encoded as a string of codepoints. >>>>>> >>>>>> (I tend to use the terms compatibility character and compatibility >>>>>> codepoint interchangably. The standard only refers to them as >>>>>> compatibility characters. However, the standard is determined to >>>>>> emphasise that characters are abstract and that codepoints are >>>>>> concrete. So I think it is often more useful and productive to think >>>>>> of compatibility or convenience codepoints). >>>>>> >>>>>> a composed character : a character made up of several codepoints >>>>>> >>>>>> Unicode encoding explained >>>>>> ===================== >>>>>> A convenience codepoint can therefore be thought of as a code point >>>>>> used for a character which also has a composed form. >>>>>> >>>>>> The way Unicode works is that sometimes you can encode a character in >>>>>> one byte, sometimes not. Sometimes you can encode it in two bytes, >>>>>> sometimes not. >>>>>> >>>>>> You can therefore have a long stream of ASCII which is single-byte >>>>>> Unicode. If there is an occasional Cyrillic or Greek character in the >>>>>> stream, it would be represented either by a compatibility character or >>>>>> by a multi-byte combination. >>>>>> >>>>>> Using compatibility characters can prevent proper sorting and >>>>>> equivalence testing. >>>>>> >>>>>> Using "pure" Unicode, ie. "normal encodings", can cause compatibility >>>>>> and round-tripping probelms. Although avoiding them can *also* cause >>>>>> compatibility issues and round-tripping problems. >>>>>> >>>>>> Currently my thinking is: >>>>>> >>>>>> a Utf8String class >>>>>> an Ordered collection, with 1 byte characters as the modal element, >>>>>> but short arrays of wider strings where necessary >>>>>> a Utf16String class >>>>>> an Ordered collection, with 2 byte characters as the modal element, >>>>>> but short arrays of wider strings >>>>>> beginning with a 2-byte endianness indicator. >>>>>> >>>>>> Utf8Strings sometimes need to be sortable, and sometimes need to be >>>>>> compatible. >>>>>> >>>>>> So my thinking is that Utf8String will contain convenience codepoints, >>>>>> for round-tripping. And where there are multiple convenience >>>>>> codepoints for a character, that it standardises on one. >>>>>> >>>>>> And that there is a Utf8SortableString which uses *only* normal >>>>>> characters. >>>>>> >>>>>> We then need methods to convert between the two. >>>>>> >>>>>> aUtf8String asUtf8SortableString >>>>>> >>>>>> and >>>>>> >>>>>> aUtf8SortableString asUtf8String >>>>>> >>>>>> >>>>>> Sort orders are culture and context dependent - Sweden and Germany >>>>>> have different sort orders for the same diacritic-ed characters. Some >>>>>> countries have one order in general usage, and another for specific >>>>>> usages, such as phone directories (e.g. UK and France) >>>>>> >>>>>> Similarly for Utf16 : Utf16String and Utf16SortableString and >>>>>> conversion methods >>>>>> >>>>>> A list of sorted words would be a SortedCollection, and there could be >>>>>> pre-prepared sortBlocks for them, e.g. frPhoneBookOrder, deOrder, >>>>>> seOrder, ukOrder, etc >>>>>> >>>>>> along the lines of >>>>>> aListOfWords := SortedCollection sortBlock: deOrder >>>>>> >>>>>> If a word is either a Utf8SortableString, or a well-formed Utf8String, >>>>>> then we can perform equivalence testing on them trivially. >>>>>> >>>>>> To make sure a Utf8String is well formed, we would need to have a way >>>>>> of cleaning up any convenience codepoints which were valid, but which >>>>>> were for a character which has multiple equally-valid alternative >>>>>> convenience codepoints, and for which the string currently had the >>>>>> "wrong" convenience codepoint. (i.e for any character with valid >>>>>> alternative convenience codepoints, we would choose one to be in the >>>>>> well-formed Utf8String, and we would need a method for cleaning the >>>>>> alternative convenience codepoints out of the string, and replacing >>>>>> them with the chosen approved convenience codepoint. >>>>>> >>>>>> aUtf8String cleanUtf8String >>>>>> >>>>>> With WideString, a lot of the issues disappear - except >>>>>> round-tripping(although I'm sure I have seen something recently about >>>>>> 4-byte strings that also have an additional bit. Which would make >>>>>> some Unicode characters 5-bytes long.) >>>>>> >>>>>> >>>>>> (I'm starting to zone out now - if I've overlooked anything - obvious, >>>>>> subtle, or somewhere in between, please let me know) >>>>>> >>>>>> Cheers, >>>>>> Euan >>>>>> >>>>>> >>>>> >>>>> >>> >>>> On 07 Dec 2015, at 04:21, EuanM wrote: >>>> >>>> This a long email. A *lot* of it is encapsulated in the Venn diagram both: >>>> http://smalltalk.uk.to/unicode-utf8.html >>>> and my Smalltalk in Small Steps blog at: >>>> http://smalltalkinsmallsteps.blogspot.co.uk/2015/12/utf-8-for-cuis-pharo-and-squeak.html >>>> >>>> My current thinking, and understanding. >>>> ============================== >>>> >>>> 0) a) ASCII and ISO-8859-1 consist of characters encoded in 1 byte. >>>> b) UTF-8 can encode all of those characters in 1 byte, but can >>>> prefer some of them to be encoded as sequences of multiple bytes. And >>>> can encode additional characters as sequences of multiple bytes. >>>> >>>> 1) Smalltalk has long had multiple String classes. >>>> >>>> 2) Any UTF16 Unicode codepoint which has a codepoint of 00nn hex >>>> is encoded as a UTF-8 codepoint of nn hex. >>>> >>>> 3) All valid ISO-8859-1 characters have a character code between 20 >>>> hex and 7E hex, or between A0 hex and FF hex. >>>> https://en.wikipedia.org/wiki/ISO/IEC_8859-1 >>>> >>>> 4) All valid ASCII characters have a character code between 00 hex and 7E hex. >>>> https://en.wikipedia.org/wiki/ASCII >>>> >>>> >>>> 5) a) All character codes which are defined within ISO-8859-1 and also >>>> defined within ASCII. (i.e. character codes 20 hex to 7E hex) are >>>> defined identically in both. >>>> >>>> b) All printable ASCII characters are defined identically in both >>>> ASCII and ISO-8859-1 >>>> >>>> 6) All character codes defined in ASCII (00 hex to 7E hex) are >>>> defined identically in Unicode UTF-8. >>>> >>>> 7) All character codes defined in ISO-8859-1 (20 hex - 7E hex ; A0 hex >>>> - FF hex ) are defined identically in UTF-8. >>>> >>>> 8) => some Unicode codepoints map to both ASCII and ISO-8859-1. >>>> all ASCII maps 1:1 to Unicode UTF-8 >>>> all ISO-8859-1 maps 1:1 to Unicode UTF-8 >>>> >>>> 9) All ByteStrings elements which are either a valid ISO-8859-1 >>>> character or a valid ASCII character are *also* a valid UTF-8 >>>> character. >>>> >>>> 10) ISO-8859-1 characters representing a character with a diacritic, >>>> or a two-character ligature, have no ASCII equivalent. In Unicode >>>> UTF-8, those character codes which are representing compound glyphs, >>>> are called "compatibility codepoints". >>>> >>>> 11) The preferred Unicode representation of the characters which have >>>> compatibility codepoints is as a a short set of codepoints >>>> representing the characters which are combined together to form the >>>> glyph of the convenience codepoint, as a sequence of bytes >>>> representing the component characters. >>>> >>>> >>>> 12) Some concrete examples: >>>> >>>> A - aka Upper Case A >>>> In ASCII, in ISO 8859-1 >>>> ASCII A - 41 hex >>>> ISO-8859-1 A - 41 hex >>>> UTF-8 A - 41 hex >>>> >>>> BEL (a bell sound, often invoked by a Ctrl-g keyboard chord) >>>> In ASCII, not in ISO 8859-1 >>>> ASCII : BEL - 07 hex >>>> ISO-8859-1 : 07 hex is not a valid character code >>>> UTF-8 : BEL - 07 hex >>>> >>>> ? (GBP currency symbol) >>>> In ISO-8859-1, not in ASCII >>>> ASCII : A3 hex is not a valid ASCII code >>>> UTF-8: ? - A3 hex >>>> ISO-8859-1: ? - A3 hex >>>> >>>> Upper Case C cedilla >>>> In ISO-8859-1, not in ASCII, in UTF-8 as a compatibility codepoint >>>> *and* a composed set of codepoints >>>> ASCII : C7 hex is not a valid ASCII character code >>>> ISO-8859-1 : Upper Case C cedilla - C7 hex >>>> UTF-8 : Upper Case C cedilla (compatibility codepoint) - C7 hex >>>> Unicode preferred Upper Case C cedilla (composed set of codepoints) >>>> Upper case C 0043 hex (Upper case C) >>>> followed by >>>> cedilla 00B8 hex (cedilla) >>>> >>>> 13) For any valid ASCII string *and* for any valid ISO-8859-1 string, >>>> aByteString is completely adequate for editing and display. >>>> >>>> 14) When sorting any valid ASCII string *or* any valid ISO-8859-1 >>>> string, upper and lower case versions of the same character will be >>>> treated differently. >>>> >>>> 15) When sorting any valid ISO-8859-1 string containing >>>> letter+diacritic combination glyphs or ligature combination glyphs, >>>> the glyphs in combination will treated differently to a "plain" glyph >>>> of the character >>>> i.e. "C" and "C cedilla" will be treated very differently. "?" and >>>> "fs" will be treated very differently. >>>> >>>> 16) Different nations have different rules about where diacritic-ed >>>> characted and ligature pairs should be placed when in alphabetical >>>> order. >>>> >>>> 17) Some nations even have multiple standards - e.g. surnames >>>> beginning either "M superscript-c" or "M superscript-a superscript-c" >>>> are treated as beginning equivalently in UK phone directories, but not >>>> in other situations. >>>> >>>> >>>> Some practical upshots >>>> ================== >>>> >>>> 1) Cuis and its ISO-8859-1 encoding is *exactly* the same as UTF-8, >>>> for any single character it considers valid, or any ByteString it has >>>> made up of characters it considers valid. >>>> >>>> 2) Any ByteString is valid UTF-8 in any of Squeak, Pharo, Cuis or any >>>> other Smalltalk with a single byte ByteString following ASCII or >>>> ISO-8859-1. >>>> >>>> 3) Any Smalltalk (or derivative language) using ByteString can >>>> immediately consider it's ByteString as valid UTF-8, as long as it >>>> also considers the ByteSring as valid ASCII and/or ISO-8859-1. >>>> >>>> 4) All of those can be successfully exported to any system using UTF-8 >>>> (e.g. HTML). >>>> >>>> 5) To successfully *accept* all UTF-8 we much be able to do either: >>>> a) accept UTF-8 strings with composed characters >>>> b) convert UTF-8 strings with composed characters into UTF-8 strings >>>> that use *only* compatibility codepoints. >>>> >>>> >>>> Class + protocol proposals >>>> >>>> >>>> >>>> a Utf8CompatibilityString class. >>>> >>>> asByteString - ensure only compatibility codepoints are used. >>>> Ensure it doews not encode characters above 00FF hex. >>>> >>>> asIso8859String - ensures only compatibility codepoints are used, >>>> and that the characters are each valid ISO 8859-1 >>>> >>>> asAsciiString - ensures only characters 00hex - 7F hex are used. >>>> >>>> asUtf8ComposedIso8859String - ensures all compatibility codepoints >>>> are expanded into small OrderedCollections of codepoints >>>> >>>> a Utf8ComposedIso8859String class - will provide sortable and >>>> comparable UTF8 strings of all ASCII and ISO 8859-1 strings. >>>> >>>> Then a Utf8SortableCollection class - a collection of >>>> Utf8ComposedIso8859Strings words and phrases. >>>> >>>> Custom sortBlocks will define the applicable sort order. >>>> >>>> We can create a collection... a Dictionary, thinking about it, of >>>> named, prefabricated sortBlocks. >>>> >>>> This will work for all UTF8 strings of ISO-8859-1 and ASCII strings. >>>> >>>> If anyone has better names for the classes, please let me know. >>>> >>>> If anyone else wants to help >>>> - build these, >>>> - create SUnit tests for these >>>> - write documentation for these >>>> Please let me know. >>>> >>>> n.b. I have had absolutely no experience of Ropes. >>>> >>>> My own background with this stuff: In the early 90's as a Project >>>> Manager implementing office automation systems across a global >>>> company, with offices in the Americas, Western, Eastern and Central >>>> Europe, (including Slavic and Cyrillic users) nations, Japan and >>>> China. The mission-critical application was word-processing. >>>> >>>> Our offices were spread around the globe, and we needed those offices >>>> to successfully exchange documents with their sister offices, and with >>>> the customers in each region the offices were in. >>>> >>>> Unicode was then new, and our platform supplier was the NeXT >>>> Corporation, who had been founder members in of the Unicode Consortium >>>> in 1990. >>>> >>>> So far: I've read the latest version of the Unicode Standard (v8.0). >>>> This is freely downloadable. >>>> I've purchased a paper copy of an earlier release. New releases >>>> typically consist additional codespaces (i.e. alphabets). So old >>>> copies are useful, as well as cheap. (Paper copies of version 4.0 >>>> are available second-hand for < $10 / ?10). >>>> >>>> The typical change with each release is the addition of further >>>> codespaces (i.e alphabets (more or less) ), so you don't lose a lot. >>>> (I'll be going through my V4.0 just to make sure) >>>> >>>> Cheers, >>>> Euan >>>> >>>> >>>> >>>> >>>> On 5 December 2015 at 13:08, stepharo wrote: >>>>> Hi EuanM >>>>> >>>>> Le 4/12/15 12:42, EuanM a ?crit : >>>>>> >>>>>> I'm currently groping my way to seeing how feature-complete our >>>>>> Unicode support is. I am doing this to establish what still needs to >>>>>> be done to provide full Unicode support. >>>>> >>>>> >>>>> this is great. Thanks for pushing this. I wrote and collected some roadmap >>>>> (analyses on different topics) >>>>> on the pharo github project feel free to add this one there. >>>>>> >>>>>> >>>>>> This seems to me to be an area where it would be best to write it >>>>>> once, and then have the same codebase incorporated into the Smalltalks >>>>>> that most share a common ancestry. >>>>>> >>>>>> I am keen to get: equality-testing for strings; sortability for >>>>>> strings which have ligatures and diacritic characters; and correct >>>>>> round-tripping of data. >>>>> >>>>> Go! >>>>> My suggestion is >>>>> start small >>>>> make steady progress >>>>> write tests >>>>> commit often :) >>>>> >>>>> Stef >>>>> >>>>> What is the french phoneBook ordering because this is the first time I hear >>>>> about it. >>>>> >>>>>> >>>>>> Call to action: >>>>>> ========== >>>>>> >>>>>> If you have comments on these proposals - such as "but we already have >>>>>> that facility" or "the reason we do not have these facilities is >>>>>> because they are dog-slow" - please let me know them. >>>>>> >>>>>> If you would like to help out, please let me know. >>>>>> >>>>>> If you have Unicode experience and expertise, and would like to be, or >>>>>> would be willing to be, in the 'council of experts' for this project, >>>>>> please let me know. >>>>>> >>>>>> If you have comments or ideas on anything mentioned in this email >>>>>> >>>>>> In the first instance, the initiative's website will be: >>>>>> http://smalltalk.uk.to/unicode.html >>>>>> >>>>>> I have created a SqueakSource.com project called UnicodeSupport >>>>>> >>>>>> I want to avoid re-inventing any facilities which already exist. >>>>>> Except where they prevent us reaching the goals of: >>>>>> - sortable UTF8 strings >>>>>> - sortable UTF16 strings >>>>>> - equivalence testing of 2 UTF8 strings >>>>>> - equivalence testing of 2 UTF16 strings >>>>>> - round-tripping UTF8 strings through Smalltalk >>>>>> - roundtripping UTF16 strings through Smalltalk. >>>>>> As I understand it, we have limited Unicode support atm. >>>>>> >>>>>> Current state of play >>>>>> =============== >>>>>> ByteString gets converted to WideString when need is automagically >>>>>> detected. >>>>>> >>>>>> Is there anything else that currently exists? >>>>>> >>>>>> Definition of Terms >>>>>> ============== >>>>>> A quick definition of terms before I go any further: >>>>>> >>>>>> Standard terms from the Unicode standard >>>>>> =============================== >>>>>> a compatibility character : an additional encoding of a *normal* >>>>>> character, for compatibility and round-trip conversion purposes. For >>>>>> instance, a 1-byte encoding of a Latin character with a diacritic. >>>>>> >>>>>> Made-up terms >>>>>> ============ >>>>>> a convenience codepoint : a single codepoint which represents an item >>>>>> that is also encoded as a string of codepoints. >>>>>> >>>>>> (I tend to use the terms compatibility character and compatibility >>>>>> codepoint interchangably. The standard only refers to them as >>>>>> compatibility characters. However, the standard is determined to >>>>>> emphasise that characters are abstract and that codepoints are >>>>>> concrete. So I think it is often more useful and productive to think >>>>>> of compatibility or convenience codepoints). >>>>>> >>>>>> a composed character : a character made up of several codepoints >>>>>> >>>>>> Unicode encoding explained >>>>>> ===================== >>>>>> A convenience codepoint can therefore be thought of as a code point >>>>>> used for a character which also has a composed form. >>>>>> >>>>>> The way Unicode works is that sometimes you can encode a character in >>>>>> one byte, sometimes not. Sometimes you can encode it in two bytes, >>>>>> sometimes not. >>>>>> >>>>>> You can therefore have a long stream of ASCII which is single-byte >>>>>> Unicode. If there is an occasional Cyrillic or Greek character in the >>>>>> stream, it would be represented either by a compatibility character or >>>>>> by a multi-byte combination. >>>>>> >>>>>> Using compatibility characters can prevent proper sorting and >>>>>> equivalence testing. >>>>>> >>>>>> Using "pure" Unicode, ie. "normal encodings", can cause compatibility >>>>>> and round-tripping probelms. Although avoiding them can *also* cause >>>>>> compatibility issues and round-tripping problems. >>>>>> >>>>>> Currently my thinking is: >>>>>> >>>>>> a Utf8String class >>>>>> an Ordered collection, with 1 byte characters as the modal element, >>>>>> but short arrays of wider strings where necessary >>>>>> a Utf16String class >>>>>> an Ordered collection, with 2 byte characters as the modal element, >>>>>> but short arrays of wider strings >>>>>> beginning with a 2-byte endianness indicator. >>>>>> >>>>>> Utf8Strings sometimes need to be sortable, and sometimes need to be >>>>>> compatible. >>>>>> >>>>>> So my thinking is that Utf8String will contain convenience codepoints, >>>>>> for round-tripping. And where there are multiple convenience >>>>>> codepoints for a character, that it standardises on one. >>>>>> >>>>>> And that there is a Utf8SortableString which uses *only* normal >>>>>> characters. >>>>>> >>>>>> We then need methods to convert between the two. >>>>>> >>>>>> aUtf8String asUtf8SortableString >>>>>> >>>>>> and >>>>>> >>>>>> aUtf8SortableString asUtf8String >>>>>> >>>>>> >>>>>> Sort orders are culture and context dependent - Sweden and Germany >>>>>> have different sort orders for the same diacritic-ed characters. Some >>>>>> countries have one order in general usage, and another for specific >>>>>> usages, such as phone directories (e.g. UK and France) >>>>>> >>>>>> Similarly for Utf16 : Utf16String and Utf16SortableString and >>>>>> conversion methods >>>>>> >>>>>> A list of sorted words would be a SortedCollection, and there could be >>>>>> pre-prepared sortBlocks for them, e.g. frPhoneBookOrder, deOrder, >>>>>> seOrder, ukOrder, etc >>>>>> >>>>>> along the lines of >>>>>> aListOfWords := SortedCollection sortBlock: deOrder >>>>>> >>>>>> If a word is either a Utf8SortableString, or a well-formed Utf8String, >>>>>> then we can perform equivalence testing on them trivially. >>>>>> >>>>>> To make sure a Utf8String is well formed, we would need to have a way >>>>>> of cleaning up any convenience codepoints which were valid, but which >>>>>> were for a character which has multiple equally-valid alternative >>>>>> convenience codepoints, and for which the string currently had the >>>>>> "wrong" convenience codepoint. (i.e for any character with valid >>>>>> alternative convenience codepoints, we would choose one to be in the >>>>>> well-formed Utf8String, and we would need a method for cleaning the >>>>>> alternative convenience codepoints out of the string, and replacing >>>>>> them with the chosen approved convenience codepoint. >>>>>> >>>>>> aUtf8String cleanUtf8String >>>>>> >>>>>> With WideString, a lot of the issues disappear - except >>>>>> round-tripping(although I'm sure I have seen something recently about >>>>>> 4-byte strings that also have an additional bit. Which would make >>>>>> some Unicode characters 5-bytes long.) >>>>>> >>>>>> >>>>>> (I'm starting to zone out now - if I've overlooked anything - obvious, >>>>>> subtle, or somewhere in between, please let me know) >>>>>> >>>>>> Cheers, >>>>>> Euan >>>>>> >>>>>> >>>>> >>>>> >>>> >>> >>> >> > > From euanmee at gmail.com Mon Dec 7 11:18:36 2015 From: euanmee at gmail.com (EuanM) Date: Mon Dec 7 11:18:38 2015 Subject: [Unicode] collation sequences (Re: [squeak-dev] Unicode Support) In-Reply-To: References: Message-ID: Hannes, The Unicode standard provide compatibility codepoints for compatibility purposes and prefers all characters to be represented composed form - as that way they are comparable and sortable. (Some composed characters have *more than one* compatibility codepoint. The canonical example is the composed character #(0041 030a) which can be represented by EITHER the compatibility codepoint #(00c5) "Latin Capital Letter A with Ring" above OR by #(212b) "Angstrom sign" ) On 7 December 2015 at 08:17, H. Hirzel wrote: > On 12/7/15, EuanM wrote: >> My current thinking for collation sequences: >> >> All strings being collated have had all compatibility codepoints >> expanded into composed sequences. > > What does the Unicode manual suggest? (www.unicode.org reference?) > > > >> >> Strings containing composed sequences and UTF >> -8 strings containing multi-byte characters have these represented by >> a very short ordered collection in place of the single Byte of a >> ByteString. >> >> When we sort characters, words or phrases of strings that contain >> zero compatibility codepoints, we simply pull a pre-defined sortBlock >> out of a Dictionary of pre-defined sortBlocks >> >> >> aDictionaryOfSorts at: ukPhoneBook put: aSortBlock >> or >> aDictionaryOfSorts at: ukPhoneBook put: '[aString representing the >> code of a sortBlock]' . >> >> ASortedCollectionOfUtf8Strings sortBlock: aDictionaryOfSorts at: >> ukPhoneBook >> >> - or some actual working code! :-) >> > > Yes, focusing on this is a real need. > > >> >> On 6 December 2015 at 15:14, H. Hirzel wrote: >>> P.S. The 30-bit value for each character in Squeak/Pharo (if necessary >>> together with an additional language tag) is a potentially very >>> capable infrastructure. Not really used much at the moment. >>> >>> The challenge is to to make _existing_ Unicode-know-how defined >>> elsewhere (e.g. www.unicode.org) available in Squeak/Pharo/Cuis. >>> >>> Most simple cases would be to start with collation sequences in >>> Italian, French, German, Spanish, Portugese. Later on more complex >>> cases like Arabic. >>> >>> --HH >>> >>> On 12/6/15, H. Hirzel wrote: >>>> Hi Euan, >>>> >>>> On 12/4/15, EuanM wrote: >>>>> I'm currently groping my way to seeing how feature-complete our >>>>> Unicode support is. I am doing this to establish what still needs to >>>>> be done to provide full Unicode support. >>>>> >>>>> This seems to me to be an area where it would be best to write it >>>>> once, and then have the same codebase incorporated into the Smalltalks >>>>> that most share a common ancestry. >>>>> >>>>> I am keen to get: equality-testing for strings; sortability for >>>>> strings which have ligatures and diacritic characters; and correct >>>>> round-tripping of data. >>>> >>>> These goals call for a package with SUnit tests which you then can >>>> run on all platforms. This will be a tool to evalutate platforms for >>>> the level of Unicode support. >>>> As mentioned in the thread I would focus on UTF8 only as far as >>>> external files are concerned. >>>> I.E. the test package writes a sample UFT8 file and then reads it to >>>> do the various tests. >>>> I have started doing this for Squeak and Cuis some time ago with a few >>>> tests. >>>> >>>> I am interested in sortability. Round-tripping is fine if you go for >>>> UTF8. >>>> Important of course is which languages you think the package should >>>> work. Some of them are easy, some not. >>>> >>>> This afternoon I did some updates on the Squeak wiki >>>> http://wiki.squeak.org/squeak/recent >>>> >>>> --Hannes >>>> >>>>> >>>>> Call to action: >>>>> ========== >>>>> >>>>> If you have comments on these proposals - such as "but we already have >>>>> that facility" or "the reason we do not have these facilities is >>>>> because they are dog-slow" - please let me know them. >>>>> >>>>> If you would like to help out, please let me know. >>>>> >>>>> If you have Unicode experience and expertise, and would like to be, or >>>>> would be willing to be, in the 'council of experts' for this project, >>>>> please let me know. >>>>> >>>>> If you have comments or ideas on anything mentioned in this email >>>>> >>>>> In the first instance, the initiative's website will be: >>>>> http://smalltalk.uk.to/unicode.html >>>>> >>>>> I have created a SqueakSource.com project called UnicodeSupport >>>>> >>>>> I want to avoid re-inventing any facilities which already exist. >>>>> Except where they prevent us reaching the goals of: >>>>> - sortable UTF8 strings >>>>> - sortable UTF16 strings >>>>> - equivalence testing of 2 UTF8 strings >>>>> - equivalence testing of 2 UTF16 strings >>>>> - round-tripping UTF8 strings through Smalltalk >>>>> - roundtripping UTF16 strings through Smalltalk. >>>>> As I understand it, we have limited Unicode support atm. >>>>> >>>>> Current state of play >>>>> =============== >>>>> ByteString gets converted to WideString when need is automagically >>>>> detected. >>>>> >>>>> Is there anything else that currently exists? >>>>> >>>>> Definition of Terms >>>>> ============== >>>>> A quick definition of terms before I go any further: >>>>> >>>>> Standard terms from the Unicode standard >>>>> =============================== >>>>> a compatibility character : an additional encoding of a *normal* >>>>> character, for compatibility and round-trip conversion purposes. For >>>>> instance, a 1-byte encoding of a Latin character with a diacritic. >>>>> >>>>> Made-up terms >>>>> ============ >>>>> a convenience codepoint : a single codepoint which represents an item >>>>> that is also encoded as a string of codepoints. >>>>> >>>>> (I tend to use the terms compatibility character and compatibility >>>>> codepoint interchangably. The standard only refers to them as >>>>> compatibility characters. However, the standard is determined to >>>>> emphasise that characters are abstract and that codepoints are >>>>> concrete. So I think it is often more useful and productive to think >>>>> of compatibility or convenience codepoints). >>>>> >>>>> a composed character : a character made up of several codepoints >>>>> >>>>> Unicode encoding explained >>>>> ===================== >>>>> A convenience codepoint can therefore be thought of as a code point >>>>> used for a character which also has a composed form. >>>>> >>>>> The way Unicode works is that sometimes you can encode a character in >>>>> one byte, sometimes not. Sometimes you can encode it in two bytes, >>>>> sometimes not. >>>>> >>>>> You can therefore have a long stream of ASCII which is single-byte >>>>> Unicode. If there is an occasional Cyrillic or Greek character in the >>>>> stream, it would be represented either by a compatibility character or >>>>> by a multi-byte combination. >>>>> >>>>> Using compatibility characters can prevent proper sorting and >>>>> equivalence testing. >>>>> >>>>> Using "pure" Unicode, ie. "normal encodings", can cause compatibility >>>>> and round-tripping probelms. Although avoiding them can *also* cause >>>>> compatibility issues and round-tripping problems. >>>>> >>>>> Currently my thinking is: >>>>> >>>>> a Utf8String class >>>>> an Ordered collection, with 1 byte characters as the modal element, >>>>> but short arrays of wider strings where necessary >>>>> a Utf16String class >>>>> an Ordered collection, with 2 byte characters as the modal element, >>>>> but short arrays of wider strings >>>>> beginning with a 2-byte endianness indicator. >>>>> >>>>> Utf8Strings sometimes need to be sortable, and sometimes need to be >>>>> compatible. >>>>> >>>>> So my thinking is that Utf8String will contain convenience codepoints, >>>>> for round-tripping. And where there are multiple convenience >>>>> codepoints for a character, that it standardises on one. >>>>> >>>>> And that there is a Utf8SortableString which uses *only* normal >>>>> characters. >>>>> >>>>> We then need methods to convert between the two. >>>>> >>>>> aUtf8String asUtf8SortableString >>>>> >>>>> and >>>>> >>>>> aUtf8SortableString asUtf8String >>>>> >>>>> >>>>> Sort orders are culture and context dependent - Sweden and Germany >>>>> have different sort orders for the same diacritic-ed characters. Some >>>>> countries have one order in general usage, and another for specific >>>>> usages, such as phone directories (e.g. UK and France) >>>>> >>>>> Similarly for Utf16 : Utf16String and Utf16SortableString and >>>>> conversion methods >>>>> >>>>> A list of sorted words would be a SortedCollection, and there could be >>>>> pre-prepared sortBlocks for them, e.g. frPhoneBookOrder, deOrder, >>>>> seOrder, ukOrder, etc >>>>> >>>>> along the lines of >>>>> aListOfWords := SortedCollection sortBlock: deOrder >>>>> >>>>> If a word is either a Utf8SortableString, or a well-formed Utf8String, >>>>> then we can perform equivalence testing on them trivially. >>>>> >>>>> To make sure a Utf8String is well formed, we would need to have a way >>>>> of cleaning up any convenience codepoints which were valid, but which >>>>> were for a character which has multiple equally-valid alternative >>>>> convenience codepoints, and for which the string currently had the >>>>> "wrong" convenience codepoint. (i.e for any character with valid >>>>> alternative convenience codepoints, we would choose one to be in the >>>>> well-formed Utf8String, and we would need a method for cleaning the >>>>> alternative convenience codepoints out of the string, and replacing >>>>> them with the chosen approved convenience codepoint. >>>>> >>>>> aUtf8String cleanUtf8String >>>>> >>>>> With WideString, a lot of the issues disappear - except >>>>> round-tripping(although I'm sure I have seen something recently about >>>>> 4-byte strings that also have an additional bit. Which would make >>>>> some Unicode characters 5-bytes long.) >>>>> >>>>> >>>>> (I'm starting to zone out now - if I've overlooked anything - obvious, >>>>> subtle, or somewhere in between, please let me know) >>>>> >>>>> Cheers, >>>>> Euan >>>>> >>>>> >>>> >> From euanmee at gmail.com Mon Dec 7 11:22:14 2015 From: euanmee at gmail.com (EuanM) Date: Mon Dec 7 11:22:16 2015 Subject: [squeak-dev] Re: [Unicode] Class Unicode --> data from unicode.org In-Reply-To: References: Message-ID: Gnu-Smalltalk also has the classes UnicodeCharacter UnicodeString which I'm having a good look at. On 7 December 2015 at 08:52, H. Hirzel wrote: > there is a class 'Unicode' both in Squeak and Pharo. > > http://wiki.squeak.org/squeak/6225 > > It surely needs attention as it loads the data from www.unicode.org to > initialize the Unicode knowhow of Squeak / Pharo. From euanmee at gmail.com Mon Dec 7 11:26:17 2015 From: euanmee at gmail.com (EuanM) Date: Mon Dec 7 11:26:19 2015 Subject: [squeak-dev] Re: [Unicode] Update of page 'Multilingual Squeak' In-Reply-To: References: Message-ID: Hi Hannes, It's looking like we should be able to conflate a bunch of existing facilities and code. The various Smalltalks seem to have approached Unicode like the old story of the blind men describing an elephant. There are lots of components when are usefully described and dealt with, and if we can just put them all together it'll will be less new work, and we should get a more portable outcome, and a more robust solution. On 7 December 2015 at 08:21, H. Hirzel wrote: > I updated the page > > 'Multilingual Squeak' > http://wiki.squeak.org/squeak/919 > > and some related pages. > > > Many pages have not been touched for 10 years. > I realized that a lot of infrastructure exists but seemingly people > are not aware of it or do not know how to use it. > > > In particular collation sequences are missing though there are > LanguageEnvironments. Pharo being a fork has most of it. It would be > an issue to check how much it differs and fold back or forth necessary > changes. The infrastructure as such seems to be capable. > > --Hannes From Yoshiki.Ohshima at acm.org Mon Dec 7 11:27:47 2015 From: Yoshiki.Ohshima at acm.org (Yoshiki Ohshima) Date: Mon Dec 7 11:27:51 2015 Subject: [squeak-dev] Re: [Pharo-dev] Unicode Support In-Reply-To: References: <5662E1D8.1070802@free.fr> Message-ID: Hi all, First of all, I'm sorry for leaving Squeak m17n work incomplete. Things are degrading a bit by bit and many things are not working as good as before, unfortunately. That said, there are a few things I'd like to mention: On Sun, Dec 6, 2015 at 7:21 PM, EuanM wrote: > This a long email. A *lot* of it is encapsulated in the Venn diagram both: > http://smalltalk.uk.to/unicode-utf8.html > and my Smalltalk in Small Steps blog at: > http://smalltalkinsmallsteps.blogspot.co.uk/2015/12/utf-8-for-cuis-pharo-and-squeak.html > > My current thinking, and understanding. > ============================== > > 0) a) ASCII and ISO-8859-1 consist of characters encoded in 1 byte. > b) UTF-8 can encode all of those characters in 1 byte, but can > prefer some of them to be encoded as sequences of multiple bytes. And > can encode additional characters as sequences of multiple bytes. > > 1) Smalltalk has long had multiple String classes. Yes, but never meant to make it user visible, in the same sense that a typical user does not (always) have to worry about the difference between SmallInteger and LargeInteger. > 2) Any UTF16 Unicode codepoint which has a codepoint of 00nn hex > is encoded as a UTF-8 codepoint of nn hex. module endianness, but yes. > 7) All character codes defined in ISO-8859-1 (20 hex - 7E hex ; A0 hex > - FF hex ) are defined identically in UTF-8. 3) to 6) are more or less correct but this 7) is not right, if you mean what I think you mean. > 8) => some Unicode codepoints map to both ASCII and ISO-8859-1. > all ASCII maps 1:1 to Unicode UTF-8 > all ISO-8859-1 maps 1:1 to Unicode UTF-8 so this is not correct in the same reason. > 9) All ByteStrings elements which are either a valid ISO-8859-1 > character or a valid ASCII character are *also* a valid UTF-8 > character. No. ByteStrings are meant to be ISO-8859-1. Unfortunately, Squeak does use ByteString to store UTF-8 (my intention was only transiently; in hindsight, it would have been a better convention to use ByteArray for this transient UTF-8 data.) > 11) The preferred Unicode representation of the characters which have > compatibility codepoints is as a a short set of codepoints > representing the characters which are combined together to form the > glyph of the convenience codepoint, as a sequence of bytes > representing the component characters. > > > 12) Some concrete examples: > > ? (GBP currency symbol) > In ISO-8859-1, not in ASCII > ASCII : A3 hex is not a valid ASCII code > UTF-8: ? - A3 hex This is 0xC2 0xA3, not A3. > Upper Case C cedilla > In ISO-8859-1, not in ASCII, in UTF-8 as a compatibility codepoint > *and* a composed set of codepoints > ASCII : C7 hex is not a valid ASCII character code > ISO-8859-1 : Upper Case C cedilla - C7 hex > UTF-8 : Upper Case C cedilla (compatibility codepoint) - C7 hex no, and, > Unicode preferred Upper Case C cedilla (composed set of codepoints) > Upper case C 0043 hex (Upper case C) > followed by > cedilla 00B8 hex (cedilla) no. The codepoint that follows is U+0327, or 0xCC 0xA7 in UTF-8. > 13) For any valid ASCII string *and* for any valid ISO-8859-1 string, > aByteString is completely adequate for editing and display. So unfortunately this is not true. > 14) When sorting any valid ASCII string *or* any valid ISO-8859-1 > string, upper and lower case versions of the same character will be > treated differently. > > 15) When sorting any valid ISO-8859-1 string containing > letter+diacritic combination glyphs or ligature combination glyphs, > the glyphs in combination will treated differently to a "plain" glyph > of the character > i.e. "C" and "C cedilla" will be treated very differently. "?" and > "fs" will be treated very differently. The statement is true but perhaps you mean ss instead of fs? > a Utf8CompatibilityString class. > > asByteString - ensure only compatibility codepoints are used. > Ensure it doews not encode characters above 00FF hex. > > asIso8859String - ensures only compatibility codepoints are used, > and that the characters are each valid ISO 8859-1 > > asAsciiString - ensures only characters 00hex - 7F hex are used. > > asUtf8ComposedIso8859String - ensures all compatibility codepoints > are expanded into small OrderedCollections of codepoints > > a Utf8ComposedIso8859String class - will provide sortable and > comparable UTF8 strings of all ASCII and ISO 8859-1 strings. > > Then a Utf8SortableCollection class - a collection of > Utf8ComposedIso8859Strings words and phrases. > > Custom sortBlocks will define the applicable sort order. > > We can create a collection... a Dictionary, thinking about it, of > named, prefabricated sortBlocks. > > This will work for all UTF8 strings of ISO-8859-1 and ASCII strings. > > If anyone has better names for the classes, please let me know. > > If anyone else wants to help > - build these, > - create SUnit tests for these > - write documentation for these > Please let me know. My feeling is that these extra classes are totally overkill and not necessary. Unfortunately, I have not been following the discussion very closely, but what is the problem that is being solved here? -- -- Yoshiki From frank.shearar at gmail.com Mon Dec 7 11:31:41 2015 From: frank.shearar at gmail.com (Frank Shearar) Date: Mon Dec 7 11:31:44 2015 Subject: [squeak-dev] Re: [Unicode] Class Unicode --> data from unicode.org In-Reply-To: References: Message-ID: Please take a look at the issues around licencing, where GNU Smalltalk is concerned: http://lists.squeakfoundation.org/pipermail/squeak-dev/2008-January/123761.html I'd suggest talking to Paolo Bonzini before continuing. frank On 7 December 2015 at 11:22, EuanM wrote: > Gnu-Smalltalk also has the classes > > UnicodeCharacter > UnicodeString > > which I'm having a good look at. > > On 7 December 2015 at 08:52, H. Hirzel wrote: >> there is a class 'Unicode' both in Squeak and Pharo. >> >> http://wiki.squeak.org/squeak/6225 >> >> It surely needs attention as it loads the data from www.unicode.org to >> initialize the Unicode knowhow of Squeak / Pharo. > From euanmee at gmail.com Mon Dec 7 12:05:12 2015 From: euanmee at gmail.com (EuanM) Date: Mon Dec 7 12:05:16 2015 Subject: [squeak-dev] Re: [Pharo-dev] Unicode Support In-Reply-To: <0F104967-2F55-431F-9EED-0275ABC7CEC1@veloxit.no> References: <5662E1D8.1070802@free.fr> <96B16DCE-A9E7-49E6-B866-E5A846801AA0@stfx.eu> <0F104967-2F55-431F-9EED-0275ABC7CEC1@veloxit.no> Message-ID: Hi Henry, To be honest, at some point I'm going to long for the for the much more succinct semantics of healthcare systems and sports scoring and administration systems again. :-) codepoints are any of *either* - the representation of a component of an abstract character, *or* eg. "A" #(0041) as a component of - the sole representation of the whole of an abstract character *or* of - a representation of an abstract character provided for backwards compatibility which is more properly represented by a series of codepoints representing a composed character e.g. The "A" #(0041) as a codepoint can be: the sole representation of the whole of an abstract character "A" #(0041) The representation of a component of the composed (i.e. preferred) version of the abstract character ? #(0041 030a) ? (#00C5) represents one valid compatibility form of the abstract character ? which is most properly represented by #(0041 030a). ? (#212b) also represents one valid compatibility form of the abstract character ? which is most properly represented by #(0041 030a). With any luck, this satisfies both our semantic understandings of the concept of "codepoint" Would you agree with that? In Unicode, codepoints are *NOT* an abstract numerical representation of a text character. At least not as we generally understand the term "text character" from our experience of non-Unicode character mappings. codepoints represent "*encoded characters*" and "a *text element* ... is represented by a sequence of one or more codepoints". (And the term "text element" is deliberately left undefined in the Unicode standard) Individual codepoints are very often *not* the encoded form of an abstract character that we are interested in. Unless we are communicating to or from another system (Which in some cases is the Smalltalk ByteString class) i.e. in other words *Some* individual codepoints *may* be a representation of a specific *abstract character*, but only in special cases. The general case in Unicode is that Unicode defines (a) representation(s) of a Unicode *abstract character*. The Unicode standard representation of an abstract character is a composed sequence of codepoints, where in some cases that sequence is as short as 1 codepoint. In other cases, Unicode has a compatibility alias of a single codepoint which is *also* a representation of an abstract character There are some cases where an abstract character can be represented by more than one single-codepoint compatibility codepoint. Cheers, Euan On 7 December 2015 at 11:11, Henrik Johansen wrote: > >> On 07 Dec 2015, at 11:51 , EuanM wrote: >> >> And indeed, in principle. >> >> On 7 December 2015 at 10:51, EuanM wrote: >>> Verifying assumptions is the key reason why you should documents like >>> this out for review. >>> >>> Sven - >>> >>> I'm confident I understand the use of UTF-8 in principal. > > I can only second Sven's sentiment that you need to better differentiate code points (an abstract numerical representation of a character, where a set of such mappings > define a charset, such as Unicode), and character encoding forms. (which are how code points are represented in bytes by a defined process such as UTF-8, UTF-16 etc). > > I know you'll probably think I'm arguing semantics again, but these are *important* semantics ;) > > Cheers, > Henry From robert.w.withers at gmail.com Mon Dec 7 12:20:15 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Mon Dec 7 12:20:26 2015 Subject: [squeak-dev] Reed Solomon Blues Message-ID: <5665797F.9050202@gmail.com> Hi Good morning, I am trying to port java code implementing Reed-Solomon encoding with a GaloisField and polynomial system. I am currently testing the easiest Mode, which is 9 data symbols, 15 code symbols and 3 symbols of error. Theer's 2 nibbles to a byte, so a split and join is needed. This all has to do with iteration where the base indexing changes from 0 to 1. I screwed it up in there somewhere and I am struggling to find teh issue. It looks like a complete code review... Since this is core crypto code, it would be an extra blessing if someone else's eyes were on this code in detail.More tests always help too. Would someone be willing to crawl into the java and squeak code to review, learn and qualify/validate? Also, you would be helping me understand all this polynomial math and where my indexing issue may be. The failure in Squeak is in the GenericGFPoly>>#divide: method. It runs forever and the quotient/remainder never reduces it's degree. So one of teh math operations is broken & failing. You can run the EncodingTests I am including a zip of the Java classes I use. This core Reed-Solomon code was written at Google and released open-source. I wrote a test and they pass, so code is working. The latest Cryptography package in the crypto repo is version 48. Please load that and run the CryptoReedSolomonTest to see the infiniteLoop of the polynomial divide. Let me know if you can help. Thanks, Robert -------------- next part -------------- A non-text attachment was scrubbed... Name: reedsolomon-java.zip Type: application/zip Size: 24035 bytes Desc: not available Url : http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151207/4fd8f40e/reedsolomon-java.zip From euanmee at gmail.com Mon Dec 7 12:24:49 2015 From: euanmee at gmail.com (EuanM) Date: Mon Dec 7 12:24:53 2015 Subject: [Pharo-dev] [squeak-dev] Re: [Unicode] Class Unicode --> data from unicode.org In-Reply-To: References: Message-ID: Thanks for the heads up - I'll be sure o stay out of their code and only work with Gnu0-Smalltalk to the extent that we can polyfilla over them. I'll contact Paolo, although his last statement pointed to: http://smalltalk.gnu.org/faq/32 "Does copyright restrict the use of GNU Smalltalk? By Paolo Bonzini - Posted on June 20th, 2007 It does, but in general you don't need to worry much about the licensing of your Smalltalk programs. The class library is licensed under the GNU LGPL. Thus, it permits using the library together with non-free programs. The virtual machine and the bindings to external libraries are licensed under the GNU GPL, with a special exception that allows Smalltalk programs to be linked with the functions exported by the virtual machine and said bindings." and "whatever happens Smalltalk programs are clear and only covered by the GPL." (http://lists.squeakfoundation.org/pipermail/squeak-dev/2008-January/123790.html) The LGPL states "You may convey a Combined Work [i.e. a work containing both LGPL'd source code and non-LGPL's source code] under ***terms of your choice*** that, taken together, effectively do not restrict modification of the portions of the [LGPL'd code] contained in the Combined Work". If we use LGPL'd Smalltalk from Gnu-Smalltalk, we may not close the LGPL'd portion of the code. Was anyone planning on closing the Unicode parts of Cuis/Pharo/Squeak? It *is* an issue with Dolphin's .exe facility, which is one of my own personal use-cases. But for Pharo/Squeak/Cuis it seems to be a non-issue. If anyone has concrete evidence to the contrary, please raise it now. (IANAL but I can read contracts and do risk management of legal issues). If anyone continues to have long-term concerns, let me know. I can get some qualified legal opinions. Tbh, very little of my work is done in the USA's legal jurisdiction. My own risk is low. On 7 December 2015 at 11:31, Frank Shearar wrote: > Please take a look at the issues around licencing, where GNU Smalltalk > is concerned: http://lists.squeakfoundation.org/pipermail/squeak-dev/2008-January/123761.html > > I'd suggest talking to Paolo Bonzini before continuing. > > frank > > On 7 December 2015 at 11:22, EuanM wrote: >> Gnu-Smalltalk also has the classes >> >> UnicodeCharacter >> UnicodeString >> >> which I'm having a good look at. >> >> On 7 December 2015 at 08:52, H. Hirzel wrote: >>> there is a class 'Unicode' both in Squeak and Pharo. >>> >>> http://wiki.squeak.org/squeak/6225 >>> >>> It surely needs attention as it loads the data from www.unicode.org to >>> initialize the Unicode knowhow of Squeak / Pharo. >> > From robert.w.withers at gmail.com Mon Dec 7 12:38:03 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Mon Dec 7 12:38:09 2015 Subject: [squeak-dev] Re: Reed Solomon Blues In-Reply-To: <5665797F.9050202@gmail.com> References: <5665797F.9050202@gmail.com> Message-ID: <56657DAB.6060605@gmail.com> Sorry, forgot to mentionthe test I am using is CryptoReedSolomonTest>>#testEncodeAndDecodeRS_15_9. The other 2 encodeAnd Decode tests have teh wrong sized data at the level they are testing. This data needs to be the size of an interleaved block of 4 chunks that fit the algorithm. Just to inform you, in RS(9)(15) ,the data block is 4 times 9 nibbles or 36 nibbles/18 bytes. Foir some reason I am adding 4 bytes of lenght in there, I think to differentiate use of different sized RS algorithms. Robert On 12/07/2015 07:20 AM, Robert Withers wrote: > Hi Good morning, > > I am trying to port java code implementing Reed-Solomon encoding with > a GaloisField and polynomial system. I am currently testing the > easiest Mode, which is 9 data symbols, 15 code symbols and 3 symbols > of error. Theer's 2 nibbles to a byte, so a split and join is needed. > This all has to do with iteration where the base indexing changes from > 0 to 1. I screwed it up in there somewhere and I am struggling to find > teh issue. It looks like a complete code review... > > Since this is core crypto code, it would be an extra blessing if > someone else's eyes were on this code in detail.More tests always help > too. Would someone be willing to crawl into the java and squeak code > to review, learn and qualify/validate? Also, you would be helping me > understand all this polynomial math and where my indexing issue may be. > > The failure in Squeak is in the GenericGFPoly>>#divide: method. It > runs forever and the quotient/remainder never reduces it's degree. So > one of teh math operations is broken & failing. You can run the > EncodingTests > > I am including a zip of the Java classes I use. This core Reed-Solomon > code was written at Google and released open-source. I wrote a test > and they pass, so code is working. The latest Cryptography package in > the crypto repo is version 48. Please load that and run the > CryptoReedSolomonTest to see the infiniteLoop of the polynomial divide. > > Let me know if you can help. > > Thanks, > Robert > From hannes.hirzel at gmail.com Mon Dec 7 12:47:24 2015 From: hannes.hirzel at gmail.com (H. Hirzel) Date: Mon Dec 7 12:47:27 2015 Subject: [squeak-dev] [Unicode][m17n] Where do I find the m17n paper? (Multilingual Squeak) Message-ID: Hello Yoshiki Your paper on the design and implementation of the m17n package [1] http://www.is.titech.ac.jp/~ohshima/squeak/squeak-multilingual-e.html is no longer accessible. Where can I get it from these days? Regards Hannes [1] Multilingual Squeak http://wiki.squeak.org/squeak/919 From henrik.s.johansen at veloxit.no Mon Dec 7 13:06:33 2015 From: henrik.s.johansen at veloxit.no (Henrik Johansen) Date: Mon Dec 7 13:06:38 2015 Subject: [squeak-dev] Re: [Pharo-dev] Unicode Support In-Reply-To: References: <5662E1D8.1070802@free.fr> <96B16DCE-A9E7-49E6-B866-E5A846801AA0@stfx.eu> <0F104967-2F55-431F-9EED-0275ABC7CEC1@veloxit.no> Message-ID: <52F49745-64AA-447A-A75A-71FB86FCE36E@veloxit.no> Skipped content of type multipart/alternative-------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 842 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151207/afa87813/signature.pgp From stephan at stack.nl Mon Dec 7 13:08:36 2015 From: stephan at stack.nl (Stephan Eggermont) Date: Mon Dec 7 13:08:48 2015 Subject: [squeak-dev] Re: [Pharo-dev] Re: [Unicode] Class Unicode --> data from unicode.org In-Reply-To: References: Message-ID: On 07-12-15 13:24, EuanM wrote: > The class library is licensed under the GNU LGPL. Thus, it permits > using the library together with non-free programs. The incompatibility of LGPL with the kind of binding we do in Smalltalk is a well known issue to the FSF. There is no known solution yet. Stephan From henrik.s.johansen at veloxit.no Mon Dec 7 13:32:36 2015 From: henrik.s.johansen at veloxit.no (Henrik Johansen) Date: Mon Dec 7 13:32:41 2015 Subject: [Pharo-dev] [Unicode] collation sequences (Re: [squeak-dev] Unicode Support) In-Reply-To: References: Message-ID: <041D13DE-3811-4052-A96A-4A80CD44D181@veloxit.no> Skipped content of type multipart/alternative-------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 842 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151207/a7e14f77/signature.pgp From henrik.s.johansen at veloxit.no Mon Dec 7 14:48:34 2015 From: henrik.s.johansen at veloxit.no (Henrik Johansen) Date: Mon Dec 7 14:48:41 2015 Subject: [squeak-dev] Re: [Pharo-dev] Unicode Support In-Reply-To: <52F49745-64AA-447A-A75A-71FB86FCE36E@veloxit.no> References: <5662E1D8.1070802@free.fr> <96B16DCE-A9E7-49E6-B866-E5A846801AA0@stfx.eu> <0F104967-2F55-431F-9EED-0275ABC7CEC1@veloxit.no> <52F49745-64AA-447A-A75A-71FB86FCE36E@veloxit.no> Message-ID: <1C2371C7-AC2E-4679-B757-C6D492C0F2AD@veloxit.no> Skipped content of type multipart/alternative-------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 842 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151207/04f48c71/signature-0001.pgp From btc at openinworld.com Mon Dec 7 14:48:42 2015 From: btc at openinworld.com (Ben Coman) Date: Mon Dec 7 14:49:11 2015 Subject: [squeak-dev] Unicode Support In-Reply-To: References: Message-ID: On Fri, Dec 4, 2015 at 7:42 PM, EuanM wrote: > I'm currently groping my way to seeing how feature-complete our > Unicode support is. I am doing this to establish what still needs to > be done to provide full Unicode support. > > This seems to me to be an area where it would be best to write it > once, and then have the same codebase incorporated into the Smalltalks > that most share a common ancestry. > > I am keen to get: equality-testing for strings; sortability for > strings which have ligatures and diacritic characters; and correct > round-tripping of data. > > Call to action: > ========== > > If you have comments on these proposals - such as "but we already have > that facility" or "the reason we do not have these facilities is > because they are dog-slow" - please let me know them. > > If you would like to help out, please let me know. > > If you have Unicode experience and expertise, and would like to be, or > would be willing to be, in the 'council of experts' for this project, > please let me know. > > If you have comments or ideas on anything mentioned in this email > > In the first instance, the initiative's website will be: > http://smalltalk.uk.to/unicode.html > > I have created a SqueakSource.com project called UnicodeSupport > > I want to avoid re-inventing any facilities which already exist. > Except where they prevent us reaching the goals of: > - sortable UTF8 strings > - sortable UTF16 strings > - equivalence testing of 2 UTF8 strings > - equivalence testing of 2 UTF16 strings > - round-tripping UTF8 strings through Smalltalk > - roundtripping UTF16 strings through Smalltalk. > As I understand it, we have limited Unicode support atm. > > Current state of play > =============== > ByteString gets converted to WideString when need is automagically detected. > > Is there anything else that currently exists? > > Definition of Terms > ============== > A quick definition of terms before I go any further: > > Standard terms from the Unicode standard > =============================== > a compatibility character : an additional encoding of a *normal* > character, for compatibility and round-trip conversion purposes. For > instance, a 1-byte encoding of a Latin character with a diacritic. > > Made-up terms > ============ > a convenience codepoint : a single codepoint which represents an item > that is also encoded as a string of codepoints. > > (I tend to use the terms compatibility character and compatibility > codepoint interchangably. The standard only refers to them as > compatibility characters. However, the standard is determined to > emphasise that characters are abstract and that codepoints are > concrete. So I think it is often more useful and productive to think > of compatibility or convenience codepoints). > > a composed character : a character made up of several codepoints > > Unicode encoding explained > ===================== > A convenience codepoint can therefore be thought of as a code point > used for a character which also has a composed form. > > The way Unicode works is that sometimes you can encode a character in > one byte, sometimes not. Sometimes you can encode it in two bytes, > sometimes not. > > You can therefore have a long stream of ASCII which is single-byte > Unicode. If there is an occasional Cyrillic or Greek character in the > stream, it would be represented either by a compatibility character or > by a multi-byte combination. > > Using compatibility characters can prevent proper sorting and > equivalence testing. > > Using "pure" Unicode, ie. "normal encodings", can cause compatibility > and round-tripping probelms. Although avoiding them can *also* cause > compatibility issues and round-tripping problems. > > Currently my thinking is: > > a Utf8String class > an Ordered collection, with 1 byte characters as the modal element, > but short arrays of wider strings where necessary > a Utf16String class > an Ordered collection, with 2 byte characters as the modal element, > but short arrays of wider strings > beginning with a 2-byte endianness indicator. > > Utf8Strings sometimes need to be sortable, and sometimes need to be compatible. > > So my thinking is that Utf8String will contain convenience codepoints, > for round-tripping. And where there are multiple convenience > codepoints for a character, that it standardises on one. > > And that there is a Utf8SortableString which uses *only* normal characters. > > We then need methods to convert between the two. > > aUtf8String asUtf8SortableString > > and > > aUtf8SortableString asUtf8String > > > Sort orders are culture and context dependent - Sweden and Germany > have different sort orders for the same diacritic-ed characters. Some > countries have one order in general usage, and another for specific > usages, such as phone directories (e.g. UK and France) > > Similarly for Utf16 : Utf16String and Utf16SortableString and > conversion methods > > A list of sorted words would be a SortedCollection, and there could be > pre-prepared sortBlocks for them, e.g. frPhoneBookOrder, deOrder, > seOrder, ukOrder, etc > > along the lines of > aListOfWords := SortedCollection sortBlock: deOrder > > If a word is either a Utf8SortableString, or a well-formed Utf8String, > then we can perform equivalence testing on them trivially. > > To make sure a Utf8String is well formed, we would need to have a way > of cleaning up any convenience codepoints which were valid, but which > were for a character which has multiple equally-valid alternative > convenience codepoints, and for which the string currently had the > "wrong" convenience codepoint. (i.e for any character with valid > alternative convenience codepoints, we would choose one to be in the > well-formed Utf8String, and we would need a method for cleaning the > alternative convenience codepoints out of the string, and replacing > them with the chosen approved convenience codepoint. > > aUtf8String cleanUtf8String > > With WideString, a lot of the issues disappear - except > round-tripping(although I'm sure I have seen something recently about > 4-byte strings that also have an additional bit. Which would make > some Unicode characters 5-bytes long.) > > > (I'm starting to zone out now - if I've overlooked anything - obvious, > subtle, or somewhere in between, please let me know) > > Cheers, > Euan > Good initiative. Here is some info I've bookmarked over time... http://www.joelonsoftware.com/articles/Unicode.html >> The Single Most Important Fact About Encodings - It does not make sense to have a string without knowing what encoding it uses. You can no longer stick your head in the sand and pretend that "plain" text is ASCII. There Ain't No Such Thing As Plain Text. If you have a string, in memory, in a file, or in an email message, you have to know what encoding it is in or you cannot interpret it or display it to users correctly. http://kunststube.net/encoding/ >> So what does it mean for a language to natively support or not support Unicode? It basically refers to whether the language assumes that one character equals one byte or not. >> What does it mean for a language to support Unicode then? Javascript for example supports Unicode. In fact, any string in Javascript is UTF-16 encoded. In fact, it's the only thing Javascript deals with. You cannot have a string in Javascript that is not UTF-16 encoded. Javascript worships Unicode to the extent that there's no facility to deal with any other encoding in the core language. >> Other languages are simply encoding-aware. Internally they store strings in a particular encoding, http://cafe.elharo.com/programming/the-ten-commandments-of-unicode/ >> 1. I am Unicode, thy character set. Thou shalt have no other character sets before me. >> 4. Thou shalt not refer to Unicode as a two-byte character set. >> 6. Thou shalt count and index Unicode characters, not UTF-16 code points. >> 7. Thou shalt use UTF-8 as the preferred encoding wherever possible. https://xkcd.com/1137/ >> ruomuh https://developer.apple.com/library/prerelease/ios/documentation/Swift/Conceptual/Swift_Programming_Language/StringsAndCharacters.html >> Strings and Characters in Swift http://oleb.net/blog/2014/07/swift-strings/ Interesting article, though I don't yet grok it all, I thought it worth sharing... >> Swift?s string implemenation makes working with Unicode easier and significantly less error-prone. >> Strings in Swift are represented by the String type. A String is a collection of Charactervalues. A Swift Character represents one perceived character (what a person thinks of as a single character, called a grapheme). Since Unicode often uses two or more code points(called a grapheme cluster) to form one perceived character, this implies that a Charactercan be composed of multiple Unicode scalar values if they form a single grapheme cluster. (Unicode scalar is the term for any Unicode code point except surrogate pair characters, which are used to encode UTF-16.) >> This change has the potential to prevent many common errors when dealing with string lengths or substrings. It is a huge difference to most other Unicode-aware string libraries (including NSString) where the building blocks of a string are usually UTF-16 code units or single Unicode scalars. cheers -ben From btc at openinworld.com Mon Dec 7 14:51:50 2015 From: btc at openinworld.com (Ben Coman) Date: Mon Dec 7 14:52:12 2015 Subject: Fwd: [squeak-dev] Unicode Support In-Reply-To: References: Message-ID: Sorry to double post, I missed the cross-post. On Fri, Dec 4, 2015 at 7:42 PM, EuanM wrote: > I'm currently groping my way to seeing how feature-complete our > Unicode support is. I am doing this to establish what still needs to > be done to provide full Unicode support. > > This seems to me to be an area where it would be best to write it > once, and then have the same codebase incorporated into the Smalltalks > that most share a common ancestry. > > I am keen to get: equality-testing for strings; sortability for > strings which have ligatures and diacritic characters; and correct > round-tripping of data. > > Call to action: > ========== > > If you have comments on these proposals - such as "but we already have > that facility" or "the reason we do not have these facilities is > because they are dog-slow" - please let me know them. > > If you would like to help out, please let me know. > > If you have Unicode experience and expertise, and would like to be, or > would be willing to be, in the 'council of experts' for this project, > please let me know. > > If you have comments or ideas on anything mentioned in this email > > In the first instance, the initiative's website will be: > http://smalltalk.uk.to/unicode.html > > I have created a SqueakSource.com project called UnicodeSupport > > I want to avoid re-inventing any facilities which already exist. > Except where they prevent us reaching the goals of: > - sortable UTF8 strings > - sortable UTF16 strings > - equivalence testing of 2 UTF8 strings > - equivalence testing of 2 UTF16 strings > - round-tripping UTF8 strings through Smalltalk > - roundtripping UTF16 strings through Smalltalk. > As I understand it, we have limited Unicode support atm. > > Current state of play > =============== > ByteString gets converted to WideString when need is automagically detected. > > Is there anything else that currently exists? > > Definition of Terms > ============== > A quick definition of terms before I go any further: > > Standard terms from the Unicode standard > =============================== > a compatibility character : an additional encoding of a *normal* > character, for compatibility and round-trip conversion purposes. For > instance, a 1-byte encoding of a Latin character with a diacritic. > > Made-up terms > ============ > a convenience codepoint : a single codepoint which represents an item > that is also encoded as a string of codepoints. > > (I tend to use the terms compatibility character and compatibility > codepoint interchangably. The standard only refers to them as > compatibility characters. However, the standard is determined to > emphasise that characters are abstract and that codepoints are > concrete. So I think it is often more useful and productive to think > of compatibility or convenience codepoints). > > a composed character : a character made up of several codepoints > > Unicode encoding explained > ===================== > A convenience codepoint can therefore be thought of as a code point > used for a character which also has a composed form. > > The way Unicode works is that sometimes you can encode a character in > one byte, sometimes not. Sometimes you can encode it in two bytes, > sometimes not. > > You can therefore have a long stream of ASCII which is single-byte > Unicode. If there is an occasional Cyrillic or Greek character in the > stream, it would be represented either by a compatibility character or > by a multi-byte combination. > > Using compatibility characters can prevent proper sorting and > equivalence testing. > > Using "pure" Unicode, ie. "normal encodings", can cause compatibility > and round-tripping probelms. Although avoiding them can *also* cause > compatibility issues and round-tripping problems. > > Currently my thinking is: > > a Utf8String class > an Ordered collection, with 1 byte characters as the modal element, > but short arrays of wider strings where necessary > a Utf16String class > an Ordered collection, with 2 byte characters as the modal element, > but short arrays of wider strings > beginning with a 2-byte endianness indicator. > > Utf8Strings sometimes need to be sortable, and sometimes need to be compatible. > > So my thinking is that Utf8String will contain convenience codepoints, > for round-tripping. And where there are multiple convenience > codepoints for a character, that it standardises on one. > > And that there is a Utf8SortableString which uses *only* normal characters. > > We then need methods to convert between the two. > > aUtf8String asUtf8SortableString > > and > > aUtf8SortableString asUtf8String > > > Sort orders are culture and context dependent - Sweden and Germany > have different sort orders for the same diacritic-ed characters. Some > countries have one order in general usage, and another for specific > usages, such as phone directories (e.g. UK and France) > > Similarly for Utf16 : Utf16String and Utf16SortableString and > conversion methods > > A list of sorted words would be a SortedCollection, and there could be > pre-prepared sortBlocks for them, e.g. frPhoneBookOrder, deOrder, > seOrder, ukOrder, etc > > along the lines of > aListOfWords := SortedCollection sortBlock: deOrder > > If a word is either a Utf8SortableString, or a well-formed Utf8String, > then we can perform equivalence testing on them trivially. > > To make sure a Utf8String is well formed, we would need to have a way > of cleaning up any convenience codepoints which were valid, but which > were for a character which has multiple equally-valid alternative > convenience codepoints, and for which the string currently had the > "wrong" convenience codepoint. (i.e for any character with valid > alternative convenience codepoints, we would choose one to be in the > well-formed Utf8String, and we would need a method for cleaning the > alternative convenience codepoints out of the string, and replacing > them with the chosen approved convenience codepoint. > > aUtf8String cleanUtf8String > > With WideString, a lot of the issues disappear - except > round-tripping(although I'm sure I have seen something recently about > 4-byte strings that also have an additional bit. Which would make > some Unicode characters 5-bytes long.) > > > (I'm starting to zone out now - if I've overlooked anything - obvious, > subtle, or somewhere in between, please let me know) > > Cheers, > Euan > Good initiative. Here is some info I've bookmarked over time... http://www.joelonsoftware.com/articles/Unicode.html >> The Single Most Important Fact About Encodings - It does not make sense to have a string without knowing what encoding it uses. You can no longer stick your head in the sand and pretend that "plain" text is ASCII. There Ain't No Such Thing As Plain Text. If you have a string, in memory, in a file, or in an email message, you have to know what encoding it is in or you cannot interpret it or display it to users correctly. http://kunststube.net/encoding/ >> So what does it mean for a language to natively support or not support Unicode? It basically refers to whether the language assumes that one character equals one byte or not. >> What does it mean for a language to support Unicode then? Javascript for example supports Unicode. In fact, any string in Javascript is UTF-16 encoded. In fact, it's the only thing Javascript deals with. You cannot have a string in Javascript that is not UTF-16 encoded. Javascript worships Unicode to the extent that there's no facility to deal with any other encoding in the core language. >> Other languages are simply encoding-aware. Internally they store strings in a particular encoding, http://cafe.elharo.com/programming/the-ten-commandments-of-unicode/ >> 1. I am Unicode, thy character set. Thou shalt have no other character sets before me. >> 4. Thou shalt not refer to Unicode as a two-byte character set. >> 6. Thou shalt count and index Unicode characters, not UTF-16 code points. >> 7. Thou shalt use UTF-8 as the preferred encoding wherever possible. https://xkcd.com/1137/ >> ruomuh https://developer.apple.com/library/prerelease/ios/documentation/Swift/Conceptual/Swift_Programming_Language/StringsAndCharacters.html >> Strings and Characters in Swift http://oleb.net/blog/2014/07/swift-strings/ Interesting article, though I don't yet grok it all, I thought it worth sharing... >> Swift?s string implemenation makes working with Unicode easier and significantly less error-prone. >> Strings in Swift are represented by the String type. A String is a collection of Charactervalues. A Swift Character represents one perceived character (what a person thinks of as a single character, called a grapheme). Since Unicode often uses two or more code points(called a grapheme cluster) to form one perceived character, this implies that a Charactercan be composed of multiple Unicode scalar values if they form a single grapheme cluster. (Unicode scalar is the term for any Unicode code point except surrogate pair characters, which are used to encode UTF-16.) >> This change has the potential to prevent many common errors when dealing with string lengths or substrings. It is a huge difference to most other Unicode-aware string libraries (including NSString) where the building blocks of a string are usually UTF-16 code units or single Unicode scalars. cheers -ben From btc at openinworld.com Mon Dec 7 15:04:24 2015 From: btc at openinworld.com (Ben Coman) Date: Mon Dec 7 15:04:52 2015 Subject: [squeak-dev] Re: [Pharo-dev] Unicode Support In-Reply-To: <1C2371C7-AC2E-4679-B757-C6D492C0F2AD@veloxit.no> References: <5662E1D8.1070802@free.fr> <96B16DCE-A9E7-49E6-B866-E5A846801AA0@stfx.eu> <0F104967-2F55-431F-9EED-0275ABC7CEC1@veloxit.no> <52F49745-64AA-447A-A75A-71FB86FCE36E@veloxit.no> <1C2371C7-AC2E-4679-B757-C6D492C0F2AD@veloxit.no> Message-ID: On Mon, Dec 7, 2015 at 10:48 PM, Henrik Johansen wrote: > > On 07 Dec 2015, at 2:06 , Henrik Johansen > wrote: > > > codepoints represent "*encoded characters*" > > > No. a codepoint is the numerical value assigned to a character. An "encoded > character" is the way a codepoint is represented in bytes using a given > encoding. > > > You were right on this point, I see I remembered the terminology of this > incorrectly. > http://www.unicode.org/versions/Unicode8.0.0/ch02.pdf figure 2.8 does use > "encoded characters" for the mapping of abstract characters to its > equivalent codepoint (s/ sequences). What I thought it meant is better > described as a codepoint's byte output using an "encoding scheme". > > An accurate description following that terminology, would be that > Pharo/Squeak Strings keep data in UTF32 encoding form, where 1 codepoint = 1 > code unit, dynamically switched between Latin1 (ByteStrings) and UTF32 > (WideStrings) encoding schemes as needed. The implication from Joel's unicode article (linked from my other post) is that whatever encoding we use to store strings, the encoding should not be implicit (i.e. by convention defined outside the image). *Every* string needs to record its encoding. Maybe we should follow Swift [1] and have Characters comprised of multiple codepoints, and/or a String be able to handle a sequence of differently encoded Characters, i.e. String being a mixed sequence of UTF-8, UTF-16, UTF-32 Characters. I have no idea what that wold do for efficiency, but maybe let Moore's Law handle that. [1] http://oleb.net/blog/2014/07/swift-strings/ >> A Swift Character represents one perceived character (what a person thinks of as a single character, called a grapheme). Since Unicode often uses two or more code points(called a grapheme cluster) to form one perceived character, this implies that a Charactercan be composed of multiple Unicode scalar values if they form a single grapheme cluster. cheers -ben > With the same terminology, the difference between a code point, a code unit, > how an encoding scheme represents a codepoint as code units/bytes, are the > concepts it is important to distinguish. > Quite a mouthful though! From robert.w.withers at gmail.com Mon Dec 7 15:11:42 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Mon Dec 7 15:11:46 2015 Subject: [squeak-dev] Name change: Mushroom ( was Re: evolutions of squeakelib & crypto) In-Reply-To: References: <565DCE53.90501@gmail.com> <565DD073.1060409@gmail.com> <565FF8ED.1030606@free.fr> <5660BCFA.4050107@gmail.com> <56616E9C.6040201@gmail.com> <56616F27.4070002@gmail.com> <566174E9.9020001@gmail.com> <566175EA.2090107@gmail.com> <60B913A3-565C-4924-9F82-4E0E96553B72@gmail.com> <5661A6A5.9030902@gmail.com> <224A3FD6-A8DA-45E3-8EDE-AA52133940CC@stfx.eu> <5661B7F1.1080701@gmail.com> <5663A08C.8030602@gmail.com> Message-ID: <5665A1AE.2030706@gmail.com> I renamed the project to Mushroom and I also dumped the encoding work to focus on shutdown, optimization and serialization. Here's the wiki: https://github.com/SqueakCryptographySquad/Mushroom/wiki thanks,Robert On 12/06/2015 01:42 AM, Ben Coman wrote: > On Sun, Dec 6, 2015 at 10:42 AM, Robert Withers > wrote: >> On 12/05/2015 09:24 PM, Ben Coman wrote: >>> On Fri, Dec 4, 2015 at 11:57 PM, Robert Withers >>> wrote: >>>> Now I think you are right on with your observation. Additionally, the >>>> number >>>> of dialects could increase further with Fuel serialization, just port >>>> SecureSession and bits. >>>> >>>> Alright, I came up with a name and it may border on the egregious ... >>>> presenting ... >>>> >>>> "Maelstrom" >>> Great sounding name. However some general advice for the community, >>> since I see a lot of great sounding project names drowned out in the >>> noise of our web-search-centric universe. A litmus test for project >>> naming is using google search to find which return low search results. >>> Today, its more important to be unique than any other attribute of a >>> name. So in general, *dictionary* english words are not the best. >>> One technique is to intentionally mispell the word you like. Here are >>> some comparative examples (note, the surrounding quotes are required >>> to avoid google trying to be helpful and correct the spelling)... >>> >>> "maelstrom" --> 7,480,000 >>> "maelstroom" --> 6,200 >>> "maelstrum" --> 2,280 >>> "maelstruum" --> 7 >>> >>> Lots of interesting other techniques can be found by searching on: >>> techniques to generate brand names or domain names. >>> >>> cheers -ben >> >> I would be happy to change the names to something more unique, though it may >> take a few. Are you suggesting "maelstruum"? >> >> cheers, >> Robert >> >> > *Suggesting* yes, but the choice is yours ;) You need to own it. > > I think maelstruum is certainly memorable with the double "u", but > maybe jarring next the the "m". I'm inclined to maelstroom, since I > associate it with "zoom". I wouldn't necessarily go for the absolute > lowest results. I have an entirely unsubstantiated belief that > anything less than 10,000 gives a reasonable chance to compete once a > user's browsing history is taken into account. Finally you need to > check existing results don't return something abhorrent (I didn't do > this). > > I'd encourage to play around testing on google search. Its quick and > easy to generate and test alternatives. I've added a few more below. > "maelstra" --> 3,560 > "maelstram" --> 504 > "maelstrim" --> 1200 > "maelstroon" --> 58 > "maelstroomi" --> 4 > > btw, I wouldn't swap the order of the "ae" since that would be > susceptible to real typing errors. > > cheers -ben > From mbaehr at email.archlab.tuwien.ac.at Mon Dec 7 15:17:28 2015 From: mbaehr at email.archlab.tuwien.ac.at (=?utf-8?q?Martin_B=C3=A4hr?=) Date: Mon Dec 7 15:18:26 2015 Subject: [Pharo-dev] [squeak-dev] Re: [Unicode] Class Unicode --> data from unicode.org In-Reply-To: References: Message-ID: <1449499341-sup-3116@email.archlab.tuwien.ac.at> Excerpts from EuanM's message of 2015-12-07 13:24:49 +0100: > If we use LGPL'd Smalltalk from Gnu-Smalltalk, we may not close the > LGPL'd portion of the code. > > Was anyone planning on closing the Unicode parts of Cuis/Pharo/Squeak? > > It *is* an issue with Dolphin's .exe facility, which is one of my own > personal use-cases. But for Pharo/Squeak/Cuis it seems to be a > non-issue. If anyone has concrete evidence to the contrary, please > raise it now. i say this as a strong proponent of the GNU GPL, and while i personally would not mind switch licenses, i do not believe that this is something anyone else would want. so the reason you can't use GNU-smalltalk code is because pharo and squeak should remain under the MIT license. if you are creating an add-on package then the license is a non-issue, but if you want the package to become part of the core, then it must be licensed under the MIT license. in other words, sure it is legal to combine MIT and LGPL code, what is not legal is relicensing LGPL code as MIT unless you are the author. and that is what the warning is about. by reading the LGPL code of gnu-smalltalk you risk accidentally copying it. adding LGPL code to the core thus would effectively change the license, and as i said, i don't think anyone wants that. randal schwartz suggests a cleanroom approach, though i personally think that is overkill. if a cleanroom approach were really necessary than i would never be allowed to switch jobs because once i have been working on one companys private code i would be tainted for life, as i could accidentally reproduce the same code in my next job. however, either you get the GNU-smalltalk authors to relicense their unicode classes under the MIT license, or they will need to be rewritten from scratch. i would suggest asking them in the name of having a common code-base for all smalltalks. greetings, martin. -- eKita - the online platform for your entire academic life -- chief engineer eKita.co pike programmer pike.lysator.liu.se caudium.net societyserver.org secretary beijinglug.org mentor fossasia.org foresight developer foresightlinux.org realss.com unix sysadmin Martin B?hr working in china http://societyserver.org/mbaehr/ From btc at openinworld.com Mon Dec 7 15:31:47 2015 From: btc at openinworld.com (Ben Coman) Date: Mon Dec 7 15:32:13 2015 Subject: [Pharo-dev] [squeak-dev] Re: [Unicode] Class Unicode --> data from unicode.org In-Reply-To: <1449499341-sup-3116@email.archlab.tuwien.ac.at> References: <1449499341-sup-3116@email.archlab.tuwien.ac.at> Message-ID: On Mon, Dec 7, 2015 at 11:17 PM, Martin B?hr wrote: > Excerpts from EuanM's message of 2015-12-07 13:24:49 +0100: >> If we use LGPL'd Smalltalk from Gnu-Smalltalk, we may not close the >> LGPL'd portion of the code. >> >> Was anyone planning on closing the Unicode parts of Cuis/Pharo/Squeak? >> >> It *is* an issue with Dolphin's .exe facility, which is one of my own >> personal use-cases. But for Pharo/Squeak/Cuis it seems to be a >> non-issue. If anyone has concrete evidence to the contrary, please >> raise it now. > > i say this as a strong proponent of the GNU GPL, and while i personally would > not mind switch licenses, i do not believe that this is something anyone else > would want. > > so the reason you can't use GNU-smalltalk code is because pharo and squeak > should remain under the MIT license. > > if you are creating an add-on package then the license is a non-issue, but if > you want the package to become part of the core, then it must be licensed under > the MIT license. > > in other words, sure it is legal to combine MIT and LGPL code, what is not > legal is relicensing LGPL code as MIT unless you are the author. and that is > what the warning is about. by reading the LGPL code of gnu-smalltalk you risk > accidentally copying it. > > adding LGPL code to the core thus would effectively change the license, and as > i said, i don't think anyone wants that. > > randal schwartz suggests a cleanroom approach, though i personally think that > is overkill. if a cleanroom approach were really necessary than i would never > be allowed to switch jobs because once i have been working on one companys > private code i would be tainted for life, as i could accidentally reproduce the > same code in my next job. > > however, either you get the GNU-smalltalk authors to relicense their unicode > classes under the MIT license, or they will need to be rewritten from scratch. I read in that thread that contributions to the GNU Smalltalk must have copyright assigned to FSF. So the FSF could be asked to relicense those classes as MIT. You would have to see the text of the assignment clause to know whether the original authors (if you can track them in the code history) can directly relicense it MIT. Now if a MIT licensed unicode library arises that is used by many Smalltalks, GNU Smalltalk might feel inclined to incorporate it downstream under their own license. One reason to argue that MIT is more free than GPL/LGPL, though the latter has the alternate advantage of enforced reciprocity. cheers -ben > i would suggest asking them in the name of having a common code-base for all > smalltalks. > > greetings, martin. From btc at openinworld.com Mon Dec 7 15:38:15 2015 From: btc at openinworld.com (Ben Coman) Date: Mon Dec 7 15:39:01 2015 Subject: [squeak-dev] Name change: Mushroom ( was Re: evolutions of squeakelib & crypto) In-Reply-To: <5665A1AE.2030706@gmail.com> References: <565DCE53.90501@gmail.com> <565DD073.1060409@gmail.com> <565FF8ED.1030606@free.fr> <5660BCFA.4050107@gmail.com> <56616E9C.6040201@gmail.com> <56616F27.4070002@gmail.com> <566174E9.9020001@gmail.com> <566175EA.2090107@gmail.com> <60B913A3-565C-4924-9F82-4E0E96553B72@gmail.com> <5661A6A5.9030902@gmail.com> <224A3FD6-A8DA-45E3-8EDE-AA52133940CC@stfx.eu> <5661B7F1.1080701@gmail.com> <5663A08C.8030602@gmail.com> <5665A1AE.2030706@gmail.com> Message-ID: I like it, but it seems you missed my point :) mushroom --> 117,000,000 is two orders of magnitude more hidden. Anyway, maybe I overplay its significance. cheers -ben On Mon, Dec 7, 2015 at 11:11 PM, Robert Withers wrote: > I renamed the project to Mushroom and I also dumped the encoding work to > focus on shutdown, optimization and serialization. Here's the wiki: > https://github.com/SqueakCryptographySquad/Mushroom/wiki > > thanks,Robert > > > On 12/06/2015 01:42 AM, Ben Coman wrote: >> >> On Sun, Dec 6, 2015 at 10:42 AM, Robert Withers >> wrote: >>> >>> On 12/05/2015 09:24 PM, Ben Coman wrote: >>>> >>>> On Fri, Dec 4, 2015 at 11:57 PM, Robert Withers >>>> wrote: >>>>> >>>>> Now I think you are right on with your observation. Additionally, the >>>>> number >>>>> of dialects could increase further with Fuel serialization, just port >>>>> SecureSession and bits. >>>>> >>>>> Alright, I came up with a name and it may border on the egregious ... >>>>> presenting ... >>>>> >>>>> "Maelstrom" >>>> >>>> Great sounding name. However some general advice for the community, >>>> since I see a lot of great sounding project names drowned out in the >>>> noise of our web-search-centric universe. A litmus test for project >>>> naming is using google search to find which return low search results. >>>> Today, its more important to be unique than any other attribute of a >>>> name. So in general, *dictionary* english words are not the best. >>>> One technique is to intentionally mispell the word you like. Here are >>>> some comparative examples (note, the surrounding quotes are required >>>> to avoid google trying to be helpful and correct the spelling)... >>>> >>>> "maelstrom" --> 7,480,000 >>>> "maelstroom" --> 6,200 >>>> "maelstrum" --> 2,280 >>>> "maelstruum" --> 7 >>>> >>>> Lots of interesting other techniques can be found by searching on: >>>> techniques to generate brand names or domain names. >>>> >>>> cheers -ben >>> >>> >>> I would be happy to change the names to something more unique, though it >>> may >>> take a few. Are you suggesting "maelstruum"? >>> >>> cheers, >>> Robert >>> >>> >> *Suggesting* yes, but the choice is yours ;) You need to own it. >> >> I think maelstruum is certainly memorable with the double "u", but >> maybe jarring next the the "m". I'm inclined to maelstroom, since I >> associate it with "zoom". I wouldn't necessarily go for the absolute >> lowest results. I have an entirely unsubstantiated belief that >> anything less than 10,000 gives a reasonable chance to compete once a >> user's browsing history is taken into account. Finally you need to >> check existing results don't return something abhorrent (I didn't do >> this). >> >> I'd encourage to play around testing on google search. Its quick and >> easy to generate and test alternatives. I've added a few more below. >> "maelstra" --> 3,560 >> "maelstram" --> 504 >> "maelstrim" --> 1200 >> "maelstroon" --> 58 >> "maelstroomi" --> 4 >> >> btw, I wouldn't swap the order of the "ae" since that would be >> susceptible to real typing errors. >> >> cheers -ben >> > > From huw.softdesigns at gmail.com Mon Dec 7 16:12:47 2015 From: huw.softdesigns at gmail.com (Huw Lloyd) Date: Mon Dec 7 16:12:50 2015 Subject: [squeak-dev] Name change: Mushroom ( was Re: evolutions of squeakelib & crypto) In-Reply-To: References: <565DCE53.90501@gmail.com> <565DD073.1060409@gmail.com> <565FF8ED.1030606@free.fr> <5660BCFA.4050107@gmail.com> <56616E9C.6040201@gmail.com> <56616F27.4070002@gmail.com> <566174E9.9020001@gmail.com> <566175EA.2090107@gmail.com> <60B913A3-565C-4924-9F82-4E0E96553B72@gmail.com> <5661A6A5.9030902@gmail.com> <224A3FD6-A8DA-45E3-8EDE-AA52133940CC@stfx.eu> <5661B7F1.1080701@gmail.com> <5663A08C.8030602@gmail.com> <5665A1AE.2030706@gmail.com> Message-ID: It seemed to me that smalltalk has always had the opportunity to avoid the pitfalls of packaging that product-oriented projects are concerned with. This seems tied up with the contradictory relation between the (democratically) relatedness of ideas and the (capitalist) constraints of money. If you look at a page of text, there are no namespaces. There are no use of words with arbitrarily assigned word-forms. The spaces (relations) are between the words (and even between the letters). The accessibility is largely subjective (knowing the meaning of a word), the accessibility is not a package. It seems to me that the practice of giving a package a vivid or redolent name (which may have nothing to do with its etymology) is falling into the package mentality twice over -- not only is a package built, but it is then sold through the imitation of a viral meme. So I would say that its "hidden" nature is of more magnitude than that. But as Ben says. Anyway, maybe I overplay its significance. Best, Huw On 7 December 2015 at 15:38, Ben Coman wrote: > I like it, but it seems you missed my point :) > mushroom --> 117,000,000 is two orders of magnitude more hidden. > Anyway, maybe I overplay its significance. > cheers -ben > > On Mon, Dec 7, 2015 at 11:11 PM, Robert Withers > wrote: > > I renamed the project to Mushroom and I also dumped the encoding work to > > focus on shutdown, optimization and serialization. Here's the wiki: > > https://github.com/SqueakCryptographySquad/Mushroom/wiki > > > > thanks,Robert > > > > > > On 12/06/2015 01:42 AM, Ben Coman wrote: > >> > >> On Sun, Dec 6, 2015 at 10:42 AM, Robert Withers > >> wrote: > >>> > >>> On 12/05/2015 09:24 PM, Ben Coman wrote: > >>>> > >>>> On Fri, Dec 4, 2015 at 11:57 PM, Robert Withers > >>>> wrote: > >>>>> > >>>>> Now I think you are right on with your observation. Additionally, the > >>>>> number > >>>>> of dialects could increase further with Fuel serialization, just port > >>>>> SecureSession and bits. > >>>>> > >>>>> Alright, I came up with a name and it may border on the egregious ... > >>>>> presenting ... > >>>>> > >>>>> "Maelstrom" > >>>> > >>>> Great sounding name. However some general advice for the community, > >>>> since I see a lot of great sounding project names drowned out in the > >>>> noise of our web-search-centric universe. A litmus test for project > >>>> naming is using google search to find which return low search results. > >>>> Today, its more important to be unique than any other attribute of a > >>>> name. So in general, *dictionary* english words are not the best. > >>>> One technique is to intentionally mispell the word you like. Here are > >>>> some comparative examples (note, the surrounding quotes are required > >>>> to avoid google trying to be helpful and correct the spelling)... > >>>> > >>>> "maelstrom" --> 7,480,000 > >>>> "maelstroom" --> 6,200 > >>>> "maelstrum" --> 2,280 > >>>> "maelstruum" --> 7 > >>>> > >>>> Lots of interesting other techniques can be found by searching on: > >>>> techniques to generate brand names or domain names. > >>>> > >>>> cheers -ben > >>> > >>> > >>> I would be happy to change the names to something more unique, though > it > >>> may > >>> take a few. Are you suggesting "maelstruum"? > >>> > >>> cheers, > >>> Robert > >>> > >>> > >> *Suggesting* yes, but the choice is yours ;) You need to own it. > >> > >> I think maelstruum is certainly memorable with the double "u", but > >> maybe jarring next the the "m". I'm inclined to maelstroom, since I > >> associate it with "zoom". I wouldn't necessarily go for the absolute > >> lowest results. I have an entirely unsubstantiated belief that > >> anything less than 10,000 gives a reasonable chance to compete once a > >> user's browsing history is taken into account. Finally you need to > >> check existing results don't return something abhorrent (I didn't do > >> this). > >> > >> I'd encourage to play around testing on google search. Its quick and > >> easy to generate and test alternatives. I've added a few more below. > >> "maelstra" --> 3,560 > >> "maelstram" --> 504 > >> "maelstrim" --> 1200 > >> "maelstroon" --> 58 > >> "maelstroomi" --> 4 > >> > >> btw, I wouldn't swap the order of the "ae" since that would be > >> susceptible to real typing errors. > >> > >> cheers -ben > >> > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151207/526ed50b/attachment.htm From mbaehr at email.archlab.tuwien.ac.at Mon Dec 7 16:06:04 2015 From: mbaehr at email.archlab.tuwien.ac.at (=?utf-8?q?Martin_B=C3=A4hr?=) Date: Mon Dec 7 16:44:29 2015 Subject: [Pharo-dev] [squeak-dev] Re: [Unicode] Class Unicode --> data from unicode.org In-Reply-To: <1449499341-sup-3116@email.archlab.tuwien.ac.at> References: <1449499341-sup-3116@email.archlab.tuwien.ac.at> Message-ID: <1449502640-sup-2085@email.archlab.tuwien.ac.at> Excerpts from Martin B?hr's message of 2015-12-07 16:17:28 +0100: > randal schwartz suggests a cleanroom approach, though i personally think that > is overkill. if a cleanroom approach were really necessary than i would never > be allowed to switch jobs because once i have been working on one companys > private code i would be tainted for life, as i could accidentally reproduce the > same code in my next job. > > however, either you get the GNU-smalltalk authors to relicense their unicode > classes under the MIT license, or they will need to be rewritten from scratch. > > i would suggest asking them in the name of having a common code-base for all > smalltalks. http://lists.squeakfoundation.org/pipermail/squeak-dev/2008-January/123817.html is worth reading in that context: So it's mainly an issue of being willing to cooperate. I think all GNU Smalltalk people (almost all?) are and it's also true at least for some Squeak people. If somebody took a package (several thousands lines of code, etc.) wholesale, ported it to Squeak, and licensed it under MIT on SqueakMap that would be copyright violation. But everything can be done "cum grano salis". Nobody is going to sue you if you copy an interface from GNU Smalltalk, reimplement in Squeak (for fun!), but you did look at the method comments in GNU Smalltalk -- and OMG maybe you got a glimpse of the source code just below! Copyright does not protect having similar implementations. I think we're all for interoperability and for exchange of opinions within the communities. so there you have the open invitation to go and ask... and read this http://lists.squeakfoundation.org/pipermail/squeak-dev/2008-January/123838.html for a good summary on the issue. greetings, martin. -- eKita - the online platform for your entire academic life -- chief engineer eKita.co pike programmer pike.lysator.liu.se caudium.net societyserver.org secretary beijinglug.org mentor fossasia.org foresight developer foresightlinux.org realss.com unix sysadmin Martin B?hr working in china http://societyserver.org/mbaehr/ From hannes.hirzel at gmail.com Mon Dec 7 16:56:56 2015 From: hannes.hirzel at gmail.com (H. Hirzel) Date: Mon Dec 7 16:56:59 2015 Subject: [Pharo-dev] [squeak-dev] Re: [Unicode] Class Unicode --> data from unicode.org In-Reply-To: <1449502640-sup-2085@email.archlab.tuwien.ac.at> References: <1449499341-sup-3116@email.archlab.tuwien.ac.at> <1449502640-sup-2085@email.archlab.tuwien.ac.at> Message-ID: As the issue is about potential reuse of GNU Smalltalk code snippets / classes regarding Unicode the most straightforward thing is to ask Paolo Bonzini (http://smalltalk.gnu.org/blog/bonzinip) if he permits the reuse under an MIT license. So far no code has been identified which would be needed under an MIT license. HTH Hannes On 12/7/15, Martin B?hr wrote: > Excerpts from Martin B?hr's message of 2015-12-07 16:17:28 +0100: >> randal schwartz suggests a cleanroom approach, though i personally think >> that >> is overkill. if a cleanroom approach were really necessary than i would >> never >> be allowed to switch jobs because once i have been working on one >> companys >> private code i would be tainted for life, as i could accidentally >> reproduce the >> same code in my next job. >> >> however, either you get the GNU-smalltalk authors to relicense their >> unicode >> classes under the MIT license, or they will need to be rewritten from >> scratch. >> >> i would suggest asking them in the name of having a common code-base for >> all >> smalltalks. > > http://lists.squeakfoundation.org/pipermail/squeak-dev/2008-January/123817.html > is worth reading in that context: > > So it's mainly an issue of being willing to cooperate. I think all GNU > Smalltalk people (almost all?) are and it's also true at least for some > Squeak people. If somebody took a package (several thousands lines of > code, etc.) wholesale, ported it to Squeak, and licensed it under MIT on > SqueakMap that would be copyright violation. But everything can be done > "cum grano salis". Nobody is going to sue you if you copy an interface > from GNU Smalltalk, reimplement in Squeak (for fun!), but you did look > at the method comments in GNU Smalltalk -- and OMG maybe you got a > glimpse of the source code just below! Copyright does not protect > having similar implementations. > > I think we're all for interoperability and for exchange of opinions > within the communities. > > so there you have the open invitation to go and ask... > > and read this > http://lists.squeakfoundation.org/pipermail/squeak-dev/2008-January/123838.html > for a good summary on the issue. > > greetings, martin. > > -- > eKita - the online platform for your entire academic > life > -- > chief engineer > eKita.co > pike programmer pike.lysator.liu.se caudium.net > societyserver.org > secretary > beijinglug.org > mentor > fossasia.org > foresight developer foresightlinux.org > realss.com > unix sysadmin > Martin B?hr working in china > http://societyserver.org/mbaehr/ > > From hannes.hirzel at gmail.com Mon Dec 7 20:52:09 2015 From: hannes.hirzel at gmail.com (H. Hirzel) Date: Mon Dec 7 20:52:11 2015 Subject: [squeak-dev] Sorting Unicode strings Message-ID: Hello According to http://www.unicode.org/cldr/charts/27/collation/de.html the German phonebook sort order is a A ? ? ?? ?? ? ? ?? ?? ?? ?? b B c C d D e E f F g G h H i I j J k K l L m M n N o O ? ? ?? ?? ? ? ?? ?? ?? ?? ?? ?? p P q Q r R s S ss ? t T u U ? ? ? ? ? ? ? ? ?? ?? ? ? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? v V w W x X y Y z Z I wonder why it looks like this. A lot of characters which never appear in a German text. For Spanish there is 'traditional' and 'standard' http://www.unicode.org/cldr/charts/27/collation/es.html standard a A ? ? b B c C d D e E ? ? f F g G h H i I ? ? j J k K l L m M n N ? ? ?? ?? ?? ?? ?? ?? ?? ?? o O ? ? p P q Q r R s S t T u U ? ? ? ? v V w W x X y Y z Z traditional a A ? ? b B c C ch Ch CH c? C? C? c? C? C? c? C? C? c? C? C? c? C? C? c? C? C? c? C? C? c? C? d D e E ? ? f F g G h H i I ? ? j J k K l L ll Ll LL l? L? L? l? L? L? l? L? L? l? L? L? l? L? L? l? L? L? l? L? L? m M n N ? ? ?? ?? ?? ?? ?? ?? ?? ?? o O ? ? p P q Q r R s S t T u U ? ? ? ? v V w W x X y Y z Z And French is not easily found http://www.unicode.org/cldr/charts/27/collation/index.html or seems to be defined elsewhere http://unicode.org/repos/cldr/tags/release-27/common/collation/fr.xml Suggestions and hints are welcome --Hannes From robert.w.withers at gmail.com Mon Dec 7 22:57:10 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Mon Dec 7 22:57:17 2015 Subject: [squeak-dev] Name change: Mushroom ( was Re: evolutions of squeakelib & crypto) In-Reply-To: References: <565DCE53.90501@gmail.com> <565DD073.1060409@gmail.com> <565FF8ED.1030606@free.fr> <5660BCFA.4050107@gmail.com> <56616E9C.6040201@gmail.com> <56616F27.4070002@gmail.com> <566174E9.9020001@gmail.com> <566175EA.2090107@gmail.com> <60B913A3-565C-4924-9F82-4E0E96553B72@gmail.com> <5661A6A5.9030902@gmail.com> <224A3FD6-A8DA-45E3-8EDE-AA52133940CC@stfx.eu> <5661B7F1.1080701@gmail.com> <5663A08C.8030602@gmail.com> <5665A1AE.2030706@gmail.com> Message-ID: <56660EC6.7030804@gmail.com> Ben, Huw, Todd and Sven, thank you all for your feedback! I suppose I could call the project "CryptOCaps" but for some reason I glommed onto mushroom as the name. Not grandiose and it is somewhat descriptive...a network of secure sessions, each one a mushroom. Ceps are highly valued. We can tag it for the catalog. For sure, we have Seaside, Morphic, Nebraska, Fuel, Alien, Cog, Monticello and that's just the squeak side of unusual naming of projects. I hope that "mushroom" gains a wide reputation as a solid, reliable, secure and performant session layer under the CryptOCaps presentation...I am thinking of splitting the secure session layer from the ocaps presentation layer, but this would require another name choice, so I hesitate...perhaps "Risotto"? What are your thoughts? Best, Robert On 12/07/2015 10:38 AM, Ben Coman wrote: > I like it, but it seems you missed my point :) > mushroom --> 117,000,000 is two orders of magnitude more hidden. > Anyway, maybe I overplay its significance. > cheers -ben > > On Mon, Dec 7, 2015 at 11:11 PM, Robert Withers > wrote: >> I renamed the project to Mushroom and I also dumped the encoding work to >> focus on shutdown, optimization and serialization. Here's the wiki: >> https://github.com/SqueakCryptographySquad/Mushroom/wiki >> >> thanks,Robert >> >> >> On 12/06/2015 01:42 AM, Ben Coman wrote: >>> On Sun, Dec 6, 2015 at 10:42 AM, Robert Withers >>> wrote: >>>> On 12/05/2015 09:24 PM, Ben Coman wrote: >>>>> On Fri, Dec 4, 2015 at 11:57 PM, Robert Withers >>>>> wrote: >>>>>> Now I think you are right on with your observation. Additionally, the >>>>>> number >>>>>> of dialects could increase further with Fuel serialization, just port >>>>>> SecureSession and bits. >>>>>> >>>>>> Alright, I came up with a name and it may border on the egregious ... >>>>>> presenting ... >>>>>> >>>>>> "Maelstrom" >>>>> Great sounding name. However some general advice for the community, >>>>> since I see a lot of great sounding project names drowned out in the >>>>> noise of our web-search-centric universe. A litmus test for project >>>>> naming is using google search to find which return low search results. >>>>> Today, its more important to be unique than any other attribute of a >>>>> name. So in general, *dictionary* english words are not the best. >>>>> One technique is to intentionally mispell the word you like. Here are >>>>> some comparative examples (note, the surrounding quotes are required >>>>> to avoid google trying to be helpful and correct the spelling)... >>>>> >>>>> "maelstrom" --> 7,480,000 >>>>> "maelstroom" --> 6,200 >>>>> "maelstrum" --> 2,280 >>>>> "maelstruum" --> 7 >>>>> >>>>> Lots of interesting other techniques can be found by searching on: >>>>> techniques to generate brand names or domain names. >>>>> >>>>> cheers -ben >>>> >>>> I would be happy to change the names to something more unique, though it >>>> may >>>> take a few. Are you suggesting "maelstruum"? >>>> >>>> cheers, >>>> Robert >>>> >>>> >>> *Suggesting* yes, but the choice is yours ;) You need to own it. >>> >>> I think maelstruum is certainly memorable with the double "u", but >>> maybe jarring next the the "m". I'm inclined to maelstroom, since I >>> associate it with "zoom". I wouldn't necessarily go for the absolute >>> lowest results. I have an entirely unsubstantiated belief that >>> anything less than 10,000 gives a reasonable chance to compete once a >>> user's browsing history is taken into account. Finally you need to >>> check existing results don't return something abhorrent (I didn't do >>> this). >>> >>> I'd encourage to play around testing on google search. Its quick and >>> easy to generate and test alternatives. I've added a few more below. >>> "maelstra" --> 3,560 >>> "maelstram" --> 504 >>> "maelstrim" --> 1200 >>> "maelstroon" --> 58 >>> "maelstroomi" --> 4 >>> >>> btw, I wouldn't swap the order of the "ae" since that would be >>> susceptible to real typing errors. >>> >>> cheers -ben >>> >> From jecel at merlintec.com Tue Dec 8 00:26:52 2015 From: jecel at merlintec.com (Jecel Assumpcao Jr.) Date: Mon Dec 7 23:27:06 2015 Subject: [squeak-dev] Name change: Mushroom ( was Re: evolutions of squeakelib & crypto) In-Reply-To: <56660EC6.7030804@gmail.com> References: <565DCE53.90501@gmail.com> <565DD073.1060409@gmail.com> <565FF8ED.1030606@free.fr> <5660BCFA.4050107@gmail.com> <56616E9C.6040201@gmail.com> <56616F27.4070002@gmail.com> <566174E9.9020001@gmail.com> <566175EA.2090107@gmail.com> <60B913A3-565C-4924-9F82-4E0E96553B72@gmail.com> <5661A6A5.9030902@gmail.com> <224A3FD6-A8DA-45E3-8EDE-AA52133940CC@stfx.eu> <5661B7F1.1080701@gmail.com> <5663A08C.8030602@gmail.com> <5665A1AE.2030706@gmail.com> <56660EC6.7030804@gmail.com> Message-ID: Robert Withers wrote: > Ben, Huw, Todd and Sven, thank you all for your feedback! I suppose I > could call the project "CryptOCaps" but for some reason I glommed onto > mushroom as the name. Not grandiose and it is somewhat descriptive...a > network of secure sessions, each one a mushroom. Ceps are highly valued. > We can tag it for the catalog. Though pretty old, the Mushroom Smalltalk computer was very well known at one time: http://www.wolczko.com/mushroom/ Though one is a hardware project and yours is software, there is still the potential for some confusion. -- Jecel From tim at rowledge.org Tue Dec 8 00:01:32 2015 From: tim at rowledge.org (tim Rowledge) Date: Tue Dec 8 00:01:36 2015 Subject: [squeak-dev] HandMorph position gridding Message-ID: <3CD8C5E4-4270-456C-8A3E-3B47BDBA00CB@rowledge.org> After much fun, I have a plausible mechanism that introduces a way to have a temporary cursor move on a grid rather than in single pixel steps. This is useful for painting programs that want to work on scaled up bitmaps, or perhaps for drawing programs wanting to place morphs on a grid (say, electrical component icons on a virtual breadboard?). It requires 4 new HandMorph methods, 1 small change to HandMorph>position: and either the use of a couple of properties in HandMorph or a couple of additional instvars. Does anyone have objections to this being added? Speak soon so I know if I have to try yet another tactic. tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Useful random insult:- Doesn't just know nothing; doesn't even suspect much. From robert.w.withers at gmail.com Tue Dec 8 00:25:00 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Tue Dec 8 00:25:09 2015 Subject: [squeak-dev] Name change: Mushroom ( was Re: evolutions of squeakelib & crypto) In-Reply-To: <566615d0.13e48c0a.b2cc4.63bcSMTPIN_ADDED_MISSING@mx.google.com> References: <565DCE53.90501@gmail.com> <565DD073.1060409@gmail.com> <565FF8ED.1030606@free.fr> <5660BCFA.4050107@gmail.com> <56616E9C.6040201@gmail.com> <56616F27.4070002@gmail.com> <566174E9.9020001@gmail.com> <566175EA.2090107@gmail.com> <60B913A3-565C-4924-9F82-4E0E96553B72@gmail.com> <5661A6A5.9030902@gmail.com> <224A3FD6-A8DA-45E3-8EDE-AA52133940CC@stfx.eu> <5661B7F1.1080701@gmail.com> <5663A08C.8030602@gmail.com> <5665A1AE.2030706@gmail.com> <56660EC6.7030804@gmail.com> <566615d0.13e48c0a.b2cc4.63bcSMTPIN_ADDED_MISSING@mx.google.com> Message-ID: <5666235C.9000906@gmail.com> On 12/07/2015 07:26 PM, Jecel Assumpcao Jr. wrote: > Robert Withers wrote: >> Ben, Huw, Todd and Sven, thank you all for your feedback! I suppose I >> could call the project "CryptOCaps" but for some reason I glommed onto >> mushroom as the name. Not grandiose and it is somewhat descriptive...a >> network of secure sessions, each one a mushroom. Ceps are highly valued. >> We can tag it for the catalog. > Though pretty old, the Mushroom Smalltalk computer was very well known > at one time: > > http://www.wolczko.com/mushroom/ > > Though one is a hardware project and yours is software, there is still > the potential for some confusion. > > -- Jecel Oh dear, that's a potential naming conflict. Perhaps this is where the idea came from, as I have read about this Mushroom Project long ago. As this hardware project is not longer active, repurposing the name may be reasonable but I cannot say wityhout that authority. There are 2 layers, session & presentation, so perhaps 2 names are needed and the one project needs splitting. And there is the consensus issue. Perhaps I should use technical names and be done with it: SecureSession & CryptOCaps. Well, I will leave off changing again until agreed to names are established, as I don't know what to do about it, but I welcome suggestions... Maelstrom, Mushroom, Crystalline & Phosphor, SecureSession & CryptOCaps, pther suggestions? Robert From mbaehr at email.archlab.tuwien.ac.at Tue Dec 8 02:28:43 2015 From: mbaehr at email.archlab.tuwien.ac.at (=?utf-8?q?Martin_B=C3=A4hr?=) Date: Tue Dec 8 02:28:47 2015 Subject: [squeak-dev] Sorting Unicode strings In-Reply-To: References: Message-ID: <1449541607-sup-6177@email.archlab.tuwien.ac.at> Excerpts from H. Hirzel's message of 2015-12-07 21:52:09 +0100: > According to http://www.unicode.org/cldr/charts/27/collation/de.html the German > phonebook sort order is > > a A ? ? ?? ?? ? ? ?? ?? ?? ?? b B c C d D e E f F g G h H i I j J k K > l L m M n N o O ? ? ?? ?? ? ? ?? ?? ?? ?? ?? ?? p P q Q r R s S ss ? t > T u U ? ? ? ? ? ? ? ? ?? ?? ? ? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? v > V w W x X y Y z Z > > I wonder why it looks like this. A lot of characters which never > appear in a German text. but they might appear in names in a german phonebook (or any other list of names) greetings, martin. -- eKita - the online platform for your entire academic life -- chief engineer eKita.co pike programmer pike.lysator.liu.se caudium.net societyserver.org secretary beijinglug.org mentor fossasia.org foresight developer foresightlinux.org realss.com unix sysadmin Martin B?hr working in china http://societyserver.org/mbaehr/ From lewis at mail.msen.com Tue Dec 8 02:56:14 2015 From: lewis at mail.msen.com (David T. Lewis) Date: Tue Dec 8 02:56:16 2015 Subject: [squeak-dev] HandMorph position gridding In-Reply-To: <3CD8C5E4-4270-456C-8A3E-3B47BDBA00CB@rowledge.org> References: <3CD8C5E4-4270-456C-8A3E-3B47BDBA00CB@rowledge.org> Message-ID: <20151208025614.GA96679@shell.msen.com> On Mon, Dec 07, 2015 at 04:01:32PM -0800, tim Rowledge wrote: > After much fun, I have a plausible mechanism that introduces a way to have a temporary cursor move on a grid rather than in single pixel steps. This is useful for painting programs that want to work on scaled up bitmaps, or perhaps for drawing programs wanting to place morphs on a grid (say, electrical component icons on a virtual breadboard?). It requires 4 new HandMorph methods, 1 small change to HandMorph>position: and either the use of a couple of properties in HandMorph or a couple of additional instvars. > > Does anyone have objections to this being added? Speak soon so I know if I have to try yet another tactic. > Sounds reasonable, can you post it to the inbox please? Thanks, Dave From commits at source.squeak.org Tue Dec 8 04:25:36 2015 From: commits at source.squeak.org (commits@source.squeak.org) Date: Tue Dec 8 04:25:37 2015 Subject: [squeak-dev] The Inbox: Morphic-tpr.1057.mcz Message-ID: tim Rowledge uploaded a new version of Morphic to project The Inbox: http://source.squeak.org/inbox/Morphic-tpr.1057.mcz ==================== Summary ==================== Name: Morphic-tpr.1057 Author: tpr Time: 7 December 2015, 8:24:53.359 pm UUID: c685257b-5ee7-49b3-a6b7-5255f28cd47f Ancestors: Morphic-dtl.1056 Add the STWCPW version of cursor gridding to HandMorph. It may very well be better to add two instance variables rather than use a couple of properties to hold the state. There may well be other methods needing changes. To use: start by sending gridTo: grid origin: origin to the hand any subsequent temporaryCursor... will be forced to screeen co-ordinates that are on grid sized steps with an offset of the origin. gridPointRaw provides the frid limited position of the current hand. Older events are *not* gridded - this is one of the places where there may be more to do. =============== Diff against Morphic-dtl.1056 =============== Item was added: + ----- Method: HandMorph>>gridPointRaw (in category 'gridded cursor') ----- + gridPointRaw + "return my latest position gridded" + ^self griddedPoint: self currentEvent position! Item was added: + ----- Method: HandMorph>>gridTo:origin: (in category 'gridded cursor') ----- + gridTo: grid origin: offset + "set a couple of properties to specify gridding for the temporaryCursor; instvars would be nicer" + self setProperty: #gridStep toValue: grid; + setProperty: #gridOffset toValue: offset! Item was added: + ----- Method: HandMorph>>griddedPoint: (in category 'gridded cursor') ----- + griddedPoint: aPoint + "return the equivalent point snapped to the grid, if indeed any gridding is set" + self valueOfProperty: #gridStep ifPresentDo: [:grid| |offset| + offset := self valueOfProperty: #gridOffset ifAbsent: [0@0]. + ^ offset + (aPoint + (grid //2) - offset truncateTo: grid)]. + ^aPoint! Item was changed: ----- Method: HandMorph>>position: (in category 'geometry') ----- position: aPoint "Overridden to align submorph origins to the grid if gridding is on." | adjustedPosition delta box | adjustedPosition := aPoint. + temporaryCursor ifNotNil: [adjustedPosition := (self griddedPoint: adjustedPosition) + temporaryCursorOffset]. - temporaryCursor ifNotNil: [adjustedPosition := adjustedPosition + temporaryCursorOffset]. "Copied from Morph to avoid owner layoutChanged" "Change the position of this morph and and all of its submorphs." delta := adjustedPosition - bounds topLeft. delta isZero ifTrue: [^ self]. "Null change" box := self fullBounds. (delta dotProduct: delta) > 100 ifTrue:[ "e.g., more than 10 pixels moved" self invalidRect: box. self invalidRect: (box translateBy: delta). ] ifFalse:[ self invalidRect: (box merge: (box translateBy: delta)). ]. self privateFullMoveBy: delta. ! Item was added: + ----- Method: HandMorph>>turnOffGridding (in category 'gridded cursor') ----- + turnOffGridding + "remove the gridding properties to stop gridding" + self removeProperty: #gridStep; removeProperty: #gridOffset! From tim at rowledge.org Tue Dec 8 04:26:36 2015 From: tim at rowledge.org (tim Rowledge) Date: Tue Dec 8 04:26:39 2015 Subject: [squeak-dev] HandMorph position gridding In-Reply-To: <20151208025614.GA96679@shell.msen.com> References: <3CD8C5E4-4270-456C-8A3E-3B47BDBA00CB@rowledge.org> <20151208025614.GA96679@shell.msen.com> Message-ID: > On 07-12-2015, at 6:56 PM, David T. Lewis wrote: > > On Mon, Dec 07, 2015 at 04:01:32PM -0800, tim Rowledge wrote: >> After much fun, I have a plausible mechanism that introduces a way to have a temporary cursor move on a grid rather than in single pixel steps. This is useful for painting programs that want to work on scaled up bitmaps, or perhaps for drawing programs wanting to place morphs on a grid (say, electrical component icons on a virtual breadboard?). It requires 4 new HandMorph methods, 1 small change to HandMorph>position: and either the use of a couple of properties in HandMorph or a couple of additional instvars. >> >> Does anyone have objections to this being added? Speak soon so I know if I have to try yet another tactic. >> > > Sounds reasonable, can you post it to the inbox please? Done. I hope. tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Long computations that yield zero are probably all for naught. From commits at source.squeak.org Tue Dec 8 05:00:51 2015 From: commits at source.squeak.org (commits@source.squeak.org) Date: Tue Dec 8 05:00:53 2015 Subject: [squeak-dev] Squeak 4.6: Environments-cwp.58.mcz Message-ID: Colin Putney uploaded a new version of Environments to project Squeak 4.6: http://source.squeak.org/squeak46/Environments-cwp.58.mcz ==================== Summary ==================== Name: Environments-cwp.58 Author: cwp Time: 7 December 2015, 9:00:03.154 pm UUID: d851b52d-ed87-40d7-88d4-d196ea0e09da Ancestors: Environments-cmm.57 Backported From: Environments-cwp.60 Backport image segment serializaton fix from trunk =============== Diff against Environments-cmm.57 =============== Item was changed: ----- Method: Binding>>objectForDataStream: (in category 'as yet unclassified') ----- + objectForDataStream: refStrm + "I am about to be written on an object file. I am a global, so write a proxy that + will hook up with the same resource in the destination system." + + | dp | + dp := DiskProxy + global: #Smalltalk + selector: #associationOrUndeclaredAt: + args: (Array with: key). + refStrm replace: self with: dp. + ^ dp! - objectForDataStream: refStream - "It's not yet clear how serialization should work in the presence of environments" - - self shouldBeImplemented.! From commits at source.squeak.org Tue Dec 8 05:14:36 2015 From: commits at source.squeak.org (commits@source.squeak.org) Date: Tue Dec 8 05:14:38 2015 Subject: [squeak-dev] Squeak 4.6: System-cwp.754.mcz Message-ID: Colin Putney uploaded a new version of System to project Squeak 4.6: http://source.squeak.org/squeak46/System-cwp.754.mcz ==================== Summary ==================== Name: System-cwp.754 Author: cwp Time: 7 December 2015, 9:14:14.615 pm UUID: 68390bf4-02ab-46d4-b3bc-96bf2723e7d5 Ancestors: System-topa.753 Backported From: System-cwp.782 Backport image segment serialization fixes from trunk =============== Diff against System-topa.753 =============== Item was removed: - ----- Method: Association>>objectForDataStream: (in category '*System-Object Storage-objects from disk') ----- - objectForDataStream: refStrm - | dp | - "I am about to be written on an object file. If I am a known global, write a proxy that will hook up with the same resource in the destination system." - self flag: #environments. - - ^ (Smalltalk globals associationAt: key ifAbsent: [nil]) == self - ifTrue: [dp := DiskProxy global: #Smalltalk selector: #associationOrUndeclaredAt: - args: (Array with: key). - refStrm replace: self with: dp. - dp] - ifFalse: [self]! Item was changed: ----- Method: DiskProxy>>comeFullyUpOnReload: (in category 'i/o') ----- comeFullyUpOnReload: smartRefStream "Internalize myself into a fully alive object after raw loading from a DataStream. (See my class comment.) DataStream will substitute the object from this eval for the DiskProxy." + | globalObj symbol pr nn arrayIndex env | - | globalObj symbol pr nn arrayIndex | - self flag: #environments. symbol := globalObjectName. "See if class is mapped to another name" (smartRefStream respondsTo: #renamed) ifTrue: [ "If in outPointers in an ImageSegment, remember original class name. See mapClass:installIn:. Would be lost otherwise." ((thisContext sender sender sender sender sender sender sender sender receiver class == ImageSegment) and: [ thisContext sender sender sender sender method == (DataStream compiledMethodAt: #readArray)]) ifTrue: [ arrayIndex := (thisContext sender sender sender sender) tempAt: 4. "index var in readArray. Later safer to find i on stack of context." smartRefStream renamedConv at: arrayIndex put: symbol]. "save original name" symbol := smartRefStream renamed at: symbol ifAbsent: [symbol]]. "map" + env := Environment current. + globalObj := env valueOf: symbol ifAbsent: [ - globalObj := Smalltalk at: symbol ifAbsent: [ preSelector == nil & (constructorSelector = #yourself) ifTrue: [ Transcript cr; show: symbol, ' is undeclared.'. + env undeclare: symbol. - (Undeclared includesKey: symbol) ifTrue: [^ Undeclared at: symbol]. - Undeclared at: symbol put: nil. ^ nil]. ^ self error: 'Global "', symbol, '" not found']. ((symbol == #World) and: [Smalltalk isMorphic not]) ifTrue: [ self inform: 'These objects will work better if opened in a Morphic World. Dismiss and reopen all menus.']. preSelector ifNotNil: [ Symbol hasInterned: preSelector ifTrue: [:selector | [globalObj := globalObj perform: selector] on: Error do: [:ex | ex messageText = 'key not found' ifTrue: [^ nil]. ^ ex signal]] ]. symbol == #Project ifTrue: [ (constructorSelector = #fromUrl:) ifTrue: [ nn := (constructorArgs first findTokens: '/') last. nn := (nn findTokens: '.|') first. pr := Project named: nn. ^ pr ifNil: [self] ifNotNil: [pr]]. pr := globalObj perform: constructorSelector withArguments: constructorArgs. ^ pr ifNil: [self] ifNotNil: [pr]]. "keep the Proxy if Project does not exist" constructorSelector ifNil: [^ globalObj]. Symbol hasInterned: constructorSelector ifTrue: [:selector | [^ globalObj perform: selector withArguments: constructorArgs] on: Error do: [:ex | ex messageText = 'key not found' ifTrue: [^ nil]. ^ ex signal] ]. "args not checked against Renamed" ^ nil "was not in proper form"! From commits at source.squeak.org Tue Dec 8 05:16:27 2015 From: commits at source.squeak.org (commits@source.squeak.org) Date: Tue Dec 8 05:16:29 2015 Subject: [squeak-dev] Squeak 4.6: System-cwp.755.mcz Message-ID: Colin Putney uploaded a new version of System to project Squeak 4.6: http://source.squeak.org/squeak46/System-cwp.755.mcz ==================== Summary ==================== Name: System-cwp.755 Author: cwp Time: 7 December 2015, 9:16:07.606 pm UUID: 8240fbad-2095-45dc-97af-65ab0d381a52 Ancestors: System-cwp.754, System-dtl.754 Merge =============== Diff against System-cwp.754 =============== Item was changed: ----- Method: SmalltalkImage>>recreateSpecialObjectsArray (in category 'special objects') ----- recreateSpecialObjectsArray "Smalltalk recreateSpecialObjectsArray" "To external package developers: **** DO NOT OVERRIDE THIS METHOD. ***** If you are writing a plugin and need additional special object(s) for your own use, use addGCRoot() function and use own, separate special objects registry " "The Special Objects Array is an array of objects used by the Squeak virtual machine. Its contents are critical and accesses to it by the VM are unchecked, so don't even think of playing here unless you know what you are doing." | newArray | newArray := Array new: 58. "Nil false and true get used throughout the interpreter" newArray at: 1 put: nil. newArray at: 2 put: false. newArray at: 3 put: true. "This association holds the active process (a ProcessScheduler)" newArray at: 4 put: (self specialObjectsArray at: 4) "(self bindingOf: #Processor) but it answers an Alias". "Numerous classes below used for type checking and instantiation" newArray at: 5 put: Bitmap. newArray at: 6 put: SmallInteger. newArray at: 7 put: ByteString. newArray at: 8 put: Array. newArray at: 9 put: Smalltalk. newArray at: 10 put: Float. newArray at: 11 put: MethodContext. newArray at: 12 put: BlockContext. newArray at: 13 put: Point. newArray at: 14 put: LargePositiveInteger. newArray at: 15 put: Display. newArray at: 16 put: Message. newArray at: 17 put: CompiledMethod. newArray at: 18 put: (self specialObjectsArray at: 18). "(low space Semaphore)" newArray at: 19 put: Semaphore. newArray at: 20 put: Character. newArray at: 21 put: #doesNotUnderstand:. newArray at: 22 put: #cannotReturn:. newArray at: 23 put: nil. "This is the process signalling low space." "An array of the 32 selectors that are compiled as special bytecodes, paired alternately with the number of arguments each takes." newArray at: 24 put: #( #+ 1 #- 1 #< 1 #> 1 #<= 1 #>= 1 #= 1 #~= 1 #* 1 #/ 1 #\\ 1 #@ 1 #bitShift: 1 #// 1 #bitAnd: 1 #bitOr: 1 #at: 1 #at:put: 2 #size 0 #next 0 #nextPut: 1 #atEnd 0 #== 1 #class 0 #blockCopy: 1 #value 0 #value: 1 #do: 1 #new 0 #new: 1 #x 0 #y 0 ). "An array of the 255 Characters in ascii order. Cog inlines table into machine code at: prim so do not regenerate it." newArray at: 25 put: (self specialObjectsArray at: 25). newArray at: 26 put: #mustBeBoolean. newArray at: 27 put: ByteArray. newArray at: 28 put: Process. "An array of up to 31 classes whose instances will have compact headers" newArray at: 29 put: self compactClassesArray. newArray at: 30 put: (self specialObjectsArray at: 30). "(delay Semaphore)" newArray at: 31 put: (self specialObjectsArray at: 31). "(user interrupt Semaphore)" "Entries 32 - 34 unreferenced. Previously these contained prototype instances to be copied for fast initialization" newArray at: 32 put: nil. "was (Float new: 2)" newArray at: 33 put: nil. "was (LargePositiveInteger new: 4)" newArray at: 34 put: nil. "was Point new" newArray at: 35 put: #cannotInterpret:. "Note: This must be fixed once we start using context prototypes (yeah, right)" "(MethodContext new: CompiledMethod fullFrameSize)." newArray at: 36 put: (self specialObjectsArray at: 36). "Is the prototype MethodContext (unused by the VM)" newArray at: 37 put: BlockClosure. "(BlockContext new: CompiledMethod fullFrameSize)." newArray at: 38 put: (self specialObjectsArray at: 38). "Is the prototype BlockContext (unused by the VM)" "array of objects referred to by external code" newArray at: 39 put: (self specialObjectsArray at: 39). "preserve external semaphores" newArray at: 40 put: nil. "Reserved for Mutex in Cog VMs" newArray at: 41 put: nil. "Reserved for a LinkedList instance for overlapped calls in CogMT" "finalization Semaphore" newArray at: 42 put: ((self specialObjectsArray at: 42) ifNil: [Semaphore new]). newArray at: 43 put: LargeNegativeInteger. "External objects for callout. Note: Written so that one can actually completely remove the FFI." newArray at: 44 put: (self at: #ExternalAddress ifAbsent: []). newArray at: 45 put: (self at: #ExternalStructure ifAbsent: []). newArray at: 46 put: (self at: #ExternalData ifAbsent: []). newArray at: 47 put: (self at: #ExternalFunction ifAbsent: []). newArray at: 48 put: (self at: #ExternalLibrary ifAbsent: []). newArray at: 49 put: #aboutToReturn:through:. newArray at: 50 put: #run:with:in:. "51 reserved for immutability message" "newArray at: 51 put: #attemptToAssign:withIndex:." newArray at: 51 put: (self specialObjectsArray at: 51 ifAbsent: []). newArray at: 52 put: #(nil "nil => generic error" #'bad receiver' #'bad argument' #'bad index' #'bad number of arguments' #'inappropriate operation' #'unsupported operation' #'no modification' #'insufficient object memory' #'insufficient C memory' #'not found' #'bad method' #'internal error in named primitive machinery' #'object may move' #'resource limit exceeded' #'object is pinned'). "53 to 55 are for Alien" newArray at: 53 put: (self at: #Alien ifAbsent: []). + newArray at: 54 put: #invokeCallbackContext:. "use invokeCallback:stack:registers:jmpbuf: for old Alien callbacks." - newArray at: 54 put: #invokeCallbackContext::. "use invokeCallback:stack:registers:jmpbuf: for old Alien callbacks." newArray at: 55 put: (self at: #UnsafeAlien ifAbsent: []). "Weak reference finalization" newArray at: 56 put: (self at: #WeakFinalizationList ifAbsent: []). "reserved for foreign callback process" newArray at: 57 put: (self specialObjectsArray at: 57 ifAbsent: []). newArray at: 58 put: #unusedBytecode. "Now replace the interpreter's reference in one atomic operation" self specialObjectsArray becomeForward: newArray ! From lewis at mail.msen.com Tue Dec 8 05:27:34 2015 From: lewis at mail.msen.com (David T. Lewis) Date: Tue Dec 8 05:27:37 2015 Subject: [squeak-dev] HandMorph position gridding In-Reply-To: References: <3CD8C5E4-4270-456C-8A3E-3B47BDBA00CB@rowledge.org> <20151208025614.GA96679@shell.msen.com> Message-ID: <20151208052734.GA22023@shell.msen.com> On Mon, Dec 07, 2015 at 08:26:36PM -0800, tim Rowledge wrote: > > > On 07-12-2015, at 6:56 PM, David T. Lewis wrote: > > > > On Mon, Dec 07, 2015 at 04:01:32PM -0800, tim Rowledge wrote: > >> After much fun, I have a plausible mechanism that introduces a way to have a temporary cursor move on a grid rather than in single pixel steps. This is useful for painting programs that want to work on scaled up bitmaps, or perhaps for drawing programs wanting to place morphs on a grid (say, electrical component icons on a virtual breadboard?). It requires 4 new HandMorph methods, 1 small change to HandMorph>position: and either the use of a couple of properties in HandMorph or a couple of additional instvars. > >> > >> Does anyone have objections to this being added? Speak soon so I know if I have to try yet another tactic. > >> > > > > Sounds reasonable, can you post it to the inbox please? > > Done. I hope. > Thanks. I won't pretend to be a competent reviewer, but it looks nicely done and has its own 'gridded cursor' method category to identify the four new methods. Dave From commits at source.squeak.org Tue Dec 8 05:30:54 2015 From: commits at source.squeak.org (commits@source.squeak.org) Date: Tue Dec 8 05:30:56 2015 Subject: [squeak-dev] The Trunk: Morphic-cwp.1057.mcz Message-ID: Colin Putney uploaded a new version of Morphic to project The Trunk: http://source.squeak.org/trunk/Morphic-cwp.1057.mcz ==================== Summary ==================== Name: Morphic-cwp.1057 Author: cwp Time: 7 December 2015, 9:30:19.787 pm UUID: c8dfb49a-6bd4-486f-8a25-592c57330da7 Ancestors: Morphic-dtl.1056 Don't clobber the existing reference pool in PasteUpMorph>>referencePool =============== Diff against Morphic-dtl.1056 =============== Item was changed: ----- Method: PasteUpMorph>>referencePool (in category 'objects from disk') ----- referencePool + ^ self + valueOfProperty: #References + ifAbsentPut: [OrderedCollection new] + + ! - | pool | - pool := OrderedCollection new. - self setProperty: #References toValue: pool. - ^ pool! From commits at source.squeak.org Tue Dec 8 05:39:14 2015 From: commits at source.squeak.org (commits@source.squeak.org) Date: Tue Dec 8 05:39:15 2015 Subject: [squeak-dev] Squeak 4.6: Morphic-cwp.994.mcz Message-ID: Colin Putney uploaded a new version of Morphic to project Squeak 4.6: http://source.squeak.org/squeak46/Morphic-cwp.994.mcz ==================== Summary ==================== Name: Morphic-cwp.994 Author: cwp Time: 7 December 2015, 9:38:39.595 pm UUID: 0da004bf-23c2-4661-bf07-be12fbe01104 Ancestors: Morphic-cmm.993 Backported From: Morphic-cwp.1057 Backport image segment serialization fixes from trunk =============== Diff against Morphic-cmm.993 =============== Item was added: + ----- Method: PasteUpMorph>>referencePool (in category 'objects from disk') ----- + referencePool + ^ self + valueOfProperty: #References + ifAbsentPut: [OrderedCollection new] + + ! From colin at wiresong.com Tue Dec 8 05:50:26 2015 From: colin at wiresong.com (Colin Putney) Date: Tue Dec 8 05:50:30 2015 Subject: [squeak-dev] Project saving and loading in Squeak 4.6 Message-ID: Hi Huw, I've backported the Environment fixes to the Squeak 4.6 repository, so you should be able to choose "Update Squeak" from the Squeak menu to load them. Colin -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151207/cec1ffc4/attachment.htm From hannes.hirzel at gmail.com Tue Dec 8 07:31:21 2015 From: hannes.hirzel at gmail.com (H. Hirzel) Date: Tue Dec 8 07:31:24 2015 Subject: [Cuis] Sorting Unicode strings (Re: [Unicode] collation sequences (Re: [squeak-dev] Unicode Support)) In-Reply-To: <5665EF88.5030901@gemtalksystems.com> References: <5665EF88.5030901@gemtalksystems.com> Message-ID: Dale Thank you for your answer with links to the ICU library and the notes about classes in Gemstone. Noteworthy that you have a class Utf8 as a subclass of ByteArray. I understand that Gemstone uses the ICU library and thus does not implement the algorithms in Smalltalk. I am currently looking into what the ICU library provides. I found as well a Ruby library [2] which implements CLDR [3] It has methods like this "Alphabetize a list using regular Ruby sort:" $> ["Art", "Wasa", "?lg", "Ved"].sort $> ["Art", "Ved", "Wasa", "?lg"] Alphabetize a list using TwitterCLDR?s locale-aware sort: $> ["Art", "Wasa", "?lg", "Ved"].localize(:de).sort.to_a $> ["?lg", "Art", "Ved", "Wasa"] I hope that given such an example it would not be too difficult to reimplement a similar sort algorithm in Squeak/Cuis/Pharo. Currently the interest is in getting sorting done in a cross-dialect-way. --Hannes [2] https://blog.twitter.com/2012/twittercldr-improving-internationalization-support-in-ruby [3] Unicode Common Locale Data Repository http://cldr.unicode.org/index On 12/7/15, Dale Henrichs wrote: > Hannes, > > For GemStone, we are using the ICU library[1]. We have Unicode7, > Unicode16 and Unicode32 classes (subclasses of CharacterCollection) for > internal Strings and the class Utf8 (a subclass of ByteArray) for Utf8 > encoded strings ... > > The ICU library provides the primitive implementations for working with > the Unicode* and Utf8 classes > > When we started considering Unicode support, we looked at what it would > take to support collation - our main reason for looking at Unicode in > the first place) -- and we saw just how complicated the collation rules > can be[2], we were glad to see that someone had already done the hard > work[1]... > > Reconciling our legacy String implementations (String, DoubleByteString, > and QuadByteString) with the Unicode* classes was also interesting, > because the rules for Unicode equality and our legacy equality > implementation were not quite compatible. > > If you are interested in more information, I can share additional > details ... > > Dale > > [1] http://site.icu-project.org/ > [2] http://unicode.org/reports/tr10/ > > On 12/07/2015 11:54 AM, H. Hirzel wrote: >> Hello >> >> According to http://www.unicode.org/cldr/charts/27/collation/de.html the >> German >> phonebook sort order is >> >> a A ? ? ?? ?? ? ? ?? ?? ?? ?? b B c C d D e E f F g G h H i I j J k K >> l L m M n N o O ? ? ?? ?? ? ? ?? ?? ?? ?? ?? ?? p P q Q r R s S ss ? t >> T u U ? ? ? ? ? ? ? ? ?? ?? ? ? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? v >> V w W x X y Y z Z >> >> I wonder why it looks like this. A lot of characters which never >> appear in a German text. >> >> >> For Spanish there is 'traditional' and 'standard' >> >> http://www.unicode.org/cldr/charts/27/collation/es.html >> >> standard a A ? ? b B c C d D e E ? ? f F g G h H i I ? ? j J k K l L m >> M n N ? ? ?? ?? ?? ?? ?? ?? ?? ?? o O ? ? p P q Q r R s S t T u U ? ? >> ? ? v V w W x X y Y z Z >> >> traditional a A ? ? b B c C ch Ch CH c? C? C? c? C? C? c? C? C? c? C? >> C? c? C? C? c? C? C? c? C? C? c? C? d D e E ? ? f F g G h H i I ? ? j >> J k K l L ll Ll LL l? L? L? l? L? L? l? L? L? l? L? L? l? L? L? l? L? >> L? l? L? L? m M n N ? ? ?? ?? ?? ?? ?? ?? ?? ?? o O ? ? p P q Q r R s >> S t T u U ? ? ? ? v V w W x X y Y z Z >> >> And French is not easily found >> http://www.unicode.org/cldr/charts/27/collation/index.html >> or seems to be defined elsewhere >> >> http://unicode.org/repos/cldr/tags/release-27/common/collation/fr.xml >> >> Suggestions and hints are welcome >> >> --Hannes >> _______________________________________________ >> Cuis mailing list >> Cuis@jvuletich.org >> http://jvuletich.org/mailman/listinfo/cuis_jvuletich.org > > > _______________________________________________ > Cuis mailing list > Cuis@jvuletich.org > http://jvuletich.org/mailman/listinfo/cuis_jvuletich.org > From robert.w.withers at gmail.com Tue Dec 8 08:37:37 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Tue Dec 8 08:37:43 2015 Subject: [squeak-dev] Re: [Pharo-users] evolutions of squeakelib & crypto (Reed Solomon) In-Reply-To: References: <565DCE53.90501@gmail.com> <565DD073.1060409@gmail.com> <565FF8ED.1030606@free.fr> <5660BCFA.4050107@gmail.com> <56616E9C.6040201@gmail.com> <56616F27.4070002@gmail.com> <566174E9.9020001@gmail.com> <566175EA.2090107@gmail.com> Message-ID: <566696D1.4050605@gmail.com> Hey Chris, I just tried this in my squeak image and it complains about not finding 'ConfigurationOfFuel' in a Monticello repository. Are there other pre-steps I need to take? Robert On 12/06/2015 02:03 PM, Chris Muller wrote: > On Fri, Dec 4, 2015 at 5:15 AM, Robert Withers > > wrote: > > I just realized that the squeak version uses ReferenceStream while > the Pharo version uses Fuel, so the binary serializations are > different and they won't speak to each other. Any chance that Fuel > is ported to Squeak? > > > Installer new merge: #fuel > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151208/a02d4498/attachment.htm From hannes.hirzel at gmail.com Tue Dec 8 13:01:36 2015 From: hannes.hirzel at gmail.com (H. Hirzel) Date: Tue Dec 8 13:01:41 2015 Subject: [squeak-dev] Project saving and loading in Squeak 4.6 In-Reply-To: References: Message-ID: Hi Colin Update from a http://ftp.squeak.org/4.6/Squeak-4.6-All-in-One.zip went well (see attached screenshot). Saving, then deleting and loading a test project went fine. Thank you! --Hannes On 12/8/15, Colin Putney wrote: > Hi Huw, > > I've backported the Environment fixes to the Squeak 4.6 repository, so you > should be able to choose "Update Squeak" from the Squeak menu to load them. > > Colin > -------------- next part -------------- A non-text attachment was scrubbed... Name: Squeak4.6_with_cwp_load_save_fix_backport_2015-12-08.png Type: image/png Size: 35189 bytes Desc: not available Url : http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151208/1235c814/Squeak4.6_with_cwp_load_save_fix_backport_2015-12-08-0001.png From lewis at mail.msen.com Tue Dec 8 13:06:19 2015 From: lewis at mail.msen.com (David T. Lewis) Date: Tue Dec 8 13:06:21 2015 Subject: [squeak-dev] Project saving and loading in Squeak 4.6 In-Reply-To: References: Message-ID: <20151208130619.GA97687@shell.msen.com> On Tue, Dec 08, 2015 at 02:01:36PM +0100, H. Hirzel wrote: > > On 12/8/15, Colin Putney wrote: > > Hi Huw, > > > > I've backported the Environment fixes to the Squeak 4.6 repository, so you > > should be able to choose "Update Squeak" from the Squeak menu to load them. > > > > Colin > > > > Hi Colin > > Update from a > http://ftp.squeak.org/4.6/Squeak-4.6-All-in-One.zip > went well (see attached screenshot). > > Saving, then deleting and loading a test project went fine. > > Thank you! > > --Hannes Yay! Thank you Colin. Dave From robert.w.withers at gmail.com Tue Dec 8 13:07:51 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Tue Dec 8 13:07:57 2015 Subject: [squeak-dev] SecureSession published to Cryptography repository Message-ID: <5666D627.9020703@gmail.com> Good morning, I managed to separate the SecureSession code out of Mushroom and I have published it into the SqueakSource Cryptography repository. Version 4 is this release, though I expect name refactoring to occur, so just take the latest. This requires the loading of Cryptography, though I did not set a package dependency in Monticello for SecureSession. Should I? Note to Pharo: Please load version 37 of Cryptography. WARNING: the Performance test will fail without the crypto plugins installed. Resulting performance data, with and without crypto plugins...there is 275% increase in performance with crypto plugins. With crypto plugins: 100 KB data chunks received: 10 send time: 272 ms, rate: 3.676 MB/sec. receive time: 600 ms. rate: 1.667 MB/sec. Without crypto plugins: 100 KB data chunks received: 1 send time: 8469 ms, rate: 0.012 MB/sec. receive time: 17233 ms. rate: 0.006 MB/sec. Regards, Robert -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151208/3844c4aa/attachment.htm From robert.w.withers at gmail.com Tue Dec 8 13:33:30 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Tue Dec 8 13:33:38 2015 Subject: [squeak-dev] Re: SecureSession published to Cryptography repository In-Reply-To: <5666D627.9020703@gmail.com> References: <5666D627.9020703@gmail.com> Message-ID: <5666DC2A.1000700@gmail.com> Is that a 27,500% increase? I am a little confused. On 12/08/2015 08:07 AM, Robert Withers wrote: > Good morning, > > I managed to separate the SecureSession code out of Mushroom and I > have published it into the SqueakSource Cryptography repository. > Version 4 is this release, though I expect name refactoring to occur, > so just take the latest. This requires the loading of Cryptography, > though I did not set a package dependency in Monticello for > SecureSession. Should I? > > Note to Pharo: Please load version 37 of Cryptography. > > WARNING: the Performance test will fail without the crypto plugins > installed. > > > Resulting performance data, with and without crypto plugins...there is > 275% increase in performance with crypto plugins. > > With crypto plugins: > > 100 KB data chunks received: 10 > send time: 272 ms, rate: 3.676 MB/sec. > receive time: 600 ms. rate: 1.667 MB/sec. > > > Without crypto plugins: > > 100 KB data chunks received: 1 > send time: 8469 ms, rate: 0.012 MB/sec. > receive time: 17233 ms. rate: 0.006 MB/sec. > > Regards, > Robert -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151208/e1a8c507/attachment.htm From btc at openinworld.com Tue Dec 8 14:00:36 2015 From: btc at openinworld.com (Ben Coman) Date: Tue Dec 8 14:01:00 2015 Subject: [squeak-dev] Name change: Mushroom ( was Re: evolutions of squeakelib & crypto) In-Reply-To: <5666235C.9000906@gmail.com> References: <565DCE53.90501@gmail.com> <565DD073.1060409@gmail.com> <565FF8ED.1030606@free.fr> <5660BCFA.4050107@gmail.com> <56616E9C.6040201@gmail.com> <56616F27.4070002@gmail.com> <566174E9.9020001@gmail.com> <566175EA.2090107@gmail.com> <60B913A3-565C-4924-9F82-4E0E96553B72@gmail.com> <5661A6A5.9030902@gmail.com> <224A3FD6-A8DA-45E3-8EDE-AA52133940CC@stfx.eu> <5661B7F1.1080701@gmail.com> <5663A08C.8030602@gmail.com> <5665A1AE.2030706@gmail.com> <56660EC6.7030804@gmail.com> <566615d0.13e48c0a.b2cc4.63bcSMTPIN_ADDED_MISSING@mx.google.com> <5666235C.9000906@gmail.com> Message-ID: On Tue, Dec 8, 2015 at 8:25 AM, Robert Withers wrote: > > > On 12/07/2015 07:26 PM, Jecel Assumpcao Jr. wrote: >> >> Robert Withers wrote: >>> >>> Ben, Huw, Todd and Sven, thank you all for your feedback! I suppose I >>> could call the project "CryptOCaps" but for some reason I glommed onto >>> mushroom as the name. Not grandiose and it is somewhat descriptive...a >>> network of secure sessions, each one a mushroom. Ceps are highly valued. >>> We can tag it for the catalog. >> >> Though pretty old, the Mushroom Smalltalk computer was very well known >> at one time: >> >> http://www.wolczko.com/mushroom/ >> >> Though one is a hardware project and yours is software, there is still >> the potential for some confusion. >> >> -- Jecel > > > Oh dear, that's a potential naming conflict. Perhaps this is where the idea > came from, as I have read about this Mushroom Project long ago. As this > hardware project is not longer active, repurposing the name may be > reasonable but I cannot say wityhout that authority. There are 2 layers, > session & presentation, so perhaps 2 names are needed and the one project > needs splitting. And there is the consensus issue. Perhaps I should use > technical names and be done with it: SecureSession & CryptOCaps. Well, I > will leave off changing again until agreed to names are established, as I > don't know what to do about it, but I welcome suggestions... > > Maelstrom, Mushroom, Crystalline & Phosphor, SecureSession & CryptOCaps, > pther suggestions? Grinding over your previous descriptions... >> binary serialization to communicate over the wire securewire --> 1900 secuawire --> 0 immunewire --> 2 wirymune --> 1 My overall favourite, if a little oblique ("wire immune"). tightwire --> 91200 >> substrate for distributed secure objects obtransecua --> 0 obtransec --> 0 Equal favourite descriptive ("object transfer secure") secuaobj --> 0 secushare --> 5880 kryptobj --> 4 kryptomesh --> 0 Equal favourite descriptive ("cryptographic mesh" of objects) meshobsec --> 0 kryptnet --> 1220 cheers -ben From robert.w.withers at gmail.com Tue Dec 8 14:07:20 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Tue Dec 8 14:07:27 2015 Subject: [squeak-dev] Name change: Mushroom ( was Re: evolutions of squeakelib & crypto) In-Reply-To: References: <565DCE53.90501@gmail.com> <565DD073.1060409@gmail.com> <565FF8ED.1030606@free.fr> <5660BCFA.4050107@gmail.com> <56616E9C.6040201@gmail.com> <56616F27.4070002@gmail.com> <566174E9.9020001@gmail.com> <566175EA.2090107@gmail.com> <60B913A3-565C-4924-9F82-4E0E96553B72@gmail.com> <5661A6A5.9030902@gmail.com> <224A3FD6-A8DA-45E3-8EDE-AA52133940CC@stfx.eu> <5661B7F1.1080701@gmail.com> <5663A08C.8030602@gmail.com> <5665A1AE.2030706@gmail.com> <56660EC6.7030804@gmail.com> <566615d0.13e48c0a.b2cc4.63bcSMTPIN_ADDED_MISSING@mx.google.com> <5666235C.9000906@gmail.com> Message-ID: <5666E418.7020706@gmail.com> Thanks for doing this research, Ben. I checked 'kryptocap' with 363 results. If not Mushroom, I kinda like this one since it is an OCap system.. Cheers, Robert On 12/08/2015 09:00 AM, Ben Coman wrote: > On Tue, Dec 8, 2015 at 8:25 AM, Robert Withers > wrote: >> >> On 12/07/2015 07:26 PM, Jecel Assumpcao Jr. wrote: >>> Robert Withers wrote: >>>> Ben, Huw, Todd and Sven, thank you all for your feedback! I suppose I >>>> could call the project "CryptOCaps" but for some reason I glommed onto >>>> mushroom as the name. Not grandiose and it is somewhat descriptive...a >>>> network of secure sessions, each one a mushroom. Ceps are highly valued. >>>> We can tag it for the catalog. >>> Though pretty old, the Mushroom Smalltalk computer was very well known >>> at one time: >>> >>> http://www.wolczko.com/mushroom/ >>> >>> Though one is a hardware project and yours is software, there is still >>> the potential for some confusion. >>> >>> -- Jecel >> >> Oh dear, that's a potential naming conflict. Perhaps this is where the idea >> came from, as I have read about this Mushroom Project long ago. As this >> hardware project is not longer active, repurposing the name may be >> reasonable but I cannot say wityhout that authority. There are 2 layers, >> session & presentation, so perhaps 2 names are needed and the one project >> needs splitting. And there is the consensus issue. Perhaps I should use >> technical names and be done with it: SecureSession & CryptOCaps. Well, I >> will leave off changing again until agreed to names are established, as I >> don't know what to do about it, but I welcome suggestions... >> >> Maelstrom, Mushroom, Crystalline & Phosphor, SecureSession & CryptOCaps, >> pther suggestions? > > Grinding over your previous descriptions... > >>> binary serialization to communicate over the wire > securewire --> 1900 > > secuawire --> 0 > > immunewire --> 2 > > wirymune --> 1 My overall favourite, if a little oblique ("wire immune"). > > tightwire --> 91200 > > >>> substrate for distributed secure objects > obtransecua --> 0 > > obtransec --> 0 Equal favourite descriptive ("object transfer secure") > > secuaobj --> 0 > > secushare --> 5880 > > kryptobj --> 4 > > kryptomesh --> 0 Equal favourite descriptive ("cryptographic mesh" of objects) > > meshobsec --> 0 > > kryptnet --> 1220 > > > cheers -ben > From robert.w.withers at gmail.com Tue Dec 8 14:53:23 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Tue Dec 8 14:53:32 2015 Subject: [squeak-dev] Re: SecureSession published to Cryptography repository In-Reply-To: <5666DC2A.1000700@gmail.com> References: <5666D627.9020703@gmail.com> <5666DC2A.1000700@gmail.com> Message-ID: <5666EEE3.8020304@gmail.com> Ok, less confused now - it IS 27500%! That's with me single-core 900Mhz machine. An i5 or i7 will have much better throughput me thinks, at least the clocking...I would love to hear some numbers...it is 13 Mbs now, in loopback; can we reach 50 Mbs? One known issue: when calling setsockopt: (squeak protocol is Socket>>#setOption:value:, one of the values fails. Here is the code where Iam calling this... ConnectionHandler>>#newListener | listener | self flag: #todo. "listener _ self socketClass newTCP." listener := Socket newTCP. listener setOption: 'SO_REUSEADDR' value: 1. listener setOption: 'SO_LINGER' value: 0. "backlog is how many simultaneous connections to accept at the same time" listener listenOn: portNumber backlogSize: 10. ^listener The stdout/stderr output from this call is: setsockopt: Invalid argument setsockopt: Invalid argument Note that I also refactored teh pipeline to eliminate a few calls, change the protocol and rename some classes. I updated the diagram at https://github.com/SqueakCryptographySquad/Mushroom/wiki. There is now a version 5 of the SecureSession package in the repository. Thanks, Robert On 12/08/2015 08:33 AM, Robert Withers wrote: > Is that a 27,500% increase? I am a little confused. > > On 12/08/2015 08:07 AM, Robert Withers wrote: >> Good morning, >> >> I managed to separate the SecureSession code out of Mushroom and I >> have published it into the SqueakSource Cryptography repository. >> Version 4 is this release, though I expect name refactoring to occur, >> so just take the latest. This requires the loading of Cryptography, >> though I did not set a package dependency in Monticello for >> SecureSession. Should I? >> >> Note to Pharo: Please load version 37 of Cryptography. >> >> WARNING: the Performance test will fail without the crypto plugins >> installed. >> >> >> Resulting performance data, with and without crypto plugins...there >> is 275% increase in performance with crypto plugins. >> >> With crypto plugins: >> >> 100 KB data chunks received: 10 >> send time: 272 ms, rate: 3.676 MB/sec. >> receive time: 600 ms. rate: 1.667 MB/sec. >> >> >> Without crypto plugins: >> >> 100 KB data chunks received: 1 >> send time: 8469 ms, rate: 0.012 MB/sec. >> receive time: 17233 ms. rate: 0.006 MB/sec. >> >> Regards, >> Robert > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151208/80017eed/attachment.htm From commits at source.squeak.org Tue Dec 8 18:16:44 2015 From: commits at source.squeak.org (commits@source.squeak.org) Date: Tue Dec 8 18:16:47 2015 Subject: [squeak-dev] The Inbox: Tools-pre.659.mcz Message-ID: A new version of Tools was added to project The Inbox: http://source.squeak.org/inbox/Tools-pre.659.mcz ==================== Summary ==================== Name: Tools-pre.659 Author: pre Time: 8 December 2015, 7:16:41.034 pm UUID: a7cb3cba-074b-46a6-8a83-fada58007b25 Ancestors: Tools-eem.658 This adds balloon help texts to the message list of the standard browser to explain the symbols shown in front of message names. =============== Diff against Tools-eem.658 =============== Item was changed: ----- Method: Browser>>messageHelpAt: (in category 'message list') ----- messageHelpAt: anIndex "Show the first n lines of the sources code of the selected message." + | source formatted | - | source formatted lineCount | Preferences balloonHelpInMessageLists ifFalse: [^ nil]. self messageList size < anIndex ifTrue: [^ nil]. source := (self selectedClassOrMetaClass >> (self messageList at: anIndex)) getSource. + source lineCount > 5 ifTrue: [ + | sourceLines | + sourceLines := (source asString lines copyFrom: 1 to: 5) asOrderedCollection. + sourceLines add: ' [...]'. + source := sourceLines joinSeparatedBy: Character cr]. + formatted := SHTextStylerST80 new classOrMetaClass: self selectedClassOrMetaClass; styledTextFor: source asText. - - lineCount := 0. - source doWithIndex: [:char :index | - char = Character cr ifTrue: [lineCount := lineCount + 1]. - lineCount > 10 ifTrue: [ - formatted := formatted copyFrom: 1 to: index-1. - formatted append: ' [...]'. - ^ formatted]]. + ^ (Text newFrom: ((self messageIconHelpAt: anIndex) ifNotEmpty: [:t | t , Character cr, Character cr])) + append: formatted; + yourself! - ^ formatted! Item was added: + ----- Method: Browser>>messageIconHelpAt: (in category 'message list') ----- + messageIconHelpAt: anIndex + + self class showMessageIcons + ifFalse: [^ nil]. + + ^ ToolIconHelp iconHelpNamed: (ToolIcons + iconForClass: self selectedClassOrMetaClass + selector: (self messageList at: anIndex))! Item was added: + Object subclass: #ToolIconHelp + instanceVariableNames: '' + classVariableNames: 'HelpTexts' + poolDictionaries: '' + category: 'Tools-Base'! Item was added: + ----- Method: ToolIconHelp class>>abstract (in category 'as yet unclassified') ----- + abstract + + ^ 'This method is abstract.' ! Item was added: + ----- Method: ToolIconHelp class>>arrowDown (in category 'as yet unclassified') ----- + arrowDown + + ^ 'This method is overriden by another method.'! Item was added: + ----- Method: ToolIconHelp class>>arrowUp (in category 'as yet unclassified') ----- + arrowUp + + ^ 'This method overrides a super method.'! Item was added: + ----- Method: ToolIconHelp class>>arrowUpAndDown (in category 'as yet unclassified') ----- + arrowUpAndDown + + ^ 'This method overrides and is overridden by other methods.'! Item was added: + ----- Method: ToolIconHelp class>>blank (in category 'as yet unclassified') ----- + blank + + ^ ''! Item was added: + ----- Method: ToolIconHelp class>>breakpoint (in category 'as yet unclassified') ----- + breakpoint + + ^ 'This method contains a breakpoint.'! Item was added: + ----- Method: ToolIconHelp class>>flag (in category 'as yet unclassified') ----- + flag + + ^ 'This method needs attention of some kind.' ! Item was added: + ----- Method: ToolIconHelp class>>helpTexts (in category 'as yet unclassified') ----- + helpTexts + + ^ HelpTexts ifNil: [HelpTexts := IdentityDictionary new]! Item was added: + ----- Method: ToolIconHelp class>>iconHelpNamed: (in category 'as yet unclassified') ----- + iconHelpNamed: aSymbol + + (self respondsTo: aSymbol) + ifTrue: [^self helpTexts at: aSymbol ifAbsentPut: [self perform: aSymbol]] + ifFalse: [^ ''] + ! Item was added: + ----- Method: ToolIconHelp class>>no (in category 'as yet unclassified') ----- + no + + ^ 'This method should not be implemented.' ! Item was added: + ----- Method: ToolIconHelp class>>notOverridden (in category 'as yet unclassified') ----- + notOverridden + + ^ 'This method is abstract and has not been overridden.' ! Item was added: + ----- Method: ToolIconHelp class>>primitive (in category 'as yet unclassified') ----- + primitive + + ^ 'This method implements a primitive.'! From patrick.rein at hpi.de Tue Dec 8 19:03:13 2015 From: patrick.rein at hpi.de (Patrick R.) Date: Tue Dec 8 19:20:24 2015 Subject: [squeak-dev] Deprecated Call in CompiledMethod Message-ID: <1449601393729-4866105.post@n4.nabble.com> Hello everyone, While wandering through the Kernel I stumbled upon the message CompiledMethod>>#valueWithReceiver: aReceiver arguments: anArray which contains the following code: Is there a reason for this being still in the image? Otherwise, I will change this to the new API. -- View this message in context: http://forum.world.st/Deprecated-Call-in-CompiledMethod-tp4866105.html Sent from the Squeak - Dev mailing list archive at Nabble.com. From dale.henrichs at gemtalksystems.com Tue Dec 8 19:20:48 2015 From: dale.henrichs at gemtalksystems.com (Dale Henrichs) Date: Tue Dec 8 19:20:53 2015 Subject: [Cuis] Sorting Unicode strings (Re: [Unicode] collation sequences (Re: [squeak-dev] Unicode Support)) In-Reply-To: References: <5665EF88.5030901@gemtalksystems.com> Message-ID: <56672D90.4020202@gemtalksystems.com> On 12/07/2015 11:31 PM, H. Hirzel wrote: > Dale > > Thank you for your answer with links to the ICU library and the notes > about classes in Gemstone. Noteworthy that you have a class Utf8 as a > subclass of ByteArray. > > I understand that Gemstone uses the ICU library and thus does not > implement the algorithms in Smalltalk. > > I am currently looking into what the ICU library provides. > > I found as well a Ruby library [2] which implements CLDR [3] > > It has methods like this > > "Alphabetize a list using regular Ruby sort:" > > $> ["Art", "Wasa", "?lg", "Ved"].sort > $> ["Art", "Ved", "Wasa", "?lg"] > > Alphabetize a list using TwitterCLDR?s locale-aware sort: > > $> ["Art", "Wasa", "?lg", "Ved"].localize(:de).sort.to_a > $> ["?lg", "Art", "Ved", "Wasa"] > > I hope that given such an example it would not be too difficult to > reimplement a similar sort algorithm in Squeak/Cuis/Pharo. Currently > the interest is in getting sorting done in a cross-dialect-way. > I think that the issue (from a performance perspective) is that you can't depend upon the value of the code point when doing collation --- the main algorithm[5] is pretty much table based --- In addition to the different sort orders based on characters there are even more arcane sort rules where characters at the end of a word can affect the sort order of the word (for more info see[4]). It is worth looking at the Conformance section of the Unicode spec[1] as there are different levels of collation conformance ..... ICU conforms[2] to to UTS #10[3], the highest level of conformance ... It looks like TwitterCLDR[6] uses the Main Algorithm[5] with tailoring[7]. They don't claim to be conformant to the Unicode Collation Algorithm[3], but they are covering a big chunk of the standard use cases .... Dale [1] http://unicode.org/reports/tr10/#Conformance [2] http://userguide.icu-project.org/collation [3] http://www.unicode.org/reports/tr10/ [4] http://www.unicode.org/reports/tr10/#Introduction [5] http://www.unicode.org/reports/tr10/#Main_Algorithm [6] https://blog.twitter.com/2012/twittercldr-improving-internationalization-support-in-ruby [7] http://unicode.org/reports/tr10/#Tailoring From robert.w.withers at gmail.com Tue Dec 8 19:39:32 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Tue Dec 8 19:39:38 2015 Subject: [squeak-dev] Deprecated Call in CompiledMethod In-Reply-To: <1449601393729-4866105.post@n4.nabble.com> References: <1449601393729-4866105.post@n4.nabble.com> Message-ID: <566731F4.80905@gmail.com> Hi Patrick, I like the looks of this. A core goal is to see these get passed over the network and stretch it's context/stack with farRefs and have it evaluate remotely. I've not spent any time exploring moving actual code, but Mushroom (currently a little bit broken) does remote objects. A CompiledMethod looks perfect for remote relocation, I think but I am not sure. This method looks to be a perfect fit for eventual evaluation. I wanted to let you know since you brought this method up. Thank you for the opportunity, I am hopeful. Best recursively, Robert On 12/08/2015 02:03 PM, Patrick R. wrote: > Hello everyone, > > While wandering through the Kernel I stumbled upon the message > CompiledMethod>>#valueWithReceiver: aReceiver arguments: anArray which > contains the following code: > > > > Is there a reason for this being still in the image? Otherwise, I will > change this to the new API. > > > > -- > View this message in context: http://forum.world.st/Deprecated-Call-in-CompiledMethod-tp4866105.html > Sent from the Squeak - Dev mailing list archive at Nabble.com. > From euanmee at gmail.com Tue Dec 8 19:59:57 2015 From: euanmee at gmail.com (EuanM) Date: Tue Dec 8 19:59:59 2015 Subject: [squeak-dev] Re: [Unicode] Normalization -- canonical equivalence vs compatibility equivalence Message-ID: canonical equivalence ================ aString = anotherStrong where both strings are of the same sequence of valid Unicode representations of abstract characters BUT each individual character MAY (or may not) be represented by different valid Unicode representations of the character e.g. #(00c5) = #(212b) = #(0041 030a) "The same meaning when printed or displayed" Every tuple of canonically equivalent sequences is also compatibility equivalent. Actually yes - the easy way to define it would be a canonically equivalent string is the same string sequence of the same abstract characters as the string you are comparing it to. (But may use different codepoints) The sequence will have the same glyph appearance. compatibility equivalence is ===================== aString = anotherStrong where both strings have the same meaning in some particular contexts, but may have a different glyph appearance e.g. comparing ligatures with the decomposed sequences making up a ligature as in "ss" = "?" This is compatibility equivalent, they *can* have the same meaning, but they never have the same appearance. The normalisation forms either - expand all compatibility codepoints into fully composed sequences, to create the "fully composed normal form" of the string OR fully condense all composed sequences into compatibility codepoints, to create a "fully decomposed normal form". (Note, that as there can be multiple compatibility codepoints for an abstract character, it is necessary to create fully decomposed forms only after first coverting to fully composed forms, or two fully decomposed strings could fail equivalance testing *despite* being compatibility equivalent. e.g. #(00c5 00c5 00c5) = #(212b 212b 212b) is actually true, for compatibility equivalence But a process of fully composing any of #(00c5 212b #(0041 030a) ) #(00c5 00c5 00c5) #(212b 212b 212b) would always produce #( #(0041 030a) #(0041 030a) #(0041 030a) ) and then a subsequent full decomposition would produce #(00c5 00c5 00c5) for all three cases. On 8 December 2015 at 16:09, H. Hirzel wrote: > It seems that going for Normalization first in order to implement > String comparison is the next necessary steps. This is already a > result by itself. > > Normalization (Unicode) > Last updated at 4:05 pm UTC on 8 December 2015 > http://wiki.squeak.org/squeak/6229 > > Can you explain to me what the difference between > > canonical equivalence > and > compatibility equivalence > > and for which we should go? > > --Hannes From ma.chris.m at gmail.com Tue Dec 8 20:03:12 2015 From: ma.chris.m at gmail.com (Chris Muller) Date: Tue Dec 8 20:03:55 2015 Subject: [squeak-dev] Re: [Pharo-users] evolutions of squeakelib & crypto (Reed Solomon) In-Reply-To: <566696D1.4050605@gmail.com> References: <565DCE53.90501@gmail.com> <565DD073.1060409@gmail.com> <565FF8ED.1030606@free.fr> <5660BCFA.4050107@gmail.com> <56616E9C.6040201@gmail.com> <56616F27.4070002@gmail.com> <566174E9.9020001@gmail.com> <566175EA.2090107@gmail.com> <566696D1.4050605@gmail.com> Message-ID: Ask Mariano. On Tue, Dec 8, 2015 at 2:37 AM, Robert Withers wrote: > Hey Chris, > > I just tried this in my squeak image and it complains about not finding > 'ConfigurationOfFuel' in a Monticello repository. Are there other pre-steps > I need to take? > > Robert > > On 12/06/2015 02:03 PM, Chris Muller wrote: > > On Fri, Dec 4, 2015 at 5:15 AM, Robert Withers > wrote: >> >> I just realized that the squeak version uses ReferenceStream while the >> Pharo version uses Fuel, so the binary serializations are different and they >> won't speak to each other. Any chance that Fuel is ported to Squeak? >> > > Installer new merge: #fuel > > > > From euanmee at gmail.com Tue Dec 8 20:22:12 2015 From: euanmee at gmail.com (EuanM) Date: Tue Dec 8 20:22:14 2015 Subject: [Cuis] Sorting Unicode strings (Re: [Unicode] collation sequences (Re: [squeak-dev] Unicode Support)) In-Reply-To: <56672D90.4020202@gemtalksystems.com> References: <5665EF88.5030901@gemtalksystems.com> <56672D90.4020202@gemtalksystems.com> Message-ID: Dale - is that you can't depend on the value of a codepoint *unless the string is either in fully-composed form (or has just been fully-decomposed from a fully-composed form) * OR are there circumstances where even those two cases cannot be relied upon? On 8 December 2015 at 19:20, Dale Henrichs wrote: > > > On 12/07/2015 11:31 PM, H. Hirzel wrote: >> >> Dale >> >> Thank you for your answer with links to the ICU library and the notes >> about classes in Gemstone. Noteworthy that you have a class Utf8 as a >> subclass of ByteArray. >> >> I understand that Gemstone uses the ICU library and thus does not >> implement the algorithms in Smalltalk. >> >> I am currently looking into what the ICU library provides. >> >> I found as well a Ruby library [2] which implements CLDR [3] >> >> It has methods like this >> >> "Alphabetize a list using regular Ruby sort:" >> >> $> ["Art", "Wasa", "?lg", "Ved"].sort >> $> ["Art", "Ved", "Wasa", "?lg"] >> >> Alphabetize a list using TwitterCLDR?s locale-aware sort: >> >> $> ["Art", "Wasa", "?lg", "Ved"].localize(:de).sort.to_a >> $> ["?lg", "Art", "Ved", "Wasa"] >> >> I hope that given such an example it would not be too difficult to >> reimplement a similar sort algorithm in Squeak/Cuis/Pharo. Currently >> the interest is in getting sorting done in a cross-dialect-way. >> > > I think that the issue (from a performance perspective) is that you can't > depend upon the value of the code point when doing collation --- the main > algorithm[5] is pretty much table based --- In addition to the different > sort orders based on characters there are even more arcane sort rules where > characters at the end of a word can affect the sort order of the word (for > more info see[4]). > > It is worth looking at the Conformance section of the Unicode spec[1] as > there are different levels of collation conformance ..... > > ICU conforms[2] to to UTS #10[3], the highest level of conformance ... > > It looks like TwitterCLDR[6] uses the Main Algorithm[5] with tailoring[7]. > They don't claim to be conformant to the Unicode Collation Algorithm[3], but > they are covering a big chunk of the standard use cases .... > > Dale > > [1] http://unicode.org/reports/tr10/#Conformance > [2] http://userguide.icu-project.org/collation > [3] http://www.unicode.org/reports/tr10/ > [4] http://www.unicode.org/reports/tr10/#Introduction > [5] http://www.unicode.org/reports/tr10/#Main_Algorithm > [6] > https://blog.twitter.com/2012/twittercldr-improving-internationalization-support-in-ruby > [7] http://unicode.org/reports/tr10/#Tailoring From Das.Linux at gmx.de Tue Dec 8 20:39:03 2015 From: Das.Linux at gmx.de (Tobias Pape) Date: Tue Dec 8 20:39:07 2015 Subject: [squeak-dev] [Pharo-users] evolutions of squeakelib & crypto (Reed Solomon) In-Reply-To: References: <565DCE53.90501@gmail.com> <565DD073.1060409@gmail.com> <565FF8ED.1030606@free.fr> <5660BCFA.4050107@gmail.com> <56616E9C.6040201@gmail.com> <56616F27.4070002@gmail.com> <566174E9.9020001@gmail.com> <566175EA.2090107@gmail.com> <566696D1.4050605@gmail.com> Message-ID: <1418E355-0D5E-40F6-A955-B57F6D5D42D7@gmx.de> Hi, On 08.12.2015, at 21:03, Chris Muller wrote: > Ask Mariano. That's uncommonly curtly? Max Leske actually makes a huge effort that Fuel always runs in Squeak[1]. I want to thank him for that here. So. Robert, you should do: (Installer monticello http: 'http://smalltalkhub.com/mc/Pharo/Fuel/') project: 'main'; package: 'ConfigurationOfFuel'; install. ((Smalltalk at: #ConfigurationOfFuel) project version: #stable) load. Fuel is loaded then. There are some development leftovers but they shouldn't affect normal operation. If you find issues, please report them here and to Max (CC). Best regards -Tobias [1]: https://github.com/theseion/Fuel/blob/master/scripts/jenkins.sh#L43 > > On Tue, Dec 8, 2015 at 2:37 AM, Robert Withers > wrote: >> Hey Chris, >> >> I just tried this in my squeak image and it complains about not finding >> 'ConfigurationOfFuel' in a Monticello repository. Are there other pre-steps >> I need to take? >> >> Robert >> >> On 12/06/2015 02:03 PM, Chris Muller wrote: >> >> On Fri, Dec 4, 2015 at 5:15 AM, Robert Withers >> wrote: >>> >>> I just realized that the squeak version uses ReferenceStream while the >>> Pharo version uses Fuel, so the binary serializations are different and they >>> won't speak to each other. Any chance that Fuel is ported to Squeak? >>> >> >> Installer new merge: #fuel >> >> >> >> > From commits at source.squeak.org Tue Dec 8 20:41:14 2015 From: commits at source.squeak.org (commits@source.squeak.org) Date: Tue Dec 8 20:41:16 2015 Subject: [squeak-dev] The Trunk: Installer-Core-topa.402.mcz Message-ID: Tobias Pape uploaded a new version of Installer-Core to project The Trunk: http://source.squeak.org/trunk/Installer-Core-topa.402.mcz ==================== Summary ==================== Name: Installer-Core-topa.402 Author: topa Time: 8 December 2015, 9:41:01.27 pm UUID: 99770767-224c-4cc1-98ba-70a62d979822 Ancestors: Installer-Core-topa.401 This won't work in any way. Until we have a good way integrating Installer/SqueakMap and Metacello, remove this as it is confusing and unhelpful. =============== Diff against Installer-Core-topa.401 =============== Item was removed: - ----- Method: Installer>>fuel (in category 'package-definitions') ----- - fuel - "Serialization package." - ^ { #ss3 -> 'Fuel'. - 'ConfigurationOfFuel' }! From robert.w.withers at gmail.com Tue Dec 8 20:42:55 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Tue Dec 8 20:43:03 2015 Subject: [squeak-dev] [Pharo-users] evolutions of squeakelib & crypto (Reed Solomon) In-Reply-To: <1418E355-0D5E-40F6-A955-B57F6D5D42D7@gmx.de> References: <565DCE53.90501@gmail.com> <565DD073.1060409@gmail.com> <565FF8ED.1030606@free.fr> <5660BCFA.4050107@gmail.com> <56616E9C.6040201@gmail.com> <56616F27.4070002@gmail.com> <566174E9.9020001@gmail.com> <566175EA.2090107@gmail.com> <566696D1.4050605@gmail.com> <1418E355-0D5E-40F6-A955-B57F6D5D42D7@gmx.de> Message-ID: <566740CF.5070104@gmail.com> On 12/08/2015 03:39 PM, Tobias Pape wrote: > Hi, > > On 08.12.2015, at 21:03, Chris Muller wrote: > >> Ask Mariano. > That's uncommonly curtly? > > I think he is German so I get it. :) From robert.w.withers at gmail.com Tue Dec 8 20:47:27 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Tue Dec 8 20:47:34 2015 Subject: [squeak-dev] [Pharo-users] evolutions of squeakelib & crypto (Reed Solomon) In-Reply-To: <1418E355-0D5E-40F6-A955-B57F6D5D42D7@gmx.de> References: <565DCE53.90501@gmail.com> <565DD073.1060409@gmail.com> <565FF8ED.1030606@free.fr> <5660BCFA.4050107@gmail.com> <56616E9C.6040201@gmail.com> <56616F27.4070002@gmail.com> <566174E9.9020001@gmail.com> <566175EA.2090107@gmail.com> <566696D1.4050605@gmail.com> <1418E355-0D5E-40F6-A955-B57F6D5D42D7@gmx.de> Message-ID: <566741DF.4060407@gmail.com> I threw that comment out there, but realized I forgot to thank you for this great info. Thank you, Tobias. I am right there ready to dive in, but will take a break. Bleiben moeglich, bitte... On 12/08/2015 03:39 PM, Tobias Pape wrote: > Hi, > > On 08.12.2015, at 21:03, Chris Muller wrote: > >> Ask Mariano. > That's uncommonly curtly? > > Max Leske actually makes a huge effort that Fuel always runs in Squeak[1]. > I want to thank him for that here. > > So. > Robert, you should do: > > (Installer monticello http: 'http://smalltalkhub.com/mc/Pharo/Fuel/') > project: 'main'; > package: 'ConfigurationOfFuel'; > install. > > ((Smalltalk at: #ConfigurationOfFuel) project version: #stable) load. > > Fuel is loaded then. There are some development leftovers but they shouldn't affect > normal operation. If you find issues, please report them here and to Max (CC). > > Best regards > -Tobias > > > [1]: https://github.com/theseion/Fuel/blob/master/scripts/jenkins.sh#L43 > >> On Tue, Dec 8, 2015 at 2:37 AM, Robert Withers >> wrote: >>> Hey Chris, >>> >>> I just tried this in my squeak image and it complains about not finding >>> 'ConfigurationOfFuel' in a Monticello repository. Are there other pre-steps >>> I need to take? >>> >>> Robert >>> >>> On 12/06/2015 02:03 PM, Chris Muller wrote: >>> >>> On Fri, Dec 4, 2015 at 5:15 AM, Robert Withers >>> wrote: >>>> I just realized that the squeak version uses ReferenceStream while the >>>> Pharo version uses Fuel, so the binary serializations are different and they >>>> won't speak to each other. Any chance that Fuel is ported to Squeak? >>>> >>> Installer new merge: #fuel >>> >>> >>> >>> > From dale.henrichs at gemtalksystems.com Tue Dec 8 21:35:46 2015 From: dale.henrichs at gemtalksystems.com (Dale Henrichs) Date: Tue Dec 8 21:35:51 2015 Subject: [Cuis] Sorting Unicode strings (Re: [Unicode] collation sequences (Re: [squeak-dev] Unicode Support)) In-Reply-To: References: <5665EF88.5030901@gemtalksystems.com> <56672D90.4020202@gemtalksystems.com> Message-ID: <56674D32.2000305@gemtalksystems.com> Euan, What I meant is that you can't _always_ use the code point for collation, i.e., sorting based on the value of code points is not always correct[1]. If I'm not mistaken the fully-composed and fully-decomposed forms can only be used for testing the equivalence of two strings[2] ... The Main Algorithm[3], starts by producing a normalized form of the string, but the subsequent steps (produce array, form sort key and compare) involves table lookups among other things .... Once you've produced a sort key for a string, the sort key does use "binary comparison" for collating , which is a byte by byte numeric comparison ... Dale [1] http://www.unicode.org/reports/tr10/#Common_Misperceptions [2] http://unicode.org/reports/tr15/pdtr15.html [3] http://www.unicode.org/reports/tr10/#Main_Algorithm On 12/08/2015 12:22 PM, EuanM wrote: > Dale - is that you can't depend on the value of a codepoint > *unless the string is either in fully-composed form > (or has just been fully-decomposed from a fully-composed form) * > > OR are there circumstances where even those two cases cannot be relied upon? > > On 8 December 2015 at 19:20, Dale Henrichs > wrote: >> >> On 12/07/2015 11:31 PM, H. Hirzel wrote: >>> Dale >>> >>> Thank you for your answer with links to the ICU library and the notes >>> about classes in Gemstone. Noteworthy that you have a class Utf8 as a >>> subclass of ByteArray. >>> >>> I understand that Gemstone uses the ICU library and thus does not >>> implement the algorithms in Smalltalk. >>> >>> I am currently looking into what the ICU library provides. >>> >>> I found as well a Ruby library [2] which implements CLDR [3] >>> >>> It has methods like this >>> >>> "Alphabetize a list using regular Ruby sort:" >>> >>> $> ["Art", "Wasa", "?lg", "Ved"].sort >>> $> ["Art", "Ved", "Wasa", "?lg"] >>> >>> Alphabetize a list using TwitterCLDR?s locale-aware sort: >>> >>> $> ["Art", "Wasa", "?lg", "Ved"].localize(:de).sort.to_a >>> $> ["?lg", "Art", "Ved", "Wasa"] >>> >>> I hope that given such an example it would not be too difficult to >>> reimplement a similar sort algorithm in Squeak/Cuis/Pharo. Currently >>> the interest is in getting sorting done in a cross-dialect-way. >>> >> I think that the issue (from a performance perspective) is that you can't >> depend upon the value of the code point when doing collation --- the main >> algorithm[5] is pretty much table based --- In addition to the different >> sort orders based on characters there are even more arcane sort rules where >> characters at the end of a word can affect the sort order of the word (for >> more info see[4]). >> >> It is worth looking at the Conformance section of the Unicode spec[1] as >> there are different levels of collation conformance ..... >> >> ICU conforms[2] to to UTS #10[3], the highest level of conformance ... >> >> It looks like TwitterCLDR[6] uses the Main Algorithm[5] with tailoring[7]. >> They don't claim to be conformant to the Unicode Collation Algorithm[3], but >> they are covering a big chunk of the standard use cases .... >> >> Dale >> >> [1] http://unicode.org/reports/tr10/#Conformance >> [2] http://userguide.icu-project.org/collation >> [3] http://www.unicode.org/reports/tr10/ >> [4] http://www.unicode.org/reports/tr10/#Introduction >> [5] http://www.unicode.org/reports/tr10/#Main_Algorithm >> [6] >> https://blog.twitter.com/2012/twittercldr-improving-internationalization-support-in-ruby >> [7] http://unicode.org/reports/tr10/#Tailoring From karlramberg at gmail.com Tue Dec 8 21:47:13 2015 From: karlramberg at gmail.com (karl ramberg) Date: Tue Dec 8 21:47:18 2015 Subject: [squeak-dev] The Inbox: Tools-pre.659.mcz In-Reply-To: <56671e94.8b4f370a.3ca22.14b7SMTPIN_ADDED_MISSING@mx.google.com> References: <56671e94.8b4f370a.3ca22.14b7SMTPIN_ADDED_MISSING@mx.google.com> Message-ID: Nice. Best, Karl On Tue, Dec 8, 2015 at 7:16 PM, wrote: > A new version of Tools was added to project The Inbox: > http://source.squeak.org/inbox/Tools-pre.659.mcz > > ==================== Summary ==================== > > Name: Tools-pre.659 > Author: pre > Time: 8 December 2015, 7:16:41.034 pm > UUID: a7cb3cba-074b-46a6-8a83-fada58007b25 > Ancestors: Tools-eem.658 > > This adds balloon help texts to the message list of the standard browser > to explain the symbols shown in front of message names. > > =============== Diff against Tools-eem.658 =============== > > Item was changed: > ----- Method: Browser>>messageHelpAt: (in category 'message list') ----- > messageHelpAt: anIndex > "Show the first n lines of the sources code of the selected > message." > > + | source formatted | > - | source formatted lineCount | > Preferences balloonHelpInMessageLists ifFalse: [^ nil]. > self messageList size < anIndex ifTrue: [^ nil]. > > source := (self selectedClassOrMetaClass >> (self messageList at: > anIndex)) getSource. > + source lineCount > 5 ifTrue: [ > + | sourceLines | > + sourceLines := (source asString lines copyFrom: 1 to: 5) > asOrderedCollection. > + sourceLines add: ' [...]'. > + source := sourceLines joinSeparatedBy: Character cr]. > + > formatted := SHTextStylerST80 new > classOrMetaClass: self selectedClassOrMetaClass; > styledTextFor: source asText. > - > - lineCount := 0. > - source doWithIndex: [:char :index | > - char = Character cr ifTrue: [lineCount := lineCount + 1]. > - lineCount > 10 ifTrue: [ > - formatted := formatted copyFrom: 1 to: index-1. > - formatted append: ' [...]'. > - ^ formatted]]. > > + ^ (Text newFrom: ((self messageIconHelpAt: anIndex) ifNotEmpty: > [:t | t , Character cr, Character cr])) > + append: formatted; > + yourself! > - ^ formatted! > > Item was added: > + ----- Method: Browser>>messageIconHelpAt: (in category 'message list') > ----- > + messageIconHelpAt: anIndex > + > + self class showMessageIcons > + ifFalse: [^ nil]. > + > + ^ ToolIconHelp iconHelpNamed: (ToolIcons > + iconForClass: self selectedClassOrMetaClass > + selector: (self messageList at: anIndex))! > > Item was added: > + Object subclass: #ToolIconHelp > + instanceVariableNames: '' > + classVariableNames: 'HelpTexts' > + poolDictionaries: '' > + category: 'Tools-Base'! > > Item was added: > + ----- Method: ToolIconHelp class>>abstract (in category 'as yet > unclassified') ----- > + abstract > + > + ^ 'This method is abstract.' ! > > Item was added: > + ----- Method: ToolIconHelp class>>arrowDown (in category 'as yet > unclassified') ----- > + arrowDown > + > + ^ 'This method is overriden by another method.'! > > Item was added: > + ----- Method: ToolIconHelp class>>arrowUp (in category 'as yet > unclassified') ----- > + arrowUp > + > + ^ 'This method overrides a super method.'! > > Item was added: > + ----- Method: ToolIconHelp class>>arrowUpAndDown (in category 'as yet > unclassified') ----- > + arrowUpAndDown > + > + ^ 'This method overrides and is overridden by other methods.'! > > Item was added: > + ----- Method: ToolIconHelp class>>blank (in category 'as yet > unclassified') ----- > + blank > + > + ^ ''! > > Item was added: > + ----- Method: ToolIconHelp class>>breakpoint (in category 'as yet > unclassified') ----- > + breakpoint > + > + ^ 'This method contains a breakpoint.'! > > Item was added: > + ----- Method: ToolIconHelp class>>flag (in category 'as yet > unclassified') ----- > + flag > + > + ^ 'This method needs attention of some kind.' ! > > Item was added: > + ----- Method: ToolIconHelp class>>helpTexts (in category 'as yet > unclassified') ----- > + helpTexts > + > + ^ HelpTexts ifNil: [HelpTexts := IdentityDictionary new]! > > Item was added: > + ----- Method: ToolIconHelp class>>iconHelpNamed: (in category 'as yet > unclassified') ----- > + iconHelpNamed: aSymbol > + > + (self respondsTo: aSymbol) > + ifTrue: [^self helpTexts at: aSymbol ifAbsentPut: [self > perform: aSymbol]] > + ifFalse: [^ ''] > + ! > > Item was added: > + ----- Method: ToolIconHelp class>>no (in category 'as yet unclassified') > ----- > + no > + > + ^ 'This method should not be implemented.' ! > > Item was added: > + ----- Method: ToolIconHelp class>>notOverridden (in category 'as yet > unclassified') ----- > + notOverridden > + > + ^ 'This method is abstract and has not been overridden.' ! > > Item was added: > + ----- Method: ToolIconHelp class>>primitive (in category 'as yet > unclassified') ----- > + primitive > + > + ^ 'This method implements a primitive.'! > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151208/9a55cb1f/attachment-0001.htm From commits at source.squeak.org Tue Dec 8 22:55:02 2015 From: commits at source.squeak.org (commits@source.squeak.org) Date: Tue Dec 8 22:55:04 2015 Subject: [squeak-dev] Daily Commit Log Message-ID: <20151208225502.9061.qmail@box4.squeakfoundation.org> Changes to Trunk (http://source.squeak.org/trunk.html) in the last 24 hours: http://lists.squeakfoundation.org/pipermail/packages/2015-December/009210.html Name: Morphic-cwp.1057 Ancestors: Morphic-dtl.1056 Don't clobber the existing reference pool in PasteUpMorph>>referencePool ============================================= http://lists.squeakfoundation.org/pipermail/packages/2015-December/009211.html Name: Installer-Core-topa.402 Ancestors: Installer-Core-topa.401 This won't work in any way. Until we have a good way integrating Installer/SqueakMap and Metacello, remove this as it is confusing and unhelpful. ============================================= From euanmee at gmail.com Tue Dec 8 23:50:38 2015 From: euanmee at gmail.com (EuanM) Date: Tue Dec 8 23:50:40 2015 Subject: [Pharo-dev] [Cuis] Sorting Unicode strings (Re: [Unicode] collation sequences (Re: [squeak-dev] Unicode Support)) In-Reply-To: References: <5665EF88.5030901@gemtalksystems.com> <56672D90.4020202@gemtalksystems.com> Message-ID: Dale, yes - sorting based on the value of codepoints is almost always guaranteed to be wrong. Sorting is an application-specific issue, not a technical Unicode issue, as there is more than one canonical sort order per culture, and there is often more than one culture per writing system. e.g. ISO Latin 1 / Latin 9 covers these cultures (amongst others) English (2 sort orders); Spanish; French (2 sort orders); German (2 sort orders); Swedish; etc German sort order differs from Swedish for the same characters, etc Todd, My thinking is that if we implement fully-composed strings as heterogenous arrays, we sidestep a lot of the complexity of the ICU. If it turns out that the performance is terrible, we can then seek to incorporate the ICU. On 8 December 2015 at 22:36, Todd Blanchard wrote: > I just want to second Dale's endorsement of the ICU library. It has been > around a long time (originally developed by Taligent) and it provides the > base unicode capabilities for an awful lot of software. > > I think it would make more sense to bring icu into Smalltalk as a > NativeBoost library than to spend resources reimplementing and maintaining > it. > > -Todd Blanchard > > On Dec 8, 2015, at 11:20, Dale Henrichs > wrote: > > On 12/07/2015 11:31 PM, H. Hirzel wrote: > > Dale > > Thank you for your answer with links to the ICU library and the notes > about classes in Gemstone. Noteworthy that you have a class Utf8 as a > subclass of ByteArray. > > I understand that Gemstone uses the ICU library and thus does not > implement the algorithms in Smalltalk. > > I am currently looking into what the ICU library provides. > > From euanmee at gmail.com Tue Dec 8 23:52:19 2015 From: euanmee at gmail.com (EuanM) Date: Tue Dec 8 23:52:21 2015 Subject: [Pharo-dev] [Cuis] Sorting Unicode strings (Re: [Unicode] collation sequences (Re: [squeak-dev] Unicode Support)) In-Reply-To: References: <5665EF88.5030901@gemtalksystems.com> <56672D90.4020202@gemtalksystems.com> Message-ID: Equally old are the NextStep Object C functions which are now embodied within MacOS X. It might also be informative for us to see what's done there. On 8 December 2015 at 23:50, EuanM wrote: > Dale, > > yes - sorting based on the value of codepoints is almost always > guaranteed to be wrong. Sorting is an application-specific issue, not > a technical Unicode issue, as there is more than one canonical sort > order per culture, and there is often more than one culture per > writing system. > > e.g. ISO Latin 1 / Latin 9 > covers these cultures (amongst others) > English (2 sort orders); Spanish; French (2 sort orders); German (2 > sort orders); Swedish; etc > > German sort order differs from Swedish for the same characters, etc > > Todd, > > My thinking is that if we implement fully-composed strings as > heterogenous arrays, we sidestep a lot of the complexity of the ICU. > > If it turns out that the performance is terrible, we can then seek to > incorporate the ICU. > > > On 8 December 2015 at 22:36, Todd Blanchard wrote: >> I just want to second Dale's endorsement of the ICU library. It has been >> around a long time (originally developed by Taligent) and it provides the >> base unicode capabilities for an awful lot of software. >> >> I think it would make more sense to bring icu into Smalltalk as a >> NativeBoost library than to spend resources reimplementing and maintaining >> it. >> >> -Todd Blanchard >> >> On Dec 8, 2015, at 11:20, Dale Henrichs >> wrote: >> >> On 12/07/2015 11:31 PM, H. Hirzel wrote: >> >> Dale >> >> Thank you for your answer with links to the ICU library and the notes >> about classes in Gemstone. Noteworthy that you have a class Utf8 as a >> subclass of ByteArray. >> >> I understand that Gemstone uses the ICU library and thus does not >> implement the algorithms in Smalltalk. >> >> I am currently looking into what the ICU library provides. >> >> From lewis at mail.msen.com Wed Dec 9 00:03:32 2015 From: lewis at mail.msen.com (David T. Lewis) Date: Wed Dec 9 00:03:33 2015 Subject: [squeak-dev] The Inbox: Tools-pre.659.mcz In-Reply-To: References: <56671e94.8b4f370a.3ca22.14b7SMTPIN_ADDED_MISSING@mx.google.com> Message-ID: <20151209000332.GA990@shell.msen.com> On Tue, Dec 08, 2015 at 10:47:13PM +0100, karl ramberg wrote: > > On Tue, Dec 8, 2015 at 7:16 PM, wrote: > > > A new version of Tools was added to project The Inbox: > > http://source.squeak.org/inbox/Tools-pre.659.mcz > > > > ==================== Summary ==================== > > > > Name: Tools-pre.659 > > Author: pre > > Time: 8 December 2015, 7:16:41.034 pm > > UUID: a7cb3cba-074b-46a6-8a83-fada58007b25 > > Ancestors: Tools-eem.658 > > > > This adds balloon help texts to the message list of the standard browser > > to explain the symbols shown in front of message names. > > > > Nice. > > Best, > Karl This sounds really good, but are there more updates on the way? I loaded Tools-pre.659, but I do not see balloon help when I hold the mouse cursor over the icons. Thanks, Dave From euanmee at gmail.com Wed Dec 9 00:15:22 2015 From: euanmee at gmail.com (EuanM) Date: Wed Dec 9 00:15:25 2015 Subject: [Pharo-dev] [Cuis] Sorting Unicode strings (Re: [Unicode] collation sequences (Re: [squeak-dev] Unicode Support)) In-Reply-To: References: <5665EF88.5030901@gemtalksystems.com> <56672D90.4020202@gemtalksystems.com> Message-ID: Hi Todd, it's taken me til now to put my thoughts into words on this issue. I think we should make it work first. This will allow us to gain more insight into the issues, and create documentation about the process that we, as a community, understand. If ICU is the right way to go, we can *then* "make it right". On 8 December 2015 at 22:36, Todd Blanchard wrote: > I just want to second Dale's endorsement of the ICU library. It has been > around a long time (originally developed by Taligent) and it provides the > base unicode capabilities for an awful lot of software. > > I think it would make more sense to bring icu into Smalltalk as a > NativeBoost library than to spend resources reimplementing and maintaining > it. > > -Todd Blanchard > > On Dec 8, 2015, at 11:20, Dale Henrichs > wrote: > > On 12/07/2015 11:31 PM, H. Hirzel wrote: > > Dale > > Thank you for your answer with links to the ICU library and the notes > about classes in Gemstone. Noteworthy that you have a class Utf8 as a > subclass of ByteArray. > > I understand that Gemstone uses the ICU library and thus does not > implement the algorithms in Smalltalk. > > I am currently looking into what the ICU library provides. > > From euanmee at gmail.com Wed Dec 9 00:33:51 2015 From: euanmee at gmail.com (EuanM) Date: Wed Dec 9 00:33:53 2015 Subject: [Cuis] Sorting Unicode strings (Re: [Unicode] collation sequences (Re: [squeak-dev] Unicode Support)) In-Reply-To: <56674D32.2000305@gemtalksystems.com> References: <5665EF88.5030901@gemtalksystems.com> <56672D90.4020202@gemtalksystems.com> <56674D32.2000305@gemtalksystems.com> Message-ID: Reading up http://www.unicode.org/reports/tr15/#Examples The Unicode standard seems to require you never to make aGermanStrasse equivalent in a sort order to the ligatured version, aGermanStra?e This seems counter-intuitive to me. Is there a reason for this? Have I just simply picked this up wrongly? On 8 December 2015 at 21:35, Dale Henrichs wrote: > Euan, > > What I meant is that you can't _always_ use the code point for collation, > i.e., sorting based on the value of code points is not always correct[1]. > > If I'm not mistaken the fully-composed and fully-decomposed forms can only > be used for testing the equivalence of two strings[2] ... > > The Main Algorithm[3], starts by producing a normalized form of the string, > but the subsequent steps (produce array, form sort key and compare) involves > table lookups among other things .... > > Once you've produced a sort key for a string, the sort key does use "binary > comparison" for collating , which is a byte by byte numeric comparison ... > > Dale > > [1] http://www.unicode.org/reports/tr10/#Common_Misperceptions > [2] http://unicode.org/reports/tr15/pdtr15.html > [3] http://www.unicode.org/reports/tr10/#Main_Algorithm > > > On 12/08/2015 12:22 PM, EuanM wrote: >> >> Dale - is that you can't depend on the value of a codepoint >> *unless the string is either in fully-composed form >> (or has just been fully-decomposed from a fully-composed form) * >> >> OR are there circumstances where even those two cases cannot be relied >> upon? >> >> On 8 December 2015 at 19:20, Dale Henrichs >> wrote: >>> >>> >>> On 12/07/2015 11:31 PM, H. Hirzel wrote: >>>> >>>> Dale >>>> >>>> Thank you for your answer with links to the ICU library and the notes >>>> about classes in Gemstone. Noteworthy that you have a class Utf8 as a >>>> subclass of ByteArray. >>>> >>>> I understand that Gemstone uses the ICU library and thus does not >>>> implement the algorithms in Smalltalk. >>>> >>>> I am currently looking into what the ICU library provides. >>>> >>>> I found as well a Ruby library [2] which implements CLDR [3] >>>> >>>> It has methods like this >>>> >>>> "Alphabetize a list using regular Ruby sort:" >>>> >>>> $> ["Art", "Wasa", "?lg", "Ved"].sort >>>> $> ["Art", "Ved", "Wasa", "?lg"] >>>> >>>> Alphabetize a list using TwitterCLDR?s locale-aware sort: >>>> >>>> $> ["Art", "Wasa", "?lg", "Ved"].localize(:de).sort.to_a >>>> $> ["?lg", "Art", "Ved", "Wasa"] >>>> >>>> I hope that given such an example it would not be too difficult to >>>> reimplement a similar sort algorithm in Squeak/Cuis/Pharo. Currently >>>> the interest is in getting sorting done in a cross-dialect-way. >>>> >>> I think that the issue (from a performance perspective) is that you can't >>> depend upon the value of the code point when doing collation --- the main >>> algorithm[5] is pretty much table based --- In addition to the different >>> sort orders based on characters there are even more arcane sort rules >>> where >>> characters at the end of a word can affect the sort order of the word >>> (for >>> more info see[4]). >>> >>> It is worth looking at the Conformance section of the Unicode spec[1] as >>> there are different levels of collation conformance ..... >>> >>> ICU conforms[2] to to UTS #10[3], the highest level of conformance ... >>> >>> It looks like TwitterCLDR[6] uses the Main Algorithm[5] with >>> tailoring[7]. >>> They don't claim to be conformant to the Unicode Collation Algorithm[3], >>> but >>> they are covering a big chunk of the standard use cases .... >>> >>> Dale >>> >>> [1] http://unicode.org/reports/tr10/#Conformance >>> [2] http://userguide.icu-project.org/collation >>> [3] http://www.unicode.org/reports/tr10/ >>> [4] http://www.unicode.org/reports/tr10/#Introduction >>> [5] http://www.unicode.org/reports/tr10/#Main_Algorithm >>> [6] >>> >>> https://blog.twitter.com/2012/twittercldr-improving-internationalization-support-in-ruby >>> [7] http://unicode.org/reports/tr10/#Tailoring > > From dale.henrichs at gemtalksystems.com Wed Dec 9 00:43:19 2015 From: dale.henrichs at gemtalksystems.com (Dale Henrichs) Date: Wed Dec 9 00:43:24 2015 Subject: [Cuis] Sorting Unicode strings (Re: [Unicode] collation sequences (Re: [squeak-dev] Unicode Support)) In-Reply-To: References: <5665EF88.5030901@gemtalksystems.com> <56672D90.4020202@gemtalksystems.com> <56674D32.2000305@gemtalksystems.com> Message-ID: <56677927.3060109@gemtalksystems.com> On 12/08/2015 04:33 PM, EuanM wrote: > Reading up http://www.unicode.org/reports/tr15/#Examples > > The Unicode standard seems to require you never to make > aGermanStrasse > equivalent in a sort order to the ligatured version, > aGermanStra?e I didn't see anything in the examples related to this ... could you be a bit more specific? Dale From euanmee at gmail.com Wed Dec 9 00:59:43 2015 From: euanmee at gmail.com (EuanM) Date: Wed Dec 9 00:59:45 2015 Subject: [Cuis] Sorting Unicode strings (Re: [Unicode] collation sequences (Re: [squeak-dev] Unicode Support)) In-Reply-To: <56677927.3060109@gemtalksystems.com> References: <5665EF88.5030901@gemtalksystems.com> <56672D90.4020202@gemtalksystems.com> <56674D32.2000305@gemtalksystems.com> <56677927.3060109@gemtalksystems.com> Message-ID: http://www.unicode.org/reports/tr15/#Stable_Code_Points Table 7, the discussion of Ligatures, (which uses the ligature of "ffi" as its example) Every time I think I'm about to grokk this standard, something like this crops up. On 9 December 2015 at 00:43, Dale Henrichs wrote: > > > On 12/08/2015 04:33 PM, EuanM wrote: >> >> Reading up http://www.unicode.org/reports/tr15/#Examples >> >> The Unicode standard seems to require you never to make >> aGermanStrasse >> equivalent in a sort order to the ligatured version, >> aGermanStra?e > > I didn't see anything in the examples related to this ... could you be a bit > more specific? > > Dale From tim at rowledge.org Wed Dec 9 01:13:58 2015 From: tim at rowledge.org (tim Rowledge) Date: Wed Dec 9 01:14:02 2015 Subject: [squeak-dev] Updating past 15464 (spur) Message-ID: <9251DA50-87A4-47FE-83A4-50E7EAE2F1A0@rowledge.org> This must have been noticed and I swear I saw messages about it not so long ago but I can?t find anything right now. Very frustrating! I have an image updated to 15464 and when attempting to update it stops at Kernel.spur-dtl.850, opening an MC merge window. Attempting to do a merge doesn?t seem to work - something in one of the conflicts is nil when it shouldn?t be and so trying the relatively obvious ?All Newer? fails. Manually choosing ?Keep? for each of the conflicts allows us to get a bit further. This time I get to Collections.spur-nice.569 where another merge window opens and, oh nice, an MNU from the timer event loop. At which point we are of course locked. It must be possible to proceed since we have newer trunk images to download. Did we not find a way to allow us to just do the update ?normally?? tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim When you earnestly believe you can compensate for a lack of skill by doubling your efforts, there's no end to what you can't do From euanmee at gmail.com Wed Dec 9 01:26:31 2015 From: euanmee at gmail.com (EuanM) Date: Wed Dec 9 01:26:34 2015 Subject: [Pharo-dev] [squeak-dev] Re: [Unicode] Class Unicode --> data from unicode.org In-Reply-To: <1449502640-sup-2085@email.archlab.tuwien.ac.at> References: <1449499341-sup-3116@email.archlab.tuwien.ac.at> <1449502640-sup-2085@email.archlab.tuwien.ac.at> Message-ID: Yes, clean room approaches are overkill. Obviousness in code has long been found to be a protection against copyright. For instance, you cannot copyright a programme in BASIC that says 10 Print "Hello World" 20 GOTO 10 (And by copyright, I mean prevent anyone else in the world from typing that programme and distributing their copy). One of the great advantage of writing weird, convoluted or just plain dense clever code is that it becomes non-obvious and therefore indubitably covered under copyright. The mere fact that code has been written somewhere, does *not* make it unusable elsewhere. (As an aside, I tend to write plain, straightforward code. Asserting copyright on a lot of it would be hard). One of the great advantages of not doing my coding in the USA is that Stallman would have to come after me in my jurisdiction, not his. Probably a special case of security through obscurity. Usual disclaimer. IANAL. Just interested in these issues from having developed legal support systems (including litigation support systems). As Hannes says, we haven't hit an area of the project where it is an issue yet. But personally, I'm quite relaxed about looking at the Gnu Smalltalk codebase. I'm pretty certain I can re-implement without violating copyright. In the same way I can pr?cis and paraphrase a short story without violating the copyright of the short story. Can any of you tell me the story of Hansel and Gretel without violating the copyright of some recently published re-telling of the tale? I'm pretty certain you could. Please remember that it is much easier to avoid copyright violation than it is to avoid patent violation, in these sorts of cases. None of the Gnu Smalltalk base is patented. Cheers, Euan (back to reading the ICU) On 7 December 2015 at 16:06, Martin B?hr wrote: > Excerpts from Martin B?hr's message of 2015-12-07 16:17:28 +0100: >> randal schwartz suggests a cleanroom approach, though i personally think that >> is overkill. if a cleanroom approach were really necessary than i would never >> be allowed to switch jobs because once i have been working on one companys >> private code i would be tainted for life, as i could accidentally reproduce the >> same code in my next job. >> >> however, either you get the GNU-smalltalk authors to relicense their unicode >> classes under the MIT license, or they will need to be rewritten from scratch. >> >> i would suggest asking them in the name of having a common code-base for all >> smalltalks. > > http://lists.squeakfoundation.org/pipermail/squeak-dev/2008-January/123817.html > is worth reading in that context: > > So it's mainly an issue of being willing to cooperate. I think all GNU > Smalltalk people (almost all?) are and it's also true at least for some > Squeak people. If somebody took a package (several thousands lines of > code, etc.) wholesale, ported it to Squeak, and licensed it under MIT on > SqueakMap that would be copyright violation. But everything can be done > "cum grano salis". Nobody is going to sue you if you copy an interface > from GNU Smalltalk, reimplement in Squeak (for fun!), but you did look > at the method comments in GNU Smalltalk -- and OMG maybe you got a > glimpse of the source code just below! Copyright does not protect > having similar implementations. > > I think we're all for interoperability and for exchange of opinions > within the communities. > > so there you have the open invitation to go and ask... > > and read this > http://lists.squeakfoundation.org/pipermail/squeak-dev/2008-January/123838.html > for a good summary on the issue. > > greetings, martin. > > -- > eKita - the online platform for your entire academic life > -- > chief engineer eKita.co > pike programmer pike.lysator.liu.se caudium.net societyserver.org > secretary beijinglug.org > mentor fossasia.org > foresight developer foresightlinux.org realss.com > unix sysadmin > Martin B?hr working in china http://societyserver.org/mbaehr/ From euanmee at gmail.com Wed Dec 9 01:30:58 2015 From: euanmee at gmail.com (EuanM) Date: Wed Dec 9 01:31:01 2015 Subject: [Cuis] Sorting Unicode strings (Re: [Unicode] collation sequences (Re: [squeak-dev] Unicode Support)) In-Reply-To: References: <5665EF88.5030901@gemtalksystems.com> <56672D90.4020202@gemtalksystems.com> <56674D32.2000305@gemtalksystems.com> <56677927.3060109@gemtalksystems.com> Message-ID: I've just discovered the ICU's official listing for an ICU wrapper for Smalltalk: https://schrievkrom.wordpress.com/tag/icu/ Listed on http://site.icu-project.org/related In turn, it mentions: ICU: Initial source code for Squeak/Pharo. http://ss3.gemstone.com/ss/ICU.html by Marten Feldtmann, Jan van de Sandt ICU-V2 Unicode support http://ss3.gemstone.com/ss/ICU-V2.html also by Marten Feldtmann, Jan van de Sandt On 9 December 2015 at 00:59, EuanM wrote: > http://www.unicode.org/reports/tr15/#Stable_Code_Points > Table 7, the discussion of Ligatures, (which uses the ligature of > "ffi" as its example) > > Every time I think I'm about to grokk this standard, something like > this crops up. > > > > On 9 December 2015 at 00:43, Dale Henrichs > wrote: >> >> >> On 12/08/2015 04:33 PM, EuanM wrote: >>> >>> Reading up http://www.unicode.org/reports/tr15/#Examples >>> >>> The Unicode standard seems to require you never to make >>> aGermanStrasse >>> equivalent in a sort order to the ligatured version, >>> aGermanStra?e >> >> I didn't see anything in the examples related to this ... could you be a bit >> more specific? >> >> Dale From euanmee at gmail.com Wed Dec 9 01:31:41 2015 From: euanmee at gmail.com (EuanM) Date: Wed Dec 9 01:31:45 2015 Subject: [Cuis] Sorting Unicode strings (Re: [Unicode] collation sequences (Re: [squeak-dev] Unicode Support)) In-Reply-To: References: <5665EF88.5030901@gemtalksystems.com> <56672D90.4020202@gemtalksystems.com> <56674D32.2000305@gemtalksystems.com> <56677927.3060109@gemtalksystems.com> Message-ID: "Accessing ICU library from Squeak/Pharo via NativeBoost interface FFI. This is my port to Squeak/Pharo - to have same interface available under VASmalltalk and Pharo/Squeak." On 9 December 2015 at 01:30, EuanM wrote: > I've just discovered the ICU's official listing for an ICU wrapper for > Smalltalk: > https://schrievkrom.wordpress.com/tag/icu/ > > Listed on > http://site.icu-project.org/related > > In turn, it mentions: > > ICU: Initial source code for Squeak/Pharo. > http://ss3.gemstone.com/ss/ICU.html > by Marten Feldtmann, Jan van de Sandt > > ICU-V2 Unicode support > http://ss3.gemstone.com/ss/ICU-V2.html > also by Marten Feldtmann, Jan van de Sandt > > On 9 December 2015 at 00:59, EuanM wrote: >> http://www.unicode.org/reports/tr15/#Stable_Code_Points >> Table 7, the discussion of Ligatures, (which uses the ligature of >> "ffi" as its example) >> >> Every time I think I'm about to grokk this standard, something like >> this crops up. >> >> >> >> On 9 December 2015 at 00:43, Dale Henrichs >> wrote: >>> >>> >>> On 12/08/2015 04:33 PM, EuanM wrote: >>>> >>>> Reading up http://www.unicode.org/reports/tr15/#Examples >>>> >>>> The Unicode standard seems to require you never to make >>>> aGermanStrasse >>>> equivalent in a sort order to the ligatured version, >>>> aGermanStra?e >>> >>> I didn't see anything in the examples related to this ... could you be a bit >>> more specific? >>> >>> Dale From mbaehr at email.archlab.tuwien.ac.at Wed Dec 9 01:43:35 2015 From: mbaehr at email.archlab.tuwien.ac.at (=?utf-8?q?Martin_B=C3=A4hr?=) Date: Wed Dec 9 01:44:34 2015 Subject: [Cuis] Sorting Unicode strings (Re: [Unicode] collation sequences (Re: [squeak-dev] Unicode Support)) In-Reply-To: References: <5665EF88.5030901@gemtalksystems.com> <56672D90.4020202@gemtalksystems.com> <56674D32.2000305@gemtalksystems.com> <56677927.3060109@gemtalksystems.com> Message-ID: <1449623848-sup-2820@email.archlab.tuwien.ac.at> Excerpts from EuanM's message of 2015-12-09 01:59:43 +0100: > http://www.unicode.org/reports/tr15/#Stable_Code_Points > Table 7, the discussion of Ligatures, (which uses the ligature of > "ffi" as its example) ? is not a ligature of ss, but is a different character. historically it evolved from a ligature of long s (?) and round s but it is no no longer a true ligature that can be decomposed without sideeffects. they are pronounced differently and there are german words where the difference of using ? vs ss results in different meaning of the word: (eg Bu?e vs Busse: penance vs busses) https://en.wikipedia.org/wiki/? it is allowed to use ss as a replacement for ? only when ? itself is not available. this is similar to the german umlauts: ?,? and ? which can be decomposed into ae, oe and ue, but those forms are a mere approximation, not equivalent. in a medium where umlauts are available, using a decomposed form can be considered an error. greetings, martin. -- eKita - the online platform for your entire academic life -- chief engineer eKita.co pike programmer pike.lysator.liu.se caudium.net societyserver.org secretary beijinglug.org mentor fossasia.org foresight developer foresightlinux.org realss.com unix sysadmin Martin B?hr working in china http://societyserver.org/mbaehr/ From mbaehr at email.archlab.tuwien.ac.at Wed Dec 9 02:36:53 2015 From: mbaehr at email.archlab.tuwien.ac.at (=?utf-8?q?Martin_B=C3=A4hr?=) Date: Wed Dec 9 02:37:51 2015 Subject: [Cuis] Sorting Unicode strings (Re: [Unicode] collation sequences (Re: [squeak-dev] Unicode Support)) In-Reply-To: <1449623848-sup-2820@email.archlab.tuwien.ac.at> References: <5665EF88.5030901@gemtalksystems.com> <56672D90.4020202@gemtalksystems.com> <56674D32.2000305@gemtalksystems.com> <56677927.3060109@gemtalksystems.com> <1449623848-sup-2820@email.archlab.tuwien.ac.at> Message-ID: <1449628048-sup-4275@email.archlab.tuwien.ac.at> Excerpts from Martin B?hr's message of 2015-12-09 02:43:35 +0100: > Excerpts from EuanM's message of 2015-12-09 01:59:43 +0100: > > http://www.unicode.org/reports/tr15/#Stable_Code_Points > > Table 7, the discussion of Ligatures, (which uses the ligature of > > "ffi" as its example) > > ? is not a ligature of ss, but is a different character. rereading this, i think i am wrong, in that this has nothing to do with ? vs ss. looking at the standard i also don't understand your conclusion. what the standard seems to say is, that the ffi ligature is not equivalent to plain ffi because, if i write a string with the ligature, then it is a different string than plain "ffi", because both forms are printable. on the other hand ? and a" (two different encodings for ?) are the same, because the printed forms are always identical. however that doesn't mean that they are sorted differently. german sorting rules for example explicitly state that ? and ss are sorted the same. (at least according to wikipedia :-) and surely, ffi ligature and ffi are sorted the same too. greetings, martin. -- eKita - the online platform for your entire academic life -- chief engineer eKita.co pike programmer pike.lysator.liu.se caudium.net societyserver.org secretary beijinglug.org mentor fossasia.org foresight developer foresightlinux.org realss.com unix sysadmin Martin B?hr working in china http://societyserver.org/mbaehr/ From euanmee at gmail.com Wed Dec 9 02:45:36 2015 From: euanmee at gmail.com (EuanM) Date: Wed Dec 9 02:45:39 2015 Subject: [Cuis] Sorting Unicode strings (Re: [Unicode] collation sequences (Re: [squeak-dev] Unicode Support)) In-Reply-To: <1449628048-sup-4275@email.archlab.tuwien.ac.at> References: <5665EF88.5030901@gemtalksystems.com> <56672D90.4020202@gemtalksystems.com> <56674D32.2000305@gemtalksystems.com> <56677927.3060109@gemtalksystems.com> <1449623848-sup-2820@email.archlab.tuwien.ac.at> <1449628048-sup-4275@email.archlab.tuwien.ac.at> Message-ID: "The ffi_ligature (U+FB03) is not decomposed, because it has a compatibility mapping, not a canonical mapping." Standard's text from Table 7 "what the standard seems to say is, that the ffi ligature is not equivalent to plain ffi because, if i write a string with the ligature, then it is a different string than plain "ffi", because both forms are printable. on the other hand ? and a" (two different encodings for ?) are the same, because the printed forms are always identical." --Martin I agree with that interpretation. I'm struggling to be clear about the consequences for equality testing. For sorting - every sort order can be created to go along with every encoding, so I suppose it just depends on which pre-assembled sort order is used. On 9 December 2015 at 02:36, Martin B?hr wrote: > Excerpts from Martin B?hr's message of 2015-12-09 02:43:35 +0100: >> Excerpts from EuanM's message of 2015-12-09 01:59:43 +0100: >> > http://www.unicode.org/reports/tr15/#Stable_Code_Points >> > Table 7, the discussion of Ligatures, (which uses the ligature of >> > "ffi" as its example) >> >> ? is not a ligature of ss, but is a different character. > > rereading this, i think i am wrong, in that this has nothing to do with ? vs ss. > > looking at the standard i also don't understand your conclusion. > > what the standard seems to say is, that the ffi ligature is not equivalent to > plain ffi because, if i write a string with the ligature, then it is a > different string than plain "ffi", because both forms are printable. > on the other hand ? and a" (two different encodings for ?) are the same, > because the printed forms are always identical. > > however that doesn't mean that they are sorted differently. > german sorting rules for example explicitly state that ? and ss are sorted the same. > (at least according to wikipedia :-) > and surely, ffi ligature and ffi are sorted the same too. > > greetings, martin. > > -- > eKita - the online platform for your entire academic life > -- > chief engineer eKita.co > pike programmer pike.lysator.liu.se caudium.net societyserver.org > secretary beijinglug.org > mentor fossasia.org > foresight developer foresightlinux.org realss.com > unix sysadmin > Martin B?hr working in china http://societyserver.org/mbaehr/ From commits at source.squeak.org Wed Dec 9 02:46:08 2015 From: commits at source.squeak.org (commits@source.squeak.org) Date: Wed Dec 9 02:46:10 2015 Subject: [squeak-dev] The Trunk: Morphic-tpr.1058.mcz Message-ID: tim Rowledge uploaded a new version of Morphic to project The Trunk: http://source.squeak.org/trunk/Morphic-tpr.1058.mcz ==================== Summary ==================== Name: Morphic-tpr.1058 Author: tpr Time: 8 December 2015, 6:45:23.463 pm UUID: c438744a-71c4-45a0-ade8-7ecf52f50f89 Ancestors: Morphic-cwp.1057, Morphic-tpr.1057 Transfer from inbox, keep up to date. =============== Diff against Morphic-cwp.1057 =============== Item was added: + ----- Method: HandMorph>>gridPointRaw (in category 'gridded cursor') ----- + gridPointRaw + "return my latest position gridded" + ^self griddedPoint: self currentEvent position! Item was added: + ----- Method: HandMorph>>gridTo:origin: (in category 'gridded cursor') ----- + gridTo: grid origin: offset + "set a couple of properties to specify gridding for the temporaryCursor; instvars would be nicer" + self setProperty: #gridStep toValue: grid; + setProperty: #gridOffset toValue: offset! Item was added: + ----- Method: HandMorph>>griddedPoint: (in category 'gridded cursor') ----- + griddedPoint: aPoint + "return the equivalent point snapped to the grid, if indeed any gridding is set" + self valueOfProperty: #gridStep ifPresentDo: [:grid| |offset| + offset := self valueOfProperty: #gridOffset ifAbsent: [0@0]. + ^ offset + (aPoint + (grid //2) - offset truncateTo: grid)]. + ^aPoint! Item was changed: ----- Method: HandMorph>>position: (in category 'geometry') ----- position: aPoint "Overridden to align submorph origins to the grid if gridding is on." | adjustedPosition delta box | adjustedPosition := aPoint. + temporaryCursor ifNotNil: [adjustedPosition := (self griddedPoint: adjustedPosition) + temporaryCursorOffset]. - temporaryCursor ifNotNil: [adjustedPosition := adjustedPosition + temporaryCursorOffset]. "Copied from Morph to avoid owner layoutChanged" "Change the position of this morph and and all of its submorphs." delta := adjustedPosition - bounds topLeft. delta isZero ifTrue: [^ self]. "Null change" box := self fullBounds. (delta dotProduct: delta) > 100 ifTrue:[ "e.g., more than 10 pixels moved" self invalidRect: box. self invalidRect: (box translateBy: delta). ] ifFalse:[ self invalidRect: (box merge: (box translateBy: delta)). ]. self privateFullMoveBy: delta. ! Item was added: + ----- Method: HandMorph>>turnOffGridding (in category 'gridded cursor') ----- + turnOffGridding + "remove the gridding properties to stop gridding" + self removeProperty: #gridStep; removeProperty: #gridOffset! From karlramberg at gmail.com Wed Dec 9 05:07:20 2015 From: karlramberg at gmail.com (karl ramberg) Date: Wed Dec 9 05:07:23 2015 Subject: [squeak-dev] The Inbox: Tools-pre.659.mcz In-Reply-To: <20151209000332.GA990@shell.msen.com> References: <56671e94.8b4f370a.3ca22.14b7SMTPIN_ADDED_MISSING@mx.google.com> <20151209000332.GA990@shell.msen.com> Message-ID: Turn on Preferences balloonHelpInMessageLists Best, Karl On Wed, Dec 9, 2015 at 1:03 AM, David T. Lewis wrote: > On Tue, Dec 08, 2015 at 10:47:13PM +0100, karl ramberg wrote: > > > > On Tue, Dec 8, 2015 at 7:16 PM, wrote: > > > > > A new version of Tools was added to project The Inbox: > > > http://source.squeak.org/inbox/Tools-pre.659.mcz > > > > > > ==================== Summary ==================== > > > > > > Name: Tools-pre.659 > > > Author: pre > > > Time: 8 December 2015, 7:16:41.034 pm > > > UUID: a7cb3cba-074b-46a6-8a83-fada58007b25 > > > Ancestors: Tools-eem.658 > > > > > > This adds balloon help texts to the message list of the standard > browser > > > to explain the symbols shown in front of message names. > > > > > > > Nice. > > > > Best, > > Karl > > This sounds really good, but are there more updates on the way? I loaded > Tools-pre.659, but I do not see balloon help when I hold the mouse cursor > over the icons. > > Thanks, > Dave > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151209/af14a829/attachment.htm From commits at source.squeak.org Wed Dec 9 05:21:52 2015 From: commits at source.squeak.org (commits@source.squeak.org) Date: Wed Dec 9 05:21:54 2015 Subject: [squeak-dev] The Trunk: Tools-pre.659.mcz Message-ID: David T. Lewis uploaded a new version of Tools to project The Trunk: http://source.squeak.org/trunk/Tools-pre.659.mcz ==================== Summary ==================== Name: Tools-pre.659 Author: pre Time: 8 December 2015, 7:16:41.034 pm UUID: a7cb3cba-074b-46a6-8a83-fada58007b25 Ancestors: Tools-eem.658 This adds balloon help texts to the message list of the standard browser to explain the symbols shown in front of message names. =============== Diff against Tools-eem.658 =============== Item was changed: ----- Method: Browser>>messageHelpAt: (in category 'message list') ----- messageHelpAt: anIndex "Show the first n lines of the sources code of the selected message." + | source formatted | - | source formatted lineCount | Preferences balloonHelpInMessageLists ifFalse: [^ nil]. self messageList size < anIndex ifTrue: [^ nil]. source := (self selectedClassOrMetaClass >> (self messageList at: anIndex)) getSource. + source lineCount > 5 ifTrue: [ + | sourceLines | + sourceLines := (source asString lines copyFrom: 1 to: 5) asOrderedCollection. + sourceLines add: ' [...]'. + source := sourceLines joinSeparatedBy: Character cr]. + formatted := SHTextStylerST80 new classOrMetaClass: self selectedClassOrMetaClass; styledTextFor: source asText. - - lineCount := 0. - source doWithIndex: [:char :index | - char = Character cr ifTrue: [lineCount := lineCount + 1]. - lineCount > 10 ifTrue: [ - formatted := formatted copyFrom: 1 to: index-1. - formatted append: ' [...]'. - ^ formatted]]. + ^ (Text newFrom: ((self messageIconHelpAt: anIndex) ifNotEmpty: [:t | t , Character cr, Character cr])) + append: formatted; + yourself! - ^ formatted! Item was added: + ----- Method: Browser>>messageIconHelpAt: (in category 'message list') ----- + messageIconHelpAt: anIndex + + self class showMessageIcons + ifFalse: [^ nil]. + + ^ ToolIconHelp iconHelpNamed: (ToolIcons + iconForClass: self selectedClassOrMetaClass + selector: (self messageList at: anIndex))! Item was added: + Object subclass: #ToolIconHelp + instanceVariableNames: '' + classVariableNames: 'HelpTexts' + poolDictionaries: '' + category: 'Tools-Base'! Item was added: + ----- Method: ToolIconHelp class>>abstract (in category 'as yet unclassified') ----- + abstract + + ^ 'This method is abstract.' ! Item was added: + ----- Method: ToolIconHelp class>>arrowDown (in category 'as yet unclassified') ----- + arrowDown + + ^ 'This method is overriden by another method.'! Item was added: + ----- Method: ToolIconHelp class>>arrowUp (in category 'as yet unclassified') ----- + arrowUp + + ^ 'This method overrides a super method.'! Item was added: + ----- Method: ToolIconHelp class>>arrowUpAndDown (in category 'as yet unclassified') ----- + arrowUpAndDown + + ^ 'This method overrides and is overridden by other methods.'! Item was added: + ----- Method: ToolIconHelp class>>blank (in category 'as yet unclassified') ----- + blank + + ^ ''! Item was added: + ----- Method: ToolIconHelp class>>breakpoint (in category 'as yet unclassified') ----- + breakpoint + + ^ 'This method contains a breakpoint.'! Item was added: + ----- Method: ToolIconHelp class>>flag (in category 'as yet unclassified') ----- + flag + + ^ 'This method needs attention of some kind.' ! Item was added: + ----- Method: ToolIconHelp class>>helpTexts (in category 'as yet unclassified') ----- + helpTexts + + ^ HelpTexts ifNil: [HelpTexts := IdentityDictionary new]! Item was added: + ----- Method: ToolIconHelp class>>iconHelpNamed: (in category 'as yet unclassified') ----- + iconHelpNamed: aSymbol + + (self respondsTo: aSymbol) + ifTrue: [^self helpTexts at: aSymbol ifAbsentPut: [self perform: aSymbol]] + ifFalse: [^ ''] + ! Item was added: + ----- Method: ToolIconHelp class>>no (in category 'as yet unclassified') ----- + no + + ^ 'This method should not be implemented.' ! Item was added: + ----- Method: ToolIconHelp class>>notOverridden (in category 'as yet unclassified') ----- + notOverridden + + ^ 'This method is abstract and has not been overridden.' ! Item was added: + ----- Method: ToolIconHelp class>>primitive (in category 'as yet unclassified') ----- + primitive + + ^ 'This method implements a primitive.'! From lewis at mail.msen.com Wed Dec 9 05:22:07 2015 From: lewis at mail.msen.com (David T. Lewis) Date: Wed Dec 9 05:22:10 2015 Subject: [squeak-dev] The Inbox: Tools-pre.659.mcz In-Reply-To: References: <56671e94.8b4f370a.3ca22.14b7SMTPIN_ADDED_MISSING@mx.google.com> <20151209000332.GA990@shell.msen.com> Message-ID: <20151209052207.GA55634@shell.msen.com> On Wed, Dec 09, 2015 at 06:07:20AM +0100, karl ramberg wrote: > > On Wed, Dec 9, 2015 at 1:03 AM, David T. Lewis wrote: > > > On Tue, Dec 08, 2015 at 10:47:13PM +0100, karl ramberg wrote: > > > > > > On Tue, Dec 8, 2015 at 7:16 PM, wrote: > > > > > > > A new version of Tools was added to project The Inbox: > > > > http://source.squeak.org/inbox/Tools-pre.659.mcz > > > > > > > > ==================== Summary ==================== > > > > > > > > Name: Tools-pre.659 > > > > Author: pre > > > > Time: 8 December 2015, 7:16:41.034 pm > > > > UUID: a7cb3cba-074b-46a6-8a83-fada58007b25 > > > > Ancestors: Tools-eem.658 > > > > > > > > This adds balloon help texts to the message list of the standard > > browser > > > > to explain the symbols shown in front of message names. > > > > > > > > > > Nice. > > > > > > Best, > > > Karl > > > > This sounds really good, but are there more updates on the way? I loaded > > Tools-pre.659, but I do not see balloon help when I hold the mouse cursor > > over the icons. > > > > Thanks, > > Dave > > > > > > > > Turn on Preferences balloonHelpInMessageLists Ah, I see it now. Very nice! Patrick, I hope you do not mind if I push this to trunk, it seems like and obviously good change. Thanks! Dave From maxleske at gmail.com Wed Dec 9 07:49:05 2015 From: maxleske at gmail.com (Max Leske) Date: Wed Dec 9 07:49:11 2015 Subject: [squeak-dev] [Pharo-users] evolutions of squeakelib & crypto (Reed Solomon) In-Reply-To: References: <565DCE53.90501@gmail.com> <565DD073.1060409@gmail.com> <565FF8ED.1030606@free.fr> <5660BCFA.4050107@gmail.com> <56616E9C.6040201@gmail.com> <56616F27.4070002@gmail.com> <566174E9.9020001@gmail.com> <566175EA.2090107@gmail.com> <60B913A3-565C-4924-9F82-4E0E96553B72@gmail.com> <5661A6A5.9030902@gmail.com> <224A3FD6-A8DA-45E3-8EDE-AA52133940CC@stfx.eu> <5661B7F1.1080701@gmail.com> Message-ID: > On 04 Dec 2015, at 17:41, Mariano Martinez Peck wrote: > > I think that either Max or Martin did the Spur needed changes for Fuel. Yes, Martin made those. > So likely it should be easy to make it work in latest Squeak. I agree. The changes for Spur were minimal. I simply haven?t had time yet for Squeak 5. > > On Fri, Dec 4, 2015 at 12:57 PM, Robert Withers > wrote: > Now I think you are right on with your observation. Additionally, the number of dialects could increase further with Fuel serialization, just port SecureSession and bits. > > Alright, I came up with a name and it may border on the egregious ... presenting ... > > "Maelstrom" > > A new distributed metric will be established in Maelstrom to measure intra- & inter-vat activity: "Sanguinity". > > Robert > > PS. thanks for pointing out Fuel support for older squeaks. Well, we support 4.6, which was released at the same time as 5. So we?re nearly up to speed ;) Cheers, Max > > > On 12/04/2015 10:25 AM, Sven Van Caekenberghe wrote: > Another suggestion: if you intend to make something cross dialect and hope to get traction, I would not use a project name which contains one dialect, but something more general like Cryptography, or some cool project name, like Seaside, Fuel, ... > > On 04 Dec 2015, at 15:43, Robert Withers > wrote: > > I use squeak 5.0 and would want the Fuel support to customize for wire serializations and substitutions, such that Squeak and Pharo could talk to each other, and all other Fuel environments. The Fuel changes I made are in the Pharo port of SqueakElib in the SqueakElib-CapTP-Serialization category and consist of a Decoder, Materializer and a Materialization. > > Robert > > On 12/04/2015 09:35 AM, Esteban Lorenzano wrote: > which squeak version? > of what? > > On 04 Dec 2015, at 12:15, Robert Withers > wrote: > > I just realized that the squeak version uses ReferenceStream while the Pharo version uses Fuel, so the binary serializations are different and they won't speak to each other. Any chance that Fuel is ported to Squeak? > > Regards, > Robert > > On 12/04/2015 06:11 AM, Robert Withers wrote: > I am unable to import these files into SqueakSource, so it may be best done from inside Pharo with Monticello. Here are working Crypto and SqueakElib in Pharo, prior to SecureSession refactoring and Reed Solomon. I include the correct version of LayeredProtocol. > > Regards, > Robert > > > On 12/04/2015 05:47 AM, Robert Withers wrote: > Best Regards > > http://www.squeaksource.com/Cryptography.html > http://www.squeaksource.com/squeakelib.html > > > On 12/04/2015 05:44 AM, Robert Withers wrote: > After my password reset on squeaksource, I committed to both Cryptography and SqueakElib, project links below. > > In the case of Cryptography, I had a version ported to Pharo ... I will organize Pharo ports of both after Reed Solomon is stable, and announce them to the Pharo list. This way both environments can be supported through this one repository. > > Robert > > On 12/04/2015 04:35 AM, Stephan Eggermont wrote: > On 03-12-15 23:06, Robert Withers wrote: > Are any of these used by both squeak and Pharo? That would be the right > move I think. I will ask about getting my password reset for > squeaksource, since that is where the old code resides. > All of them. Mostly timing of project start/high activity and who are > maintaining it decided on platforms. There was a time when squeaksource was not so stable and then many projects migrated, and I currently hear least about stability issues from ss3, but the load on smalltalkhub is much higher, I assume, as that is used for the pharo ci. > > In the not so far future Pharo is likely to move to a git based infrastructure, using libgit2. Early adopters are already using it. > I haven't heard the squeak ideas about that. > > Stephan > > > > > > > > > > -- > Mariano > http://marianopeck.wordpress.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151209/85e1a6d6/attachment-0001.htm From maxleske at gmail.com Wed Dec 9 07:50:54 2015 From: maxleske at gmail.com (Max Leske) Date: Wed Dec 9 07:50:58 2015 Subject: [squeak-dev] [Pharo-users] evolutions of squeakelib & crypto (Reed Solomon) In-Reply-To: <1418E355-0D5E-40F6-A955-B57F6D5D42D7@gmx.de> References: <565DCE53.90501@gmail.com> <565DD073.1060409@gmail.com> <565FF8ED.1030606@free.fr> <5660BCFA.4050107@gmail.com> <56616E9C.6040201@gmail.com> <56616F27.4070002@gmail.com> <566174E9.9020001@gmail.com> <566175EA.2090107@gmail.com> <566696D1.4050605@gmail.com> <1418E355-0D5E-40F6-A955-B57F6D5D42D7@gmx.de> Message-ID: > On 08 Dec 2015, at 21:39, Tobias Pape wrote: > > Hi, > > On 08.12.2015, at 21:03, Chris Muller wrote: > >> Ask Mariano. > > That's uncommonly curtly? > > Max Leske actually makes a huge effort that Fuel always runs in Squeak[1]. > I want to thank him for that here. Thanks Tobias! It?s good to know that Fuel is appreciated in the Squeak community. I?ll try to get support for Squeak 5 out soon. Cheers, Max > > So. > Robert, you should do: > > (Installer monticello http: 'http://smalltalkhub.com/mc/Pharo/Fuel/') > project: 'main'; > package: 'ConfigurationOfFuel'; > install. > > ((Smalltalk at: #ConfigurationOfFuel) project version: #stable) load. > > Fuel is loaded then. There are some development leftovers but they shouldn't affect > normal operation. If you find issues, please report them here and to Max (CC). > > Best regards > -Tobias > > > [1]: https://github.com/theseion/Fuel/blob/master/scripts/jenkins.sh#L43 > >> >> On Tue, Dec 8, 2015 at 2:37 AM, Robert Withers >> wrote: >>> Hey Chris, >>> >>> I just tried this in my squeak image and it complains about not finding >>> 'ConfigurationOfFuel' in a Monticello repository. Are there other pre-steps >>> I need to take? >>> >>> Robert >>> >>> On 12/06/2015 02:03 PM, Chris Muller wrote: >>> >>> On Fri, Dec 4, 2015 at 5:15 AM, Robert Withers >>> wrote: >>>> >>>> I just realized that the squeak version uses ReferenceStream while the >>>> Pharo version uses Fuel, so the binary serializations are different and they >>>> won't speak to each other. Any chance that Fuel is ported to Squeak? >>>> >>> >>> Installer new merge: #fuel >>> >>> >>> >>> >> > From stephan at stack.nl Wed Dec 9 08:44:03 2015 From: stephan at stack.nl (Stephan Eggermont) Date: Wed Dec 9 08:44:17 2015 Subject: [squeak-dev] Re: [Cuis] Sorting Unicode strings (Re: [Unicode] collation sequences (Re: Unicode Support)) In-Reply-To: <56674D32.2000305@gemtalksystems.com> References: <5665EF88.5030901@gemtalksystems.com> <56672D90.4020202@gemtalksystems.com> <56674D32.2000305@gemtalksystems.com> Message-ID: On 08-12-15 22:35, Dale Henrichs wrote: > What I meant is that you can't _always_ use the code point for > collation, i.e., sorting based on the value of code points is not always > correct[1]. I have given up on universal sorting when I learned that dutch libraries sorting of author names depends on the country of origin of the author. So if Jan van Beek is dutch he will be sorted under B, while if he's belgian under V. I haven't checked what happens if the author emigrates, or changes nationality... Stephan From lewis at mail.msen.com Wed Dec 9 13:14:18 2015 From: lewis at mail.msen.com (David T. Lewis) Date: Wed Dec 9 13:14:21 2015 Subject: [squeak-dev] [Pharo-users] evolutions of squeakelib & crypto (Reed Solomon) In-Reply-To: References: <56616E9C.6040201@gmail.com> <56616F27.4070002@gmail.com> <566174E9.9020001@gmail.com> <566175EA.2090107@gmail.com> <566696D1.4050605@gmail.com> <1418E355-0D5E-40F6-A955-B57F6D5D42D7@gmx.de> Message-ID: <20151209131418.GA42149@shell.msen.com> On Wed, Dec 09, 2015 at 08:50:54AM +0100, Max Leske wrote: > > > On 08 Dec 2015, at 21:39, Tobias Pape wrote: > > > > Hi, > > > > On 08.12.2015, at 21:03, Chris Muller wrote: > > > >> Ask Mariano. > > > > That's uncommonly curtly? > > > > Max Leske actually makes a huge effort that Fuel always runs in Squeak[1]. > > I want to thank him for that here. > > Thanks Tobias! It?s good to know that Fuel is appreciated in the Squeak community. I?ll try to get support for Squeak 5 out soon. > > Cheers, > Max > It is very much appreciated. Thank you! Dave From euanmee at gmail.com Wed Dec 9 13:16:18 2015 From: euanmee at gmail.com (EuanM) Date: Wed Dec 9 13:16:21 2015 Subject: [squeak-dev] Re: [Pharo-dev] Unicode Support In-Reply-To: <3163C756-AEA6-4483-A68D-3F06D7E089D4@stfx.eu> References: <5662E1D8.1070802@free.fr> <96B16DCE-A9E7-49E6-B866-E5A846801AA0@stfx.eu> <0F104967-2F55-431F-9EED-0275ABC7CEC1@veloxit.no> <52F49745-64AA-447A-A75A-71FB86FCE36E@veloxit.no> <434B72EE-528B-4ECC-9DCE-78A8F7A6E43E@gmail.com> <3163C756-AEA6-4483-A68D-3F06D7E089D4@stfx.eu> Message-ID: "To encode Unicode for external representation as bytes, we use UTF-8 like the rest of the modern world. So far, so good. Why all the confusion ?" The confusion arises because simply providing *a* valid UTF-8 encoding of does not ensure sortability, nor equivalence testability. It might provide sortable strings. It might not. It might provide a string that can be compared to another string successfully. It might not. So being able to perform valid UTF-8 encoding is *necessary*, but *not sufficient*. i.e. the confusion arises because UTF-8 can provide for several competing, non-sortable encodings of even a single character. This means that *valid* UTF-8 cannot be relied upon to provide these facilities *unless* all the UTF-8 strings can be relied upon to have been encoded to UTF-8 by the same specification of process. i.e. *unless* it has gone through a process of being converted by *a specific* valid method of encoding to UTF-8. Understanding the concept of abstract character is, imo key to understanding the differences between the various valid UTF-8 forms of a given abstract character. Cheers, Euan On 9 December 2015 at 10:45, Sven Van Caekenberghe wrote: > >> On 09 Dec 2015, at 10:35, Guillermo Polito wrote: >> >> >>> On 8 dic 2015, at 10:07 p.m., EuanM wrote: >>> >>> "No. a codepoint is the numerical value assigned to a character. An >>> "encoded character" is the way a codepoint is represented in bytes >>> using a given encoding." >>> >>> No. >>> >>> A codepoint may represent a component part of an abstract character, >>> or may represent an abstract character, or it may do both (but not >>> always at the same time). >>> >>> Codepoints represent a single encoding of a single concept. >>> >>> Sometimes that concept represents a whole abstract character. >>> Sometimes it represent part of an abstract character. >> >> Well. I do not agree with this. I agree with the quote. >> >> Can you explain a bit more about what you mean by abstract character and concept? > > I am pretty sure that this whole discussion does more harm than good for most people's understanding of Unicode. > > It is best and (mostly) correct to think of a Unicode string as a sequence of Unicode characters, each defined/identified by a code point (out of 10.000s covering all languages). That is what we have today in Pharo (with the distinction between ByteString and WideString as mostly invisible implementation details). > > To encode Unicode for external representation as bytes, we use UTF-8 like the rest of the modern world. > > So far, so good. > > Why all the confusion ? Because the world is a complex place and the Unicode standard tries to cover all possible things. Citing all these exceptions and special cases will make people crazy and give up. I am sure that most stopped reading this thread. > > Why then is there confusion about the seemingly simple concept of a character ? Because Unicode allows different ways to say the same thing. The simplest example in a common language is (the French letter ?) is > > LATIN SMALL LETTER E WITH ACUTE [U+00E9] > > which can also be written as > > LATIN SMALL LETTER E [U+0065] followed by COMBINING ACUTE ACCENT [U+0301] > > The former being a composed normal form, the latter a decomposed normal form. (And yes, it is even much more complicated than that, it goes on for 1000s of pages). > > In the above example, the concept of character/string is indeed fuzzy. > > HTH, > > Sven > > From euanmee at gmail.com Wed Dec 9 13:17:01 2015 From: euanmee at gmail.com (EuanM) Date: Wed Dec 9 13:17:04 2015 Subject: [squeak-dev] Fwd: [Pharo-dev] Unicode Support In-Reply-To: References: <5662E1D8.1070802@free.fr> <96B16DCE-A9E7-49E6-B866-E5A846801AA0@stfx.eu> <0F104967-2F55-431F-9EED-0275ABC7CEC1@veloxit.no> <52F49745-64AA-447A-A75A-71FB86FCE36E@veloxit.no> <434B72EE-528B-4ECC-9DCE-78A8F7A6E43E@gmail.com> Message-ID: "Well. I do not agree with this. I agree with the quote. Can you explain a bit more about what you mean by abstract character and concept?" --Guillermo The problem with the quote is, that *while true*, it *does not disambiguate* between: either compatibility character and abstract character; or character as composable component of an abstract character and character as the entire embodiment of an abstract character; Abstract character is the key concept of Unicode. Differentiation between abstract character and codepoints is the key differentiator of the Unicode approach and most previous approaches to character encoding, e,g, ASCII, EBCDIC, ISO Latin 1, etc Please see my previous posts which use the example of Angstrom, Capital A with circle (or whatever the canonical name is) and the composed sequence of "Capital A" and "circle above a letter" for a fuller explanation of the concept of "abstract character". On 9 December 2015 at 09:35, Guillermo Polito wrote: > >> On 8 dic 2015, at 10:07 p.m., EuanM wrote: >> >> "No. a codepoint is the numerical value assigned to a character. An >> "encoded character" is the way a codepoint is represented in bytes >> using a given encoding." >> >> No. >> >> A codepoint may represent a component part of an abstract character, >> or may represent an abstract character, or it may do both (but not >> always at the same time). >> >> Codepoints represent a single encoding of a single concept. >> >> Sometimes that concept represents a whole abstract character. >> Sometimes it represent part of an abstract character. > > Well. I do not agree with this. I agree with the quote. > > Can you explain a bit more about what you mean by abstract character and concept? > >> >> This is the key difference between Unicode and most character encodings. >> >> A codepoint does not always represent a whole character. >> >> On 7 December 2015 at 13:06, Henrik Johansen >> wrote: >>> >>> On 07 Dec 2015, at 1:05 , EuanM wrote: >>> >>> Hi Henry, >>> >>> To be honest, at some point I'm going to long for the for the much >>> more succinct semantics of healthcare systems and sports scoring and >>> administration systems again. :-) >>> >>> codepoints are any of *either* >>> - the representation of a component of an abstract character, *or* >>> eg. "A" #(0041) as a component of >>> - the sole representation of the whole of an abstract character *or* of >>> - a representation of an abstract character provided for backwards >>> compatibility which is more properly represented by a series of >>> codepoints representing a composed character >>> >>> e.g. >>> >>> The "A" #(0041) as a codepoint can be: >>> the sole representation of the whole of an abstract character "A" #(0041) >>> >>> The representation of a component of the composed (i.e. preferred) >>> version of the abstract character ? #(0041 030a) >>> >>> ? (#00C5) represents one valid compatibility form of the abstract >>> character ? which is most properly represented by #(0041 030a). >>> >>> ? (#212b) also represents one valid compatibility form of the abstract >>> character ? which is most properly represented by #(0041 030a). >>> >>> With any luck, this satisfies both our semantic understandings of the >>> concept of "codepoint" >>> >>> Would you agree with that? >>> >>> In Unicode, codepoints are *NOT* an abstract numerical representation >>> of a text character. >>> >>> At least not as we generally understand the term "text character" from >>> our experience of non-Unicode character mappings. >>> >>> >>> I agree, they are numerical representations of what Unicode refers to as >>> characters. >>> >>> >>> codepoints represent "*encoded characters*" >>> >>> >>> No. a codepoint is the numerical value assigned to a character. An "encoded >>> character" is the way a codepoint is represented in bytes using a given >>> encoding. >>> >>> and "a *text element* ... >>> is represented by a sequence of one or more codepoints". (And the >>> term "text element" is deliberately left undefined in the Unicode >>> standard) >>> >>> Individual codepoints are very often *not* the encoded form of an >>> abstract character that we are interested in. Unless we are >>> communicating to or from another system (Which in some cases is the >>> Smalltalk ByteString class) >>> >>> >>> >>> >>> i.e. in other words >>> >>> *Some* individual codepoints *may* be a representation of a specific >>> *abstract character*, but only in special cases. >>> >>> The general case in Unicode is that Unicode defines (a) >>> representation(s) of a Unicode *abstract character*. >>> >>> The Unicode standard representation of an abstract character is a >>> composed sequence of codepoints, where in some cases that sequence is >>> as short as 1 codepoint. >>> >>> In other cases, Unicode has a compatibility alias of a single >>> codepoint which is *also* a representation of an abstract character >>> >>> There are some cases where an abstract character can be represented by >>> more than one single-codepoint compatibility codepoint. >>> >>> Cheers, >>> Euan >>> >>> >>> I agree you have a good grasp of the distinction between an abstract >>> character (characters and character sequences which should be treated >>> equivalent wrt, equality / sorting / display, etc.) and a character (which >>> each have a code point assigned). >>> That is besides the point both Sven and I tried to get through, which is the >>> difference between a code point and the encoded form(s) of said code point. >>> When you write: >>> "and therefore encodable in UTF-8 as compatibility codepoint e9 hex >>> and as the composed character #(0065 00b4) (all in hex) and as the >>> same composed character as both >>> #(feff 0065 00b4) and #(ffef 0065 00b4) when endianness markers are >>> included" >>> >>> I's quite clear you confuse the two. 0xFEFF is the codepoint of the >>> character used as bom. >>> When you state that it can be written ffef (I assume you meant FFFE), you >>> are again confusing the code point and its encoded value (an encoded value >>> which only occurs in UTF16/32, no less). >>> >>> When this distinction is clear, it might be easier to see that value in that >>> Strings are kept as Unicode code points arrays, and converted to encoded >>> forms when entering/exiting the system. >>> >>> Cheers, >>> Henry >>> >> > > From hannes.hirzel at gmail.com Wed Dec 9 14:11:35 2015 From: hannes.hirzel at gmail.com (H. Hirzel) Date: Wed Dec 9 14:11:37 2015 Subject: [squeak-dev] Re: [Cuis] Sorting Unicode strings (Re: [Unicode] collation sequences (Re: Unicode Support)) In-Reply-To: References: <5665EF88.5030901@gemtalksystems.com> <56672D90.4020202@gemtalksystems.com> <56674D32.2000305@gemtalksystems.com> Message-ID: Hi Stephan What you mention is an edge case. A regular case which is not implemented yet is language insensitive sorting http://www.unicode.org/versions/Unicode8.0.0/UnicodeStandard-8.0.pdf In some circumstances, an application may need to do language-insensitive sorting?that is, sorting of textual data without consideration of language-specific cultural expectations about how strings should be ordered. This is currently not the case as there is no normalization. As the Unicode Character Database [1] is already available in Squeak / Pharo (and could easily be loaded into Cuis) the implementation effort for language-insensitive sorting seems to be in reach without a big effort. [1] http://wiki.squeak.org/squeak/6244 Hannes On 12/9/15, Stephan Eggermont wrote: > On 08-12-15 22:35, Dale Henrichs wrote: >> What I meant is that you can't _always_ use the code point for >> collation, i.e., sorting based on the value of code points is not always >> correct[1]. > > I have given up on universal sorting when I learned that dutch libraries > sorting of author names depends on the country of origin of the author. > So if Jan van Beek is dutch he will be sorted under B, while if he's > belgian under V. I haven't checked what happens if the author emigrates, > or changes nationality... > > Stephan > > > > From hannes.hirzel at gmail.com Wed Dec 9 15:03:54 2015 From: hannes.hirzel at gmail.com (H. Hirzel) Date: Wed Dec 9 15:04:09 2015 Subject: [Pharo-dev] [Cuis] Sorting Unicode strings (Re: [Unicode] collation sequences (Re: [squeak-dev] Unicode Support)) In-Reply-To: References: <5665EF88.5030901@gemtalksystems.com> <56672D90.4020202@gemtalksystems.com> Message-ID: On 12/8/15, Todd Blanchard wrote: > I just want to second Dale's endorsement of the ICU library. It has been > around a long time (originally developed by Taligent) and it provides the > base unicode capabilities for an awful lot of software. > > I think it would make more sense to bring icu into Smalltalk as a > NativeBoost library than to spend resources reimplementing and maintaining > it. > > -Todd Blanchard ICU already seems to be available for Smalltalk accessed through wrappers http://wiki.squeak.org/squeak/6234 --Hannes >> On Dec 8, 2015, at 11:20, Dale Henrichs >> wrote: >> >> On 12/07/2015 11:31 PM, H. Hirzel wrote: >>> Dale >>> >>> Thank you for your answer with links to the ICU library and the notes >>> about classes in Gemstone. Noteworthy that you have a class Utf8 as a >>> subclass of ByteArray. >>> >>> I understand that Gemstone uses the ICU library and thus does not >>> implement the algorithms in Smalltalk. >>> >>> I am currently looking into what the ICU library provides. > > From hannes.hirzel at gmail.com Wed Dec 9 17:23:35 2015 From: hannes.hirzel at gmail.com (H. Hirzel) Date: Wed Dec 9 17:23:37 2015 Subject: [Pharo-dev] [Cuis] Sorting Unicode strings (Re: [Unicode] collation sequences (Re: [squeak-dev] Unicode Support)) In-Reply-To: <530E19B5-317C-4405-A9DB-DF4423A858F3@mac.com> References: <5665EF88.5030901@gemtalksystems.com> <56672D90.4020202@gemtalksystems.com> <530E19B5-317C-4405-A9DB-DF4423A858F3@mac.com> Message-ID: It seems that people at Twitter just did this for Ruby -- reimplementation from scratch with an API oriented towards Ruby users. A summary http://wiki.squeak.org/squeak/6263 Of course it depends on what you expect to achieve. This thread starts with aiming at getting German, French and Spanish sorting done. Plus some other similar cases. The algorithms are table driven and the tables are read into the Smalltalk image as is. As of now the Unicode Character Database is in the Squeak/Pharo image http://wiki.squeak.org/squeak/6244 Getting the sorting done does not seem to be extraordinary hard. However making use of the Smalltalk ICU wrapper is surely an option. --Hannes On 12/9/15, Todd Blanchard wrote: > They are practically the same thing. > > ICU was developed by Taligent which was a joint venture between Apple and > IBM. Makes sense that NSString and ICU's UnicodeString are pretty close in > implementation. ICU was also ported to Java for Sun by IBM. The point is - > this is a very elaborate chunk of code with far reach. If ICU is wrong on > some point - it is universally wrong and thus likely to be taken as "right" > as it is at least consistent. I think re-implementing it is folly TBH. > Just use it. > >> On Dec 8, 2015, at 15:52, EuanM wrote: >> >> Equally old are the NextStep Object C functions which are now embodied >> within MacOS X. >> > > > From smalltalk at stefan-marr.de Wed Dec 9 19:33:21 2015 From: smalltalk at stefan-marr.de (Stefan Marr) Date: Wed Dec 9 19:33:25 2015 Subject: [squeak-dev] Call for Papers: Reflect'16, Workshop on Reflection and Runtime Meta-Programming Techniques Message-ID: <0941D2B5-B13E-4F97-A6CF-459AFD10239A@stefan-marr.de> Call for Papers: Reflect?16 =========================== Workshop on Reflection and Runtime Meta-Programming Techniques Co-located with Modularity 2016 March 14 or 15, 2016, M?laga, Spain Twitter @ReflectWorkshop http://2016.modularity.info/track/Reflect-2016-papers With modern mainstream languages embracing runtime reflection, for instance in JavaScript with proxies and Ruby with its culture of using meta-programming, the research on meta-architectures, reflective programming, and other meta-programming techniques have become relevant and timely once again. Over the last couple of years, these techniques saw a surge of interest benefiting from the JavaScript standardization process as well as performance improvements based on just-in-time compilation that increased their general acceptance. The Reflect?16 workshop aims to bring together people who do research on reflection and runtime meta-programming, as well as users of such techniques to e.g. build applications, language extensions, or software tools. We invite contributions to the workshop on a wide range of topics related to design, implementation, and application of reflective APIs and runtime meta-programming techniques, as well as empirical studies and typing for such systems and languages. We welcome technical papers as well as work-in-progress and position papers from the academic as well as industrial perspective. Position paper should take a perhaps controversial stance on a specific topic and argue the position well. Topics of Interest ------------------ The topics of interest for the workshop include, but are not limited to: - applications to middleware, frameworks, and DSLs - reflection and metaobject protocols to enable tooling - meta-level architectures and reflective middleware for modern runtime platforms (e.g. IoT, cyber-physical systems, cloud/grid computing, exa-scale systems, smart grids, mobile systems) - optimization techniques to minimize runtime overhead of reflection - use for application-level runtime optimization - new language constructs for reflection and meta-programming - security in reflective systems and capability-based designs - application of reflective techniques to achieve adaptability, separation of concerns, code reuse, etc. - empirical studies to the dynamic behavior of reflective programs - application to enable complex concurrent systems - typing of reflective programs Workshop Format and Submissions ------------------------------- This workshop welcomes the presentation of mature work as well as discussion of new ideas and emerging problems as part of a mini-conference format. Furthermore, we plan for more interactive brainstorming and demonstration sessions between the formal presentations to enable an active exchange of ideas. The workshop papers will be published in both the electronic proceedings of the Modularity conference and in the ACM Digital Library, if not requested otherwise by the authors. Papers are to be submitted using the ACM sigplanconf style at 9pt font size. See http://www.acm.org/publications/article-templates/proceedings-template.html. position and work-in-progress paper: max. 4 pages technical paper: max. 8 pages demos and posters: 1-page abstract For the submission, please use the EasyChair system: https://easychair.org/conferences/?conf=reflect16 Important Dates --------------- abstract submission: January 11, 2016 paper submission: January 15, 2016 notification: February 6, 2016 camera-ready: February 13, 2016 all deadlines: Anywhere on Earth (AoE), i.e., GMT/UTC?12:00 hour Workshop Organizers ------------------- Gilad Bracha, Google Shigeru Chiba, University of Tokyo Elisa Gonzalez Boix, Vrije Universiteit Brussel Stefan Marr, Johannes Kepler University Linz Program Committee ----------------- Daniele Bonetta, Oracle Labs, Austria Damien Cassou, University of Lille 1, France Siobhan Clarke, Trinity College Dublin, Ireland Stephane Ducasse, Inria, France Robert Hirschfeld, HPI, Germany Hridesh Rajan, Iowa State University, USA Romain Rouvoy, University Lille 1 and INRIA, France Eric Tanter, University of Chile, Chile Laurie Tratt, King?s College, UK Tom Van Cutsem, Bell Labs, Belgium Takuo Wantanabe, Tokyo Institute of Technology, Japan Tijs van der Storm, CWI, NL -- Stefan Marr Johannes Kepler Universit?t Linz http://stefan-marr.de/research/ From dale.henrichs at gemtalksystems.com Wed Dec 9 21:53:49 2015 From: dale.henrichs at gemtalksystems.com (Dale Henrichs) Date: Wed Dec 9 21:53:54 2015 Subject: [Cuis] Sorting Unicode strings (Re: [Unicode] collation sequences (Re: [squeak-dev] Unicode Support)) In-Reply-To: References: <5665EF88.5030901@gemtalksystems.com> <56672D90.4020202@gemtalksystems.com> <56674D32.2000305@gemtalksystems.com> <56677927.3060109@gemtalksystems.com> Message-ID: <5668A2ED.4070208@gemtalksystems.com> On 12/08/2015 04:59 PM, EuanM wrote: > http://www.unicode.org/reports/tr15/#Stable_Code_Points > Table 7, the discussion of Ligatures, (which uses the ligature of > "ffi" as its example) > > Every time I think I'm about to grokk this standard, something like > this crops up. > Table 7 and Table 8 are showing different normalization forms and I think Table 8 is an example where the two strings are equivalent (the are two categories of normalization: canonical equivalent [table 7] and compatibility equivalent [table 8]) .... it seems (I haven't delved this far myself) that the normalization form is chosen based on the actual set of characters and possibly some application specific choices .... So I don't think there is a fixed one-size fits all interpretation ... which makes duplicating the functionality ICU challenging.... Dale From dale.henrichs at gemtalksystems.com Wed Dec 9 22:00:07 2015 From: dale.henrichs at gemtalksystems.com (Dale Henrichs) Date: Wed Dec 9 22:00:11 2015 Subject: [squeak-dev] Re: [Cuis] Sorting Unicode strings (Re: [Unicode] collation sequences (Re: Unicode Support)) In-Reply-To: References: <5665EF88.5030901@gemtalksystems.com> <56672D90.4020202@gemtalksystems.com> <56674D32.2000305@gemtalksystems.com> Message-ID: <5668A467.7060302@gemtalksystems.com> On 12/09/2015 12:44 AM, Stephan Eggermont wrote: > On 08-12-15 22:35, Dale Henrichs wrote: >> What I meant is that you can't _always_ use the code point for >> collation, i.e., sorting based on the value of code points is not always >> correct[1]. > > I have given up on universal sorting when I learned that dutch > libraries sorting of author names depends on the country of origin of > the author. So if Jan van Beek is dutch he will be sorted under B, > while if he's belgian under V. I haven't checked what happens if the > author emigrates, or changes nationality... > > Stephan > Well, with ICU (and GemStone's implementation) you can choose which collator to use (Country specific) at the image level or on a comparison by comparison bases ... for example for an indexed collection (Unicode) Strings, you can choose the collator to use for that particular index ... so while it's true that universal sorter is not possible, it is possible to choose a collator that will satisfy a particlar customer .... Dale From commits at source.squeak.org Wed Dec 9 22:55:02 2015 From: commits at source.squeak.org (commits@source.squeak.org) Date: Wed Dec 9 22:55:03 2015 Subject: [squeak-dev] Daily Commit Log Message-ID: <20151209225502.6519.qmail@box4.squeakfoundation.org> Changes to Trunk (http://source.squeak.org/trunk.html) in the last 24 hours: http://lists.squeakfoundation.org/pipermail/packages/2015-December/009212.html Name: Morphic-tpr.1058 Ancestors: Morphic-cwp.1057, Morphic-tpr.1057 Transfer from inbox, keep up to date. ============================================= http://lists.squeakfoundation.org/pipermail/packages/2015-December/009213.html Name: Tools-pre.659 Ancestors: Tools-eem.658 This adds balloon help texts to the message list of the standard browser to explain the symbols shown in front of message names. ============================================= From leves at caesar.elte.hu Thu Dec 10 00:31:46 2015 From: leves at caesar.elte.hu (Levente Uzonyi) Date: Thu Dec 10 00:31:50 2015 Subject: [squeak-dev] Re: [Cuis] Sorting Unicode strings (Re: [Unicode] collation sequences (Re: Unicode Support)) In-Reply-To: <5668A467.7060302@gemtalksystems.com> References: <5665EF88.5030901@gemtalksystems.com> <56672D90.4020202@gemtalksystems.com> <56674D32.2000305@gemtalksystems.com> <5668A467.7060302@gemtalksystems.com> Message-ID: On Wed, 9 Dec 2015, Dale Henrichs wrote: > > > On 12/09/2015 12:44 AM, Stephan Eggermont wrote: >> On 08-12-15 22:35, Dale Henrichs wrote: >>> What I meant is that you can't _always_ use the code point for >>> collation, i.e., sorting based on the value of code points is not always >>> correct[1]. >> >> I have given up on universal sorting when I learned that dutch libraries >> sorting of author names depends on the country of origin of the author. So >> if Jan van Beek is dutch he will be sorted under B, while if he's belgian >> under V. I haven't checked what happens if the author emigrates, or changes >> nationality... >> >> Stephan >> > Well, with ICU (and GemStone's implementation) you can choose which collator > to use (Country specific) at the image level or on a comparison by comparison > bases ... for example for an indexed collection (Unicode) Strings, you can > choose the collator to use for that particular index ... so while it's true > that universal sorter is not possible, it is possible to choose a collator > that will satisfy a particlar customer .... I expect my image to compare strings using the codepoint-based (+language tags) lexicographical method, because it's simple, deterministic and fast. Imagine having failing tests just because your image uses different default comparison methods based on some (external) parameter... It's also a nightmare to find out why your program is slow on some machine, while it's fast on another. Levente > > Dale > > > From tim at rowledge.org Thu Dec 10 00:45:39 2015 From: tim at rowledge.org (tim Rowledge) Date: Thu Dec 10 00:45:44 2015 Subject: [squeak-dev] Explorer bug when inspecting a ColorForm Message-ID: <8D6DE919-ED85-4375-B756-00FD018D8A3A@rowledge.org> (In a 15566 update image) If you try to ?explore? a morph that returns a ColorForm for #iconOrThumbnail we have a oroblem because Form>scaledIntoFormOfSize:smoothing: makes the assumption that 32bpp is appropriate - which isn?t true for ColorForms, as you can see if you look at ColorForm>setExtent:depth: I guess it?s the use of ?self species? that is the real problem. Latest version is from this august, by marcel, so maybe you have a simple solution? Im in the midst of more complication than I can describe, so I?m not about to fix this right now... tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Useful Latin Phrases:- Illiud Latine dici non potest = You can't say that in Latin. From tim at rowledge.org Thu Dec 10 00:45:39 2015 From: tim at rowledge.org (tim Rowledge) Date: Thu Dec 10 02:02:14 2015 Subject: [squeak-dev] Explorer bug when inspecting a ColorForm Message-ID: <8D6DE919-ED85-4375-B756-00FD018D8A3A@rowledge.org> (In a 15566 update image) If you try to ?explore? a morph that returns a ColorForm for #iconOrThumbnail we have a oroblem because Form>scaledIntoFormOfSize:smoothing: makes the assumption that 32bpp is appropriate - which isn?t true for ColorForms, as you can see if you look at ColorForm>setExtent:depth: I guess it?s the use of ?self species? that is the real problem. Latest version is from this august, by marcel, so maybe you have a simple solution? Im in the midst of more complication than I can describe, so I?m not about to fix this right now... tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Useful Latin Phrases:- Illiud Latine dici non potest = You can't say that in Latin. From eliot.miranda at gmail.com Thu Dec 10 03:06:11 2015 From: eliot.miranda at gmail.com (Eliot Miranda) Date: Thu Dec 10 03:06:12 2015 Subject: Broken Update [was [squeak-dev] Daily Commit Log] Message-ID: Hi, whatever happened in unloading Universes, it broke the ability to update a release Squeak 5.0 image to the current tip. Hence my automated scripts to build 64-bit images are broken. Can whoever broke the update either fix it, or provide some reasonable workaround so that I can restore my scripts? What's at stake here is, well, a high performance 64-bit Squeak and/or Pharo. I think it's reasonable, even mandatory that we should be able to update a release N image to the tip of release N. We should have CI servers that try and do this (I think we do, don't we?), and so IMO it's incumbent on whoever breaks the update to fix things, e.g. by changing update maps appropriately. [and I don't mean this as a punishment, or something that the person has to do on their own; I'm happy to help fix things, it's /really/ important that the update works, and not just from the last update]. Thanks in advance. Apologies if I appear rude; it's not my intent. I just want my auto-update back ;-) On Thu, Nov 26, 2015 at 2:55 PM, wrote: > Changes to Trunk (http://source.squeak.org/trunk.html) in the last 24 > hours: > > > http://lists.squeakfoundation.org/pipermail/packages/2015-November/009199.html > > Name: Squeak-Version-kfr.4713 > Ancestors: Squeak-Version-kfr.4712 > > Unload and unregister Universes > > ============================================= > _,,,^..^,,,_ best, Eliot -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151209/f1bfefb7/attachment.htm From tim at rowledge.org Thu Dec 10 04:40:17 2015 From: tim at rowledge.org (tim Rowledge) Date: Thu Dec 10 04:40:22 2015 Subject: Broken Update [was [squeak-dev] Daily Commit Log] In-Reply-To: References: Message-ID: > On 09-12-2015, at 7:06 PM, Eliot Miranda wrote: > > Hi, > > whatever happened in unloading Universes, it broke the ability to update a release Squeak 5.0 image to the current tip. Hence my automated scripts to build 64-bit images are broken. Can whoever broke the update either fix it, or provide some reasonable workaround so that I can restore my scripts? What's at stake here is, well, a high performance 64-bit Squeak and/or Pharo. That?s pretty much what I was asking in an email I (failed to actually send out! dammit!) sent yesterday. We need to solve this asap. Not to mention me solving the ?why have several emails not got outside my internal network? question. tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Strange OpCodes: MC: Melt down Core From karlramberg at gmail.com Thu Dec 10 05:27:18 2015 From: karlramberg at gmail.com (karl ramberg) Date: Thu Dec 10 05:27:21 2015 Subject: Broken Update [was [squeak-dev] Daily Commit Log] In-Reply-To: References: Message-ID: Hi, It it related with this Levente mentioned in a mail earlier ? "Sadly the update process will break if your image has Universes-mt.48 in it, because it'll try to load Universes-kfr.49 after it has been removed by Squeak-Version. I think removing Universes from the update map will help." I asked how to proceed but I got no answer. I'm not sure how one make a update map Best, Karl On Thu, Dec 10, 2015 at 4:06 AM, Eliot Miranda wrote: > Hi, > > whatever happened in unloading Universes, it broke the ability to > update a release Squeak 5.0 image to the current tip. Hence my automated > scripts to build 64-bit images are broken. Can whoever broke the update > either fix it, or provide some reasonable workaround so that I can restore > my scripts? What's at stake here is, well, a high performance 64-bit > Squeak and/or Pharo. > > I think it's reasonable, even mandatory that we should be able to update a > release N image to the tip of release N. We should have CI servers that > try and do this (I think we do, don't we?), and so IMO it's incumbent on > whoever breaks the update to fix things, e.g. by changing update maps > appropriately. [and I don't mean this as a punishment, or something that > the person has to do on their own; I'm happy to help fix things, it's > /really/ important that the update works, and not just from the last > update]. > > Thanks in advance. Apologies if I appear rude; it's not my intent. I > just want my auto-update back ;-) > > On Thu, Nov 26, 2015 at 2:55 PM, wrote: > >> Changes to Trunk (http://source.squeak.org/trunk.html) in the last 24 >> hours: >> >> >> http://lists.squeakfoundation.org/pipermail/packages/2015-November/009199.html >> >> Name: Squeak-Version-kfr.4713 >> Ancestors: Squeak-Version-kfr.4712 >> >> Unload and unregister Universes >> >> ============================================= >> > > _,,,^..^,,,_ > best, Eliot > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151210/ac343c35/attachment.htm From Patrick.Rein at hpi.de Thu Dec 10 07:45:18 2015 From: Patrick.Rein at hpi.de (Rein, Patrick) Date: Thu Dec 10 07:45:21 2015 Subject: AW: [squeak-dev] The Inbox: Tools-pre.659.mcz In-Reply-To: <20151209052207.GA55634@shell.msen.com> References: <56671e94.8b4f370a.3ca22.14b7SMTPIN_ADDED_MISSING@mx.google.com> <20151209000332.GA990@shell.msen.com> , <20151209052207.GA55634@shell.msen.com> Message-ID: <1449733525113.93327@hpi.de> Not at all! This was inspired by a suggestion from Robert by the way :) Bests Patrick ________________________________________ Von: squeak-dev-bounces@lists.squeakfoundation.org im Auftrag von David T. Lewis Gesendet: Mittwoch, 9. Dezember 2015 06:22 An: The general-purpose Squeak developers list Betreff: Re: [squeak-dev] The Inbox: Tools-pre.659.mcz On Wed, Dec 09, 2015 at 06:07:20AM +0100, karl ramberg wrote: > > On Wed, Dec 9, 2015 at 1:03 AM, David T. Lewis wrote: > > > On Tue, Dec 08, 2015 at 10:47:13PM +0100, karl ramberg wrote: > > > > > > On Tue, Dec 8, 2015 at 7:16 PM, wrote: > > > > > > > A new version of Tools was added to project The Inbox: > > > > http://source.squeak.org/inbox/Tools-pre.659.mcz > > > > > > > > ==================== Summary ==================== > > > > > > > > Name: Tools-pre.659 > > > > Author: pre > > > > Time: 8 December 2015, 7:16:41.034 pm > > > > UUID: a7cb3cba-074b-46a6-8a83-fada58007b25 > > > > Ancestors: Tools-eem.658 > > > > > > > > This adds balloon help texts to the message list of the standard > > browser > > > > to explain the symbols shown in front of message names. > > > > > > > > > > Nice. > > > > > > Best, > > > Karl > > > > This sounds really good, but are there more updates on the way? I loaded > > Tools-pre.659, but I do not see balloon help when I hold the mouse cursor > > over the icons. > > > > Thanks, > > Dave > > > > > > > > Turn on Preferences balloonHelpInMessageLists Ah, I see it now. Very nice! Patrick, I hope you do not mind if I push this to trunk, it seems like and obviously good change. Thanks! Dave From Marcel.Taeumel at hpi.de Thu Dec 10 08:18:57 2015 From: Marcel.Taeumel at hpi.de (marcel.taeumel) Date: Thu Dec 10 08:36:20 2015 Subject: [squeak-dev] Re: Explorer bug when inspecting a ColorForm In-Reply-To: <8D6DE919-ED85-4375-B756-00FD018D8A3A@rowledge.org> References: <8D6DE919-ED85-4375-B756-00FD018D8A3A@rowledge.org> Message-ID: <1449735537955-4866473.post@n4.nabble.com> Where can I find a useful instance of ColorForm? =) Best, Marcel -- View this message in context: http://forum.world.st/Explorer-bug-when-inspecting-a-ColorForm-tp4866459p4866473.html Sent from the Squeak - Dev mailing list archive at Nabble.com. From Das.Linux at gmx.de Thu Dec 10 08:37:28 2015 From: Das.Linux at gmx.de (Tobias Pape) Date: Thu Dec 10 08:37:33 2015 Subject: [squeak-dev] Re: Explorer bug when inspecting a ColorForm In-Reply-To: <1449735537955-4866473.post@n4.nabble.com> References: <8D6DE919-ED85-4375-B756-00FD018D8A3A@rowledge.org> <1449735537955-4866473.post@n4.nabble.com> Message-ID: <6E0FA07D-01CA-4EC7-9894-135F8C18A9B8@gmx.de> On 10.12.2015, at 09:18, marcel.taeumel wrote: > Where can I find a useful instance of ColorForm? =) Ask your local drug sto.. ahh ColorForm not ChloroForm, sorry? > > Best, > Marcel > > > > -- > View this message in context: http://forum.world.st/Explorer-bug-when-inspecting-a-ColorForm-tp4866459p4866473.html > Sent from the Squeak - Dev mailing list archive at Nabble.com. > From Marcel.Taeumel at hpi.de Thu Dec 10 08:23:22 2015 From: Marcel.Taeumel at hpi.de (marcel.taeumel) Date: Thu Dec 10 08:40:43 2015 Subject: [squeak-dev] Re: Explorer bug when inspecting a ColorForm In-Reply-To: <6E0FA07D-01CA-4EC7-9894-135F8C18A9B8@gmx.de> References: <8D6DE919-ED85-4375-B756-00FD018D8A3A@rowledge.org> <1449735537955-4866473.post@n4.nabble.com> <6E0FA07D-01CA-4EC7-9894-135F8C18A9B8@gmx.de> Message-ID: <1449735802543-4866475.post@n4.nabble.com> Ah, "ColorForm allInstances explore" reproduces the bug. :) Best, Marcel -- View this message in context: http://forum.world.st/Explorer-bug-when-inspecting-a-ColorForm-tp4866459p4866475.html Sent from the Squeak - Dev mailing list archive at Nabble.com. From commits at source.squeak.org Thu Dec 10 08:50:34 2015 From: commits at source.squeak.org (commits@source.squeak.org) Date: Thu Dec 10 08:50:37 2015 Subject: [squeak-dev] The Trunk: Morphic-mt.1059.mcz Message-ID: Marcel Taeumel uploaded a new version of Morphic to project The Trunk: http://source.squeak.org/trunk/Morphic-mt.1059.mcz ==================== Summary ==================== Name: Morphic-mt.1059 Author: mt Time: 10 December 2015, 9:49:58.039 am UUID: cbaef52e-fd80-4151-9a73-53b4eededc97 Ancestors: Morphic-tpr.1058 Fixes bug regarding the scaling of 8-bit color forms. =============== Diff against Morphic-tpr.1058 =============== Item was changed: ----- Method: Form>>scaledIntoFormOfSize:smoothing: (in category '*Morphic') ----- scaledIntoFormOfSize: aNumberOrPoint smoothing: factor "Scale and center the receiver into a form of a given size" | extent scaledForm result | extent := aNumberOrPoint asPoint. extent = self extent ifTrue: [^ self copy]. scaledForm := self scaledToSize: extent smoothing: factor. + result := self species extent: extent depth: self depth. - result := self species extent: extent depth: 32. result getCanvas translucentImage: scaledForm at: extent - scaledForm extent // 2. ^ result ! From Marcel.Taeumel at hpi.de Thu Dec 10 08:34:33 2015 From: Marcel.Taeumel at hpi.de (marcel.taeumel) Date: Thu Dec 10 08:51:56 2015 Subject: [squeak-dev] Re: Explorer bug when inspecting a ColorForm In-Reply-To: <1449735802543-4866475.post@n4.nabble.com> References: <8D6DE919-ED85-4375-B756-00FD018D8A3A@rowledge.org> <1449735537955-4866473.post@n4.nabble.com> <6E0FA07D-01CA-4EC7-9894-135F8C18A9B8@gmx.de> <1449735802543-4866475.post@n4.nabble.com> Message-ID: <1449736473537-4866478.post@n4.nabble.com> Fixed: http://forum.world.st/The-Trunk-Morphic-mt-1059-mcz-td4866477.html Best, Marcel -- View this message in context: http://forum.world.st/Explorer-bug-when-inspecting-a-ColorForm-tp4866459p4866478.html Sent from the Squeak - Dev mailing list archive at Nabble.com. From bert at freudenbergs.de Thu Dec 10 14:06:29 2015 From: bert at freudenbergs.de (Bert Freudenberg) Date: Thu Dec 10 14:06:33 2015 Subject: [squeak-dev] The Inbox: Tools-pre.659.mcz In-Reply-To: <1449733525113.93327@hpi.de> References: <56671e94.8b4f370a.3ca22.14b7SMTPIN_ADDED_MISSING@mx.google.com> <20151209000332.GA990@shell.msen.com> <20151209052207.GA55634@shell.msen.com> <1449733525113.93327@hpi.de> Message-ID: On 10.12.2015, at 08:45, Rein, Patrick wrote: > > This was inspired by a suggestion from Robert by the way :) > > Bests > Patrick > > On Wed, Dec 09, 2015 at 06:07:20AM +0100, karl ramberg wrote: >> >> Turn on Preferences balloonHelpInMessageLists Why do we need a preference for this? - Bert - -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4115 bytes Desc: not available Url : http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151210/cc335264/smime.bin From cunningham.cb at gmail.com Thu Dec 10 15:36:43 2015 From: cunningham.cb at gmail.com (Chris Cunningham) Date: Thu Dec 10 15:36:47 2015 Subject: Broken Update [was [squeak-dev] Daily Commit Log] In-Reply-To: References: Message-ID: Hi Karl, I think to update the update map, open up trunk, go down and find the package called 'update', browse that. It should bring up an MCConfigurationBrowser. The order of the packages in that browser is the order that an update will try to upgrade the packages - and is the list of packages that the upgrade stream will upgrade. I think you'll want to remove from that list the packages that were removed in the SqueakVersion, and then save the configuration back to trunk. When you are doing tricky things, you can specify exactly which versions of packages should load in an update - and stack them one after the other - so that the update will load specific packages in a specific order. Probably not needed for this fix, though, right? All of this said, I don't actually have access to do this or test it, so take the info as you will. Good luck, cbc On Wed, Dec 9, 2015 at 9:27 PM, karl ramberg wrote: > Hi, > It it related with this Levente mentioned in a mail earlier ? > > "Sadly the update process will break if your image has Universes-mt.48 in > it, because it'll try to load Universes-kfr.49 after it has been removed by > Squeak-Version. I think removing Universes from the update map will help." > > I asked how to proceed but I got no answer. > I'm not sure how one make a update map > > > Best, > Karl > > On Thu, Dec 10, 2015 at 4:06 AM, Eliot Miranda > wrote: > >> Hi, >> >> whatever happened in unloading Universes, it broke the ability to >> update a release Squeak 5.0 image to the current tip. Hence my automated >> scripts to build 64-bit images are broken. Can whoever broke the update >> either fix it, or provide some reasonable workaround so that I can restore >> my scripts? What's at stake here is, well, a high performance 64-bit >> Squeak and/or Pharo. >> >> I think it's reasonable, even mandatory that we should be able to update >> a release N image to the tip of release N. We should have CI servers that >> try and do this (I think we do, don't we?), and so IMO it's incumbent on >> whoever breaks the update to fix things, e.g. by changing update maps >> appropriately. [and I don't mean this as a punishment, or something that >> the person has to do on their own; I'm happy to help fix things, it's >> /really/ important that the update works, and not just from the last >> update]. >> >> Thanks in advance. Apologies if I appear rude; it's not my intent. I >> just want my auto-update back ;-) >> >> On Thu, Nov 26, 2015 at 2:55 PM, wrote: >> >>> Changes to Trunk (http://source.squeak.org/trunk.html) in the last 24 >>> hours: >>> >>> >>> http://lists.squeakfoundation.org/pipermail/packages/2015-November/009199.html >>> >>> Name: Squeak-Version-kfr.4713 >>> Ancestors: Squeak-Version-kfr.4712 >>> >>> Unload and unregister Universes >>> >>> ============================================= >>> >> >> _,,,^..^,,,_ >> best, Eliot >> >> >> >> > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151210/1cab060d/attachment.htm From eliot.miranda at gmail.com Thu Dec 10 16:00:39 2015 From: eliot.miranda at gmail.com (Eliot Miranda) Date: Thu Dec 10 16:00:43 2015 Subject: Broken Update [was [squeak-dev] Daily Commit Log] In-Reply-To: References: Message-ID: Skipped content of type multipart/alternative-------------- next part -------------- A non-text attachment was scrubbed... Name: SpurBootstrapMonticelloPackagePatcher.st Type: application/octet-stream Size: 42258 bytes Desc: not available Url : http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151210/72eb66cc/SpurBootstrapMonticelloPackagePatcher.obj From karlramberg at gmail.com Thu Dec 10 16:34:38 2015 From: karlramberg at gmail.com (karl ramberg) Date: Thu Dec 10 16:34:42 2015 Subject: Broken Update [was [squeak-dev] Daily Commit Log] In-Reply-To: References: Message-ID: Thanks Best, Karl On Thu, Dec 10, 2015 at 4:36 PM, Chris Cunningham wrote: > Hi Karl, > I think to update the update map, open up trunk, go down and find the > package called 'update', browse that. It should bring up an > MCConfigurationBrowser. The order of the packages in that browser is the > order that an update will try to upgrade the packages - and is the list of > packages that the upgrade stream will upgrade. > > I think you'll want to remove from that list the packages that were > removed in the SqueakVersion, and then save the configuration back to trunk. > > When you are doing tricky things, you can specify exactly which versions > of packages should load in an update - and stack them one after the other - > so that the update will load specific packages in a specific order. > Probably not needed for this fix, though, right? > > All of this said, I don't actually have access to do this or test it, so > take the info as you will. > > Good luck, > cbc > > On Wed, Dec 9, 2015 at 9:27 PM, karl ramberg > wrote: > >> Hi, >> It it related with this Levente mentioned in a mail earlier ? >> >> "Sadly the update process will break if your image has Universes-mt.48 in >> it, because it'll try to load Universes-kfr.49 after it has been removed by >> Squeak-Version. I think removing Universes from the update map will >> help." >> >> I asked how to proceed but I got no answer. >> I'm not sure how one make a update map >> >> >> Best, >> Karl >> >> On Thu, Dec 10, 2015 at 4:06 AM, Eliot Miranda >> wrote: >> >>> Hi, >>> >>> whatever happened in unloading Universes, it broke the ability to >>> update a release Squeak 5.0 image to the current tip. Hence my automated >>> scripts to build 64-bit images are broken. Can whoever broke the update >>> either fix it, or provide some reasonable workaround so that I can restore >>> my scripts? What's at stake here is, well, a high performance 64-bit >>> Squeak and/or Pharo. >>> >>> I think it's reasonable, even mandatory that we should be able to update >>> a release N image to the tip of release N. We should have CI servers that >>> try and do this (I think we do, don't we?), and so IMO it's incumbent on >>> whoever breaks the update to fix things, e.g. by changing update maps >>> appropriately. [and I don't mean this as a punishment, or something that >>> the person has to do on their own; I'm happy to help fix things, it's >>> /really/ important that the update works, and not just from the last >>> update]. >>> >>> Thanks in advance. Apologies if I appear rude; it's not my intent. I >>> just want my auto-update back ;-) >>> >>> On Thu, Nov 26, 2015 at 2:55 PM, wrote: >>> >>>> Changes to Trunk (http://source.squeak.org/trunk.html) in the last 24 >>>> hours: >>>> >>>> >>>> http://lists.squeakfoundation.org/pipermail/packages/2015-November/009199.html >>>> >>>> Name: Squeak-Version-kfr.4713 >>>> Ancestors: Squeak-Version-kfr.4712 >>>> >>>> Unload and unregister Universes >>>> >>>> ============================================= >>>> >>> >>> _,,,^..^,,,_ >>> best, Eliot >>> >>> >>> >>> >> >> >> >> > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151210/3f0d64a0/attachment-0001.htm From karlramberg at gmail.com Thu Dec 10 16:35:17 2015 From: karlramberg at gmail.com (karl ramberg) Date: Thu Dec 10 16:35:21 2015 Subject: Broken Update [was [squeak-dev] Daily Commit Log] In-Reply-To: References: Message-ID: Thanks, Best, Karl On Thu, Dec 10, 2015 at 5:00 PM, Eliot Miranda wrote: > Hi Karl, > > On Wed, Dec 9, 2015 at 9:27 PM, karl ramberg > wrote: > >> Hi, >> It it related with this Levente mentioned in a mail earlier ? >> >> "Sadly the update process will break if your image has Universes-mt.48 in >> it, because it'll try to load Universes-kfr.49 after it has been removed by >> Squeak-Version. I think removing Universes from the update map will >> help." >> >> I asked how to proceed but I got no answer. >> I'm not sure how one make a update map >> > > As Chris says the updates live in the "update" package. Open up a > repository browser on source.squeak.org/trunk and you'll see them towards > the bottom (update.spur contains older updates from the time of the > bootstrap from 4.6 to what became 5.0, you can ignore them). > > What I would do is create a local mirror, either by fully populating my > package cache with all the 5.0 packages and updates, or creating a > directory repository and downloading all the packages and updates to there > and then play with the local repository until I got the update working > again, and then I would upload the changed updates to trunk. > > I'm attaching SpurBootstrapMonticelloPackagePatcher which is the hack I > used to automate patching 4.6 packages to make them Spur compatible in the > run-up to 5.0. I'm doing this because it contains methods like > downloadToSourceDirAllPackageVersionsStartingWith: that download all > versions of a package to a local repository starting from a specific > version. Sao these hacks will help you in populating your local repository > prior to experimenting. Once I'd downloaded all packages I would then take > a local copy and modify the copy, I could then diff the original and the > modified clone to see what had changed. > > HTH > > _,,,^..^,,,_ > best, Eliot > > Best, >> Karl >> >> On Thu, Dec 10, 2015 at 4:06 AM, Eliot Miranda >> wrote: >> >>> Hi, >>> >>> whatever happened in unloading Universes, it broke the ability to >>> update a release Squeak 5.0 image to the current tip. Hence my automated >>> scripts to build 64-bit images are broken. Can whoever broke the update >>> either fix it, or provide some reasonable workaround so that I can restore >>> my scripts? What's at stake here is, well, a high performance 64-bit >>> Squeak and/or Pharo. >>> >>> I think it's reasonable, even mandatory that we should be able to update >>> a release N image to the tip of release N. We should have CI servers that >>> try and do this (I think we do, don't we?), and so IMO it's incumbent on >>> whoever breaks the update to fix things, e.g. by changing update maps >>> appropriately. [and I don't mean this as a punishment, or something that >>> the person has to do on their own; I'm happy to help fix things, it's >>> /really/ important that the update works, and not just from the last >>> update]. >>> >>> Thanks in advance. Apologies if I appear rude; it's not my intent. I >>> just want my auto-update back ;-) >>> >>> On Thu, Nov 26, 2015 at 2:55 PM, wrote: >>> >>>> Changes to Trunk (http://source.squeak.org/trunk.html) in the last 24 >>>> hours: >>>> >>>> >>>> http://lists.squeakfoundation.org/pipermail/packages/2015-November/009199.html >>>> >>>> Name: Squeak-Version-kfr.4713 >>>> Ancestors: Squeak-Version-kfr.4712 >>>> >>>> Unload and unregister Universes >>>> >>>> ============================================= >>>> >>> >>> _,,,^..^,,,_ >>> best, Eliot >>> >> > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151210/1728e3dd/attachment.htm From leves at caesar.elte.hu Thu Dec 10 17:01:28 2015 From: leves at caesar.elte.hu (Levente Uzonyi) Date: Thu Dec 10 17:01:35 2015 Subject: Broken Update [was [squeak-dev] Daily Commit Log] In-Reply-To: References: Message-ID: IMHO all you need to do is to fire up a Squeak image which has Squeak-Version-kfr.4713 loaded, open update-mt.339.mcm (open the Trunk repository, select this update map, and click Browse), remove Universes-mt.48 from the list (select, right click, remove), and update Squeak-Version to kfr.4713 (select, right click, update from image (or repository if you're brave), and finally save it (Store) using the same name (update-mt.339). Here comes the tricky part: you can't overwrite an existing update map, nor can you move it away from the repository using the web interface, so an administrator of source.squeak.org will have to do one of these things to let you save the correct version. Alternatively, you could try to save it using different initials but same version (e.g. update-kfr.339) and hope that the update process will sort the maps lexicographically, but I'd rather not rely on this. Levente On Thu, 10 Dec 2015, karl ramberg wrote: > Thanks, > Best, > Karl > > On Thu, Dec 10, 2015 at 5:00 PM, Eliot Miranda wrote: > Hi Karl, > On Wed, Dec 9, 2015 at 9:27 PM, karl ramberg wrote: > Hi,It it related with this Levente mentioned in a mail earlier ? > > "Sadly the update process will break if your image has Universes-mt.48 in it, because it'll try to load Universes-kfr.49 after it has been removed by?Squeak-Version. I think removing Universes from the update map will help." > > I asked how to?proceed but I got no answer. > I'm not sure how one make a update map > > > As Chris says the updates live in the "update" package.? Open up a repository browser on source.squeak.org/trunk and you'll see them towards the bottom (update.spur contains older updates from the time of the bootstrap from 4.6 to what became 5.0, you can ignore > them). > > What I would do is create a local mirror, either by fully populating my package cache with all the 5.0 packages and updates, or creating a directory repository and downloading all the packages and updates to there and then play with the local repository until I > got the update working again, and then I would upload the changed updates to trunk. > > I'm attaching SpurBootstrapMonticelloPackagePatcher which is the hack I used to automate patching 4.6 packages to make them Spur compatible in the run-up to 5.0.? I'm doing this because it contains methods like downloadToSourceDirAllPackageVersionsStartingWith: > that download all versions of a package to a local repository starting from a specific version.? Sao these hacks will help you in populating your local repository prior to experimenting. Once I'd downloaded all packages I would then take a local copy and modify > the copy, I could then diff the original and the modified clone to see what had changed. > > HTH > > ?_,,,^..^,,,_ > best,?Eliot > > Best, > Karl? > [cleardot.gif] > > On Thu, Dec 10, 2015 at 4:06 AM, Eliot Miranda wrote: > Hi, > ? ? whatever happened in unloading Universes, it broke the ability to update a release Squeak 5.0 image to the current tip.? Hence my automated scripts to build 64-bit images are broken.? Can whoever broke the update either fix it, or provide some > reasonable workaround so that I can restore my scripts?? What's at stake here is, well, a high performance 64-bit Squeak and/or Pharo. > > I think it's reasonable, even mandatory that we should be able to update a release N image to the tip of release N.? We should have CI servers that try and do this (I think we do, don't we?), and so IMO it's incumbent on whoever breaks the update to > fix things, e.g. by changing update maps appropriately. ?[and I don't mean this as a punishment, or something that the person has to do on their own; I'm happy to help fix things, it's /really/ important that the update works, and not just from the > last update]. > > Thanks in advance.? Apologies if I appear rude; it's not my intent.? I just want my auto-update back ;-) > > On Thu, Nov 26, 2015 at 2:55 PM, wrote: > Changes to Trunk (http://source.squeak.org/trunk.html) in the last 24 hours: > > http://lists.squeakfoundation.org/pipermail/packages/2015-November/009199.html > > Name: Squeak-Version-kfr.4713 > Ancestors: Squeak-Version-kfr.4712 > > Unload and unregister Universes > > ============================================= > > ? > _,,,^..^,,,_ > best,?Eliot > > > > > > > > From Marcel.Taeumel at hpi.de Thu Dec 10 17:03:11 2015 From: Marcel.Taeumel at hpi.de (marcel.taeumel) Date: Thu Dec 10 17:20:34 2015 Subject: [squeak-dev] Re: The Inbox: Tools-pre.659.mcz In-Reply-To: References: <20151209000332.GA990@shell.msen.com> <20151209052207.GA55634@shell.msen.com> <1449733525113.93327@hpi.de> Message-ID: <1449766991012-4866519.post@n4.nabble.com> I don't know. There has been one for a long time. It just never worked. :-) Best, Marcel -- View this message in context: http://forum.world.st/The-Inbox-Tools-pre-659-mcz-tp4866102p4866519.html Sent from the Squeak - Dev mailing list archive at Nabble.com. From karlramberg at gmail.com Thu Dec 10 17:34:06 2015 From: karlramberg at gmail.com (karl ramberg) Date: Thu Dec 10 17:34:07 2015 Subject: Broken Update [was [squeak-dev] Daily Commit Log] In-Reply-To: References: Message-ID: On Thu, Dec 10, 2015 at 6:01 PM, Levente Uzonyi wrote: > IMHO all you need to do is to fire up a Squeak image which has > Squeak-Version-kfr.4713 loaded, open update-mt.339.mcm (open the Trunk > repository, select this update map, and click Browse), remove > Universes-mt.48 from the list (select, right click, remove), and update > Squeak-Version to kfr.4713 (select, right click, update from image (or > repository if you're brave), and finally save it (Store) using the same > name (update-mt.339). > I tried this, but there is no stored file from the MCConfigurationBrowser... I'm not sure what is going on, why it is not outputting a update file to my local repository. I also updated from a Squeak 5.0 image and I did not run into any problems with the update... Best, Karl > Here comes the tricky part: you can't overwrite an existing update map, > nor can you move it away from the repository using the web interface, so an > administrator of source.squeak.org will have to do one of these things to > let you save the correct version. > Alternatively, you could try to save it using different initials but same > version (e.g. update-kfr.339) and hope that the update process will sort > the maps lexicographically, but I'd rather not rely on this. > > Levente > > On Thu, 10 Dec 2015, karl ramberg wrote: > > Thanks, >> Best, >> Karl >> >> On Thu, Dec 10, 2015 at 5:00 PM, Eliot Miranda >> wrote: >> Hi Karl, >> On Wed, Dec 9, 2015 at 9:27 PM, karl ramberg >> wrote: >> Hi,It it related with this Levente mentioned in a mail >> earlier ? >> >> "Sadly the update process will break if your image has Universes-mt.48 in >> it, because it'll try to load Universes-kfr.49 after it has been removed >> by Squeak-Version. I think removing Universes from the update map will >> help." >> >> I asked how to proceed but I got no answer. >> I'm not sure how one make a update map >> >> >> As Chris says the updates live in the "update" package. Open up a >> repository browser on source.squeak.org/trunk and you'll see them >> towards the bottom (update.spur contains older updates from the time of the >> bootstrap from 4.6 to what became 5.0, you can ignore >> them). >> >> What I would do is create a local mirror, either by fully populating my >> package cache with all the 5.0 packages and updates, or creating a >> directory repository and downloading all the packages and updates to there >> and then play with the local repository until I >> got the update working again, and then I would upload the changed updates >> to trunk. >> >> I'm attaching SpurBootstrapMonticelloPackagePatcher which is the hack I >> used to automate patching 4.6 packages to make them Spur compatible in the >> run-up to 5.0. I'm doing this because it contains methods like >> downloadToSourceDirAllPackageVersionsStartingWith: >> that download all versions of a package to a local repository starting >> from a specific version. Sao these hacks will help you in populating your >> local repository prior to experimenting. Once I'd downloaded all packages I >> would then take a local copy and modify >> the copy, I could then diff the original and the modified clone to see >> what had changed. >> >> HTH >> >> _,,,^..^,,,_ >> best, Eliot >> >> Best, >> Karl >> [cleardot.gif] >> >> On Thu, Dec 10, 2015 at 4:06 AM, Eliot Miranda >> wrote: >> Hi, >> whatever happened in unloading Universes, it broke the ability to >> update a release Squeak 5.0 image to the current tip. Hence my automated >> scripts to build 64-bit images are broken. Can whoever broke the update >> either fix it, or provide some >> reasonable workaround so that I can restore my scripts? What's at stake >> here is, well, a high performance 64-bit Squeak and/or Pharo. >> >> I think it's reasonable, even mandatory that we should be able to update >> a release N image to the tip of release N. We should have CI servers that >> try and do this (I think we do, don't we?), and so IMO it's incumbent on >> whoever breaks the update to >> fix things, e.g. by changing update maps appropriately. [and I don't >> mean this as a punishment, or something that the person has to do on their >> own; I'm happy to help fix things, it's /really/ important that the update >> works, and not just from the >> last update]. >> >> Thanks in advance. Apologies if I appear rude; it's not my intent. I >> just want my auto-update back ;-) >> >> On Thu, Nov 26, 2015 at 2:55 PM, wrote: >> Changes to Trunk (http://source.squeak.org/trunk.html) in the last >> 24 hours: >> >> >> http://lists.squeakfoundation.org/pipermail/packages/2015-November/009199.html >> >> Name: Squeak-Version-kfr.4713 >> Ancestors: Squeak-Version-kfr.4712 >> >> Unload and unregister Universes >> >> ============================================= >> >> >> _,,,^..^,,,_ >> best, Eliot >> >> >> >> >> >> >> >> > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151210/98d355d1/attachment.htm From karlramberg at gmail.com Thu Dec 10 17:36:36 2015 From: karlramberg at gmail.com (karl ramberg) Date: Thu Dec 10 17:36:38 2015 Subject: [squeak-dev] Re: The Inbox: Tools-pre.659.mcz In-Reply-To: <1449766991012-4866519.post@n4.nabble.com> References: <20151209000332.GA990@shell.msen.com> <20151209052207.GA55634@shell.msen.com> <1449733525113.93327@hpi.de> <1449766991012-4866519.post@n4.nabble.com> Message-ID: The balloons may take a little time getting used to, and they may not be everyone's favorite, because they are quite big and obscures the list. Best, Karl On Thu, Dec 10, 2015 at 6:03 PM, marcel.taeumel wrote: > I don't know. There has been one for a long time. It just never worked. :-) > > Best, > Marcel > > > > -- > View this message in context: > http://forum.world.st/The-Inbox-Tools-pre-659-mcz-tp4866102p4866519.html > Sent from the Squeak - Dev mailing list archive at Nabble.com. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151210/5afe7809/attachment.htm From bert at freudenbergs.de Thu Dec 10 18:31:37 2015 From: bert at freudenbergs.de (Bert Freudenberg) Date: Thu Dec 10 18:31:41 2015 Subject: [squeak-dev] Re: The Inbox: Tools-pre.659.mcz In-Reply-To: References: <20151209000332.GA990@shell.msen.com> <20151209052207.GA55634@shell.msen.com> <1449733525113.93327@hpi.de> <1449766991012-4866519.post@n4.nabble.com> Message-ID: On 10.12.2015, at 18:36, karl ramberg wrote: > > The balloons may take a little time getting used to, and they may not be everyone's favorite, because they are quite big and obscures the list. We might have a preference to turn off all help balloons for people bothered by them. If someone complains. Which may or may not happen ;) But individual preferences to disable some balloons do not seem useful to me. In any case it should be *enabled* by default. I for one would be happy to see help for all UI elements. - Bert - -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4115 bytes Desc: not available Url : http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151210/7133da39/smime.bin From karlramberg at gmail.com Thu Dec 10 18:36:58 2015 From: karlramberg at gmail.com (karl ramberg) Date: Thu Dec 10 18:37:01 2015 Subject: [squeak-dev] Re: The Inbox: Tools-pre.659.mcz In-Reply-To: References: <20151209000332.GA990@shell.msen.com> <20151209052207.GA55634@shell.msen.com> <1449733525113.93327@hpi.de> <1449766991012-4866519.post@n4.nabble.com> Message-ID: On Thu, Dec 10, 2015 at 7:31 PM, Bert Freudenberg wrote: > On 10.12.2015, at 18:36, karl ramberg wrote: > > > > The balloons may take a little time getting used to, and they may not be > everyone's favorite, because they are quite big and obscures the list. > > > We might have a preference to turn off all help balloons for people > bothered by them. If someone complains. Which may or may not happen ;) > Yes, general preference for all balloon help seems more reasonable. Best, Karl > > But individual preferences to disable some balloons do not seem useful to > me. In any case it should be *enabled* by default. I for one would be happy > to see help for all UI elements. > > - Bert - > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151210/82e2c44c/attachment.htm From lecteur at zogotounga.net Thu Dec 10 18:37:08 2015 From: lecteur at zogotounga.net (=?UTF-8?Q?St=c3=a9phane_Rollandin?=) Date: Thu Dec 10 18:37:11 2015 Subject: [squeak-dev] Re: The Inbox: Tools-pre.659.mcz In-Reply-To: References: <20151209000332.GA990@shell.msen.com> <20151209052207.GA55634@shell.msen.com> <1449733525113.93327@hpi.de> <1449766991012-4866519.post@n4.nabble.com> Message-ID: <5669C654.3050001@zogotounga.net> >> The balloons may take a little time getting used to, and they may not be everyone's favorite, because they are quite big and obscures the list. > > > We might have a preference to turn off all help balloons for people bothered by them. If someone complains. Which may or may not happen ;) I would definitely turn that off. But of course I don't mind if it's on by default... Stef From tim at rowledge.org Thu Dec 10 18:54:24 2015 From: tim at rowledge.org (tim Rowledge) Date: Thu Dec 10 18:54:29 2015 Subject: [squeak-dev] Explorer bug when inspecting a ColorForm In-Reply-To: <1449736473537-4866478.post@n4.nabble.com> References: <8D6DE919-ED85-4375-B756-00FD018D8A3A@rowledge.org> <1449735537955-4866473.post@n4.nabble.com> <6E0FA07D-01CA-4EC7-9894-135F8C18A9B8@gmx.de> <1449735802543-4866475.post@n4.nabble.com> <1449736473537-4866478.post@n4.nabble.com> Message-ID: <38E77446-580D-4CDB-9265-DA5B796019E3@rowledge.org> > On 10-12-2015, at 12:34 AM, marcel.taeumel wrote: > > Fixed: http://forum.world.st/The-Trunk-Morphic-mt-1059-mcz-td4866477.html Thanks- that?s actually a solution I didn?t think of at the time because I was sure that the #translucentImage:at: required a 32bpp Form. Much better this way. tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Strange OpCodes: BYEBYE: Store in Write-Only Storage From lecteur at zogotounga.net Thu Dec 10 18:56:13 2015 From: lecteur at zogotounga.net (=?UTF-8?Q?St=c3=a9phane_Rollandin?=) Date: Thu Dec 10 18:56:19 2015 Subject: [squeak-dev] Re: The Inbox: Tools-pre.659.mcz In-Reply-To: References: <20151209000332.GA990@shell.msen.com> <20151209052207.GA55634@shell.msen.com> <1449733525113.93327@hpi.de> <1449766991012-4866519.post@n4.nabble.com> Message-ID: <5669CACD.2060909@zogotounga.net> > Yes, general preference for all balloon help seems more reasonable. Please reconsider. Some applications may define their own balloons and want them to appear (or let the user choose), while at the same time allowing other balloons such as the message lists ones to be turned off. It is a matter of workflow and management of the attention of the application user. I think there are no hard rules here. So having the possibility to turn off help balloons selectively is a good thing IMO, even if it is a little more complex to implement. Stef From karlramberg at gmail.com Thu Dec 10 19:12:01 2015 From: karlramberg at gmail.com (karl ramberg) Date: Thu Dec 10 19:12:03 2015 Subject: Broken Update [was [squeak-dev] Daily Commit Log] In-Reply-To: References: Message-ID: Skipped content of type multipart/alternative-------------- next part -------------- A non-text attachment was scrubbed... Name: update-kfr.339.mcm.gz Type: application/x-gzip Size: 3353 bytes Desc: not available Url : http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151210/e4703ab8/update-kfr.339.mcm.bin From karlramberg at gmail.com Thu Dec 10 19:15:52 2015 From: karlramberg at gmail.com (karl ramberg) Date: Thu Dec 10 19:15:54 2015 Subject: [squeak-dev] Re: The Inbox: Tools-pre.659.mcz In-Reply-To: <5669CACD.2060909@zogotounga.net> References: <20151209000332.GA990@shell.msen.com> <20151209052207.GA55634@shell.msen.com> <1449733525113.93327@hpi.de> <1449766991012-4866519.post@n4.nabble.com> <5669CACD.2060909@zogotounga.net> Message-ID: On Thu, Dec 10, 2015 at 7:56 PM, St?phane Rollandin wrote: > Yes, general preference for all balloon help seems more reasonable. >> > > Please reconsider. > > Some applications may define their own balloons and want them to appear > (or let the user choose), while at the same time allowing other balloons > such as the message lists ones to be turned off. > > It is a matter of workflow and management of the attention of the > application user. I think there are no hard rules here. > > So having the possibility to turn off help balloons selectively is a good > thing IMO, even if it is a little more complex to implement. > > Stef > > Ok, sounds reasonable. But the preferences list is quite overwhelming and ever expanding. Best, Karl -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151210/c67f55ec/attachment.htm From lecteur at zogotounga.net Thu Dec 10 19:37:33 2015 From: lecteur at zogotounga.net (=?UTF-8?Q?St=c3=a9phane_Rollandin?=) Date: Thu Dec 10 19:37:36 2015 Subject: [squeak-dev] Re: The Inbox: Tools-pre.659.mcz In-Reply-To: References: <20151209000332.GA990@shell.msen.com> <20151209052207.GA55634@shell.msen.com> <1449733525113.93327@hpi.de> <1449766991012-4866519.post@n4.nabble.com> <5669CACD.2060909@zogotounga.net> Message-ID: <5669D47D.7060902@zogotounga.net> > But the preferences list is quite overwhelming and ever expanding. Then maybe we could implement preferences themes ? "Experienced user mode", "Novice mode", "Developer mode", specific application mode... (another layer of complexity I'm afraid... but possibly making things simpler in the end) Stef From karlramberg at gmail.com Thu Dec 10 19:56:24 2015 From: karlramberg at gmail.com (karl ramberg) Date: Thu Dec 10 19:56:26 2015 Subject: [squeak-dev] Re: The Inbox: Tools-pre.659.mcz In-Reply-To: <5669D47D.7060902@zogotounga.net> References: <20151209000332.GA990@shell.msen.com> <20151209052207.GA55634@shell.msen.com> <1449733525113.93327@hpi.de> <1449766991012-4866519.post@n4.nabble.com> <5669CACD.2060909@zogotounga.net> <5669D47D.7060902@zogotounga.net> Message-ID: On Thu, Dec 10, 2015 at 8:37 PM, St?phane Rollandin wrote: > But the preferences list is quite overwhelming and ever expanding. >> > > Then maybe we could implement preferences themes ? "Experienced user > mode", "Novice mode", "Developer mode", specific application mode... > (another layer of complexity I'm afraid... but possibly making things > simpler in the end) > > Stef > I want tool specific preferences to be reached from the window menu of the tool. That way the organization could be delegated to each tool and not reside in a gigantic list with many categories. But it's a big job to reorganize it. Best, Karl -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151210/9f8decfd/attachment-0001.htm From dale.henrichs at gemtalksystems.com Thu Dec 10 20:27:36 2015 From: dale.henrichs at gemtalksystems.com (Dale Henrichs) Date: Thu Dec 10 20:27:41 2015 Subject: [squeak-dev] Re: [Cuis] Sorting Unicode strings (Re: [Unicode] collation sequences (Re: Unicode Support)) In-Reply-To: References: <5665EF88.5030901@gemtalksystems.com> <56672D90.4020202@gemtalksystems.com> <56674D32.2000305@gemtalksystems.com> <5668A467.7060302@gemtalksystems.com> Message-ID: <5669E038.4090103@gemtalksystems.com> On 12/09/2015 04:31 PM, Levente Uzonyi wrote: > On Wed, 9 Dec 2015, Dale Henrichs wrote: > >> >> >> On 12/09/2015 12:44 AM, Stephan Eggermont wrote: >>> On 08-12-15 22:35, Dale Henrichs wrote: >>>> What I meant is that you can't _always_ use the code point for >>>> collation, i.e., sorting based on the value of code points is not >>>> always >>>> correct[1]. >>> >>> I have given up on universal sorting when I learned that dutch >>> libraries sorting of author names depends on the country of origin >>> of the author. So if Jan van Beek is dutch he will be sorted under >>> B, while if he's belgian under V. I haven't checked what happens if >>> the author emigrates, or changes nationality... >>> >>> Stephan >>> >> Well, with ICU (and GemStone's implementation) you can choose which >> collator to use (Country specific) at the image level or on a >> comparison by comparison bases ... for example for an indexed >> collection (Unicode) Strings, you can choose the collator to use for >> that particular index ... so while it's true that universal sorter is >> not possible, it is possible to choose a collator that will satisfy a >> particlar customer .... > > I expect my image to compare strings using the codepoint-based > (+language tags) lexicographical method, because it's simple, > deterministic and fast. > Imagine having failing tests just because your image uses different > default comparison methods based on some (external) parameter... > It's also a nightmare to find out why your program is slow on some > machine, while it's fast on another. When we implemented the Unicode support in GemStone we preserved the legacy string classes and their legacy behavior ... We added new Unicode* classes with the new collator-based behavior for sorting and comparison ... That way legacy applications (and legacy) tests were not impacted by the choice of collator ... And folks could choose whether or not their application would benefit by the use of the new Unicode* classes.... The ICU library performance is actually comparable to our original implementations, so there isn't a noticeable performance difference - we built the support into our vm and if folks are interested in some of the gory technical details, we'd be willing to share our experience, as there are several things that we did to minimize potential performance impacts --- Dale From tim at rowledge.org Thu Dec 10 22:47:06 2015 From: tim at rowledge.org (tim Rowledge) Date: Thu Dec 10 22:47:12 2015 Subject: [squeak-dev] HandMorph etc #grabTransform Message-ID: <6D41EA3D-8470-4B02-B0E7-CC63CF0C4304@rowledge.org> Has anybody used the #grabTransform message that is sent by HandMorph as part of picking up a Morph? Every use I can find of it seems to set it to an Identity transform and trying to use it for make my dragged morph appear half-size is not working in a very understandable manner. HandMorph>grabMorph:from: is the major player here; it asks the formerOwner for the grabTransform (without specifying for which grabbed morph, which seems odd) and uses that along withe formerOwner?s transform to do things to the grabbed morph. If I make my owner-morph implement grabTransform to return an identity transform, the grabbing works as normal. If I provide atransform that does any scaling (which is what I want to use) then a) the positions and offsets get all screwy b) dropping doesn?t work the same - you have to drop the morph and then click-release again to get it to actually drop. That part really confuses me right now. I doubt this code was stuck in for the fun of it so I feel sure somebody must have had a use for it in the past. Does anyone remember anything about it? Oh joy; power cut. Hopefully the UPS will last long enough to get this sent. tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Useful random insult:- Overdue for deincarnation. From commits at source.squeak.org Thu Dec 10 22:55:02 2015 From: commits at source.squeak.org (commits@source.squeak.org) Date: Thu Dec 10 22:55:03 2015 Subject: [squeak-dev] Daily Commit Log Message-ID: <20151210225502.23205.qmail@box4.squeakfoundation.org> Changes to Trunk (http://source.squeak.org/trunk.html) in the last 24 hours: http://lists.squeakfoundation.org/pipermail/packages/2015-December/009214.html Name: Morphic-mt.1059 Ancestors: Morphic-tpr.1058 Fixes bug regarding the scaling of 8-bit color forms. ============================================= From eliot.miranda at gmail.com Fri Dec 11 00:29:10 2015 From: eliot.miranda at gmail.com (Eliot Miranda) Date: Fri Dec 11 00:29:13 2015 Subject: Broken Update [was [squeak-dev] Daily Commit Log] In-Reply-To: References: Message-ID: Thanks Karl, my auto-update script now works perfectly again. Thanks. On Thu, Dec 10, 2015 at 11:12 AM, karl ramberg wrote: > Here is a update-kfr.339.mcm with the difference from update-mt.339.mcm : > > dependency ('Squeak-Version' 'Squeak-Version-kfr.4713' > '96c171dc-c340-492e-b376-30e0212b9c8e') > > and universes are not in it > > On Thu, Dec 10, 2015 at 6:34 PM, karl ramberg > wrote: > >> >> >> On Thu, Dec 10, 2015 at 6:01 PM, Levente Uzonyi >> wrote: >> >>> IMHO all you need to do is to fire up a Squeak image which has >>> Squeak-Version-kfr.4713 loaded, open update-mt.339.mcm (open the Trunk >>> repository, select this update map, and click Browse), remove >>> Universes-mt.48 from the list (select, right click, remove), and update >>> Squeak-Version to kfr.4713 (select, right click, update from image (or >>> repository if you're brave), and finally save it (Store) using the same >>> name (update-mt.339). >>> >> >> I tried this, but there is no stored file from the >> MCConfigurationBrowser... I'm not sure what is going on, why it is not >> outputting a update file to my local repository. >> >> I also updated from a Squeak 5.0 image and I did not run into any >> problems with the update... >> >> >> Best, >> Karl >> >> >> >>> Here comes the tricky part: you can't overwrite an existing update map, >>> nor can you move it away from the repository using the web interface, so an >>> administrator of source.squeak.org will have to do one of these things >>> to let you save the correct version. >>> Alternatively, you could try to save it using different initials but >>> same version (e.g. update-kfr.339) and hope that the update process will >>> sort the maps lexicographically, but I'd rather not rely on this. >>> >>> Levente >>> >>> On Thu, 10 Dec 2015, karl ramberg wrote: >>> >>> Thanks, >>>> Best, >>>> Karl >>>> >>>> On Thu, Dec 10, 2015 at 5:00 PM, Eliot Miranda >>>> wrote: >>>> Hi Karl, >>>> On Wed, Dec 9, 2015 at 9:27 PM, karl ramberg < >>>> karlramberg@gmail.com> wrote: >>>> Hi,It it related with this Levente mentioned in a mail >>>> earlier ? >>>> >>>> "Sadly the update process will break if your image has Universes-mt.48 >>>> in it, because it'll try to load Universes-kfr.49 after it has been removed >>>> by Squeak-Version. I think removing Universes from the update map will >>>> help." >>>> >>>> I asked how to proceed but I got no answer. >>>> I'm not sure how one make a update map >>>> >>>> >>>> As Chris says the updates live in the "update" package. Open up a >>>> repository browser on source.squeak.org/trunk and you'll see them >>>> towards the bottom (update.spur contains older updates from the time of the >>>> bootstrap from 4.6 to what became 5.0, you can ignore >>>> them). >>>> >>>> What I would do is create a local mirror, either by fully populating my >>>> package cache with all the 5.0 packages and updates, or creating a >>>> directory repository and downloading all the packages and updates to there >>>> and then play with the local repository until I >>>> got the update working again, and then I would upload the changed >>>> updates to trunk. >>>> >>>> I'm attaching SpurBootstrapMonticelloPackagePatcher which is the hack I >>>> used to automate patching 4.6 packages to make them Spur compatible in the >>>> run-up to 5.0. I'm doing this because it contains methods like >>>> downloadToSourceDirAllPackageVersionsStartingWith: >>>> that download all versions of a package to a local repository starting >>>> from a specific version. Sao these hacks will help you in populating your >>>> local repository prior to experimenting. Once I'd downloaded all packages I >>>> would then take a local copy and modify >>>> the copy, I could then diff the original and the modified clone to see >>>> what had changed. >>>> >>>> HTH >>>> >>>> _,,,^..^,,,_ >>>> best, Eliot >>>> >>>> Best, >>>> Karl >>>> [cleardot.gif] >>>> >>>> On Thu, Dec 10, 2015 at 4:06 AM, Eliot Miranda >>>> wrote: >>>> Hi, >>>> whatever happened in unloading Universes, it broke the ability to >>>> update a release Squeak 5.0 image to the current tip. Hence my automated >>>> scripts to build 64-bit images are broken. Can whoever broke the update >>>> either fix it, or provide some >>>> reasonable workaround so that I can restore my scripts? What's at >>>> stake here is, well, a high performance 64-bit Squeak and/or Pharo. >>>> >>>> I think it's reasonable, even mandatory that we should be able to >>>> update a release N image to the tip of release N. We should have CI >>>> servers that try and do this (I think we do, don't we?), and so IMO it's >>>> incumbent on whoever breaks the update to >>>> fix things, e.g. by changing update maps appropriately. [and I don't >>>> mean this as a punishment, or something that the person has to do on their >>>> own; I'm happy to help fix things, it's /really/ important that the update >>>> works, and not just from the >>>> last update]. >>>> >>>> Thanks in advance. Apologies if I appear rude; it's not my intent. I >>>> just want my auto-update back ;-) >>>> >>>> On Thu, Nov 26, 2015 at 2:55 PM, wrote: >>>> Changes to Trunk (http://source.squeak.org/trunk.html) in the >>>> last 24 hours: >>>> >>>> >>>> http://lists.squeakfoundation.org/pipermail/packages/2015-November/009199.html >>>> >>>> Name: Squeak-Version-kfr.4713 >>>> Ancestors: Squeak-Version-kfr.4712 >>>> >>>> Unload and unregister Universes >>>> >>>> ============================================= >>>> >>>> >>>> _,,,^..^,,,_ >>>> best, Eliot >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>> >>> >>> >> > > > > -- _,,,^..^,,,_ best, Eliot -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151210/971974a5/attachment.htm From leves at caesar.elte.hu Fri Dec 11 01:03:27 2015 From: leves at caesar.elte.hu (Levente Uzonyi) Date: Fri Dec 11 01:03:32 2015 Subject: Broken Update [was [squeak-dev] Daily Commit Log] In-Reply-To: References: Message-ID: Well, I just realized that there's no need for magic to swap configurations, because update-mt.339 contains consistent versions of Squeak-Version and Universes. So the best is to just save a new .mcm file (v340) having Squeak-Version updated to the latest version and having Universes removed. Levente On Thu, 10 Dec 2015, karl ramberg wrote: > Here is a update-kfr.339.mcm with the difference from update-mt.339.mcm : > dependency ('Squeak-Version' 'Squeak-Version-kfr.4713' '96c171dc-c340-492e-b376-30e0212b9c8e') > > and universes are not in it > > On Thu, Dec 10, 2015 at 6:34 PM, karl ramberg wrote: > > > On Thu, Dec 10, 2015 at 6:01 PM, Levente Uzonyi wrote: > IMHO all you need to do is to fire up a Squeak image which has Squeak-Version-kfr.4713 loaded, open update-mt.339.mcm (open the Trunk repository, select this update map, and click Browse), remove Universes-mt.48 from the list (select, right > click, remove), and update Squeak-Version to kfr.4713 (select, right click, update from image (or repository if you're brave), and finally save it (Store) using the same name (update-mt.339). > > > I tried this, but there is no stored file from the MCConfigurationBrowser... I'm not sure what is going on, why it is not outputting a update file to my local repository. > > I also updated from a Squeak 5.0 image and I did not run into any problems with the update... > > > Best, > Karl > > ? > Here comes the tricky part: you can't overwrite an existing update map, nor can you move it away from the repository using the web interface, so an administrator of source.squeak.org will have to do one of these things to let you save the correct > version. > Alternatively, you could try to save it using different initials but same version (e.g. update-kfr.339) and hope that the update process will sort the maps lexicographically, but I'd rather not rely on this. > > Levente > > On Thu, 10 Dec 2015, karl ramberg wrote: > > Thanks, > Best, > Karl > > On Thu, Dec 10, 2015 at 5:00 PM, Eliot Miranda wrote: > ? ? ? Hi Karl, > ? ? ? On Wed, Dec 9, 2015 at 9:27 PM, karl ramberg wrote: > ? ? ? ? ? ? Hi,It it related with this Levente mentioned in a mail earlier ? > > "Sadly the update process will break if your image has Universes-mt.48 in it, because it'll try to load Universes-kfr.49 after it has been removed by?Squeak-Version. I think removing Universes from the update map will help." > > I asked how to?proceed but I got no answer. > I'm not sure how one make a update map > > > As Chris says the updates live in the "update" package.? Open up a repository browser on source.squeak.org/trunk and you'll see them towards the bottom (update.spur contains older updates from the time of the bootstrap from 4.6 to what > became 5.0, you can ignore > them). > > What I would do is create a local mirror, either by fully populating my package cache with all the 5.0 packages and updates, or creating a directory repository and downloading all the packages and updates to there and then play with the > local repository until I > got the update working again, and then I would upload the changed updates to trunk. > > I'm attaching SpurBootstrapMonticelloPackagePatcher which is the hack I used to automate patching 4.6 packages to make them Spur compatible in the run-up to 5.0.? I'm doing this because it contains methods like > downloadToSourceDirAllPackageVersionsStartingWith: > that download all versions of a package to a local repository starting from a specific version.? Sao these hacks will help you in populating your local repository prior to experimenting. Once I'd downloaded all packages I would then take a > local copy and modify > the copy, I could then diff the original and the modified clone to see what had changed. > > HTH > > ?_,,,^..^,,,_ > best,?Eliot > > ? ? ? Best, > Karl? > [cleardot.gif] > > On Thu, Dec 10, 2015 at 4:06 AM, Eliot Miranda wrote: > ? ? ? Hi, > ? ? whatever happened in unloading Universes, it broke the ability to update a release Squeak 5.0 image to the current tip.? Hence my automated scripts to build 64-bit images are broken.? Can whoever broke the update either fix it, or > provide some > reasonable workaround so that I can restore my scripts?? What's at stake here is, well, a high performance 64-bit Squeak and/or Pharo. > > I think it's reasonable, even mandatory that we should be able to update a release N image to the tip of release N.? We should have CI servers that try and do this (I think we do, don't we?), and so IMO it's incumbent on whoever breaks the > update to > fix things, e.g. by changing update maps appropriately. ?[and I don't mean this as a punishment, or something that the person has to do on their own; I'm happy to help fix things, it's /really/ important that the update works, and not just > from the > last update]. > > Thanks in advance.? Apologies if I appear rude; it's not my intent.? I just want my auto-update back ;-) > > On Thu, Nov 26, 2015 at 2:55 PM, wrote: > ? ? ? Changes to Trunk (http://source.squeak.org/trunk.html) in the last 24 hours: > > ? ? ? http://lists.squeakfoundation.org/pipermail/packages/2015-November/009199.html > > ? ? ? Name: Squeak-Version-kfr.4713 > ? ? ? Ancestors: Squeak-Version-kfr.4712 > > ? ? ? Unload and unregister Universes > > ? ? ? ============================================= > > ? > _,,,^..^,,,_ > best,?Eliot > > > > > > > > > > > > > > From euanmee at gmail.com Fri Dec 11 02:43:49 2015 From: euanmee at gmail.com (EuanM) Date: Fri Dec 11 02:43:52 2015 Subject: [squeak-dev] Re: [Pharo-dev] Unicode Support In-Reply-To: References: <5662E1D8.1070802@free.fr> <96B16DCE-A9E7-49E6-B866-E5A846801AA0@stfx.eu> <0F104967-2F55-431F-9EED-0275ABC7CEC1@veloxit.no> <52F49745-64AA-447A-A75A-71FB86FCE36E@veloxit.no> <434B72EE-528B-4ECC-9DCE-78A8F7A6E43E@gmail.com> Message-ID: I agree with all of that, Ben. I'm currently fairly certain that fully-composed abstract characters is a term that is 1:1 mapped with the term "grapheme cluster" (i.e. one is an older Unicode description of a newer Unicode term). And once we create these, I think this sort of implementation is straightforward. For particular values of "straightforward", of course :-) i.e. the Swift approach is equivalent to the approach I originally proposed and asked for critiques of. One thing I don't understand.... why does the fact the composed abstract character (aka grapheme cluster) is a sequence mean that an array cannot be used to hold the sequence? If people then also want a compatibility-codepoints-only UTF-8 representation, it is simple to provide comparable (i.e equivalence-testable) versions of any UTF-8 string - because we are creating them from composed forms by a *single* defined method. For my part, the reason I think we ought to implement it *in* Smalltalk is ... this is the String class of the new age. I want Smalltalk to be handle Strings as native objects. On 10 December 2015 at 23:41, Ben Coman wrote: > On Wed, Dec 9, 2015 at 5:35 PM, Guillermo Polito > wrote: >> >>> On 8 dic 2015, at 10:07 p.m., EuanM wrote: >>> >>> "No. a codepoint is the numerical value assigned to a character. An >>> "encoded character" is the way a codepoint is represented in bytes >>> using a given encoding." >>> >>> No. >>> >>> A codepoint may represent a component part of an abstract character, >>> or may represent an abstract character, or it may do both (but not >>> always at the same time). >>> >>> Codepoints represent a single encoding of a single concept. >>> >>> Sometimes that concept represents a whole abstract character. >>> Sometimes it represent part of an abstract character. >> >> Well. I do not agree with this. I agree with the quote. >> >> Can you explain a bit more about what you mean by abstract character and concept? > > This seems to be what Swift is doing, where Strings are not composed > not of codepoints but of graphemes. > >>>> "Every instance of Swift?s Character type represents a single extended grapheme cluster. An extended grapheme cluster is a sequence** of one or more Unicode scalars that (when combined) produce a single human-readable character. [1] > > ** i.e. not an array > >>>> Here?s an example. The letter ? can be represented as the single Unicode scalar ? (LATIN SMALL LETTER E WITH ACUTE, or U+00E9). However, the same letter can also be represented as a pair of scalars?a standard letter e (LATIN SMALL LETTER E, or U+0065), followed by the COMBINING ACUTE ACCENT scalar (U+0301). TheCOMBINING ACUTE ACCENT scalar is graphically applied to the scalar that precedes it, turning an e into an ?when it is rendered by a Unicode-aware text-rendering system. [1] > >>>> In both cases, the letter ? is represented as a single Swift Character value that represents an extended grapheme cluster. In the first case, the cluster contains a single scalar; in the second case, it is a cluster of two scalars:" [1] > >>>> Swift?s string implemenation makes working with Unicode easier and significantly less error-prone. As a programmer, you still have to be aware of possible edge cases, but this probably cannot be avoided completely considering the characteristics of Unicode. [2] > > Indeed I've tried searched for what problems it causes and get a null > result. So I read *all*good* things about Swift's unicode > implementation reducing common errors dealing with Unicode. Can > anyone point to complaints about Swift's unicode implementation? > Maybe this... > >>>> An argument could be made that the implementation of String as a sequence that requires iterating over characters from the beginning of the string for many operations poses a significant performance problem but I do not think so. My guess is that Apple?s engineers have considered the implications of their implementation and apps that do not deal with enormous amounts of text will be fine. Moreover, the idea that you could get away with an implementation that supports random access of characters is an illusion given the complexity of Unicode. [2] > > Considering our common pattern: Make it work, Make it right, Make it > fast -- maybe Strings as arrays are a premature optimisation, that > was the right choice in the past prior to Unicode, but considering > Moore's Law versus programmer time, is not the best choice now. > Should we at least start with a UnicodeString and UnicodeCharacter > that operates like Swift, and over time *maybe* move the tools to use > them. > > [1] https://developer.apple.com/library/ios/documentation/Swift/Conceptual/Swift_Programming_Language/StringsAndCharacters.html > [2] http://oleb.net/blog/2014/07/swift-strings/ > > cheers -ben > >> >>> >>> This is the key difference between Unicode and most character encodings. >>> >>> A codepoint does not always represent a whole character. >>> >>> On 7 December 2015 at 13:06, Henrik Johansen > From robert.w.withers at gmail.com Fri Dec 11 04:00:45 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Fri Dec 11 04:00:55 2015 Subject: [squeak-dev] looking for an image file Message-ID: <566A4A6D.4080702@gmail.com> Where could I find an image file version 6505? I need to see to something first and hopefully all will be well. Thank you, Robert From robert.w.withers at gmail.com Fri Dec 11 04:09:12 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Fri Dec 11 04:09:17 2015 Subject: [squeak-dev] Re: looking for an image file In-Reply-To: <566A4A6D.4080702@gmail.com> References: <566A4A6D.4080702@gmail.com> Message-ID: <566A4C68.5000002@gmail.com> Scratch that. I need to get setup with a 4.6 image so I can look into Fuel on the squeak side. Where could I find that work? thanks, Robert On 12/10/2015 11:00 PM, Robert Withers wrote: > Where could I find an image file version 6505? I need to see to > something first and hopefully all will be well. > > Thank you, > Robert From robert.w.withers at gmail.com Fri Dec 11 04:19:20 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Fri Dec 11 04:19:25 2015 Subject: [squeak-dev] Re: looking for an image file In-Reply-To: <566A4C68.5000002@gmail.com> References: <566A4A6D.4080702@gmail.com> <566A4C68.5000002@gmail.com> Message-ID: <566A4EC8.9000805@gmail.com> I do need to hit up 4.6. But also, the newest interpreter is for image version 6505 and the updated image is 6521. I am not sure howe to proceed.. On 12/10/2015 11:09 PM, Robert Withers wrote: > Scratch that. I need to get setup with a 4.6 image so I can look into > Fuel on the squeak side. Where could I find that work? > > thanks, > Robert > > On 12/10/2015 11:00 PM, Robert Withers wrote: >> Where could I find an image file version 6505? I need to see to >> something first and hopefully all will be well. >> >> Thank you, >> Robert > From karlramberg at gmail.com Fri Dec 11 05:59:33 2015 From: karlramberg at gmail.com (karl ramberg) Date: Fri Dec 11 05:59:36 2015 Subject: Broken Update [was [squeak-dev] Daily Commit Log] In-Reply-To: References: Message-ID: Hm, I'm a little confused/ worried because I did not change anything yet. Best, Karl On Fri, Dec 11, 2015 at 1:29 AM, Eliot Miranda wrote: > Thanks Karl, > > my auto-update script now works perfectly again. Thanks. > > On Thu, Dec 10, 2015 at 11:12 AM, karl ramberg > wrote: > >> Here is a update-kfr.339.mcm with the difference from update-mt.339.mcm : >> >> dependency ('Squeak-Version' 'Squeak-Version-kfr.4713' >> '96c171dc-c340-492e-b376-30e0212b9c8e') >> >> and universes are not in it >> >> On Thu, Dec 10, 2015 at 6:34 PM, karl ramberg >> wrote: >> >>> >>> >>> On Thu, Dec 10, 2015 at 6:01 PM, Levente Uzonyi >>> wrote: >>> >>>> IMHO all you need to do is to fire up a Squeak image which has >>>> Squeak-Version-kfr.4713 loaded, open update-mt.339.mcm (open the Trunk >>>> repository, select this update map, and click Browse), remove >>>> Universes-mt.48 from the list (select, right click, remove), and update >>>> Squeak-Version to kfr.4713 (select, right click, update from image (or >>>> repository if you're brave), and finally save it (Store) using the same >>>> name (update-mt.339). >>>> >>> >>> I tried this, but there is no stored file from the >>> MCConfigurationBrowser... I'm not sure what is going on, why it is not >>> outputting a update file to my local repository. >>> >>> I also updated from a Squeak 5.0 image and I did not run into any >>> problems with the update... >>> >>> >>> Best, >>> Karl >>> >>> >>> >>>> Here comes the tricky part: you can't overwrite an existing update map, >>>> nor can you move it away from the repository using the web interface, so an >>>> administrator of source.squeak.org will have to do one of these things >>>> to let you save the correct version. >>>> Alternatively, you could try to save it using different initials but >>>> same version (e.g. update-kfr.339) and hope that the update process will >>>> sort the maps lexicographically, but I'd rather not rely on this. >>>> >>>> Levente >>>> >>>> On Thu, 10 Dec 2015, karl ramberg wrote: >>>> >>>> Thanks, >>>>> Best, >>>>> Karl >>>>> >>>>> On Thu, Dec 10, 2015 at 5:00 PM, Eliot Miranda < >>>>> eliot.miranda@gmail.com> wrote: >>>>> Hi Karl, >>>>> On Wed, Dec 9, 2015 at 9:27 PM, karl ramberg < >>>>> karlramberg@gmail.com> wrote: >>>>> Hi,It it related with this Levente mentioned in a mail >>>>> earlier ? >>>>> >>>>> "Sadly the update process will break if your image has Universes-mt.48 >>>>> in it, because it'll try to load Universes-kfr.49 after it has been removed >>>>> by Squeak-Version. I think removing Universes from the update map will >>>>> help." >>>>> >>>>> I asked how to proceed but I got no answer. >>>>> I'm not sure how one make a update map >>>>> >>>>> >>>>> As Chris says the updates live in the "update" package. Open up a >>>>> repository browser on source.squeak.org/trunk and you'll see them >>>>> towards the bottom (update.spur contains older updates from the time of the >>>>> bootstrap from 4.6 to what became 5.0, you can ignore >>>>> them). >>>>> >>>>> What I would do is create a local mirror, either by fully populating >>>>> my package cache with all the 5.0 packages and updates, or creating a >>>>> directory repository and downloading all the packages and updates to there >>>>> and then play with the local repository until I >>>>> got the update working again, and then I would upload the changed >>>>> updates to trunk. >>>>> >>>>> I'm attaching SpurBootstrapMonticelloPackagePatcher which is the hack >>>>> I used to automate patching 4.6 packages to make them Spur compatible in >>>>> the run-up to 5.0. I'm doing this because it contains methods like >>>>> downloadToSourceDirAllPackageVersionsStartingWith: >>>>> that download all versions of a package to a local repository starting >>>>> from a specific version. Sao these hacks will help you in populating your >>>>> local repository prior to experimenting. Once I'd downloaded all packages I >>>>> would then take a local copy and modify >>>>> the copy, I could then diff the original and the modified clone to see >>>>> what had changed. >>>>> >>>>> HTH >>>>> >>>>> _,,,^..^,,,_ >>>>> best, Eliot >>>>> >>>>> Best, >>>>> Karl >>>>> [cleardot.gif] >>>>> >>>>> On Thu, Dec 10, 2015 at 4:06 AM, Eliot Miranda < >>>>> eliot.miranda@gmail.com> wrote: >>>>> Hi, >>>>> whatever happened in unloading Universes, it broke the ability to >>>>> update a release Squeak 5.0 image to the current tip. Hence my automated >>>>> scripts to build 64-bit images are broken. Can whoever broke the update >>>>> either fix it, or provide some >>>>> reasonable workaround so that I can restore my scripts? What's at >>>>> stake here is, well, a high performance 64-bit Squeak and/or Pharo. >>>>> >>>>> I think it's reasonable, even mandatory that we should be able to >>>>> update a release N image to the tip of release N. We should have CI >>>>> servers that try and do this (I think we do, don't we?), and so IMO it's >>>>> incumbent on whoever breaks the update to >>>>> fix things, e.g. by changing update maps appropriately. [and I don't >>>>> mean this as a punishment, or something that the person has to do on their >>>>> own; I'm happy to help fix things, it's /really/ important that the update >>>>> works, and not just from the >>>>> last update]. >>>>> >>>>> Thanks in advance. Apologies if I appear rude; it's not my intent. I >>>>> just want my auto-update back ;-) >>>>> >>>>> On Thu, Nov 26, 2015 at 2:55 PM, wrote: >>>>> Changes to Trunk (http://source.squeak.org/trunk.html) in the >>>>> last 24 hours: >>>>> >>>>> >>>>> http://lists.squeakfoundation.org/pipermail/packages/2015-November/009199.html >>>>> >>>>> Name: Squeak-Version-kfr.4713 >>>>> Ancestors: Squeak-Version-kfr.4712 >>>>> >>>>> Unload and unregister Universes >>>>> >>>>> ============================================= >>>>> >>>>> >>>>> _,,,^..^,,,_ >>>>> best, Eliot >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>> >>>> >>>> >>> >> >> >> >> > > > -- > _,,,^..^,,,_ > best, Eliot > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151211/1041dd73/attachment.htm From btc at openinworld.com Fri Dec 11 06:31:13 2015 From: btc at openinworld.com (Ben Coman) Date: Fri Dec 11 06:31:36 2015 Subject: [squeak-dev] Re: [Pharo-dev] Unicode Support In-Reply-To: References: <5662E1D8.1070802@free.fr> <96B16DCE-A9E7-49E6-B866-E5A846801AA0@stfx.eu> <0F104967-2F55-431F-9EED-0275ABC7CEC1@veloxit.no> <52F49745-64AA-447A-A75A-71FB86FCE36E@veloxit.no> <434B72EE-528B-4ECC-9DCE-78A8F7A6E43E@gmail.com> Message-ID: On Fri, Dec 11, 2015 at 10:43 AM, EuanM wrote: > I agree with all of that, Ben. > > I'm currently fairly certain that fully-composed abstract characters > is a term that is 1:1 mapped with the term "grapheme cluster" (i.e. > one is an older Unicode description of a newer Unicode term). > > And once we create these, I think this sort of implementation is > straightforward. For particular values of "straightforward", of > course :-) > > i.e. the Swift approach is equivalent to the approach I originally > proposed and asked for critiques of. > > One thing I don't understand.... why does the fact the composed > abstract character (aka grapheme cluster) is a sequence mean that an > array cannot be used to hold the sequence? I realised the same question after I had posted. I don't know the answer. Maybe just that particular implementation. cheers -ben > If people then also want a compatibility-codepoints-only UTF-8 > representation, it is simple to provide comparable (i.e > equivalence-testable) versions of any UTF-8 string - because we are > creating them from composed forms by a *single* defined method. > > For my part, the reason I think we ought to implement it *in* > Smalltalk is ... this is the String class of the new age. I want > Smalltalk to be handle Strings as native objects. > > > On 10 December 2015 at 23:41, Ben Coman wrote: >> On Wed, Dec 9, 2015 at 5:35 PM, Guillermo Polito >> wrote: >>> >>>> On 8 dic 2015, at 10:07 p.m., EuanM wrote: >>>> >>>> "No. a codepoint is the numerical value assigned to a character. An >>>> "encoded character" is the way a codepoint is represented in bytes >>>> using a given encoding." >>>> >>>> No. >>>> >>>> A codepoint may represent a component part of an abstract character, >>>> or may represent an abstract character, or it may do both (but not >>>> always at the same time). >>>> >>>> Codepoints represent a single encoding of a single concept. >>>> >>>> Sometimes that concept represents a whole abstract character. >>>> Sometimes it represent part of an abstract character. >>> >>> Well. I do not agree with this. I agree with the quote. >>> >>> Can you explain a bit more about what you mean by abstract character and concept? >> >> This seems to be what Swift is doing, where Strings are not composed >> not of codepoints but of graphemes. >> >>>>> "Every instance of Swift?s Character type represents a single extended grapheme cluster. An extended grapheme cluster is a sequence** of one or more Unicode scalars that (when combined) produce a single human-readable character. [1] >> >> ** i.e. not an array >> >>>>> Here?s an example. The letter ? can be represented as the single Unicode scalar ? (LATIN SMALL LETTER E WITH ACUTE, or U+00E9). However, the same letter can also be represented as a pair of scalars?a standard letter e (LATIN SMALL LETTER E, or U+0065), followed by the COMBINING ACUTE ACCENT scalar (U+0301). TheCOMBINING ACUTE ACCENT scalar is graphically applied to the scalar that precedes it, turning an e into an ?when it is rendered by a Unicode-aware text-rendering system. [1] >> >>>>> In both cases, the letter ? is represented as a single Swift Character value that represents an extended grapheme cluster. In the first case, the cluster contains a single scalar; in the second case, it is a cluster of two scalars:" [1] >> >>>>> Swift?s string implemenation makes working with Unicode easier and significantly less error-prone. As a programmer, you still have to be aware of possible edge cases, but this probably cannot be avoided completely considering the characteristics of Unicode. [2] >> >> Indeed I've tried searched for what problems it causes and get a null >> result. So I read *all*good* things about Swift's unicode >> implementation reducing common errors dealing with Unicode. Can >> anyone point to complaints about Swift's unicode implementation? >> Maybe this... >> >>>>> An argument could be made that the implementation of String as a sequence that requires iterating over characters from the beginning of the string for many operations poses a significant performance problem but I do not think so. My guess is that Apple?s engineers have considered the implications of their implementation and apps that do not deal with enormous amounts of text will be fine. Moreover, the idea that you could get away with an implementation that supports random access of characters is an illusion given the complexity of Unicode. [2] >> >> Considering our common pattern: Make it work, Make it right, Make it >> fast -- maybe Strings as arrays are a premature optimisation, that >> was the right choice in the past prior to Unicode, but considering >> Moore's Law versus programmer time, is not the best choice now. >> Should we at least start with a UnicodeString and UnicodeCharacter >> that operates like Swift, and over time *maybe* move the tools to use >> them. >> >> [1] https://developer.apple.com/library/ios/documentation/Swift/Conceptual/Swift_Programming_Language/StringsAndCharacters.html >> [2] http://oleb.net/blog/2014/07/swift-strings/ >> >> cheers -ben >> >>> >>>> >>>> This is the key difference between Unicode and most character encodings. >>>> >>>> A codepoint does not always represent a whole character. >>>> >>>> On 7 December 2015 at 13:06, Henrik Johansen >> > From eliot.miranda at gmail.com Fri Dec 11 09:29:13 2015 From: eliot.miranda at gmail.com (Eliot Miranda) Date: Fri Dec 11 09:29:21 2015 Subject: [squeak-dev] Re: [Pharo-dev] Unicode Support In-Reply-To: References: <5662E1D8.1070802@free.fr> <96B16DCE-A9E7-49E6-B866-E5A846801AA0@stfx.eu> <0F104967-2F55-431F-9EED-0275ABC7CEC1@veloxit.no> <52F49745-64AA-447A-A75A-71FB86FCE36E@veloxit.no> <434B72EE-528B-4ECC-9DCE-78A8F7A6E43E@gmail.com> Message-ID: <7CC2EE2F-676A-4183-99DD-A27C6480FC64@gmail.com> Hi Euan, > On Dec 10, 2015, at 6:43 PM, EuanM wrote: > > I agree with all of that, Ben. > > I'm currently fairly certain that fully-composed abstract characters > is a term that is 1:1 mapped with the term "grapheme cluster" (i.e. > one is an older Unicode description of a newer Unicode term). > > And once we create these, I think this sort of implementation is > straightforward. For particular values of "straightforward", of > course :-) > > i.e. the Swift approach is equivalent to the approach I originally > proposed and asked for critiques of. > > One thing I don't understand.... why does the fact the composed > abstract character (aka grapheme cluster) is a sequence mean that an > array cannot be used to hold the sequence? Of course an Array can be used, but one good reason to use bits organized as four-byte units is that the garbage collector spends no time scanning them, whereas as far as its concerned the Array representation is all objects and must be scanned. Another reason is that foreign code may find the bits representation compatible and so they can be passed through the FFI to other languages whereas the Array of tagged characters will always require conversion. Yet another reason is that in 64-bits the Array takes twice the space of the bits object. > If people then also want a compatibility-codepoints-only UTF-8 > representation, it is simple to provide comparable (i.e > equivalence-testable) versions of any UTF-8 string - because we are > creating them from composed forms by a *single* defined method. > > For my part, the reason I think we ought to implement it *in* > Smalltalk is ... this is the String class of the new age. I want > Smalltalk to be handle Strings as native objects. There's little if any difference in convenience of use between an Array of characters and a bits array with the string at:/at:put: primitives since both require at:/at:put: to access, but the latter is (efficiently) type checked (by the VM), whereas there's nothing to prevent storing other than characters in the Areay unless one introduces the overhead of skier explicit type checks in Smalltalk, and the Areay starts life as a sequence of nils (invalid until every element is set to a character) whereas the bits representation begins fully initialized with 0 asCharacter. So there's nothing more "natively objecty" about the Array. Smalltalk objects hide their representation from clients and externally they behave the same, except for space and time. Given that this is a dynamically-typed language there's nothing to prevent one providing both implementations beyond maintenance cost and complexity/confusion. So at least it's easy to do performance comparisons between the two. But I still think the bits representation is superior if what you want is a sequence of Characters. >> On 10 December 2015 at 23:41, Ben Coman wrote: >> On Wed, Dec 9, 2015 at 5:35 PM, Guillermo Polito >> wrote: >>> >>>> On 8 dic 2015, at 10:07 p.m., EuanM wrote: >>>> >>>> "No. a codepoint is the numerical value assigned to a character. An >>>> "encoded character" is the way a codepoint is represented in bytes >>>> using a given encoding." >>>> >>>> No. >>>> >>>> A codepoint may represent a component part of an abstract character, >>>> or may represent an abstract character, or it may do both (but not >>>> always at the same time). >>>> >>>> Codepoints represent a single encoding of a single concept. >>>> >>>> Sometimes that concept represents a whole abstract character. >>>> Sometimes it represent part of an abstract character. >>> >>> Well. I do not agree with this. I agree with the quote. >>> >>> Can you explain a bit more about what you mean by abstract character and concept? >> >> This seems to be what Swift is doing, where Strings are not composed >> not of codepoints but of graphemes. >> >>>>> "Every instance of Swift?s Character type represents a single extended grapheme cluster. An extended grapheme cluster is a sequence** of one or more Unicode scalars that (when combined) produce a single human-readable character. [1] >> >> ** i.e. not an array >> >>>>> Here?s an example. The letter ? can be represented as the single Unicode scalar ? (LATIN SMALL LETTER E WITH ACUTE, or U+00E9). However, the same letter can also be represented as a pair of scalars?a standard letter e (LATIN SMALL LETTER E, or U+0065), followed by the COMBINING ACUTE ACCENT scalar (U+0301). TheCOMBINING ACUTE ACCENT scalar is graphically applied to the scalar that precedes it, turning an e into an ?when it is rendered by a Unicode-aware text-rendering system. [1] >> >>>>> In both cases, the letter ? is represented as a single Swift Character value that represents an extended grapheme cluster. In the first case, the cluster contains a single scalar; in the second case, it is a cluster of two scalars:" [1] >> >>>>> Swift?s string implemenation makes working with Unicode easier and significantly less error-prone. As a programmer, you still have to be aware of possible edge cases, but this probably cannot be avoided completely considering the characteristics of Unicode. [2] >> >> Indeed I've tried searched for what problems it causes and get a null >> result. So I read *all*good* things about Swift's unicode >> implementation reducing common errors dealing with Unicode. Can >> anyone point to complaints about Swift's unicode implementation? >> Maybe this... >> >>>>> An argument could be made that the implementation of String as a sequence that requires iterating over characters from the beginning of the string for many operations poses a significant performance problem but I do not think so. My guess is that Apple?s engineers have considered the implications of their implementation and apps that do not deal with enormous amounts of text will be fine. Moreover, the idea that you could get away with an implementation that supports random access of characters is an illusion given the complexity of Unicode. [2] >> >> Considering our common pattern: Make it work, Make it right, Make it >> fast -- maybe Strings as arrays are a premature optimisation, that >> was the right choice in the past prior to Unicode, but considering >> Moore's Law versus programmer time, is not the best choice now. >> Should we at least start with a UnicodeString and UnicodeCharacter >> that operates like Swift, and over time *maybe* move the tools to use >> them. >> >> [1] https://developer.apple.com/library/ios/documentation/Swift/Conceptual/Swift_Programming_Language/StringsAndCharacters.html >> [2] http://oleb.net/blog/2014/07/swift-strings/ >> >> cheers -ben >> >>> >>>> >>>> This is the key difference between Unicode and most character encodings. >>>> >>>> A codepoint does not always represent a whole character. >>>> >>>> On 7 December 2015 at 13:06, Henrik Johansen _,,,^..^,,,_ (phone) From lewis at mail.msen.com Fri Dec 11 14:20:06 2015 From: lewis at mail.msen.com (David T. Lewis) Date: Fri Dec 11 14:20:07 2015 Subject: [squeak-dev] Re: [Vm-dev] Re: looking for an image file In-Reply-To: <566A4EC8.9000805@gmail.com> References: <566A4A6D.4080702@gmail.com> <566A4C68.5000002@gmail.com> <566A4EC8.9000805@gmail.com> Message-ID: <20151211142006.GA57925@shell.msen.com> On Thu, Dec 10, 2015 at 11:19:20PM -0500, Robert Withers wrote: > > I do need to hit up 4.6. But also, the newest interpreter is for image > version 6505 and the updated image is 6521. I am not sure howe to proceed.. For an explanation of the various image formats, load package ImageFormat from SqueakMap. Read the class comment first, then look at the class side utility methods. Examples: ImageFormat versionDescriptions do: [:e | Transcript cr; show: e] (ImageFormat fromFile: Smalltalk imageName) description Loosely speaking, Squeak 4.6 is image format 6405, and works with Cog and interpreter VMs. Squeak 5.0 and trunk are image format 6521, and work with Spur. Dave > > On 12/10/2015 11:09 PM, Robert Withers wrote: > >Scratch that. I need to get setup with a 4.6 image so I can look into > >Fuel on the squeak side. Where could I find that work? > > > >thanks, > >Robert > > > >On 12/10/2015 11:00 PM, Robert Withers wrote: > >>Where could I find an image file version 6505? I need to see to > >>something first and hopefully all will be well. > >> > >>Thank you, > >>Robert > > From eliot.miranda at gmail.com Fri Dec 11 16:07:35 2015 From: eliot.miranda at gmail.com (Eliot Miranda) Date: Fri Dec 11 16:07:42 2015 Subject: [squeak-dev] Re: looking for an image file In-Reply-To: <566A4C68.5000002@gmail.com> References: <566A4A6D.4080702@gmail.com> <566A4C68.5000002@gmail.com> Message-ID: <2DBABD81-CB1D-4B9C-B090-FF84572D9B40@gmail.com> Hi Robert, off the top of my head I can't see why Fuel wouldn't work in 5.0. There might be a few issues around Characters, which are immediates in 5.0, but that shouldn't be hard to fix. _,,,^..^,,,_ (phone) > On Dec 10, 2015, at 8:09 PM, Robert Withers wrote: > > Scratch that. I need to get setup with a 4.6 image so I can look into Fuel on the squeak side. Where could I find that work? > > thanks, > Robert > >> On 12/10/2015 11:00 PM, Robert Withers wrote: >> Where could I find an image file version 6505? I need to see to something first and hopefully all will be well. >> >> Thank you, >> Robert > > From cunningham.cb at gmail.com Fri Dec 11 16:19:56 2015 From: cunningham.cb at gmail.com (Chris Cunningham) Date: Fri Dec 11 16:19:59 2015 Subject: [squeak-dev] Re: looking for an image file In-Reply-To: <2DBABD81-CB1D-4B9C-B090-FF84572D9B40@gmail.com> References: <566A4A6D.4080702@gmail.com> <566A4C68.5000002@gmail.com> <2DBABD81-CB1D-4B9C-B090-FF84572D9B40@gmail.com> Message-ID: Having just loaded it into 5.0 myself, the first (biggest?) problem was that there was no squeak compatibility defined for 5.0. I suspect that if you opened the Fuel repository, though, and loaded the squeak46 compatibility package, that should work. And then there appears to be some issue with FLLargeIdentitySet initializing it's array to 4096, but our LargeIdentityHash returning significantly larger numbers. Has primitive 75 changed in 5.0? -cbc On Fri, Dec 11, 2015 at 8:07 AM, Eliot Miranda wrote: > Hi Robert, > > off the top of my head I can't see why Fuel wouldn't work in 5.0. > There might be a few issues around Characters, which are immediates in 5.0, > but that shouldn't be hard to fix. > > _,,,^..^,,,_ (phone) > > > On Dec 10, 2015, at 8:09 PM, Robert Withers > wrote: > > > > Scratch that. I need to get setup with a 4.6 image so I can look into > Fuel on the squeak side. Where could I find that work? > > > > thanks, > > Robert > > > >> On 12/10/2015 11:00 PM, Robert Withers wrote: > >> Where could I find an image file version 6505? I need to see to > something first and hopefully all will be well. > >> > >> Thank you, > >> Robert > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151211/3aadd6f1/attachment.htm From maxleske at gmail.com Fri Dec 11 16:25:48 2015 From: maxleske at gmail.com (Max Leske) Date: Fri Dec 11 16:25:51 2015 Subject: [squeak-dev] ftp down? Message-ID: <365CC8D7-DF0D-47CF-B71A-4174EAC3C27A@gmail.com> Hi I?m trying to get the Fuel build set up for 5.0 but I can?t seem to connect to ftp.squeak.org. It?s available via http but not via ftp. Alternatively, how do you access that server for the Squeak builds? Should I use a different protocol? Cheers, Max From maxleske at gmail.com Fri Dec 11 16:38:45 2015 From: maxleske at gmail.com (Max Leske) Date: Fri Dec 11 16:38:49 2015 Subject: [squeak-dev] Re: ftp down? In-Reply-To: <365CC8D7-DF0D-47CF-B71A-4174EAC3C27A@gmail.com> References: <365CC8D7-DF0D-47CF-B71A-4174EAC3C27A@gmail.com> Message-ID: <4F9371EE-D9F8-4AD7-A055-C5AF0D4D007B@gmail.com> > On 11 Dec 2015, at 17:25, Max Leske wrote: > > Hi > > I?m trying to get the Fuel build set up for 5.0 but I can?t seem to connect to ftp.squeak.org. It?s available via http but not via ftp. > > Alternatively, how do you access that server for the Squeak builds? Should I use a different protocol? > > Cheers, > Max I can in fact download specific files but I can?t log in or do listings. That works for me (although it?s a bit of a pain). Cheers, Max From eliot.miranda at gmail.com Fri Dec 11 16:52:51 2015 From: eliot.miranda at gmail.com (Eliot Miranda) Date: Fri Dec 11 16:52:57 2015 Subject: [squeak-dev] Re: looking for an image file In-Reply-To: References: <566A4A6D.4080702@gmail.com> <566A4C68.5000002@gmail.com> <2DBABD81-CB1D-4B9C-B090-FF84572D9B40@gmail.com> Message-ID: <69172E5A-B322-4F8A-93B1-2B94CABB6126@gmail.com> Hi Chris, > On Dec 11, 2015, at 8:19 AM, Chris Cunningham wrote: > > Having just loaded it into 5.0 myself, the first (biggest?) problem was that there was no squeak compatibility defined for 5.0. > I suspect that if you opened the Fuel repository, though, and loaded the squeak46 compatibility package, that should work. > And then there appears to be some issue with FLLargeIdentitySet initializing it's array to 4096, but our LargeIdentityHash returning significantly larger numbers. > > Has primitive 75 changed in 5.0? Effectively yes. Spur provides a 22 bit identity hash as opposed to the (IIRC) 11 bit field in 4.6 and prior. > -cbc _,,,^..^,,,_ (phone) > >> On Fri, Dec 11, 2015 at 8:07 AM, Eliot Miranda wrote: >> Hi Robert, >> >> off the top of my head I can't see why Fuel wouldn't work in 5.0. There might be a few issues around Characters, which are immediates in 5.0, but that shouldn't be hard to fix. >> >> _,,,^..^,,,_ (phone) >> >> > On Dec 10, 2015, at 8:09 PM, Robert Withers wrote: >> > >> > Scratch that. I need to get setup with a 4.6 image so I can look into Fuel on the squeak side. Where could I find that work? >> > >> > thanks, >> > Robert >> > >> >> On 12/10/2015 11:00 PM, Robert Withers wrote: >> >> Where could I find an image file version 6505? I need to see to something first and hopefully all will be well. >> >> >> >> Thank you, >> >> Robert >> > >> > >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151211/d028e62c/attachment.htm From eliot.miranda at gmail.com Fri Dec 11 18:02:51 2015 From: eliot.miranda at gmail.com (Eliot Miranda) Date: Fri Dec 11 18:02:57 2015 Subject: Broken Update [was [squeak-dev] Daily Commit Log] In-Reply-To: References: Message-ID: <05294A00-FB41-4DAF-8F88-9128319552DD@gmail.com> Hi Karl, > On Dec 10, 2015, at 9:59 PM, karl ramberg wrote: > > Hm, > I'm a little confused/ worried because I did not change anything yet. So was I when I replied yesterday cuz I didn't see what's changed. But it is fixed. Levente did you fix it? Is that what should be read from your last message? If so, much thanks! > > Best, > Karl > >> On Fri, Dec 11, 2015 at 1:29 AM, Eliot Miranda wrote: >> Thanks Karl, >> >> my auto-update script now works perfectly again. Thanks. >> >>> On Thu, Dec 10, 2015 at 11:12 AM, karl ramberg wrote: >>> Here is a update-kfr.339.mcm with the difference from update-mt.339.mcm : >>> >>> dependency ('Squeak-Version' 'Squeak-Version-kfr.4713' '96c171dc-c340-492e-b376-30e0212b9c8e') >>> >>> and universes are not in it >>> >>>> On Thu, Dec 10, 2015 at 6:34 PM, karl ramberg wrote: >>>> >>>> >>>>> On Thu, Dec 10, 2015 at 6:01 PM, Levente Uzonyi wrote: >>>>> IMHO all you need to do is to fire up a Squeak image which has Squeak-Version-kfr.4713 loaded, open update-mt.339.mcm (open the Trunk repository, select this update map, and click Browse), remove Universes-mt.48 from the list (select, right click, remove), and update Squeak-Version to kfr.4713 (select, right click, update from image (or repository if you're brave), and finally save it (Store) using the same name (update-mt.339). >>>> >>>> I tried this, but there is no stored file from the MCConfigurationBrowser... I'm not sure what is going on, why it is not outputting a update file to my local repository. >>>> >>>> I also updated from a Squeak 5.0 image and I did not run into any problems with the update... >>>> >>>> >>>> Best, >>>> Karl >>>> >>>> >>>>> Here comes the tricky part: you can't overwrite an existing update map, nor can you move it away from the repository using the web interface, so an administrator of source.squeak.org will have to do one of these things to let you save the correct version. >>>>> Alternatively, you could try to save it using different initials but same version (e.g. update-kfr.339) and hope that the update process will sort the maps lexicographically, but I'd rather not rely on this. >>>>> >>>>> Levente >>>>> >>>>> On Thu, 10 Dec 2015, karl ramberg wrote: >>>>> >>>>>> Thanks, >>>>>> Best, >>>>>> Karl >>>>>> >>>>>> On Thu, Dec 10, 2015 at 5:00 PM, Eliot Miranda wrote: >>>>>> Hi Karl, >>>>>> On Wed, Dec 9, 2015 at 9:27 PM, karl ramberg wrote: >>>>>> Hi,It it related with this Levente mentioned in a mail earlier ? >>>>>> >>>>>> "Sadly the update process will break if your image has Universes-mt.48 in it, because it'll try to load Universes-kfr.49 after it has been removed by Squeak-Version. I think removing Universes from the update map will help." >>>>>> >>>>>> I asked how to proceed but I got no answer. >>>>>> I'm not sure how one make a update map >>>>>> >>>>>> >>>>>> As Chris says the updates live in the "update" package. Open up a repository browser on source.squeak.org/trunk and you'll see them towards the bottom (update.spur contains older updates from the time of the bootstrap from 4.6 to what became 5.0, you can ignore >>>>>> them). >>>>>> >>>>>> What I would do is create a local mirror, either by fully populating my package cache with all the 5.0 packages and updates, or creating a directory repository and downloading all the packages and updates to there and then play with the local repository until I >>>>>> got the update working again, and then I would upload the changed updates to trunk. >>>>>> >>>>>> I'm attaching SpurBootstrapMonticelloPackagePatcher which is the hack I used to automate patching 4.6 packages to make them Spur compatible in the run-up to 5.0. I'm doing this because it contains methods like downloadToSourceDirAllPackageVersionsStartingWith: >>>>>> that download all versions of a package to a local repository starting from a specific version. Sao these hacks will help you in populating your local repository prior to experimenting. Once I'd downloaded all packages I would then take a local copy and modify >>>>>> the copy, I could then diff the original and the modified clone to see what had changed. >>>>>> >>>>>> HTH >>>>>> >>>>>> _,,,^..^,,,_ >>>>>> best, Eliot >>>>>> >>>>>> Best, >>>>>> Karl >>>>>> [cleardot.gif] >>>>>> >>>>>> On Thu, Dec 10, 2015 at 4:06 AM, Eliot Miranda wrote: >>>>>> Hi, >>>>>> whatever happened in unloading Universes, it broke the ability to update a release Squeak 5.0 image to the current tip. Hence my automated scripts to build 64-bit images are broken. Can whoever broke the update either fix it, or provide some >>>>>> reasonable workaround so that I can restore my scripts? What's at stake here is, well, a high performance 64-bit Squeak and/or Pharo. >>>>>> >>>>>> I think it's reasonable, even mandatory that we should be able to update a release N image to the tip of release N. We should have CI servers that try and do this (I think we do, don't we?), and so IMO it's incumbent on whoever breaks the update to >>>>>> fix things, e.g. by changing update maps appropriately. [and I don't mean this as a punishment, or something that the person has to do on their own; I'm happy to help fix things, it's /really/ important that the update works, and not just from the >>>>>> last update]. >>>>>> >>>>>> Thanks in advance. Apologies if I appear rude; it's not my intent. I just want my auto-update back ;-) >>>>>> >>>>>> On Thu, Nov 26, 2015 at 2:55 PM, wrote: >>>>>> Changes to Trunk (http://source.squeak.org/trunk.html) in the last 24 hours: >>>>>> >>>>>> http://lists.squeakfoundation.org/pipermail/packages/2015-November/009199.html >>>>>> >>>>>> Name: Squeak-Version-kfr.4713 >>>>>> Ancestors: Squeak-Version-kfr.4712 >>>>>> >>>>>> Unload and unregister Universes >>>>>> >>>>>> ============================================= >>>>>> >>>>>> >>>>>> _,,,^..^,,,_ >>>>>> best, Eliot >> >> >> >> -- >> _,,,^..^,,,_ >> best, Eliot > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151211/5cbe7040/attachment.htm From colin at wiresong.com Fri Dec 11 19:05:35 2015 From: colin at wiresong.com (Colin Putney) Date: Fri Dec 11 19:05:38 2015 Subject: [squeak-dev] Re: [Pharo-dev] Unicode Support In-Reply-To: <7CC2EE2F-676A-4183-99DD-A27C6480FC64@gmail.com> References: <5662E1D8.1070802@free.fr> <96B16DCE-A9E7-49E6-B866-E5A846801AA0@stfx.eu> <0F104967-2F55-431F-9EED-0275ABC7CEC1@veloxit.no> <52F49745-64AA-447A-A75A-71FB86FCE36E@veloxit.no> <434B72EE-528B-4ECC-9DCE-78A8F7A6E43E@gmail.com> <7CC2EE2F-676A-4183-99DD-A27C6480FC64@gmail.com> Message-ID: On Fri, Dec 11, 2015 at 1:29 AM, Eliot Miranda wrote: > > For my part, the reason I think we ought to implement it *in* > > Smalltalk is ... this is the String class of the new age. I want > > Smalltalk to be handle Strings as native objects. > > There's little if any difference in convenience of use between an Array of > characters and a bits array with the string at:/at:put: primitives since > both require at:/at:put: to access, but the latter is (efficiently) type > checked (by the VM), whereas there's nothing to prevent storing other than > characters in the Areay unless one introduces the overhead of skier > explicit type checks in Smalltalk, and the Areay starts life as a sequence > of nils (invalid until every element is set to a character) whereas the > bits representation begins fully initialized with 0 asCharacter. So > there's nothing more "natively objecty" about the Array. Smalltalk objects > hide their representation from clients and externally they behave the same, > except for space and time. > I think Euan was referring to the Gemstone strategy of storing the string content as bits, then calling into ICU (a C++ library for Unicode processing) to manipulate them. So he's saying sure, store the string data as bits, but write Smalltalk code to sort them, render on screen etc. Colin -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151211/32660794/attachment.htm From robert.w.withers at gmail.com Fri Dec 11 19:11:02 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Fri Dec 11 19:11:07 2015 Subject: [squeak-dev] Re: [Vm-dev] Re: looking for an image file In-Reply-To: <20151211142006.GA57925@shell.msen.com> References: <566A4A6D.4080702@gmail.com> <566A4C68.5000002@gmail.com> <566A4EC8.9000805@gmail.com> <20151211142006.GA57925@shell.msen.com> Message-ID: <566B1FC6.9010103@gmail.com> Thanks, Dave, I am starting to read about ImageFormats. The confusing part to me is the cog/spur vm I built asks fro image format 6505, while my image is 6521, but the ImageFormat descriptions list 6521 as a cog/spur image. Robert On 12/11/2015 09:20 AM, David T. Lewis wrote: > On Thu, Dec 10, 2015 at 11:19:20PM -0500, Robert Withers wrote: >> I do need to hit up 4.6. But also, the newest interpreter is for image >> version 6505 and the updated image is 6521. I am not sure howe to proceed.. > For an explanation of the various image formats, load package ImageFormat > from SqueakMap. Read the class comment first, then look at the class > side utility methods. Examples: > > ImageFormat versionDescriptions do: [:e | Transcript cr; show: e] > > (ImageFormat fromFile: Smalltalk imageName) description > > Loosely speaking, Squeak 4.6 is image format 6405, and works with Cog > and interpreter VMs. Squeak 5.0 and trunk are image format 6521, and > work with Spur. > > Dave > > > >> On 12/10/2015 11:09 PM, Robert Withers wrote: >>> Scratch that. I need to get setup with a 4.6 image so I can look into >>> Fuel on the squeak side. Where could I find that work? >>> >>> thanks, >>> Robert >>> >>> On 12/10/2015 11:00 PM, Robert Withers wrote: >>>> Where could I find an image file version 6505? I need to see to >>>> something first and hopefully all will be well. >>>> >>>> Thank you, >>>> Robert From robert.w.withers at gmail.com Fri Dec 11 19:13:31 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Fri Dec 11 19:13:35 2015 Subject: [squeak-dev] Re: looking for an image file In-Reply-To: <2DBABD81-CB1D-4B9C-B090-FF84572D9B40@gmail.com> References: <566A4A6D.4080702@gmail.com> <566A4C68.5000002@gmail.com> <2DBABD81-CB1D-4B9C-B090-FF84572D9B40@gmail.com> Message-ID: <566B205B.8050209@gmail.com> Thanks Eliot. I'm slowly heading toward Fuel. I just got a new 64-bit laptop I am configuring with 64-bit Ubuntu and trying to resolve those issues to build the vm. Robert On 12/11/2015 11:07 AM, Eliot Miranda wrote: > Hi Robert, > > off the top of my head I can't see why Fuel wouldn't work in 5.0. There might be a few issues around Characters, which are immediates in 5.0, but that shouldn't be hard to fix. > > _,,,^..^,,,_ (phone) > >> On Dec 10, 2015, at 8:09 PM, Robert Withers wrote: >> >> Scratch that. I need to get setup with a 4.6 image so I can look into Fuel on the squeak side. Where could I find that work? >> >> thanks, >> Robert >> >>> On 12/10/2015 11:00 PM, Robert Withers wrote: >>> Where could I find an image file version 6505? I need to see to something first and hopefully all will be well. >>> >>> Thank you, >>> Robert >> From eliot.miranda at gmail.com Fri Dec 11 19:25:42 2015 From: eliot.miranda at gmail.com (Eliot Miranda) Date: Fri Dec 11 19:25:44 2015 Subject: [squeak-dev] Re: [Vm-dev] Re: looking for an image file In-Reply-To: <566B1FC6.9010103@gmail.com> References: <566A4A6D.4080702@gmail.com> <566A4C68.5000002@gmail.com> <566A4EC8.9000805@gmail.com> <20151211142006.GA57925@shell.msen.com> <566B1FC6.9010103@gmail.com> Message-ID: Hi Rob, On Fri, Dec 11, 2015 at 11:11 AM, Robert Withers wrote: > Thanks, Dave, I am starting to read about ImageFormats. The confusing part > to me is the cog/spur vm I built asks fro image format 6505, while my image > is 6521, but the ImageFormat descriptions list 6521 as a cog/spur image. I expect you haven't built a Spur VM, but have built a standard v3 Cog VM. What platform are you on and which directory did you build in? > > > Robert > > > > On 12/11/2015 09:20 AM, David T. Lewis wrote: > >> On Thu, Dec 10, 2015 at 11:19:20PM -0500, Robert Withers wrote: >> >>> I do need to hit up 4.6. But also, the newest interpreter is for image >>> version 6505 and the updated image is 6521. I am not sure howe to >>> proceed.. >>> >> For an explanation of the various image formats, load package ImageFormat >> from SqueakMap. Read the class comment first, then look at the class >> side utility methods. Examples: >> >> ImageFormat versionDescriptions do: [:e | Transcript cr; show: e] >> >> (ImageFormat fromFile: Smalltalk imageName) description >> >> Loosely speaking, Squeak 4.6 is image format 6405, and works with Cog >> and interpreter VMs. Squeak 5.0 and trunk are image format 6521, and >> work with Spur. >> >> Dave >> >> >> >> On 12/10/2015 11:09 PM, Robert Withers wrote: >>> >>>> Scratch that. I need to get setup with a 4.6 image so I can look into >>>> Fuel on the squeak side. Where could I find that work? >>>> >>>> thanks, >>>> Robert >>>> >>>> On 12/10/2015 11:00 PM, Robert Withers wrote: >>>> >>>>> Where could I find an image file version 6505? I need to see to >>>>> something first and hopefully all will be well. >>>>> >>>>> Thank you, >>>>> Robert >>>>> >>>> > > -- _,,,^..^,,,_ best, Eliot -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151211/2e84a317/attachment.htm From robert.w.withers at gmail.com Fri Dec 11 19:27:07 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Fri Dec 11 19:27:13 2015 Subject: [squeak-dev] loading Ubuntu 64-bit Message-ID: <566B238B.6010807@gmail.com> Hi, I just got a new 64-bit laptop I am configuring with 64-bit Ubuntu and trying to resolve those issues to build the vm. I ran the apt-get for everything I was aware of, but I get an error when running buildspurtrunkvmmakermage.sh: cogspurlinuxht/squeak trunk50.image UpdateSqueakTrunkImage.st squeak: could not find any display driver ./updatespurimage.sh: line 14: 27399 Aborted (core dumped) $VM trunk50.image UpdateSqueakTrunkImage.st cogspurlinuxht/squeak SpurVMMaker.image BuildSqueakSpurTrunkVMMakerImage.st squeak: could not find any display driver ./buildspurtrunkvmmakerimage.sh: line 12: 27431 Aborted (core dumped) $VM SpurVMMaker.image BuildSqueakSpurTrunkVMMakerImage.st For some reason it isn't finding the vm-display-X11 and I do not recall why from my prior install. Does this ring a bell with anyone who could assist me? Thank you, Robert ps. all the apt-get commands I ran: sudo apt-get install cmake zip bash-completion ruby git xz-utils debhelper devscripts sudo apt-get install libc6-dev:i386 libasound2:i386 libasound2-dev:i386 libasound2-plugins:i386 libssl-dev:i386 i#!libssl0.9.8:i386 libfreetype6-dev:i386 libx11-dev:i386 libsm-dev:i386 libice-dev:i386 sudo apt-get install build-essential gcc-multilib g++ sudo apt-get install libgl1-mesa-dev libgl1-mesa-glx:i386 sudo apt-get install binutils:i386 g++-4.6:i386 gcc-4.6:i386 sudo ln -s /usr/lib/i386-linux-gnu/mesa/libGL.so /usr/lib/i386-linux-gnu/libGL.so sudo ln -s /usr/lib/i386-linux-gnu/mesa/libGL.so.1 /usr/lib/i386-linux-gnu/mesa/libGL.so sudo apt-get install uuid-dev:i386 sudo apt-get install subversion From eliot.miranda at gmail.com Fri Dec 11 19:27:57 2015 From: eliot.miranda at gmail.com (Eliot Miranda) Date: Fri Dec 11 19:28:00 2015 Subject: [squeak-dev] Re: [Pharo-dev] Unicode Support In-Reply-To: References: <5662E1D8.1070802@free.fr> <96B16DCE-A9E7-49E6-B866-E5A846801AA0@stfx.eu> <0F104967-2F55-431F-9EED-0275ABC7CEC1@veloxit.no> <52F49745-64AA-447A-A75A-71FB86FCE36E@veloxit.no> <434B72EE-528B-4ECC-9DCE-78A8F7A6E43E@gmail.com> <7CC2EE2F-676A-4183-99DD-A27C6480FC64@gmail.com> Message-ID: On Fri, Dec 11, 2015 at 11:05 AM, Colin Putney wrote: > > > On Fri, Dec 11, 2015 at 1:29 AM, Eliot Miranda > wrote: > >> > For my part, the reason I think we ought to implement it *in* >> > Smalltalk is ... this is the String class of the new age. I want >> > Smalltalk to be handle Strings as native objects. >> >> There's little if any difference in convenience of use between an Array >> of characters and a bits array with the string at:/at:put: primitives since >> both require at:/at:put: to access, but the latter is (efficiently) type >> checked (by the VM), whereas there's nothing to prevent storing other than >> characters in the Areay unless one introduces the overhead of skier >> explicit type checks in Smalltalk, and the Areay starts life as a sequence >> of nils (invalid until every element is set to a character) whereas the >> bits representation begins fully initialized with 0 asCharacter. So >> there's nothing more "natively objecty" about the Array. Smalltalk objects >> hide their representation from clients and externally they behave the same, >> except for space and time. >> > > I think Euan was referring to the Gemstone strategy of storing the string > content as bits, then calling into ICU (a C++ library for Unicode > processing) to manipulate them. So he's saying sure, store the string data > as bits, but write Smalltalk code to sort them, render on screen etc. > Ah, thanks for the clarification. Unwise of me to read complex emails on the phone :-). Letter box problem. I agree. ICU should be steered clear of at all costs. > Colin > > > > > > -- _,,,^..^,,,_ best, Eliot -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151211/8b17600a/attachment.htm From eliot.miranda at gmail.com Fri Dec 11 19:29:51 2015 From: eliot.miranda at gmail.com (Eliot Miranda) Date: Fri Dec 11 19:29:54 2015 Subject: [squeak-dev] loading Ubuntu 64-bit In-Reply-To: <566B238B.6010807@gmail.com> References: <566B238B.6010807@gmail.com> Message-ID: Hi Robert, if you run ldd on the vm-display-X11 library it should list the libraries it expects. If any are missing it won't load. On Fri, Dec 11, 2015 at 11:27 AM, Robert Withers wrote: > Hi, > > I just got a new 64-bit laptop I am configuring with 64-bit Ubuntu and > trying to resolve those issues to build the vm. I ran the apt-get for > everything I was aware of, but I get an error when running > buildspurtrunkvmmakermage.sh: > > cogspurlinuxht/squeak trunk50.image UpdateSqueakTrunkImage.st > squeak: could not find any display driver > ./updatespurimage.sh: line 14: 27399 Aborted (core dumped) > $VM trunk50.image UpdateSqueakTrunkImage.st > cogspurlinuxht/squeak SpurVMMaker.image BuildSqueakSpurTrunkVMMakerIma > ge.st > squeak: could not find any display driver > ./buildspurtrunkvmmakerimage.sh: line 12: 27431 Aborted > (core dumped) $VM SpurVMMaker.image BuildSqueakSpurTrunkVMMakerImage.st > > For some reason it isn't finding the vm-display-X11 and I do not recall > why from my prior install. Does this ring a bell with anyone who could > assist me? > > Thank you, > Robert > > > ps. all the apt-get commands I ran: > > sudo apt-get install cmake zip bash-completion ruby git xz-utils debhelper > devscripts > sudo apt-get install libc6-dev:i386 libasound2:i386 libasound2-dev:i386 > libasound2-plugins:i386 libssl-dev:i386 i#!libssl0.9.8:i386 > libfreetype6-dev:i386 libx11-dev:i386 libsm-dev:i386 libice-dev:i386 > sudo apt-get install build-essential gcc-multilib g++ > sudo apt-get install libgl1-mesa-dev libgl1-mesa-glx:i386 > sudo apt-get install binutils:i386 g++-4.6:i386 gcc-4.6:i386 > sudo ln -s /usr/lib/i386-linux-gnu/mesa/libGL.so > /usr/lib/i386-linux-gnu/libGL.so > sudo ln -s /usr/lib/i386-linux-gnu/mesa/libGL.so.1 > /usr/lib/i386-linux-gnu/mesa/libGL.so > sudo apt-get install uuid-dev:i386 > sudo apt-get install subversion > > -- _,,,^..^,,,_ best, Eliot -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151211/77e2293b/attachment.htm From robert.w.withers at gmail.com Fri Dec 11 19:53:15 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Fri Dec 11 19:53:20 2015 Subject: [squeak-dev] loading Ubuntu 64-bit In-Reply-To: References: <566B238B.6010807@gmail.com> Message-ID: <566B29AB.5000300@gmail.com> Thanks Eliot! ldd said libSM.so.6 and libICE.so.6 are missing. It turns out it is the i386 32-bit version it wants. So I apt-get install libsm6:i386 libice6:i386. I am now underway building the vmmaker image. Robert On 12/11/2015 02:29 PM, Eliot Miranda wrote: > Hi Robert, > > if you run ldd on the vm-display-X11 library it should list the > libraries it expects. If any are missing it won't load. > > On Fri, Dec 11, 2015 at 11:27 AM, Robert Withers > > wrote: > > Hi, > > I just got a new 64-bit laptop I am configuring with 64-bit Ubuntu > and trying to resolve those issues to build the vm. I ran the > apt-get for everything I was aware of, but I get an error when > running buildspurtrunkvmmakermage.sh: > > cogspurlinuxht/squeak trunk50.image UpdateSqueakTrunkImage.st > squeak: could not find any display driver > ./updatespurimage.sh: line 14: 27399 Aborted (core dumped) $VM > trunk50.image UpdateSqueakTrunkImage.st > cogspurlinuxht/squeak SpurVMMaker.image > BuildSqueakSpurTrunkVMMakerImage.st > squeak: could not find any display driver > ./buildspurtrunkvmmakerimage.sh: line 12: 27431 Aborted > (core dumped) $VM SpurVMMaker.image > BuildSqueakSpurTrunkVMMakerImage.st > > For some reason it isn't finding the vm-display-X11 and I do not > recall why from my prior install. Does this ring a bell with > anyone who could assist me? > > Thank you, > Robert > > > ps. all the apt-get commands I ran: > > sudo apt-get install cmake zip bash-completion ruby git xz-utils > debhelper devscripts > sudo apt-get install libc6-dev:i386 libasound2:i386 > libasound2-dev:i386 libasound2-plugins:i386 libssl-dev:i386 > i#!libssl0.9.8:i386 libfreetype6-dev:i386 libx11-dev:i386 > libsm-dev:i386 libice-dev:i386 > sudo apt-get install build-essential gcc-multilib g++ > sudo apt-get install libgl1-mesa-dev libgl1-mesa-glx:i386 > sudo apt-get install binutils:i386 g++-4.6:i386 gcc-4.6:i386 > sudo ln -s /usr/lib/i386-linux-gnu/mesa/libGL.so > /usr/lib/i386-linux-gnu/libGL.so > sudo ln -s /usr/lib/i386-linux-gnu/mesa/libGL.so.1 > /usr/lib/i386-linux-gnu/mesa/libGL.so > sudo apt-get install uuid-dev:i386 > sudo apt-get install subversion > > > > > -- > _,,,^..^,,,_ > best, Eliot > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151211/94be0d59/attachment.htm From frank-lesser at lesser-software.com Fri Dec 11 19:55:24 2015 From: frank-lesser at lesser-software.com (Frank Lesser) Date: Fri Dec 11 19:55:32 2015 Subject: [squeak-dev] bitshift: In-Reply-To: References: <566B238B.6010807@gmail.com> Message-ID: <6E15F02ADD7542FF99002AFF9C58723B@LSWWin764NB> ( -16r100000001 bitShift: -32 ) ( -16r100000000 bitShift: -32 ) = -2 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151211/9dcb8eb6/attachment.htm From robert.w.withers at gmail.com Fri Dec 11 20:00:19 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Fri Dec 11 20:00:26 2015 Subject: [squeak-dev] Re: [Vm-dev] Re: looking for an image file In-Reply-To: References: <566A4A6D.4080702@gmail.com> <566A4C68.5000002@gmail.com> <566A4EC8.9000805@gmail.com> <20151211142006.GA57925@shell.msen.com> <566B1FC6.9010103@gmail.com> Message-ID: <566B2B53.8030702@gmail.com> Yes Eliot, I generated the Squeak Cog Spur VM (to /spursrc) and then built in 'cogvm/build.linux32x86/squeak.cog.spur/build'. Before I installed it and ran squeak SpurVMMaker.image out of image and it gave me this warning that the vm was 6505 and the image 6521. Now I am trying to rebuild after updating the cogvm source and the SpurVMMaker image and it is generating this when compiling: /home/rabbit/warren/cogvm/spursrc/vm/cogitIA32.c:1095:28: warning: ?picAbortTrampolineFor? used but never defined [enabled by default] static sqInt NoDbgRegParms picAbortTrampolineFor(sqInt numArgs); ^ make[1]: *** [cogit.o] Error 1 make: *** [vm/vm.a] Error 2 I don't quite yet grasp all the approaches. Robert On 12/11/2015 02:25 PM, Eliot Miranda wrote: > Hi Rob, > > On Fri, Dec 11, 2015 at 11:11 AM, Robert Withers > > wrote: > > Thanks, Dave, I am starting to read about ImageFormats. The > confusing part to me is the cog/spur vm I built asks fro image > format 6505, while my image is 6521, but the ImageFormat > descriptions list 6521 as a cog/spur image. > > > I expect you haven't built a Spur VM, but have built a standard v3 Cog > VM. What platform are you on and which directory did you build in? > > > > Robert > > > > On 12/11/2015 09:20 AM, David T. Lewis wrote: > > On Thu, Dec 10, 2015 at 11:19:20PM -0500, Robert Withers wrote: > > I do need to hit up 4.6. But also, the newest interpreter > is for image > version 6505 and the updated image is 6521. I am not sure > howe to proceed.. > > For an explanation of the various image formats, load package > ImageFormat > from SqueakMap. Read the class comment first, then look at the > class > side utility methods. Examples: > > ImageFormat versionDescriptions do: [:e | Transcript cr; > show: e] > > (ImageFormat fromFile: Smalltalk imageName) description > > Loosely speaking, Squeak 4.6 is image format 6405, and works > with Cog > and interpreter VMs. Squeak 5.0 and trunk are image format > 6521, and > work with Spur. > > Dave > > > > On 12/10/2015 11:09 PM, Robert Withers wrote: > > Scratch that. I need to get setup with a 4.6 image so > I can look into > Fuel on the squeak side. Where could I find that work? > > thanks, > Robert > > On 12/10/2015 11:00 PM, Robert Withers wrote: > > Where could I find an image file version 6505? I > need to see to > something first and hopefully all will be well. > > Thank you, > Robert > > > > > > > -- > _,,,^..^,,,_ > best, Eliot > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151211/027f15e4/attachment-0001.htm From rpboland at gmail.com Fri Dec 11 20:00:34 2015 From: rpboland at gmail.com (Ralph Boland) Date: Fri Dec 11 20:00:37 2015 Subject: [squeak-dev] Recovering method changes after crash Message-ID: I recently upgraded from Squeak 3.11 to Squeak 4.6 Overall it seems better and particular much faster. However, I do have a problem. In Squeak 3.11 whenever my image crashed for any reason (The most common was I got into an infinite loop I couldn't kill from within Squeak and had to do so from the Linux command line.) I could recover the changes not in the must recent save of my image by restarting squeak on that image and then opening changes>>recently logged changes and selecting the methods I wanted reloaded into my image and then reloading them. But in Squeak 4.6 there seems to be no way to do this. In Squeak 4.6, after a crash of the image, how do I recover the (possibly many) methods/changes in the changes file but not in the image? If there is no easy way to do this in 4.6 then this is a major disadvantage over 3.11 to my mind. I assume I am actually missing something simple here so apologies for being slow here. Sincerely, Ralph Boland From frank-lesser at lesser-software.com Fri Dec 11 20:00:58 2015 From: frank-lesser at lesser-software.com (Frank Lesser) Date: Fri Dec 11 20:01:07 2015 Subject: [squeak-dev] bitshift: In-Reply-To: <6E15F02ADD7542FF99002AFF9C58723B@LSWWin764NB> References: <566B238B.6010807@gmail.com> <6E15F02ADD7542FF99002AFF9C58723B@LSWWin764NB> Message-ID: <92A7EEEA2A8A475C9735EFEEB13157CE@LSWWin764NB> Hi, ( -16r100000001 bitShift: -32 ) = -2 ( -16r100000000 bitShift: -32 ) = -1 looks odd to me, guess the prim needs to be fixed Frank -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151211/0ebdbb25/attachment.htm From tim.olson.mail at gmail.com Fri Dec 11 20:16:23 2015 From: tim.olson.mail at gmail.com (Tim Olson) Date: Fri Dec 11 20:16:30 2015 Subject: [squeak-dev] bitshift: In-Reply-To: <92A7EEEA2A8A475C9735EFEEB13157CE@LSWWin764NB> References: <566B238B.6010807@gmail.com> <6E15F02ADD7542FF99002AFF9C58723B@LSWWin764NB> <92A7EEEA2A8A475C9735EFEEB13157CE@LSWWin764NB> Message-ID: <6BFA1764-1200-48DA-8898-5C22B0A2F53E@gmail.com> On Dec 11, 2015, at 2:00 PM, Frank Lesser wrote: > Hi, > > ( -16r100000001 bitShift: -32 ) = -2 > ( -16r100000000 bitShift: -32 ) = -1 > > looks odd to me, > > guess the prim needs to be fixed Looks correct to me: (-16r100000001 bitAnd: 16rFFFFFFFFFF) hex -> ?16rFEFFFFFFFF? downshift this by 32 bits and you get 16rFFFFFFFE, which is -2 (-16r100000000 bitAnd: 16rFFFFFFFFFF) hex -> ?16rFF00000000' downshift this by 32 bits and you get 16rFFFFFFFF, which is -1. ? tim -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151211/cbb59397/attachment.htm From euanmee at gmail.com Fri Dec 11 20:19:13 2015 From: euanmee at gmail.com (EuanM) Date: Fri Dec 11 20:19:16 2015 Subject: [squeak-dev] Re: [Pharo-dev] Unicode Support In-Reply-To: <1449848269163-4866610.post@n4.nabble.com> References: <5662E1D8.1070802@free.fr> <1449848269163-4866610.post@n4.nabble.com> Message-ID: "If it hasn't already been said, please do not conflate Unicode and UTF-8. I think that would be a recipe for a high P.I.T.A. factor." --Richard Sargent I agree. :-) Regarding UTF-16, I just want to be able to export to, and receive from, Windows (and any other platforms using UTF-16 as their native character representation). Windows will always be able to accept UTF-16. All Windows apps *might well* export UTF-16. There may be other platforms which use UTF-16 as their native format. I'd just like to be able to cope with those situations. Nothing more. All this is requires is a Utf16String class that has an asUtf8String method (and any other required conversion methods). And other string classes to have asUtf16String classes. Once we have the other classes and methods, this should be a trivial extensions. Export will just be transformations of existing formats of valid strings. Import just needs to transform to (one of) our preferred format(s), and have a validity check performed after the transform is complete. On 11 December 2015 at 15:37, Richard Sargent wrote: > EuanM wrote >> ... >> all ISO-8859-1 maps 1:1 to Unicode UTF-8 >> ... > > I am late coming in to this conversation. If it hasn't already been said, > please do not conflate Unicode and UTF-8. I think that would be a recipe for > a high P.I.T.A. factor. > > Unicode defines the meaning of the code points. > UTF-8 (and -16) define an interchange mechanism. > > In other words, when you write the code points to an external medium > (socket, file, whatever), encode them via UTF-whatever. Read UTF-whatever > from an external medium and re-instantiate the code points. > (Personally, I see no use for UTF-16 as an interchange mechanism. Others may > have justification for it. I don't.) > > Having characters be a consistent size in their object representation makes > everything easier. #at:, #indexOf:, #includes: ... no one wants to be > scanning through bytes representing variable sized characters. > > Model Unicode strings using classes such as e.g. Unicode7, Unicode16, and > Unicode32, with automatic coercion to the larger character width. > > > > > -- > View this message in context: http://forum.world.st/Unicode-Support-tp4865139p4866610.html > Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com. > From euanmee at gmail.com Fri Dec 11 20:20:21 2015 From: euanmee at gmail.com (EuanM) Date: Fri Dec 11 20:20:23 2015 Subject: [Pharo-dev] [squeak-dev] Re: Unicode Support In-Reply-To: <7CC2EE2F-676A-4183-99DD-A27C6480FC64@gmail.com> References: <5662E1D8.1070802@free.fr> <96B16DCE-A9E7-49E6-B866-E5A846801AA0@stfx.eu> <0F104967-2F55-431F-9EED-0275ABC7CEC1@veloxit.no> <52F49745-64AA-447A-A75A-71FB86FCE36E@veloxit.no> <434B72EE-528B-4ECC-9DCE-78A8F7A6E43E@gmail.com> <7CC2EE2F-676A-4183-99DD-A27C6480FC64@gmail.com> Message-ID: Eliot - thank you for explaining to me why my original idea was bad. :-) I always assumed it would be. Otherwise I'd've just built it the way I proposed. I'm thoroughly delighted to have more knowledgeable people contributing. On 11 December 2015 at 09:29, Eliot Miranda wrote: > Hi Euan, > >> On Dec 10, 2015, at 6:43 PM, EuanM wrote: >> >> I agree with all of that, Ben. >> >> I'm currently fairly certain that fully-composed abstract characters >> is a term that is 1:1 mapped with the term "grapheme cluster" (i.e. >> one is an older Unicode description of a newer Unicode term). >> >> And once we create these, I think this sort of implementation is >> straightforward. For particular values of "straightforward", of >> course :-) >> >> i.e. the Swift approach is equivalent to the approach I originally >> proposed and asked for critiques of. >> >> One thing I don't understand.... why does the fact the composed >> abstract character (aka grapheme cluster) is a sequence mean that an >> array cannot be used to hold the sequence? > > Of course an Array can be used, but one good reason to use bits organized as four-byte units is that the garbage collector spends no time scanning them, whereas as far as its concerned the Array representation is all objects and must be scanned. Another reason is that foreign code may find the bits representation compatible and so they can be passed through the FFI to other languages whereas the Array of tagged characters will always require conversion. Yet another reason is that in 64-bits the Array takes twice the space of the bits object. > >> If people then also want a compatibility-codepoints-only UTF-8 >> representation, it is simple to provide comparable (i.e >> equivalence-testable) versions of any UTF-8 string - because we are >> creating them from composed forms by a *single* defined method. >> >> For my part, the reason I think we ought to implement it *in* >> Smalltalk is ... this is the String class of the new age. I want >> Smalltalk to be handle Strings as native objects. > > There's little if any difference in convenience of use between an Array of characters and a bits array with the string at:/at:put: primitives since both require at:/at:put: to access, but the latter is (efficiently) type checked (by the VM), whereas there's nothing to prevent storing other than characters in the Areay unless one introduces the overhead of skier explicit type checks in Smalltalk, and the Areay starts life as a sequence of nils (invalid until every element is set to a character) whereas the bits representation begins fully initialized with 0 asCharacter. So there's nothing more "natively objecty" about the Array. Smalltalk objects hide their representation from clients and externally they behave the same, except for space and time. > > Given that this is a dynamically-typed language there's nothing to prevent one providing both implementations beyond maintenance cost and complexity/confusion. So at least it's easy to do performance comparisons between the two. But I still think the bits representation is superior if what you want is a sequence of Characters. > >>> On 10 December 2015 at 23:41, Ben Coman wrote: >>> On Wed, Dec 9, 2015 at 5:35 PM, Guillermo Polito >>> wrote: >>>> >>>>> On 8 dic 2015, at 10:07 p.m., EuanM wrote: >>>>> >>>>> "No. a codepoint is the numerical value assigned to a character. An >>>>> "encoded character" is the way a codepoint is represented in bytes >>>>> using a given encoding." >>>>> >>>>> No. >>>>> >>>>> A codepoint may represent a component part of an abstract character, >>>>> or may represent an abstract character, or it may do both (but not >>>>> always at the same time). >>>>> >>>>> Codepoints represent a single encoding of a single concept. >>>>> >>>>> Sometimes that concept represents a whole abstract character. >>>>> Sometimes it represent part of an abstract character. >>>> >>>> Well. I do not agree with this. I agree with the quote. >>>> >>>> Can you explain a bit more about what you mean by abstract character and concept? >>> >>> This seems to be what Swift is doing, where Strings are not composed >>> not of codepoints but of graphemes. >>> >>>>>> "Every instance of Swift?s Character type represents a single extended grapheme cluster. An extended grapheme cluster is a sequence** of one or more Unicode scalars that (when combined) produce a single human-readable character. [1] >>> >>> ** i.e. not an array >>> >>>>>> Here?s an example. The letter ? can be represented as the single Unicode scalar ? (LATIN SMALL LETTER E WITH ACUTE, or U+00E9). However, the same letter can also be represented as a pair of scalars?a standard letter e (LATIN SMALL LETTER E, or U+0065), followed by the COMBINING ACUTE ACCENT scalar (U+0301). TheCOMBINING ACUTE ACCENT scalar is graphically applied to the scalar that precedes it, turning an e into an ?when it is rendered by a Unicode-aware text-rendering system. [1] >>> >>>>>> In both cases, the letter ? is represented as a single Swift Character value that represents an extended grapheme cluster. In the first case, the cluster contains a single scalar; in the second case, it is a cluster of two scalars:" [1] >>> >>>>>> Swift?s string implemenation makes working with Unicode easier and significantly less error-prone. As a programmer, you still have to be aware of possible edge cases, but this probably cannot be avoided completely considering the characteristics of Unicode. [2] >>> >>> Indeed I've tried searched for what problems it causes and get a null >>> result. So I read *all*good* things about Swift's unicode >>> implementation reducing common errors dealing with Unicode. Can >>> anyone point to complaints about Swift's unicode implementation? >>> Maybe this... >>> >>>>>> An argument could be made that the implementation of String as a sequence that requires iterating over characters from the beginning of the string for many operations poses a significant performance problem but I do not think so. My guess is that Apple?s engineers have considered the implications of their implementation and apps that do not deal with enormous amounts of text will be fine. Moreover, the idea that you could get away with an implementation that supports random access of characters is an illusion given the complexity of Unicode. [2] >>> >>> Considering our common pattern: Make it work, Make it right, Make it >>> fast -- maybe Strings as arrays are a premature optimisation, that >>> was the right choice in the past prior to Unicode, but considering >>> Moore's Law versus programmer time, is not the best choice now. >>> Should we at least start with a UnicodeString and UnicodeCharacter >>> that operates like Swift, and over time *maybe* move the tools to use >>> them. >>> >>> [1] https://developer.apple.com/library/ios/documentation/Swift/Conceptual/Swift_Programming_Language/StringsAndCharacters.html >>> [2] http://oleb.net/blog/2014/07/swift-strings/ >>> >>> cheers -ben >>> >>>> >>>>> >>>>> This is the key difference between Unicode and most character encodings. >>>>> >>>>> A codepoint does not always represent a whole character. >>>>> >>>>> On 7 December 2015 at 13:06, Henrik Johansen > > _,,,^..^,,,_ (phone) From jakob.reschke at student.hpi.de Fri Dec 11 20:58:37 2015 From: jakob.reschke at student.hpi.de (Jakob Reschke) Date: Fri Dec 11 20:59:00 2015 Subject: [squeak-dev] Recovering method changes after crash In-Reply-To: <5d20c7a3df6b409ca72857a23f0adb81@MX2015-DAG2.hpi.uni-potsdam.de> References: <5d20c7a3df6b409ca72857a23f0adb81@MX2015-DAG2.hpi.uni-potsdam.de> Message-ID: Hi Ralph, I think what you are looking for is "Extras > Recover Changes" in the world main docking bar. But the path you mentioned with "World Menu > changes > recently logged changes" does exist as well in my 4.6 images, so I wonder why it was not there for you. Best regards, Jakob 2015-12-11 21:00 GMT+01:00 Ralph Boland : > I recently upgraded from Squeak 3.11 to Squeak 4.6 > Overall it seems better and particular much faster. > However, I do have a problem. > In Squeak 3.11 whenever my image crashed for any reason > (The most common was I got into an infinite loop I couldn't kill > from within Squeak and had to do so from the Linux command line.) > I could recover the changes not in the must recent save of my image > by restarting squeak on that image and then > opening changes>>recently logged changes and selecting the methods > I wanted reloaded into my image and then reloading them. > > But in Squeak 4.6 there seems to be no way to do this. > In Squeak 4.6, after a crash of the image, how do I recover the > (possibly many) methods/changes > in the changes file but not in the image? > If there is no easy way to do this in 4.6 then this is a major > disadvantage over 3.11 to my mind. > I assume I am actually missing something simple here so apologies for > being slow here. > > Sincerely, > > Ralph Boland > From frank-lesser at lesser-software.com Fri Dec 11 21:22:01 2015 From: frank-lesser at lesser-software.com (Frank Lesser) Date: Fri Dec 11 21:22:09 2015 Subject: AW: [squeak-dev] bitshift: In-Reply-To: <6BFA1764-1200-48DA-8898-5C22B0A2F53E@gmail.com> References: <566B238B.6010807@gmail.com><6E15F02ADD7542FF99002AFF9C58723B@LSWWin764NB><92A7EEEA2A8A475C9735EFEEB13157CE@LSWWin764NB> <6BFA1764-1200-48DA-8898-5C22B0A2F53E@gmail.com> Message-ID: <6561B0F0C9D048C48A4EAE8160D0FF30@LSWWin764NB> Hi Tim, thx for fast answer, need to get sorted my newsgroups as it was posted wrong. It is wrong in Dolphin: ( -16r10000001 bitShift: -32 )-1 ( -16r10000000 bitShift: -32 )-1 ( -16r1000001 bitShift: -24 ) -2 ( -16r1000000 bitShift: -24 ) -1 but according to Dolphin's comment it is correct. Frank _____ Von: squeak-dev-bounces@lists.squeakfoundation.org [mailto:squeak-dev-bounces@lists.squeakfoundation.org] Im Auftrag von Tim Olson Gesendet: Freitag, 11. Dezember 2015 21:16 An: The general-purpose Squeak developers list Betreff: Re: [squeak-dev] bitshift: On Dec 11, 2015, at 2:00 PM, Frank Lesser wrote: Hi, ( -16r100000001 bitShift: -32 ) = -2 ( -16r100000000 bitShift: -32 ) = -1 looks odd to me, guess the prim needs to be fixed Looks correct to me: (-16r100000001 bitAnd: 16rFFFFFFFFFF) hex -> '16rFEFFFFFFFF' downshift this by 32 bits and you get 16rFFFFFFFE, which is -2 (-16r100000000 bitAnd: 16rFFFFFFFFFF) hex -> '16rFF00000000' downshift this by 32 bits and you get 16rFFFFFFFF, which is -1. - tim -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151211/268c9fd8/attachment.htm From tim at rowledge.org Fri Dec 11 22:39:22 2015 From: tim at rowledge.org (tim Rowledge) Date: Fri Dec 11 22:39:29 2015 Subject: [squeak-dev] Re: HandMorph etc #grabTransform In-Reply-To: <6D41EA3D-8470-4B02-B0E7-CC63CF0C4304@rowledge.org> References: <6D41EA3D-8470-4B02-B0E7-CC63CF0C4304@rowledge.org> Message-ID: <5C80A9AF-1D0D-4139-BB05-97DF728CBED8@rowledge.org> With power back, I can do some googling and it seems nothing has ever really been done with #grabTransform that has leaked out into the public googlesphere. Somebody somewhere must remember what it was intended for? > On 10-12-2015, at 2:47 PM, tim Rowledge wrote: > > Has anybody used the #grabTransform message that is sent by HandMorph as part of picking up a Morph? Every use I can find of it seems to set it to an Identity transform and trying to use it for make my dragged morph appear half-size is not working in a very understandable manner. > > HandMorph>grabMorph:from: is the major player here; it asks the formerOwner for the grabTransform (without specifying for which grabbed morph, which seems odd) and uses that along withe formerOwner?s transform to do things to the grabbed morph. If I make my owner-morph implement grabTransform to return an identity transform, the grabbing works as normal. If I provide atransform that does any scaling (which is what I want to use) then > a) the positions and offsets get all screwy > b) dropping doesn?t work the same - you have to drop the morph and then click-release again to get it to actually drop. That part really confuses me right now. > > I doubt this code was stuck in for the fun of it so I feel sure somebody must have had a use for it in the past. Does anyone remember anything about it? > > Oh joy; power cut. Hopefully the UPS will last long enough to get this sent. tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Ignoranus (n): A person who's both stupid and an asshole. From robert.w.withers at gmail.com Fri Dec 11 22:54:47 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Fri Dec 11 22:54:53 2015 Subject: [squeak-dev] Re: [Vm-dev] Re: looking for an image file In-Reply-To: <566B2B53.8030702@gmail.com> References: <566A4A6D.4080702@gmail.com> <566A4C68.5000002@gmail.com> <566A4EC8.9000805@gmail.com> <20151211142006.GA57925@shell.msen.com> <566B1FC6.9010103@gmail.com> <566B2B53.8030702@gmail.com> Message-ID: <566B5437.8090600@gmail.com> Skipped content of type multipart/alternative-------------- next part -------------- A non-text attachment was scrubbed... Name: mvm.logs.zip Type: application/zip Size: 6699 bytes Desc: not available Url : http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151211/e594a367/mvm.logs.zip From euanmee at gmail.com Sat Dec 12 01:45:54 2015 From: euanmee at gmail.com (EuanM) Date: Sat Dec 12 01:45:58 2015 Subject: [squeak-dev] Re: [Pharo-dev] Unicode Support In-Reply-To: <007287EA-40D5-46E4-8D93-CF7BFE70743A@gmail.com> References: <5662E1D8.1070802@free.fr> <1449848269163-4866610.post@n4.nabble.com> <5EE3E92D-7822-44C7-8781-D72A71C3882F@mac.com> <007287EA-40D5-46E4-8D93-CF7BFE70743A@gmail.com> Message-ID: Elliot, what's your take on having heterogenous collections for the composed Unicode? i.e. collections with one element for each character, with some characters being themselves a collection of characters (Simple character like "a" is one char, and a character which is a collection of characters is the fully composed version of ? (01d5), a U (0055) with a diaeresis - ? - ( 00a8 aka 0308 in combining form ) on top to form the compatibility character ? (ooDC) which then gets a macron- ? -( 0304) on top of that so a #(0061) ? #(01d5) = #( 00dc 0304) = #( 0055 0308 0304) i.e a string which alternated those two characters 'a?a?a?a?' would be represented by something equivalent to: #( 0061 #( 0055 0308 0304) 0061 #( 0055 0308 0304) 0061 #( 0055 0308 0304) 0061 #( 0055 0308 0304) ) as opposed to a string of compatibility characters: #( 0061 01d5 0061 01d5 0061 01d5 0061 01d5) Does alternating the type used for characters in a string have a significant effect on speed? On 11 December 2015 at 23:08, Eliot Miranda wrote: > Hi Todd, > > On Dec 11, 2015, at 12:57 PM, Todd Blanchard wrote: > > > On Dec 11, 2015, at 12:19, EuanM wrote: > > "If it hasn't already been said, please do not conflate Unicode and > UTF-8. I think that would be a recipe for > a high P.I.T.A. factor." --Richard Sargent > > > Well, yes. But I think you guys are making this way too hard. > > A unicode character is an abstract idea - for instance the letter 'a'. > The letter 'a' has a code point - its the number 97. How the number 97 is > represented in the computer is irrelevant. > > Now we get to transfer encodings. These are UTF8, UTF16, etc.... A > transfer encoding specifies the binary representation of the sequence of > code points. > > UTF8 is a variable length byte encoding. You read it one byte at a time, > aggregating byte sequences to 'code points'. ByteArray would be an > excellent choice as a superclass but it must be understood that #at: or > #at:put refers to a byte, not a character. If you want characters, you have > to start at the beginning and process it sequentially, like a stream (if > working in the ASCII domain - you can generally 'cheat' this a bit). A C > representation would be char utf8[] > > UTF16 is also a variable length encoding of two byte quantities - what C > used to call a 'short int'. You process it in two byte chunks instead of > one byte chunks. Like UTF8, you must read it sequentially to interpret the > characters. #at and #at:put: would necessarily refer to byte pairs and not > characters. A C representation would be short utf16[]; It would also to > 50% space inefficient for ASCII - which is normally the bulk of your text. > > Realistically, you need exactly one in-memory format and stream > readers/writers that can convert (these are typically table driven state > machines). My choice would be UTF8 for the internal memory format and the > ability to read and write from UTF8 to UTF16. > > But I stress again...strings don't really need indexability as much as you > think and neither UTF8 nor UTF16 provide this property anyhow as they are > variable length encodings. I don't see any sensible reason to have more > than one in-memory binary format in the image. > > > The only reasons are space and time. If a string only contains code points > in the range 0-255 there's no point in squandering 4 bytes per code point > (same goes for 0-65535). Further, if in some application interchange is > more important than random access it may make sense in performance grounds > to use utf-8 directly. > > Again, Smalltalk's dynamic typing makes it easy to have one's cake and eat > it too. > > My $0.02c > > > _,,,^..^,,,_ (phone) > > > I agree. :-) > > Regarding UTF-16, I just want to be able to export to, and receive > from, Windows (and any other platforms using UTF-16 as their native > character representation). > > Windows will always be able to accept UTF-16. All Windows apps *might > well* export UTF-16. There may be other platforms which use UTF-16 as > their native format. I'd just like to be able to cope with those > situations. Nothing more. > > All this is requires is a Utf16String class that has an asUtf8String > method (and any other required conversion methods). > > From eliot.miranda at gmail.com Sat Dec 12 02:31:59 2015 From: eliot.miranda at gmail.com (Eliot Miranda) Date: Sat Dec 12 02:32:03 2015 Subject: [squeak-dev] Re: [Pharo-dev] Unicode Support In-Reply-To: References: <5662E1D8.1070802@free.fr> <1449848269163-4866610.post@n4.nabble.com> <5EE3E92D-7822-44C7-8781-D72A71C3882F@mac.com> <007287EA-40D5-46E4-8D93-CF7BFE70743A@gmail.com> Message-ID: Hi Euan, On Fri, Dec 11, 2015 at 5:45 PM, EuanM wrote: > Elliot, what's your take on having heterogenous collections for the > composed Unicode? > I'm not sure I'm understanding the question, but... I'm told by someone in the know that string concatenation is a big deal in certain applications, so providing tree-like representations for strings can be a win since concatenation is O(1) (allocate a new root and assign the two subtrees). It seems reasonable to have a rich library with several representations available with different trade-offs. But I'd let requirements drive design, not feature dreams. > i.e. collections with one element for each character, with some > characters being themselves a collection of characters > > (Simple character like "a" is one char, and a character which is a > collection of characters is the fully composed version of ? (01d5), a > U (0055) with a diaeresis - ? - ( 00a8 aka 0308 in combining form ) > on top to form the compatibility character ? (ooDC) which then gets a > macron- ? -( 0304) on top of that > > so > a #(0061) > > ? #(01d5) = #( 00dc 0304) = #( 0055 0308 0304) > > i.e a string which alternated those two characters > > 'a?a?a?a?' > > would be represented by something equivalent to: > > #( 0061 #( 0055 0308 0304) 0061 #( 0055 0308 0304) 0061 #( 0055 0308 > 0304) 0061 #( 0055 0308 0304) ) > > as opposed to a string of compatibility characters: > #( 0061 01d5 0061 01d5 0061 01d5 0061 01d5) > > Does alternating the type used for characters in a string have a > significant effect on speed? > I honestly don't know. You've just gone well beyond my familiarity with the issues :-). I'm just a VM guy :-). But I will say that in cases like this, real applications and the profiler are your friends. Be guided by what you need now, not by what you think you'll need further down the road. > On 11 December 2015 at 23:08, Eliot Miranda > wrote: > > Hi Todd, > > > > On Dec 11, 2015, at 12:57 PM, Todd Blanchard wrote: > > > > > > On Dec 11, 2015, at 12:19, EuanM wrote: > > > > "If it hasn't already been said, please do not conflate Unicode and > > UTF-8. I think that would be a recipe for > > a high P.I.T.A. factor." --Richard Sargent > > > > > > Well, yes. But I think you guys are making this way too hard. > > > > A unicode character is an abstract idea - for instance the letter 'a'. > > The letter 'a' has a code point - its the number 97. How the number 97 > is > > represented in the computer is irrelevant. > > > > Now we get to transfer encodings. These are UTF8, UTF16, etc.... A > > transfer encoding specifies the binary representation of the sequence of > > code points. > > > > UTF8 is a variable length byte encoding. You read it one byte at a time, > > aggregating byte sequences to 'code points'. ByteArray would be an > > excellent choice as a superclass but it must be understood that #at: or > > #at:put refers to a byte, not a character. If you want characters, you > have > > to start at the beginning and process it sequentially, like a stream (if > > working in the ASCII domain - you can generally 'cheat' this a bit). A C > > representation would be char utf8[] > > > > UTF16 is also a variable length encoding of two byte quantities - what C > > used to call a 'short int'. You process it in two byte chunks instead of > > one byte chunks. Like UTF8, you must read it sequentially to interpret > the > > characters. #at and #at:put: would necessarily refer to byte pairs and > not > > characters. A C representation would be short utf16[]; It would also to > > 50% space inefficient for ASCII - which is normally the bulk of your > text. > > > > Realistically, you need exactly one in-memory format and stream > > readers/writers that can convert (these are typically table driven state > > machines). My choice would be UTF8 for the internal memory format and > the > > ability to read and write from UTF8 to UTF16. > > > > But I stress again...strings don't really need indexability as much as > you > > think and neither UTF8 nor UTF16 provide this property anyhow as they are > > variable length encodings. I don't see any sensible reason to have more > > than one in-memory binary format in the image. > > > > > > The only reasons are space and time. If a string only contains code > points > > in the range 0-255 there's no point in squandering 4 bytes per code point > > (same goes for 0-65535). Further, if in some application interchange is > > more important than random access it may make sense in performance > grounds > > to use utf-8 directly. > > > > Again, Smalltalk's dynamic typing makes it easy to have one's cake and > eat > > it too. > > > > My $0.02c > > > > > > _,,,^..^,,,_ (phone) > > > > > > I agree. :-) > > > > Regarding UTF-16, I just want to be able to export to, and receive > > from, Windows (and any other platforms using UTF-16 as their native > > character representation). > > > > Windows will always be able to accept UTF-16. All Windows apps *might > > well* export UTF-16. There may be other platforms which use UTF-16 as > > their native format. I'd just like to be able to cope with those > > situations. Nothing more. > > > > All this is requires is a Utf16String class that has an asUtf8String > > method (and any other required conversion methods). > > > > > > -- _,,,^..^,,,_ best, Eliot -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151211/d1592ad9/attachment.htm From robert.w.withers at gmail.com Sat Dec 12 04:53:57 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Sat Dec 12 04:54:04 2015 Subject: [squeak-dev] squeak & pharo versions of crypto and SecureSession Message-ID: <566BA865.7050603@gmail.com> Folks, I cross-tested with Pharo and here are the good versions: Cryptography - squeak: Cryptography-rww.49.mcz - pharo: Cryptography-Pharo-rww.38.mcz SecureSession: - squeak & pharo: SecureSession-RobertWithers.20.mcz I hope this helps, Robert From lecteur at zogotounga.net Sat Dec 12 09:09:24 2015 From: lecteur at zogotounga.net (=?UTF-8?Q?St=c3=a9phane_Rollandin?=) Date: Sat Dec 12 09:09:25 2015 Subject: [squeak-dev] Re: [Pharo-dev] Unicode Support In-Reply-To: References: <5662E1D8.1070802@free.fr> <1449848269163-4866610.post@n4.nabble.com> <5EE3E92D-7822-44C7-8781-D72A71C3882F@mac.com> <007287EA-40D5-46E4-8D93-CF7BFE70743A@gmail.com> Message-ID: <566BE444.2090402@zogotounga.net> Eliot Miranda: > But I'd let requirements drive design, not feature dreams. ... > Be guided by what you need now, not by what you think you'll need further > down the road. These two sentences should be written in human sized gold letters in your bathroom, people ! Wisdom ! I mean, +1 Stef From karlramberg at gmail.com Sat Dec 12 10:53:29 2015 From: karlramberg at gmail.com (karl ramberg) Date: Sat Dec 12 10:53:31 2015 Subject: [squeak-dev] HandMorph etc #grabTransform In-Reply-To: <6D41EA3D-8470-4B02-B0E7-CC63CF0C4304@rowledge.org> References: <6D41EA3D-8470-4B02-B0E7-CC63CF0C4304@rowledge.org> Message-ID: On Thu, Dec 10, 2015 at 11:47 PM, tim Rowledge wrote: > Has anybody used the #grabTransform message that is sent by HandMorph as > part of picking up a Morph? Every use I can find of it seems to set it to > an Identity transform and trying to use it for make my dragged morph appear > half-size is not working in a very understandable manner. > I think this is used for moving a morph that is transformed and also grabbing a duplicated morph that is transformed. > HandMorph>grabMorph:from: is the major player here; it asks the > formerOwner for the grabTransform (without specifying for which grabbed > morph, which seems odd) and uses that along withe formerOwner?s transform > to do things to the grabbed morph. If I make my owner-morph implement > grabTransform to return an identity transform, the grabbing works as > normal. If I provide atransform that does any scaling (which is what I want > to use) then > a) the positions and offsets get all screwy > b) dropping doesn?t work the same - you have to drop the morph and then > click-release again to get it to actually drop. That part really confuses > me right now. > > Maybe attachMorph: will work easier as you can do the trickery to the morph and just attach it to the hand ? Best, Karl I doubt this code was stuck in for the fun of it so I feel sure somebody > must have had a use for it in the past. Does anyone remember anything about > it? > > Oh joy; power cut. Hopefully the UPS will last long enough to get this > sent. > > tim > -- > tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim > Useful random insult:- Overdue for deincarnation. > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151212/cab6b5da/attachment.htm From Lou at Keystone-Software.com Sat Dec 12 13:19:42 2015 From: Lou at Keystone-Software.com (Louis LaBrunda) Date: Sat Dec 12 13:19:53 2015 Subject: [squeak-dev] [Pharo-dev] Unicode Support References: <5662E1D8.1070802@free.fr> <1449848269163-4866610.post@n4.nabble.com> <5EE3E92D-7822-44C7-8781-D72A71C3882F@mac.com> <007287EA-40D5-46E4-8D93-CF7BFE70743A@gmail.com> <566BE444.2090402@zogotounga.net> Message-ID: Hi Guys, +0.95 I certainly agree we shouldn't write code now that we may need in the future but it doesn't hurt to look a little ahead and plan a bit. If the people defining database tables thought a little ahead and didn't try to save two characters per date by leaving off the century there never would have been a Y2K scare. Lou On Sat, 12 Dec 2015 10:09:24 +0100, St?phane Rollandin wrote: >Eliot Miranda: > >> But I'd let requirements drive design, not feature dreams. > >... > >> Be guided by what you need now, not by what you think you'll need further >> down the road. > > >These two sentences should be written in human sized gold letters in >your bathroom, people ! Wisdom ! > >I mean, +1 > > >Stef > -- Louis LaBrunda Keystone Software Corp. SkypeMe callto://PhotonDemon From asqueaker at gmail.com Sat Dec 12 15:37:25 2015 From: asqueaker at gmail.com (Chris Muller) Date: Sat Dec 12 15:38:09 2015 Subject: [squeak-dev] drawing an arc Message-ID: Does anyone know how to draw an arc on a Canvas? It has drawOval / fillOval, but I want to draw a piece of pie. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151212/6396193a/attachment.htm From javier_diaz_r at mac.com Sat Dec 12 16:14:26 2015 From: javier_diaz_r at mac.com (Javier Diaz-Reinoso) Date: Sat Dec 12 16:15:06 2015 Subject: [squeak-dev] drawing an arc In-Reply-To: References: Message-ID: I make a ?naive? implementation of a GroupMorph, one of the class examples is a pie: -------------- next part -------------- A non-text attachment was scrubbed... Name: GroupMorph.st.zip Type: application/zip Size: 3445 bytes Desc: not available Url : http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151212/82f2839f/GroupMorph.st.zip -------------- next part -------------- > On Dec 12, 2015, at 10:37, Chris Muller wrote: > > Does anyone know how to draw an arc on a Canvas? It has drawOval / fillOval, but I want to draw a piece of pie. > From leves at caesar.elte.hu Sat Dec 12 17:30:06 2015 From: leves at caesar.elte.hu (Levente Uzonyi) Date: Sat Dec 12 17:30:10 2015 Subject: Broken Update [was [squeak-dev] Daily Commit Log] In-Reply-To: <05294A00-FB41-4DAF-8F88-9128319552DD@gmail.com> References: <05294A00-FB41-4DAF-8F88-9128319552DD@gmail.com> Message-ID: Hi All, I haven't changed anything, so it must be some randomness in the update process. I still think that we should make a new .mcm with the last version of Squeak-Version and with no Universes package in it. Levente On Fri, 11 Dec 2015, Eliot Miranda wrote: > Hi Karl, > > On Dec 10, 2015, at 9:59 PM, karl ramberg wrote: > > Hm,I'm a little confused/ worried because I did not change anything yet. > > > So was I when I replied yesterday cuz I didn't see what's changed. ?But it is fixed. ?Levente did you fix it? ?Is that what should be read from your last message? ?If so, much thanks! > > Best, > Karl > > On Fri, Dec 11, 2015 at 1:29 AM, Eliot Miranda wrote: > Thanks Karl, > ? ? my auto-update script now works perfectly again.? Thanks. > > On Thu, Dec 10, 2015 at 11:12 AM, karl ramberg wrote: > Here is a update-kfr.339.mcm with the difference from update-mt.339.mcm : > dependency ('Squeak-Version' 'Squeak-Version-kfr.4713' '96c171dc-c340-492e-b376-30e0212b9c8e') > > and universes are not in it > > On Thu, Dec 10, 2015 at 6:34 PM, karl ramberg wrote: > > > On Thu, Dec 10, 2015 at 6:01 PM, Levente Uzonyi wrote: > IMHO all you need to do is to fire up a Squeak image which has Squeak-Version-kfr.4713 loaded, open update-mt.339.mcm (open the Trunk repository, select this update map, and click Browse), remove Universes-mt.48 from the list > (select, right click, remove), and update Squeak-Version to kfr.4713 (select, right click, update from image (or repository if you're brave), and finally save it (Store) using the same name (update-mt.339). > > > I tried this, but there is no stored file from the MCConfigurationBrowser... I'm not sure what is going on, why it is not outputting a update file to my local repository. > > I also updated from a Squeak 5.0 image and I did not run into any problems with the update... > > > Best, > Karl > > ? > Here comes the tricky part: you can't overwrite an existing update map, nor can you move it away from the repository using the web interface, so an administrator of source.squeak.org will have to do one of these things to let you > save the correct version. > Alternatively, you could try to save it using different initials but same version (e.g. update-kfr.339) and hope that the update process will sort the maps lexicographically, but I'd rather not rely on this. > > Levente > > On Thu, 10 Dec 2015, karl ramberg wrote: > > Thanks, > Best, > Karl > > On Thu, Dec 10, 2015 at 5:00 PM, Eliot Miranda wrote: > ? ? ? Hi Karl, > ? ? ? On Wed, Dec 9, 2015 at 9:27 PM, karl ramberg wrote: > ? ? ? ? ? ? Hi,It it related with this Levente mentioned in a mail earlier ? > > "Sadly the update process will break if your image has Universes-mt.48 in it, because it'll try to load Universes-kfr.49 after it has been removed by?Squeak-Version. I think removing Universes from the update map will > help." > > I asked how to?proceed but I got no answer. > I'm not sure how one make a update map > > > As Chris says the updates live in the "update" package.? Open up a repository browser on source.squeak.org/trunk and you'll see them towards the bottom (update.spur contains older updates from the time of the bootstrap > from 4.6 to what became 5.0, you can ignore > them). > > What I would do is create a local mirror, either by fully populating my package cache with all the 5.0 packages and updates, or creating a directory repository and downloading all the packages and updates to there and > then play with the local repository until I > got the update working again, and then I would upload the changed updates to trunk. > > I'm attaching SpurBootstrapMonticelloPackagePatcher which is the hack I used to automate patching 4.6 packages to make them Spur compatible in the run-up to 5.0.? I'm doing this because it contains methods like > downloadToSourceDirAllPackageVersionsStartingWith: > that download all versions of a package to a local repository starting from a specific version.? Sao these hacks will help you in populating your local repository prior to experimenting. Once I'd downloaded all packages I > would then take a local copy and modify > the copy, I could then diff the original and the modified clone to see what had changed. > > HTH > > ?_,,,^..^,,,_ > best,?Eliot > > ? ? ? Best, > Karl? > [cleardot.gif] > > On Thu, Dec 10, 2015 at 4:06 AM, Eliot Miranda wrote: > ? ? ? Hi, > ? ? whatever happened in unloading Universes, it broke the ability to update a release Squeak 5.0 image to the current tip.? Hence my automated scripts to build 64-bit images are broken.? Can whoever broke the update > either fix it, or provide some > reasonable workaround so that I can restore my scripts?? What's at stake here is, well, a high performance 64-bit Squeak and/or Pharo. > > I think it's reasonable, even mandatory that we should be able to update a release N image to the tip of release N.? We should have CI servers that try and do this (I think we do, don't we?), and so IMO it's incumbent on > whoever breaks the update to > fix things, e.g. by changing update maps appropriately. ?[and I don't mean this as a punishment, or something that the person has to do on their own; I'm happy to help fix things, it's /really/ important that the update > works, and not just from the > last update]. > > Thanks in advance.? Apologies if I appear rude; it's not my intent.? I just want my auto-update back ;-) > > On Thu, Nov 26, 2015 at 2:55 PM, wrote: > ? ? ? Changes to Trunk (http://source.squeak.org/trunk.html) in the last 24 hours: > > ? ? ? http://lists.squeakfoundation.org/pipermail/packages/2015-November/009199.html > > ? ? ? Name: Squeak-Version-kfr.4713 > ? ? ? Ancestors: Squeak-Version-kfr.4712 > > ? ? ? Unload and unregister Universes > > ? ? ? ============================================= > > ? > _,,,^..^,,,_ > best,?Eliot > > > > > > > > > > > > > > > > > > > -- > _,,,^..^,,,_ > best,?Eliot > > > > > > > From rpboland at gmail.com Sat Dec 12 17:50:04 2015 From: rpboland at gmail.com (Ralph Boland) Date: Sat Dec 12 17:50:08 2015 Subject: [squeak-dev] Re: Recovering method changes after crash (Ralph Boland) (Sorry for being brain dead) Message-ID: > Hi Ralph, > I think what you are looking for is "Extras > Recover Changes" in the > world main docking bar. But the path you mentioned with "World Menu > > changes > recently logged changes" does exist as well in my 4.6 > images, so I wonder why it was not there for you. > Best regards, > Jakob You are of course totally correct. I was behaving too brain dead to figure out that to file in selections I use a menu. In Squeak 3.11 this could be done by clicking on a button. Ralph (If I only had a brain; from the Wizard of Oz) From eliot.miranda at gmail.com Sat Dec 12 18:21:19 2015 From: eliot.miranda at gmail.com (Eliot Miranda) Date: Sat Dec 12 18:21:25 2015 Subject: [squeak-dev] [Pharo-dev] Unicode Support In-Reply-To: References: <5662E1D8.1070802@free.fr> <1449848269163-4866610.post@n4.nabble.com> <5EE3E92D-7822-44C7-8781-D72A71C3882F@mac.com> <007287EA-40D5-46E4-8D93-CF7BFE70743A@gmail.com> <566BE444.2090402@zogotounga.net> Message-ID: <51EEA113-D82A-4D3A-80D4-44BACAB736B9@gmail.com> Hi Louis, _,,,^..^,,,_ (phone) > On Dec 12, 2015, at 5:19 AM, Louis LaBrunda wrote: > > Hi Guys, > > +0.95 I certainly agree we shouldn't write code now that we may need in > the future but it doesn't hurt to look a little ahead and plan a bit. If > the people defining database tables thought a little ahead and didn't try > to save two characters per date by leaving off the century there never > would have been a Y2K scare. > > Lou I am /not/ saying one shouldn't look forward; I am /not/ saying cut corners or do weak design. I am saying don't implement what you're not necessarily going to use. The y2k bug is either bad design or a space optimisation, depending on your point of view, but it isn't an example of requirements driving design. The requirement is to represent dates; the step to use 2 digits for the year is a subsequent step. Here's perhaps a better example. I've just spent two years working on Spur, the VM & object representation under Squeak 5 and Pharo 5. I designed in support for 64-bits from the start. Because I looked forward, this year the 64-bit version was relatively straight-forward to add. I /could/ have designed the system to support 128 bits on the assumption that some time 64-bits will be superseded by 128 bits. There there was a requirement for 64 bits. There is no requirement for 128 bits. Had I aimed for 128 bits the system would be a mess; too big, slow, more complex to try and get it to work in 32- & 64-bits. Even further back I designed into the Cogit (the VM's JIT compiler) a split between the generic code generator and an object-representation-specific code generator that generates all code to do with object representation, such as testing for tagged typed, fetching inst vars or allocating new objects, because I knew I wanted to replace the object representation as soon as possible. Had I not done so, adding Spur would have taken much more work; I would have had to engineer the split by refactoring instead of by design. So being driven by requirements doesn't mean not looking to the future, it means implementing what you need and what you /know/ you're going to need, not what "would be nice to have" or what "sounds like a really cool idea". _,,,^..^,,,_ (phone) > On Sat, 12 Dec 2015 10:09:24 +0100, St?phane Rollandin > wrote: > >> Eliot Miranda: >> >>> But I'd let requirements drive design, not feature dreams. >> >> ... >> >>> Be guided by what you need now, not by what you think you'll need further >>> down the road. >> >> >> These two sentences should be written in human sized gold letters in >> your bathroom, people ! Wisdom ! >> >> I mean, +1 >> >> >> Stef > -- > Louis LaBrunda > Keystone Software Corp. > SkypeMe callto://PhotonDemon > > From Lou at Keystone-Software.com Sat Dec 12 19:57:08 2015 From: Lou at Keystone-Software.com (Louis LaBrunda) Date: Sat Dec 12 19:57:31 2015 Subject: [squeak-dev] [Pharo-dev] Unicode Support References: <1449848269163-4866610.post@n4.nabble.com> <5EE3E92D-7822-44C7-8781-D72A71C3882F@mac.com> <007287EA-40D5-46E4-8D93-CF7BFE70743A@gmail.com> <566BE444.2090402@zogotounga.net> <51EEA113-D82A-4D3A-80D4-44BACAB736B9@gmail.com> Message-ID: <2mso6b957s5b3hnr0747fbre0vacfr8lgq@4ax.com> Hi Eliot, >Hi Louis, >_,,,^..^,,,_ (phone) >> On Dec 12, 2015, at 5:19 AM, Louis LaBrunda wrote: >> >> Hi Guys, >> >> +0.95 I certainly agree we shouldn't write code now that we may need in >> the future but it doesn't hurt to look a little ahead and plan a bit. If >> the people defining database tables thought a little ahead and didn't try >> to save two characters per date by leaving off the century there never >> would have been a Y2K scare. >> >> Lou > >I am /not/ saying one shouldn't look forward; I am /not/ saying cut corners or do weak design. I'm sure you not. And I didn't mean to imply otherwise. And I don't disagree with anything you've said. I just wanted to be sure everyone understood there is a difference between not implementing what isn't needed now and looking ahead enough to allow easy changes in the future. Your example designing in support for 64-bits from the start is spot on. >I am saying don't implement what you're not necessarily going to use. >The y2k bug is either bad design or a space optimisation, depending on your point of view, but it isn't an example of requirements driving design. The requirement is to represent dates; the step to use 2 digits for the year is a subsequent step. Just a little history because I'm old enough to remember when disk space was limited and at a premium (I think IBM 3330 were about 200MB). The choice to leave off the century (back then dates and times were often stored as strings) was a space saving effort. It was a bad choice, if it was a conscious one. I'm not sure it was a conscious choice, as many programmers didn't look very far ahead even to see the approaching century. >Here's perhaps a better example. I've just spent two years working on Spur, the VM & object representation under Squeak 5 and Pharo 5. I designed in support for 64-bits from the start. Because I looked forward, this year the 64-bit version was relatively straight-forward to add. I /could/ have designed the system to support 128 bits on the assumption that some time 64-bits will be superseded by 128 bits. There there was a requirement for 64 bits. There is no requirement for 128 bits. Had I aimed for 128 bits the system would be a mess; too big, slow, more complex to try and get it to work in 32- & 64-bits. > >Even further back I designed into the Cogit (the VM's JIT compiler) a split between the generic code generator and an object-representation-specific code generator that generates all code to do with object representation, such as testing for tagged typed, fetching inst vars or allocating new objects, because I knew I wanted to replace the object representation as soon as possible. Had I not done so, adding Spur would have taken much more work; I would have had to engineer the split by refactoring instead of by design. > >So being driven by requirements doesn't mean not looking to the future, it means implementing what you need and what you /know/ you're going to need, not what "would be nice to have" or what "sounds like a really cool idea". > >_,,,^..^,,,_ (phone) > >> On Sat, 12 Dec 2015 10:09:24 +0100, St?phane Rollandin >> wrote: >> >>> Eliot Miranda: >>> >>>> But I'd let requirements drive design, not feature dreams. >>> >>> ... >>> >>>> Be guided by what you need now, not by what you think you'll need further >>>> down the road. >>> >>> >>> These two sentences should be written in human sized gold letters in >>> your bathroom, people ! Wisdom ! >>> >>> I mean, +1 >>> >>> >>> Stef >> -- >> Louis LaBrunda >> Keystone Software Corp. >> SkypeMe callto://PhotonDemon >> >> > -- Louis LaBrunda Keystone Software Corp. SkypeMe callto://PhotonDemon From leves at caesar.elte.hu Sat Dec 12 22:03:07 2015 From: leves at caesar.elte.hu (Levente Uzonyi) Date: Sat Dec 12 22:03:11 2015 Subject: Broken Update [was [squeak-dev] Daily Commit Log] In-Reply-To: References: <05294A00-FB41-4DAF-8F88-9128319552DD@gmail.com> Message-ID: I've uploaded a new mcm with the described changes. Levente On Sat, 12 Dec 2015, Levente Uzonyi wrote: > Hi All, > > I haven't changed anything, so it must be some randomness in the update > process. I still think that we should make a new .mcm with the last version > of Squeak-Version and with no Universes package in it. > > Levente > > On Fri, 11 Dec 2015, Eliot Miranda wrote: > >> Hi Karl, >> >> On Dec 10, 2015, at 9:59 PM, karl ramberg wrote: >> >> Hm,I'm a little confused/ worried because I did not change anything >> yet. >> >> >> So was I when I replied yesterday cuz I didn't see what's changed. ?But it >> is fixed. ?Levente did you fix it? ?Is that what should be read from your >> last message? ?If so, much thanks! >> >> Best, >> Karl >> >> On Fri, Dec 11, 2015 at 1:29 AM, Eliot Miranda >> wrote: >> Thanks Karl, >> ? ? my auto-update script now works perfectly again.? Thanks. >> >> On Thu, Dec 10, 2015 at 11:12 AM, karl ramberg >> wrote: >> Here is a update-kfr.339.mcm with the difference from >> update-mt.339.mcm : >> dependency ('Squeak-Version' 'Squeak-Version-kfr.4713' >> '96c171dc-c340-492e-b376-30e0212b9c8e') >> >> and universes are not in it >> >> On Thu, Dec 10, 2015 at 6:34 PM, karl ramberg >> wrote: >> >> >> On Thu, Dec 10, 2015 at 6:01 PM, Levente Uzonyi >> wrote: >> IMHO all you need to do is to fire up a Squeak image which has >> Squeak-Version-kfr.4713 loaded, open update-mt.339.mcm (open the Trunk >> repository, select this update map, and click Browse), remove >> Universes-mt.48 from the list >> (select, right click, remove), and update Squeak-Version to >> kfr.4713 (select, right click, update from image (or repository if you're >> brave), and finally save it (Store) using the same name (update-mt.339). >> >> >> I tried this, but there is no stored file from the >> MCConfigurationBrowser... I'm not sure what is going on, why it is not >> outputting a update file to my local repository. >> >> I also updated from a Squeak 5.0 image and I did not run into any problems >> with the update... >> >> >> Best, >> Karl >> >> ? >> Here comes the tricky part: you can't overwrite an existing update >> map, nor can you move it away from the repository using the web interface, >> so an administrator of source.squeak.org will have to do one of these >> things to let you >> save the correct version. >> Alternatively, you could try to save it using different initials but >> same version (e.g. update-kfr.339) and hope that the update process will >> sort the maps lexicographically, but I'd rather not rely on this. >> >> Levente >> >> On Thu, 10 Dec 2015, karl ramberg wrote: >> >> Thanks, >> Best, >> Karl >> >> On Thu, Dec 10, 2015 at 5:00 PM, Eliot Miranda >> wrote: >> ? ? ? Hi Karl, >> ? ? ? On Wed, Dec 9, 2015 at 9:27 PM, karl ramberg >> wrote: >> ? ? ? ? ? ? Hi,It it related with this Levente mentioned in a >> mail earlier ? >> >> "Sadly the update process will break if your image has >> Universes-mt.48 in it, because it'll try to load Universes-kfr.49 after it >> has been removed by?Squeak-Version. I think removing Universes from the >> update map will >> help." >> >> I asked how to?proceed but I got no answer. >> I'm not sure how one make a update map >> >> >> As Chris says the updates live in the "update" package.? Open >> up a repository browser on source.squeak.org/trunk and you'll see them >> towards the bottom (update.spur contains older updates from the time of the >> bootstrap >> from 4.6 to what became 5.0, you can ignore >> them). >> >> What I would do is create a local mirror, either by fully >> populating my package cache with all the 5.0 packages and updates, or >> creating a directory repository and downloading all the packages and >> updates to there and >> then play with the local repository until I >> got the update working again, and then I would upload the >> changed updates to trunk. >> >> I'm attaching SpurBootstrapMonticelloPackagePatcher which is >> the hack I used to automate patching 4.6 packages to make them Spur >> compatible in the run-up to 5.0.? I'm doing this because it contains >> methods like >> downloadToSourceDirAllPackageVersionsStartingWith: >> that download all versions of a package to a local repository >> starting from a specific version.? Sao these hacks will help you in >> populating your local repository prior to experimenting. Once I'd >> downloaded all packages I >> would then take a local copy and modify >> the copy, I could then diff the original and the modified clone >> to see what had changed. >> >> HTH >> >> ?_,,,^..^,,,_ >> best,?Eliot >> >> ? ? ? Best, >> Karl? >> [cleardot.gif] >> >> On Thu, Dec 10, 2015 at 4:06 AM, Eliot Miranda >> wrote: >> ? ? ? Hi, >> ? ? whatever happened in unloading Universes, it broke the >> ability to update a release Squeak 5.0 image to the current tip.? Hence my >> automated scripts to build 64-bit images are broken.? Can whoever broke the >> update >> either fix it, or provide some >> reasonable workaround so that I can restore my scripts?? What's >> at stake here is, well, a high performance 64-bit Squeak and/or Pharo. >> >> I think it's reasonable, even mandatory that we should be able >> to update a release N image to the tip of release N.? We should have CI >> servers that try and do this (I think we do, don't we?), and so IMO it's >> incumbent on >> whoever breaks the update to >> fix things, e.g. by changing update maps appropriately. ?[and I >> don't mean this as a punishment, or something that the person has to do on >> their own; I'm happy to help fix things, it's /really/ important that the >> update >> works, and not just from the >> last update]. >> >> Thanks in advance.? Apologies if I appear rude; it's not my >> intent.? I just want my auto-update back ;-) >> >> On Thu, Nov 26, 2015 at 2:55 PM, >> wrote: >> ? ? ? Changes to Trunk (http://source.squeak.org/trunk.html) in >> the last 24 hours: >> >> ? ? ? >> http://lists.squeakfoundation.org/pipermail/packages/2015-November/009199.html >> >> ? ? ? Name: Squeak-Version-kfr.4713 >> ? ? ? Ancestors: Squeak-Version-kfr.4712 >> >> ? ? ? Unload and unregister Universes >> >> ? ? ? ============================================= >> >> ? >> _,,,^..^,,,_ >> best,?Eliot >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> -- >> _,,,^..^,,,_ >> best,?Eliot >> >> >> >> >> >> > From karlramberg at gmail.com Sat Dec 12 22:09:56 2015 From: karlramberg at gmail.com (karl ramberg) Date: Sat Dec 12 22:09:59 2015 Subject: Broken Update [was [squeak-dev] Daily Commit Log] In-Reply-To: References: <05294A00-FB41-4DAF-8F88-9128319552DD@gmail.com> Message-ID: Thanks Best, Karl On Sat, Dec 12, 2015 at 11:03 PM, Levente Uzonyi wrote: > I've uploaded a new mcm with the described changes. > > Levente > > > On Sat, 12 Dec 2015, Levente Uzonyi wrote: > > Hi All, >> >> I haven't changed anything, so it must be some randomness in the update >> process. I still think that we should make a new .mcm with the last version >> of Squeak-Version and with no Universes package in it. >> >> Levente >> >> On Fri, 11 Dec 2015, Eliot Miranda wrote: >> >> Hi Karl, >>> >>> On Dec 10, 2015, at 9:59 PM, karl ramberg wrote: >>> >>> Hm,I'm a little confused/ worried because I did not change >>> anything yet. >>> >>> >>> So was I when I replied yesterday cuz I didn't see what's changed. But >>> it is fixed. Levente did you fix it? Is that what should be read from >>> your last message? If so, much thanks! >>> >>> Best, >>> Karl >>> >>> On Fri, Dec 11, 2015 at 1:29 AM, Eliot Miranda >>> wrote: >>> Thanks Karl, >>> my auto-update script now works perfectly again. Thanks. >>> >>> On Thu, Dec 10, 2015 at 11:12 AM, karl ramberg >>> wrote: >>> Here is a update-kfr.339.mcm with the difference from >>> update-mt.339.mcm : >>> dependency ('Squeak-Version' 'Squeak-Version-kfr.4713' >>> '96c171dc-c340-492e-b376-30e0212b9c8e') >>> >>> and universes are not in it >>> >>> On Thu, Dec 10, 2015 at 6:34 PM, karl ramberg >>> wrote: >>> >>> >>> On Thu, Dec 10, 2015 at 6:01 PM, Levente Uzonyi < >>> leves@caesar.elte.hu> wrote: >>> IMHO all you need to do is to fire up a Squeak image which >>> has Squeak-Version-kfr.4713 loaded, open update-mt.339.mcm (open the Trunk >>> repository, select this update map, and click Browse), remove >>> Universes-mt.48 from the list >>> (select, right click, remove), and update Squeak-Version to >>> kfr.4713 (select, right click, update from image (or repository if you're >>> brave), and finally save it (Store) using the same name (update-mt.339). >>> >>> >>> I tried this, but there is no stored file from the >>> MCConfigurationBrowser... I'm not sure what is going on, why it is not >>> outputting a update file to my local repository. >>> >>> I also updated from a Squeak 5.0 image and I did not run into any >>> problems with the update... >>> >>> >>> Best, >>> Karl >>> >>> >>> Here comes the tricky part: you can't overwrite an existing update >>> map, nor can you move it away from the repository using the web interface, >>> so an administrator of source.squeak.org will have to do one of these >>> things to let you >>> save the correct version. >>> Alternatively, you could try to save it using different initials >>> but same version (e.g. update-kfr.339) and hope that the update process >>> will sort the maps lexicographically, but I'd rather not rely on this. >>> >>> Levente >>> >>> On Thu, 10 Dec 2015, karl ramberg wrote: >>> >>> Thanks, >>> Best, >>> Karl >>> >>> On Thu, Dec 10, 2015 at 5:00 PM, Eliot Miranda < >>> eliot.miranda@gmail.com> wrote: >>> Hi Karl, >>> On Wed, Dec 9, 2015 at 9:27 PM, karl ramberg < >>> karlramberg@gmail.com> wrote: >>> Hi,It it related with this Levente mentioned in >>> a mail earlier ? >>> >>> "Sadly the update process will break if your image has >>> Universes-mt.48 in it, because it'll try to load Universes-kfr.49 after it >>> has been removed by Squeak-Version. I think removing Universes from the >>> update map will >>> help." >>> >>> I asked how to proceed but I got no answer. >>> I'm not sure how one make a update map >>> >>> >>> As Chris says the updates live in the "update" package. >>> Open up a repository browser on source.squeak.org/trunk and you'll see >>> them towards the bottom (update.spur contains older updates from the time >>> of the bootstrap >>> from 4.6 to what became 5.0, you can ignore >>> them). >>> >>> What I would do is create a local mirror, either by fully >>> populating my package cache with all the 5.0 packages and updates, or >>> creating a directory repository and downloading all the packages and >>> updates to there and >>> then play with the local repository until I >>> got the update working again, and then I would upload the >>> changed updates to trunk. >>> >>> I'm attaching SpurBootstrapMonticelloPackagePatcher which is >>> the hack I used to automate patching 4.6 packages to make them Spur >>> compatible in the run-up to 5.0. I'm doing this because it contains >>> methods like >>> downloadToSourceDirAllPackageVersionsStartingWith: >>> that download all versions of a package to a local >>> repository starting from a specific version. Sao these hacks will help you >>> in populating your local repository prior to experimenting. Once I'd >>> downloaded all packages I >>> would then take a local copy and modify >>> the copy, I could then diff the original and the modified >>> clone to see what had changed. >>> >>> HTH >>> >>> _,,,^..^,,,_ >>> best, Eliot >>> >>> Best, >>> Karl >>> [cleardot.gif] >>> >>> On Thu, Dec 10, 2015 at 4:06 AM, Eliot Miranda < >>> eliot.miranda@gmail.com> wrote: >>> Hi, >>> whatever happened in unloading Universes, it broke the >>> ability to update a release Squeak 5.0 image to the current tip. Hence my >>> automated scripts to build 64-bit images are broken. Can whoever broke the >>> update >>> either fix it, or provide some >>> reasonable workaround so that I can restore my scripts? >>> What's at stake here is, well, a high performance 64-bit Squeak and/or >>> Pharo. >>> >>> I think it's reasonable, even mandatory that we should be >>> able to update a release N image to the tip of release N. We should have >>> CI servers that try and do this (I think we do, don't we?), and so IMO it's >>> incumbent on >>> whoever breaks the update to >>> fix things, e.g. by changing update maps appropriately. >>> [and I don't mean this as a punishment, or something that the person has >>> to do on their own; I'm happy to help fix things, it's /really/ important >>> that the update >>> works, and not just from the >>> last update]. >>> >>> Thanks in advance. Apologies if I appear rude; it's not my >>> intent. I just want my auto-update back ;-) >>> >>> On Thu, Nov 26, 2015 at 2:55 PM, >>> wrote: >>> Changes to Trunk (http://source.squeak.org/trunk.html) >>> in the last 24 hours: >>> >>> >>> http://lists.squeakfoundation.org/pipermail/packages/2015-November/009199.html >>> >>> Name: Squeak-Version-kfr.4713 >>> Ancestors: Squeak-Version-kfr.4712 >>> >>> Unload and unregister Universes >>> >>> ============================================= >>> >>> >>> _,,,^..^,,,_ >>> best, Eliot >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> -- >>> _,,,^..^,,,_ >>> best, Eliot >>> >>> >>> >>> >>> >>> >>> > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151212/88c25e82/attachment.htm From lewis at mail.msen.com Sat Dec 12 22:58:56 2015 From: lewis at mail.msen.com (David T. Lewis) Date: Sat Dec 12 22:58:58 2015 Subject: [squeak-dev] DirectoryPlugin updated on SqueakSource Message-ID: <20151212225856.GA19870@shell.msen.com> A few years ago (well, more than a few), I implemented DirectoryPlugin. This claimed to (and I believe actually did) provide very large performance benefits on images of that time. It also required some intrusive changes on the image side, so it really was not appropriate for integration into the image. I recently updated the primitive and provided a separate image side access class, This is now on SqueakSource.com as package DirectoryPlugin. Since this was originally written circa 2003, the Squeak image has apparently improved immensely. The directory stream primitives, which were the original motivation for writing it, no longer provide any benefit at all. In fact, they seem to be slightly slower than the standard (modern) image). The file stat primitives do provide substantial performance benefit, so these may still be of interest for modern images. Here is the description from SqueakSource: ------- Project Description Home page http://wiki.squeak.org/squeak/2274 DirectoryPlugin provides access to a Posix directory stream for efficient access to directory contents. It also provides methods for testing file status and how to creating directory entries for a files and directories. The directory stream primitives use the address of a directory stream as a handle to identify a directory stream. This permits several directory streams to be opened and manipulated independently by separate Smalltalk processes. However, care must be taken to avoid passing the handle for an invalid directory stream (such as one which has already been closed) to one of these methods. The plugin has been tested on Linux and on Windows. The directory stream support produces a performance improvement only on Unix (Linux) systems. The file access primitives provide substantial improvements for both Unix and Windows systems. Other platforms have not been tested. DirectoryPluginAccess provides access to primitives in DirectoryPlugin, along with utility methods for various directory and file access functions. Performance characteristics have changed significantly since Squeak circa 2003. The readdir() primitives no longer provide any benefit, but the file testing primitives still yield a couple orders of magnitude improvement for some functions. From robert.w.withers at gmail.com Sun Dec 13 02:04:34 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Sun Dec 13 02:04:39 2015 Subject: [squeak-dev] squeak: could not find any display driver Message-ID: <566CD232.7070904@gmail.com> Are you tired of seeing this result: rabbit@rabbithole:~/warren/cogvm/image$ cogvm Mushroom.4.image squeak: could not find any display driver Aborted (core dumped) I am! Thank you for the help today. Unfortunately this is still an issue for me on both laptops: 32-bit and 64-bit OS. Both were working as well. I reinstalled the 64-bit and it failed right off the start. The commonality to all three failures was the fact that I did these below commands prior to it failing this way, everytime. I think it is the upgrade that does it in. I'll reinstall 32-bit and try without these commands; let you know. Robert sudo apt-get update sudo apt-get upgrade and I ran this script for system libraries for 32-bit squeak. #!/bin/bash # ARGUMENT HANDLING ============================================================= if { [ "$1" = "-h" ] || [ "$1" = "--help" ]; }; then echo "Install libraries required to build the pharo VM under ubuntu. " exit 0; elif [ $# -gt 0 ]; then echo "--help/-h is the only argument allowed" exit 1; fi # INSTALL BUILD LIBRARIES ====================================================== sudo apt-get install cmake zip bash-completion ruby git xz-utils debhelper devscripts sudo apt-get install libc6-dev:i386 libasound2:i386 libasound2-dev:i386 libasound2-plugins:i386 libssl-dev:i386 i#!libssl0.9.8:i386 libfreetype6-dev:i386 libx11-dev:i386 libsm-dev:i386 libice-dev:i386 sudo apt-get install build-essential gcc-multilib g++ sudo apt-get install libgl1-mesa-dev libgl1-mesa-glx:i386 sudo apt-get install binutils:i386 g++-4.6:i386 gcc-4.6:i386 sudo ln -s /usr/lib/i386-linux-gnu/mesa/libGL.so /usr/lib/i386-linux-gnu/libGL.so sudo ln -s /usr/lib/i386-linux-gnu/mesa/libGL.so.1 /usr/lib/i386-linux-gnu/mesa/libGL.so sudo apt-get install uuid-dev:i386 sudo apt-get install subversion -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151212/bd1d878f/attachment.htm From leves at caesar.elte.hu Sun Dec 13 02:54:02 2015 From: leves at caesar.elte.hu (Levente Uzonyi) Date: Sun Dec 13 02:54:08 2015 Subject: [squeak-dev] squeak: could not find any display driver In-Reply-To: <566CD232.7070904@gmail.com> References: <566CD232.7070904@gmail.com> Message-ID: What does "ldd vm-display-X11" print? Levente On Sat, 12 Dec 2015, Robert Withers wrote: > Are you tired of seeing this result: > > rabbit@rabbithole:~/warren/cogvm/image$ cogvm Mushroom.4.image > squeak: could not find any display driver > Aborted (core dumped) > > I am! Thank you for the help today. Unfortunately this is still an issue for me on both laptops: 32-bit and 64-bit OS.? Both were working as well. I reinstalled the 64-bit and it failed right off the start. The commonality to all three failures was the fact that I did > these below commands prior to it failing this way, everytime. I think it is the upgrade that does it in. I'll reinstall 32-bit and try without these commands; let you know. > > Robert > > sudo apt-get update > sudo apt-get upgrade > > and I ran this script for system libraries for 32-bit squeak. > #!/bin/bash > > # ARGUMENT HANDLING ============================================================= > if { [ "$1" = "-h" ] || [ "$1" = "--help" ]; }; then > ??? echo "Install libraries required to build the pharo VM under ubuntu. > " > ??? exit 0; > elif [ $# -gt 0 ]; then > ??? echo "--help/-h is the only argument allowed" > ??? exit 1; > fi > > # INSTALL BUILD LIBRARIES ====================================================== > sudo apt-get install cmake zip bash-completion ruby git xz-utils debhelper devscripts > sudo apt-get install libc6-dev:i386 libasound2:i386 libasound2-dev:i386 libasound2-plugins:i386 libssl-dev:i386 i#!libssl0.9.8:i386 libfreetype6-dev:i386 libx11-dev:i386 libsm-dev:i386 libice-dev:i386 > sudo apt-get install build-essential gcc-multilib g++ > sudo apt-get install libgl1-mesa-dev libgl1-mesa-glx:i386 > sudo apt-get install binutils:i386 g++-4.6:i386 gcc-4.6:i386 > sudo ln -s /usr/lib/i386-linux-gnu/mesa/libGL.so /usr/lib/i386-linux-gnu/libGL.so > sudo ln -s /usr/lib/i386-linux-gnu/mesa/libGL.so.1 /usr/lib/i386-linux-gnu/mesa/libGL.so > sudo apt-get install uuid-dev:i386 > sudo apt-get install subversion > > > > > > From robert.w.withers at gmail.com Sun Dec 13 03:03:53 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Sun Dec 13 03:04:01 2015 Subject: [squeak-dev] squeak: could not find any display driver In-Reply-To: References: <566CD232.7070904@gmail.com> Message-ID: <66BA6777-5BD2-4FEE-8754-B476C76CACE3@gmail.com> Yes, I learned this today, or rather I was reminded of old UNIX cmds I used to know. The results of this were good. Every dependency was linked. This was why I've been using that script, which covers those 32-bit dependencies pretty thoroughly. Alright --- robert > On Dec 12, 2015, at 9:54 PM, Levente Uzonyi wrote: > > What does "ldd vm-display-X11" print? > > Levente > >> On Sat, 12 Dec 2015, Robert Withers wrote: >> >> Are you tired of seeing this result: >> rabbit@rabbithole:~/warren/cogvm/image$ cogvm Mushroom.4.image >> squeak: could not find any display driver >> Aborted (core dumped) >> I am! Thank you for the help today. Unfortunately this is still an issue for me on both laptops: 32-bit and 64-bit OS. Both were working as well. I reinstalled the 64-bit and it failed right off the start. The commonality to all three failures was the fact that I did >> these below commands prior to it failing this way, everytime. I think it is the upgrade that does it in. I'll reinstall 32-bit and try without these commands; let you know. >> Robert >> sudo apt-get update >> sudo apt-get upgrade >> and I ran this script for system libraries for 32-bit squeak. >> #!/bin/bash >> >> # ARGUMENT HANDLING ============================================================= >> if { [ "$1" = "-h" ] || [ "$1" = "--help" ]; }; then >> echo "Install libraries required to build the pharo VM under ubuntu. >> " >> exit 0; >> elif [ $# -gt 0 ]; then >> echo "--help/-h is the only argument allowed" >> exit 1; >> fi >> >> # INSTALL BUILD LIBRARIES ====================================================== >> sudo apt-get install cmake zip bash-completion ruby git xz-utils debhelper devscripts >> sudo apt-get install libc6-dev:i386 libasound2:i386 libasound2-dev:i386 libasound2-plugins:i386 libssl-dev:i386 i#!libssl0.9.8:i386 libfreetype6-dev:i386 libx11-dev:i386 libsm-dev:i386 libice-dev:i386 >> sudo apt-get install build-essential gcc-multilib g++ >> sudo apt-get install libgl1-mesa-dev libgl1-mesa-glx:i386 >> sudo apt-get install binutils:i386 g++-4.6:i386 gcc-4.6:i386 >> sudo ln -s /usr/lib/i386-linux-gnu/mesa/libGL.so /usr/lib/i386-linux-gnu/libGL.so >> sudo ln -s /usr/lib/i386-linux-gnu/mesa/libGL.so.1 /usr/lib/i386-linux-gnu/mesa/libGL.so >> sudo apt-get install uuid-dev:i386 >> sudo apt-get install subversion > From leves at caesar.elte.hu Sun Dec 13 03:23:14 2015 From: leves at caesar.elte.hu (Levente Uzonyi) Date: Sun Dec 13 03:23:18 2015 Subject: [squeak-dev] squeak: could not find any display driver In-Reply-To: <66BA6777-5BD2-4FEE-8754-B476C76CACE3@gmail.com> References: <566CD232.7070904@gmail.com> <66BA6777-5BD2-4FEE-8754-B476C76CACE3@gmail.com> Message-ID: Is cogvm a script or a binary? If it's the former, then you should try running the binary. You can also try being explicit about the display plugin you want to use. E.g.: cogvm --vm-display-X11 Mushroom.4.image Levente On Sat, 12 Dec 2015, Robert Withers wrote: > Yes, I learned this today, or rather I was reminded of old UNIX cmds I used to know. The results of this were good. Every dependency was linked. This was why I've been using that script, which covers those 32-bit dependencies pretty thoroughly. > > Alright > --- > robert > >> On Dec 12, 2015, at 9:54 PM, Levente Uzonyi wrote: >> >> What does "ldd vm-display-X11" print? >> >> Levente >> >>> On Sat, 12 Dec 2015, Robert Withers wrote: >>> >>> Are you tired of seeing this result: >>> rabbit@rabbithole:~/warren/cogvm/image$ cogvm Mushroom.4.image >>> squeak: could not find any display driver >>> Aborted (core dumped) >>> I am! Thank you for the help today. Unfortunately this is still an issue for me on both laptops: 32-bit and 64-bit OS. Both were working as well. I reinstalled the 64-bit and it failed right off the start. The commonality to all three failures was the fact that I did >>> these below commands prior to it failing this way, everytime. I think it is the upgrade that does it in. I'll reinstall 32-bit and try without these commands; let you know. >>> Robert >>> sudo apt-get update >>> sudo apt-get upgrade >>> and I ran this script for system libraries for 32-bit squeak. >>> #!/bin/bash >>> >>> # ARGUMENT HANDLING ============================================================= >>> if { [ "$1" = "-h" ] || [ "$1" = "--help" ]; }; then >>> echo "Install libraries required to build the pharo VM under ubuntu. >>> " >>> exit 0; >>> elif [ $# -gt 0 ]; then >>> echo "--help/-h is the only argument allowed" >>> exit 1; >>> fi >>> >>> # INSTALL BUILD LIBRARIES ====================================================== >>> sudo apt-get install cmake zip bash-completion ruby git xz-utils debhelper devscripts >>> sudo apt-get install libc6-dev:i386 libasound2:i386 libasound2-dev:i386 libasound2-plugins:i386 libssl-dev:i386 i#!libssl0.9.8:i386 libfreetype6-dev:i386 libx11-dev:i386 libsm-dev:i386 libice-dev:i386 >>> sudo apt-get install build-essential gcc-multilib g++ >>> sudo apt-get install libgl1-mesa-dev libgl1-mesa-glx:i386 >>> sudo apt-get install binutils:i386 g++-4.6:i386 gcc-4.6:i386 >>> sudo ln -s /usr/lib/i386-linux-gnu/mesa/libGL.so /usr/lib/i386-linux-gnu/libGL.so >>> sudo ln -s /usr/lib/i386-linux-gnu/mesa/libGL.so.1 /usr/lib/i386-linux-gnu/mesa/libGL.so >>> sudo apt-get install uuid-dev:i386 >>> sudo apt-get install subversion >> > > From robert.w.withers at gmail.com Sun Dec 13 03:47:50 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Sun Dec 13 03:47:58 2015 Subject: [squeak-dev] squeak: could not find any display driver In-Reply-To: References: <566CD232.7070904@gmail.com> <66BA6777-5BD2-4FEE-8754-B476C76CACE3@gmail.com> Message-ID: <566CEA66.9070305@gmail.com> It is a renamed binary that was working, but no longer. I just reinstalled again, 64-bit Ubuntu 15.10. I did NOT upgrade. I ran the script but it was missing libsm6:i386 and libsm6:i386 and also had a bug. I'll attach the new one. It all installs but I still have the same issue "could not find any display driver". I also ran the all-in-One and it said "Running 32-bit Squeak on a 64-bit System. install-libs32 may install them.". I ran that script as well. Perhaps it is a 64-bit issue running 32-bits. I will reinstall 32-bit ubuntu, although my first box was 32-bit before it failed. Thanks for lending a hand. Best, Robert On 12/12/2015 10:23 PM, Levente Uzonyi wrote: > Is cogvm a script or a binary? If it's the former, then you should try > running the binary. > You can also try being explicit about the display plugin you want to > use. E.g.: cogvm --vm-display-X11 Mushroom.4.image > > Levente > > On Sat, 12 Dec 2015, Robert Withers wrote: > >> Yes, I learned this today, or rather I was reminded of old UNIX cmds >> I used to know. The results of this were good. Every dependency was >> linked. This was why I've been using that script, which covers those >> 32-bit dependencies pretty thoroughly. >> >> Alright >> --- >> robert >> >>> On Dec 12, 2015, at 9:54 PM, Levente Uzonyi >>> wrote: >>> >>> What does "ldd vm-display-X11" print? >>> >>> Levente >>> >>>> On Sat, 12 Dec 2015, Robert Withers wrote: >>>> >>>> Are you tired of seeing this result: >>>> rabbit@rabbithole:~/warren/cogvm/image$ cogvm Mushroom.4.image >>>> squeak: could not find any display driver >>>> Aborted (core dumped) >>>> I am! Thank you for the help today. Unfortunately this is still an >>>> issue for me on both laptops: 32-bit and 64-bit OS. Both were >>>> working as well. I reinstalled the 64-bit and it failed right off >>>> the start. The commonality to all three failures was the fact that >>>> I did >>>> these below commands prior to it failing this way, everytime. I >>>> think it is the upgrade that does it in. I'll reinstall 32-bit and >>>> try without these commands; let you know. >>>> Robert >>>> sudo apt-get update >>>> sudo apt-get upgrade >>>> and I ran this script for system libraries for 32-bit squeak. >>>> #!/bin/bash >>>> >>>> # ARGUMENT HANDLING >>>> ============================================================= >>>> if { [ "$1" = "-h" ] || [ "$1" = "--help" ]; }; then >>>> echo "Install libraries required to build the pharo VM >>>> under ubuntu. >>>> " >>>> exit 0; >>>> elif [ $# -gt 0 ]; then >>>> echo "--help/-h is the only argument allowed" >>>> exit 1; >>>> fi >>>> >>>> # INSTALL BUILD LIBRARIES >>>> ====================================================== >>>> sudo apt-get install cmake zip bash-completion ruby git >>>> xz-utils debhelper devscripts >>>> sudo apt-get install libc6-dev:i386 libasound2:i386 >>>> libasound2-dev:i386 libasound2-plugins:i386 libssl-dev:i386 >>>> i#!libssl0.9.8:i386 libfreetype6-dev:i386 libx11-dev:i386 >>>> libsm-dev:i386 libice-dev:i386 >>>> sudo apt-get install build-essential gcc-multilib g++ >>>> sudo apt-get install libgl1-mesa-dev libgl1-mesa-glx:i386 >>>> sudo apt-get install binutils:i386 g++-4.6:i386 gcc-4.6:i386 >>>> sudo ln -s /usr/lib/i386-linux-gnu/mesa/libGL.so >>>> /usr/lib/i386-linux-gnu/libGL.so >>>> sudo ln -s /usr/lib/i386-linux-gnu/mesa/libGL.so.1 >>>> /usr/lib/i386-linux-gnu/mesa/libGL.so >>>> sudo apt-get install uuid-dev:i386 >>>> sudo apt-get install subversion >>> >> >> > -------------- next part -------------- A non-text attachment was scrubbed... Name: setup-ubuntu.sh Type: application/x-shellscript Size: 1152 bytes Desc: not available Url : http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151212/ff03dcce/setup-ubuntu.bin From btc at openinworld.com Sun Dec 13 07:58:12 2015 From: btc at openinworld.com (Ben Coman) Date: Sun Dec 13 07:58:36 2015 Subject: [squeak-dev] [Pharo-dev] Unicode Support In-Reply-To: <2mso6b957s5b3hnr0747fbre0vacfr8lgq@4ax.com> References: <1449848269163-4866610.post@n4.nabble.com> <5EE3E92D-7822-44C7-8781-D72A71C3882F@mac.com> <007287EA-40D5-46E4-8D93-CF7BFE70743A@gmail.com> <566BE444.2090402@zogotounga.net> <51EEA113-D82A-4D3A-80D4-44BACAB736B9@gmail.com> <2mso6b957s5b3hnr0747fbre0vacfr8lgq@4ax.com> Message-ID: On Sun, Dec 13, 2015 at 3:57 AM, Louis LaBrunda wrote: > Hi Eliot, > >>Hi Louis, >>_,,,^..^,,,_ (phone) >>> On Dec 12, 2015, at 5:19 AM, Louis LaBrunda wrote: >>> >>> Hi Guys, >>> >>> +0.95 I certainly agree we shouldn't write code now that we may need in >>> the future but it doesn't hurt to look a little ahead and plan a bit. If >>> the people defining database tables thought a little ahead and didn't try >>> to save two characters per date by leaving off the century there never >>> would have been a Y2K scare. >>> >>> Lou >> >>I am /not/ saying one shouldn't look forward; I am /not/ saying cut corners or do weak design. > > I'm sure you not. And I didn't mean to imply otherwise. And I don't > disagree with anything you've said. I just wanted to be sure everyone > understood there is a difference between not implementing what isn't needed > now and looking ahead enough to allow easy changes in the future. Your > example designing in support for 64-bits from the start is spot on. > >>I am saying don't implement what you're not necessarily going to use. > >>The y2k bug is either bad design or a space optimisation, depending on your point of view, but it isn't an example of requirements driving design. The requirement is to represent dates; the step to use 2 digits for the year is a subsequent step. > > Just a little history because I'm old enough to remember when disk space > was limited and at a premium (I think IBM 3330 were about 200MB). The > choice to leave off the century (back then dates and times were often > stored as strings) was a space saving effort. It was a bad choice, if it > was a conscious one. I'm not sure it was a conscious choice, as many > programmers didn't look very far ahead even to see the approaching century. Obligatory... http://www.businessinsider.com.au/picture-of-ibm-hard-drive-on-airplane-2014-1 now get off my lawn... ;) cheers -ben From Lou at Keystone-Software.com Sun Dec 13 13:49:52 2015 From: Lou at Keystone-Software.com (Louis LaBrunda) Date: Sun Dec 13 13:50:07 2015 Subject: [squeak-dev] [Pharo-dev] Unicode Support References: <5EE3E92D-7822-44C7-8781-D72A71C3882F@mac.com> <007287EA-40D5-46E4-8D93-CF7BFE70743A@gmail.com> <566BE444.2090402@zogotounga.net> <51EEA113-D82A-4D3A-80D4-44BACAB736B9@gmail.com> <2mso6b957s5b3hnr0747fbre0vacfr8lgq@4ax.com> Message-ID: Hi Ben, >Obligatory... >http://www.businessinsider.com.au/picture-of-ibm-hard-drive-on-airplane-2014-1 >now get off my lawn... ;) > >cheers -ben I'm not sure what you mean be Obligatory but thanks for the link/picture. I was eight years old then so I don't really remember of that size. Lou -- Louis LaBrunda Keystone Software Corp. SkypeMe callto://PhotonDemon From tim at rowledge.org Mon Dec 14 01:19:55 2015 From: tim at rowledge.org (tim Rowledge) Date: Mon Dec 14 01:20:00 2015 Subject: [squeak-dev] HandMorph etc #grabTransform In-Reply-To: References: <6D41EA3D-8470-4B02-B0E7-CC63CF0C4304@rowledge.org> Message-ID: <8746E820-9425-4960-BCAE-ADD807CA5CE3@rowledge.org> > On 12-12-2015, at 2:53 AM, karl ramberg wrote: > > > > On Thu, Dec 10, 2015 at 11:47 PM, tim Rowledge wrote: > Has anybody used the #grabTransform message that is sent by HandMorph as part of picking up a Morph? Every use I can find of it seems to set it to an Identity transform and trying to use it for make my dragged morph appear half-size is not working in a very understandable manner. > > I think this is used for moving a morph that is transformed and also grabbing a duplicated morph that is transformed. Actually it doesn?t seem to be; a morph that has been rotated/scaled/etc by manipulating the halo gets a FlexMorph shell built around it and that is dragged and then when you drop the composite the flex shell is stripped away; which seems pretty heavyweight way to do it to be honest. > > > HandMorph>grabMorph:from: is the major player here; it asks the formerOwner for the grabTransform (without specifying for which grabbed morph, which seems odd) and uses that along withe formerOwner?s transform to do things to the grabbed morph. If I make my owner-morph implement grabTransform to return an identity transform, the grabbing works as normal. If I provide atransform that does any scaling (which is what I want to use) then > a) the positions and offsets get all screwy > b) dropping doesn?t work the same - you have to drop the morph and then click-release again to get it to actually drop. That part really confuses me right now. > > Maybe attachMorph: will work easier as you can do the trickery to the morph and just attach it to the hand ? We can?t use attachMorph: since ? oh wait, I see what you mean. I was about to say that grabbing the morph prevents us but of course the code is actually #startDrag: and we get to choose to use attach or grab. Yeah, that should work I guess, though I did some experiments with it before noticing the grabTransform stuff; you can get all sorts of surprises if you wrap your morph in a TransformationMorph and forget to unwrap it on drop. If nobody has any clue about what it was originally for, what it was ever used for or if any surviving code has a use for it then we should remove it (or rework it to do something helpful). tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Useful Latin Phrases:- Nihil curo de ista tua stulta superstitione = I'm not interested in your dopey religious cult. From eliot.miranda at gmail.com Mon Dec 14 01:30:52 2015 From: eliot.miranda at gmail.com (Eliot Miranda) Date: Mon Dec 14 01:30:57 2015 Subject: [squeak-dev] Re: [Vm-dev] DirectoryPlugin updated on SqueakSource In-Reply-To: <20151212225856.GA19870@shell.msen.com> References: <20151212225856.GA19870@shell.msen.com> Message-ID: Hi David, On Sat, Dec 12, 2015 at 2:58 PM, David T. Lewis wrote: > > A few years ago (well, more than a few), I implemented DirectoryPlugin. > This claimed to (and I believe actually did) provide very large performance > benefits on images of that time. It also required some intrusive changes > on the image side, so it really was not appropriate for integration into > the image. > > I recently updated the primitive and provided a separate image side access > class, This is now on SqueakSource.com as package DirectoryPlugin. > > Since this was originally written circa 2003, the Squeak image has > apparently > improved immensely. The directory stream primitives, which were the > original > motivation for writing it, no longer provide any benefit at all. In fact, > they seem to be slightly slower than the standard (modern) image). > > The file stat primitives do provide substantial performance benefit, so > these may still be of interest for modern images. > Then what's involved in integrating only the stat primitives? Both in the plugin and the image? > > Here is the description from SqueakSource: > > ------- > Project Description > > Home page http://wiki.squeak.org/squeak/2274 > > DirectoryPlugin provides access to a Posix directory stream for efficient > access to directory contents. It also provides methods for testing file > status and how to creating directory entries for a files and directories. > > The directory stream primitives use the address of a directory stream as > a handle to identify a directory stream. This permits several directory > streams to be opened and manipulated independently by separate Smalltalk > processes. However, care must be taken to avoid passing the handle for an > invalid directory stream (such as one which has already been closed) to > one of these methods. > > The plugin has been tested on Linux and on Windows. The directory stream > support produces a performance improvement only on Unix (Linux) systems. > The file access primitives provide substantial improvements for both Unix > and Windows systems. Other platforms have not been tested. > > DirectoryPluginAccess provides access to primitives in DirectoryPlugin, > along with utility methods for various directory and file access functions. > > Performance characteristics have changed significantly since Squeak circa > 2003. The readdir() primitives no longer provide any benefit, but the file > testing primitives still yield a couple orders of magnitude improvement > for some functions. > > -- _,,,^..^,,,_ best, Eliot -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151213/040aaedf/attachment.htm From eliot.miranda at gmail.com Mon Dec 14 01:32:47 2015 From: eliot.miranda at gmail.com (Eliot Miranda) Date: Mon Dec 14 01:32:49 2015 Subject: [squeak-dev] squeak: could not find any display driver In-Reply-To: <566CEA66.9070305@gmail.com> References: <566CD232.7070904@gmail.com> <66BA6777-5BD2-4FEE-8754-B476C76CACE3@gmail.com> <566CEA66.9070305@gmail.com> Message-ID: Hi Robert, and email me any text I should add to the HowToBuild on installing needed packages. thx On Sat, Dec 12, 2015 at 7:47 PM, Robert Withers wrote: > It is a renamed binary that was working, but no longer. I just reinstalled > again, 64-bit Ubuntu 15.10. I did NOT upgrade. I ran the script but it was > missing libsm6:i386 and libsm6:i386 and also had a bug. I'll attach the new > one. It all installs but I still have the same issue "could not find any > display driver". > > I also ran the all-in-One and it said "Running 32-bit Squeak on a 64-bit > System. install-libs32 may install them.". I ran that script as well. > > Perhaps it is a 64-bit issue running 32-bits. I will reinstall 32-bit > ubuntu, although my first box was 32-bit before it failed. > > Thanks for lending a hand. > > Best, > Robert > > > > > On 12/12/2015 10:23 PM, Levente Uzonyi wrote: > >> Is cogvm a script or a binary? If it's the former, then you should try >> running the binary. >> You can also try being explicit about the display plugin you want to use. >> E.g.: cogvm --vm-display-X11 Mushroom.4.image >> >> Levente >> >> On Sat, 12 Dec 2015, Robert Withers wrote: >> >> Yes, I learned this today, or rather I was reminded of old UNIX cmds I >>> used to know. The results of this were good. Every dependency was linked. >>> This was why I've been using that script, which covers those 32-bit >>> dependencies pretty thoroughly. >>> >>> Alright >>> --- >>> robert >>> >>> On Dec 12, 2015, at 9:54 PM, Levente Uzonyi >>>> wrote: >>>> >>>> What does "ldd vm-display-X11" print? >>>> >>>> Levente >>>> >>>> On Sat, 12 Dec 2015, Robert Withers wrote: >>>>> >>>>> Are you tired of seeing this result: >>>>> rabbit@rabbithole:~/warren/cogvm/image$ cogvm Mushroom.4.image >>>>> squeak: could not find any display driver >>>>> Aborted (core dumped) >>>>> I am! Thank you for the help today. Unfortunately this is still an >>>>> issue for me on both laptops: 32-bit and 64-bit OS. Both were working as >>>>> well. I reinstalled the 64-bit and it failed right off the start. The >>>>> commonality to all three failures was the fact that I did >>>>> these below commands prior to it failing this way, everytime. I think >>>>> it is the upgrade that does it in. I'll reinstall 32-bit and try without >>>>> these commands; let you know. >>>>> Robert >>>>> sudo apt-get update >>>>> sudo apt-get upgrade >>>>> and I ran this script for system libraries for 32-bit squeak. >>>>> #!/bin/bash >>>>> >>>>> # ARGUMENT HANDLING >>>>> ============================================================= >>>>> if { [ "$1" = "-h" ] || [ "$1" = "--help" ]; }; then >>>>> echo "Install libraries required to build the pharo VM under >>>>> ubuntu. >>>>> " >>>>> exit 0; >>>>> elif [ $# -gt 0 ]; then >>>>> echo "--help/-h is the only argument allowed" >>>>> exit 1; >>>>> fi >>>>> >>>>> # INSTALL BUILD LIBRARIES >>>>> ====================================================== >>>>> sudo apt-get install cmake zip bash-completion ruby git xz-utils >>>>> debhelper devscripts >>>>> sudo apt-get install libc6-dev:i386 libasound2:i386 >>>>> libasound2-dev:i386 libasound2-plugins:i386 libssl-dev:i386 >>>>> i#!libssl0.9.8:i386 libfreetype6-dev:i386 libx11-dev:i386 libsm-dev:i386 >>>>> libice-dev:i386 >>>>> sudo apt-get install build-essential gcc-multilib g++ >>>>> sudo apt-get install libgl1-mesa-dev libgl1-mesa-glx:i386 >>>>> sudo apt-get install binutils:i386 g++-4.6:i386 gcc-4.6:i386 >>>>> sudo ln -s /usr/lib/i386-linux-gnu/mesa/libGL.so >>>>> /usr/lib/i386-linux-gnu/libGL.so >>>>> sudo ln -s /usr/lib/i386-linux-gnu/mesa/libGL.so.1 >>>>> /usr/lib/i386-linux-gnu/mesa/libGL.so >>>>> sudo apt-get install uuid-dev:i386 >>>>> sudo apt-get install subversion >>>>> >>>> >>>> >>> >>> >> > > > > -- _,,,^..^,,,_ best, Eliot -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151213/1de4030e/attachment.htm From lewis at mail.msen.com Mon Dec 14 03:01:12 2015 From: lewis at mail.msen.com (David T. Lewis) Date: Mon Dec 14 03:01:14 2015 Subject: [squeak-dev] Re: [Vm-dev] DirectoryPlugin updated on SqueakSource In-Reply-To: References: <20151212225856.GA19870@shell.msen.com> Message-ID: <20151214030112.GB87089@shell.msen.com> On Sun, Dec 13, 2015 at 05:30:52PM -0800, Eliot Miranda wrote: > > Hi David, > > On Sat, Dec 12, 2015 at 2:58 PM, David T. Lewis wrote: > > > > > A few years ago (well, more than a few), I implemented DirectoryPlugin. > > This claimed to (and I believe actually did) provide very large performance > > benefits on images of that time. It also required some intrusive changes > > on the image side, so it really was not appropriate for integration into > > the image. > > > > I recently updated the primitive and provided a separate image side access > > class, This is now on SqueakSource.com as package DirectoryPlugin. > > > > Since this was originally written circa 2003, the Squeak image has > > apparently > > improved immensely. The directory stream primitives, which were the > > original > > motivation for writing it, no longer provide any benefit at all. In fact, > > they seem to be slightly slower than the standard (modern) image). > > > > The file stat primitives do provide substantial performance benefit, so > > these may still be of interest for modern images. > > > > Then what's involved in integrating only the stat primitives? Both in the > plugin and the image? > For the plugin, we could just remove the four directory stream primitives, leave the remaining ten file stat primitives, and rename the module to something like 'FileStatPlugin'. For the image, the ten example methods in DirectoryPluginAccess class in category 'file primitive access' illustrate how to access the primitives. The tricky part is that these are unix/posix only, so they would have to be optional primitives and integrated into the image in such a way that things still work reasonably on other platforms. I have not really looked into that aspect of the problem since 2003, but it might be worth a second look now. Dave From robert.w.withers at gmail.com Mon Dec 14 06:30:58 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Mon Dec 14 06:31:03 2015 Subject: [squeak-dev] squeak: could not find any display driver In-Reply-To: References: <566CD232.7070904@gmail.com> <66BA6777-5BD2-4FEE-8754-B476C76CACE3@gmail.com> <566CEA66.9070305@gmail.com> Message-ID: <566E6222.7050200@gmail.com> Hi Eliot, I just posted this to Ask Ubuntu and I let you know the results. http://askubuntu.com/questions/709301/squeak-could-not-find-any-display-driver Robert http://askubuntu.com/questions/709301/squeak-could-not-find-any-display-driver On 12/13/2015 08:32 PM, Eliot Miranda wrote: > Hi Robert, > > and email me any text I should add to the HowToBuild on installing > needed packages. thx > > On Sat, Dec 12, 2015 at 7:47 PM, Robert Withers > > wrote: > > It is a renamed binary that was working, but no longer. I just > reinstalled again, 64-bit Ubuntu 15.10. I did NOT upgrade. I ran > the script but it was missing libsm6:i386 and libsm6:i386 and also > had a bug. I'll attach the new one. It all installs but I still > have the same issue "could not find any display driver". > > I also ran the all-in-One and it said "Running 32-bit Squeak on a > 64-bit System. install-libs32 may install them.". I ran that > script as well. > > Perhaps it is a 64-bit issue running 32-bits. I will reinstall > 32-bit ubuntu, although my first box was 32-bit before it failed. > > Thanks for lending a hand. > > Best, > Robert > > > > > On 12/12/2015 10:23 PM, Levente Uzonyi wrote: > > Is cogvm a script or a binary? If it's the former, then you > should try running the binary. > You can also try being explicit about the display plugin you > want to use. E.g.: cogvm --vm-display-X11 Mushroom.4.image > > Levente > > On Sat, 12 Dec 2015, Robert Withers wrote: > > Yes, I learned this today, or rather I was reminded of old > UNIX cmds I used to know. The results of this were good. > Every dependency was linked. This was why I've been using > that script, which covers those 32-bit dependencies pretty > thoroughly. > > Alright > --- > robert > > On Dec 12, 2015, at 9:54 PM, Levente Uzonyi > > > wrote: > > What does "ldd vm-display-X11" print? > > Levente > > On Sat, 12 Dec 2015, Robert Withers wrote: > > Are you tired of seeing this result: > rabbit@rabbithole:~/warren/cogvm/image$ cogvm > Mushroom.4.image > squeak: could not find any display driver > Aborted (core dumped) > I am! Thank you for the help today. Unfortunately > this is still an issue for me on both laptops: > 32-bit and 64-bit OS. Both were working as well. > I reinstalled the 64-bit and it failed right off > the start. The commonality to all three failures > was the fact that I did > these below commands prior to it failing this way, > everytime. I think it is the upgrade that does it > in. I'll reinstall 32-bit and try without these > commands; let you know. > Robert > sudo apt-get update > sudo apt-get upgrade > and I ran this script for system libraries for > 32-bit squeak. > #!/bin/bash > > # ARGUMENT HANDLING > ============================================================= > if { [ "$1" = "-h" ] || [ "$1" = "--help" ]; > }; then > echo "Install libraries required to build > the pharo VM under ubuntu. > " > exit 0; > elif [ $# -gt 0 ]; then > echo "--help/-h is the only argument allowed" > exit 1; > fi > > # INSTALL BUILD LIBRARIES > ====================================================== > sudo apt-get install cmake zip > bash-completion ruby git xz-utils debhelper devscripts > sudo apt-get install libc6-dev:i386 > libasound2:i386 libasound2-dev:i386 > libasound2-plugins:i386 libssl-dev:i386 > i#!libssl0.9.8:i386 libfreetype6-dev:i386 > libx11-dev:i386 libsm-dev:i386 libice-dev:i386 > sudo apt-get install build-essential > gcc-multilib g++ > sudo apt-get install libgl1-mesa-dev > libgl1-mesa-glx:i386 > sudo apt-get install binutils:i386 > g++-4.6:i386 gcc-4.6:i386 > sudo ln -s > /usr/lib/i386-linux-gnu/mesa/libGL.so > /usr/lib/i386-linux-gnu/libGL.so > sudo ln -s > /usr/lib/i386-linux-gnu/mesa/libGL.so.1 > /usr/lib/i386-linux-gnu/mesa/libGL.so > sudo apt-get install uuid-dev:i386 > sudo apt-get install subversion > > > > > > > > > > > > > -- > _,,,^..^,,,_ > best, Eliot > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151214/618f7ab3/attachment.htm From nicolaihess at gmail.com Mon Dec 14 09:26:20 2015 From: nicolaihess at gmail.com (Nicolai Hess) Date: Mon Dec 14 09:26:24 2015 Subject: [squeak-dev] HandMorph etc #grabTransform In-Reply-To: <8746E820-9425-4960-BCAE-ADD807CA5CE3@rowledge.org> References: <6D41EA3D-8470-4B02-B0E7-CC63CF0C4304@rowledge.org> <8746E820-9425-4960-BCAE-ADD807CA5CE3@rowledge.org> Message-ID: 2015-12-14 2:19 GMT+01:00 tim Rowledge : > > > On 12-12-2015, at 2:53 AM, karl ramberg wrote: > > > > > > > > On Thu, Dec 10, 2015 at 11:47 PM, tim Rowledge wrote: > > Has anybody used the #grabTransform message that is sent by HandMorph as > part of picking up a Morph? Every use I can find of it seems to set it to > an Identity transform and trying to use it for make my dragged morph appear > half-size is not working in a very understandable manner. > > > > I think this is used for moving a morph that is transformed and also > grabbing a duplicated morph that is transformed. > > Actually it doesn?t seem to be; It makes a difference, if the visible position and orientation is a composition from multiple transformed morphs. For example: e:=EllipseMorph new . e addMorphBack:StringMorph new. e openInHand Now - show halo for ellipse morph - rotate the ellipse morph - show halo for string morph - rotate string morph - grab or duplicate the string morph -> new string morphs transform automatically gets the combined transformation of the original string morph and the ellipse morph. > a morph that has been rotated/scaled/etc by manipulating the halo gets a > FlexMorph shell built around it and that is dragged and then when you drop > the composite the flex shell is stripped away; which seems pretty > heavyweight way to do it to be honest. > > > > > > > HandMorph>grabMorph:from: is the major player here; it asks the > formerOwner for the grabTransform (without specifying for which grabbed > morph, which seems odd) and uses that along withe formerOwner?s transform > to do things to the grabbed morph. If I make my owner-morph implement > grabTransform to return an identity transform, the grabbing works as > normal. If I provide atransform that does any scaling (which is what I want > to use) then > > a) the positions and offsets get all screwy > > b) dropping doesn?t work the same - you have to drop the morph and then > click-release again to get it to actually drop. That part really confuses > me right now. > > > > Maybe attachMorph: will work easier as you can do the trickery to the > morph and just attach it to the hand ? > > We can?t use attachMorph: since ? oh wait, I see what you mean. I was > about to say that grabbing the morph prevents us but of course the code is > actually #startDrag: and we get to choose to use attach or grab. Yeah, that > should work I guess, though I did some experiments with it before noticing > the grabTransform stuff; you can get all sorts of surprises if you wrap > your morph in a TransformationMorph and forget to unwrap it on drop. > > If nobody has any clue about what it was originally for, what it was ever > used for or if any surviving code has a use for it then we should remove it > (or rework it to do something helpful). > > > tim > -- > tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim > Useful Latin Phrases:- Nihil curo de ista tua stulta superstitione = I'm > not interested in your dopey religious cult. > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151214/56375124/attachment.htm From brasspen at gmail.com Mon Dec 14 16:25:00 2015 From: brasspen at gmail.com (Chris Cunnington) Date: Mon Dec 14 16:25:06 2015 Subject: [squeak-dev] Merry Christmas from OopluCon 2016 Message-ID: <566EED5C.9080907@gmail.com> OopluCon 2016 A Day for the Smalltalk Programming Language and its Friends Sunday 10 April 2016 8am - 4pm The Box SF 1069 Howard Street (between 6th and 7th) three blocks from Moscone Center, SoMa San Francisco, USA http://www.theboxsf.com/ * Dr. Eliot Miranda will deliver the keynote presentation Cog - The Open Source Smaltalk VM * Chris Thorgrimsson of Lam Research is presenting. Talk name to be announced in 2016 * Chris Cunnington is presenting An Introduction To Smalltalk * a request for further presentations will be announced on 11 Jan. 2016 * this event will be promoted to the Ruby, Python, JavaScript and Objective C communities * the event is free * The latest details will be available at http://www.ooplu.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151214/3067ab2a/attachment.htm From robert.w.withers at gmail.com Mon Dec 14 16:46:34 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Mon Dec 14 16:46:39 2015 Subject: [squeak-dev] Re: Crypto plugins for ubuntu 32-bits In-Reply-To: <5662527E.8010509@gmail.com> References: <5662527E.8010509@gmail.com> Message-ID: <566EF26A.8020307@gmail.com> On 12/04/2015 09:57 PM, Robert Withers wrote: > I loaded the CryptographyPlugins package, from the Cryptography > repository into a VMMaker image, and generated the crypto plugins > externally. Although I have not measured it rigorously, there is a > noticeable performance improvement with SqueakElib tests. > > Untar these Ubuntu 32-bit crypto plugins into > /usr/local/lib/squeak/5.0... > > https://www.dropbox.com/s/v502zudwsbtr243/CryptoPlugins.ubuntu32.12.4.15.tgz?dl=0 > > > Let me know any issues. > Good morning, Did anyone happen to grab this tgz file? I have since deleted the file and I don't have a copy. If you did grab it, could you email it to me, out of band? I am unable to build it again, bright now. --- ... ^,^ best, robert > Regards, > Robert From kilon.alios at gmail.com Mon Dec 14 17:05:04 2015 From: kilon.alios at gmail.com (Dimitris Chloupis) Date: Mon Dec 14 17:05:17 2015 Subject: [squeak-dev] Re: [Vm-dev] Merry Christmas from OopluCon 2016 In-Reply-To: <566EED5C.9080907@gmail.com> References: <566EED5C.9080907@gmail.com> Message-ID: Any chance there will be videos uploaded for us that can't attend ? On Mon, 14 Dec 2015 at 18:25, Chris Cunnington wrote: > > OopluCon 2016 > A Day for the Smalltalk Programming Language and its Friends > > Sunday 10 April 2016 > 8am - 4pm > > The Box SF > 1069 Howard Street (between 6th and 7th) > three blocks from Moscone Center, SoMa > San Francisco, USA > http://www.theboxsf.com/ > > > - Dr. Eliot Miranda will deliver the keynote presentation Cog - The > Open Source Smaltalk VM > - Chris Thorgrimsson of Lam Research is presenting. Talk name to be > announced in 2016 > - Chris Cunnington is presenting An Introduction To Smalltalk > - a request for further presentations will be announced on 11 Jan. 2016 > - this event will be promoted to the Ruby, Python, JavaScript and > Objective C communities > - the event is free > - The latest details will be available at http://www.ooplu.com > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151214/30994106/attachment-0001.htm From brasspen at gmail.com Mon Dec 14 17:38:36 2015 From: brasspen at gmail.com (Chris Cunnington) Date: Mon Dec 14 17:38:44 2015 Subject: [squeak-dev] Re: [Vm-dev] Merry Christmas from OopluCon 2016 In-Reply-To: References: <566EED5C.9080907@gmail.com> Message-ID: <1D968CCE-9AA5-44E4-B536-6EEBC23CCB18@gmail.com> > On Dec 14, 2015, at 12:05 PM, Dimitris Chloupis wrote: > > Any chance there will be videos uploaded for us that can't attend ? I?d be happy to have somebody shoot a video, but at the moment the entire staff is me alone and I?m not looking to add more variables this first year. If somebody from Hasso Plattner Stanford wants to do that, then super. I?m sort of at my capacity. For example ? I will add a second ?l? to fix the spelling mistake on my webpage for the event. I really should know how to spell Smalltalk. But doing these things, being who I am, means I?m going to make silly mistakes along the way. Chris > On Mon, 14 Dec 2015 at 18:25, Chris Cunnington > wrote: > > OopluCon 2016 > A Day for the Smalltalk Programming Language and its Friends > > Sunday 10 April 2016 > 8am - 4pm > > The Box SF > 1069 Howard Street (between 6th and 7th) > three blocks from Moscone Center, SoMa > San Francisco, USA > http://www.theboxsf.com/ > > Dr. Eliot Miranda will deliver the keynote presentation Cog - The Open Source Smaltalk VM > Chris Thorgrimsson of Lam Research is presenting. Talk name to be announced in 2016 > Chris Cunnington is presenting An Introduction To Smalltalk > a request for further presentations will be announced on 11 Jan. 2016 > this event will be promoted to the Ruby, Python, JavaScript and Objective C communities > the event is free > The latest details will be available at http://www.ooplu.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151214/8aac8646/attachment.htm From kilon.alios at gmail.com Mon Dec 14 17:41:38 2015 From: kilon.alios at gmail.com (Dimitris Chloupis) Date: Mon Dec 14 17:41:52 2015 Subject: [squeak-dev] Re: [Vm-dev] Merry Christmas from OopluCon 2016 In-Reply-To: <1D968CCE-9AA5-44E4-B536-6EEBC23CCB18@gmail.com> References: <566EED5C.9080907@gmail.com> <1D968CCE-9AA5-44E4-B536-6EEBC23CCB18@gmail.com> Message-ID: That's ok it's great you organize such things and don't worry about mistakes , effort counts the most :) On Mon, 14 Dec 2015 at 19:38, Chris Cunnington wrote: > On Dec 14, 2015, at 12:05 PM, Dimitris Chloupis > wrote: > > Any chance there will be videos uploaded for us that can't attend ? > > > I?d be happy to have somebody shoot a video, but at the moment the entire > staff is me alone and I?m not looking to add more variables this first > year. If somebody from Hasso Plattner Stanford wants to do that, then > super. I?m sort of at my capacity. For example ? > > I will add a second ?l? to fix the spelling mistake on my webpage for the > event. I really should know how to spell Smalltalk. But doing these things, > being who I am, means I?m going to make silly mistakes along the way. > > Chris > > On Mon, 14 Dec 2015 at 18:25, Chris Cunnington wrote: > >> >> OopluCon 2016 >> A Day for the Smalltalk Programming Language and its Friends >> >> Sunday 10 April 2016 >> 8am - 4pm >> >> The Box SF >> 1069 Howard Street (between 6th and 7th) >> three blocks from Moscone Center, SoMa >> San Francisco, USA >> http://www.theboxsf.com/ >> >> >> - Dr. Eliot Miranda will deliver the keynote presentation Cog - The >> Open Source Smaltalk VM >> - Chris Thorgrimsson of Lam Research is presenting. Talk name to be >> announced in 2016 >> - Chris Cunnington is presenting An Introduction To Smalltalk >> - a request for further presentations will be announced on 11 Jan. >> 2016 >> - this event will be promoted to the Ruby, Python, JavaScript and >> Objective C communities >> - the event is free >> - The latest details will be available at >> http://www.ooplu.com >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151214/ecd565a8/attachment.htm From bert at freudenbergs.de Mon Dec 14 18:22:27 2015 From: bert at freudenbergs.de (Bert Freudenberg) Date: Mon Dec 14 18:22:30 2015 Subject: [squeak-dev] drawing an arc In-Reply-To: References: Message-ID: <0BA25E7C-C635-4803-9870-E91E3E001A65@freudenbergs.de> On 12.12.2015, at 16:37, Chris Muller wrote: > > Does anyone know how to draw an arc on a Canvas? It has drawOval / fillOval, but I want to draw a piece of pie. I don?t think we have that. In the Etoys image there is a SectorMorph but it?s a subclass of PolygonMorph, updating its vertices, not drawing directly on the canvas. - Bert - -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4115 bytes Desc: not available Url : http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151214/1aff3416/smime.bin From robert.w.withers at gmail.com Mon Dec 14 18:27:55 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Mon Dec 14 18:28:00 2015 Subject: [squeak-dev] Re: Crypto plugins for ubuntu 32-bits In-Reply-To: <566EF26A.8020307@gmail.com> References: <5662527E.8010509@gmail.com> <566EF26A.8020307@gmail.com> Message-ID: <566F0A2B.1030302@gmail.com> I should mention I have never tested Mushroom performance on my new dual-core laptop with plugins, yet. I got to 20.1 Mbs on the old laptop so I am anxious. I hope I didn;t come across as such poorly. best, robert On 12/14/2015 11:46 AM, Robert Withers wrote: > > On 12/04/2015 09:57 PM, Robert Withers wrote: >> I loaded the CryptographyPlugins package, from the Cryptography >> repository into a VMMaker image, and generated the crypto plugins >> externally. Although I have not measured it rigorously, there is a >> noticeable performance improvement with SqueakElib tests. >> >> Untar these Ubuntu 32-bit crypto plugins into >> /usr/local/lib/squeak/5.0... >> >> https://www.dropbox.com/s/v502zudwsbtr243/CryptoPlugins.ubuntu32.12.4.15.tgz?dl=0 >> >> >> Let me know any issues. >> > > Good morning, > > Did anyone happen to grab this tgz file? I have since deleted the file > and I don't have a copy. If you did grab it, could you email it to me, > out of band? I am unable to build it again, bright now. > > > --- > ... ^,^ best, robert > >> I should mention thRegards, >> Robert > -- ... ^,^ best, robert From bert at freudenbergs.de Mon Dec 14 21:35:26 2015 From: bert at freudenbergs.de (Bert Freudenberg) Date: Mon Dec 14 21:35:29 2015 Subject: [squeak-dev] bitshift: In-Reply-To: <6561B0F0C9D048C48A4EAE8160D0FF30@LSWWin764NB> References: <566B238B.6010807@gmail.com> <6E15F02ADD7542FF99002AFF9C58723B@LSWWin764NB> <92A7EEEA2A8A475C9735EFEEB13157CE@LSWWin764NB> <6BFA1764-1200-48DA-8898-5C22B0A2F53E@gmail.com> <6561B0F0C9D048C48A4EAE8160D0FF30@LSWWin764NB> Message-ID: <177F8819-441A-48F0-B967-A77422736D34@freudenbergs.de> Skipped content of type multipart/alternative-------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4115 bytes Desc: not available Url : http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151214/234210f7/smime.bin From frank-lesser at lesser-software.com Mon Dec 14 22:28:12 2015 From: frank-lesser at lesser-software.com (Frank Lesser) Date: Mon Dec 14 22:28:34 2015 Subject: [squeak-dev] bitshift: In-Reply-To: <177F8819-441A-48F0-B967-A77422736D34@freudenbergs.de> References: <566B238B.6010807@gmail.com><6E15F02ADD7542FF99002AFF9C58723B@LSWWin764NB><92A7EEEA2A8A475C9735EFEEB13157CE@LSWWin764NB><6BFA1764-1200-48DA-8898-5C22B0A2F53E@gmail.com><6561B0F0C9D048C48A4EAE8160D0FF30@LSWWin764NB> <177F8819-441A-48F0-B967-A77422736D34@freudenbergs.de> Message-ID: Hi Bert, yes funny, but to be correct, in Dolphin bitShift: is correct ( so far our tests covered it ), in Squeak, Pharo it is also correct, in Digitalk it is broken, in Dolphin Integer division prim is broken. Other Smalltalk's I haven't tested yet. Frank _____ Von: squeak-dev-bounces@lists.squeakfoundation.org [mailto:squeak-dev-bounces@lists.squeakfoundation.org] Im Auftrag von Bert Freudenberg Gesendet: Montag, 14. Dezember 2015 22:35 An: The general-purpose Squeak developers list Betreff: Re: [squeak-dev] bitshift: Fun fact - in JavaScript, this is true: (-0x10000001 >> 32) == -0x10000001 - Bert - On 11.12.2015, at 22:22, Frank Lesser wrote: Hi Tim, thx for fast answer, need to get sorted my newsgroups as it was posted wrong. It is wrong in Dolphin: ( -16r10000001 bitShift: -32 )-1 ( -16r10000000 bitShift: -32 )-1 ( -16r1000001 bitShift: -24 ) -2 ( -16r1000000 bitShift: -24 ) -1 but according to Dolphin's comment it is correct. Frank _____ Von: squeak-dev-bounces@lists.squeakfoundation.org [mailto:squeak-dev-bounces@lists.squeakfoundation.org] Im Auftrag von Tim Olson Gesendet: Freitag, 11. Dezember 2015 21:16 An: The general-purpose Squeak developers list Betreff: Re: [squeak-dev] bitshift: On Dec 11, 2015, at 2:00 PM, Frank Lesser wrote: Hi, ( -16r100000001 bitShift: -32 ) = -2 ( -16r100000000 bitShift: -32 ) = -1 looks odd to me, guess the prim needs to be fixed Looks correct to me: (-16r100000001 bitAnd: 16rFFFFFFFFFF) hex -> '16rFEFFFFFFFF' downshift this by 32 bits and you get 16rFFFFFFFE, which is -2 (-16r100000000 bitAnd: 16rFFFFFFFFFF) hex -> '16rFF00000000' downshift this by 32 bits and you get 16rFFFFFFFF, which is -1. - tim -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151214/b9a16653/attachment.htm From robert.w.withers at gmail.com Mon Dec 14 23:08:52 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Mon Dec 14 23:08:58 2015 Subject: [squeak-dev] Re: Crypto plugins for ubuntu 32-bits In-Reply-To: <566EF26A.8020307@gmail.com> References: <5662527E.8010509@gmail.com> <566EF26A.8020307@gmail.com> Message-ID: <566F4C04.1000000@gmail.com> If not my zip, could someone zip up their crypto plugins and send them to me, please. This would make me happy after more bad news. regards, robert On 12/14/2015 11:46 AM, Robert Withers wrote: > > On 12/04/2015 09:57 PM, Robert Withers wrote: >> I loaded the CryptographyPlugins package, from the Cryptography >> repository into a VMMaker image, and generated the crypto plugins >> externally. Although I have not measured it rigorously, there is a >> noticeable performance improvement with SqueakElib tests. >> >> Untar these Ubuntu 32-bit crypto plugins into >> /usr/local/lib/squeak/5.0... >> >> https://www.dropbox.com/s/v502zudwsbtr243/CryptoPlugins.ubuntu32.12.4.15.tgz?dl=0 >> >> >> Let me know any issues. >> > > Good morning, > > Did anyone happen to grab this tgz file? I have since deleted the file > and I don't have a copy. If you did grab it, could you email it to me, > out of band? I am unable to build it again, bright now. > > > --- > ... ^,^ best, robert > >> Regards, >> Robert > -- ... ^,^ best, robert From lewis at mail.msen.com Mon Dec 14 23:52:08 2015 From: lewis at mail.msen.com (David T. Lewis) Date: Mon Dec 14 23:53:02 2015 Subject: [squeak-dev] Re: Crypto plugins for ubuntu 32-bits In-Reply-To: <566F4C04.1000000@gmail.com> References: <5662527E.8010509@gmail.com> <566EF26A.8020307@gmail.com> <566F4C04.1000000@gmail.com> Message-ID: <20151214235208.GA21413@shell.msen.com> Hi Robert, Attached is a copy of the tgz that I downloaded from your dropbox link on December 5. Dave On Mon, Dec 14, 2015 at 06:08:52PM -0500, Robert Withers wrote: > If not my zip, could someone zip up their crypto plugins and send them > to me, please. This would make me happy after more bad news. > > regards, > robert > > On 12/14/2015 11:46 AM, Robert Withers wrote: > > > >On 12/04/2015 09:57 PM, Robert Withers wrote: > >>I loaded the CryptographyPlugins package, from the Cryptography > >>repository into a VMMaker image, and generated the crypto plugins > >>externally. Although I have not measured it rigorously, there is a > >>noticeable performance improvement with SqueakElib tests. > >> > >>Untar these Ubuntu 32-bit crypto plugins into > >>/usr/local/lib/squeak/5.0... > >> > >>https://www.dropbox.com/s/v502zudwsbtr243/CryptoPlugins.ubuntu32.12.4.15.tgz?dl=0 > >> > >> > >>Let me know any issues. > >> > > > >Good morning, > > > >Did anyone happen to grab this tgz file? I have since deleted the file > >and I don't have a copy. If you did grab it, could you email it to me, > >out of band? I am unable to build it again, bright now. > > > > > >--- > >... ^,^ best, robert > > > >>Regards, > >>Robert > > > > -- > ... ^,^ best, robert -------------- next part -------------- A non-text attachment was scrubbed... Name: CryptoPlugins.ubuntu32.12.4.15.tgz Type: application/x-gtar Size: 45460 bytes Desc: not available Url : http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151214/f4038d9a/CryptoPlugins.ubuntu32.12.4.15-0001.tgz From robert.w.withers at gmail.com Tue Dec 15 00:53:30 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Tue Dec 15 00:53:35 2015 Subject: [Pharo-dev] [squeak-dev] Re: Crypto plugins for ubuntu 32-bits In-Reply-To: <20151214235208.GA21413@shell.msen.com> References: <5662527E.8010509@gmail.com> <566EF26A.8020307@gmail.com> <566F4C04.1000000@gmail.com> <20151214235208.GA21413@shell.msen.com> Message-ID: <566F648A.8030507@gmail.com> Skipped content of type multipart/alternative-------------- next part -------------- A non-text attachment was scrubbed... Name: Spy Results.text.gz Type: application/gzip Size: 1924 bytes Desc: not available Url : http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151214/dde375d5/SpyResults.text.bin From robert.w.withers at gmail.com Tue Dec 15 10:00:12 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Tue Dec 15 10:00:18 2015 Subject: [squeak-dev] potential to include cryptography in base image Message-ID: <566FE4AC.4060803@gmail.com> Skipped content of type multipart/alternative-------------- next part -------------- A non-text attachment was scrubbed... Name: Cryptography Spy Results.text.gz Type: application/gzip Size: 1661 bytes Desc: not available Url : http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151215/dcb220a6/CryptographySpyResults.text.bin From robert.w.withers at gmail.com Tue Dec 15 10:44:20 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Tue Dec 15 10:44:25 2015 Subject: [squeak-dev] Re: potential to include cryptography in base image In-Reply-To: <7E133BBD-690B-4ACD-9886-E0F610F20F08@stfx.eu> References: <566FE4AC.4060803@gmail.com> <7E133BBD-690B-4ACD-9886-E0F610F20F08@stfx.eu> Message-ID: <566FEF04.7020504@gmail.com> Thank you, Sven, giving me the opportunity to scope the right mailing lists into the discussion. I love that Pharo is oriented towards business. My particular excitement is to see all the scientific computing. These recent threads on expanding capacity and targeting marketing are good. I think Cryptography in the base, including SecureSession, adds to that effort. The Cryptography package, both for squeak and the Cryptography-Pharo package, and in the Cryptography repository (http://www.squeaksource.com/Cryptography. Through the Monticello Browser this is a loadable item. The loading of SecureSession, which is loadable by both squeak and pharo, ois a separate load. I appreciate that pharo has established a different mechanism for loading. At this time, I am deep into fixing ReedSolomon FEC code, so I must be conservative (which I am) and defer learning and including Cryptography in that mechanism. I must be careful what promises I make and I cannot make a commitment here. If you are interested in seeing this in pharo's catalog, then the Cryptography team welcomes new members, especially from new environments. Welcome. We appreciate all the work that you do! Again, I appreciate the opportunity to engage the community in discussion around the definition and example of a minimal modular image that includes Cryptography & SecureSession in the base. NB: I think that business computing and scientific computing does not equal mobile devices, necessarily. I have long made the case, with varying degrees of effectiveness, that squeak belongs on the server. A SecureSession & Cryptography capability in the base, minimal, headless image is desired: the first question in the OP. Therefore, the correct target of growth is not mobile, it is BigData! On 12/15/2015 05:20 AM, Sven Van Caekenberghe wrote: > Robert, > > I think that the Pharo community, part of which is more business oriented, is absolutely interested in more and better Crypto code. In any case, I am. > > What we absolutely want, if it is not already the case (I did not check), is that the Crypto code can be loaded using 1 single action (through our validated Monticello configurations and Catalog mechanism) - I am sure you will find help to achieve and maintain (through a CI process) that goal. > > Whether it should be a base part of the image is another question. Modularity is a huge goal for Pharo. This is a much harder discussion (as the same can be said of or asked for for many packages that are generally useful: XML, CSV, JSON, SQL, ...). In any the case, the first step is the one described in the previous paragraph. Then you need traction, usage, and maybe demand for full inclusion. > > Regards, > > Sven > >> On 15 Dec 2015, at 11:00, Robert Withers wrote: >> >> It was suggested to me that I write to the list and raise the question about cryptography being included in the base image. Really I have 3 questions I would ask you all: >> >> ? is it desirable to include cryptography? >> ? is it feasible to include cryptography? >> ? what is the time frame for including cryptography? >> Given the thread on password hashing (and salting and so on), there are good, solid implementations in the cryptography package. Looking in the Cryptography repository, there is a Pharo 5.0 compatible Cryptography package. >> >> In light of another recent thread discussing random number generation, discussion about the best approach to random algorithms in cryptography ought be engaged. For instance, the SecureRandom algorithm evidently provides some level of guarantee. >> >> To underline the solidity I am attaching a profile of all 102 cryptography tests passing green. This profile demonstrates that there are no areas of particular inefficiency - nothing stands out to be improved - means that the entire library is maximally efficient. >> >> And so I please ask that we have these discussions, for there is a lot of value in this package for general and basic use. >> >> >> -- >> . .. ... ^,^ best, robert >> -- . .. .. ^,^ best, robert From robert.w.withers at gmail.com Tue Dec 15 11:09:03 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Tue Dec 15 11:09:08 2015 Subject: [squeak-dev] Looking for a working mailreader Message-ID: <566FF4CF.4020402@gmail.com> Is there such for squeak? I tried Celeste but it didn't load correctly. I prefer all my tools tin squeak for immersion and full screen experience. -- . .. .. ^,^ best, robert From hannes.hirzel at gmail.com Tue Dec 15 11:15:59 2015 From: hannes.hirzel at gmail.com (H. Hirzel) Date: Tue Dec 15 11:16:02 2015 Subject: [squeak-dev] Re: [Pharo-dev] Unicode Support - Normalization NFD -- #decomposedStringWithCanonicalMapping Message-ID: On 12/13/15, stepharo wrote: > >> It is best and (mostly) correct to think of a Unicode string as a sequence >> of Unicode characters, each defined/identified by a code point (out of >> 10.000s covering all languages). That is what we have today in Pharo (with >> the distinction between ByteString and WideString as mostly invisible >> implementation details). >> >> To encode Unicode for external representation as bytes, we use UTF-8 like >> the rest of the modern world. >> So far, so good. >> ...> > > like me ;) > I will wait for a conclusion with code :) > > Stef > Some code with illustration by the e acute example see below >> Why then is there confusion about the seemingly simple concept of a >> character ? Because Unicode allows different ways to say the same thing. >> The simplest example in a common language is (the French letter ?) is >> >> LATIN SMALL LETTER E WITH ACUTE [U+00E9] >> >> which can also be written as >> >> LATIN SMALL LETTER E [U+0065] followed by COMBINING ACUTE ACCENT [U+0301] >> >> The former being a composed normal form, the latter a decomposed normal >> form. (And yes, it is even much more complicated than that, it goes on for >> 1000s of pages). >> >> In the above example, the concept of character/string is indeed fuzzy. >> >> HTH, >> >> Sven The text below shows how to deal with the Unicode e acute example brought up by Sven in terms of comparing strings. Currently Pharo and Cuis do not do Normalization of strings. Limited support is in Squeak. It will be shown how NFD normalization may be implemented. Swift programming language ----------------------------------------- How does the Swift programming language [1] deal with Unicode strings? // "Voulez-vous un caf??" using LATIN SMALL LETTER E WITH ACUTE let eAcuteQuestion = "Voulez-vous un caf\u{E9}?" // "Voulez-vous un café?" using LATIN SMALL LETTER E and COMBINING ACUTE ACCENT let combinedEAcuteQuestion = "Voulez-vous un caf\u{65}\u{301}?" if eAcuteQuestion == combinedEAcuteQuestion { print("These two strings are considered equal") } // prints "These two strings are considered equal" The equality operator uses the NFD (Normalization Form Decomposed)[2] form for the comparison appyling a method #decomposedStringWithCanonicalMapping[3] Squeak / Pharo ----------------------- Comparison without NFD [3] "Voulez-vous un caf??" eAcuteQuestion := 'Voulez-vous un caf', 16rE9 asCharacter asString, '?'. combinedEAcuteQuestion := 'Voulez-vous un cafe', 16r301 asCharacter asString, '?'. eAcuteQuestion = combinedEAcuteQuestion false eAcuteQuestion == combinedEAcuteQuestion false The result is false. A Unicode conformant application however should return *true*. Reason for this is that Squeak / Pharo strings are not put into NFD before testing for equality = Squeak Unicode strings may be tested for Unicode conformant equality by converting them to NFD before testing. Squeak using NFD asDecomposedUnicode[4] transforms a string into NFD for cases where a Unicode code point if decomposed, is decomposed only to two code points [5]. This is so because when initializing [6] the Unicode Character Database in Squeak this is a limitation imposed by the code which reads UnicodeData.txt [7][8]. This is not a necessary limitation. The code may be rewritten at the price of a more complex implementation of #asDecomposedUnicode. "Voulez-vous un caf??" eAcuteQuestion := 'Voulez-vous un caf', 16rE9 asCharacter asString, '?'. combinedEAcuteQuestion := 'Voulez-vous un cafe', 16r301 asCharacter asString, '?'. eAcuteQuestion asDecomposedUnicode = combinedEAcuteQuestion asDecomposedUnicode true Conclusion ------------------ Implementing a method like #decomposedStringWithCanonicalMapping (swift) which puts a string into NFD (Normalization Form D) is an important building block towards better Unicode compliance. A Squeak proposal is given by [4]. It needs to be reviewed.extended. It should probably be extended for cases where there are more than two code points in the decomposed form (3 or more?) The implementing of NFD comparison gives us an equality test for a comparatively small effort for simple cases covering a large number of use cases (Languages using the Latin script). The algorithm is table driven by the UCD [8]. From this follows an simple but important fact for conformant implementations need runtime access to information from the Unicode Character Database [UCD][9]. [1] https://developer.apple.com/library/ios/documentation/Swift/Conceptual/Swift_Programming_Language/StringsAndCharacters.html#//apple_ref/doc/uid/TP40014097-CH7-ID285 [2] http://www.unicode.org/glossary/#normalization_form_d [3] https://developer.apple.com/library/ios/documentation/Cocoa/Reference/Foundation/Classes/NSString_Class/index.html#//apple_ref/occ/instm/NSString/decomposedStringWithCanonicalMapping [4] String asDecomposedUnicode http://wiki.squeak.org/squeak/6250 [5] http://www.unicode.org/glossary/#code_point [6] Unicode initialize http://wiki.squeak.org/squeak/6248 [7] http://www.unicode.org/Public/UNIDATA/UnicodeData.txt [8] Unicode Character Database documentation http://unicode.org/reports/tr44/ [9] http://www.unicode.org/reports/tr23/ From robert.w.withers at gmail.com Tue Dec 15 11:30:19 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Tue Dec 15 11:30:24 2015 Subject: [squeak-dev] review, validation and test development for GenericGF Message-ID: <566FF9CB.7000804@gmail.com> Skipped content of type multipart/alternative-------------- next part -------------- A non-text attachment was scrubbed... Name: GenericGF.java Type: text/x-java Size: 5950 bytes Desc: not available Url : http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151215/650f7892/GenericGF.java From edgardec2005 at gmail.com Tue Dec 15 11:44:25 2015 From: edgardec2005 at gmail.com (Edgar J. De Cleene) Date: Tue Dec 15 11:44:43 2015 Subject: [squeak-dev] drawing an arc In-Reply-To: <0BA25E7C-C635-4803-9870-E91E3E001A65@freudenbergs.de> Message-ID: Second try, the first have a picture but was rejected Yes, we have. See atachment. Original code I think is Hernan Morales Durand, but could be wrong On 12/14/15, 3:22 PM, "Bert Freudenberg" wrote: > On 12.12.2015, at 16:37, Chris Muller wrote: >> >> Does anyone know how to draw an arc on a Canvas? It has drawOval / fillOval, >> but I want to draw a piece of pie. > > > I don?t think we have that. In the Etoys image there is a SectorMorph but it?s > a subclass of PolygonMorph, updating its vertices, not drawing directly on the > canvas. > > - Bert - -------------- next part -------------- A non-text attachment was scrubbed... Name: PieChartMorph-gm.5.cs Type: application/octet-stream Size: 59190 bytes Desc: not available Url : http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151215/7ab61a32/PieChartMorph-gm.5-0001.obj From robert.w.withers at gmail.com Tue Dec 15 11:45:02 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Tue Dec 15 11:45:08 2015 Subject: [squeak-dev] Re: review, validation and test development for GenericGF In-Reply-To: <566FF9CB.7000804@gmail.com> References: <566FF9CB.7000804@gmail.com> Message-ID: <566FFD3E.3010807@gmail.com> The review I need is especially the case with the multiply(a, b), in Java, or #multiply:scalar:, in Smalltalk. Thank you! On 12/15/2015 06:30 AM, Robert Withers wrote: > Hi all, sticking with the distribution for discussing Crypto in the > base, I am currently trying to stabilize a port of Reed-Solomon such > that FEC can be added to SecureSession. I am doing this work in > SecureSession. I just posted an update of the GenericGF class, with > the Java implementation attached here. > > I need this critical code reviewed and tested. You need Cryptography > and SecureSession to be included in the catalog. If you can help me > with the former, I could help with the latter, if that suits you. > Please let me know! > > Please load Cryptography(-Pharo) and SecureSession.23 for review. > > "ported & refactored GenericGF from the java code. This includes > reindexing from 0 to 1. Having no tests, hopefully someone can > give a review of this specific class. Where some tests to be > developed at this scope, that would be stellar." > > > I appreciate. > -- > . .. .. ^,^ best, robert -- . .. .. ^,^ best, robert -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151215/113e422e/attachment.htm From hannes.hirzel at gmail.com Tue Dec 15 16:05:28 2015 From: hannes.hirzel at gmail.com (H. Hirzel) Date: Tue Dec 15 16:05:32 2015 Subject: [Pharo-dev] [Cuis] Sorting Unicode strings (Re: [Unicode] collation sequences (Re: [squeak-dev] Unicode Support)) In-Reply-To: References: <5665EF88.5030901@gemtalksystems.com> <56672D90.4020202@gemtalksystems.com> <530E19B5-317C-4405-A9DB-DF4423A858F3@mac.com> Message-ID: On 12/15/15, Ben Coman wrote: > On Thu, Dec 10, 2015 at 12:37 AM, Todd Blanchard > wrote: >> They are practically the same thing. >> >> ICU was developed by Taligent which was a joint venture between Apple and >> IBM. Makes sense that NSString and ICU's UnicodeString are pretty close >> in implementation. ICU was also ported to Java for Sun by IBM. The point >> is - this is a very elaborate chunk of code with far reach. If ICU is >> wrong on some point - it is universally wrong and thus likely to be taken >> as "right" as it is at least consistent. I think re-implementing it is >> folly TBH. Just use it. > > Apple seem to have moved on from NSString to support Unicode in a > different way in Switft... Could you please give some more details? I have read https://www.objc.io/issues/9-strings/unicode/#nsstring-and-unicode so far. It says that an NSString object actually represents an array of UTF-16-encoded code units. This in contrast to Squeak / Pharo where a String is an ArrayedCollection of 21 bit Unicode code points (transparently optimizing to a ByteArray if the string only contains values of the first code page). >> >>> On Dec 8, 2015, at 15:52, EuanM wrote: >>> >>> Equally old are the NextStep Object C functions which are now embodied >>> within MacOS X. >>> >> >> > > From robert.w.withers at gmail.com Tue Dec 15 17:29:59 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Tue Dec 15 17:30:05 2015 Subject: [squeak-dev] Re: Looking for a working mailreader In-Reply-To: <566FF4CF.4020402@gmail.com> References: <566FF4CF.4020402@gmail.com> Message-ID: <56704E17.3070708@gmail.com> I guess it was left behind. On 12/15/2015 06:09 AM, Robert Withers wrote: > Is there such for squeak? I tried Celeste but it didn't load > correctly. I prefer all my tools tin squeak for immersion and full > screen experience. > -- . .. .. ^,^ robert From btc at openinworld.com Tue Dec 15 17:58:24 2015 From: btc at openinworld.com (Ben Coman) Date: Tue Dec 15 18:01:02 2015 Subject: [Pharo-dev] [Cuis] Sorting Unicode strings (Re: [Unicode] collation sequences (Re: [squeak-dev] Unicode Support)) In-Reply-To: References: <5665EF88.5030901@gemtalksystems.com> <56672D90.4020202@gemtalksystems.com> <530E19B5-317C-4405-A9DB-DF4423A858F3@mac.com> Message-ID: On Wed, Dec 16, 2015 at 12:05 AM, H. Hirzel wrote: > On 12/15/15, Ben Coman wrote: >> On Thu, Dec 10, 2015 at 12:37 AM, Todd Blanchard >> wrote: >>> They are practically the same thing. >>> >>> ICU was developed by Taligent which was a joint venture between Apple and >>> IBM. Makes sense that NSString and ICU's UnicodeString are pretty close >>> in implementation. ICU was also ported to Java for Sun by IBM. The point >>> is - this is a very elaborate chunk of code with far reach. If ICU is >>> wrong on some point - it is universally wrong and thus likely to be taken >>> as "right" as it is at least consistent. I think re-implementing it is >>> folly TBH. Just use it. >> >> Apple seem to have moved on from NSString to support Unicode in a >> different way in Switft... > > Could you please give some more details? http://oleb.net/blog/2014/07/swift-strings/ "Swift's change has the potential to prevent many common errors when dealing with string lengths or substrings. It is a huge difference to most other Unicode-aware string libraries (including NSString) where the building blocks of a string are usually UTF-16 code units or single Unicode scalars" cheers -ben > > I have read > https://www.objc.io/issues/9-strings/unicode/#nsstring-and-unicode > so far. > > It says that an NSString object actually represents an array of > UTF-16-encoded code units. > > This in contrast to Squeak / Pharo where a String is an > ArrayedCollection of 21 bit Unicode code points (transparently > optimizing to a ByteArray if the string only contains values of the > first code page). > > >>> >>>> On Dec 8, 2015, at 15:52, EuanM wrote: >>>> >>>> Equally old are the NextStep Object C functions which are now embodied >>>> within MacOS X. >>>> >>> >>> >> >> > From eliot.miranda at gmail.com Tue Dec 15 21:46:24 2015 From: eliot.miranda at gmail.com (Eliot Miranda) Date: Tue Dec 15 21:46:28 2015 Subject: [squeak-dev] Re: [Cuis] Sorting Unicode strings (Re: [Unicode] collation sequences (Re: Unicode Support)) In-Reply-To: <5669E038.4090103@gemtalksystems.com> References: <5665EF88.5030901@gemtalksystems.com> <56672D90.4020202@gemtalksystems.com> <56674D32.2000305@gemtalksystems.com> <5668A467.7060302@gemtalksystems.com> <5669E038.4090103@gemtalksystems.com> Message-ID: Hi Dale, On Thu, Dec 10, 2015 at 12:27 PM, Dale Henrichs < dale.henrichs@gemtalksystems.com> wrote: > > > On 12/09/2015 04:31 PM, Levente Uzonyi wrote: > >> On Wed, 9 Dec 2015, Dale Henrichs wrote: >> >> >>> >>> On 12/09/2015 12:44 AM, Stephan Eggermont wrote: >>> >>>> On 08-12-15 22:35, Dale Henrichs wrote: >>>> >>>>> What I meant is that you can't _always_ use the code point for >>>>> collation, i.e., sorting based on the value of code points is not >>>>> always >>>>> correct[1]. >>>>> >>>> >>>> I have given up on universal sorting when I learned that dutch >>>> libraries sorting of author names depends on the country of origin of the >>>> author. So if Jan van Beek is dutch he will be sorted under B, while if >>>> he's belgian under V. I haven't checked what happens if the author >>>> emigrates, or changes nationality... >>>> >>>> Stephan >>>> >>>> Well, with ICU (and GemStone's implementation) you can choose which >>> collator to use (Country specific) at the image level or on a comparison by >>> comparison bases ... for example for an indexed collection (Unicode) >>> Strings, you can choose the collator to use for that particular index ... >>> so while it's true that universal sorter is not possible, it is possible to >>> choose a collator that will satisfy a particlar customer .... >>> >> >> I expect my image to compare strings using the codepoint-based (+language >> tags) lexicographical method, because it's simple, deterministic and fast. >> Imagine having failing tests just because your image uses different >> default comparison methods based on some (external) parameter... >> It's also a nightmare to find out why your program is slow on some >> machine, while it's fast on another. >> > > When we implemented the Unicode support in GemStone we preserved the > legacy string classes and their legacy behavior ... We added new Unicode* > classes with the new collator-based behavior for sorting and comparison ... > That way legacy applications (and legacy) tests were not impacted by the > choice of collator ... And folks could choose whether or not their > application would benefit by the use of the new Unicode* classes.... > > The ICU library performance is actually comparable to our original > implementations, so there isn't a noticeable performance difference - we > built the support into our vm and if folks are interested in some of the > gory technical details, we'd be willing to share our experience, as there > are several things that we did to minimize potential performance impacts --- > Just so you know, I will dig my heels in as deeply as I am able to prevent the use of C++ libraries in the VM. It destroys the simulator, which is the most important thing we have for VM development productivity. As far as I'm concerned any use of external libraries to implement core functionality kills the VM-in-Smalltalk concept that Squeak (and Pharo) are built upon. So for me it's a non-starter. I hope others agree. _,,,^..^,,,_ best, Eliot -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151215/48354678/attachment.htm From jecel at merlintec.com Tue Dec 15 23:16:42 2015 From: jecel at merlintec.com (Jecel Assumpcao Jr.) Date: Tue Dec 15 22:16:57 2015 Subject: [squeak-dev] Re: Looking for a working mailreader In-Reply-To: <56704E17.3070708@gmail.com> References: <566FF4CF.4020402@gmail.com> <56704E17.3070708@gmail.com> Message-ID: Robert Withers replied to self: > I guess it was left behind. > > On 12/15/2015 06:09 AM, Robert Withers wrote: > > Is there such for squeak? I tried Celeste but it didn't load > > correctly. I prefer all my tools tin squeak for immersion and full > > screen experience. I am writing this reply in Celeste using a Squeak 4.1 image. I have not tried loading Celeste in a newer image than that. I do have a small set of fixes which I have not sent to be included in the official Celeste because most of them are very wrong and were only enough to keep only loading emails even though some were very broken. -- Jecel From robert.w.withers at gmail.com Tue Dec 15 22:49:12 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Tue Dec 15 22:49:16 2015 Subject: [squeak-dev] Re: Looking for a working mailreader In-Reply-To: <5670915d.882b8c0a.e1cb5.fffff985SMTPIN_ADDED_MISSING@mx.google.com> References: <566FF4CF.4020402@gmail.com> <56704E17.3070708@gmail.com> <5670915d.882b8c0a.e1cb5.fffff985SMTPIN_ADDED_MISSING@mx.google.com> Message-ID: <567098E8.5060505@gmail.com> That's unfortunate. I was hoping to hear of an updated stable version. thanks, robert On 12/15/2015 06:16 PM, Jecel Assumpcao Jr. wrote: > Robert Withers replied to self: >> I guess it was left behind. >> >> On 12/15/2015 06:09 AM, Robert Withers wrote: >>> Is there such for squeak? I tried Celeste but it didn't load >>> correctly. I prefer all my tools tin squeak for immersion and full >>> screen experience. > I am writing this reply in Celeste using a Squeak 4.1 image. I have not > tried loading Celeste in a newer image than that. I do have a small set > of fixes which I have not sent to be included in the official Celeste > because most of them are very wrong and were only enough to keep only > loading emails even though some were very broken. > > -- Jecel > > -- . .. .. ^,^ best, robert From commits at source.squeak.org Wed Dec 16 00:33:19 2015 From: commits at source.squeak.org (commits@source.squeak.org) Date: Wed Dec 16 00:33:20 2015 Subject: [squeak-dev] The Trunk: Sound-tpr.47.mcz Message-ID: tim Rowledge uploaded a new version of Sound to project The Trunk: http://source.squeak.org/trunk/Sound-tpr.47.mcz ==================== Summary ==================== Name: Sound-tpr.47 Author: tpr Time: 15 December 2015, 4:33:09.091 pm UUID: 998878c2-2ae5-41e1-b60a-dbc169c238cd Ancestors: Sound-nice.46 Protect SoundRecorder from primitiveFailed inside a critical block; raise a Warning signal afterwards instead of a raw error =============== Diff against Sound-nice.46 =============== Item was changed: ----- Method: SoundRecorder>>primStartRecordingDesiredSampleRate:stereo:semaIndex: (in category 'primitives') ----- primStartRecordingDesiredSampleRate: samplesPerSec stereo: stereoFlag semaIndex: anInteger + "Start sound recording with the given stereo setting. Use a sampling rate as close to the desired rate as the underlying platform will support. If the given semaphore index is > 0, it is taken to be the index of a Semaphore in the external objects array to be signalled every time a recording buffer is filled. + We do *not* raise a primitiveFailed error here since this prim is called insdied a critical blcok and that often makes things painful. The only really likely case where this prim fails is a linux machine with no sound input hardware (a Raspberry Pi for example). See the startRecording method for how the failure is handled" - "Start sound recording with the given stereo setting. Use a sampling rate as close to the desired rate as the underlying platform will support. If the given semaphore index is > 0, it is taken to be the index of a Semaphore in the external objects array to be signalled every time a recording buffer is filled." + "self primitiveFailed" - self primitiveFailed ! Item was changed: ----- Method: SoundRecorder>>startRecording (in category 'recording controls') ----- startRecording + "Turn on the sound input driver and start the recording process. Initially, recording is paused. + If the primStartRecordingDesiredSampleRate:... fails it almost certainly means we have no usable + sound input device. Rather than having the prim raise a failure error we let it quietly do nothing + (since I hate trying to debug errors inside a critical block) and check the actual sampling rate later. + If the sampling rate is 0 we know the startup failed and raise an application level Signal to let any + user code know about the problem. + You might think we should also use the stopRecording message to close things down cleanly but + that simply results in astorm of attempts to start recording so it is simpler to let it be deluded. An + attempts to start recording will repeat the test and thereby handle any plug-in hardware etc." - "Turn of the sound input driver and start the recording process. Initially, recording is paused." recordLevel ifNil: [recordLevel := 0.5]. "lazy initialization" CanRecordWhilePlaying ifFalse: [SoundPlayer shutDown]. recordProcess ifNotNil: [self stopRecording]. paused := true. meteringBuffer := SoundBuffer newMonoSampleCount: 1024. meterLevel := 0. self allocateBuffer. Smalltalk newExternalSemaphoreDo: [ :semaphore :index | bufferAvailableSema := semaphore. self primStartRecordingDesiredSampleRate: samplingRate asInteger stereo: stereo semaIndex: index ]. RecorderActive := true. samplingRate := self primGetActualRecordingSampleRate. + samplingRate = 0 ifTrue: [ Warning signal: 'SoundRecorder: unable to connect to sound input device']. self primSetRecordLevel: (1000.0 * recordLevel) asInteger. recordProcess := [self recordLoop] newProcess. recordProcess priority: Processor userInterruptPriority. recordProcess resume! From eliot.miranda at gmail.com Wed Dec 16 01:35:38 2015 From: eliot.miranda at gmail.com (Eliot Miranda) Date: Wed Dec 16 01:35:42 2015 Subject: [Pharo-dev] [squeak-dev] Re: [Cuis] Sorting Unicode strings (Re: [Unicode] collation sequences (Re: Unicode Support)) In-Reply-To: <8CA11731-BA34-4000-8603-962FD6839F29@mac.com> References: <5665EF88.5030901@gemtalksystems.com> <56672D90.4020202@gemtalksystems.com> <56674D32.2000305@gemtalksystems.com> <5668A467.7060302@gemtalksystems.com> <5669E038.4090103@gemtalksystems.com> <8CA11731-BA34-4000-8603-962FD6839F29@mac.com> Message-ID: Hi Todd, On Tue, Dec 15, 2015 at 3:46 PM, Todd Blanchard wrote: > Hi Eliot, > > On Dec 15, 2015, at 13:46, Eliot Miranda wrote: > > Just so you know, I will dig my heels in as deeply as I am able to prevent > the use of C++ libraries in the VM. It destroys the simulator, which is > the most important thing we have for VM development productivity. As far > as I'm concerned any use of external libraries to implement core > functionality kills the VM-in-Smalltalk concept that Squeak (and Pharo) are > built upon. > > > OK, I defer to you because you certainly know more about the VM internals > and what does and doesn't work well than anyone else. > > So I guess I would like to know your recommendation for 1) how best to > store strings - byte arrays (UTF8), - 2-byte word arrays (UTF16 - now we > get to worry about endian). > Raw Unicode, either as 8-bit, 16-bit or 32-bit. When creating a String it should start as an 8-bit-per-Unicode-character string. Attempts to store Character values that won't fit cause the String to become a String whose element size is large enough to accommodate the character. In Spur, become: is cheap so this growth pays only for the reallocation and copying of the at a, not for an expensive heap scan necessary to do the become:. > Bearing in mind that both representations are variable length and so while > accessing the n'th byte/word is O(1), accessing the n'th character is > necessarily O(n) unless you know you have no surrogates in your string. > Right, so UTF-8 and UTF-16 are not convenient representations and to be provided only for interchange. > > Also...since NSString has been mentioned...it is worth noting that > NSString is built atop CFString (source code here: > https://www.opensource.apple.com/source/CF/CF-855.11/CFString.c) which > does a fair job of optimizing memory by using bytes where it can and shorts > where it cannot. It is also worth noting that characterAt: actually does > the wrong thing, since it assumes characters are no bigger than FFFF rather > than 10FFFF. > Yes, and Squeak (and AFAIA, Pharo) has been doing this for ages. If one has become: it is very easy to manage. Now with Spur not only do we have become:, we have a fairly fast become:. Does this make sense? > Also...I'll just toss in this very nice article on unicode and how > NSString deals with it. > https://www.objc.io/issues/9-strings/unicode/ > > -Todd Blanchard > _,,,^..^,,,_ best, Eliot -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151215/9a38c998/attachment.htm From lewis at mail.msen.com Wed Dec 16 03:02:23 2015 From: lewis at mail.msen.com (David T. Lewis) Date: Wed Dec 16 03:02:25 2015 Subject: [squeak-dev] potential to include cryptography in base image In-Reply-To: <566FE4AC.4060803@gmail.com> References: <566FE4AC.4060803@gmail.com> Message-ID: <20151216030223.GA16566@shell.msen.com> On Tue, Dec 15, 2015 at 05:00:12AM -0500, Robert Withers wrote: > It was suggested to me that I write to the list and raise the question > about cryptography being included in the base image. Really I have 3 > questions I would ask you all: > > 1. is it desirable to include cryptography? > 2. is it feasible to include cryptography? > 3. what is the time frame for including cryptography? I'm not sure whether it is a good idea to include it in the base image, versus maintain it in an external package that can be easily loaded from SqueakMap. Either way, I think we should get it updated on SqueakMap first so that more people can (and hopefully will) load it in their images, run the tests, and get experience with it. FWIW, I personally tend to favor maintaining packages externally in cases where we have an interest in supporting the package on various kinds of images, such as Squeak/Cuis/Pharo. Robert, are you in a position to update the SqueakMap entries? Currently they are "crypto", "Cryptography", and "Cryptography Team Package". They are all out of date, and I'm not sure who owns which version. We should pick one of them as the official one, and update it so that it loads the latest version (Cryptography-rww.49) into a Squeak 4.6 or 5.0 image. It is of course quite easy to load Cryptography directly from squeaksource, but there are quite a few packages in that repository that may seem confusing for someone loading Cryptography for the first time, so I think a one-click download from SqueakMap would be helpful. Dave From Marcel.Taeumel at hpi.de Wed Dec 16 09:08:54 2015 From: Marcel.Taeumel at hpi.de (marcel.taeumel) Date: Wed Dec 16 09:26:59 2015 Subject: [squeak-dev] CurrentReadOnlySourceFiles annoys me ... Message-ID: <1450256934663-4867253.post@n4.nabble.com> Hi there! The following snippet does not work: [(Morph >> #submorphCount) getSource] on: Exception do: [:ex | ] Because "CurrentReadOnlySourceFiles" is a subclass of Exception. This smells. Does somebody know, why? Rationale: I want to catch all exceptions in some silently but frequently called method. Best, Marcel -- View this message in context: http://forum.world.st/CurrentReadOnlySourceFiles-annoys-me-tp4867253.html Sent from the Squeak - Dev mailing list archive at Nabble.com. From hannes.hirzel at gmail.com Wed Dec 16 10:22:37 2015 From: hannes.hirzel at gmail.com (H. Hirzel) Date: Wed Dec 16 10:22:42 2015 Subject: [Pharo-dev] [squeak-dev] Re: [Cuis] Sorting Unicode strings (Re: [Unicode] collation sequences (Re: Unicode Support)) In-Reply-To: References: <5665EF88.5030901@gemtalksystems.com> <56672D90.4020202@gemtalksystems.com> <56674D32.2000305@gemtalksystems.com> <5668A467.7060302@gemtalksystems.com> <5669E038.4090103@gemtalksystems.com> <8CA11731-BA34-4000-8603-962FD6839F29@mac.com> Message-ID: On 12/16/15, Eliot Miranda wrote: > Hi Todd, > > On Tue, Dec 15, 2015 at 3:46 PM, Todd Blanchard wrote: > >> Hi Eliot, >> >> On Dec 15, 2015, at 13:46, Eliot Miranda wrote: >> >> Just so you know, I will dig my heels in as deeply as I am able to >> prevent >> the use of C++ libraries in the VM. It destroys the simulator, which is >> the most important thing we have for VM development productivity. As far >> as I'm concerned any use of external libraries to implement core >> functionality kills the VM-in-Smalltalk concept that Squeak (and Pharo) >> are >> built upon. >> >> >> OK, I defer to you because you certainly know more about the VM internals >> and what does and doesn't work well than anyone else. >> >> So I guess I would like to know your recommendation for 1) how best to >> store strings - byte arrays (UTF8), - 2-byte word arrays (UTF16 - now we >> get to worry about endian). >> > > Raw Unicode, either as 8-bit, 16-bit or 32-bit. When creating a String it > should start as an 8-bit-per-Unicode-character string. Attempts to store > Character values that won't fit cause the String to become a String whose > element size is large enough to accommodate the character. This is the case: see tests here http://wiki.squeak.org/squeak/6316 In Spur, > become: is cheap so this growth pays only for the reallocation and copying > of the at a, not for an expensive heap scan necessary to do the become:. Could you elaborate on this please? > > >> Bearing in mind that both representations are variable length and so >> while >> accessing the n'th byte/word is O(1), accessing the n'th character is >> necessarily O(n) unless you know you have no surrogates in your string. >> > > Right, so UTF-8 and UTF-16 are not convenient representations and to be > provided only for interchange. +1 Squeak/Pharo uses 8bit/32bit (UTF-32) internally and UTF-8 externally. There are converters for UTF-8 and UTF-16. > >> >> Also...since NSString has been mentioned...it is worth noting that >> NSString is built atop CFString (source code here: >> https://www.opensource.apple.com/source/CF/CF-855.11/CFString.c) which >> does a fair job of optimizing memory by using bytes where it can and >> shorts >> where it cannot. It is also worth noting that characterAt: actually does >> the wrong thing, since it assumes characters are no bigger than FFFF >> rather >> than 10FFFF. >> > > Yes, and Squeak (and AFAIA, Pharo) has been doing this for ages. If one > has become: it is very easy to manage. Now with Spur not only do we have > become:, we have a fairly fast become:. > > Does this make sense? > > >> Also...I'll just toss in this very nice article on unicode and how >> NSString deals with it. >> https://www.objc.io/issues/9-strings/unicode/ >> >> -Todd Blanchard >> > > _,,,^..^,,,_ > best, Eliot > From btc at openinworld.com Wed Dec 16 12:18:20 2015 From: btc at openinworld.com (Ben Coman) Date: Wed Dec 16 12:18:44 2015 Subject: [Vm-dev] Re: [Pharo-dev] [squeak-dev] Re: [Cuis] Sorting Unicode strings (Re: [Unicode] collation sequences (Re: Unicode Support)) In-Reply-To: References: <5665EF88.5030901@gemtalksystems.com> <56672D90.4020202@gemtalksystems.com> <56674D32.2000305@gemtalksystems.com> <5668A467.7060302@gemtalksystems.com> <5669E038.4090103@gemtalksystems.com> <8CA11731-BA34-4000-8603-962FD6839F29@mac.com> Message-ID: On Wed, Dec 16, 2015 at 6:22 PM, H. Hirzel wrote: > > On 12/16/15, Eliot Miranda wrote: >> Hi Todd, >> >> On Tue, Dec 15, 2015 at 3:46 PM, Todd Blanchard wrote: >> >>> Hi Eliot, >>> >>> On Dec 15, 2015, at 13:46, Eliot Miranda wrote: >>> >>> Just so you know, I will dig my heels in as deeply as I am able to >>> prevent >>> the use of C++ libraries in the VM. It destroys the simulator, which is >>> the most important thing we have for VM development productivity. As far >>> as I'm concerned any use of external libraries to implement core >>> functionality kills the VM-in-Smalltalk concept that Squeak (and Pharo) >>> are >>> built upon. >>> >>> >>> OK, I defer to you because you certainly know more about the VM internals >>> and what does and doesn't work well than anyone else. >>> >>> So I guess I would like to know your recommendation for 1) how best to >>> store strings - byte arrays (UTF8), - 2-byte word arrays (UTF16 - now we >>> get to worry about endian). >>> >> >> Raw Unicode, either as 8-bit, 16-bit or 32-bit. When creating a String it >> should start as an 8-bit-per-Unicode-character string. Attempts to store >> Character values that won't fit cause the String to become a String whose >> element size is large enough to accommodate the character. > > This is the case: see tests here http://wiki.squeak.org/squeak/6316 > > In Spur, >> become: is cheap so this growth pays only for the reallocation and copying >> of the at a, not for an expensive heap scan necessary to do the become:. > > Could you elaborate on this please? https://hal.inria.fr/hal-01152610/file/partialReadBarrier.pdf cheers -ben >> >> >>> Bearing in mind that both representations are variable length and so >>> while >>> accessing the n'th byte/word is O(1), accessing the n'th character is >>> necessarily O(n) unless you know you have no surrogates in your string. >>> >> >> Right, so UTF-8 and UTF-16 are not convenient representations and to be >> provided only for interchange. > > +1 > > Squeak/Pharo uses 8bit/32bit (UTF-32) internally and UTF-8 externally. > There are converters for UTF-8 and UTF-16. > > > >> >>> >>> Also...since NSString has been mentioned...it is worth noting that >>> NSString is built atop CFString (source code here: >>> https://www.opensource.apple.com/source/CF/CF-855.11/CFString.c) which >>> does a fair job of optimizing memory by using bytes where it can and >>> shorts >>> where it cannot. It is also worth noting that characterAt: actually does >>> the wrong thing, since it assumes characters are no bigger than FFFF >>> rather >>> than 10FFFF. >>> >> >> Yes, and Squeak (and AFAIA, Pharo) has been doing this for ages. If one >> has become: it is very easy to manage. Now with Spur not only do we have >> become:, we have a fairly fast become:. >> >> Does this make sense? >> >> >>> Also...I'll just toss in this very nice article on unicode and how >>> NSString deals with it. >>> https://www.objc.io/issues/9-strings/unicode/ >>> >>> -Todd Blanchard >>> >> >> _,,,^..^,,,_ >> best, Eliot >> From leves at caesar.elte.hu Wed Dec 16 12:23:54 2015 From: leves at caesar.elte.hu (Levente Uzonyi) Date: Wed Dec 16 12:24:00 2015 Subject: [squeak-dev] CurrentReadOnlySourceFiles annoys me ... In-Reply-To: <1450256934663-4867253.post@n4.nabble.com> References: <1450256934663-4867253.post@n4.nabble.com> Message-ID: CurrentReadOnlySourceFiles could have been a ProcessLocalVariable, but at the time I added it, there were no PLVs in the Trunk. Changing it to PLV is possible, but it would take some effort. Catching all exceptions doesn't sound like a good idea. Why would you want to do that? Levente On Wed, 16 Dec 2015, marcel.taeumel wrote: > Hi there! > > The following snippet does not work: > > [(Morph >> #submorphCount) getSource] > on: Exception > do: [:ex | ] > > Because "CurrentReadOnlySourceFiles" is a subclass of Exception. This > smells. Does somebody know, why? > > Rationale: I want to catch all exceptions in some silently but frequently > called method. > > Best, > Marcel > > > > -- > View this message in context: http://forum.world.st/CurrentReadOnlySourceFiles-annoys-me-tp4867253.html > Sent from the Squeak - Dev mailing list archive at Nabble.com. > > From Marcel.Taeumel at hpi.de Wed Dec 16 12:38:10 2015 From: Marcel.Taeumel at hpi.de (marcel.taeumel) Date: Wed Dec 16 12:56:15 2015 Subject: [squeak-dev] Re: CurrentReadOnlySourceFiles annoys me ... In-Reply-To: References: <1450256934663-4867253.post@n4.nabble.com> Message-ID: <1450269490206-4867327.post@n4.nabble.com> I want to catch all exceptions because it is kind of test code where only success matters. :-) It is an experiment where the participants should program stuff and receive a green tick whenever the task is accomplished. Raising a debugger when the participant puts a "self halt" or anything into the control flow of the test code would confuse the participant. Conceptionally, CurrentReadOnlySourceFiles is no kind of an exception. Thank you for the explanation. Can you sketch the steps to transform that into a PLV? Why would it take some effort? Bset, Marcel -- View this message in context: http://forum.world.st/CurrentReadOnlySourceFiles-annoys-me-tp4867253p4867327.html Sent from the Squeak - Dev mailing list archive at Nabble.com. From eliot.miranda at gmail.com Wed Dec 16 14:46:22 2015 From: eliot.miranda at gmail.com (Eliot Miranda) Date: Wed Dec 16 14:46:31 2015 Subject: [Vm-dev] Re: [Pharo-dev] [squeak-dev] Re: [Cuis] Sorting Unicode strings (Re: [Unicode] collation sequences (Re: Unicode Support)) In-Reply-To: References: <5665EF88.5030901@gemtalksystems.com> <56672D90.4020202@gemtalksystems.com> <56674D32.2000305@gemtalksystems.com> <5668A467.7060302@gemtalksystems.com> <5669E038.4090103@gemtalksystems.com> <8CA11731-BA34-4000-8603-962FD6839F29@mac.com> Message-ID: <167919A7-8B22-4FE0-8179-58D16AE3DCA6@gmail.com> Hi Hannes, > On Dec 16, 2015, at 2:22 AM, H. Hirzel wrote: > > >> On 12/16/15, Eliot Miranda wrote: >> Hi Todd, >> >>> On Tue, Dec 15, 2015 at 3:46 PM, Todd Blanchard wrote: >>> >>> Hi Eliot, >>> >>> On Dec 15, 2015, at 13:46, Eliot Miranda wrote: >>> >>> Just so you know, I will dig my heels in as deeply as I am able to >>> prevent >>> the use of C++ libraries in the VM. It destroys the simulator, which is >>> the most important thing we have for VM development productivity. As far >>> as I'm concerned any use of external libraries to implement core >>> functionality kills the VM-in-Smalltalk concept that Squeak (and Pharo) >>> are >>> built upon. >>> >>> >>> OK, I defer to you because you certainly know more about the VM internals >>> and what does and doesn't work well than anyone else. >>> >>> So I guess I would like to know your recommendation for 1) how best to >>> store strings - byte arrays (UTF8), - 2-byte word arrays (UTF16 - now we >>> get to worry about endian). >> >> Raw Unicode, either as 8-bit, 16-bit or 32-bit. When creating a String it >> should start as an 8-bit-per-Unicode-character string. Attempts to store >> Character values that won't fit cause the String to become a String whose >> element size is large enough to accommodate the character. > > This is the case: see tests here http://wiki.squeak.org/squeak/6316 > > In Spur, >> become: is cheap so this growth pays only for the reallocation and copying >> of the at a, not for an expensive heap scan necessary to do the become:. > > Could you elaborate on this please? Well, there are two presentations online, one at ESUG 2014, and one at Splash 2015, a paper at ISMM 2015, and a blog post if you want a full account, but... Spur supports lazy become via forwarders. When an object is becommed, it is turned into a forwarder to the object it becomes, and when a pair of objects are becommed each is copied and the two originals turned into forwarders to the opposite copy. Immediately after the forwarding the receiver in each stack frame in the stack zone is scanned to follow forwarders so that no read barrier is needed when accessing inst vars. Forwarders have a unique class index so any message send to a forwarder will fail. The check for sends to forwarders needs to be done only just before a full lookup. A forwarder looks different to other objects (they have a unique format field in their object header) so any primitive that encounters a forwarder in its operands will fail (since primitives validate their arguments). So in primitive failure the VM checks for forwarders amongst the operands and if any are found, they are followed and the primitive is retried. There are read barriers in some operations, hence I call the scheme a partial read barrier. The main cost is the stack zone scan which must be performed for pointer objects (since only these can have inst vars). This takes very little time on current machinery. Becoming bit objects like strings is faster because no scan is needed. >>> Bearing in mind that both representations are variable length and so >>> while >>> accessing the n'th byte/word is O(1), accessing the n'th character is >>> necessarily O(n) unless you know you have no surrogates in your string. >> >> Right, so UTF-8 and UTF-16 are not convenient representations and to be >> provided only for interchange. > > +1 > > Squeak/Pharo uses 8bit/32bit (UTF-32) internally and UTF-8 externally. > There are converters for UTF-8 and UTF-16. > >>> Also...since NSString has been mentioned...it is worth noting that >>> NSString is built atop CFString (source code here: >>> https://www.opensource.apple.com/source/CF/CF-855.11/CFString.c) which >>> does a fair job of optimizing memory by using bytes where it can and >>> shorts >>> where it cannot. It is also worth noting that characterAt: actually does >>> the wrong thing, since it assumes characters are no bigger than FFFF >>> rather >>> than 10FFFF. >> >> Yes, and Squeak (and AFAIA, Pharo) has been doing this for ages. If one >> has become: it is very easy to manage. Now with Spur not only do we have >> become:, we have a fairly fast become:. >> >> Does this make sense? >> >> >>> Also...I'll just toss in this very nice article on unicode and how >>> NSString deals with it. >>> https://www.objc.io/issues/9-strings/unicode/ >>> >>> -Todd Blanchard >> >> _,,,^..^,,,_ >> best, Eliot From eliot.miranda at gmail.com Wed Dec 16 14:53:40 2015 From: eliot.miranda at gmail.com (Eliot Miranda) Date: Wed Dec 16 14:53:45 2015 Subject: [squeak-dev] Re: CurrentReadOnlySourceFiles annoys me ... In-Reply-To: <1450269490206-4867327.post@n4.nabble.com> References: <1450256934663-4867253.post@n4.nabble.com> <1450269490206-4867327.post@n4.nabble.com> Message-ID: <35DA6F81-42C6-4761-B43E-CD5593EF7953@gmail.com> Hi all, IMO we don't need CurrentReadOnkySourceFiles, instead we should have SourceFilesArray maintain one read-only copy per source file. The objection is that using a single file will make debugging source file access break, since the debuggers own access to source will perturb the source files whose access is being debugged, but the debugger can easily temporarily install a different read-init copy whenever it accesses source, insulating the debugged code from this effect. _,,,^..^,,,_ (phone) > On Dec 16, 2015, at 4:38 AM, marcel.taeumel wrote: > > I want to catch all exceptions because it is kind of test code where only > success matters. :-) It is an experiment where the participants should > program stuff and receive a green tick whenever the task is accomplished. > Raising a debugger when the participant puts a "self halt" or anything into > the control flow of the test code would confuse the participant. > > Conceptionally, CurrentReadOnlySourceFiles is no kind of an exception. Thank > you for the explanation. Can you sketch the steps to transform that into a > PLV? Why would it take some effort? > > Bset, > Marcel > > > > -- > View this message in context: http://forum.world.st/CurrentReadOnlySourceFiles-annoys-me-tp4867253p4867327.html > Sent from the Squeak - Dev mailing list archive at Nabble.com. > From leves at caesar.elte.hu Wed Dec 16 22:43:22 2015 From: leves at caesar.elte.hu (Levente Uzonyi) Date: Wed Dec 16 22:43:26 2015 Subject: [squeak-dev] Re: CurrentReadOnlySourceFiles annoys me ... In-Reply-To: <35DA6F81-42C6-4761-B43E-CD5593EF7953@gmail.com> References: <1450256934663-4867253.post@n4.nabble.com> <1450269490206-4867327.post@n4.nabble.com> <35DA6F81-42C6-4761-B43E-CD5593EF7953@gmail.com> Message-ID: My objection is that concurrent access of the read-only source files would cause race conditions. We might simplify the thing by using the regular source files (or some shared read-only copies) when requested from the UI process, and create a read-only copy for all other processes. That would cover most potential slow-downs in the Trunk. Levente On Wed, 16 Dec 2015, Eliot Miranda wrote: > Hi all, > > IMO we don't need CurrentReadOnkySourceFiles, instead we should have SourceFilesArray maintain one read-only copy per source file. The objection is that using a single file will make debugging source file access break, since the debuggers own access to source will perturb the source files whose access is being debugged, but the debugger can easily temporarily install a different read-init copy whenever it accesses source, insulating the debugged code from this effect. > > _,,,^..^,,,_ (phone) > >> On Dec 16, 2015, at 4:38 AM, marcel.taeumel wrote: >> >> I want to catch all exceptions because it is kind of test code where only >> success matters. :-) It is an experiment where the participants should >> program stuff and receive a green tick whenever the task is accomplished. >> Raising a debugger when the participant puts a "self halt" or anything into >> the control flow of the test code would confuse the participant. >> >> Conceptionally, CurrentReadOnlySourceFiles is no kind of an exception. Thank >> you for the explanation. Can you sketch the steps to transform that into a >> PLV? Why would it take some effort? >> >> Bset, >> Marcel >> >> >> >> -- >> View this message in context: http://forum.world.st/CurrentReadOnlySourceFiles-annoys-me-tp4867253p4867327.html >> Sent from the Squeak - Dev mailing list archive at Nabble.com. >> > > From commits at source.squeak.org Wed Dec 16 22:55:02 2015 From: commits at source.squeak.org (commits@source.squeak.org) Date: Wed Dec 16 22:55:04 2015 Subject: [squeak-dev] Daily Commit Log Message-ID: <20151216225502.1536.qmail@box4.squeakfoundation.org> Changes to Trunk (http://source.squeak.org/trunk.html) in the last 24 hours: http://lists.squeakfoundation.org/pipermail/packages/2015-December/009215.html Name: Sound-tpr.47 Ancestors: Sound-nice.46 Protect SoundRecorder from primitiveFailed inside a critical block; raise a Warning signal afterwards instead of a raw error ============================================= From nicolas.cellier.aka.nice at gmail.com Wed Dec 16 23:08:24 2015 From: nicolas.cellier.aka.nice at gmail.com (Nicolas Cellier) Date: Wed Dec 16 23:08:29 2015 Subject: [squeak-dev] bitshift: In-Reply-To: References: <566B238B.6010807@gmail.com> <6E15F02ADD7542FF99002AFF9C58723B@LSWWin764NB> <92A7EEEA2A8A475C9735EFEEB13157CE@LSWWin764NB> <6BFA1764-1200-48DA-8898-5C22B0A2F53E@gmail.com> <6561B0F0C9D048C48A4EAE8160D0FF30@LSWWin764NB> <177F8819-441A-48F0-B967-A77422736D34@freudenbergs.de> Message-ID: Hmm, that remind me something like https://github.com/nicolas-cellier-aka-nice/arbitrary-precision-float/wiki/ArbitraryPrecisionFloatForDolphin With the ArbitraryPrecisionFloat package I think I pretty much stressed the LargeInteger primitives, and I can say that VW and Squeak/Pharo haven't proved buggy so far. gst uses gmp, so porting ArbitraryPrecisionFloat there makes less sense and I can't tell. It's been years since I didn't test VA or stx... Nicolas 2015-12-14 23:28 GMT+01:00 Frank Lesser : > Hi Bert, > > yes funny, > > but to be correct, in Dolphin bitShift: is correct ( so far our tests > covered it ), in Squeak, Pharo it is also correct, in Digitalk it is > broken, in Dolphin Integer division prim is broken. > > Other Smalltalk's I haven't tested yet. > > Frank > > > ------------------------------ > > *Von:* squeak-dev-bounces@lists.squeakfoundation.org [mailto:squea > k-dev-bounces@lists.squeakfoundation.org] *Im Auftrag von *Bert > Freudenberg > *Gesendet:* Montag, 14. Dezember 2015 22:35 > > *An:* The general-purpose Squeak developers list > *Betreff:* Re: [squeak-dev] bitshift: > > > > Fun fact - in JavaScript, this is true: > > > > (-0x10000001 >> 32) == -0x10000001 > > > > - Bert - > > > > On 11.12.2015, at 22:22, Frank Lesser > wrote: > > > > Hi Tim, thx for fast answer, need to get sorted my newsgroups as it was > posted wrong. > > It is wrong in Dolphin: > > > > ( -16r10000001 bitShift: -32 )-1 > > ( -16r10000000 bitShift: -32 )-1 > > ( -16r1000001 bitShift: -24 ) -2 > > ( -16r1000000 bitShift: -24 ) -1 > > > > but according to Dolphin's comment it is correct. > > Frank > > > ------------------------------ > > *Von:* squeak-dev-bounces@lists.squeakfoundation.org [ > mailto:squeak-dev-bounces@lists.squeakfoundation.org > ] *Im Auftrag von *Tim > Olson > *Gesendet:* Freitag, 11. Dezember 2015 21:16 > *An:* The general-purpose Squeak developers list > *Betreff:* Re: [squeak-dev] bitshift: > > > > > > On Dec 11, 2015, at 2:00 PM, Frank Lesser < > frank-lesser@lesser-software.com> wrote: > > > > > Hi, > > > > ( -16r100000001 bitShift: -32 ) = -2 > > ( -16r100000000 bitShift: -32 ) = -1 > > > > looks odd to me, > > > > guess the prim needs to be fixed > > > > Looks correct to me: > > > > (-16r100000001 bitAnd: 16rFFFFFFFFFF) hex -> ?16rFEFFFFFFFF? > > downshift this by 32 bits and you get > 16rFFFFFFFE, which is -2 > > > > (-16r100000000 bitAnd: 16rFFFFFFFFFF) hex -> ?16rFF00000000' > > downshift this by 32 bits and you get > 16rFFFFFFFF, which is -1. > > > > ? tim > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151217/36ef97c7/attachment-0001.htm From tim at rowledge.org Wed Dec 16 23:29:35 2015 From: tim at rowledge.org (tim Rowledge) Date: Wed Dec 16 23:29:39 2015 Subject: [squeak-dev] CurrentReadOnlySourceFiles annoys me ... In-Reply-To: References: <1450256934663-4867253.post@n4.nabble.com> <1450269490206-4867327.post@n4.nabble.com> <35DA6F81-42C6-4761-B43E-CD5593EF7953@gmail.com> Message-ID: <0EFA74A2-257E-4C29-B35F-04FF6BEB23A4@rowledge.org> > On 16-12-2015, at 2:43 PM, Levente Uzonyi wrote: > > My objection is that concurrent access of the read-only source files would cause race conditions. > We might simplify the thing by using the regular source files (or some shared read-only copies) when requested from the UI process, and create a read-only copy for all other processes. That would cover most potential slow-downs in the Trunk. One of the best ways to avoid file read (and many write) issues would be to get rid of the {deletable expletive} stupid separate move & read/write operations. If the primitive were readBytes: num from: file to: buffer at: startposition and so on a lot of problems could not exist. It?s like error codes being found via a distinct api instead of being returned directly - a catastrophe waiting to happen. tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim This is all a lot simpler and a lot more complicated than you could possibly imagine From eliot.miranda at gmail.com Thu Dec 17 02:49:20 2015 From: eliot.miranda at gmail.com (Eliot Miranda) Date: Thu Dec 17 02:49:22 2015 Subject: [squeak-dev] Re: CurrentReadOnlySourceFiles annoys me ... In-Reply-To: References: <1450256934663-4867253.post@n4.nabble.com> <1450269490206-4867327.post@n4.nabble.com> <35DA6F81-42C6-4761-B43E-CD5593EF7953@gmail.com> Message-ID: Hi Levente, On Wed, Dec 16, 2015 at 2:43 PM, Levente Uzonyi wrote: > My objection is that concurrent access of the read-only source files would > cause race conditions. > We might simplify the thing by using the regular source files (or some > shared read-only copies) when requested from the UI process, and create a > read-only copy for all other processes. That would cover most potential > slow-downs in the Trunk. Apart form the debugger can you think of situations in which this will really happen? I can't. If you can convince me that concurrent access is a real danger then I'll think again, but in all my time doing Smalltalk with Smalltalk-80 v2, with VisualWorks and with Croquet, Squeak and Pharo the only time this has been an issue was in the debugger debugging source file access. Given that that's easy to solve I don't see the point of living with such slow source file access for any longer :-) > > > Levente > > > On Wed, 16 Dec 2015, Eliot Miranda wrote: > > Hi all, >> >> IMO we don't need CurrentReadOnkySourceFiles, instead we should have >> SourceFilesArray maintain one read-only copy per source file. The >> objection is that using a single file will make debugging source file >> access break, since the debuggers own access to source will perturb the >> source files whose access is being debugged, but the debugger can easily >> temporarily install a different read-init copy whenever it accesses source, >> insulating the debugged code from this effect. >> >> _,,,^..^,,,_ (phone) >> >> On Dec 16, 2015, at 4:38 AM, marcel.taeumel >>> wrote: >>> >>> I want to catch all exceptions because it is kind of test code where only >>> success matters. :-) It is an experiment where the participants should >>> program stuff and receive a green tick whenever the task is accomplished. >>> Raising a debugger when the participant puts a "self halt" or anything >>> into >>> the control flow of the test code would confuse the participant. >>> >>> Conceptionally, CurrentReadOnlySourceFiles is no kind of an exception. >>> Thank >>> you for the explanation. Can you sketch the steps to transform that into >>> a >>> PLV? Why would it take some effort? >>> >>> Bset, >>> Marcel >>> >>> >>> >>> -- >>> View this message in context: >>> http://forum.world.st/CurrentReadOnlySourceFiles-annoys-me-tp4867253p4867327.html >>> Sent from the Squeak - Dev mailing list archive at Nabble.com. >>> >>> >> >> > -- _,,,^..^,,,_ best, Eliot -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151216/cbc34548/attachment.htm From eliot.miranda at gmail.com Thu Dec 17 03:24:07 2015 From: eliot.miranda at gmail.com (Eliot Miranda) Date: Thu Dec 17 03:24:09 2015 Subject: [squeak-dev] CurrentReadOnlySourceFiles annoys me ... In-Reply-To: <0EFA74A2-257E-4C29-B35F-04FF6BEB23A4@rowledge.org> References: <1450256934663-4867253.post@n4.nabble.com> <1450269490206-4867327.post@n4.nabble.com> <35DA6F81-42C6-4761-B43E-CD5593EF7953@gmail.com> <0EFA74A2-257E-4C29-B35F-04FF6BEB23A4@rowledge.org> Message-ID: On Wed, Dec 16, 2015 at 3:29 PM, tim Rowledge wrote: > > > On 16-12-2015, at 2:43 PM, Levente Uzonyi wrote: > > > > My objection is that concurrent access of the read-only source files > would cause race conditions. > > We might simplify the thing by using the regular source files (or some > shared read-only copies) when requested from the UI process, and create a > read-only copy for all other processes. That would cover most potential > slow-downs in the Trunk. > > One of the best ways to avoid file read (and many write) issues would be > to get rid of the {deletable expletive} stupid separate move & read/write > operations. If the primitive were readBytes: num from: file to: buffer at: > startposition and so on a lot of problems could not exist. It?s like error > codes being found via a distinct api instead of being returned directly - a > catastrophe waiting to happen. > Agreed. How do we make this so? tim > -- > tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim > This is all a lot simpler and a lot more complicated than you could > possibly imagine > > > > -- _,,,^..^,,,_ best, Eliot -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151216/0cc764db/attachment.htm From btc at openinworld.com Thu Dec 17 03:31:58 2015 From: btc at openinworld.com (Ben Coman) Date: Thu Dec 17 03:32:25 2015 Subject: [squeak-dev] Raspberry Pi Scratch with GPIO Message-ID: I was just looking to show a friend at work programming for kids on the Raspberry Pi, away from my usual Smalltalk activities at home, and was surprised to find an article mentioning Scratch with GPIO and our very own Tim & Eliot, which I thought I'd share https://www.raspberrypi.org/blog/a-new-version-of-scratch-for-raspberry-pi-now-with-added-gpio/ cheers -ben From lewis at mail.msen.com Thu Dec 17 04:24:21 2015 From: lewis at mail.msen.com (David T. Lewis) Date: Thu Dec 17 04:24:23 2015 Subject: [squeak-dev] Re: CurrentReadOnlySourceFiles annoys me ... In-Reply-To: References: <1450256934663-4867253.post@n4.nabble.com> <1450269490206-4867327.post@n4.nabble.com> <35DA6F81-42C6-4761-B43E-CD5593EF7953@gmail.com> Message-ID: <20151217042421.GA99370@shell.msen.com> On Wed, Dec 16, 2015 at 06:49:20PM -0800, Eliot Miranda wrote: > Hi Levente, > > On Wed, Dec 16, 2015 at 2:43 PM, Levente Uzonyi > wrote: > > > My objection is that concurrent access of the read-only source files would > > cause race conditions. > > We might simplify the thing by using the regular source files (or some > > shared read-only copies) when requested from the UI process, and create a > > read-only copy for all other processes. That would cover most potential > > slow-downs in the Trunk. > > > Apart form the debugger can you think of situations in which this will > really happen? I can't. If you can convince me that concurrent access is a > real danger then I'll think again, but in all my time doing Smalltalk with > Smalltalk-80 v2, with VisualWorks and with Croquet, Squeak and Pharo the > only time this has been an issue was in the debugger debugging source file > access. Given that that's easy to solve I don't see the point of living > with such slow source file access for any longer :-) > The CurrentReadOnlySourceFiles changes were added in March 2011 in Files-ul.104 and Files-ul.105. I do not recall the specific problems that motivated this change, but they were real problems and I don't think that it was anything related to the debugger. Levente can probably refresh our collective memory but I think that it may have been something related to running SUnit tests or the MC update stream processing. Be that as it may, it seems to me that accessing source files entries is a fundamental thing that needs to work reliably in a multi-process environment, period (*). Dave (*) I was tempted to say "without exception" rather that "period", but that would imply an implementation bias ;-) From eliot.miranda at gmail.com Thu Dec 17 17:29:28 2015 From: eliot.miranda at gmail.com (Eliot Miranda) Date: Thu Dec 17 17:29:30 2015 Subject: [squeak-dev] OT: Convince me github is a wise choice Message-ID: Hi Ben, Hi All, I'm quite conservative when it comes to relying on others' infrastructure so I need some help making me take the plunge. Please see below: On Thu, Dec 17, 2015 at 7:15 AM, Ben Coman wrote: > > > On Wed, Dec 16, 2015 at 1:00 AM, Ben Coman wrote: > >> > >> > >> On Wed, Dec 16, 2015 at 10:43 AM, Ryan Macnak > wrote: > >> > > >> > What would be more helpful is if the VM build was fixed to work with > a cross compiler, so it would compile fast enough to test ARM and MIPS on > Travis CI alongside IA32 and X64. > >> > > >> > It would also help if the top-of-tree Intel VMs were always kept > working so we'd know which change broke something. Moving the Subversion > repository to a more reliable host (which likely means migrating to Git) > would also cut down on the false positives Travis reports because the > Subversion server has a habit of dropping connections. > >> > >> +1 github :) > > btw, Did you know that github supports subversion clients since 2011 [1]? > Here are supported features [2]. Are these sufficient for your > current svn workflows? > Potentially we could have ONE repository and those liking subversion > can stick with it and those liking git can use that. Of course, this > would need to be proven. > Ah, that's interesting. So my concern is whether github is a safe long-term bet. Specifically what is there to prevent some third party from buying github, or of github going public and the board taking the decision, or github on its own, deciding to charge for hosting, keeping the data hostage to extract payment? What safeguards are in place to prevent this? I'm not interested in "this will never happen" arguments. I'm interested in hard data please. [4] Provides pragmatic advice for cutting over. Esteban appears to > have done similar to step 1 and 2 [3] - but it seem sometimes his > modifications directly update this mirror so its not clear to see when > that branch is an *exact* copy of the current svn trunk. So I'd love > to see a github repository that is always an *exact* mirror of the svn > repository, with any pharo mods occurring in a branch off that. Even > better if the repository for svn users resides on github in place of > that mirror. > > I've been googling around for problems reported using github via an > svn client, and haven't found any smoking guns. > Is this something we can trial? I'm willing to put some effort into > it. A key requirement would be not interrupting Eliots work on > Spur-64. Potentially we could stay for months on step 3 [4] with the > CI infrastructure running on the git side, but code check-ins > continuing onthe svn side. > > btw2, [5] provides a use case for the advantages of a full switch. > > cheers -ben > > [1] https://github.com/blog/966-improved-subversion-client-support > [2] https://help.github.com/articles/support-for-subversion-clients/ > [3] https://github.com/pharo-project/pharo-vm/network > [4] > http://blogs.atlassian.com/2013/01/atlassian-svn-to-git-migration-technical-side/ > [5] > http://blogs.atlassian.com/2013/01/svn-to-git-how-atlassian-made-the-switch-without-sacrificing-active-development/ > -- _,,,^..^,,,_ best, Eliot -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151217/3a444513/attachment.htm From tim at rowledge.org Thu Dec 17 18:35:40 2015 From: tim at rowledge.org (tim Rowledge) Date: Thu Dec 17 18:35:44 2015 Subject: [squeak-dev] CurrentReadOnlySourceFiles annoys me ... In-Reply-To: References: <1450256934663-4867253.post@n4.nabble.com> <1450269490206-4867327.post@n4.nabble.com> <35DA6F81-42C6-4761-B43E-CD5593EF7953@gmail.com> <0EFA74A2-257E-4C29-B35F-04FF6BEB23A4@rowledge.org> Message-ID: > On 16-12-2015, at 7:24 PM, Eliot Miranda wrote: > > > > On Wed, Dec 16, 2015 at 3:29 PM, tim Rowledge wrote: > > > On 16-12-2015, at 2:43 PM, Levente Uzonyi wrote: > > > > My objection is that concurrent access of the read-only source files would cause race conditions. > > We might simplify the thing by using the regular source files (or some shared read-only copies) when requested from the UI process, and create a read-only copy for all other processes. That would cover most potential slow-downs in the Trunk. > > One of the best ways to avoid file read (and many write) issues would be to get rid of the {deletable expletive} stupid separate move & read/write operations. If the primitive were readBytes: num from: file to: buffer at: startposition and so on a lot of problems could not exist. It?s like error codes being found via a distinct api instead of being returned directly - a catastrophe waiting to happen. > > Agreed. How do we make this so? Extended FilePlugin for new systems, maintain old api as well for a few years until all users of the old stuff die, new images use new prims. If there are particularly annoying users of the old stuff, send out ninja teams to expedite their retirement. tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Strange OpCodes: ESR: Emulate Slide Rule From lewis at mail.msen.com Fri Dec 18 00:53:10 2015 From: lewis at mail.msen.com (David T. Lewis) Date: Fri Dec 18 00:54:12 2015 Subject: [squeak-dev] Re: [Pharo-dev] [Vm-dev] VM Maker: VMMaker.oscog-eem.1609.mcz In-Reply-To: References: <56730263.e1108c0a.603c2.ffffb660SMTPIN_ADDED_MISSING@mx.google.com> Message-ID: <20151218005310.GA7232@shell.msen.com> On Thu, Dec 17, 2015 at 11:12:35AM -0800, Eliot Miranda wrote: > Hi All, > > the real Cog 64-bit Spur x64 VM just evaluated 3+4 correctly on Mac OS X: > > > ??? > > Early days yet. This is a debug VM. The optimised VM does not yet display > the prompt. But we can be confident that a 64-bit JIT for Spur will be > available some time in January. > ??? > Outstanding! Congratulations. Dave From robert.w.withers at gmail.com Fri Dec 18 06:35:21 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Fri Dec 18 06:35:28 2015 Subject: [squeak-dev] New version (47) of SecureSession for Pharo & Squeak Message-ID: <5673A929.9070209@gmail.com> Please load the SecureSession.47 and drive it about a bit. I am missing crypto plugins on Pharo, so avoid the performance test, tto stay responsive. Here are some of the changes: - frame-based pipeline - header per msg so no mutable session state - header is written with message in Asn1Der encoding. - most header fields currently optional - adding per-connection msgId to header - timeSent set also - reworked TransportEndpoint to be better scoped. Looking forward: - extend frames into the strategy utilities - events Any issues or questions, please reach out. Thanks, robert From bert at freudenbergs.de Fri Dec 18 10:21:37 2015 From: bert at freudenbergs.de (Bert Freudenberg) Date: Fri Dec 18 10:21:43 2015 Subject: [squeak-dev] Re: [Vm-dev] VM Maker: VMMaker.oscog-eem.1609.mcz In-Reply-To: References: <56730263.e1108c0a.603c2.ffffb660SMTPIN_ADDED_MISSING@mx.google.com> Message-ID: On 17.12.2015, at 20:12, Eliot Miranda wrote: > > Hi All, > > the real Cog 64-bit Spur x64 VM just evaluated 3+4 correctly on Mac OS X: Awesome! Congrats :) - Bert - -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4115 bytes Desc: not available Url : http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151218/6746d89a/smime.bin From robert.w.withers at gmail.com Fri Dec 18 11:50:51 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Fri Dec 18 11:50:56 2015 Subject: [squeak-dev] version 50 of SecureSession is released Message-ID: <5673F31B.2090702@gmail.com> SecureSession is now FEC encoded. thanks, -- . .. .. ^,^ robert From robert.w.withers at gmail.com Fri Dec 18 12:15:20 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Fri Dec 18 12:15:24 2015 Subject: [squeak-dev] Re: version 50 of SecureSession is released In-Reply-To: <5673F31B.2090702@gmail.com> References: <5673F31B.2090702@gmail.com> Message-ID: <5673F8D8.8020106@gmail.com> Well, I must correct myself. It encodes/decodes in the absence of errors, but it does not recover from damage, yet. It must be the EuclideanAlgorithm it is using. I'll let you know when that's fixed, but it is usable without damage, in the meantime. Sorry about that, Robert On 12/18/2015 06:50 AM, Robert Withers wrote: > SecureSession is now FEC encoded. > > thanks, -- . .. .. ^,^ robert From hannes.hirzel at gmail.com Fri Dec 18 13:47:24 2015 From: hannes.hirzel at gmail.com (H. Hirzel) Date: Fri Dec 18 13:47:26 2015 Subject: [squeak-dev] [Unicode] Summary (Re: [Pharo-dev] Unicode Support // e acute example --> decomposition in Pharo?) Message-ID: Hello Sven Thank you for your report about about your experimental, proof of concept, prototype project, that aims to improve Unicode support. Please include me in the loop. Below is is my attempt at summarizing the Unicode discussion of the last weeks. Corrections /comments / additions are welcome. Kind regards Hannes 1) There is a need for improved Unicode support implemented _within_ the image , probably as a library. 1a) This follows the example of the the Twitter CLDR library (i.e. re-implementation of ICU components for Ruby). https://github.com/twitter/twitter-cldr-rb Other languages/libraries have similar approaches - dotNet, https://msdn.microsoft.com/en-us/library/System.Globalization.CharUnicodeInfo%28v=vs.110%29.aspx) - Python https://docs.python.org/3/howto/unicode.html - Go http://blog.golang.org/strings - Swift, https://developer.apple.com/library/ios/documentation/Swift/Conceptual/Swift_Programming_Language/StringsAndCharacters.html - Perl http://blog.golang.org/strings 1b) ICU is _not_ the way to go (http://site.icu-project.org/) . This is because of security and portability reasons (Eliot Miranda) and because of the Smalltalk approach that wants to expose basic algorithms in Smalltalk code. In addition the 16bit based ICU library does not fit well with the Squeak/Pharo UTF32 model. 2) The Unicode infrastructure (21(32) bit wide Characters as immediate objects, use of UTF-32 internally, indexable strings, UTF8 for outside communication, support of code converters) is a very valuable foundation which makes algorithms more straightforward at the expense of a more memory usage. It not used to its full potential at all currently though a lot of hard work has been done. 3) The Unicode algorithms are mostly table / database driven. This means that dictionary lookup is a prominent part of the algorithms. The essential building block for this is that the Unicode character database UCD (http://www.unicode.org/ucd/) is made available _within_ the image with the full content as needed by the target languages / scripts one wants to deal with. The process of loading the UCD should be made configurable. 3a) a lot of people are interested in the Latin script (and scripts of similar complexity) only. 3b) The UCD data in XML form http://www.unicode.org/Public/8.0.0/ucdxml/ offers a download with and without the CJK characters. 4) The next step is to implement normalization (http://www.unicode.org/reports/tr15/#Norm_Forms). Glad to read that you have reached results here with the test data: http://www.unicode.org/Public/UNIDATA/NormalizationTest.txt. 5) Pharo offers nice inspectors to view dictionaries and ordered collections (table view, drill down) which facilitates the development to table driven algorithms. The data structures and algorithm are do not depend on a particular dialect though and may be ported to Squeak or Cuis. 6) After having implemented normalization, comparison may be implemented. This needs CLDR access (collation, Unicode Common Locale Data Repository, http://cldr.unicode.org/ ). 7) An architecture has the following subsystems 7a) Basic character handling (21(32)bit characters in indexable strings, point 2) 7b) Runtime access to the Unicode Character Database (point 3) 7c) Converters 7d) Normalization (point 4) 7e) CLDR access (point 6) 8) The implementation should be driven by the current needs. An attainable next goal is to release 8a) a StringBuilder utility class for easier construction of test strings i.e. instead of > normalizer composeString: (#(68 117 776 115 115 101 108 100 111 114 102 32 > 75 111 776 110 105 103 115 97 108 108 101 101) collect: #asCharacter as: > String). do normalizer composeString: (StringBuilder construct: 'Du\u0308sseldorf Ko\u0308nigsallee') and construct some test cases with it which illustrate some basic Unicode issues. 8b) identity testing for major languages (e.g. French, German, Spanish) and scripts of similar complexity. I 8c) to provide some more documentation of past and concurrent efforts. Note: This summary has only covered string manipulation, not rendering on the screen which is a different issue. On 12/16/15, Sven Van Caekenberghe wrote: > Hi Hannes, > > My detailed comments/answers below, after quoting 2 of your emails: > >> On 10 Dec 2015, at 22:17, H. Hirzel wrote: >> >> Hello Sven >> >> On 12/9/15, Sven Van Caekenberghe wrote: >> >>> The simplest example in a common language is (the French letter ?) is >>> >>> LATIN SMALL LETTER E WITH ACUTE [U+00E9] >>> >>> which can also be written as >>> >>> LATIN SMALL LETTER E [U+0065] followed by COMBINING ACUTE ACCENT >>> [U+0301] >>> >>> The former being a composed normal form, the latter a decomposed normal >>> form. (And yes, it is even much more complicated than that, it goes on >>> for >>> 1000s of pages). >>> >>> In the above example, the concept of character/string is indeed fuzzy. >>> >>> HTH, >>> >>> Sven >> >> Thanks for this example. I have created a wiki page with it >> >> I wonder what the Pharo equivalent is of the following Squeak expression >> >> $? asString asDecomposedUnicode >> >> Regards >> >> Hannes > > You also wrote: > >> The text below shows how to deal with the Unicode e acute example >> brought up by Sven in terms of comparing strings. Currently Pharo and >> Cuis do not do Normalization of strings. Limited support is in Squeak. >> It will be shown how NFD normalization may be implemented. >> >> >> Swift programming language >> ----------------------------------------- >> >> How does the Swift programming language [1] deal with Unicode strings? >> >> // "Voulez-vous un caf??" using LATIN SMALL LETTER E WITH ACUTE >> let eAcuteQuestion = "Voulez-vous un caf\u{E9}?" >> >> // "Voulez-vous un café?" using LATIN SMALL LETTER E and >> COMBINING ACUTE ACCENT >> let combinedEAcuteQuestion = "Voulez-vous un caf\u{65}\u{301}?" >> >> if eAcuteQuestion == combinedEAcuteQuestion { >> print("These two strings are considered equal") >> } >> // prints "These two strings are considered equal" >> >> The equality operator uses the NFD (Normalization Form Decomposed)[2] >> form for the comparison appyling a method >> #decomposedStringWithCanonicalMapping[3] >> >> >> Squeak / Pharo >> ----------------------- >> >> Comparison without NFD [3] >> >> >> "Voulez-vous un caf??" >> eAcuteQuestion := 'Voulez-vous un caf', 16rE9 asCharacter asString, '?'. >> combinedEAcuteQuestion := 'Voulez-vous un cafe', 16r301 asCharacter >> asString, '?'. >> >> >> eAcuteQuestion = combinedEAcuteQuestion >> false >> >> eAcuteQuestion == combinedEAcuteQuestion >> false >> >> The result is false. A Unicode conformant application however should >> return *true*. >> >> Reason for this is that Squeak / Pharo strings are not put into NFD >> before testing for equality = >> >> >> Squeak Unicode strings may be tested for Unicode conformant equality >> by converting them to NFD before testing. >> >> >> >> Squeak using NFD >> >> asDecomposedUnicode[4] transforms a string into NFD for cases where a >> Unicode code point if decomposed, is decomposed only to two code >> points [5]. This is so because when initializing [6] the Unicode >> Character Database in Squeak this is a limitation imposed by the code >> which reads UnicodeData.txt [7][8]. This is not a necessary >> limitation. The code may be rewritten at the price of a more complex >> implementation of #asDecomposedUnicode. >> >> "Voulez-vous un caf??" >> eAcuteQuestion := 'Voulez-vous un caf', 16rE9 asCharacter asString, '?'. >> combinedEAcuteQuestion := 'Voulez-vous un cafe', 16r301 asCharacter >> asString, '?'. >> >> >> eAcuteQuestion asDecomposedUnicode = >> combinedEAcuteQuestion asDecomposedUnicode >> >> true >> >> >> >> Conclusion >> ------------------ >> >> Implementing a method like #decomposedStringWithCanonicalMapping >> (swift) which puts a string into NFD (Normalization Form D) is an >> important building block towards better Unicode compliance. A Squeak >> proposal is given by [4]. It needs to be reviewed.extended. >> >> It should probably be extended for cases where there are more than >> two code points in the decomposed form (3 or more?) >> >> The implementing of NFD comparison gives us an equality test for a >> comparatively small effort for simple cases covering a large number of >> use cases (Languages using the Latin script). >> >> The algorithm is table driven by the UCD [8]. From this follows an >> simple but important fact for conformant implementations need runtime >> access to information from the Unicode Character Database [UCD][9]. >> >> >> [1] >> https://developer.apple.com/library/ios/documentation/Swift/Conceptual/Swift_Programming_Language/StringsAndCharacters.html#//apple_ref/doc/uid/TP40014097-CH7-ID285 >> [2] http://www.unicode.org/glossary/#normalization_form_d >> [3] >> https://developer.apple.com/library/ios/documentation/Cocoa/Reference/Foundation/Classes/NSString_Class/index.html#//apple_ref/occ/instm/NSString/decomposedStringWithCanonicalMapping >> [4] String asDecomposedUnicode http://wiki.squeak.org/squeak/6250 >> [5] http://www.unicode.org/glossary/#code_point >> [6] Unicode initialize http://wiki.squeak.org/squeak/6248 >> [7] http://www.unicode.org/Public/UNIDATA/UnicodeData.txt >> [8] Unicode Character Database documentation >> http://unicode.org/reports/tr44/ >> [9] http://www.unicode.org/reports/tr23/ > > > Today, we have a Unicode and CombinedCharacter class in Pharo, and there is > different but similar Unicode code in Squeak. These are too simple (even > though they might work, partially). > > The scope of the original threads is way too wide: a new string type, > normalisation, collation, being cross dialect, mixing all kinds of character > and encoding definitions. All interesting, but not much will come out of it. > But the point that we cannot leave proper text string handling to an outside > library is indeed key. > > That is why a couple of people in the Pharo community (myself included) > started an experimental, proof of concept, prototype project, that aims to > improve Unicode support. We will announce it to a wider public when we feel > we have something to show for. The goal is in the first place to understand > and implement the fundamental algorithms, starting with the 4 forms of > Normalisation. But we're working on collation/sorting too. > > This work is of course being done for/in Pharo, using some of the facilities > only available there. It probably won't be difficult to port, but we can't > be bothered with probability right now. > > What we started with is loading UCD data and making it available as a nice > objects (30.000 of them). > > So now you can do things like > > $? unicodeCharacterData. > > => "U+00E9 LATIN SMALL LETTER E WITH ACUTE (LATIN SMALL LETTER E ACUTE)" > > $? unicodeCharacterData uppercase asCharacter. > > => "$?" > > $? unicodeCharacterData decompositionMapping. > > => "#(101 769)" > > There is also a cool GT Inspector view: > > > > Next we started implementing a normaliser. It was rather easy to get support > for simpler languages going. The next code snippets use explicit code > arrays, because copying decomposed diacritics to my mail client does not > work (they get automatically composed), in a Pharo Workspace this does work > nicely with plain strings. The higher numbers are the diacritics. > > (normalizer decomposeString: 'les ?l?ves Fran?ais') collect: #codePoint as: > Array. > > => "#(108 101 115 32 101 769 108 101 768 118 101 115 32 70 114 97 110 99 > 807 97 105 115)" > > (normalizer decomposeString: 'D?sseldorf K?nigsallee') collect: #codePoint > as: Array. > > => "#(68 117 776 115 115 101 108 100 111 114 102 32 75 111 776 110 105 103 > 115 97 108 108 101 101)" > > normalizer composeString: (#(108 101 115 32 101 769 108 101 768 118 101 115 > 32 70 114 97 110 99 807 97 105 115) collect: #asCharacter as: String). > > => "'les ?l?ves Fran?ais'" > > normalizer composeString: (#(68 117 776 115 115 101 108 100 111 114 102 32 > 75 111 776 110 105 103 115 97 108 108 101 101) collect: #asCharacter as: > String). > > => "'D?sseldorf K?nigsallee'" > > However, the real algorithm following the official specification (and other > elements of Unicode that interact with it) is way more complicated (think > about all those special languages/scripts out there). We're focused on > understanding/implementing that now. > > Next, unit tests were added (of course). As well as a test that uses > http://www.unicode.org/Public/UNIDATA/NormalizationTest.txt to run about > 75.000 individual test cases to check conformance to the official Unicode > Normalization specification. > > Right now (with super cool hangul / jamo code by Henrik), we hit the > following stats: > > #testNFC 16998/18593 (91.42%) > #testNFD 16797/18593 (90.34%) > #testNFKC 13321/18593 (71.65%) > #testNFKD 16564/18593 (89.09%) > > Way better than the naive implementations, but not yet there. > > We are also experimenting and thinking a lot about how to best implement all > this, trying out different models/ideas/apis/representations. > > It will move slowly, but you will hear from us again in the coming > weeks/months. > > Sven > > PS: Pharo developers with a good understanding of this subject area that > want to help, let me know and we'll put you in the loop. Hacking and > specification reading are required ;-) > > From peter at ozzard.org Fri Dec 18 14:25:38 2015 From: peter at ozzard.org (Peter Crowther) Date: Fri Dec 18 14:25:40 2015 Subject: [squeak-dev] OT: Convince me github is a wise choice In-Reply-To: References: Message-ID: On 17 December 2015 at 17:29, Eliot Miranda wrote: > So my concern is whether github is a safe long-term > bet. Specifically what is there to prevent some third party from buying > github, or of github going public and the board taking the decision, or > github on its own, deciding to charge for hosting, keeping the data hostage > to extract payment? What safeguards are in place to prevent this? I'm not > interested in "this will never happen" arguments. I'm interested in hard > data please. There is nothing in place commercially to prevent this, but there are some technical safeguards. Principal among these is that a clone of a repo includes a full version history. One or more of us could choose to "git clone" regularly from the github repo, and treat those as backups against github becoming unavailable or unreasonable. - Peter From Marcel.Taeumel at hpi.de Fri Dec 18 14:34:23 2015 From: Marcel.Taeumel at hpi.de (marcel.taeumel) Date: Fri Dec 18 14:52:43 2015 Subject: [squeak-dev] Re: CurrentReadOnlySourceFiles annoys me ... In-Reply-To: <1450269490206-4867327.post@n4.nabble.com> References: <1450256934663-4867253.post@n4.nabble.com> <1450269490206-4867327.post@n4.nabble.com> Message-ID: <1450449263326-4867752.post@n4.nabble.com> We might want to add something to support this: [ ... ] on: Exception - CurrentReadOnlyFiles do: [:ex | ... ]. Best, Marcel -- View this message in context: http://forum.world.st/CurrentReadOnlySourceFiles-annoys-me-tp4867253p4867752.html Sent from the Squeak - Dev mailing list archive at Nabble.com. From patrick.rein at hpi.de Fri Dec 18 15:39:42 2015 From: patrick.rein at hpi.de (Patrick R.) Date: Fri Dec 18 15:58:02 2015 Subject: [squeak-dev] Re: OT: Convince me github is a wise choice In-Reply-To: References: Message-ID: <1450453182285-4867767.post@n4.nabble.com> On this regard: We could for sure set up a server pulling all branches of the the repository periodically. Additionally, there are tools to back up the meta data associated with the repository (issues, wiki, etc.) :) http://heyrod.com/snippets/github-backup.html https://github.com/joeyh/github-backup - Patrick -- View this message in context: http://forum.world.st/OT-Convince-me-github-is-a-wise-choice-tp4867613p4867767.html Sent from the Squeak - Dev mailing list archive at Nabble.com. From robert.w.withers at gmail.com Fri Dec 18 19:10:38 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Fri Dec 18 19:10:49 2015 Subject: [squeak-dev] potential to include cryptography in base image In-Reply-To: <20151216030223.GA16566@shell.msen.com> References: <566FE4AC.4060803@gmail.com> <20151216030223.GA16566@shell.msen.com> Message-ID: <56745A2E.2080906@gmail.com> On 12/15/2015 10:02 PM, David T. Lewis wrote: > On Tue, Dec 15, 2015 at 05:00:12AM -0500, Robert Withers wrote: >> It was suggested to me that I write to the list and raise the question >> about cryptography being included in the base image. Really I have 3 >> questions I would ask you all: >> >> 1. is it desirable to include cryptography? >> 2. is it feasible to include cryptography? >> 3. what is the time frame for including cryptography? > I'm not sure whether it is a good idea to include it in the base image, > versus maintain it in an external package that can be easily loaded > from SqueakMap. Either way, I think we should get it updated on > SqueakMap first so that more people can (and hopefully will) load > it in their images, run the tests, and get experience with it. FWIW, > I personally tend to favor maintaining packages externally in cases > where we have an interest in supporting the package on various > kinds of images, such as Squeak/Cuis/Pharo. > > Robert, are you in a position to update the SqueakMap entries? Currently > they are "crypto", "Cryptography", and "Cryptography Team Package". They > are all out of date, and I'm not sure who owns which version. We should > pick one of them as the official one, and update it so that it loads the > latest version (Cryptography-rww.49) into a Squeak 4.6 or 5.0 image. > > It is of course quite easy to load Cryptography directly from squeaksource, > but there are quite a few packages in that repository that may seem confusing > for someone loading Cryptography for the first time, so I think a one-click > download from SqueakMap would be helpful. Thank you for your view of these matters. I must apologize for being forgetful about SqueakMap. I have no idea if I can login or not, it has been years. I have requested a new passwd so shortly I will be in. Do we want these artifacts on SqueakSource and SqueakMap? There are a lot of other packages in Cryptography on SqueakSource. I have cleaned up some of the packages I have in Mushroom. Some, like Blowfish, may want to be folded into the core package. Plugins will want to stay. I suppose better documentation would help, some for Pharo and some for Squeak. I got frustrated in the course of this as I think others did, as well. My apologies for simplifying a complex area and being to simplistic, at times. Regards, Robert > Dave > > -- . .. .. ^,^ robert From asqueaker at gmail.com Fri Dec 18 19:57:06 2015 From: asqueaker at gmail.com (Chris Muller) Date: Fri Dec 18 19:57:48 2015 Subject: [squeak-dev] potential to include cryptography in base image In-Reply-To: <56745A2E.2080906@gmail.com> References: <566FE4AC.4060803@gmail.com> <20151216030223.GA16566@shell.msen.com> <56745A2E.2080906@gmail.com> Message-ID: > > Do we want these artifacts on SqueakSource and SqueakMap? >> > Yes, please. Put the code on SqueakSouce and publish just the script for consuming it on SqueakMap. The best practice is to have one script that a fixed version into a fixed Squeak (e.g., currently 5.0) so there is always a working reference one can go back to, and then one more for the latest code. http://wiki.squeak.org/squeak/6182 has details and examples. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151218/5c8dd6a7/attachment.htm From robert.w.withers at gmail.com Fri Dec 18 20:05:40 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Fri Dec 18 20:05:54 2015 Subject: [squeak-dev] potential to include cryptography in base image In-Reply-To: References: <566FE4AC.4060803@gmail.com> <20151216030223.GA16566@shell.msen.com> <56745A2E.2080906@gmail.com> Message-ID: <56746714.9030302@gmail.com> I created a new account, since my old one has the wrong email that I can't access or change. Is there a way someone could update my email address to this one, please? Otherwise, I will need to get my new account added to the existing packages. robert On 12/18/2015 02:57 PM, Chris Muller wrote: > > Do we want these artifacts on SqueakSource and SqueakMap? > > > Yes, please. Put the code on SqueakSouce and publish just the script > for consuming it on SqueakMap. The best practice is to have one > script that a fixed version into a fixed Squeak (e.g., currently 5.0) > so there is always a working reference one can go back to, and then > one more for the latest code. > > http://wiki.squeak.org/squeak/6182 has details and examples. > > > -- . .. .. ^,^ robert -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151218/69d0a3cd/attachment-0001.htm From robert.w.withers at gmail.com Fri Dec 18 20:22:28 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Fri Dec 18 20:22:40 2015 Subject: [squeak-dev] Version 52... Re: version 52 of SecureSession is released In-Reply-To: <5673F8D8.8020106@gmail.com> References: <5673F31B.2090702@gmail.com> <5673F8D8.8020106@gmail.com> Message-ID: <56746B04.4000004@gmail.com> The newest version is cross-environment: squeakj & pharo, version 52. Don't run the perf test without plugins, please, to set the expectation. There is a class var to control whether RS encoding occurs (SecureSessionServer class>>#setUseFEC/useFEC:). By default it is disabled since it seriously affects performance. After profiling and cleanup, without FEC and with plugins, I consistently get over 20.5 Mbs. I think that's decent, though i am not sure where it measures with other crypto connections. 100 KB data chunks received: 22 send time: 417 ms, rate: 42.206 Mbs. receive time: 858 ms. rate: 20.513 Mbs. Hopefully we'll get RS pluginized. Then we could talk over ham radios, with non-commercial use. robert On 12/18/2015 07:15 AM, Robert Withers wrote: > Well, I must correct myself. It encodes/decodes in the absence of > errors, but it does not recover from damage, yet. It must be the > EuclideanAlgorithm it is using. I'll let you know when that's fixed, > but it is usable without damage, in the meantime. > > Sorry about that, > Robert > > On 12/18/2015 06:50 AM, Robert Withers wrote: >> SecureSession is now FEC encoded. >> >> thanks, > -- . .. .. ^,^ robert -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151218/5c8f221c/attachment.htm From asqueaker at gmail.com Fri Dec 18 20:45:46 2015 From: asqueaker at gmail.com (Chris Muller) Date: Fri Dec 18 20:46:29 2015 Subject: [squeak-dev] potential to include cryptography in base image In-Reply-To: <56746714.9030302@gmail.com> References: <566FE4AC.4060803@gmail.com> <20151216030223.GA16566@shell.msen.com> <56745A2E.2080906@gmail.com> <56746714.9030302@gmail.com> Message-ID: Yes. I've just set the #email: of your 'rww' account to your gmail address. For sake of Catalog model cleanliness, if you wouldn't deleting your new account, it would be appreciated. On Fri, Dec 18, 2015 at 2:05 PM, Robert Withers wrote: > I created a new account, since my old one has the wrong email that I can't > access or change. Is there a way someone could update my email address to > this one, please? Otherwise, I will need to get my new account added to the > existing packages. > > robert > > > On 12/18/2015 02:57 PM, Chris Muller wrote: > > Do we want these artifacts on SqueakSource and SqueakMap? >>> >> > Yes, please. Put the code on SqueakSouce and publish just the script for > consuming it on SqueakMap. The best practice is to have one script that a > fixed version into a fixed Squeak (e.g., currently 5.0) so there is always > a working reference one can go back to, and then one more for the latest > code. > > http://wiki.squeak.org/squeak/6182 has details and examples. > > > > > -- > . .. .. ^,^ robert > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151218/9f0d08b7/attachment.htm From sean at clipperadams.com Fri Dec 18 22:26:34 2015 From: sean at clipperadams.com (Sean P. DeNigris) Date: Fri Dec 18 22:44:56 2015 Subject: [squeak-dev] Re: Name change: Mushroom ( was Re: evolutions of squeakelib & crypto) In-Reply-To: <5666235C.9000906@gmail.com> References: <56616F27.4070002@gmail.com> <566175EA.2090107@gmail.com> <5661B7F1.1080701@gmail.com> <5663A08C.8030602@gmail.com> <5665A1AE.2030706@gmail.com> <56660EC6.7030804@gmail.com> <5666235C.9000906@gmail.com> Message-ID: <1450477594020-4867855.post@n4.nabble.com> Robert Withers wrote > Perhaps I should use technical names and be done with it Yes, please! I have used Squeak and Pharo almost every day for 6 years, closely following the mailing lists, and still find the buffet of clever project names baffling. Unless we're not trying to take over the world with a particular project (like maybe Seaside, which could've blown Rails out of the water given a little luck), I don't see what we gain in trade giving up clarity. ----- Cheers, Sean -- View this message in context: http://forum.world.st/evolutions-of-squeakelib-crypto-Reed-Solomon-tp4864527p4867855.html Sent from the Squeak - Dev mailing list archive at Nabble.com. From robert.w.withers at gmail.com Fri Dec 18 22:50:43 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Fri Dec 18 22:50:55 2015 Subject: [squeak-dev] Re: Name change: Mushroom ( was Re: evolutions of squeakelib & crypto) In-Reply-To: <1450477594020-4867855.post@n4.nabble.com> References: <56616F27.4070002@gmail.com> <566175EA.2090107@gmail.com> <5661B7F1.1080701@gmail.com> <5663A08C.8030602@gmail.com> <5665A1AE.2030706@gmail.com> <56660EC6.7030804@gmail.com> <5666235C.9000906@gmail.com> <1450477594020-4867855.post@n4.nabble.com> Message-ID: <56748DC3.5090804@gmail.com> Branding and marketing connected to mythology, perhaps. I really don't know. There is something of value in a well-known name that brings inspiration; this is typically false inspiration, though, so I do see your point. robert On 12/18/2015 05:26 PM, Sean P. DeNigris wrote: > Robert Withers wrote >> Perhaps I should use technical names and be done with it > Yes, please! I have used Squeak and Pharo almost every day for 6 years, > closely following the mailing lists, and still find the buffet of clever > project names baffling. Unless we're not trying to take over the world with > a particular project (like maybe Seaside, which could've blown Rails out of > the water given a little luck), I don't see what we gain in trade giving up > clarity. > > > > ----- > Cheers, > Sean > -- > View this message in context: http://forum.world.st/evolutions-of-squeakelib-crypto-Reed-Solomon-tp4864527p4867855.html > Sent from the Squeak - Dev mailing list archive at Nabble.com. > -- . .. .. ^,^ robert From robert.w.withers at gmail.com Fri Dec 18 23:03:45 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Fri Dec 18 23:03:57 2015 Subject: [squeak-dev] Re: Name change: Mushroom ( was Re: evolutions of squeakelib & crypto) In-Reply-To: <1450477594020-4867855.post@n4.nabble.com> References: <56616F27.4070002@gmail.com> <566175EA.2090107@gmail.com> <5661B7F1.1080701@gmail.com> <5663A08C.8030602@gmail.com> <5665A1AE.2030706@gmail.com> <56660EC6.7030804@gmail.com> <5666235C.9000906@gmail.com> <1450477594020-4867855.post@n4.nabble.com> Message-ID: <567490D1.9050003@gmail.com> On 12/18/2015 05:26 PM, Sean P. DeNigris wrote: > Robert Withers wrote >> Perhaps I should use technical names and be done with it > Yes, please! I have used Squeak and Pharo almost every day for 6 years, > closely following the mailing lists, and still find the buffet of clever > project names baffling. Unless we're not trying to take over the world with > a particular project (like maybe Seaside, which could've blown Rails out of > the water given a little luck), I don't see what we gain in trade giving up > clarity. What do you think would make a good name for what I am calling Mushroom, which was SqueakElib, but is really the 6th presentation layer above what are now SecureSession? It establishes a modified CapTP protocol (no searchpath & not all working). It is the presentation layer for distributed objects with promise pipelining : messages are sent to future results, and distributed result continuations. It's pretty neat. 5, 2. I am not sure what name accurately names this, what it does, where it is. distributed object computer? > > > > ----- > Cheers, > Sean > -- > View this message in context: http://forum.world.st/evolutions-of-squeakelib-crypto-Reed-Solomon-tp4864527p4867855.html > Sent from the Squeak - Dev mailing list archive at Nabble.com. > -- . .. .. ^,^ robert From huw.softdesigns at gmail.com Fri Dec 18 23:19:26 2015 From: huw.softdesigns at gmail.com (Huw Lloyd) Date: Fri Dec 18 23:19:29 2015 Subject: [squeak-dev] Re: Name change: Mushroom ( was Re: evolutions of squeakelib & crypto) In-Reply-To: <567490D1.9050003@gmail.com> References: <56616F27.4070002@gmail.com> <566175EA.2090107@gmail.com> <5661B7F1.1080701@gmail.com> <5663A08C.8030602@gmail.com> <5665A1AE.2030706@gmail.com> <56660EC6.7030804@gmail.com> <5666235C.9000906@gmail.com> <1450477594020-4867855.post@n4.nabble.com> <567490D1.9050003@gmail.com> Message-ID: Name it by its defining responsibilities, not how it is implemented. It sounds like you have something that creates/establishes/maintains/manages CapTP sessions/connections. Best, Huw On 18 December 2015 at 23:03, Robert Withers wrote: > > > On 12/18/2015 05:26 PM, Sean P. DeNigris wrote: > >> Robert Withers wrote >> >>> Perhaps I should use technical names and be done with it >>> >> Yes, please! I have used Squeak and Pharo almost every day for 6 years, >> closely following the mailing lists, and still find the buffet of clever >> project names baffling. Unless we're not trying to take over the world >> with >> a particular project (like maybe Seaside, which could've blown Rails out >> of >> the water given a little luck), I don't see what we gain in trade giving >> up >> clarity. >> > What do you think would make a good name for what I am calling Mushroom, > which was SqueakElib, but is really the 6th presentation layer above what > are now SecureSession? It establishes a modified CapTP protocol (no > searchpath & not all working). It is the presentation layer for distributed > objects with promise pipelining : messages are sent to future results, and > distributed result continuations. It's pretty neat. 5, 2. I am not > sure what name accurately names this, what it does, where it is. > > distributed object computer? > > >> >> >> ----- >> Cheers, >> Sean >> -- >> View this message in context: >> http://forum.world.st/evolutions-of-squeakelib-crypto-Reed-Solomon-tp4864527p4867855.html >> Sent from the Squeak - Dev mailing list archive at Nabble.com. >> >> > -- > . .. .. ^,^ robert > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151218/530b091d/attachment.htm From leves at caesar.elte.hu Fri Dec 18 23:55:45 2015 From: leves at caesar.elte.hu (Levente Uzonyi) Date: Fri Dec 18 23:55:50 2015 Subject: [squeak-dev] Re: CurrentReadOnlySourceFiles annoys me ... In-Reply-To: <1450269490206-4867327.post@n4.nabble.com> References: <1450256934663-4867253.post@n4.nabble.com> <1450269490206-4867327.post@n4.nabble.com> Message-ID: On Wed, 16 Dec 2015, marcel.taeumel wrote: > I want to catch all exceptions because it is kind of test code where only > success matters. :-) It is an experiment where the participants should > program stuff and receive a green tick whenever the task is accomplished. > Raising a debugger when the participant puts a "self halt" or anything into > the control flow of the test code would confuse the participant. In that case, you should catch subinstances of Exception. An Exception is not an Error, it won't raise a debugger. Actually it will do nothing by default. IMHO you should catch what needs to be cached: Error, Abort, UnhandledError, Halt >From that list, Abort seems to be unused in the image, so it's a candidate for removal. > > Conceptionally, CurrentReadOnlySourceFiles is no kind of an exception. Thank It is an exception. You "throw" it, and when it's resumed, it'll contain read-only source files. > you for the explanation. Can you sketch the steps to transform that into a > PLV? Why would it take some effort? Make a new PLV variable which implements the same API, then rewrite the users. It takes effort, because it must work while it's being used. Levente > > Bset, > Marcel > > > > -- > View this message in context: http://forum.world.st/CurrentReadOnlySourceFiles-annoys-me-tp4867253p4867327.html > Sent from the Squeak - Dev mailing list archive at Nabble.com. > > From leves at caesar.elte.hu Sat Dec 19 00:05:47 2015 From: leves at caesar.elte.hu (Levente Uzonyi) Date: Sat Dec 19 00:05:51 2015 Subject: [squeak-dev] Re: CurrentReadOnlySourceFiles annoys me ... In-Reply-To: References: <1450256934663-4867253.post@n4.nabble.com> <1450269490206-4867327.post@n4.nabble.com> <35DA6F81-42C6-4761-B43E-CD5593EF7953@gmail.com> Message-ID: Hi Eliot, I don't remember the exact case which made the Pharo guys do this change, but if you implement a web service, which shows source code on a web page, and reads the code from the image, then a non-UI process will read the source code from SourceFiles, which can lead to the problems I've described. IIRC Seaside has such example service. Levente On Wed, 16 Dec 2015, Eliot Miranda wrote: > Hi Levente, > On Wed, Dec 16, 2015 at 2:43 PM, Levente Uzonyi wrote: > My objection is that concurrent access of the read-only source files would cause race conditions. > We might simplify the thing by using the regular source files (or some shared read-only copies) when requested from the UI process, and create a read-only copy for all other processes. That would cover most potential slow-downs in the Trunk. > > > Apart form the debugger can you think of situations in which this will really happen?? I can't. If you can convince me that concurrent access is a real danger then I'll think again, but in all my time doing Smalltalk with Smalltalk-80 v2, with VisualWorks and with > Croquet, Squeak and Pharo the only time this has been an issue was in the debugger debugging source file access.? Given that that's easy to solve I don't see the point of living with such slow source file access for any longer :-) > > ? > > > Levente > > On Wed, 16 Dec 2015, Eliot Miranda wrote: > > Hi all, > > ? ?IMO we don't need CurrentReadOnkySourceFiles, instead we should have SourceFilesArray maintain one read-only copy per source file.? The objection is that using a single file will make debugging source file access break, since the debuggers > own access to source will perturb the source files whose access is being debugged, but the debugger can easily temporarily install a different read-init copy whenever it accesses source, insulating the debugged code from this effect. > > _,,,^..^,,,_ (phone) > > On Dec 16, 2015, at 4:38 AM, marcel.taeumel wrote: > > I want to catch all exceptions because it is kind of test code where only > success matters. :-)? It is an experiment where the participants should > program stuff and receive a green tick whenever the task is accomplished. > Raising a debugger when the participant puts a "self halt" or anything into > the control flow of the test code would confuse the participant. > > Conceptionally, CurrentReadOnlySourceFiles is no kind of an exception. Thank > you for the explanation. Can you sketch the steps to transform that into a > PLV? Why would it take some effort? > > Bset, > Marcel > > > > -- > View this message in context: http://forum.world.st/CurrentReadOnlySourceFiles-annoys-me-tp4867253p4867327.html > Sent from the Squeak - Dev mailing list archive at Nabble.com. > > > > > > > > -- > _,,,^..^,,,_ > best,?Eliot > > From leves at caesar.elte.hu Sat Dec 19 00:08:47 2015 From: leves at caesar.elte.hu (Levente Uzonyi) Date: Sat Dec 19 00:08:53 2015 Subject: [squeak-dev] [Unicode] Summary (Re: [Pharo-dev] Unicode Support // e acute example --> decomposition in Pharo?) In-Reply-To: References: Message-ID: Isn't it strange that the development is not open source? Levente On Fri, 18 Dec 2015, H. Hirzel wrote: > Hello Sven > > Thank you for your report about about your experimental, proof of > concept, prototype project, that aims to improve Unicode support. > Please include me in the loop. > > Below is is my attempt at summarizing the Unicode discussion of the last weeks. > Corrections /comments / additions are welcome. > > Kind regards > > Hannes > > > 1) There is a need for improved Unicode support implemented _within_ > the image , probably as a library. > > 1a) This follows the example of the the Twitter CLDR library (i.e. > re-implementation of ICU components for Ruby). > https://github.com/twitter/twitter-cldr-rb > > Other languages/libraries have similar approaches > - dotNet, https://msdn.microsoft.com/en-us/library/System.Globalization.CharUnicodeInfo%28v=vs.110%29.aspx) > - Python https://docs.python.org/3/howto/unicode.html > - Go http://blog.golang.org/strings > - Swift, https://developer.apple.com/library/ios/documentation/Swift/Conceptual/Swift_Programming_Language/StringsAndCharacters.html > - Perl http://blog.golang.org/strings > > 1b) ICU is _not_ the way to go (http://site.icu-project.org/) . This > is because of security and portability reasons (Eliot Miranda) and > because of the Smalltalk approach that wants to expose basic > algorithms in Smalltalk code. In addition the 16bit based ICU library > does not fit well with the Squeak/Pharo UTF32 model. > > 2) The Unicode infrastructure (21(32) bit wide Characters as immediate > objects, use of UTF-32 internally, indexable strings, UTF8 for outside > communication, support of code converters) is a very valuable > foundation which makes algorithms more straightforward at the expense > of a more memory usage. It not used to its full potential at all > currently though a lot of hard work has been done. > > 3) The Unicode algorithms are mostly table / database driven. This > means that dictionary lookup is a prominent part of the algorithms. > The essential building block for this is that the Unicode character > database UCD (http://www.unicode.org/ucd/) is made available > _within_ the image with the full content as needed by the target > languages / scripts one wants to deal with. The process of loading the > UCD should be made configurable. > > 3a) a lot of people are interested in the Latin script (and scripts of > similar complexity) only. > 3b) The UCD data in XML form > http://www.unicode.org/Public/8.0.0/ucdxml/ offers a download with > and without the CJK characters. > > 4) The next step is to implement normalization > (http://www.unicode.org/reports/tr15/#Norm_Forms). Glad to read that > you have reached results here with the test data: > http://www.unicode.org/Public/UNIDATA/NormalizationTest.txt. > > 5) Pharo offers nice inspectors to view dictionaries and ordered > collections (table view, drill down) which facilitates the development > to table driven algorithms. The data structures and algorithm are do > not depend on a particular dialect though and may be ported to Squeak > or Cuis. > > 6) After having implemented normalization, comparison may be > implemented. This needs CLDR access (collation, Unicode Common Locale > Data Repository, http://cldr.unicode.org/ ). > > > 7) An architecture has the following subsystems > > 7a) Basic character handling (21(32)bit characters in indexable > strings, point 2) > 7b) Runtime access to the Unicode Character Database (point 3) > 7c) Converters > 7d) Normalization (point 4) > 7e) CLDR access (point 6) > > > 8) The implementation should be driven by the current needs. > > An attainable next goal is to release > > 8a) a StringBuilder utility class for easier construction of test strings > i.e. instead of > >> normalizer composeString: (#(68 117 776 115 115 101 108 100 111 114 102 32 >> 75 111 776 110 105 103 115 97 108 108 101 101) collect: #asCharacter as: >> String). > > do > normalizer composeString: > (StringBuilder construct: 'Du\u0308sseldorf Ko\u0308nigsallee') > > and construct some test cases with it which illustrate some basic > Unicode issues. > > 8b) identity testing for major languages (e.g. French, German, > Spanish) and scripts of similar complexity. I > > 8c) to provide some more documentation of past and concurrent efforts. > > Note: This summary has only covered string manipulation, not rendering > on the screen which is a different issue. > > > On 12/16/15, Sven Van Caekenberghe wrote: >> Hi Hannes, >> >> My detailed comments/answers below, after quoting 2 of your emails: >> >>> On 10 Dec 2015, at 22:17, H. Hirzel wrote: >>> >>> Hello Sven >>> >>> On 12/9/15, Sven Van Caekenberghe wrote: >>> >>>> The simplest example in a common language is (the French letter ?) is >>>> >>>> LATIN SMALL LETTER E WITH ACUTE [U+00E9] >>>> >>>> which can also be written as >>>> >>>> LATIN SMALL LETTER E [U+0065] followed by COMBINING ACUTE ACCENT >>>> [U+0301] >>>> >>>> The former being a composed normal form, the latter a decomposed normal >>>> form. (And yes, it is even much more complicated than that, it goes on >>>> for >>>> 1000s of pages). >>>> >>>> In the above example, the concept of character/string is indeed fuzzy. >>>> >>>> HTH, >>>> >>>> Sven >>> >>> Thanks for this example. I have created a wiki page with it >>> >>> I wonder what the Pharo equivalent is of the following Squeak expression >>> >>> $? asString asDecomposedUnicode >>> >>> Regards >>> >>> Hannes >> >> You also wrote: >> >>> The text below shows how to deal with the Unicode e acute example >>> brought up by Sven in terms of comparing strings. Currently Pharo and >>> Cuis do not do Normalization of strings. Limited support is in Squeak. >>> It will be shown how NFD normalization may be implemented. >>> >>> >>> Swift programming language >>> ----------------------------------------- >>> >>> How does the Swift programming language [1] deal with Unicode strings? >>> >>> // "Voulez-vous un caf??" using LATIN SMALL LETTER E WITH ACUTE >>> let eAcuteQuestion = "Voulez-vous un caf\u{E9}?" >>> >>> // "Voulez-vous un café?" using LATIN SMALL LETTER E and >>> COMBINING ACUTE ACCENT >>> let combinedEAcuteQuestion = "Voulez-vous un caf\u{65}\u{301}?" >>> >>> if eAcuteQuestion == combinedEAcuteQuestion { >>> print("These two strings are considered equal") >>> } >>> // prints "These two strings are considered equal" >>> >>> The equality operator uses the NFD (Normalization Form Decomposed)[2] >>> form for the comparison appyling a method >>> #decomposedStringWithCanonicalMapping[3] >>> >>> >>> Squeak / Pharo >>> ----------------------- >>> >>> Comparison without NFD [3] >>> >>> >>> "Voulez-vous un caf??" >>> eAcuteQuestion := 'Voulez-vous un caf', 16rE9 asCharacter asString, '?'. >>> combinedEAcuteQuestion := 'Voulez-vous un cafe', 16r301 asCharacter >>> asString, '?'. >>> >>> >>> eAcuteQuestion = combinedEAcuteQuestion >>> false >>> >>> eAcuteQuestion == combinedEAcuteQuestion >>> false >>> >>> The result is false. A Unicode conformant application however should >>> return *true*. >>> >>> Reason for this is that Squeak / Pharo strings are not put into NFD >>> before testing for equality = >>> >>> >>> Squeak Unicode strings may be tested for Unicode conformant equality >>> by converting them to NFD before testing. >>> >>> >>> >>> Squeak using NFD >>> >>> asDecomposedUnicode[4] transforms a string into NFD for cases where a >>> Unicode code point if decomposed, is decomposed only to two code >>> points [5]. This is so because when initializing [6] the Unicode >>> Character Database in Squeak this is a limitation imposed by the code >>> which reads UnicodeData.txt [7][8]. This is not a necessary >>> limitation. The code may be rewritten at the price of a more complex >>> implementation of #asDecomposedUnicode. >>> >>> "Voulez-vous un caf??" >>> eAcuteQuestion := 'Voulez-vous un caf', 16rE9 asCharacter asString, '?'. >>> combinedEAcuteQuestion := 'Voulez-vous un cafe', 16r301 asCharacter >>> asString, '?'. >>> >>> >>> eAcuteQuestion asDecomposedUnicode = >>> combinedEAcuteQuestion asDecomposedUnicode >>> >>> true >>> >>> >>> >>> Conclusion >>> ------------------ >>> >>> Implementing a method like #decomposedStringWithCanonicalMapping >>> (swift) which puts a string into NFD (Normalization Form D) is an >>> important building block towards better Unicode compliance. A Squeak >>> proposal is given by [4]. It needs to be reviewed.extended. >>> >>> It should probably be extended for cases where there are more than >>> two code points in the decomposed form (3 or more?) >>> >>> The implementing of NFD comparison gives us an equality test for a >>> comparatively small effort for simple cases covering a large number of >>> use cases (Languages using the Latin script). >>> >>> The algorithm is table driven by the UCD [8]. From this follows an >>> simple but important fact for conformant implementations need runtime >>> access to information from the Unicode Character Database [UCD][9]. >>> >>> >>> [1] >>> https://developer.apple.com/library/ios/documentation/Swift/Conceptual/Swift_Programming_Language/StringsAndCharacters.html#//apple_ref/doc/uid/TP40014097-CH7-ID285 >>> [2] http://www.unicode.org/glossary/#normalization_form_d >>> [3] >>> https://developer.apple.com/library/ios/documentation/Cocoa/Reference/Foundation/Classes/NSString_Class/index.html#//apple_ref/occ/instm/NSString/decomposedStringWithCanonicalMapping >>> [4] String asDecomposedUnicode http://wiki.squeak.org/squeak/6250 >>> [5] http://www.unicode.org/glossary/#code_point >>> [6] Unicode initialize http://wiki.squeak.org/squeak/6248 >>> [7] http://www.unicode.org/Public/UNIDATA/UnicodeData.txt >>> [8] Unicode Character Database documentation >>> http://unicode.org/reports/tr44/ >>> [9] http://www.unicode.org/reports/tr23/ >> >> >> Today, we have a Unicode and CombinedCharacter class in Pharo, and there is >> different but similar Unicode code in Squeak. These are too simple (even >> though they might work, partially). >> >> The scope of the original threads is way too wide: a new string type, >> normalisation, collation, being cross dialect, mixing all kinds of character >> and encoding definitions. All interesting, but not much will come out of it. >> But the point that we cannot leave proper text string handling to an outside >> library is indeed key. >> >> That is why a couple of people in the Pharo community (myself included) >> started an experimental, proof of concept, prototype project, that aims to >> improve Unicode support. We will announce it to a wider public when we feel >> we have something to show for. The goal is in the first place to understand >> and implement the fundamental algorithms, starting with the 4 forms of >> Normalisation. But we're working on collation/sorting too. >> >> This work is of course being done for/in Pharo, using some of the facilities >> only available there. It probably won't be difficult to port, but we can't >> be bothered with probability right now. >> >> What we started with is loading UCD data and making it available as a nice >> objects (30.000 of them). >> >> So now you can do things like >> >> $? unicodeCharacterData. >> >> => "U+00E9 LATIN SMALL LETTER E WITH ACUTE (LATIN SMALL LETTER E ACUTE)" >> >> $? unicodeCharacterData uppercase asCharacter. >> >> => "$?" >> >> $? unicodeCharacterData decompositionMapping. >> >> => "#(101 769)" >> >> There is also a cool GT Inspector view: >> >> >> >> Next we started implementing a normaliser. It was rather easy to get support >> for simpler languages going. The next code snippets use explicit code >> arrays, because copying decomposed diacritics to my mail client does not >> work (they get automatically composed), in a Pharo Workspace this does work >> nicely with plain strings. The higher numbers are the diacritics. >> >> (normalizer decomposeString: 'les ?l?ves Fran?ais') collect: #codePoint as: >> Array. >> >> => "#(108 101 115 32 101 769 108 101 768 118 101 115 32 70 114 97 110 99 >> 807 97 105 115)" >> >> (normalizer decomposeString: 'D?sseldorf K?nigsallee') collect: #codePoint >> as: Array. >> >> => "#(68 117 776 115 115 101 108 100 111 114 102 32 75 111 776 110 105 103 >> 115 97 108 108 101 101)" >> >> normalizer composeString: (#(108 101 115 32 101 769 108 101 768 118 101 115 >> 32 70 114 97 110 99 807 97 105 115) collect: #asCharacter as: String). >> >> => "'les ?l?ves Fran?ais'" >> >> normalizer composeString: (#(68 117 776 115 115 101 108 100 111 114 102 32 >> 75 111 776 110 105 103 115 97 108 108 101 101) collect: #asCharacter as: >> String). >> >> => "'D?sseldorf K?nigsallee'" >> >> However, the real algorithm following the official specification (and other >> elements of Unicode that interact with it) is way more complicated (think >> about all those special languages/scripts out there). We're focused on >> understanding/implementing that now. >> >> Next, unit tests were added (of course). As well as a test that uses >> http://www.unicode.org/Public/UNIDATA/NormalizationTest.txt to run about >> 75.000 individual test cases to check conformance to the official Unicode >> Normalization specification. >> >> Right now (with super cool hangul / jamo code by Henrik), we hit the >> following stats: >> >> #testNFC 16998/18593 (91.42%) >> #testNFD 16797/18593 (90.34%) >> #testNFKC 13321/18593 (71.65%) >> #testNFKD 16564/18593 (89.09%) >> >> Way better than the naive implementations, but not yet there. >> >> We are also experimenting and thinking a lot about how to best implement all >> this, trying out different models/ideas/apis/representations. >> >> It will move slowly, but you will hear from us again in the coming >> weeks/months. >> >> Sven >> >> PS: Pharo developers with a good understanding of this subject area that >> want to help, let me know and we'll put you in the loop. Hacking and >> specification reading are required ;-) >> >> > > From tim at rowledge.org Sat Dec 19 00:22:56 2015 From: tim at rowledge.org (tim Rowledge) Date: Sat Dec 19 00:23:02 2015 Subject: [squeak-dev] MenuMorph>popUpAdjacent... seems to mis-position sub-menus Message-ID: <9300D1F7-25AB-4A75-86E8-B59D90727116@rowledge.org> When a menu has sub-menus they are placed to the right of the parent but slid back a little left so the overlap and the pointer travelling rightwards to the new menu will not suddenly trigger mouseleave vents and close the menu tree unexpectedly. If however the parent menu is nearly at the right side of the screen, the sub-menu has to be placed to the left. So far as I can work out the current code gets this slightly wrong and places it too far left so that the mouse does indeed tend to send a mouseleave event etc. Result is very annoying for users since they can?t select what they wanted to. The culprit appears to me to be in the MenuMorph>popUpAdjacentTo:forHand:from: method where ?leftPoint? is calculated. We subtract the layoutinset and borderwidth as well as the width; whereas I?m fairly sure it ought to be *add* the layoutinset and borderwidth but subtract the width. Certainly it seems to provide a better visual layout. I can?t see any cases that get broken by such a change but since it?s hard to be sure I?ll just commit this to the inbox for a couple of days before trunking it. tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim People who deal with bits should expect to get bitten. From commits at source.squeak.org Sat Dec 19 00:27:16 2015 From: commits at source.squeak.org (commits@source.squeak.org) Date: Sat Dec 19 00:27:19 2015 Subject: [squeak-dev] The Inbox: Morphic-tpr.1059.mcz Message-ID: tim Rowledge uploaded a new version of Morphic to project The Inbox: http://source.squeak.org/inbox/Morphic-tpr.1059.mcz ==================== Summary ==================== Name: Morphic-tpr.1059 Author: tpr Time: 18 December 2015, 4:26:36.049 pm UUID: ce2af9d7-dcb1-4225-8fea-1a0b3ae28ab4 Ancestors: Morphic-tpr.1058 When opening sub-menus with insufficient rightwards space on the Display we instead put them to the left of the parent. This change makes the leftward slide a bit smaller so that the mouse stays over menus was the user slides to the new menu; thus no strange closing of menus etc. =============== Diff against Morphic-tpr.1058 =============== Item was changed: ----- Method: MenuMorph>>popUpAdjacentTo:forHand:from: (in category 'control') ----- popUpAdjacentTo: rightOrLeftPoint forHand: hand from: sourceItem "Present this menu at the given point under control of the given hand." | tryToPlace selectedOffset rightPoint leftPoint | hand world startSteppingSubmorphsOf: self. popUpOwner := sourceItem. self fullBounds. self updateColor. "ensure layout is current" selectedOffset := (selectedItem ifNil: [self items first]) position - self position. tryToPlace := [:where :mustFit | | delta | self position: where - selectedOffset. delta := self boundsInWorld amountToTranslateWithin: sourceItem worldBounds. (delta x = 0 or: [mustFit]) ifTrue: [delta = (0 @ 0) ifFalse: [self position: self position + delta]. sourceItem world addMorphFront: self. ^ self]]. rightPoint := rightOrLeftPoint first + ((self layoutInset + self borderWidth) @ 0). + leftPoint := rightOrLeftPoint last + ((self layoutInset + self borderWidth - self width) @ 0). - leftPoint := rightOrLeftPoint last - ((self layoutInset + self borderWidth + self width) @ 0). tryToPlace value: rightPoint value: false; value: leftPoint value: false; value: rightPoint value: true.! From robert.w.withers at gmail.com Sat Dec 19 00:59:20 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Sat Dec 19 00:59:27 2015 Subject: [squeak-dev] compression problem Message-ID: <5674ABE8.9060903@gmail.com> I am seeing a bitshift: problem using GZip. I can show you, it's east to test. To replicate, load Crypto and SecureSession.58. In the SecureSessionServer>>#initializeOnVatTPIdentity: you can set compressed to true. Then run the connection and data tests. install3DesNoCompressionOn: is where it is used. Thank you, -- . .. .. ^,^ robert From robert.w.withers at gmail.com Sat Dec 19 01:15:49 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Sat Dec 19 01:15:55 2015 Subject: [squeak-dev] Fwd: picking up a little speed In-Reply-To: <5674AE4A.8070706@gmail.com> References: <5674AE4A.8070706@gmail.com> Message-ID: <5674AFC5.2050908@gmail.com> This is without encryption. I think that is 15 MB/sec. Will video fit in that throughput? 100 KB data chunks received: 22 send time: 54 ms, rate: 325.926 Mbs. receive time: 143 ms. rate: 123.077 Mbs. regards, robert -------- Forwarded Message -------- Subject: picking up a little speed Date: Fri, 18 Dec 2015 20:09:30 -0500 From: Robert Withers To: Pharo Development List I was looking at Zinc to see how it works. My SecureSession was built without my knowing about all of Zinc and it looks delicious on the control side. Here is my latest stats: 100 KB data chunks received: 22 send time: 54 ms, rate: 325.926 Mbs. receive time: 143 ms. rate: 123.077 Mbs. -- . .. .. ^,^ robert -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151218/fdaa58af/attachment.htm From avalloud at smalltalk.comcastbiz.net Sat Dec 19 02:04:45 2015 From: avalloud at smalltalk.comcastbiz.net (Andres Valloud) Date: Sat Dec 19 02:04:48 2015 Subject: [squeak-dev] [Unicode] Summary (Re: [Pharo-dev] Unicode Support // e acute example --> decomposition in Pharo?) In-Reply-To: References: Message-ID: <5674BB3D.9010200@smalltalk.comcastbiz.net> So a lot of Windows APIs require UTF-16. What's up with UTF-8 being the only choice mentioned for external communication? Unicode string encodings like UTF-* and strings of "characters" (that is, sequences of Unicode code points) should be clearly distinguished. Do you really mean "UTF-32", or do you mean "UCS-4"? Even those two are not exactly the same. On 12/18/15 5:47 , H. Hirzel wrote: > Hello Sven > > Thank you for your report about about your experimental, proof of > concept, prototype project, that aims to improve Unicode support. > Please include me in the loop. > > Below is is my attempt at summarizing the Unicode discussion of the last weeks. > Corrections /comments / additions are welcome. > > Kind regards > > Hannes > > > 1) There is a need for improved Unicode support implemented _within_ > the image , probably as a library. > > 1a) This follows the example of the the Twitter CLDR library (i.e. > re-implementation of ICU components for Ruby). > https://github.com/twitter/twitter-cldr-rb > > Other languages/libraries have similar approaches > - dotNet, https://msdn.microsoft.com/en-us/library/System.Globalization.CharUnicodeInfo%28v=vs.110%29.aspx) > - Python https://docs.python.org/3/howto/unicode.html > - Go http://blog.golang.org/strings > - Swift, https://developer.apple.com/library/ios/documentation/Swift/Conceptual/Swift_Programming_Language/StringsAndCharacters.html > - Perl http://blog.golang.org/strings > > 1b) ICU is _not_ the way to go (http://site.icu-project.org/) . This > is because of security and portability reasons (Eliot Miranda) and > because of the Smalltalk approach that wants to expose basic > algorithms in Smalltalk code. In addition the 16bit based ICU library > does not fit well with the Squeak/Pharo UTF32 model. > > 2) The Unicode infrastructure (21(32) bit wide Characters as immediate > objects, use of UTF-32 internally, indexable strings, UTF8 for outside > communication, support of code converters) is a very valuable > foundation which makes algorithms more straightforward at the expense > of a more memory usage. It not used to its full potential at all > currently though a lot of hard work has been done. > > 3) The Unicode algorithms are mostly table / database driven. This > means that dictionary lookup is a prominent part of the algorithms. > The essential building block for this is that the Unicode character > database UCD (http://www.unicode.org/ucd/) is made available > _within_ the image with the full content as needed by the target > languages / scripts one wants to deal with. The process of loading the > UCD should be made configurable. > > 3a) a lot of people are interested in the Latin script (and scripts of > similar complexity) only. > 3b) The UCD data in XML form > http://www.unicode.org/Public/8.0.0/ucdxml/ offers a download with > and without the CJK characters. > > 4) The next step is to implement normalization > (http://www.unicode.org/reports/tr15/#Norm_Forms). Glad to read that > you have reached results here with the test data: > http://www.unicode.org/Public/UNIDATA/NormalizationTest.txt. > > 5) Pharo offers nice inspectors to view dictionaries and ordered > collections (table view, drill down) which facilitates the development > to table driven algorithms. The data structures and algorithm are do > not depend on a particular dialect though and may be ported to Squeak > or Cuis. > > 6) After having implemented normalization, comparison may be > implemented. This needs CLDR access (collation, Unicode Common Locale > Data Repository, http://cldr.unicode.org/ ). > > > 7) An architecture has the following subsystems > > 7a) Basic character handling (21(32)bit characters in indexable > strings, point 2) > 7b) Runtime access to the Unicode Character Database (point 3) > 7c) Converters > 7d) Normalization (point 4) > 7e) CLDR access (point 6) > > > 8) The implementation should be driven by the current needs. > > An attainable next goal is to release > > 8a) a StringBuilder utility class for easier construction of test strings > i.e. instead of > >> normalizer composeString: (#(68 117 776 115 115 101 108 100 111 114 102 32 >> 75 111 776 110 105 103 115 97 108 108 101 101) collect: #asCharacter as: >> String). > > do > normalizer composeString: > (StringBuilder construct: 'Du\u0308sseldorf Ko\u0308nigsallee') > > and construct some test cases with it which illustrate some basic > Unicode issues. > > 8b) identity testing for major languages (e.g. French, German, > Spanish) and scripts of similar complexity. I > > 8c) to provide some more documentation of past and concurrent efforts. > > Note: This summary has only covered string manipulation, not rendering > on the screen which is a different issue. > > > On 12/16/15, Sven Van Caekenberghe wrote: >> Hi Hannes, >> >> My detailed comments/answers below, after quoting 2 of your emails: >> >>> On 10 Dec 2015, at 22:17, H. Hirzel wrote: >>> >>> Hello Sven >>> >>> On 12/9/15, Sven Van Caekenberghe wrote: >>> >>>> The simplest example in a common language is (the French letter ?) is >>>> >>>> LATIN SMALL LETTER E WITH ACUTE [U+00E9] >>>> >>>> which can also be written as >>>> >>>> LATIN SMALL LETTER E [U+0065] followed by COMBINING ACUTE ACCENT >>>> [U+0301] >>>> >>>> The former being a composed normal form, the latter a decomposed normal >>>> form. (And yes, it is even much more complicated than that, it goes on >>>> for >>>> 1000s of pages). >>>> >>>> In the above example, the concept of character/string is indeed fuzzy. >>>> >>>> HTH, >>>> >>>> Sven >>> >>> Thanks for this example. I have created a wiki page with it >>> >>> I wonder what the Pharo equivalent is of the following Squeak expression >>> >>> $? asString asDecomposedUnicode >>> >>> Regards >>> >>> Hannes >> >> You also wrote: >> >>> The text below shows how to deal with the Unicode e acute example >>> brought up by Sven in terms of comparing strings. Currently Pharo and >>> Cuis do not do Normalization of strings. Limited support is in Squeak. >>> It will be shown how NFD normalization may be implemented. >>> >>> >>> Swift programming language >>> ----------------------------------------- >>> >>> How does the Swift programming language [1] deal with Unicode strings? >>> >>> // "Voulez-vous un caf??" using LATIN SMALL LETTER E WITH ACUTE >>> let eAcuteQuestion = "Voulez-vous un caf\u{E9}?" >>> >>> // "Voulez-vous un café?" using LATIN SMALL LETTER E and >>> COMBINING ACUTE ACCENT >>> let combinedEAcuteQuestion = "Voulez-vous un caf\u{65}\u{301}?" >>> >>> if eAcuteQuestion == combinedEAcuteQuestion { >>> print("These two strings are considered equal") >>> } >>> // prints "These two strings are considered equal" >>> >>> The equality operator uses the NFD (Normalization Form Decomposed)[2] >>> form for the comparison appyling a method >>> #decomposedStringWithCanonicalMapping[3] >>> >>> >>> Squeak / Pharo >>> ----------------------- >>> >>> Comparison without NFD [3] >>> >>> >>> "Voulez-vous un caf??" >>> eAcuteQuestion := 'Voulez-vous un caf', 16rE9 asCharacter asString, '?'. >>> combinedEAcuteQuestion := 'Voulez-vous un cafe', 16r301 asCharacter >>> asString, '?'. >>> >>> >>> eAcuteQuestion = combinedEAcuteQuestion >>> false >>> >>> eAcuteQuestion == combinedEAcuteQuestion >>> false >>> >>> The result is false. A Unicode conformant application however should >>> return *true*. >>> >>> Reason for this is that Squeak / Pharo strings are not put into NFD >>> before testing for equality = >>> >>> >>> Squeak Unicode strings may be tested for Unicode conformant equality >>> by converting them to NFD before testing. >>> >>> >>> >>> Squeak using NFD >>> >>> asDecomposedUnicode[4] transforms a string into NFD for cases where a >>> Unicode code point if decomposed, is decomposed only to two code >>> points [5]. This is so because when initializing [6] the Unicode >>> Character Database in Squeak this is a limitation imposed by the code >>> which reads UnicodeData.txt [7][8]. This is not a necessary >>> limitation. The code may be rewritten at the price of a more complex >>> implementation of #asDecomposedUnicode. >>> >>> "Voulez-vous un caf??" >>> eAcuteQuestion := 'Voulez-vous un caf', 16rE9 asCharacter asString, '?'. >>> combinedEAcuteQuestion := 'Voulez-vous un cafe', 16r301 asCharacter >>> asString, '?'. >>> >>> >>> eAcuteQuestion asDecomposedUnicode = >>> combinedEAcuteQuestion asDecomposedUnicode >>> >>> true >>> >>> >>> >>> Conclusion >>> ------------------ >>> >>> Implementing a method like #decomposedStringWithCanonicalMapping >>> (swift) which puts a string into NFD (Normalization Form D) is an >>> important building block towards better Unicode compliance. A Squeak >>> proposal is given by [4]. It needs to be reviewed.extended. >>> >>> It should probably be extended for cases where there are more than >>> two code points in the decomposed form (3 or more?) >>> >>> The implementing of NFD comparison gives us an equality test for a >>> comparatively small effort for simple cases covering a large number of >>> use cases (Languages using the Latin script). >>> >>> The algorithm is table driven by the UCD [8]. From this follows an >>> simple but important fact for conformant implementations need runtime >>> access to information from the Unicode Character Database [UCD][9]. >>> >>> >>> [1] >>> https://developer.apple.com/library/ios/documentation/Swift/Conceptual/Swift_Programming_Language/StringsAndCharacters.html#//apple_ref/doc/uid/TP40014097-CH7-ID285 >>> [2] http://www.unicode.org/glossary/#normalization_form_d >>> [3] >>> https://developer.apple.com/library/ios/documentation/Cocoa/Reference/Foundation/Classes/NSString_Class/index.html#//apple_ref/occ/instm/NSString/decomposedStringWithCanonicalMapping >>> [4] String asDecomposedUnicode http://wiki.squeak.org/squeak/6250 >>> [5] http://www.unicode.org/glossary/#code_point >>> [6] Unicode initialize http://wiki.squeak.org/squeak/6248 >>> [7] http://www.unicode.org/Public/UNIDATA/UnicodeData.txt >>> [8] Unicode Character Database documentation >>> http://unicode.org/reports/tr44/ >>> [9] http://www.unicode.org/reports/tr23/ >> >> >> Today, we have a Unicode and CombinedCharacter class in Pharo, and there is >> different but similar Unicode code in Squeak. These are too simple (even >> though they might work, partially). >> >> The scope of the original threads is way too wide: a new string type, >> normalisation, collation, being cross dialect, mixing all kinds of character >> and encoding definitions. All interesting, but not much will come out of it. >> But the point that we cannot leave proper text string handling to an outside >> library is indeed key. >> >> That is why a couple of people in the Pharo community (myself included) >> started an experimental, proof of concept, prototype project, that aims to >> improve Unicode support. We will announce it to a wider public when we feel >> we have something to show for. The goal is in the first place to understand >> and implement the fundamental algorithms, starting with the 4 forms of >> Normalisation. But we're working on collation/sorting too. >> >> This work is of course being done for/in Pharo, using some of the facilities >> only available there. It probably won't be difficult to port, but we can't >> be bothered with probability right now. >> >> What we started with is loading UCD data and making it available as a nice >> objects (30.000 of them). >> >> So now you can do things like >> >> $? unicodeCharacterData. >> >> => "U+00E9 LATIN SMALL LETTER E WITH ACUTE (LATIN SMALL LETTER E ACUTE)" >> >> $? unicodeCharacterData uppercase asCharacter. >> >> => "$?" >> >> $? unicodeCharacterData decompositionMapping. >> >> => "#(101 769)" >> >> There is also a cool GT Inspector view: >> >> >> >> Next we started implementing a normaliser. It was rather easy to get support >> for simpler languages going. The next code snippets use explicit code >> arrays, because copying decomposed diacritics to my mail client does not >> work (they get automatically composed), in a Pharo Workspace this does work >> nicely with plain strings. The higher numbers are the diacritics. >> >> (normalizer decomposeString: 'les ?l?ves Fran?ais') collect: #codePoint as: >> Array. >> >> => "#(108 101 115 32 101 769 108 101 768 118 101 115 32 70 114 97 110 99 >> 807 97 105 115)" >> >> (normalizer decomposeString: 'D?sseldorf K?nigsallee') collect: #codePoint >> as: Array. >> >> => "#(68 117 776 115 115 101 108 100 111 114 102 32 75 111 776 110 105 103 >> 115 97 108 108 101 101)" >> >> normalizer composeString: (#(108 101 115 32 101 769 108 101 768 118 101 115 >> 32 70 114 97 110 99 807 97 105 115) collect: #asCharacter as: String). >> >> => "'les ?l?ves Fran?ais'" >> >> normalizer composeString: (#(68 117 776 115 115 101 108 100 111 114 102 32 >> 75 111 776 110 105 103 115 97 108 108 101 101) collect: #asCharacter as: >> String). >> >> => "'D?sseldorf K?nigsallee'" >> >> However, the real algorithm following the official specification (and other >> elements of Unicode that interact with it) is way more complicated (think >> about all those special languages/scripts out there). We're focused on >> understanding/implementing that now. >> >> Next, unit tests were added (of course). As well as a test that uses >> http://www.unicode.org/Public/UNIDATA/NormalizationTest.txt to run about >> 75.000 individual test cases to check conformance to the official Unicode >> Normalization specification. >> >> Right now (with super cool hangul / jamo code by Henrik), we hit the >> following stats: >> >> #testNFC 16998/18593 (91.42%) >> #testNFD 16797/18593 (90.34%) >> #testNFKC 13321/18593 (71.65%) >> #testNFKD 16564/18593 (89.09%) >> >> Way better than the naive implementations, but not yet there. >> >> We are also experimenting and thinking a lot about how to best implement all >> this, trying out different models/ideas/apis/representations. >> >> It will move slowly, but you will hear from us again in the coming >> weeks/months. >> >> Sven >> >> PS: Pharo developers with a good understanding of this subject area that >> want to help, let me know and we'll put you in the loop. Hacking and >> specification reading are required ;-) >> >> > > . > From eliot.miranda at gmail.com Sat Dec 19 03:11:45 2015 From: eliot.miranda at gmail.com (Eliot Miranda) Date: Sat Dec 19 03:11:49 2015 Subject: [squeak-dev] Re: CurrentReadOnlySourceFiles annoys me ... In-Reply-To: References: <1450256934663-4867253.post@n4.nabble.com> <1450269490206-4867327.post@n4.nabble.com> <35DA6F81-42C6-4761-B43E-CD5593EF7953@gmail.com> Message-ID: Hi Levente, On Fri, Dec 18, 2015 at 4:05 PM, Levente Uzonyi wrote: > Hi Eliot, > > I don't remember the exact case which made the Pharo guys do this change, > but if you implement a web service, which shows source code on a web page, > and reads the code from the image, then a non-UI process will read the > source code from SourceFiles, which can lead to the problems I've > described. IIRC Seaside has such example service. For this I'd be tempted to implement something like withClonedReadOnlySourceFilesDuring: that installs a new read-only copy for the extent of a block, sort of analogous to deferred UI actions. The requirement would be that any accessor of source outside of normal user (IDE) priority would use it. What do you think, worse than the disease, or acceptable? > > > Levente > > On Wed, 16 Dec 2015, Eliot Miranda wrote: > > Hi Levente, >> On Wed, Dec 16, 2015 at 2:43 PM, Levente Uzonyi >> wrote: >> My objection is that concurrent access of the read-only source >> files would cause race conditions. >> We might simplify the thing by using the regular source files (or >> some shared read-only copies) when requested from the UI process, and >> create a read-only copy for all other processes. That would cover most >> potential slow-downs in the Trunk. >> >> >> Apart form the debugger can you think of situations in which this will >> really happen? I can't. If you can convince me that concurrent access is a >> real danger then I'll think again, but in all my time doing Smalltalk with >> Smalltalk-80 v2, with VisualWorks and with >> Croquet, Squeak and Pharo the only time this has been an issue was in the >> debugger debugging source file access. Given that that's easy to solve I >> don't see the point of living with such slow source file access for any >> longer :-) >> >> >> >> >> Levente >> >> On Wed, 16 Dec 2015, Eliot Miranda wrote: >> >> Hi all, >> >> IMO we don't need CurrentReadOnkySourceFiles, instead we >> should have SourceFilesArray maintain one read-only copy per source file. >> The objection is that using a single file will make debugging source file >> access break, since the debuggers >> own access to source will perturb the source files whose >> access is being debugged, but the debugger can easily temporarily install a >> different read-init copy whenever it accesses source, insulating the >> debugged code from this effect. >> >> _,,,^..^,,,_ (phone) >> >> On Dec 16, 2015, at 4:38 AM, marcel.taeumel < >> Marcel.Taeumel@hpi.de> wrote: >> >> I want to catch all exceptions because it is kind of >> test code where only >> success matters. :-) It is an experiment where the >> participants should >> program stuff and receive a green tick whenever the >> task is accomplished. >> Raising a debugger when the participant puts a "self >> halt" or anything into >> the control flow of the test code would confuse the >> participant. >> >> Conceptionally, CurrentReadOnlySourceFiles is no kind >> of an exception. Thank >> you for the explanation. Can you sketch the steps to >> transform that into a >> PLV? Why would it take some effort? >> >> Bset, >> Marcel >> >> >> >> -- >> View this message in context: >> http://forum.world.st/CurrentReadOnlySourceFiles-annoys-me-tp4867253p4867327.html >> Sent from the Squeak - Dev mailing list archive at >> Nabble.com. >> >> >> >> >> >> >> >> -- >> _,,,^..^,,,_ >> best, Eliot >> >> > > > -- _,,,^..^,,,_ best, Eliot -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151218/532fb3cc/attachment-0001.htm From robert.w.withers at gmail.com Sat Dec 19 03:37:49 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Sat Dec 19 03:37:54 2015 Subject: Fwd: Re: [Pharo-users] [squeak-dev] Name change: Mushroom ( was Re: evolutions of squeakelib & crypto) In-Reply-To: <5674D0DC.2030404@gmail.com> References: <5674D0DC.2030404@gmail.com> Message-ID: <5674D10D.9040209@gmail.com> :) Thank you! -------- Forwarded Message -------- Subject: Re: [Pharo-users] [squeak-dev] Name change: Mushroom ( was Re: evolutions of squeakelib & crypto) Date: Fri, 18 Dec 2015 22:37:00 -0500 From: Robert Withers To: pharo-users@lists.pharo.org The Ode to CryptoCeps: Bouncer: "No, I'm sorry. This is a restricted area..." *checks around in case there's a problem* Me: " 'cep I got this..." *hands over teensie tiny tasty little mushoom* Note to self: 'did I eat one of those???' *can't seem to recall the particular moment. Oh@* Community: *looking sideways at each other* "What the hell is a mushoom?" Thank You! so much. On 12/18/2015 10:27 PM, Robert Withers wrote: > Ah ha!!! Thank you...how about CryptOCeps? ;) > > On 12/18/2015 10:22 PM, Offray Vladimir Luna C?rdenas wrote: >> Hi, >> >> On the names side, I would advice some kind of unique combination, >> for example "Jupyter notebook", works fine to find particularly the >> project this community is trying to build. In my own case, I spend a >> lot of time thinking in names and I discuss them with friends with a >> drink or a meal. Despite of not having a strong web presence in the >> sense of continuous updates in the web pages of my projects (my main >> web page is still under construction!) they position relatively well >> on google and DDG engines: >> >> [1] https://encrypted.google.com/search?hl=es&q=grafoscopio (5th link) >> [2] https://encrypted.google.com/search?hl=es&q=mutabit (1st link) >> [3] https://duckduckgo.com/?q=mutabit&t=ffsb (1st link) >> [4] https://duckduckgo.com/?q=grafoscopio&t=ffsb (2nd link) >> >> It didn't happen overnight but was a fluid organic process. So may be >> "Mushroom crypto" or "Risotto crypto" could be important descriptors >> in the main web presence sites of your projects (BTW, I can't find >> the repos and GitHub answers with 404). >> >> Cheers, >> >> Offray >> >> >> On 07/12/15 17:57, Robert Withers wrote: >>> Ben, Huw, Todd and Sven, thank you all for your feedback! I suppose >>> I could call the project "CryptOCaps" but for some reason I glommed >>> onto mushroom as the name. Not grandiose and it is somewhat >>> descriptive...a network of secure sessions, each one a mushroom. >>> Ceps are highly valued. We can tag it for the catalog. >>> >>> For sure, we have Seaside, Morphic, Nebraska, Fuel, Alien, Cog, >>> Monticello and that's just the squeak side of unusual naming of >>> projects. I hope that "mushroom" gains a wide reputation as a solid, >>> reliable, secure and performant session layer under the CryptOCaps >>> presentation...I am thinking of splitting the secure session layer >>> from the ocaps presentation layer, but this would require another >>> name choice, so I hesitate...perhaps "Risotto"? What are your thoughts? >>> >>> Best, >>> Robert >>> >>> On 12/07/2015 10:38 AM, Ben Coman wrote: >>>> I like it, but it seems you missed my point :) >>>> mushroom --> 117,000,000 is two orders of magnitude more hidden. >>>> Anyway, maybe I overplay its significance. >>>> cheers -ben >>>> >>>> On Mon, Dec 7, 2015 at 11:11 PM, Robert Withers >>>> wrote: >>>>> I renamed the project to Mushroom and I also dumped the encoding >>>>> work to >>>>> focus on shutdown, optimization and serialization. Here's the wiki: >>>>> https://github.com/SqueakCryptographySquad/Mushroom/wiki >>>>> >>>>> thanks,Robert >>>>> >>>>> >>>>> On 12/06/2015 01:42 AM, Ben Coman wrote: >>>>>> On Sun, Dec 6, 2015 at 10:42 AM, Robert Withers >>>>>> wrote: >>>>>>> On 12/05/2015 09:24 PM, Ben Coman wrote: >>>>>>>> On Fri, Dec 4, 2015 at 11:57 PM, Robert Withers >>>>>>>> wrote: >>>>>>>>> Now I think you are right on with your observation. >>>>>>>>> Additionally, the >>>>>>>>> number >>>>>>>>> of dialects could increase further with Fuel serialization, >>>>>>>>> just port >>>>>>>>> SecureSession and bits. >>>>>>>>> >>>>>>>>> Alright, I came up with a name and it may border on the >>>>>>>>> egregious ... >>>>>>>>> presenting ... >>>>>>>>> >>>>>>>>> "Maelstrom" >>>>>>>> Great sounding name. However some general advice for the >>>>>>>> community, >>>>>>>> since I see a lot of great sounding project names drowned out >>>>>>>> in the >>>>>>>> noise of our web-search-centric universe. A litmus test for >>>>>>>> project >>>>>>>> naming is using google search to find which return low search >>>>>>>> results. >>>>>>>> Today, its more important to be unique than any other attribute >>>>>>>> of a >>>>>>>> name. So in general, *dictionary* english words are not the best. >>>>>>>> One technique is to intentionally mispell the word you like. >>>>>>>> Here are >>>>>>>> some comparative examples (note, the surrounding quotes are >>>>>>>> required >>>>>>>> to avoid google trying to be helpful and correct the spelling)... >>>>>>>> >>>>>>>> "maelstrom" --> 7,480,000 >>>>>>>> "maelstroom" --> 6,200 >>>>>>>> "maelstrum" --> 2,280 >>>>>>>> "maelstruum" --> 7 >>>>>>>> >>>>>>>> Lots of interesting other techniques can be found by searching on: >>>>>>>> techniques to generate brand names or domain names. >>>>>>>> >>>>>>>> cheers -ben >>>>>>> >>>>>>> I would be happy to change the names to something more unique, >>>>>>> though it >>>>>>> may >>>>>>> take a few. Are you suggesting "maelstruum"? >>>>>>> >>>>>>> cheers, >>>>>>> Robert >>>>>>> >>>>>>> >>>>>> *Suggesting* yes, but the choice is yours ;) You need to own it. >>>>>> >>>>>> I think maelstruum is certainly memorable with the double "u", but >>>>>> maybe jarring next the the "m". I'm inclined to maelstroom, since I >>>>>> associate it with "zoom". I wouldn't necessarily go for the >>>>>> absolute >>>>>> lowest results. I have an entirely unsubstantiated belief that >>>>>> anything less than 10,000 gives a reasonable chance to compete >>>>>> once a >>>>>> user's browsing history is taken into account. Finally you need to >>>>>> check existing results don't return something abhorrent (I didn't do >>>>>> this). >>>>>> >>>>>> I'd encourage to play around testing on google search. Its quick and >>>>>> easy to generate and test alternatives. I've added a few more below. >>>>>> "maelstra" --> 3,560 >>>>>> "maelstram" --> 504 >>>>>> "maelstrim" --> 1200 >>>>>> "maelstroon" --> 58 >>>>>> "maelstroomi" --> 4 >>>>>> >>>>>> btw, I wouldn't swap the order of the "ae" since that would be >>>>>> susceptible to real typing errors. >>>>>> >>>>>> cheers -ben >>>>>> >>>>> >>> >>> >>> >> >> > -- . .. .. ^,^ robert -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151218/1b75028d/attachment.htm From robert.w.withers at gmail.com Sat Dec 19 06:09:45 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Sat Dec 19 06:09:51 2015 Subject: [squeak-dev] encoding challenge Message-ID: <5674F4A9.90101@gmail.com> A core part of communications, whether online or to persistence, is encoding, or marshalling. Let's do an example! Thank you for your thoughtful, earnest participation, as God wills it. Here are 2 methods that encode and decode a DateAndTime object, but rather poorly. I've been going without much sleep for a bit and I can't figure it out. Here is the code in hopes that some will find this an interesting exercise in team and community building. For that intention, as a beginning. What's the best solution? Let's show them how to code when given the space in the environment. Tnks! :) In class ASN1UTCTimeType.. encodeValue: anObject withDERStream: derStream | yy mo dd hh mm ss utcDateTime | yy := anObject year asString copyFrom: 3 to: 4. mo := anObject month asString padded: #left to: 2 with: $0. dd := anObject dayOfMonth asString padded: #left to: 2 with: $0. hh := anObject hour asString padded: #left to: 2 with: $0. mm := anObject minute asString padded: #left to: 2 with: $0. ss := anObject seconds asString padded: #left to: 2 with: $0. utcDateTime := (yy, mo, dd, hh, mm, ss, 'Z') asByteArray. derStream nextPutAll: utcDateTime. decodeValueWithDERStream: derStream length: length | aUTCDateTime | aUTCDateTime := (derStream next: length) asByteArray asString. ^ (DateAndTime readFromString: ( ((aUTCDateTime copyFrom: 1 to: 2) asInteger > 50 ifTrue: ['19'] ifFalse: ['20']), (aUTCDateTime copyFrom: 1 to: 2), '-', (aUTCDateTime copyFrom: 3 to: 4), '-', (aUTCDateTime copyFrom: 5 to: 6), 'T', (aUTCDateTime copyFrom: 7 to: 8), ':', (aUTCDateTime copyFrom: 9 to: 10), ':', (aUTCDateTime copyFrom: 11 to: 12))) offset: 0 -- . .. .. ^,^ robert From robert.w.withers at gmail.com Sat Dec 19 08:22:51 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Sat Dec 19 08:22:57 2015 Subject: [squeak-dev] Re: encoding challenge In-Reply-To: <5674F4A9.90101@gmail.com> References: <5674F4A9.90101@gmail.com> Message-ID: <567513DB.7050106@gmail.com> It turns out my problem was elsewhere. These convert just fine. My issue is the combination of flipping the whole stack to frames and adding headers and header method calls down at the Transport layer. The session layer is actually 2 sub-layers, data on top of control. In this case, we need a data frame but no control frame, so the transport is frameless. It is blowing up on me because control messages have an artificial frame. I will look into Asn1Der encoding of the control messages, however. cheers,, robert On 12/19/2015 01:09 AM, Robert Withers wrote: > A core part of communications, whether online or to persistence, is > encoding, or marshalling. Let's do an example! Thank you for your > thoughtful, earnest participation, as God wills it. > > Here are 2 methods that encode and decode a DateAndTime object, but > rather poorly. I've been going without much sleep for a bit and I > can't figure it out. Here is the code in hopes that some will find > this an interesting exercise in team and community building. For that > intention, as a beginning. What's the best solution? > > Let's show them how to code when given the space in the environment. > Tnks! :) > > In class ASN1UTCTimeType.. > > encodeValue: anObject withDERStream: derStream > > | yy mo dd hh mm ss utcDateTime | > yy := anObject year asString copyFrom: 3 to: 4. > mo := anObject month asString padded: #left to: 2 with: $0. > dd := anObject dayOfMonth asString padded: #left to: 2 with: $0. > hh := anObject hour asString padded: #left to: 2 with: $0. > mm := anObject minute asString padded: #left to: 2 with: $0. > ss := anObject seconds asString padded: #left to: 2 with: $0. > utcDateTime := (yy, mo, dd, hh, mm, ss, 'Z') asByteArray. > derStream nextPutAll: utcDateTime. > > > > decodeValueWithDERStream: derStream length: length > > | aUTCDateTime | > aUTCDateTime := (derStream next: length) asByteArray asString. > ^ (DateAndTime readFromString: ( > ((aUTCDateTime copyFrom: 1 to: 2) asInteger > 50 ifTrue: > ['19'] ifFalse: ['20']), > (aUTCDateTime copyFrom: 1 to: 2), '-', > (aUTCDateTime copyFrom: 3 to: 4), '-', > (aUTCDateTime copyFrom: 5 to: 6), 'T', > (aUTCDateTime copyFrom: 7 to: 8), ':', > (aUTCDateTime copyFrom: 9 to: 10), ':', > (aUTCDateTime copyFrom: 11 to: 12))) > offset: 0 > > -- . .. .. ^,^ robert From marcus.denker at inria.fr Sat Dec 19 08:36:42 2015 From: marcus.denker at inria.fr (Marcus Denker) Date: Sat Dec 19 08:36:49 2015 Subject: [squeak-dev] Re: [Pharo-users] encoding challenge In-Reply-To: <567513DB.7050106@gmail.com> References: <5674F4A9.90101@gmail.com> <567513DB.7050106@gmail.com> Message-ID: Hi, If you are interested in ASN.1, there is an implementation of that as part of the SS7 project: CI Server is here: https://ci.hartl.name/job/ASN.1/ This implements a ASN.1 parser. This means you do not have to do the encoding yourself but you can use the ASN.1 as published. Very useful when implementing protocols that are specified using ASN.1, e.g. all the telephony stuff for mobile networks. If it makes sense to use in this context I am not sure. Marcus > On 19 Dec 2015, at 09:22, Robert Withers wrote: > > It turns out my problem was elsewhere. These convert just fine. > > My issue is the combination of flipping the whole stack to frames and adding headers and header method calls down at the Transport layer. The session layer is actually 2 sub-layers, data on top of control. In this case, we need a data frame but no control frame, so the transport is frameless. It is blowing up on me because control messages have an artificial frame. > > I will look into Asn1Der encoding of the control messages, however. > > cheers,, > robert > > On 12/19/2015 01:09 AM, Robert Withers wrote: >> A core part of communications, whether online or to persistence, is encoding, or marshalling. Let's do an example! Thank you for your thoughtful, earnest participation, as God wills it. >> >> Here are 2 methods that encode and decode a DateAndTime object, but rather poorly. I've been going without much sleep for a bit and I can't figure it out. Here is the code in hopes that some will find this an interesting exercise in team and community building. For that intention, as a beginning. What's the best solution? >> >> Let's show them how to code when given the space in the environment. Tnks! :) >> >> In class ASN1UTCTimeType.. >> >> encodeValue: anObject withDERStream: derStream >> >> | yy mo dd hh mm ss utcDateTime | >> yy := anObject year asString copyFrom: 3 to: 4. >> mo := anObject month asString padded: #left to: 2 with: $0. >> dd := anObject dayOfMonth asString padded: #left to: 2 with: $0. >> hh := anObject hour asString padded: #left to: 2 with: $0. >> mm := anObject minute asString padded: #left to: 2 with: $0. >> ss := anObject seconds asString padded: #left to: 2 with: $0. >> utcDateTime := (yy, mo, dd, hh, mm, ss, 'Z') asByteArray. >> derStream nextPutAll: utcDateTime. >> >> >> >> decodeValueWithDERStream: derStream length: length >> >> | aUTCDateTime | >> aUTCDateTime := (derStream next: length) asByteArray asString. >> ^ (DateAndTime readFromString: ( >> ((aUTCDateTime copyFrom: 1 to: 2) asInteger > 50 ifTrue: ['19'] ifFalse: ['20']), >> (aUTCDateTime copyFrom: 1 to: 2), '-', >> (aUTCDateTime copyFrom: 3 to: 4), '-', >> (aUTCDateTime copyFrom: 5 to: 6), 'T', >> (aUTCDateTime copyFrom: 7 to: 8), ':', >> (aUTCDateTime copyFrom: 9 to: 10), ':', >> (aUTCDateTime copyFrom: 11 to: 12))) >> offset: 0 >> >> > > -- > . .. .. ^,^ robert > From robert.w.withers at gmail.com Sat Dec 19 08:55:03 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Sat Dec 19 08:55:09 2015 Subject: [squeak-dev] Re: [Pharo-users] encoding challenge In-Reply-To: References: <5674F4A9.90101@gmail.com> <567513DB.7050106@gmail.com> Message-ID: <56751B67.4040305@gmail.com> Hi Marcus, That is what I am using, an ASN.1 parser I wrote years ago. It is decently robust, but I defer marshalling to the objects, which define their schemas as well. The schemas rely on getter/setters. I do not recall if I started it or picked up where others left off. For sure, Ron and others helped build it adn for that I am grateful. If you want to see the inside of structure and mapped type marshalling, please load version 61 of SecureSession and run the Connect test and look at the stack in the debugger, from the error. I have had to refamiliarize myself with it. This is not to say the other parser you mentioned isn't a candidate, yet I have to manage my focus. If you are familiar with that parser or ASN.1 in genral, I would really welcome some feedback on this implementation. best, robert On 12/19/2015 03:36 AM, Marcus Denker wrote: > Hi, > > If you are interested in ASN.1, there is an implementation of that as part of the SS7 project: > CI Server is here: > > https://ci.hartl.name/job/ASN.1/ > > This implements a ASN.1 parser. This means you do not have to do the encoding > yourself but you can use the ASN.1 as published. Very useful when implementing protocols that > are specified using ASN.1, e.g. all the telephony stuff for mobile networks. > > If it makes sense to use in this context I am not sure. > > Marcus > >> On 19 Dec 2015, at 09:22, Robert Withers wrote: >> >> It turns out my problem was elsewhere. These convert just fine. >> >> My issue is the combination of flipping the whole stack to frames and adding headers and header method calls down at the Transport layer. The session layer is actually 2 sub-layers, data on top of control. In this case, we need a data frame but no control frame, so the transport is frameless. It is blowing up on me because control messages have an artificial frame. >> >> I will look into Asn1Der encoding of the control messages, however. >> >> cheers,, >> robert >> >> On 12/19/2015 01:09 AM, Robert Withers wrote: >>> A core part of communications, whether online or to persistence, is encoding, or marshalling. Let's do an example! Thank you for your thoughtful, earnest participation, as God wills it. >>> >>> Here are 2 methods that encode and decode a DateAndTime object, but rather poorly. I've been going without much sleep for a bit and I can't figure it out. Here is the code in hopes that some will find this an interesting exercise in team and community building. For that intention, as a beginning. What's the best solution? >>> >>> Let's show them how to code when given the space in the environment. Tnks! :) >>> >>> In class ASN1UTCTimeType.. >>> >>> encodeValue: anObject withDERStream: derStream >>> >>> | yy mo dd hh mm ss utcDateTime | >>> yy := anObject year asString copyFrom: 3 to: 4. >>> mo := anObject month asString padded: #left to: 2 with: $0. >>> dd := anObject dayOfMonth asString padded: #left to: 2 with: $0. >>> hh := anObject hour asString padded: #left to: 2 with: $0. >>> mm := anObject minute asString padded: #left to: 2 with: $0. >>> ss := anObject seconds asString padded: #left to: 2 with: $0. >>> utcDateTime := (yy, mo, dd, hh, mm, ss, 'Z') asByteArray. >>> derStream nextPutAll: utcDateTime. >>> >>> >>> >>> decodeValueWithDERStream: derStream length: length >>> >>> | aUTCDateTime | >>> aUTCDateTime := (derStream next: length) asByteArray asString. >>> ^ (DateAndTime readFromString: ( >>> ((aUTCDateTime copyFrom: 1 to: 2) asInteger > 50 ifTrue: ['19'] ifFalse: ['20']), >>> (aUTCDateTime copyFrom: 1 to: 2), '-', >>> (aUTCDateTime copyFrom: 3 to: 4), '-', >>> (aUTCDateTime copyFrom: 5 to: 6), 'T', >>> (aUTCDateTime copyFrom: 7 to: 8), ':', >>> (aUTCDateTime copyFrom: 9 to: 10), ':', >>> (aUTCDateTime copyFrom: 11 to: 12))) >>> offset: 0 >>> >>> >> -- >> . .. .. ^,^ robert >> > -- . .. .. ^,^ robert From Marcel.Taeumel at hpi.de Sat Dec 19 09:46:28 2015 From: Marcel.Taeumel at hpi.de (marcel.taeumel) Date: Sat Dec 19 10:04:54 2015 Subject: [squeak-dev] Re: MenuMorph>popUpAdjacent... seems to mis-position sub-menus In-Reply-To: <9300D1F7-25AB-4A75-86E8-B59D90727116@rowledge.org> References: <9300D1F7-25AB-4A75-86E8-B59D90727116@rowledge.org> Message-ID: <1450518388539-4867918.post@n4.nabble.com> Hi Tim, you are right, the line should rather be: ... leftPoint := rightOrLeftPoint last + ((self layoutInset + self borderWidth - self width) @ 0). ... Feel free to commit it to trunk. Best, Marcel -- View this message in context: http://forum.world.st/MenuMorph-popUpAdjacent-seems-to-mis-position-sub-menus-tp4867868p4867918.html Sent from the Squeak - Dev mailing list archive at Nabble.com. From frank.shearar at gmail.com Sat Dec 19 14:48:13 2015 From: frank.shearar at gmail.com (Frank Shearar) Date: Sat Dec 19 14:48:15 2015 Subject: [squeak-dev] Re: CurrentReadOnlySourceFiles annoys me ... In-Reply-To: <1450449263326-4867752.post@n4.nabble.com> References: <1450256934663-4867253.post@n4.nabble.com> <1450269490206-4867327.post@n4.nabble.com> <1450449263326-4867752.post@n4.nabble.com> Message-ID: On 18 December 2015 at 14:34, marcel.taeumel wrote: > We might want to add something to support this: > > [ ... ] > on: Exception - CurrentReadOnlyFiles > do: [:ex | ... ]. In other words, #- returns an ExceptionSet of some kind. Maybe a special kind of ExceptionSet that can represent an infinitude of Exceptions. That ought to be fairly simple to implement. It just needs to have a #handle: that says something like handle: anException ^ (anException isKindOf: CurrentReadOnlyFiles) not or something. I did something similar for my experiments in delimited dynamic binding: https://github.com/frankshearar/Control/blob/master/packages/Control.package/DelimitedDynamicVariableSet.class/instance/handles..st frank > Best, > Marcel From g.cotelli at gmail.com Sat Dec 19 15:33:49 2015 From: g.cotelli at gmail.com (Gabriel Cotelli) Date: Sat Dec 19 15:33:52 2015 Subject: [squeak-dev] Re: CurrentReadOnlySourceFiles annoys me ... In-Reply-To: <1450449263326-4867752.post@n4.nabble.com> References: <1450256934663-4867253.post@n4.nabble.com> <1450269490206-4867327.post@n4.nabble.com> <1450449263326-4867752.post@n4.nabble.com> Message-ID: I've implemented this support for Pharo a few years ago. You can harvest it from there. Just a couple of methods and some test cases. On Dec 18, 2015 11:52, "marcel.taeumel" wrote: > We might want to add something to support this: > > [ ... ] > on: Exception - CurrentReadOnlyFiles > do: [:ex | ... ]. > > Best, > Marcel > > > > -- > View this message in context: > http://forum.world.st/CurrentReadOnlySourceFiles-annoys-me-tp4867253p4867752.html > Sent from the Squeak - Dev mailing list archive at Nabble.com. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151219/12116533/attachment.htm From leves at caesar.elte.hu Sat Dec 19 20:51:35 2015 From: leves at caesar.elte.hu (Levente Uzonyi) Date: Sat Dec 19 20:51:40 2015 Subject: [squeak-dev] Re: CurrentReadOnlySourceFiles annoys me ... In-Reply-To: References: <1450256934663-4867253.post@n4.nabble.com> <1450269490206-4867327.post@n4.nabble.com> <35DA6F81-42C6-4761-B43E-CD5593EF7953@gmail.com> Message-ID: Hi Eliot, I don't see how it would be any better than the current solution. Currently users get correct behavior no matter what they do, and get good performance if they add some extra code (CurrentReadOnlySourceFiles cacheDuring: [ ... ]). With your suggestion, users would have to add some extra code to get correct behavior. Levente On Fri, 18 Dec 2015, Eliot Miranda wrote: > Hi Levente, > On Fri, Dec 18, 2015 at 4:05 PM, Levente Uzonyi wrote: > Hi Eliot, > > I don't remember the exact case which made the Pharo guys do this change, but if you implement a web service, which shows source code on a web page, and reads the code from the image, then a non-UI process will read the source code from SourceFiles, which > can lead to the problems I've described. IIRC Seaside has such example service. > > > For this I'd be tempted to implement something like withClonedReadOnlySourceFilesDuring: that installs a new read-only copy for the extent of a block, sort of analogous to deferred UI actions.? The requirement would be that any accessor of source outside of normal user > (IDE) priority would use it.? What do you think, worse than the disease, or acceptable? > ? > > > Levente > > On Wed, 16 Dec 2015, Eliot Miranda wrote: > > Hi Levente, > On Wed, Dec 16, 2015 at 2:43 PM, Levente Uzonyi wrote: > ? ? ? My objection is that concurrent access of the read-only source files would cause race conditions. > ? ? ? We might simplify the thing by using the regular source files (or some shared read-only copies) when requested from the UI process, and create a read-only copy for all other processes. That would cover most potential slow-downs in the > Trunk. > > > Apart form the debugger can you think of situations in which this will really happen?? I can't. If you can convince me that concurrent access is a real danger then I'll think again, but in all my time doing Smalltalk with Smalltalk-80 v2, with > VisualWorks and with > Croquet, Squeak and Pharo the only time this has been an issue was in the debugger debugging source file access.? Given that that's easy to solve I don't see the point of living with such slow source file access for any longer :-) > > ? > > > ? ? ? Levente > > ? ? ? On Wed, 16 Dec 2015, Eliot Miranda wrote: > > ? ? ? ? ? ? Hi all, > > ? ? ? ? ? ? ? ?IMO we don't need CurrentReadOnkySourceFiles, instead we should have SourceFilesArray maintain one read-only copy per source file.? The objection is that using a single file will make debugging source file access break, since the > debuggers > ? ? ? ? ? ? own access to source will perturb the source files whose access is being debugged, but the debugger can easily temporarily install a different read-init copy whenever it accesses source, insulating the debugged code from this effect. > > ? ? ? ? ? ? _,,,^..^,,,_ (phone) > > ? ? ? ? ? ? ? ? ? On Dec 16, 2015, at 4:38 AM, marcel.taeumel wrote: > > ? ? ? ? ? ? ? ? ? I want to catch all exceptions because it is kind of test code where only > ? ? ? ? ? ? ? ? ? success matters. :-)? It is an experiment where the participants should > ? ? ? ? ? ? ? ? ? program stuff and receive a green tick whenever the task is accomplished. > ? ? ? ? ? ? ? ? ? Raising a debugger when the participant puts a "self halt" or anything into > ? ? ? ? ? ? ? ? ? the control flow of the test code would confuse the participant. > > ? ? ? ? ? ? ? ? ? Conceptionally, CurrentReadOnlySourceFiles is no kind of an exception. Thank > ? ? ? ? ? ? ? ? ? you for the explanation. Can you sketch the steps to transform that into a > ? ? ? ? ? ? ? ? ? PLV? Why would it take some effort? > > ? ? ? ? ? ? ? ? ? Bset, > ? ? ? ? ? ? ? ? ? Marcel > > > > ? ? ? ? ? ? ? ? ? -- > ? ? ? ? ? ? ? ? ? View this message in context: http://forum.world.st/CurrentReadOnlySourceFiles-annoys-me-tp4867253p4867327.html > ? ? ? ? ? ? ? ? ? Sent from the Squeak - Dev mailing list archive at Nabble.com. > > > > > > > > -- > _,,,^..^,,,_ > best,?Eliot > > > > > > > > -- > _,,,^..^,,,_ > best,?Eliot > > From eliot.miranda at gmail.com Sat Dec 19 21:20:35 2015 From: eliot.miranda at gmail.com (Eliot Miranda) Date: Sat Dec 19 21:20:37 2015 Subject: [squeak-dev] Re: CurrentReadOnlySourceFiles annoys me ... In-Reply-To: References: <1450256934663-4867253.post@n4.nabble.com> <1450269490206-4867327.post@n4.nabble.com> <35DA6F81-42C6-4761-B43E-CD5593EF7953@gmail.com> Message-ID: On Sat, Dec 19, 2015 at 12:51 PM, Levente Uzonyi wrote: > Hi Eliot, > > I don't see how it would be any better than the current solution. > Here's why. Right now I have to type CurrentReadOnlySourceFiles cacheDuring: [self systemNavigation browseAllSelect: [:m| m getSourceFromFile includesSubString: 'foo']] instead of self systemNavigation browseAllSelect: [:m| m getSourceFromFile includesSubString: 'foo'] which, frankly, is absurd. We're penalising the common case. If we provide a single read-only source file by default and have the uncommon case work around it, the common case, my daily life, is much easier. Currently users get correct behavior no matter what they do, and get good > performance if they add some extra code (CurrentReadOnlySourceFiles > cacheDuring: [ ... ]). > With your suggestion, users would have to add some extra code to get > correct behavior. So what if, instead, the source files become process-aware and answer the single file if the accessor is in "the current IDE process", such that the notion of what's the owning process is weak, and by default the source files are unowned? This approach has a chance of solving the debugger issue too. > > > Levente > > > On Fri, 18 Dec 2015, Eliot Miranda wrote: > > Hi Levente, >> On Fri, Dec 18, 2015 at 4:05 PM, Levente Uzonyi >> wrote: >> Hi Eliot, >> >> I don't remember the exact case which made the Pharo guys do this >> change, but if you implement a web service, which shows source code on a >> web page, and reads the code from the image, then a non-UI process will >> read the source code from SourceFiles, which >> can lead to the problems I've described. IIRC Seaside has such >> example service. >> >> >> For this I'd be tempted to implement something like >> withClonedReadOnlySourceFilesDuring: that installs a new read-only copy for >> the extent of a block, sort of analogous to deferred UI actions. The >> requirement would be that any accessor of source outside of normal user >> (IDE) priority would use it. What do you think, worse than the disease, >> or acceptable? >> >> >> >> Levente >> >> On Wed, 16 Dec 2015, Eliot Miranda wrote: >> >> Hi Levente, >> On Wed, Dec 16, 2015 at 2:43 PM, Levente Uzonyi < >> leves@caesar.elte.hu> wrote: >> My objection is that concurrent access of the read-only >> source files would cause race conditions. >> We might simplify the thing by using the regular source >> files (or some shared read-only copies) when requested from the UI process, >> and create a read-only copy for all other processes. That would cover most >> potential slow-downs in the >> Trunk. >> >> >> Apart form the debugger can you think of situations in which >> this will really happen? I can't. If you can convince me that concurrent >> access is a real danger then I'll think again, but in all my time doing >> Smalltalk with Smalltalk-80 v2, with >> VisualWorks and with >> Croquet, Squeak and Pharo the only time this has been an >> issue was in the debugger debugging source file access. Given that that's >> easy to solve I don't see the point of living with such slow source file >> access for any longer :-) >> >> >> >> >> Levente >> >> On Wed, 16 Dec 2015, Eliot Miranda wrote: >> >> Hi all, >> >> IMO we don't need CurrentReadOnkySourceFiles, >> instead we should have SourceFilesArray maintain one read-only copy per >> source file. The objection is that using a single file will make debugging >> source file access break, since the >> debuggers >> own access to source will perturb the source >> files whose access is being debugged, but the debugger can easily >> temporarily install a different read-init copy whenever it accesses source, >> insulating the debugged code from this effect. >> >> _,,,^..^,,,_ (phone) >> >> On Dec 16, 2015, at 4:38 AM, marcel.taeumel >> wrote: >> >> I want to catch all exceptions because it >> is kind of test code where only >> success matters. :-) It is an experiment >> where the participants should >> program stuff and receive a green tick >> whenever the task is accomplished. >> Raising a debugger when the participant >> puts a "self halt" or anything into >> the control flow of the test code would >> confuse the participant. >> >> Conceptionally, CurrentReadOnlySourceFiles >> is no kind of an exception. Thank >> you for the explanation. Can you sketch the >> steps to transform that into a >> PLV? Why would it take some effort? >> >> Bset, >> Marcel >> >> >> >> -- >> View this message in context: >> http://forum.world.st/CurrentReadOnlySourceFiles-annoys-me-tp4867253p4867327.html >> Sent from the Squeak - Dev mailing list >> archive at Nabble.com. >> >> >> >> >> >> >> >> -- >> _,,,^..^,,,_ >> best, Eliot >> >> >> >> >> >> >> >> -- >> _,,,^..^,,,_ >> best, Eliot >> >> > > > -- _,,,^..^,,,_ best, Eliot -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151219/ca7b1a46/attachment.htm From leves at caesar.elte.hu Sat Dec 19 21:47:51 2015 From: leves at caesar.elte.hu (Levente Uzonyi) Date: Sat Dec 19 21:47:56 2015 Subject: [squeak-dev] Re: CurrentReadOnlySourceFiles annoys me ... In-Reply-To: References: <1450256934663-4867253.post@n4.nabble.com> <1450269490206-4867327.post@n4.nabble.com> <35DA6F81-42C6-4761-B43E-CD5593EF7953@gmail.com> Message-ID: On Sat, 19 Dec 2015, Eliot Miranda wrote: > > > On Sat, Dec 19, 2015 at 12:51 PM, Levente Uzonyi wrote: > Hi Eliot, > > I don't see how it would be any better than the current solution. > > > Here's why.? Right now I have to type > > ?CurrentReadOnlySourceFiles cacheDuring: [self systemNavigation browseAllSelect: [:m| m getSourceFromFile includesSubString: 'foo']] > > instead of > > ? ? ? ? self systemNavigation browseAllSelect: [:m| m getSourceFromFile includesSubString: 'foo'] No, you don't have to. Caching is totally optional. But if you don't use it, a new FileStream will be created for every single method your code will read. [ CurrentReadOnlySourceFiles cacheDuring: [self systemNavigation browseAllSelect: [:m| m getSourceFromFile asString includesSubString: 'foo']] ] timeToRun. "==> 880" [ self systemNavigation browseAllSelect: [:m| m getSourceFromFile asString includesSubString: 'foo'] ] timeToRun. "==> 4380" > > which, frankly, is absurd.? We're penalising the common case.? If we provide a single read-only source file by default and have the uncommon case work around it, the common case, my daily life, is much easier. > > > Currently users get correct behavior no matter what they do, and get good performance if they add some extra code (CurrentReadOnlySourceFiles cacheDuring: [ ... ]). > With your suggestion, users would have to add some extra code to get correct behavior. > > > So what if, instead, the source files become process-aware and answer the single file if the accessor is in "the current IDE process", such that the notion of what's the owning process is weak, and by default the source files are unowned?? This approach has a chance of > solving the debugger issue too. That was my last-resort suggestion a few mails earlier, but I still find using a ProcessLocalVariable superior to this, since that way all processes can have their own read-only copies. I'm not sure how well the debugger can work with PLVs though. Levente > ? > > > Levente > > On Fri, 18 Dec 2015, Eliot Miranda wrote: > > Hi Levente, > On Fri, Dec 18, 2015 at 4:05 PM, Levente Uzonyi wrote: > ? ? ? Hi Eliot, > > ? ? ? I don't remember the exact case which made the Pharo guys do this change, but if you implement a web service, which shows source code on a web page, and reads the code from the image, then a non-UI process will read the source code from > SourceFiles, which > ? ? ? can lead to the problems I've described. IIRC Seaside has such example service. > > > For this I'd be tempted to implement something like withClonedReadOnlySourceFilesDuring: that installs a new read-only copy for the extent of a block, sort of analogous to deferred UI actions.? The requirement would be that any accessor of > source outside of normal user > (IDE) priority would use it.? What do you think, worse than the disease, or acceptable? > ? > > > ? ? ? Levente > > ? ? ? On Wed, 16 Dec 2015, Eliot Miranda wrote: > > ? ? ? ? ? ? Hi Levente, > ? ? ? ? ? ? On Wed, Dec 16, 2015 at 2:43 PM, Levente Uzonyi wrote: > ? ? ? ? ? ? ? ? ? My objection is that concurrent access of the read-only source files would cause race conditions. > ? ? ? ? ? ? ? ? ? We might simplify the thing by using the regular source files (or some shared read-only copies) when requested from the UI process, and create a read-only copy for all other processes. That would cover most potential slow-downs > in the > ? ? ? ? ? ? Trunk. > > > ? ? ? ? ? ? Apart form the debugger can you think of situations in which this will really happen?? I can't. If you can convince me that concurrent access is a real danger then I'll think again, but in all my time doing Smalltalk with > Smalltalk-80 v2, with > ? ? ? ? ? ? VisualWorks and with > ? ? ? ? ? ? Croquet, Squeak and Pharo the only time this has been an issue was in the debugger debugging source file access.? Given that that's easy to solve I don't see the point of living with such slow source file access for any longer :-) > > ? ? ? ? ? ? ? > > > ? ? ? ? ? ? ? ? ? Levente > > ? ? ? ? ? ? ? ? ? On Wed, 16 Dec 2015, Eliot Miranda wrote: > > ? ? ? ? ? ? ? ? ? ? ? ? Hi all, > > ? ? ? ? ? ? ? ? ? ? ? ? ? ?IMO we don't need CurrentReadOnkySourceFiles, instead we should have SourceFilesArray maintain one read-only copy per source file.? The objection is that using a single file will make debugging source file access > break, since the > ? ? ? ? ? ? debuggers > ? ? ? ? ? ? ? ? ? ? ? ? own access to source will perturb the source files whose access is being debugged, but the debugger can easily temporarily install a different read-init copy whenever it accesses source, insulating the debugged code from > this effect. > > ? ? ? ? ? ? ? ? ? ? ? ? _,,,^..^,,,_ (phone) > > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? On Dec 16, 2015, at 4:38 AM, marcel.taeumel wrote: > > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? I want to catch all exceptions because it is kind of test code where only > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? success matters. :-)? It is an experiment where the participants should > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? program stuff and receive a green tick whenever the task is accomplished. > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? Raising a debugger when the participant puts a "self halt" or anything into > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? the control flow of the test code would confuse the participant. > > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? Conceptionally, CurrentReadOnlySourceFiles is no kind of an exception. Thank > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? you for the explanation. Can you sketch the steps to transform that into a > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? PLV? Why would it take some effort? > > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? Bset, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? Marcel > > > > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? -- > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? View this message in context: http://forum.world.st/CurrentReadOnlySourceFiles-annoys-me-tp4867253p4867327.html > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? Sent from the Squeak - Dev mailing list archive at Nabble.com. > > > > > > > > ? ? ? ? ? ? -- > ? ? ? ? ? ? _,,,^..^,,,_ > ? ? ? ? ? ? best,?Eliot > > > > > > > > -- > _,,,^..^,,,_ > best,?Eliot > > > > > > > > -- > _,,,^..^,,,_ > best,?Eliot > > From robert.w.withers at gmail.com Sun Dec 20 12:01:33 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Sun Dec 20 12:01:41 2015 Subject: [squeak-dev] Re: [Vm-dev] Re: SecureSession frame types In-Reply-To: References: <56768277.3020207@gmail.com> <56768388.8000902@gmail.com> Message-ID: <5676989D.8020201@gmail.com> Good morning, just a few comments and questions.. 1) current 4bits for msgVersion and 4bits for hdrType. With 14 messages and some not implemented (suspend/resume/...) we have little headroom in 4bits of hdrType. Shall I make the msgVersion 3bits and the hdrType 5bits? 2) the smallest interleaved FEC encoded msg is 30 bytes. With 8 bytes of data: a msgSpec, using RS(15,9) with 4bit symbols, and 4 symbol block interleaving, the smallest on the wire is 30 bytes. Without FEC enabled is it 8 bytes. 3) when run to the end of hdrType room, we can add a jump tag and have a second hdrType field as the 9th byte. Again, the valid document link is: https://www.dropbox.com/s/zj2i1votg432b2p/FrameTypes.pptx?dl=0 Thank you, robert On 12/20/2015 06:20 AM, Cl?ment Bera wrote: > > > > Hello, > > I tried to get your document but I got: > > Dropbox - error > > Is it only me ? > > 2015-12-20 11:31 GMT+01:00 Robert Withers >: > > > Excuse me for failing to complete this email./ I wanted to ask for > feedback on these structures, especially the header specification > word structure. So these fields make sense? > > thank you, > robert > > > On 12/20/2015 05:27 AM, Robert Withers wrote: > > I wrote up the various SecureSession frame types in > presentation document, which you can get here: > https://www.dropbox.com/s/3xfa5oqs8ihj88v/FrameTypes.pptx?dl=0 > > These are not yet implemented, but it is progressing. Ne quid > nemis. > > thanks, > > > -- > . .. .. ^,^ robert > > -- . .. .. ^,^ robert -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151220/7a813f98/attachment-0001.htm From robert.w.withers at gmail.com Sun Dec 20 14:20:28 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Sun Dec 20 14:20:33 2015 Subject: [squeak-dev] Re: [Vm-dev] Re: SecureSession frame types In-Reply-To: <5676989D.8020201@gmail.com> References: <56768277.3020207@gmail.com> <56768388.8000902@gmail.com> <5676989D.8020201@gmail.com> Message-ID: <5676B92C.4070101@gmail.com> My apologies for #2 below; what I said is wrong. The smallest message will be 8 bytes, as the messageSpecification is not FEC encoded. I will repair the doc as there are some issues with the spec and layout...done. Once again the link is: https://www.dropbox.com/s/zj2i1votg432b2p/FrameTypes.pptx?dl=0 * MessageSpecification is 8 bytes binary encoded. * Header is always asn.1 encoded * All messages may be FEC encoded with the FEC header asn.1 encoded. * Data Message payload is encrypted. Change query: Regarding the Message Specification, with a hdrSize...I don't think we need it. The message size includes everything + the 8 bytes spec. The hdrType will specify the kind of message and header. The specific header knows it's own size and layout. Therefor, we have 16bits free if we keep an 8 byte msgSpec. Let's rework it to a 7 byte msgSpec as below. Perhaps it is useful to be 64bit aligned, so leave it at 8. Thoughts? Does this work? *msgSpec: 7 bytes binary encoded* * HeaderSpecification: 3 bytes o multicastSymbol: 6 bits o sanguinity: 7 bits o msgVersion: 5 bits o hdrType: 6 bits * msgSize: 2nd word, 4 bytes o 4 bytes (total size with msgSpec, header, payload sizes) regards, robert On 12/20/2015 07:01 AM, Robert Withers wrote: > Good morning, just a few comments and questions.. > > 1) current 4bits for msgVersion and 4bits for hdrType. With 14 > messages and some not implemented (suspend/resume/...) we have > little headroom in 4bits of hdrType. Shall I make the msgVersion > 3bits and the hdrType 5bits? > > 2) the smallest interleaved FEC encoded msg is 30 bytes. With 8 > bytes of data: a msgSpec, using RS(15,9) with 4bit symbols, and 4 > symbol block interleaving, the smallest on the wire is 30 bytes. > Without FEC enabled is it 8 bytes. > > 3) when run to the end of hdrType room, we can add a jump tag and > have a second hdrType field as the 9th byte. > > > Again, the valid document link is: > https://www.dropbox.com/s/zj2i1votg432b2p/FrameTypes.pptx?dl=0 > > Thank you, > robert > > > > On 12/20/2015 06:20 AM, Cl?ment Bera wrote: >> >> >> >> Hello, >> >> I tried to get your document but I got: >> >> Dropbox - error >> >> Is it only me ? >> >> 2015-12-20 11:31 GMT+01:00 Robert Withers > >: >> >> >> Excuse me for failing to complete this email./ I wanted to ask >> for feedback on these structures, especially the header >> specification word structure. So these fields make sense? >> >> thank you, >> robert >> >> >> On 12/20/2015 05:27 AM, Robert Withers wrote: >> >> I wrote up the various SecureSession frame types in >> presentation document, which you can get here: >> https://www.dropbox.com/s/3xfa5oqs8ihj88v/FrameTypes.pptx?dl=0 >> >> These are not yet implemented, but it is progressing. Ne quid >> nemis. >> >> thanks, >> >> >> -- >> . .. .. ^,^ robert >> >> > > -- > . .. .. ^,^ robert -- . .. .. ^,^ robert -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151220/92a89dad/attachment.htm From robert.w.withers at gmail.com Sun Dec 20 14:33:55 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Sun Dec 20 14:34:02 2015 Subject: [squeak-dev] Re: [Vm-dev] Re: SecureSession frame types In-Reply-To: <5676B92C.4070101@gmail.com> References: <56768277.3020207@gmail.com> <56768388.8000902@gmail.com> <5676989D.8020201@gmail.com> <5676B92C.4070101@gmail.com> Message-ID: <5676BC53.4050800@gmail.com> Ok, let me make one last change to this 7 byte alternate proposal for the message specification. In the first 3 bytes with 4 fields, use supersymmetry to specify 6bits per field. SO the alternate would look like below... The whole point of the multicast symbol is to provide for low-level routers to not even have to decode the asn.1 header and distinguish all our message and header types. Just grab the first byte and mask the upper 6bits and shift, there is a destination specifier, at least in the large, as an alternate tunneling capability and would support reuse of 4byte IP addresses for NAT. If we loose that and knock sanguinity back to 2bits, then we are down to a 6 byte spec + payload. I'm mulling iot over, obviously. *msgSpec: 7 bytes binary encoded* * HeaderSpecification: 3 bytes o multicastSymbol: 6 bits o sanguinity: 6 bits o msgVersion: 6 bits o hdrType: 6 bits * msgSize: 2nd word, 4 bytes o 4 bytes (total size with msgSpec, header, payload sizes) -- . .. .. ^,^ robert -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151220/27dfc57d/attachment.htm From Marcel.Taeumel at hpi.de Sun Dec 20 14:26:18 2015 From: Marcel.Taeumel at hpi.de (marcel.taeumel) Date: Sun Dec 20 14:44:52 2015 Subject: [squeak-dev] Re: The Inbox: Morphic-tpr.1059.mcz In-Reply-To: References: Message-ID: <1450621578414-4868044.post@n4.nabble.com> +1 Marcel -- View this message in context: http://forum.world.st/The-Inbox-Morphic-tpr-1059-mcz-tp4867870p4868044.html Sent from the Squeak - Dev mailing list archive at Nabble.com. From nicolas.cellier.aka.nice at gmail.com Sun Dec 20 17:18:02 2015 From: nicolas.cellier.aka.nice at gmail.com (Nicolas Cellier) Date: Sun Dec 20 17:18:05 2015 Subject: [squeak-dev] review, validation and test development for GenericGF In-Reply-To: <566FF9CB.7000804@gmail.com> References: <566FF9CB.7000804@gmail.com> Message-ID: After a first glimpse, I noted these: 1) GenericGFPoly evaluateAt: does not set the result in the loop... 2) why using an OrderedCollection for the coefficients ? an Array is faster. same for exp and log tables of GenericGF 3) GenericGF>>multiply:scalar: you changed <= size into < size but shouldn't... 2015-12-15 12:30 GMT+01:00 Robert Withers : > Hi all, sticking with the distribution for discussing Crypto in the base, > I am currently trying to stabilize a port of Reed-Solomon such that FEC can > be added to SecureSession. I am doing this work in SecureSession. I just > posted an update of the GenericGF class, with the Java implementation > attached here. > > I need this critical code reviewed and tested. You need Cryptography and > SecureSession to be included in the catalog. If you can help me with the > former, I could help with the latter, if that suits you. Please let me know! > > Please load Cryptography(-Pharo) and SecureSession.23 for review. > > "ported & refactored GenericGF from the java code. This includes > reindexing from 0 to 1. Having no tests, hopefully someone can give a > review of this specific class. Where some tests to be developed at this > scope, that would be stellar." > > > I appreciate. > -- > . .. .. ^,^ best, robert > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151220/7ef08e1e/attachment.htm From commits at source.squeak.org Sun Dec 20 18:53:24 2015 From: commits at source.squeak.org (commits@source.squeak.org) Date: Sun Dec 20 18:53:25 2015 Subject: [squeak-dev] The Trunk: Morphic-tpr.1059.mcz Message-ID: tim Rowledge uploaded a new version of Morphic to project The Trunk: http://source.squeak.org/trunk/Morphic-tpr.1059.mcz ==================== Summary ==================== Name: Morphic-tpr.1059 Author: tpr Time: 18 December 2015, 4:26:36.049 pm UUID: ce2af9d7-dcb1-4225-8fea-1a0b3ae28ab4 Ancestors: Morphic-tpr.1058 When opening sub-menus with insufficient rightwards space on the Display we instead put them to the left of the parent. This change makes the leftward slide a bit smaller so that the mouse stays over menus was the user slides to the new menu; thus no strange closing of menus etc. =============== Diff against Morphic-tpr.1058 =============== Item was changed: ----- Method: MenuMorph>>popUpAdjacentTo:forHand:from: (in category 'control') ----- popUpAdjacentTo: rightOrLeftPoint forHand: hand from: sourceItem "Present this menu at the given point under control of the given hand." | tryToPlace selectedOffset rightPoint leftPoint | hand world startSteppingSubmorphsOf: self. popUpOwner := sourceItem. self fullBounds. self updateColor. "ensure layout is current" selectedOffset := (selectedItem ifNil: [self items first]) position - self position. tryToPlace := [:where :mustFit | | delta | self position: where - selectedOffset. delta := self boundsInWorld amountToTranslateWithin: sourceItem worldBounds. (delta x = 0 or: [mustFit]) ifTrue: [delta = (0 @ 0) ifFalse: [self position: self position + delta]. sourceItem world addMorphFront: self. ^ self]]. rightPoint := rightOrLeftPoint first + ((self layoutInset + self borderWidth) @ 0). + leftPoint := rightOrLeftPoint last + ((self layoutInset + self borderWidth - self width) @ 0). - leftPoint := rightOrLeftPoint last - ((self layoutInset + self borderWidth + self width) @ 0). tryToPlace value: rightPoint value: false; value: leftPoint value: false; value: rightPoint value: true.! From commits at source.squeak.org Sun Dec 20 18:59:09 2015 From: commits at source.squeak.org (commits@source.squeak.org) Date: Sun Dec 20 18:59:10 2015 Subject: [squeak-dev] The Trunk: Morphic-tpr.1060.mcz Message-ID: tim Rowledge uploaded a new version of Morphic to project The Trunk: http://source.squeak.org/trunk/Morphic-tpr.1060.mcz ==================== Summary ==================== Name: Morphic-tpr.1060 Author: tpr Time: 20 December 2015, 10:57:42.928 am UUID: c999504e-8f74-4941-a116-911eb62f0c74 Ancestors: Morphic-tpr.1059, Morphic-mt.1059 Attempt to correct stupid (probably( mistake in MC usage; I used the 'copy image versions here' menu item in the belief it would do something more sensible than simply copy a file. This clashed with marcel's change and so this is an attempt to re-merge and save a proper version. =============== Diff against Morphic-tpr.1059 =============== Item was changed: ----- Method: Form>>scaledIntoFormOfSize:smoothing: (in category '*Morphic') ----- scaledIntoFormOfSize: aNumberOrPoint smoothing: factor "Scale and center the receiver into a form of a given size" | extent scaledForm result | extent := aNumberOrPoint asPoint. extent = self extent ifTrue: [^ self copy]. scaledForm := self scaledToSize: extent smoothing: factor. + result := self species extent: extent depth: self depth. - result := self species extent: extent depth: 32. result getCanvas translucentImage: scaledForm at: extent - scaledForm extent // 2. ^ result ! From robert.w.withers at gmail.com Sun Dec 20 21:30:35 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Sun Dec 20 21:30:41 2015 Subject: [squeak-dev] Go Panthers! Message-ID: <56771DFB.2010004@gmail.com> That is all. ...she wrote!! 14 - 0. BWAHAHAHA! -- . .. .. ^,^ robert From commits at source.squeak.org Sun Dec 20 22:55:02 2015 From: commits at source.squeak.org (commits@source.squeak.org) Date: Sun Dec 20 22:55:04 2015 Subject: [squeak-dev] Daily Commit Log Message-ID: <20151220225502.6005.qmail@box4.squeakfoundation.org> Changes to Trunk (http://source.squeak.org/trunk.html) in the last 24 hours: http://lists.squeakfoundation.org/pipermail/packages/2015-December/009216.html Name: Morphic-tpr.1059 Ancestors: Morphic-tpr.1058 When opening sub-menus with insufficient rightwards space on the Display we instead put them to the left of the parent. This change makes the leftward slide a bit smaller so that the mouse stays over menus was the user slides to the new menu; thus no strange closing of menus etc. ============================================= http://lists.squeakfoundation.org/pipermail/packages/2015-December/009217.html Name: Morphic-tpr.1060 Ancestors: Morphic-tpr.1059, Morphic-mt.1059 Attempt to correct stupid (probably( mistake in MC usage; I used the 'copy image versions here' menu item in the belief it would do something more sensible than simply copy a file. This clashed with marcel's change and so this is an attempt to re-merge and save a proper version. ============================================= From robert.w.withers at gmail.com Mon Dec 21 00:19:35 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Mon Dec 21 00:19:40 2015 Subject: [squeak-dev] review, validation and test development for GenericGF In-Reply-To: References: <566FF9CB.7000804@gmail.com> Message-ID: <56774597.3020609@gmail.com> Hi Nicolas, I tried the fixes you suggested (1 and 3) and it still isn't working. I published tothe repository with these changes, though the pipeline is broken as I rewire my skull (intro frames). This will take several days to complete, I think, although progress is being made. Then I will be able to turn my full attention back to RS and Arrays for starters. Please pardon the delay, but I think frames will bring an advantage, perhaps not performance but design and flexibility. Many thanks for taking a close look at RS, robert On 12/20/2015 12:18 PM, Nicolas Cellier wrote: > After a first glimpse, I noted these: > > 1) GenericGFPoly evaluateAt: does not set the result in the loop... > 2) why using an OrderedCollection for the coefficients ? an Array is > faster. > same for exp and log tables of GenericGF > 3) GenericGF>>multiply:scalar: you changed <= size into < size but > shouldn't... > > 2015-12-15 12:30 GMT+01:00 Robert Withers >: > > Hi all, sticking with the distribution for discussing Crypto in > the base, I am currently trying to stabilize a port of > Reed-Solomon such that FEC can be added to SecureSession. I am > doing this work in SecureSession. I just posted an update of the > GenericGF class, with the Java implementation attached here. > > I need this critical code reviewed and tested. You need > Cryptography and SecureSession to be included in the catalog. If > you can help me with the former, I could help with the latter, if > that suits you. Please let me know! > > Please load Cryptography(-Pharo) and SecureSession.23 for review. > > "ported & refactored GenericGF from the java code. This > includes reindexing from 0 to 1. Having no tests, hopefully > someone can give a review of this specific class. Where some > tests to be developed at this scope, that would be stellar." > > > I appreciate. > -- > . .. .. ^,^ best, robert > > > > > > -- . .. .. ^,^ robert -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151220/fb4d23a4/attachment.htm From asqueaker at gmail.com Mon Dec 21 01:45:04 2015 From: asqueaker at gmail.com (Chris Muller) Date: Mon Dec 21 01:45:47 2015 Subject: [squeak-dev] The Trunk: Morphic-tpr.1060.mcz In-Reply-To: <5676fa81.97108c0a.9f5fc.5092SMTPIN_ADDED_MISSING@mx.google.com> References: <5676fa81.97108c0a.9f5fc.5092SMTPIN_ADDED_MISSING@mx.google.com> Message-ID: When mistakes like this happen, I think its better to revert to the ancestor and try again, rather than embalm these kinds of meta-notes forever into the history. 'copy image versions here' does exactly what its meant to and what it says it does. On Sun, Dec 20, 2015 at 12:58 PM, wrote: > tim Rowledge uploaded a new version of Morphic to project The Trunk: > http://source.squeak.org/trunk/Morphic-tpr.1060.mcz > > ==================== Summary ==================== > > Name: Morphic-tpr.1060 > Author: tpr > Time: 20 December 2015, 10:57:42.928 am > UUID: c999504e-8f74-4941-a116-911eb62f0c74 > Ancestors: Morphic-tpr.1059, Morphic-mt.1059 > > Attempt to correct stupid (probably( mistake in MC usage; I used the 'copy > image versions here' menu item in the belief it would do something more > sensible than simply copy a file. This clashed with marcel's change and so > this is an attempt to re-merge and save a proper version. > > =============== Diff against Morphic-tpr.1059 =============== > > Item was changed: > ----- Method: Form>>scaledIntoFormOfSize:smoothing: (in category > '*Morphic') ----- > scaledIntoFormOfSize: aNumberOrPoint smoothing: factor > "Scale and center the receiver into a form of a given size" > > | extent scaledForm result | > > extent := aNumberOrPoint asPoint. > extent = self extent ifTrue: [^ self copy]. > > scaledForm := self scaledToSize: extent smoothing: factor. > > + result := self species extent: extent depth: self depth. > - result := self species extent: extent depth: 32. > result getCanvas > translucentImage: scaledForm > at: extent - scaledForm extent // 2. > > ^ result > ! > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151220/a34357ba/attachment.htm From colin at wiresong.com Mon Dec 21 06:17:13 2015 From: colin at wiresong.com (Colin Putney) Date: Mon Dec 21 06:17:16 2015 Subject: [squeak-dev] OT: Convince me github is a wise choice In-Reply-To: References: Message-ID: On Thu, Dec 17, 2015 at 9:29 AM, Eliot Miranda wrote: > Ah, that's interesting. So my concern is whether github is a safe > long-term bet. Specifically what is there to prevent some third party from > buying github, or of github going public and the board taking the decision, > or github on its own, deciding to charge for hosting, keeping the data > hostage to extract payment? What safeguards are in place to prevent this? > I'm not interested in "this will never happen" arguments. I'm interested > in hard data please. > This sounds like a risk management problem. We want to minimize the risk that we lose access to the source code and it's history, right? Is there other data that you are concerned about? With regard to GitHub, I think these are the interesting questions: 1. What are the chances that GitHub will stop providing free hosting to open source projects? 2. What are the consequences if #1 occurs? 3. What can we do about it? First, let's look at #1. This sort of thing does happen. Holding data hostage is unusual, but free online services get shut down all the time. What might cause *Github* to do it? Could they be forced to cut expenses? Github has been around for almost 8 years, and have stuck with their model of "free public repositories, pay for privacy" throughout that time. It seems to be working for them. Three years ago one of their investors said they've been profitable over most of their life, and are growing revenue at 300% per year[1]. This summer, they raised $250 million more, with the company valued at $2 billion[2]. That indicates that they're still growing quickly, and think they'll be able to expand into new markets. So running out of money and dropping free hosting as a way to cut costs seems unlikely. How about a change in control? Maybe Oracle will buy them and squeeze as much profit out of them as possible before tossing the dry husk away. For that to happen, the offer would have to be spectacular. Github's investors need at least a 10x return, and probably more, to make money for their funds. If they were worth $2 billion this summer, the acquisition price would have to be something like $20-50 billion. That just doesn't allow the buyer much room to maneuver. There's no special technology behind Github that would make sense to to acquire at that price. Github's value is entirely in market position, customer relationships, goodwill etc. To make back the money, the buyer would need to keep running Github and keep earning revenue from it. Going public? Even less likely. Because of regulatory changes, tech companies have been waiting longer to go public and doing so at a much higher valuation. (Lots of different takes on this, but see eg. [3]) If Github went public, it would be because its valuation was so high that employees and investors wanted to (more easily) sell some shares and enjoy their wealth. That would be a huge endorsement of the business model and current management team. With few investors?only five so far[4]?the founders would undoubtedly retain control, similar to the IPOs of Google and Facebook. Messing with the business model would be unthinkable at that point. What if Github decided to change strategies without some sort of external impetus? That seems unlikely as well. The economics underlying the freemium strategy are getting more and more compelling over time. Disks are cheap, and the cost of storage keeps going down. I just ran across a new cloud storage service that charges half-a-cent per GB per month[5]. Computing power is also getting cheaper, and with cluster managers like Mesos and Kubernetes, we're using it more efficiently as well. The "burden" of providing free hosting is low and will be getting lower as time goes on. On the other hand, Github is *the* go-to place for hosting source code. There are millions of users that have both free public repositories and paid private ones. (Github reports 12 million users[6], and I bet a large fraction of them at least have access to both public and private repositories.) Taking away the free repositories would alienate a LOT of customers, and hurt revenue. So, without saying "this will never happen," I will say that Github shutting down free hosting would be unlikely. Alright, let's look at #2. If the unlikely did happen, what would be the consequences? As others have mentioned, the architecture of git makes it impossible to hold the source code and history hostage. Everyone who clones a git repository has a complete copy of the data. If they decided to lock everyone out of the repositories we'd just get another server and do this: cd coggit remote add origin git://git.squeak.org/cog.git git push origin master At the same time, we'd be in good company. Github currently has 30 million repositories[6]. Let's be really generous and say that half of those are private, and thus paid-for and exempt from hostage-taking. That means 15 million repositories are now subject to extortion from Github. Sure, most of those are personal forks with no significant changes. But even if there were only, say, 100,000 "real" repositories, that would be a *cataclysm* for the open source world. Alternate hosting would be popping up all over the place, and whatever inconvenience we might have about moving would be quickly solved by larger and richer open source projects. It wouldn't take much more than "here's our new git hosting" posted on the mailing list and squeak.org to make the change, because *everybody* would know about the problem. Finally, #3, what can we do about it? Well, in terms of influencing Github's business model, nothing. We have no leverage. So #1 is out of our control. But, there are a few things we can do to improve #2. First, we could mirror all commits to another repository. That could be a Github competitor, like BitBucket, or just a server that we host with Rackspace or whatever, or even "offline" storage like S3. I believe the Pharo folks are already mirroring the VM source, from the current hosting, so that helps reduce the risk as well. Second, we could move more of the VM source into Smalltalk. That might mean generating more of the source files with VM maker, running builds from within the image instead of using CMake etc. It probably wouldn't be worth it to make *all* the platform sources versioned in MC, but we could go further in that direction from where we are now. Finally, if it really did come down to Github holding the sources hostage and we had no other copies, we could just pay up. Currently, their cheapest plan is $7/month for 5 private repositories, which ought to cover our needs. Even with the meager donations that Squeak attracts today, surely we could raise $85 to get a year of paid hosting, and use that time to figure out what to do for the long term. Github might raise their prices (Why not? This scenario already has them being suicidally irrational.), but I can't see them exceeding our fundraising capabilities. What's the point of extortion if the victim can't pay? (As a side note, I would be shocked if hosting squeakvm.org currently costs less than $7/month. No idea who's paying for it, but how confident are we that they'll continue to do so?) In summary, Github is a very safe bet. Your nightmare scenario involves a series of very improbable events: Github would have to stop offering free hosting. They'd have to actively alienate their paying customers by holding their source code hostage. There would have to be sudden disk failures on dozens of laptops and servers where the repository is cloned. And to top it all off, the larger Squeak community, including Pharo, Cuis, Newspeak, Scratch and Croquet would have to be unable to come up with a few dozen dollars to pay for the hosting. This will never happen. Colin [1] http://peter.a16z.com/2012/07/09/software-eats-software-development/ [2] http://fortune.com/2015/07/29/github-raises-250-million-in-new-funding-now-valued-at-2-billion/ [3] http://www.forbes.com/sites/samanthasharf/2014/12/24/is-the-ipo-outmoded-why-venture-backed-companies-are-waiting-longer-to-go-public/ [4] https://www.crunchbase.com/organization/github/investors [5] https://www.backblaze.com/b2/cloud-storage.html [6] https://github.com/about/ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151220/78d77c0c/attachment.htm From marcus.denker at inria.fr Mon Dec 21 07:15:58 2015 From: marcus.denker at inria.fr (Marcus Denker) Date: Mon Dec 21 07:16:01 2015 Subject: [squeak-dev] Re: [Pharo-users] Go Panthers! In-Reply-To: <56771DFB.2010004@gmail.com> References: <56771DFB.2010004@gmail.com> Message-ID: Hello, To me your email is completely useless. I have not even the beginning of a clue what you want to say and in which context. We are all busy. This mailing lists get *a lot* of traffic. Could you evaluate in the future if your email contributes something? if not, it might be better to not send it. Marcus > On 20 Dec 2015, at 22:30, Robert Withers wrote: > > That is all. ...she wrote!! 14 - 0. BWAHAHAHA! > -- > . .. .. ^,^ robert > From lists at fniephaus.com Mon Dec 21 10:52:11 2015 From: lists at fniephaus.com (Fabio Niephaus) Date: Mon Dec 21 10:52:24 2015 Subject: [squeak-dev] OT: Convince me github is a wise choice In-Reply-To: References: Message-ID: On Mon, Dec 21, 2015 at 7:17 AM Colin Putney wrote: > On Thu, Dec 17, 2015 at 9:29 AM, Eliot Miranda > wrote: > > >> Ah, that's interesting. So my concern is whether github is a safe >> long-term bet. Specifically what is there to prevent some third party from >> buying github, or of github going public and the board taking the decision, >> or github on its own, deciding to charge for hosting, keeping the data >> hostage to extract payment? What safeguards are in place to prevent this? >> I'm not interested in "this will never happen" arguments. I'm interested >> in hard data please. >> > > This sounds like a risk management problem. We want to minimize the risk > that we lose access to the source code and it's history, right? Is there > other data that you are concerned about? > > With regard to GitHub, I think these are the interesting questions: > > > 1. What are the chances that GitHub will stop providing free hosting > to open source projects? > 2. What are the consequences if #1 occurs? > 3. What can we do about it? > > First, let's look at #1. This sort of thing does happen. Holding data > hostage is unusual, but free online services get shut down all the time. > What might cause *Github* to do it? > > Could they be forced to cut expenses? Github has been around for almost 8 > years, and have stuck with their model of "free public repositories, pay > for privacy" throughout that time. It seems to be working for them. Three > years ago one of their investors said they've been profitable over most of > their life, and are growing revenue at 300% per year[1]. This summer, they > raised $250 million more, with the company valued at $2 billion[2]. That > indicates that they're still growing quickly, and think they'll be able to > expand into new markets. So running out of money and dropping free hosting > as a way to cut costs seems unlikely. > > How about a change in control? Maybe Oracle will buy them and squeeze as > much profit out of them as possible before tossing the dry husk away. For > that to happen, the offer would have to be spectacular. Github's investors > need at least a 10x return, and probably more, to make money for their > funds. If they were worth $2 billion this summer, the acquisition price > would have to be something like $20-50 billion. That just doesn't allow the > buyer much room to maneuver. There's no special technology behind Github > that would make sense to to acquire at that price. Github's value is > entirely in market position, customer relationships, goodwill etc. To make > back the money, the buyer would need to keep running Github and keep > earning revenue from it. > > Going public? Even less likely. Because of regulatory changes, tech > companies have been waiting longer to go public and doing so at a much > higher valuation. (Lots of different takes on this, but see eg. [3]) If > Github went public, it would be because its valuation was so high that > employees and investors wanted to (more easily) sell some shares and enjoy > their wealth. That would be a huge endorsement of the business model and > current management team. With few investors?only five so far[4]?the > founders would undoubtedly retain control, similar to the IPOs of Google > and Facebook. Messing with the business model would be unthinkable at that > point. > > What if Github decided to change strategies without some sort of external > impetus? That seems unlikely as well. The economics underlying the freemium > strategy are getting more and more compelling over time. Disks are cheap, > and the cost of storage keeps going down. I just ran across a new cloud > storage service that charges half-a-cent per GB per month[5]. Computing > power is also getting cheaper, and with cluster managers like Mesos and > Kubernetes, we're using it more efficiently as well. The "burden" of > providing free hosting is low and will be getting lower as time goes on. > > On the other hand, Github is *the* go-to place for hosting source code. > There are millions of users that have both free public repositories and > paid private ones. (Github reports 12 million users[6], and I bet a large > fraction of them at least have access to both public and private > repositories.) Taking away the free repositories would alienate a LOT of > customers, and hurt revenue. > > So, without saying "this will never happen," I will say that Github > shutting down free hosting would be unlikely. > > Alright, let's look at #2. If the unlikely did happen, what would be the > consequences? > > As others have mentioned, the architecture of git makes it impossible to > hold the source code and history hostage. Everyone who clones a git > repository has a complete copy of the data. If they decided to lock > everyone out of the repositories we'd just get another server and do this: > > cd coggit remote add origin git://git.squeak.org/cog.git > git push origin master > > At the same time, we'd be in good company. Github currently has 30 million > repositories[6]. Let's be really generous and say that half of those are > private, and thus paid-for and exempt from hostage-taking. That means 15 > million repositories are now subject to extortion from Github. Sure, most > of those are personal forks with no significant changes. But even if there > were only, say, 100,000 "real" repositories, that would be a *cataclysm* > for the open source world. Alternate hosting would be popping up all over > the place, and whatever inconvenience we might have about moving would be > quickly solved by larger and richer open source projects. It wouldn't take > much more than "here's our new git hosting" posted on the mailing list and > squeak.org to make the change, because *everybody* would know about the > problem. > > Finally, #3, what can we do about it? > > Well, in terms of influencing Github's business model, nothing. We have no > leverage. So #1 is out of our control. > > But, there are a few things we can do to improve #2. First, we could > mirror all commits to another repository. That could be a Github > competitor, like BitBucket, or just a server that we host with Rackspace or > whatever, or even "offline" storage like S3. I believe the Pharo folks are > already mirroring the VM source, from the current hosting, so that helps > reduce the risk as well. > > Second, we could move more of the VM source into Smalltalk. That might > mean generating more of the source files with VM maker, running builds from > within the image instead of using CMake etc. It probably wouldn't be worth > it to make *all* the platform sources versioned in MC, but we could go > further in that direction from where we are now. > > Finally, if it really did come down to Github holding the sources hostage > and we had no other copies, we could just pay up. Currently, their cheapest > plan is $7/month for 5 private repositories, which ought to cover our > needs. Even with the meager donations that Squeak attracts today, surely we > could raise $85 to get a year of paid hosting, and use that time to figure > out what to do for the long term. Github might raise their prices (Why not? > This scenario already has them being suicidally irrational.), but I can't > see them exceeding our fundraising capabilities. What's the point of > extortion if the victim can't pay? > > (As a side note, I would be shocked if hosting squeakvm.org currently > costs less than $7/month. No idea who's paying for it, but how confident > are we that they'll continue to do so?) > > In summary, Github is a very safe bet. Your nightmare scenario involves a > series of very improbable events: Github would have to stop offering free > hosting. They'd have to actively alienate their paying customers by holding > their source code hostage. There would have to be sudden disk failures on > dozens of laptops and servers where the repository is cloned. And to top it > all off, the larger Squeak community, including Pharo, Cuis, Newspeak, > Scratch and Croquet would have to be unable to come up with a few dozen > dollars to pay for the hosting. > > This will never happen. > > Colin > > > [1] http://peter.a16z.com/2012/07/09/software-eats-software-development/ > [2] > http://fortune.com/2015/07/29/github-raises-250-million-in-new-funding-now-valued-at-2-billion/ > [3] > http://www.forbes.com/sites/samanthasharf/2014/12/24/is-the-ipo-outmoded-why-venture-backed-companies-are-waiting-longer-to-go-public/ > [4] https://www.crunchbase.com/organization/github/investors > [5] https://www.backblaze.com/b2/cloud-storage.html > [6] https://github.com/about/ > > There's not much I can add to Colin's great write-up except that others must have asked themselves the same questions as well. If moving to GitHub was a big risk, many companies wouldn't have done it already including the big ones (e.g. Microsoft , Google and now even Apple has even released Swift on GitHub). I am also +1 for GitHub. We have been successfully using it as a hosting platform for student projects [1]. IMHO, it is very convenient to not having to worry about solved problems including infrastructure. Also, mirroring a git repository can be done with a simple cronjob. However, I must admit, that there's still potential to improve client-side tooling (git + Filetree), but at least I don't have to worry about running a server and maintaining a SqueakSource/SqueakMap instance anymore. Lastly, we have been working on bringing Smalltalk support to Travis CI which will hopefully make it very easy to enable CI for any Smalltalk project on GitHub. An announcement will follow very soon. Happy holidays, Fabio [1] https://github.com/hpi-swa-teaching -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151221/8598d958/attachment-0001.htm From sumi at seagreen.ocn.ne.jp Mon Dec 21 12:04:44 2015 From: sumi at seagreen.ocn.ne.jp (Masato Sumi) Date: Mon Dec 21 12:05:27 2015 Subject: [squeak-dev] Please don't kill the in-place find feature of the former CTRL-J Message-ID: I'm very worry about that the new CTRL-J (aka "again") of tunk seems lacking very old but elegant feature of the former CTRL-J, "in-place find" function. When you just inserted text at a caret, the former version of CTRL-J (or select "again" from yellow-button menu) sets the text as a search key then delete it and then select next found text. I prefer and everyday use this feature because it make me find the text very seamlessly. I know my English is too bad to explain about this feature, so I made a screen-recording below. (Although it is very old Apple Smalltalk, you can do the same thing in current Squeak but not trunk) https://www.youtube.com/watch?v=-jKdn_9dKeQ If it was my misunderstanding and the new CTRL-J still keeps this feature, please let me how to use it. Thanks -- sumim From Marcel.Taeumel at hpi.de Mon Dec 21 12:08:03 2015 From: Marcel.Taeumel at hpi.de (marcel.taeumel) Date: Mon Dec 21 12:26:43 2015 Subject: [squeak-dev] Re: Please don't kill the in-place find feature of the former CTRL-J In-Reply-To: References: Message-ID: <1450699683717-4868121.post@n4.nabble.com> Now this is some magic feature! :) It feels like the concept of "print it" where you destroy your text contents with some debugging output. For in-place search/replace, this makes sense because you specify the first search/replace operation and then do again. Sorry, in-place find is not working this way right now. However, you can select any piece of text and "find again" if you like to. I write it down. There are still 2 or 3 things on my list regarding multiple-undo and in-place search/replace. Best, Marcel -- View this message in context: http://forum.world.st/Please-don-t-kill-the-in-place-find-feature-of-the-former-CTRL-J-tp4868119p4868121.html Sent from the Squeak - Dev mailing list archive at Nabble.com. From bert at freudenbergs.de Mon Dec 21 12:27:56 2015 From: bert at freudenbergs.de (Bert Freudenberg) Date: Mon Dec 21 12:27:59 2015 Subject: [squeak-dev] Please don't kill the in-place find feature of the former CTRL-J In-Reply-To: References: Message-ID: <7DEC0CDA-2F00-4DA2-81FA-6B9E203A7422@freudenbergs.de> > On 21.12.2015, at 13:04, Masato Sumi wrote: > > I'm very worry about that the new CTRL-J (aka "again") of tunk seems > lacking very old but elegant feature of the former CTRL-J, "in-place > find" function. > > When you just inserted text at a caret, the former version of CTRL-J > (or select "again" from yellow-button menu) sets the text as a search > key then delete it and then select next found text. > > I prefer and everyday use this feature because it make me find the > text very seamlessly. > > I know my English is too bad to explain about this feature, so I made > a screen-recording below. (Although it is very old Apple Smalltalk, > you can do the same thing in current Squeak but not trunk) > > https://www.youtube.com/watch?v=-jKdn_9dKeQ Yep. It?s Cmd-J for me on a Mac, and works nicely: http://try.squeak.org/#pixelated&fullscreen > If it was my misunderstanding and the new CTRL-J still keeps this > feature, please let me how to use it. It probably got broken accidentally ? the editor features are woefully under-documented. - Bert - -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4115 bytes Desc: not available Url : http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151221/7747cd68/smime.bin From Marcel.Taeumel at hpi.de Mon Dec 21 12:20:14 2015 From: Marcel.Taeumel at hpi.de (marcel.taeumel) Date: Mon Dec 21 12:38:55 2015 Subject: [squeak-dev] Re: Please don't kill the in-place find feature of the former CTRL-J In-Reply-To: <1450699683717-4868121.post@n4.nabble.com> References: <1450699683717-4868121.post@n4.nabble.com> Message-ID: <1450700414787-4868124.post@n4.nabble.com> Btw: We god rid of this magical "do again" thing. There is "find again" (CMD+G) and "find/replace again" (CMD+J). Maybe we can think of a way to also provide in-place find. Why don't you hit CMD+F and type your search string there? :-) *scnr* Best, Marcel -- View this message in context: http://forum.world.st/Please-don-t-kill-the-in-place-find-feature-of-the-former-CTRL-J-tp4868119p4868124.html Sent from the Squeak - Dev mailing list archive at Nabble.com. From Marcel.Taeumel at hpi.de Mon Dec 21 12:46:08 2015 From: Marcel.Taeumel at hpi.de (marcel.taeumel) Date: Mon Dec 21 13:04:47 2015 Subject: [squeak-dev] Re: Please don't kill the in-place find feature of the former CTRL-J In-Reply-To: <1450700414787-4868124.post@n4.nabble.com> References: <1450699683717-4868121.post@n4.nabble.com> <1450700414787-4868124.post@n4.nabble.com> Message-ID: <1450701968225-4868130.post@n4.nabble.com> I think I can add it like this: - You just moved your cursor to some place without selecting something. - You type something. - You hit CMD+G to "find again" - The previous input will be removed and taken as search text. - The next match will be selected. - (If you err here, you can redo with CMD+SHIFT+Z and insert the input again.) Still, it would lacke the same issues as current "in-place find/replace" has, that is, you have to type your string correctly in one run without using backspace or moving the cursor with the cursor keys. I am working on figuring out, which commands in the history belong to one subsequent edit. I plan to use position/intervals to be independent from typing speed. Best, Marcel -- View this message in context: http://forum.world.st/Please-don-t-kill-the-in-place-find-feature-of-the-former-CTRL-J-tp4868119p4868130.html Sent from the Squeak - Dev mailing list archive at Nabble.com. From Marcel.Taeumel at hpi.de Mon Dec 21 14:08:33 2015 From: Marcel.Taeumel at hpi.de (marcel.taeumel) Date: Mon Dec 21 14:27:15 2015 Subject: [squeak-dev] Re: Please don't kill the in-place find feature of the former CTRL-J In-Reply-To: <1450701968225-4868130.post@n4.nabble.com> References: <1450699683717-4868121.post@n4.nabble.com> <1450700414787-4868124.post@n4.nabble.com> <1450701968225-4868130.post@n4.nabble.com> Message-ID: <1450706913890-4868139.post@n4.nabble.com> Hi, here is a change set that should make in-place-find work: undo-inplace-find-fix.cs Please test it, then I can commit it into trunk. Here is how it works: - You just moved your cursor to some place without selecting something. - You type something. - You hit *CMD+G* to "find again" - The previous input will be removed and taken as search text. - The next match will be selected. - (If you err here, you can *undo* with CMD+Z and insert the input again.) If you hit CMD+J, nothing will happen because there was no replace operation detected. If you, however, hit CMD+G to find the next match and then continue wit a CMD+J, it will find/replace the next match (with nothing) because your insertion was removed automatically. Does this make sense? Best, Marcel -- View this message in context: http://forum.world.st/Please-don-t-kill-the-in-place-find-feature-of-the-former-CTRL-J-tp4868119p4868139.html Sent from the Squeak - Dev mailing list archive at Nabble.com. From sumi at seagreen.ocn.ne.jp Mon Dec 21 14:38:09 2015 From: sumi at seagreen.ocn.ne.jp (Masato Sumi) Date: Mon Dec 21 14:38:51 2015 Subject: [squeak-dev] Re: Please don't kill the in-place find feature of the former CTRL-J In-Reply-To: <1450706913890-4868139.post@n4.nabble.com> References: <1450699683717-4868121.post@n4.nabble.com> <1450700414787-4868124.post@n4.nabble.com> <1450701968225-4868130.post@n4.nabble.com> <1450706913890-4868139.post@n4.nabble.com> Message-ID: Marcel !! Thank you very very much for your such quick response. I've tried the ChangeSet and your new "type something then hit CMD+G to find again" feature works fine for me. I'm relieved and looking forward to next Squeak release. -- sumim 2015-12-21 23:27 GMT+09:00 marcel.taeumel : > Hi, > > here is a change set that should make in-place-find work: > undo-inplace-find-fix.cs > > > Please test it, then I can commit it into trunk. Here is how it works: > > - You just moved your cursor to some place without selecting something. > - You type something. > - You hit *CMD+G* to "find again" > - The previous input will be removed and taken as search text. > - The next match will be selected. > - (If you err here, you can *undo* with CMD+Z and insert the input again.) > > If you hit CMD+J, nothing will happen because there was no replace operation > detected. If you, however, hit CMD+G to find the next match and then > continue wit a CMD+J, it will find/replace the next match (with nothing) > because your insertion was removed automatically. > > Does this make sense? > > Best, > Marcel > > > > -- > View this message in context: http://forum.world.st/Please-don-t-kill-the-in-place-find-feature-of-the-former-CTRL-J-tp4868119p4868139.html > Sent from the Squeak - Dev mailing list archive at Nabble.com. > From jdbellomm at gmail.com Mon Dec 21 16:54:29 2015 From: jdbellomm at gmail.com (Jerry Bell) Date: Mon Dec 21 16:54:33 2015 Subject: [squeak-dev] Please don't kill the in-place find feature of the former CTRL-J In-Reply-To: References: Message-ID: I don't recall seeing that Apple Smalltalk before. Is it available somewhere? Does it run on a Squeak VM? Thanks Jerry > On Dec 21, 2015, at 6:04 AM, Masato Sumi wrote: > > I'm very worry about that the new CTRL-J (aka "again") of tunk seems > lacking very old but elegant feature of the former CTRL-J, "in-place > find" function. > > When you just inserted text at a caret, the former version of CTRL-J > (or select "again" from yellow-button menu) sets the text as a search > key then delete it and then select next found text. > > I prefer and everyday use this feature because it make me find the > text very seamlessly. > > I know my English is too bad to explain about this feature, so I made > a screen-recording below. (Although it is very old Apple Smalltalk, > you can do the same thing in current Squeak but not trunk) > > https://www.youtube.com/watch?v=-jKdn_9dKeQ > > If it was my misunderstanding and the new CTRL-J still keeps this > feature, please let me how to use it. > > Thanks > -- > sumim > From tim at rowledge.org Mon Dec 21 18:32:28 2015 From: tim at rowledge.org (tim Rowledge) Date: Mon Dec 21 18:32:33 2015 Subject: [squeak-dev] Please don't kill the in-place find feature of the former CTRL-J In-Reply-To: References: Message-ID: <9450C99B-6D4A-4A52-8431-D92ED413F928@rowledge.org> > On 21-12-2015, at 8:54 AM, Jerry Bell wrote: > > I don't recall seeing that Apple Smalltalk before. Is it available somewhere? Does it run on a Squeak VM? That?s an ancient Apple vm running a slightly modified Smalltalk -80 direct from SCG PARC; Apple took part in the early experiments detailed in the Green Book[1]. This was actually available from Apple for a while when the Mac was young, I think via one or other of the developer programs. It also ?grew up? to become the first Squeak image as released in ?96 and of course that has grown up to become the current Squeak and Pharo images. So in a sense, yes it actually does run on a Squeak VM. As for actually running the old Apple image as shown in that video? I doubt anyone has built a vm for that in a while. It?s just faintly possible that a really old vm from http://ftp.squeak.org/1.1/ might run it, if you could get hold of it. And a Mac that such an old VM can run on. Take a look at http://squeak.org/downloads/ and try a few older downloads out to see which is the oldest you can find a way to run. tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim <-------- The information went data way --------> From robert.w.withers at gmail.com Mon Dec 21 18:50:27 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Mon Dec 21 18:50:31 2015 Subject: [squeak-dev] Please don't kill the in-place find feature of the former CTRL-J In-Reply-To: <9450C99B-6D4A-4A52-8431-D92ED413F928@rowledge.org> References: <9450C99B-6D4A-4A52-8431-D92ED413F928@rowledge.org> Message-ID: <567849F3.7050507@gmail.com> On 12/21/2015 01:32 PM, tim Rowledge wrote: >> On 21-12-2015, at 8:54 AM, Jerry Bell wrote: >> >> I don't recall seeing that Apple Smalltalk before. Is it available somewhere? Does it run on a Squeak VM? > That?s an ancient Apple vm running a slightly modified Smalltalk -80 direct from SCG PARC; Apple took part in the early experiments detailed in the Green Book[1]. This was actually available from Apple for a while when the Mac was young, I think via one or other of the developer programs. It also ?grew up? to become the first Squeak image as released in ?96 and of course that has grown up to become the current Squeak and Pharo images. So in a sense, yes it actually does run on a Squeak VM. > > As for actually running the old Apple image as shown in that video? I doubt anyone has built a vm for that in a while. It?s just faintly possible that a really old vm from http://ftp.squeak.org/1.1/ might run it, if you could get hold of it. And a Mac that such an old VM can run on. > > Take a look at http://squeak.org/downloads/ and try a few older downloads out to see which is the oldest you can find a way to run. Oh jeez, this is like Ready Player one all over again! :_) awesome, robert > > tim > -- > tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim > <-------- The information went data way --------> > > > -- . .. ... ^,^ robert Go Panthers! From nicolas.cellier.aka.nice at gmail.com Mon Dec 21 21:01:34 2015 From: nicolas.cellier.aka.nice at gmail.com (Nicolas Cellier) Date: Mon Dec 21 21:01:36 2015 Subject: [squeak-dev] review, validation and test development for GenericGF In-Reply-To: <56774597.3020609@gmail.com> References: <566FF9CB.7000804@gmail.com> <56774597.3020609@gmail.com> Message-ID: Since GF is all about algebra and arithmetic it should be possible to write tests and check some invariants... For example, this works with SecureSession-rww.70: | gf | GenericGF xxinitialize. gf := GenericGF RS_256_A. 1 to: gf size - 1 do: [:a | self assert: (gf multiply: a scalar: (gf inverse: a)) = 1] One could test polynom product vs division for example, for some monomials... 2015-12-21 1:19 GMT+01:00 Robert Withers : > Hi Nicolas, > > I tried the fixes you suggested (1 and 3) and it still isn't working. I > published tothe repository with these changes, though the pipeline is > broken as I rewire my skull (intro frames). This will take several days to > complete, I think, although progress is being made. Then I will be able to > turn my full attention back to RS and Arrays for starters. Please pardon > the delay, but I think frames will bring an advantage, perhaps not > performance but design and flexibility. > > Many thanks for taking a close look at RS, > robert > > > On 12/20/2015 12:18 PM, Nicolas Cellier wrote: > > After a first glimpse, I noted these: > > 1) GenericGFPoly evaluateAt: does not set the result in the loop... > 2) why using an OrderedCollection for the coefficients ? an Array is > faster. > same for exp and log tables of GenericGF > 3) GenericGF>>multiply:scalar: you changed <= size into < size but > shouldn't... > > 2015-12-15 12:30 GMT+01:00 Robert Withers : > >> Hi all, sticking with the distribution for discussing Crypto in the base, >> I am currently trying to stabilize a port of Reed-Solomon such that FEC can >> be added to SecureSession. I am doing this work in SecureSession. I just >> posted an update of the GenericGF class, with the Java implementation >> attached here. >> >> I need this critical code reviewed and tested. You need Cryptography and >> SecureSession to be included in the catalog. If you can help me with the >> former, I could help with the latter, if that suits you. Please let me know! >> >> Please load Cryptography(-Pharo) and SecureSession.23 for review. >> >> "ported & refactored GenericGF from the java code. This includes >> reindexing from 0 to 1. Having no tests, hopefully someone can give a >> review of this specific class. Where some tests to be developed at this >> scope, that would be stellar." >> >> >> I appreciate. >> -- >> . .. .. ^,^ best, robert >> >> >> >> > > > > -- > . .. .. ^,^ robert > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151221/beb46735/attachment.htm From robert.w.withers at gmail.com Mon Dec 21 21:03:03 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Mon Dec 21 21:03:09 2015 Subject: [Pharo-users] [squeak-dev] encoding challenge In-Reply-To: References: <5674F4A9.90101@gmail.com> Message-ID: <56786907.3040802@gmail.com> Thank you, this is good. I was modeling DateAndTime as UTCTime, which is a specified format for two digit years: http://www.obj-sys.com/asn1tutorial/node15.html. I noticed this other time representation while lookin... GeneralizedTime: http://www.obj-sys.com/asn1tutorial/node14.html. To meet that format with padding, I decided to convert to strings. Not pretty but there is a test! :) Unfortunately the ASN.1 DER endoding for time is specific format. I coudl just throws milliseconds in an integer slot, but I wanted to meet the spec as well for a full ASN.1 tool. I am undecided whether to use ASN.1 for header encoding in SecureSession, or custom binary specification - I use the second option at this time. Yet I am in the middle of redefining that message protocol, so now would be the time to decide. Fuel and I have a date coming up, for sure. thank you, Robert On 12/21/2015 03:34 PM, Sven Van Caekenberghe wrote: > Robert, > >> On 19 Dec 2015, at 07:09, Robert Withers wrote: >> >> A core part of communications, whether online or to persistence, is encoding, or marshalling. Let's do an example! Thank you for your thoughtful, earnest participation, as God wills it. >> >> Here are 2 methods that encode and decode a DateAndTime object, but rather poorly. I've been going without much sleep for a bit and I can't figure it out. Here is the code in hopes that some will find this an interesting exercise in team and community building. For that intention, as a beginning. What's the best solution? >> >> Let's show them how to code when given the space in the environment. Tnks! :) >> >> In class ASN1UTCTimeType.. >> >> encodeValue: anObject withDERStream: derStream >> >> | yy mo dd hh mm ss utcDateTime | >> yy := anObject year asString copyFrom: 3 to: 4. >> mo := anObject month asString padded: #left to: 2 with: $0. >> dd := anObject dayOfMonth asString padded: #left to: 2 with: $0. >> hh := anObject hour asString padded: #left to: 2 with: $0. >> mm := anObject minute asString padded: #left to: 2 with: $0. >> ss := anObject seconds asString padded: #left to: 2 with: $0. >> utcDateTime := (yy, mo, dd, hh, mm, ss, 'Z') asByteArray. >> derStream nextPutAll: utcDateTime. >> >> >> >> decodeValueWithDERStream: derStream length: length >> >> | aUTCDateTime | >> aUTCDateTime := (derStream next: length) asByteArray asString. >> ^ (DateAndTime readFromString: ( >> ((aUTCDateTime copyFrom: 1 to: 2) asInteger > 50 ifTrue: ['19'] ifFalse: ['20']), >> (aUTCDateTime copyFrom: 1 to: 2), '-', >> (aUTCDateTime copyFrom: 3 to: 4), '-', >> (aUTCDateTime copyFrom: 5 to: 6), 'T', >> (aUTCDateTime copyFrom: 7 to: 8), ':', >> (aUTCDateTime copyFrom: 9 to: 10), ':', >> (aUTCDateTime copyFrom: 11 to: 12))) >> offset: 0 >> >> >> -- >> . .. .. ^,^ robert > Like you say, this is *not* good, because it is pretty inefficient. You want to encode binary, yet you create a string representation. But even that is done badly ;-) Any you are introducing 2YK all over again. > > Anyway, the solution is to encode the timestamp using one or more numbers, directly. From your code I assume you are only interested in second precision and the UTC/GMT/Z timezone. In that case either one rather large number (seconds since X, like unix time) or two smaller numbers (julian day number and seconds since midnight) are way more efficient. > > For inspiration, have a look at Fuel's DateAndTime>>#serializeOn: (that also encodes nanoseconds and the timezone). > > HTH, > > Sven > > > -- . .. ... ^,^ robert Go Panthers! From robert.w.withers at gmail.com Mon Dec 21 21:07:18 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Mon Dec 21 21:07:24 2015 Subject: [squeak-dev] review, validation and test development for GenericGF In-Reply-To: References: <566FF9CB.7000804@gmail.com> <56774597.3020609@gmail.com> Message-ID: <56786A06.7040202@gmail.com> Yes, the first step is to write a bunch of tests. You are testing the math with (a * (1/a)) == 1, a good test. It's in the queue, especially the multiply, divide and monomials testing, as you suggest. All of it, not just the outer layer with something this complex. thank you, Robert On 12/21/2015 04:01 PM, Nicolas Cellier wrote: > Since GF is all about algebra and arithmetic it should be possible to > write tests and check some invariants... > For example, this works with SecureSession-rww.70: > > | gf | > GenericGF xxinitialize. > gf := GenericGF RS_256_A. > 1 to: gf size - 1 do: [:a | > self assert: (gf multiply: a scalar: (gf inverse: a)) = 1] > > One could test polynom product vs division for example, for some > monomials... > > 2015-12-21 1:19 GMT+01:00 Robert Withers >: > > Hi Nicolas, > > I tried the fixes you suggested (1 and 3) and it still isn't > working. I published tothe repository with these changes, though > the pipeline is broken as I rewire my skull (intro frames). This > will take several days to complete, I think, although progress is > being made. Then I will be able to turn my full attention back to > RS and Arrays for starters. Please pardon the delay, but I think > frames will bring an advantage, perhaps not performance but design > and flexibility. > > Many thanks for taking a close look at RS, > robert > > > On 12/20/2015 12:18 PM, Nicolas Cellier wrote: >> After a first glimpse, I noted these: >> >> 1) GenericGFPoly evaluateAt: does not set the result in the loop... >> 2) why using an OrderedCollection for the coefficients ? an Array >> is faster. >> same for exp and log tables of GenericGF >> 3) GenericGF>>multiply:scalar: you changed <= size into < size >> but shouldn't... >> >> 2015-12-15 12:30 GMT+01:00 Robert Withers >> >: >> >> Hi all, sticking with the distribution for discussing Crypto >> in the base, I am currently trying to stabilize a port of >> Reed-Solomon such that FEC can be added to SecureSession. I >> am doing this work in SecureSession. I just posted an update >> of the GenericGF class, with the Java implementation attached >> here. >> >> I need this critical code reviewed and tested. You need >> Cryptography and SecureSession to be included in the catalog. >> If you can help me with the former, I could help with the >> latter, if that suits you. Please let me know! >> >> Please load Cryptography(-Pharo) and SecureSession.23 for >> review. >> >> "ported & refactored GenericGF from the java code. This >> includes reindexing from 0 to 1. Having no tests, >> hopefully someone can give a review of this specific >> class. Where some tests to be developed at this scope, >> that would be stellar." >> >> >> I appreciate. >> -- >> . .. .. ^,^ best, robert >> >> >> >> >> >> > > -- > . .. .. ^,^ robert > > > > > > -- . .. ... ^,^ robert Go Panthers! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151221/fb7c31ac/attachment.htm From robert.w.withers at gmail.com Tue Dec 22 00:08:28 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Tue Dec 22 00:08:35 2015 Subject: [squeak-dev] review, validation and test development for GenericGF In-Reply-To: <56786A06.7040202@gmail.com> References: <566FF9CB.7000804@gmail.com> <56774597.3020609@gmail.com> <56786A06.7040202@gmail.com> Message-ID: <5678947C.7010702@gmail.com> I ought to clarify something I said that was misleading to whomever read this. I think normally people think of "being in the queue" as forthcoming momentarily. To myself, it means I am planning to work on it as soon as the stack is whittled down, being the short-term task manager. In other words, the stack is short-term, the queue is medium-term and the pool is long-term. The stack is interruptible, the queue is one of priority and the pool is re-entrant. I try to frame my mind to work this way but I didn't want to raise the wrong expectations through old miscommunication - it's common. regards, Robert On 12/21/2015 04:07 PM, Robert Withers wrote: > Yes, the first step is to write a bunch of tests. You are testing the > math with (a * (1/a)) == 1, a good test. It's in the queue, especially > the multiply, divide and monomials testing, as you suggest. All of it, > not just the outer layer with something this complex. > > thank you, > Robert > > On 12/21/2015 04:01 PM, Nicolas Cellier wrote: >> Since GF is all about algebra and arithmetic it should be possible to >> write tests and check some invariants... >> For example, this works with SecureSession-rww.70: >> >> | gf | >> GenericGF xxinitialize. >> gf := GenericGF RS_256_A. >> 1 to: gf size - 1 do: [:a | >> self assert: (gf multiply: a scalar: (gf inverse: a)) = 1] >> >> One could test polynom product vs division for example, for some >> monomials... >> >> 2015-12-21 1:19 GMT+01:00 Robert Withers > >: >> >> Hi Nicolas, >> >> I tried the fixes you suggested (1 and 3) and it still isn't >> working. I published tothe repository with these changes, though >> the pipeline is broken as I rewire my skull (intro frames). This >> will take several days to complete, I think, although progress is >> being made. Then I will be able to turn my full attention back to >> RS and Arrays for starters. Please pardon the delay, but I think >> frames will bring an advantage, perhaps not performance but >> design and flexibility. >> >> Many thanks for taking a close look at RS, >> robert >> >> >> On 12/20/2015 12:18 PM, Nicolas Cellier wrote: >>> After a first glimpse, I noted these: >>> >>> 1) GenericGFPoly evaluateAt: does not set the result in the loop... >>> 2) why using an OrderedCollection for the coefficients ? an >>> Array is faster. >>> same for exp and log tables of GenericGF >>> 3) GenericGF>>multiply:scalar: you changed <= size into < size >>> but shouldn't... >>> >>> 2015-12-15 12:30 GMT+01:00 Robert Withers >>> : >>> >>> Hi all, sticking with the distribution for discussing Crypto >>> in the base, I am currently trying to stabilize a port of >>> Reed-Solomon such that FEC can be added to SecureSession. I >>> am doing this work in SecureSession. I just posted an update >>> of the GenericGF class, with the Java implementation >>> attached here. >>> >>> I need this critical code reviewed and tested. You need >>> Cryptography and SecureSession to be included in the >>> catalog. If you can help me with the former, I could help >>> with the latter, if that suits you. Please let me know! >>> >>> Please load Cryptography(-Pharo) and SecureSession.23 for >>> review. >>> >>> "ported & refactored GenericGF from the java code. This >>> includes reindexing from 0 to 1. Having no tests, >>> hopefully someone can give a review of this specific >>> class. Where some tests to be developed at this scope, >>> that would be stellar." >>> >>> >>> I appreciate. >>> -- >>> . .. .. ^,^ best, robert >>> >>> >>> >>> >>> >>> >> >> -- >> . .. .. ^,^ robert >> >> >> >> >> >> > > -- > . .. ... ^,^ robert > Go Panthers! -- . .. ... ^,^ robert Go Panthers! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151221/9a1735d2/attachment.htm From sumi at seagreen.ocn.ne.jp Tue Dec 22 02:12:34 2015 From: sumi at seagreen.ocn.ne.jp (Masato Sumi) Date: Tue Dec 22 02:13:16 2015 Subject: [squeak-dev] Please don't kill the in-place find feature of the former CTRL-J In-Reply-To: <567849F3.7050507@gmail.com> References: <9450C99B-6D4A-4A52-8431-D92ED413F928@rowledge.org> <567849F3.7050507@gmail.com> Message-ID: Thank you for your follow-up, tim . Hi, robert I purchased it from APDA (Apple Programmer's Developer's Association) about 25 years ago. Currently, you can get its image, changes, sources from here. (but it seems lacking VM, sorry) Apple Smalltalk-80 v0.4.zip http://www.esug.org/data/Smalltalk/Squeak/History/ You can also read simple manual on this site. Apple Smalltalk-80 manual https://archive.org/details/bitsavers_applemacSmntoshAug85_1587556 Even if you will able to get its VM, that is for very old Mac System (ver 6.0.8 or so). So, you also need Mac Plus emulation software (such as vMac mini), Mac Plus' ROM file, System Software and so on. -- sumim 2015-12-22 3:50 GMT+09:00 Robert Withers : > > > On 12/21/2015 01:32 PM, tim Rowledge wrote: >>> >>> On 21-12-2015, at 8:54 AM, Jerry Bell wrote: >>> >>> I don't recall seeing that Apple Smalltalk before. Is it available >>> somewhere? Does it run on a Squeak VM? >> >> That?s an ancient Apple vm running a slightly modified Smalltalk -80 >> direct from SCG PARC; Apple took part in the early experiments detailed in >> the Green Book[1]. This was actually available from Apple for a while when >> the Mac was young, I think via one or other of the developer programs. It >> also ?grew up? to become the first Squeak image as released in ?96 and of >> course that has grown up to become the current Squeak and Pharo images. So >> in a sense, yes it actually does run on a Squeak VM. >> >> As for actually running the old Apple image as shown in that video? I >> doubt anyone has built a vm for that in a while. It?s just faintly possible >> that a really old vm from http://ftp.squeak.org/1.1/ might run it, if you >> could get hold of it. And a Mac that such an old VM can run on. >> >> Take a look at http://squeak.org/downloads/ and try a few older downloads >> out to see which is the oldest you can find a way to run. > > > Oh jeez, this is like Ready Player one all over again! :_) > > > awesome, > robert > > >> >> tim >> -- >> tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim >> <-------- The information went data way --------> >> >> >> > > -- > . .. ... ^,^ robert > Go Panthers! > > From horrido.hobbies at gmail.com Tue Dec 22 11:11:12 2015 From: horrido.hobbies at gmail.com (Richard Eng) Date: Tue Dec 22 11:11:16 2015 Subject: [squeak-dev] A Suggested Promotion Message-ID: I have a suggestion for future Smalltalk advocates... The advent of Pi Zero last month is an interesting opportunity. This is a very, very cheap Raspberry Pi. Anyone can afford it. I'm thinking of kids, in particular. What if we could get kids to delve into Squeak or Pharo (Squeak is better optimized for Raspberry Pi) via the Pi Zero? A competition could be set up in various countries, such as Canada, Britain, Australia, Japan, etc., where the best student submission for a Smalltalk-based Pi Zero application would win a small scholarship prize (say, $500). A panel of judges in each country would decide the winner. The contest would be much less ambitious than my now-defunct Canadian Smalltalk Competition (CSC). Less control (no real need to prevent cheating). Easier to raise the prize money. The nice thing is that it would be a multi-national promotion (it's tough getting a national school competition like CSC going in other countries). Of course, because it's less ambitious, we can expect to reach fewer kids. Instead of thousands, probably more like hundreds. I could be totally wrong on this; I could be totally surprised! I might even come out of retirement to offer some creative services (eg, promotional graphics, websites, etc.). ? Anyway, it was just an interesting thought I had. Don't know if it stands up to scrutiny. Cheers, Richard -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151222/9f75a00a/attachment.htm From commits at source.squeak.org Tue Dec 22 15:47:09 2015 From: commits at source.squeak.org (commits@source.squeak.org) Date: Tue Dec 22 15:47:10 2015 Subject: [squeak-dev] The Trunk: Morphic-mt.1061.mcz Message-ID: Marcel Taeumel uploaded a new version of Morphic to project The Trunk: http://source.squeak.org/trunk/Morphic-mt.1061.mcz ==================== Summary ==================== Name: Morphic-mt.1061 Author: mt Time: 22 December 2015, 4:46:35.738 pm UUID: af0fca47-4c9f-4973-ab53-a9fa9edcccef Ancestors: Morphic-tpr.1060 Text editing: code clean-up and fix for in-place find. Here is how it works: - You just moved your cursor to some place without selecting something. - You type something. - You hit CMD+G to "find again" - The previous input will be removed and taken as search text. - The next match will be selected. - (If you err here, you can undo with CMD+Z to insert the input from above again.) If you hit CMD+J, nothing will happen because there was no replace operation detected. If you, however, hit CMD+G to find the next match and then continue wit a CMD+J, it will find/replace the next match (with nothing) because your insertion was removed automatically as described above. =============== Diff against Morphic-tpr.1060 =============== Item was changed: ----- Method: TextEditor>>again (in category 'menu messages') ----- again + "Do the same find/replace command again. Looks up the editor's own command history and uses the previous command to determine find string and replace string." - "Do the same replace command again. Unlike #findReplaceAgain, this looks up the editor's own command history and uses the previous command." + self history hasReplacedSomething - (self history hasPrevious and: [self history previous hasReplacedSomething]) ifFalse: [morph flash. ^ false]. self setSearchFromSelectionOrHistory; setReplacementFromHistory. "If we have no selection, give the user one to avoid annoying surprises." ^ self hasSelection ifTrue: [self findReplaceAgainNow] ifFalse: [self findAgainNow. false "see #againUpToEnd"]! Item was changed: ----- Method: TextEditor>>findReplaceAgain (in category 'menu messages') ----- findReplaceAgain - - self - setSearchFromSelectionOrHistory; - setReplacementFromHistory. + ^ self again! - ^ self findReplaceAgainNow! Item was added: + ----- Method: TextEditor>>removePreviousInsertion (in category 'typing support') ----- + removePreviousInsertion + "If the previous command was an insertion, remove it by not undoing it but replacing it with nothing." + + self history hasInsertedSomething ifFalse: [morph flash. ^ false]. + + self + selectInvisiblyFrom: self history previous intervalBefore first + to: self history previous intervalAfter last. + + self replaceSelectionWith: self nullText. + + ^ true + ! Item was changed: ----- Method: TextEditor>>setReplacementFromHistory (in category 'accessing') ----- setReplacementFromHistory "Use history to get the previous replacement." + self history hasReplacedSomething - (self history hasPrevious and: [self history previous hasReplacedSomething]) ifTrue: [ChangeText := self history previous contentsAfter].! Item was changed: ----- Method: TextEditor>>setSearchFromSelectionOrHistory (in category 'accessing') ----- setSearchFromSelectionOrHistory "Updates the current string to find with the current selection or the last change if it replaced something and thus had a prior selection." self hasSelection ifTrue: [FindText := self selection] + ifFalse: [self history hasReplacedSomething + ifTrue: [FindText := self history previous contentsBefore] + ifFalse: [self history hasInsertedSomething + ifTrue: [ + FindText := self history previous contentsAfter. + self removePreviousInsertion. "Undoable."]]]! - ifFalse: [(self history hasPrevious and: [self history previous hasReplacedSomething]) - ifTrue: [FindText := self history previous contentsBefore]].! Item was added: + ----- Method: TextEditorCommand>>hasInsertedSomething (in category 'testing') ----- + hasInsertedSomething + + ^ self contentsBefore isEmpty! Item was added: + ----- Method: TextEditorCommandHistory>>hasInsertedSomething (in category 'testing') ----- + hasInsertedSomething + + ^ self hasPrevious and: [self previous hasInsertedSomething]! Item was added: + ----- Method: TextEditorCommandHistory>>hasReplacedSomething (in category 'testing') ----- + hasReplacedSomething + + ^ self hasPrevious and: [self previous hasReplacedSomething]! From Marcel.Taeumel at hpi.de Tue Dec 22 15:30:39 2015 From: Marcel.Taeumel at hpi.de (marcel.taeumel) Date: Tue Dec 22 15:49:27 2015 Subject: [squeak-dev] Re: Please don't kill the in-place find feature of the former CTRL-J In-Reply-To: References: <1450699683717-4868121.post@n4.nabble.com> <1450700414787-4868124.post@n4.nabble.com> <1450701968225-4868130.post@n4.nabble.com> <1450706913890-4868139.post@n4.nabble.com> Message-ID: <1450798239222-4868267.post@n4.nabble.com> In trunk it is: http://forum.world.st/The-Trunk-Morphic-mt-1061-mcz-td4868266.html :-) Best, Marcel -- View this message in context: http://forum.world.st/Please-don-t-kill-the-in-place-find-feature-of-the-former-CTRL-J-tp4868119p4868267.html Sent from the Squeak - Dev mailing list archive at Nabble.com. From nicolaihess at gmail.com Tue Dec 22 16:08:23 2015 From: nicolaihess at gmail.com (Nicolai Hess) Date: Tue Dec 22 16:08:27 2015 Subject: [squeak-dev] some dead (?) links (smalltalk.org/opencobalt/iliadproject/swazoo) Message-ID: don't know where to post this, but I think some of you may be or was involved in this projects or know someone how did: smalltalk.org -> Error The requested URL could not be retrieved www.opencobalt.org -> This website is temporarily unavailable, please try again later. http://www.swazoo.org -> "This domain has expired. If you owned this domain, ...." http://www.iliadproject.org -> Does not look like a web frame work :) http://gjallar.se/ -> does not like like an issue management system. wikipedia links to smalltalk.org the other links are from http://www.world.st/try/projects and squeak.org has a link to opencobalt.org -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151222/299cece5/attachment.htm From Das.Linux at gmx.de Tue Dec 22 18:31:05 2015 From: Das.Linux at gmx.de (Tobias Pape) Date: Tue Dec 22 18:31:09 2015 Subject: [squeak-dev] ftp change: /trunk Message-ID: <59EAFE8F-EC8D-4085-922B-E52EC1158324@gmx.de> Dear all I just noticed, that http://ftp.squeak.org/trunk actually was a deep copy of http://ftp.squeak.org/4.2alpha. I removed the content from /trunk and made it a link to http://ftp.squeak.org/5.1alpha. Best regards -Tobias PS: the usage of FTP to access the files is actually steadily decreasing. What about renaming it to files.squeak.org, keeping content? [and, important for box admins, moving it away from the box2-on-box4 infra]. Thoughs? From tim at rowledge.org Tue Dec 22 19:35:50 2015 From: tim at rowledge.org (tim Rowledge) Date: Tue Dec 22 19:35:54 2015 Subject: [squeak-dev] some dead (?) links (smalltalk.org/opencobalt/iliadproject/swazoo) In-Reply-To: References: Message-ID: <84F7CF8E-5693-4E82-90FB-DDE03C1A2D69@rowledge.org> Also - sqlite points to a self-proclaimed obsolete page on Squeaksource, which in turn points to a page on SmalltalkHub that declares the package obsolete. It refers instead to a Pharo only version. Scratch refers only to the MIT site where a very old and not runnable on modern VMs image lives. With no support. It would be nicer to refer at least additionally to the Raspberry Pi release that runs on the latest VMs. swiki -points to the page about swikis, which should be the right place, but it?s *so* out of date it is embarrassing. A pointer to downloading a 3.7 based system? FunSqueak - bad pointer tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Never forget: 2 + 2 = 5 for extremely large values of 2. From leves at caesar.elte.hu Tue Dec 22 20:11:05 2015 From: leves at caesar.elte.hu (Levente Uzonyi) Date: Tue Dec 22 20:11:10 2015 Subject: [squeak-dev] Re: [Box-Admins] ftp change: /trunk In-Reply-To: <59EAFE8F-EC8D-4085-922B-E52EC1158324@gmx.de> References: <59EAFE8F-EC8D-4085-922B-E52EC1158324@gmx.de> Message-ID: On Tue, 22 Dec 2015, Tobias Pape wrote: > Dear all > > I just noticed, that http://ftp.squeak.org/trunk > actually was a deep copy of http://ftp.squeak.org/4.2alpha. > > I removed the content from /trunk and made it a link to > http://ftp.squeak.org/5.1alpha. > > Best regards > -Tobias > > PS: the usage of FTP to access the files is actually steadily decreasing. > What about renaming it to files.squeak.org, keeping content? > [and, important for box admins, moving it away from the box2-on-box4 > infra]. Thoughs? > I agree. The old subdomain should probably be kept as a cname record. Levente From commits at source.squeak.org Tue Dec 22 21:33:26 2015 From: commits at source.squeak.org (commits@source.squeak.org) Date: Tue Dec 22 21:33:28 2015 Subject: [squeak-dev] The Trunk: ToolBuilder-Morphic-cmm.154.mcz Message-ID: Chris Muller uploaded a new version of ToolBuilder-Morphic to project The Trunk: http://source.squeak.org/trunk/ToolBuilder-Morphic-cmm.154.mcz ==================== Summary ==================== Name: ToolBuilder-Morphic-cmm.154 Author: cmm Time: 22 December 2015, 3:33:18.425 pm UUID: 9c0f9cf0-1b21-448f-8401-f69edc04e17e Ancestors: ToolBuilder-Morphic-mt.153 Encountered the model already nil'd out before entering PluggableSystemWindow>>#delete on occasion. Protect against that. =============== Diff against ToolBuilder-Morphic-mt.153 =============== Item was changed: ----- Method: PluggableSystemWindow>>delete (in category 'initialization') ----- delete + (model notNil and: [ closeWindowSelector notNil ]) ifTrue: [ model perform: closeWindowSelector ]. + super delete! - closeWindowSelector ifNotNil:[model perform: closeWindowSelector]. - super delete. - ! From commits at source.squeak.org Tue Dec 22 21:49:09 2015 From: commits at source.squeak.org (commits@source.squeak.org) Date: Tue Dec 22 21:49:10 2015 Subject: [squeak-dev] The Trunk: Network-cmm.169.mcz Message-ID: Chris Muller uploaded a new version of Network to project The Trunk: http://source.squeak.org/trunk/Network-cmm.169.mcz ==================== Summary ==================== Name: Network-cmm.169 Author: cmm Time: 22 December 2015, 3:48:56.805 pm UUID: 16bcae5a-5b22-4892-af74-18a7f58d0450 Ancestors: Network-dtl.168 Update MailComposition menu for recent find-again changes. =============== Diff against Network-dtl.168 =============== Item was changed: ----- Method: MailComposition>>menuGet:shifted: (in category 'interface') ----- menuGet: aMenu shifted: shifted aMenu addList: { {'find...(f)' translated. #find}. + {'find selection again (g)' translated. #findAgain}. - {'find again (g)' translated. #findAgain}. - {'set search string (h)' translated. #setSearchString}. #-. {'accept (s)' translated. #accept}. {'send message' translated. #submit}}. ^aMenu.! From lists at fniephaus.com Tue Dec 22 22:35:49 2015 From: lists at fniephaus.com (Fabio Niephaus) Date: Tue Dec 22 22:36:02 2015 Subject: [squeak-dev] some dead (?) links (smalltalk.org/opencobalt/iliadproject/swazoo) In-Reply-To: <84F7CF8E-5693-4E82-90FB-DDE03C1A2D69@rowledge.org> References: <84F7CF8E-5693-4E82-90FB-DDE03C1A2D69@rowledge.org> Message-ID: Hi all, Thanks for the feedback, I will look into this as soon as I can. In the meantime, feel free to edit the corresponding Markdown files in [1] and open pull requests if you like. Best, Fabio [1] https://github.com/squeak-smalltalk/squeak.org On Tue, Dec 22, 2015 at 8:35 PM tim Rowledge wrote: > Also - > > sqlite points to a self-proclaimed obsolete page on Squeaksource, which in > turn points to a page on SmalltalkHub that declares the package obsolete. > It refers instead to a Pharo only version. > > Scratch refers only to the MIT site where a very old and not runnable on > modern VMs image lives. With no support. It would be nicer to refer at > least additionally to the Raspberry Pi release that runs on the latest VMs. > > swiki -points to the page about swikis, which should be the right place, > but it?s *so* out of date it is embarrassing. A pointer to downloading a > 3.7 based system? > > FunSqueak - bad pointer > > tim > -- > tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim > Never forget: 2 + 2 = 5 for extremely large values of 2. > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151222/e6069671/attachment.htm From commits at source.squeak.org Tue Dec 22 22:55:02 2015 From: commits at source.squeak.org (commits@source.squeak.org) Date: Tue Dec 22 22:55:03 2015 Subject: [squeak-dev] Daily Commit Log Message-ID: <20151222225502.15928.qmail@box4.squeakfoundation.org> Changes to Trunk (http://source.squeak.org/trunk.html) in the last 24 hours: http://lists.squeakfoundation.org/pipermail/packages/2015-December/009218.html Name: Morphic-mt.1061 Ancestors: Morphic-tpr.1060 Text editing: code clean-up and fix for in-place find. Here is how it works: - You just moved your cursor to some place without selecting something. - You type something. - You hit CMD+G to "find again" - The previous input will be removed and taken as search text. - The next match will be selected. - (If you err here, you can undo with CMD+Z to insert the input from above again.) If you hit CMD+J, nothing will happen because there was no replace operation detected. If you, however, hit CMD+G to find the next match and then continue wit a CMD+J, it will find/replace the next match (with nothing) because your insertion was removed automatically as described above. ============================================= http://lists.squeakfoundation.org/pipermail/packages/2015-December/009219.html Name: ToolBuilder-Morphic-cmm.154 Ancestors: ToolBuilder-Morphic-mt.153 Encountered the model already nil'd out before entering PluggableSystemWindow>>#delete on occasion. Protect against that. ============================================= http://lists.squeakfoundation.org/pipermail/packages/2015-December/009220.html Name: Network-cmm.169 Ancestors: Network-dtl.168 Update MailComposition menu for recent find-again changes. ============================================= From jecel at merlintec.com Wed Dec 23 00:10:24 2015 From: jecel at merlintec.com (Jecel Assumpcao Jr.) Date: Tue Dec 22 23:10:40 2015 Subject: [squeak-dev] A Suggested Promotion In-Reply-To: References: Message-ID: Richard Eng wrote on Tue, 22 Dec 2015 06:11:12 -0500 > I have a suggestion for future Smalltalk advocates... > [Squeak and Raspberry Pi contest idea] One problem is that the standard Linux distributions (including those optimized for the Pi) don't include a complete Squeak (or Pharo). An option would be to create a new distribution just for such a contest. In fact, it could be an extremely stripped down Linux focused exclusively on running Squeak (like Dan Ingalls did for his weather station). One advantage of that is that the boot time would be much smaller than what people are used to. The only thing better than that would be SqueakNOS, which would be a lot more work. -- Jecel From sumi at seagreen.ocn.ne.jp Tue Dec 22 23:22:01 2015 From: sumi at seagreen.ocn.ne.jp (Masato Sumi) Date: Tue Dec 22 23:22:43 2015 Subject: [squeak-dev] Re: Please don't kill the in-place find feature of the former CTRL-J In-Reply-To: <1450798239222-4868267.post@n4.nabble.com> References: <1450699683717-4868121.post@n4.nabble.com> <1450700414787-4868124.post@n4.nabble.com> <1450701968225-4868130.post@n4.nabble.com> <1450706913890-4868139.post@n4.nabble.com> <1450798239222-4868267.post@n4.nabble.com> Message-ID: Great! Thank you very much again, May the in-place find feature be with Squeak. -- sumim 2015/12/23 0:49 "marcel.taeumel" : > In trunk it is: > http://forum.world.st/The-Trunk-Morphic-mt-1061-mcz-td4868266.html :-) > > Best, > Marcel > > > > -- > View this message in context: > http://forum.world.st/Please-don-t-kill-the-in-place-find-feature-of-the-former-CTRL-J-tp4868119p4868267.html > Sent from the Squeak - Dev mailing list archive at Nabble.com. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151223/c2fe0c40/attachment.htm From tim at rowledge.org Wed Dec 23 00:03:55 2015 From: tim at rowledge.org (tim Rowledge) Date: Wed Dec 23 00:04:00 2015 Subject: [squeak-dev] A Suggested Promotion In-Reply-To: <201512222310.tBMNAfM2022888@mail63c0.megamailservers.com> References: <201512222310.tBMNAfM2022888@mail63c0.megamailservers.com> Message-ID: <41C30AEE-FEAE-4009-AA55-92A260F8EBCF@rowledge.org> > On 22-12-2015, at 4:10 PM, Jecel Assumpcao Jr. wrote: > > Richard Eng wrote on Tue, 22 Dec 2015 06:11:12 -0500 >> I have a suggestion for future Smalltalk advocates... >> [Squeak and Raspberry Pi contest idea] > > One problem is that the standard Linux distributions (including those > optimized for the Pi) don't include a complete Squeak (or Pharo). True, though the Pi Raspbian does include the full VM setup (currently a Cog Spur 34xx level vm), which means that a fairly small package of image/change/sources/tutorials/examples/support files is al lthat would be needed. Oh, wait, we don?t have a well thought out tutorial world. By far the biggest problem would be raising money I fear. The foundation can?t offer any because it basically doesn?t have any. It?s just as well the SFC provide free hosting because we?d not be able to pay for commercial support. tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Time to start the War on Errorism before stupidity finally gets us. From robert.w.withers at gmail.com Wed Dec 23 02:00:43 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Wed Dec 23 02:00:50 2015 Subject: [squeak-dev] [SecureSession] version 74 is out with compression enabled Message-ID: <567A004B.9050509@gmail.com> Check it in the crypto repo. Next up? refactor over to new frames. On deck? ReedSolomon -- . .. ... ^,^ robert Go Panthers! From jon at huv.com Wed Dec 23 02:45:00 2015 From: jon at huv.com (Jon Hylands) Date: Wed Dec 23 02:45:03 2015 Subject: [squeak-dev] A Suggested Promotion In-Reply-To: <5679de49.4356ca0a.6a8ae.710bMFETCHER_ADDED@google.com> References: <5679de49.4356ca0a.6a8ae.710bMFETCHER_ADDED@google.com> Message-ID: On Tue, Dec 22, 2015 at 7:10 PM, Jecel Assumpcao Jr. wrote: > One problem is that the standard Linux distributions (including those > optimized for the Pi) don't include a complete Squeak (or Pharo). An > option would be to create a new distribution just for such a contest. In > fact, it could be an extremely stripped down Linux focused exclusively > on running Squeak (like Dan Ingalls did for his weather station). One > advantage of that is that the boot time would be much smaller than what > people are used to. > My brother and I have been experimenting with Tiny Core Linux, which is available on the Pi as piCore. It boots in 12 seconds, from power on to login prompt, on a Pi 2. That gives you wifi plus an openssh server. It would probably add a few seconds to get a graphical shell running, which it also includes. - Jon -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151222/c50396f7/attachment.htm From edgardec2005 at gmail.com Tue Dec 22 20:09:19 2015 From: edgardec2005 at gmail.com (Edgar J. De Cleene) Date: Wed Dec 23 10:55:02 2015 Subject: [squeak-dev] some dead (?) links (smalltalk.org/opencobalt/iliadproject/swazoo) In-Reply-To: <84F7CF8E-5693-4E82-90FB-DDE03C1A2D69@rowledge.org> Message-ID: On 12/22/15, 4:35 PM, "tim Rowledge" wrote: > FunSqueak - bad pointer Could explain more ? Edgar From lists at fniephaus.com Wed Dec 23 10:58:11 2015 From: lists at fniephaus.com (Fabio Niephaus) Date: Wed Dec 23 10:58:24 2015 Subject: [squeak-dev] some dead (?) links (smalltalk.org/opencobalt/iliadproject/swazoo) In-Reply-To: References: <84F7CF8E-5693-4E82-90FB-DDE03C1A2D69@rowledge.org> Message-ID: I'm guessing it's a "bad pointer", because it points to our ftp using ftp:// and that seems to be broken at the moment [1]. I will update the link now... Fabio [1] http://forum.world.st/ftp-down-td4866617.html On Wed, Dec 23, 2015 at 11:55 AM Edgar J. De Cleene wrote: > > > > On 12/22/15, 4:35 PM, "tim Rowledge" wrote: > > > FunSqueak - bad pointer > > > Could explain more ? > > Edgar > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151223/21b9e663/attachment.htm From astares at gmx.de Wed Dec 23 12:53:04 2015 From: astares at gmx.de (Torsten Bergmann) Date: Wed Dec 23 12:53:56 2015 Subject: [squeak-dev] A Suggested Promotion Message-ID: Jecel Assumpcao Jr. wrote on squeak-dev: >>In fact, it could be an extremely stripped down Linux focused exclusively >>on running Squeak (like Dan Ingalls did for his weather station). One >>advantage of that is that the boot time would be much smaller than what >>people are used to. John Hylands wrote on squeak-dev >My brother and I have been experimenting with Tiny Core Linux, which is >available on the Pi as piCore. > >It boots in 12 seconds, from power on to login prompt, on a Pi 2. That >gives you wifi plus an openssh server. It would probably add a few seconds >to get a graphical shell running, which it also includes. Note that for Pharo there is a project from Mike Filonov (owner of pharocloud.com) which was initially called "PharoNOS". It is also based on TinyCore Linux, here x86 based directly booting into Pharo. A description and an downloadable ISO is available on [1]. Runs fine on VirtualBox. >From the name often people thought it was a project similar to "SqueakNOS" (but SqueakNOS boots into Squeak and needed no Linux or other underneath as even the drivers were written in Smalltalk). I guess therefore Mike renamed the project to "Pharocloud OS" now, he also made the build scripts available on [2]. I also did some experiments with Pharo on the Pi - described in an article on [3]. The result was similar: directly booting into the Pharo environment/ a Smalltalk application - but on the Pi. There is Raspbian underneath. A video is available on [4]. However: if there is now some interest in having a fast and direct Squeak/Pharo booting on the Pi maybe this could be a shared effort between both communities as well. Beside squeak-dev I therefore CC the pharo-dev mailinglist as well. Bye T. [1] http://pillarhub.pharocloud.com/hub/mikefilonov/pharonos [2] http://os.pharocloud.com [3] https://medium.com/concerning-pharo/pharo-pi-9eef257b6a21#.qgqg6765p [4] https://www.youtube.com/watch?v=sfxFqQIuawg From tim at rowledge.org Wed Dec 23 17:45:25 2015 From: tim at rowledge.org (tim Rowledge) Date: Wed Dec 23 17:46:20 2015 Subject: [squeak-dev] A Suggested Promotion In-Reply-To: References: Message-ID: <6B98DE23-4253-4729-8AF0-79A0EC3FA338@rowledge.org> Raspbian already has an option to boot straight to Scratch, so I don?t imagine it would be hard to copy whatever settings are used. I don?t know if it currently runs it with -vm-display-fb but I imagine it could. I?m not a huge fan (having tried it in the past) of ?nothing but Smalltalk? because it?s so damn hard. All those tricky hardware drivers to futz with and changing hardware and ? ugh. It seems much more sensible to me to use a Smalltalk as the layer above a decent kernel, to add tools to do all the things one normally needs to do and get the benefits of both systems. Of course, there?s still a *staggering* amount of work to do that. tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim "How many Grogs does it take to change a lightbulb?? "One. Something with manipulatory appendages will be along eventually." From tim at rowledge.org Wed Dec 23 17:46:47 2015 From: tim at rowledge.org (tim Rowledge) Date: Wed Dec 23 17:46:52 2015 Subject: [squeak-dev] some dead (?) links (smalltalk.org/opencobalt/iliadproject/swazoo) In-Reply-To: References: <84F7CF8E-5693-4E82-90FB-DDE03C1A2D69@rowledge.org> Message-ID: <915575A2-C29D-451D-9756-204E68681BBE@rowledge.org> > On 23-12-2015, at 2:58 AM, Fabio Niephaus wrote: > > I'm guessing it's a "bad pointer", because it points to our ftp using ftp:// and that seems to be broken at the moment [1]. That?s correct. tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim "Bo***x" said Pooh when Piglet kneed him in the groin. From jecel at merlintec.com Wed Dec 23 20:26:34 2015 From: jecel at merlintec.com (Jecel Assumpcao Jr.) Date: Wed Dec 23 19:26:49 2015 Subject: [squeak-dev] A Suggested Promotion In-Reply-To: <6B98DE23-4253-4729-8AF0-79A0EC3FA338@rowledge.org> References: <6B98DE23-4253-4729-8AF0-79A0EC3FA338@rowledge.org> Message-ID: Tim Rowledge wrote on Wed, 23 Dec 2015 09:45:25 -0800 > I?m not a huge fan (having tried it in the past) of ?nothing but Smalltalk? > because it?s so damn hard. All those tricky hardware drivers to futz with > and changing hardware and ? ugh. It seems much more sensible to me > to use a Smalltalk as the layer above a decent kernel, to add tools to do > all the things one normally needs to do and get the benefits of both > systems. Of course, there?s still a *staggering* amount of work to do that. C code doesn't write itself, and is as much work to produce it as the equivalent Smalltalk code. Of course, in practically all situations with machines like PCs or the Raspberry Pi someone has already done the C version but the Smalltalk version would still have to be written if anyone were interested in it. I fully agree this is essentially pointless. Since I am interested in new machines, the situation is a bit different. There is a way to do an OS in Smalltalk and still make use of existing C code: https://marcusdenker.de/Squeak/SqueakOS/ Unfortunately, not only is this way out of date but the FluxOS/OSKit project on which it is based has been dead for a very long time (as far as I know). Anyway, isn't there an additional complication in the original Raspberry Pi where the boot code runs on the graphics processor instead of the ARM? The issue is which is the simplest way to get Pi users to run Squeak? A package they can load into Raspbian with everything they need to use the pre-installed VM? An All-In-One package? A complete SD card image? Actual SD cards (this costs money to distribute)? -- Jecel From eliot.miranda at gmail.com Wed Dec 23 22:45:06 2015 From: eliot.miranda at gmail.com (Eliot Miranda) Date: Wed Dec 23 22:45:13 2015 Subject: [squeak-dev] Re: [Pharo-dev] Can I access symbolic constants from NB FFI? In-Reply-To: References: Message-ID: Hi Mariano, > On Dec 22, 2015, at 8:36 PM, Mariano Martinez Peck wrote: > > > >> On Tue, Dec 22, 2015 at 5:24 PM, tesonep@gmail.com wrote: >> Hello Mariano, >> WNOHANG is a constant is defined in sys/wait.h, after the compilation is not included in the object file. >> >> You will have to duplicate its value in the Smalltalk Code. It's value is 1, in linux is defined in bits/waitflags.h as >> >> #define WNOHANG 1 > > mmmmmm but then I would need to take care about its different values in different OS right? > I am reading for example this thread: https://groups.google.com/forum/#!topic/comp.lang.python/_5rCpfYR_ZE > > sunos 4.1.3 : #define WNOHANG 1 > Solaris 2.4 : #define WNOHANG 0100 > OSF1 2.0 : #define WNOHANG 0x1 > Ultrix 4.4 : #define WNOHANG 1 > irix 4.0.5C : #define WNOHANG 0x1 > irix 5.2 : #define WNOHANG 0100 > irix 6.0.1 : #define WNOHANG 0100 > Exactly. I've discussed this before. Please read http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/2012-May/064382.html > OK..the thread is from 1995... I thought the POSIX standard would standardize that... > > I am now checking at /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/usr/include/sys/wait.h > > and it says: > #define WNOHANG 0x00000001 > > But again..I wonder how portable is to stick to that... > > Ok...maybe I will need to go with a VM primitive as #primGetChildExitStatus: that way I can do the: > > self cCode: 'waitpid ( pidToHandle, &exitStatus, WNOHANG )' inSmalltalk: [ exitStatus := -1 ]. > > > pd: que groso Pablito verte por aca!!!! > >> Cheers, >> Pablo >> >>> On Tue, Dec 22, 2015 at 5:12 PM, Mariano Martinez Peck wrote: >>> Hi guys, >>> >>> Say I want to call to waitpid like this: >>> >>> waitpid(childID, &status, WNOHANG); >>> >>> How can I do since I don't have access to WNOHANG int value in order to send it via argument .... ??? >>> >>> Any idea? >>> >>> Thanks in advance, >>> >>> Mariano >>> http://marianopeck.wordpress.com >> >> Pablo Tesone. >> tesonep@gmail.com > > Mariano > http://marianopeck.wordpress.com _,,,^..^,,,_ (phone) -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151223/0183f8db/attachment.htm From horrido.hobbies at gmail.com Thu Dec 24 13:34:07 2015 From: horrido.hobbies at gmail.com (horrido) Date: Thu Dec 24 13:53:08 2015 Subject: [squeak-dev] Re: A Suggested Promotion In-Reply-To: <41C30AEE-FEAE-4009-AA55-92A260F8EBCF@rowledge.org> References: <41C30AEE-FEAE-4009-AA55-92A260F8EBCF@rowledge.org> Message-ID: <1450964047154-4868434.post@n4.nabble.com> Raising the prize money should be a lot easier than for the CSC. Use Kickstarter or Indiegogo. Make a spiffy advertising video (I'm a terrible videographer). Do a nice write-up for the crowdfunding campaign (I can edit your draft). Set a modest funding goal of, say, $2500 (for 5 contest countries). Offer a gift of a Pi Zero with Squeak preinstalled for the first 100 contributors of $30 or more. I think this could work. (Cautionary note: I see that there are quite a few campaigns at Kickstarter that offer Raspberry Pi products. We may be a little late to the game. ) tim Rowledge wrote >> On 22-12-2015, at 4:10 PM, Jecel Assumpcao Jr. < > jecel@ > > wrote: >> >> Richard Eng wrote on Tue, 22 Dec 2015 06:11:12 -0500 >>> I have a suggestion for future Smalltalk advocates... >>> [Squeak and Raspberry Pi contest idea] >> >> One problem is that the standard Linux distributions (including those >> optimized for the Pi) don't include a complete Squeak (or Pharo). > > True, though the Pi Raspbian does include the full VM setup (currently a > Cog Spur 34xx level vm), which means that a fairly small package of > image/change/sources/tutorials/examples/support files is al lthat would be > needed. Oh, wait, we don?t have a well thought out tutorial world. > > By far the biggest problem would be raising money I fear. The foundation > can?t offer any because it basically doesn?t have any. It?s just as well > the SFC provide free hosting because we?d not be able to pay for > commercial support. > > > tim > -- > tim Rowledge; > tim@ > ; http://www.rowledge.org/tim > Time to start the War on Errorism before stupidity finally gets us. -- View this message in context: http://forum.world.st/A-Suggested-Promotion-tp4868244p4868434.html Sent from the Squeak - Dev mailing list archive at Nabble.com. From asqueaker at gmail.com Thu Dec 24 21:41:27 2015 From: asqueaker at gmail.com (Chris Muller) Date: Thu Dec 24 21:42:09 2015 Subject: [squeak-dev] Preferences search broken Message-ID: Desperately trying to turn off 'Sort Message Categories Alphabetically' but the Preferences browser search isn't working. I tried searching for "sort", "alph", "categor". Clearly, something is broken with Preferences search.. From asqueaker at gmail.com Thu Dec 24 21:50:16 2015 From: asqueaker at gmail.com (Chris Muller) Date: Thu Dec 24 21:50:59 2015 Subject: [squeak-dev] Re: Preferences search broken In-Reply-To: References: Message-ID: Looks like the preferencesDictionary is missing many preferences. Ahh Pragmas ("arghh Pragmas"?)... So we just need to figure out how to re-set all the Pragmas? I wonder how they got out-of-sync in the first place.. On Thu, Dec 24, 2015 at 3:41 PM, Chris Muller wrote: > Desperately trying to turn off 'Sort Message Categories > Alphabetically' but the Preferences browser search isn't working. I > tried searching for "sort", "alph", "categor". > > Clearly, something is broken with Preferences search.. From Marcel.Taeumel at hpi.de Fri Dec 25 15:05:01 2015 From: Marcel.Taeumel at hpi.de (marcel.taeumel) Date: Fri Dec 25 15:24:10 2015 Subject: [squeak-dev] Re: Preferences search broken In-Reply-To: References: Message-ID: <1451055901189-4868484.post@n4.nabble.com> Do it: "Preferences registerForEvents." It works again after that. I wonder, what has happened... :-/ Best, Marcel -- View this message in context: http://forum.world.st/Preferences-search-broken-tp4868453p4868484.html Sent from the Squeak - Dev mailing list archive at Nabble.com. From robert.w.withers at gmail.com Sun Dec 27 02:57:48 2015 From: robert.w.withers at gmail.com (Robert Withers) Date: Sun Dec 27 02:57:52 2015 Subject: [squeak-dev] Scratch user Message-ID: I just wanted to post some novel news I heard over Christmas. My 11 year old nephew has used Scratch in school! Seeing as how I gave him a Pi 2 for his gift, he's going to get lots of hands on. My family was surprised & excited to hear that it was based in Squeak, knowing my predilections. Rockin it! Cheers, robert From asqueaker at gmail.com Sun Dec 27 22:02:01 2015 From: asqueaker at gmail.com (Chris Muller) Date: Sun Dec 27 22:02:45 2015 Subject: [squeak-dev] #flash UI output Message-ID: Hi Marcel, one thing I miss is the flash associated with search-string not-found. When I press Command+g and its not found, I want it to flash, not change my selection. In another example, I'm working on phasing out an instVar, I browse references to it and eliminate the references one-by-one. Now before closing the window, I used to be able to arrow though and simply make sure I get a flash on every press. Now it doesn't flash so I really have to look closely.. From Das.Linux at gmx.de Mon Dec 28 16:05:28 2015 From: Das.Linux at gmx.de (Tobias Pape) Date: Mon Dec 28 16:05:33 2015 Subject: [squeak-dev] unblock me? Message-ID: Hi all, I'm not receiving mails from the list since Dec 23. Can someone please unblock me? Best regards -Tobias PS: Merry Christmas From craig at netjam.org Mon Dec 28 16:24:43 2015 From: craig at netjam.org (Craig Latta) Date: Mon Dec 28 16:24:54 2015 Subject: [squeak-dev] re: unblock me? In-Reply-To: References: Message-ID: Hi Tobias-- > I'm not receiving mails from the list since Dec 23. > Can someone please unblock me? Hm, the ban list for squeak-dev is currently empty. Does your mailman settings page for squeak-dev do anything useful? thanks, -C p.s. Happy new year! -- Craig Latta netjam.org +31 6 2757 7177 (SMS ok) + 1 415 287 3547 (no SMS) From Marcel.Taeumel at hpi.de Mon Dec 28 16:29:34 2015 From: Marcel.Taeumel at hpi.de (marcel.taeumel) Date: Mon Dec 28 16:49:04 2015 Subject: [squeak-dev] Re: #flash UI output In-Reply-To: References: Message-ID: <1451320174344-4868679.post@n4.nabble.com> I think we should keep the interaction model consistent and simple. "Find again" selects the next match or nothing if nothing matches. Modifying that might break the (in-place) search/replace behavior. What about doing both: clear the selection *and* flash? Best, Marcel -- View this message in context: http://forum.world.st/flash-UI-output-tp4868595p4868679.html Sent from the Squeak - Dev mailing list archive at Nabble.com. From Das.Linux at gmx.de Mon Dec 28 17:09:41 2015 From: Das.Linux at gmx.de (Tobias Pape) Date: Mon Dec 28 17:29:12 2015 Subject: [squeak-dev] Re: re: unblock me? In-Reply-To: References: Message-ID: <1451322581874-4868691.post@n4.nabble.com> Hey Craig The subscriber page that I have a "bounce score [of] 1.0 out of a maximum of 5.0" (We have received some recent bounces from your address. Your current bounce score is 1.0 out of a maximum of 5.0. Please double check that your subscribed address is correct and that there are no problems with delivery to this address. Your bounce score will be automatically reset if the problems are corrected soon.) I am experiencing the same for the glass list? so I probably also have to check with my isp? -- View this message in context: http://forum.world.st/unblock-me-tp4868674p4868691.html Sent from the Squeak - Dev mailing list archive at Nabble.com. From jakob.reschke at student.hpi.de Mon Dec 28 19:21:52 2015 From: jakob.reschke at student.hpi.de (Jakob Reschke) Date: Mon Dec 28 19:22:21 2015 Subject: [squeak-dev] Re: #flash UI output In-Reply-To: <617171ed30c14bfc99498337f13c5b0a@MX2015-DAG2.hpi.uni-potsdam.de> References: <617171ed30c14bfc99498337f13c5b0a@MX2015-DAG2.hpi.uni-potsdam.de> Message-ID: I would like to add here that neither Notepad, nor Visual Studio nor Eclipse nor Vim change the selection when the search text could not be found. Instead, they pop up modal dialogs, play an exclamation sound and/or display "pattern not found" at the bottom. So at least, it is not consistent with "the outside world" at the moment. The flash would be an unintrusive indication of that state, although its meaning only becomes clear on second thought for first time users (Well, what was that? Ah, probably there are no more occurrences.), unlike in the other editors. Since not everyone likes modal dialogs and there is no general status line in Squeak's text morphs, another alternative could be to insert "no more occurrences found" (or likewise) at the cursor position and make that selected (like the "Nothing more expected ->" which appears before unbalanced brackets when you Do or Print something in a workspace) and have it removed with the next interaction (probably complicated to implement). But I am quite sure someone will find this disrupting as well (not even sure about myself), so maybe just take it as a brainstorming idea... Is there a default means for non-disrupting UI notifications in Squeak? Marcel's Vivide has short text messages appear briefly in the lower right corner of a widget when list filtering yields no more results or is cancelled. I could imagine something like this for search string not found as well. Best regards, Jakob 2015-12-28 17:29 GMT+01:00 Taeumel, Marcel : > I think we should keep the interaction model consistent and simple. "Find > again" selects the next match or nothing if nothing matches. Modifying that > might break the (in-place) search/replace behavior. > > What about doing both: clear the selection *and* flash? > > Best, > Marcel > > > > > > -- > View this message in context: http://forum.world.st/flash-UI-output-tp4868595p4868679.html > Sent from the Squeak - Dev mailing list archive at Nabble.com. > From herbertkoenig at gmx.net Tue Dec 29 12:04:17 2015 From: herbertkoenig at gmx.net (=?UTF-8?Q?Herbert_K=c3=b6nig?=) Date: Tue Dec 29 12:04:20 2015 Subject: [squeak-dev] I seem to get no more posts from Squeak dev Message-ID: <568276C1.9000800@gmx.net> Hi, the last post I received from squeak dev dates 11 December. Curious id this post gets through. Cheers, Herbert From asqueaker at gmail.com Tue Dec 29 15:34:16 2015 From: asqueaker at gmail.com (Chris Muller) Date: Tue Dec 29 15:34:58 2015 Subject: [squeak-dev] I seem to get no more posts from Squeak dev In-Reply-To: <568276C1.9000800@gmx.net> References: <568276C1.9000800@gmx.net> Message-ID: Your post got through fine. I've been receiving squeak-dev emails just fine. Maybe check your Spam folder; I just checked mine, there was one message directed at squeak-dev that was marked, and several addressed to pharo-dev. On Tue, Dec 29, 2015 at 6:04 AM, Herbert K?nig wrote: > Hi, > > the last post I received from squeak dev dates 11 December. > Curious id this post gets through. > > Cheers, > > Herbert > From tim at rowledge.org Tue Dec 29 20:34:10 2015 From: tim at rowledge.org (tim Rowledge) Date: Tue Dec 29 20:34:14 2015 Subject: [squeak-dev] SPARCStation 20 vs RaspberryPi Message-ID: <46918E9E-FE8E-4D9C-8F25-E0899588C66C@rowledge.org> Here?s an interesting performance comparison of a Pi vs the old SPARCStation 20 that was an object of lust back in the old days. http://eschatologist.net/blog/?p=266 tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim "Bother" said Pooh as he said f**k in the wrong conf. From colin at wiresong.com Tue Dec 29 20:46:32 2015 From: colin at wiresong.com (Colin Putney) Date: Tue Dec 29 20:46:35 2015 Subject: [squeak-dev] [OT] What is this called? Message-ID: Hi folks, Sorry for the off-topic post. I'm posting it here because I know there are lots of high-powered comp-sci folks lurking, and I hope to benefit from their wisdom. I have in mind a simple problem. Imagine we have a data structure in the memory of some program. The exact structure it has doesn't matter, but does have structure; it's not just a buffer full of bytes. For argument's sake, we'll assume it's a tree. Our task it to copy this tree. We want to make another tree in memory that is logically equivalent to the first. This is pretty straightforward. We need to walk the tree, allocating new nodes and copying over any internal values they contain, then recursing into the children. So far so good. But what if we generalize the problem? One way to do that would be to make the copy more distant from the original. We could copy into a different address space, in another process or on another machine. We could make the copy more distant in time. Perhaps we need to reclaim the memory that our tree uses, and the reconstruct it later. This means we'll have to do IO of some kind. The simplest adaptation of our tree-walking algorithm might be to allocate space for the nodes on disk, rather than in memory. This is really simple if we have access to raw disk storage, but gets a little more complicated (and slooow) if we're using files in a file system. We can gain back some simplicity by putting all the nodes in one file, and having some kind of internal structure to the file that lets us recover the nodes and links between them. We could also allocate storage space via communicating with another process. That might be over a network, or using some kind or IPC mechanism supplied by the operating system. Once again we'll need to transmit the data within the nodes, along with some metadata describing the connections between them. We can also loosen our definition of "logically equivalent". We might want to construct an equivalent tree in a process running a different version of our program. Or another program entirely, potentially written in another programming language. This forces us to raise the level of abstraction. We can't just copy each node as a blob of raw memory, and assume that the "receiving" program will know how to interpret it. We need some semantic definitions agreed upon by the two programs, and some means of representing those semantics as byte sequences that can be copied between them. Now, this is not an intractable problem. We do it all the time. In fact, I'd say a large fraction of the code I've written in my career has been a solution to some version of this problem. I'm sure many of you can relate. :-) So what is this problem called? What theory describes the possible solutions? Are there classes of solutions that have similar trade-offs? Where can I learn more? Colin -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151229/0265c071/attachment.htm From ron at usmedrec.com Tue Dec 29 21:31:01 2015 From: ron at usmedrec.com (Ron Teitelbaum) Date: Tue Dec 29 21:31:03 2015 Subject: [squeak-dev] [OT] What is this called? In-Reply-To: References: Message-ID: <2e3801d14280$37c9e380$a75daa80$@usmedrec.com> Hi Colin, I think you have mentioned a number of parts of the problem already. Schema Management, Address Space Management, and Version Management. Schema management gives you a protocol that allows you to define the source and destination schema along with any information that might be useful for transferring the data to a new environment. Address Space Management allows you to move data from one location to another while maintaining links and data integrity. What I think of here is that as long as the address space is well defined (move all data from UID to Header+UID) the links can maintain themselves. It?s also possible to allow nodes to move and manage data instead of having one big process that does it for you. For example say you have an address space that separates nodes into general address spaces instead of hard addresses. This would allow you to duplicate data to nodes that grab the closest data to their own address. Finding data in that structure would be looking for data next to addresses that match stored data index of some kind. See Distributed Hash Tables for more info. What?s nice about this method is that the actual structure is not important, nodes can be deleted without losing data, and nodes can maintain their own address space and storage and heal themselves. Version Management is good for adding links to data that point to versions of objects and allow you to find the latest version. You could have maps of versions that give you a point in time version of data. Much like Monticello Configurations. Combining them all is an interesting idea. A query of data might first find the right version in time, locate the right data, and then apply the right transformation to give you an answer. Copying may select just a single version, apply a transformation, and then copy the data to a new address space. That?s my take at least. All the best, Ron Teitelbaum Head Of Engineering 3D Immersive Collaboration Consulting ron@3Dicc.com Follow Me On Twitter: @RonTeitelbaum www.3Dicc.com https://www.google.com/+3Dicc From: squeak-dev-bounces@lists.squeakfoundation.org [mailto:squeak-dev-bounces@lists.squeakfoundation.org] On Behalf Of Colin Putney Sent: Tuesday, December 29, 2015 3:47 PM To: The general-purpose Squeak developers list Subject: [squeak-dev] [OT] What is this called? Hi folks, Sorry for the off-topic post. I'm posting it here because I know there are lots of high-powered comp-sci folks lurking, and I hope to benefit from their wisdom. I have in mind a simple problem. Imagine we have a data structure in the memory of some program. The exact structure it has doesn't matter, but does have structure; it's not just a buffer full of bytes. For argument's sake, we'll assume it's a tree. Our task it to copy this tree. We want to make another tree in memory that is logically equivalent to the first. This is pretty straightforward. We need to walk the tree, allocating new nodes and copying over any internal values they contain, then recursing into the children. So far so good. But what if we generalize the problem? One way to do that would be to make the copy more distant from the original. We could copy into a different address space, in another process or on another machine. We could make the copy more distant in time. Perhaps we need to reclaim the memory that our tree uses, and the reconstruct it later. This means we'll have to do IO of some kind. The simplest adaptation of our tree-walking algorithm might be to allocate space for the nodes on disk, rather than in memory. This is really simple if we have access to raw disk storage, but gets a little more complicated (and slooow) if we're using files in a file system. We can gain back some simplicity by putting all the nodes in one file, and having some kind of internal structure to the file that lets us recover the nodes and links between them. We could also allocate storage space via communicating with another process. That might be over a network, or using some kind or IPC mechanism supplied by the operating system. Once again we'll need to transmit the data within the nodes, along with some metadata describing the connections between them. We can also loosen our definition of "logically equivalent". We might want to construct an equivalent tree in a process running a different version of our program. Or another program entirely, potentially written in another programming language. This forces us to raise the level of abstraction. We can't just copy each node as a blob of raw memory, and assume that the "receiving" program will know how to interpret it. We need some semantic definitions agreed upon by the two programs, and some means of representing those semantics as byte sequences that can be copied between them. Now, this is not an intractable problem. We do it all the time. In fact, I'd say a large fraction of the code I've written in my career has been a solution to some version of this problem. I'm sure many of you can relate. :-) So what is this problem called? What theory describes the possible solutions? Are there classes of solutions that have similar trade-offs? Where can I learn more? Colin -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151229/7ba9f164/attachment.htm From asqueaker at gmail.com Tue Dec 29 21:32:01 2015 From: asqueaker at gmail.com (Chris Muller) Date: Tue Dec 29 21:32:44 2015 Subject: [squeak-dev] [OT] What is this called? In-Reply-To: References: Message-ID: > So what is this problem called? I refer to them as the problems of "transparent persistence" and "normalization". > What theory describes the possible > solutions? I'm not sure what theories, but database implementations address these issues in practice. Your were describing Magma exactly up until the "loosen our definition" paragraph, however, there's no reason the other program could not be coded to access the Magma DB or files directly -- the complete meta-definitions are included as part of the connection process. > Are there classes of solutions that have similar trade-offs? > Where can I learn more? But, obviously you know about ODBMS's, so probably this isn't the answer you're looking for. From huw.softdesigns at gmail.com Tue Dec 29 23:27:24 2015 From: huw.softdesigns at gmail.com (Huw Lloyd) Date: Tue Dec 29 23:27:27 2015 Subject: [squeak-dev] [OT] What is this called? In-Reply-To: References: Message-ID: Hi Colin, To the degree that your conception is object-oriented, in the deeper sense, it is not a problem. Your object(s) know how to persist themselves and may be considered to be extensive over that persistence -- it is merely that their morphology has changed. Are you trying to identify names in a known problem space (e.g. a space defined by the limits of the techniques used) or to identify the problem space that you're in? For the latter, if you're going to generalise then perhaps you should also wave the specialisation of 'perfect' reproduction. So, even more generally, what you seem to describing is a special case of morphogenesis in which a 'copy' is made, e.g. lazy evaluation and cached computation can be thought of as slices in that process. Best, Huw On 29 December 2015 at 21:32, Chris Muller wrote: > > So what is this problem called? > > I refer to them as the problems of "transparent persistence" and > "normalization". > > > What theory describes the possible > > solutions? > > I'm not sure what theories, but database implementations address these > issues in practice. Your were describing Magma exactly up until the > "loosen our definition" paragraph, however, there's no reason the > other program could not be coded to access the Magma DB or files > directly -- the complete meta-definitions are included as part of the > connection process. > > > Are there classes of solutions that have similar trade-offs? > > Where can I learn more? > > But, obviously you know about ODBMS's, so probably this isn't the > answer you're looking for. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151229/5a8652b2/attachment.htm From colin at wiresong.com Wed Dec 30 06:49:41 2015 From: colin at wiresong.com (Colin Putney) Date: Wed Dec 30 06:49:43 2015 Subject: [squeak-dev] [OT] What is this called? In-Reply-To: References: Message-ID: On Tue, Dec 29, 2015 at 1:32 PM, Chris Muller wrote: > But, obviously you know about ODBMS's, so probably this isn't the > answer you're looking for. > Well, it's the start of an answer. Sure, I know about ODBMSs, they're one solution to this problem. I can copy my data structure (objects, in the case of Squeak/Magma) to the database, then read it back later. But how does that actually work? Since the database is running in a separate process, you have to serialize the objects somehow, and transmit them somehow, and when they're read back into a different image, you have to detect and accommodate missing classes and shape changes and so on. I'm sure you've implemented all this in a way that's sensible for the intended uses of Magma. If we compare your solution to say, Gemstone, I'm sure you can talk about this feature or that tradeoff that's different between the two implementations. My question is, despite those differences, are the two solutions essentially the same, or are the differences fundamental? How about if we compare with Croquet? Or CORBA? Or JSON? I'm hoping this is something that people study, like oh, graph theory or category theory or parsing. Thanks! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151229/6d889ad7/attachment.htm From colin at wiresong.com Wed Dec 30 06:50:49 2015 From: colin at wiresong.com (Colin Putney) Date: Wed Dec 30 06:50:51 2015 Subject: [squeak-dev] [OT] What is this called? In-Reply-To: <2e3801d14280$37c9e380$a75daa80$@usmedrec.com> References: <2e3801d14280$37c9e380$a75daa80$@usmedrec.com> Message-ID: On Tue, Dec 29, 2015 at 1:31 PM, Ron Teitelbaum wrote: > Hi Colin, > > > > I think you have mentioned a number of parts of the problem already. > Schema Management, Address Space Management, and Version Management. > That sounds interesting. Where do these terms come from? Colin -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151229/2629354d/attachment.htm From pbpublist at gmail.com Wed Dec 30 07:05:38 2015 From: pbpublist at gmail.com (Phil (list)) Date: Wed Dec 30 07:05:43 2015 Subject: [squeak-dev] [OT] What is this called? In-Reply-To: References: Message-ID: <1451459138.5899.15.camel@gmail.com> On Tue, 2015-12-29 at 12:46 -0800, Colin Putney wrote: > > So what is this problem called? What theory describes the possible > solutions? Are there classes of solutions that have similar trade- > offs? Where can I learn more? > Serialization/marshalling??https://en.wikipedia.org/wiki/Serialization I think of it as serialization if the object is 'dead' when it reaches its destination, marshalling if I need it 'alive' From colin at wiresong.com Wed Dec 30 07:07:38 2015 From: colin at wiresong.com (Colin Putney) Date: Wed Dec 30 07:07:42 2015 Subject: [squeak-dev] [OT] What is this called? In-Reply-To: References: Message-ID: On Tue, Dec 29, 2015 at 3:27 PM, Huw Lloyd wrote: > To the degree that your conception is object-oriented, in the deeper > sense, it is not a problem. Your object(s) know how to persist themselves > and may be considered to be extensive over that persistence -- it is merely > that their morphology has changed. > Sure, but that's just hiding the issue behind an abstraction. I'm interested in the mechanics of the change in morphology. If the objects can persist themselves, great, but what technique do they use? Are you trying to identify names in a known problem space (e.g. a space > defined by the limits of the techniques used) or to identify the problem > space that you're in? For the latter, if you're going to generalise then > perhaps you should also wave the specialisation of 'perfect' reproduction. > So, even more generally, what you seem to describing is a special case of > morphogenesis in which a 'copy' is made, e.g. lazy evaluation and cached > computation can be thought of as slices in that process. > I think I'm trying to identify the problem space. "Morphogenesis" looks like an interesting term. Lazy evaluation and cached computation seem like opposites sides of the same coin. Do you mean that they are another form of "copying" that might be considered? I suppose that opens up new methods for performing the copy - eg, if the data-structure we're copying is a cached computation, instead of transmitting a description of the data structure, we might transmit the computation that produced it. Of course, then we still have the problem of how to accomplish *that*. Colin -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151229/c2ea92b5/attachment.htm From colin at wiresong.com Wed Dec 30 07:10:49 2015 From: colin at wiresong.com (Colin Putney) Date: Wed Dec 30 07:10:52 2015 Subject: [squeak-dev] [OT] What is this called? In-Reply-To: <1451459138.5899.15.camel@gmail.com> References: <1451459138.5899.15.camel@gmail.com> Message-ID: On Tue, Dec 29, 2015 at 11:05 PM, Phil (list) wrote: > Serialization/marshalling? https://en.wikipedia.org/wiki/Serialization > > I think of it as serialization if the object is 'dead' when it reaches > its destination, marshalling if I need it 'alive' > Interesting, thanks. What's the difference between dead and alive? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151229/a030ee29/attachment.htm From pbpublist at gmail.com Wed Dec 30 07:21:40 2015 From: pbpublist at gmail.com (Phil (list)) Date: Wed Dec 30 07:21:49 2015 Subject: [squeak-dev] [OT] What is this called? In-Reply-To: References: <1451459138.5899.15.camel@gmail.com> Message-ID: <1451460100.5899.20.camel@gmail.com> On Tue, 2015-12-29 at 23:10 -0800, Colin Putney wrote: > > > On Tue, Dec 29, 2015 at 11:05 PM, Phil (list) > wrote: > ? > > Serialization/marshalling??https://en.wikipedia.org/wiki/Serializat > > ion > > > > I think of it as serialization if the object is 'dead' when it > > reaches > > its destination, marshalling if I need it 'alive' > Interesting, thanks. What's the difference between dead and alive? > > ? Dead meaning you just want to persist/transfer the object state (i.e. write to disk etc.), alive meaning you want an actual instance with the specified object state to exist when it reaches its destination (i.e. recreate the object at the destination whether it be in another instance on the same machine, across the network, or even in an entirely different language/runtime environment) From huw.softdesigns at gmail.com Wed Dec 30 09:53:17 2015 From: huw.softdesigns at gmail.com (Huw Lloyd) Date: Wed Dec 30 09:53:23 2015 Subject: [squeak-dev] [OT] What is this called? In-Reply-To: References: Message-ID: On 30 December 2015 at 07:07, Colin Putney wrote: > > > On Tue, Dec 29, 2015 at 3:27 PM, Huw Lloyd > wrote: > > >> To the degree that your conception is object-oriented, in the deeper >> sense, it is not a problem. Your object(s) know how to persist themselves >> and may be considered to be extensive over that persistence -- it is merely >> that their morphology has changed. >> > > Sure, but that's just hiding the issue behind an abstraction. I'm > interested in the mechanics of the change in morphology. If the objects can > persist themselves, great, but what technique do they use? > I thought that handling a polymorphic structure was half of your problem, Colin? This is one of those cases where this mechanism can be delegated to each successive object. > > Are you trying to identify names in a known problem space (e.g. a space >> defined by the limits of the techniques used) or to identify the problem >> space that you're in? For the latter, if you're going to generalise then >> perhaps you should also wave the specialisation of 'perfect' reproduction. >> So, even more generally, what you seem to describing is a special case of >> morphogenesis in which a 'copy' is made, e.g. lazy evaluation and cached >> computation can be thought of as slices in that process. >> > > I think I'm trying to identify the problem space. "Morphogenesis" looks > like an interesting term. Lazy evaluation and cached computation seem like > opposites sides of the same coin. Do you mean that they are another form of > "copying" that might be considered? I suppose that opens up new methods for > performing the copy - eg, if the data-structure we're copying is a cached > computation, instead of transmitting a description of the data structure, > we might transmit the computation that produced it > Yes, or the 'transmit' might be pure calculation. Really, the storing and sending of values is a special case (short-cut) of a process. See SICP on streams: https://www.youtube.com/watch?v=_0i4NJYP9gM > . Of course, then we still have the problem of how to accomplish *that*. > Quite so, but that is something that can be localised too, if it was desirable. There's no mechanistic reason why it has to be the same across objects. I vaguely recall that William Kent's "Data and Reality" has some of these ideas in the form of entity - object relations which might help you too. Best, Huw > Colin > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151230/abc49f63/attachment-0001.htm From ron at usmedrec.com Wed Dec 30 16:11:41 2015 From: ron at usmedrec.com (Ron Teitelbaum) Date: Wed Dec 30 16:11:40 2015 Subject: [squeak-dev] [OT] What is this called? In-Reply-To: References: <2e3801d14280$37c9e380$a75daa80$@usmedrec.com> Message-ID: <300601d1431c$c5519b20$4ff4d160$@usmedrec.com> On Tue, Dec 29, 2015 at 1:31 PM, Ron Teitelbaum wrote: Hi Colin, I think you have mentioned a number of parts of the problem already. Schema Management, Address Space Management, and Version Management. That sounds interesting. Where do these terms come from? >From me but I didn?t make them up. I?ve worked with a number of databases. When I started I wrote my own data processing programs in a spreadsheet environment called Symphony. This was just data and functions to manipulate data. Later I worked with dBase and Paradox. These added the concept of a defined Schema Management and indexes. I then moved to Oracle that added the concept of versions, locking and logging, (the locking and logging part is Version Management (adding lock then commit or rollback). Versant the object database gave me the concepts of Address Spaces and finally DHT added the details behind what Address Spaces could do. When dealing with Schema Management I developed a the method to manage upgrading a database where I added a db upgrade table. It contained the information needed to move the database forward or backwards. Basically I added a version number and a row for each table that changed, the method to change the schema, and the transforms needed to move or convert the data. Clients would connect and verify the schema version from that table and could trigger an upgrade with the proper permissions or error if the version didn?t match. The table is a way to automate what would have been a manual process (run this query to update the table definition, then run these queries to transform the data, now call yourself this version). We also added the reverse to go back a version. This was really just a failsafe to allow you to rollback and recover from an upgrade error. Having to do both caused some interesting discussions. Data that could have been lost, which was no longer needed or collected in a newer version of the system could cause issues when rolling back, this would usually require developers to add default data, ?no data collected?, or something of that type. It made the process more thoughtful and helped to protect the system. The idea of a more detailed Version Management that gives you point in time is really just a DB design issue. Linked versions of objects can be traversed for versions and index snapshots of a point in time are easy to design in if the data is important. Thinking in terms of data is something that happens pretty naturally especially if you started working with data. I had a friend that always said that no good programmers started programming after 1985. The reason for that was that before 1985 you had to work with data structures and you learned them. I?m sure that is not entirely true but I am glad that I?ve been so exposed to data and can think in terms of data and it?s structure. I developed a system that did some amazing data transformations (reading what a physician wrote and converting into what a pharmacist would dispense). This sounds like an easy problem converting dose to strength but it was a very complicated problem. Think in terms of Brand, Generic, Formularies and Inventory, drug-drug interactions and other health checks, the number of different sizes of pills and forms (tablet, gel caps), then add on automated packaging, billing, tracking and medical records. It was quite a system. When I was designing it, I kept getting the question when are you going to show us a screen for how this will work. I said not until I have a data structure that can handle it. I worked on the underlying data structure, then the programming to support it. People were getting quite upset with me since you could not see anything that worked. I knew it worked and could show them in code but that didn?t seem to be good enough. We added the UI at the end and it took only a few days. Had we started with the UI and gone from that to the database we may never have been able to solve the problem. Real systems require detailed data structures all of these methods for managing them are part of that. Does that help? All the best, Ron Teitelbaum Head Of Engineering 3D Immersive Collaboration Consulting ron@3Dicc.com Follow Me On Twitter: @RonTeitelbaum www.3Dicc.com https://www.google.com/+3Dicc Colin -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151230/f8a106de/attachment.htm From commits at source.squeak.org Wed Dec 30 16:27:47 2015 From: commits at source.squeak.org (commits@source.squeak.org) Date: Wed Dec 30 16:27:50 2015 Subject: [squeak-dev] The Inbox: EToys-kfr.132.mcz Message-ID: A new version of EToys was added to project The Inbox: http://source.squeak.org/inbox/EToys-kfr.132.mcz ==================== Summary ==================== Name: EToys-kfr.132 Author: kfr Time: 30 December 2015, 5:27:24.767 pm UUID: 1f9e02c3-cc2a-48c5-9853-66ab3260436f Ancestors: EToys-cmm.131 Added some methods so UnscriptedPlayer and UnscriptedCardPlayer work with the ObjectTool =============== Diff against EToys-cmm.131 =============== Item was added: + ----- Method: UnscriptedCardPlayer class>>newVariableDocks: (in category 'as yet unclassified') ----- + newVariableDocks: dockList! Item was added: + ----- Method: UnscriptedCardPlayer class>>setNewInstVarNames: (in category 'as yet unclassified') ----- + setNewInstVarNames: listOfStrings! Item was changed: Player subclass: #UnscriptedPlayer + instanceVariableNames: 'patch' - instanceVariableNames: '' classVariableNames: '' poolDictionaries: '' category: 'Etoys-Scripting'! UnscriptedPlayer class instanceVariableNames: 'ephemeralPlayerRef'! !UnscriptedPlayer commentStamp: '' prior: 0! My instances are Player objects that have not been scripted, and which hence do not require a unique scripts dictionary, etc. As soon as the needed, I am transformed automatically into a unique subclass of Player.! UnscriptedPlayer class instanceVariableNames: 'ephemeralPlayerRef'! Item was added: + ----- Method: UnscriptedPlayer>>getPatch (in category 'access') ----- + getPatch + ^ patch! Item was added: + ----- Method: UnscriptedPlayer>>setPatch: (in category 'access') ----- + setPatch: t1 + patch := t1! From asqueaker at gmail.com Thu Dec 31 16:19:03 2015 From: asqueaker at gmail.com (Chris Muller) Date: Thu Dec 31 16:19:46 2015 Subject: [squeak-dev] Re: [Vm-dev] Memory leaks In-Reply-To: <1451551006.5899.45.camel@gmail.com> References: <1451551006.5899.45.camel@gmail.com> Message-ID: Go to the "Help" | "About This System" and then select "Space Analysis". On Thu, Dec 31, 2015 at 2:36 AM, Phil (list) wrote: > > I was curious if the VM has any facilities to assist in tracking down > memory leaks? (not in the VM itself, but rather in the image it is > running) The scenario that comes to mind are object reference loops > though there are likely others to consider as well. Any pointers are > appreciated. > > Thanks, > Phil From pbpublist at gmail.com Thu Dec 31 20:44:20 2015 From: pbpublist at gmail.com (Phil (list)) Date: Thu Dec 31 20:44:25 2015 Subject: [squeak-dev] Re: [Vm-dev] Memory leaks In-Reply-To: References: <1451551006.5899.45.camel@gmail.com> Message-ID: <1451594660.2513.42.camel@gmail.com> Chris, On Thu, 2015-12-31 at 10:19 -0600, Chris Muller wrote: > Go to the "Help" | "About This System" and then select "Space > Analysis". > Hmm... this looks handy (specifically the space used #). ?I'll have to look at how this is being calculated (I'm running a Cuis image so all I have right now are instance counts) Thanks very much, Phil > On Thu, Dec 31, 2015 at 2:36 AM, Phil (list) > wrote: > > > > I was curious if the VM has any facilities to assist in tracking > > down > > memory leaks? (not in the VM itself, but rather in the image it is > > running)??The scenario that comes to mind are object reference > > loops > > though there are likely others to consider as well.??Any pointers > > are > > appreciated. > > > > Thanks, > > Phil From karlramberg at gmail.com Thu Dec 31 20:53:52 2015 From: karlramberg at gmail.com (karl ramberg) Date: Thu Dec 31 20:53:54 2015 Subject: [squeak-dev] Delayed preference color updating Message-ID: I have a issue with non modal color changing. The NewColorPicker is not modal and the method returns a value before the user click 'close' button in NewColorPicker. How can I change behavior to update color when the button is clicked? PBColorPreferenceView>>changeColor: aButton aButton changeColor. "returns the current color when not modal" self preference preferenceValue: aButton fillStyle. button label: self preference preferenceValue asString. self adjustLabelColor Best, Karl -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20151231/127eb7c4/attachment.htm From pbpublist at gmail.com Thu Dec 31 21:36:00 2015 From: pbpublist at gmail.com (Phil (list)) Date: Thu Dec 31 21:36:05 2015 Subject: [squeak-dev] Re: [Vm-dev] Memory leaks In-Reply-To: <1451594660.2513.42.camel@gmail.com> References: <1451551006.5899.45.camel@gmail.com> <1451594660.2513.42.camel@gmail.com> Message-ID: <1451597760.2513.60.camel@gmail.com> On Thu, 2015-12-31 at 15:44 -0500, Phil (list) wrote: > Chris, > > On Thu, 2015-12-31 at 10:19 -0600, Chris Muller wrote: > > Go to the "Help" | "About This System" and then select "Space > > Analysis". > > > > Hmm... this looks handy (specifically the space used #). ?I'll have > to > look at how this is being calculated (I'm running a Cuis image so all > I > have right now are instance counts) Heh... turns out Cuis already has SpaceTally. ?Good on Juan for having it, my mistake for not noticing it... From commits at source.squeak.org Thu Dec 31 22:05:31 2015 From: commits at source.squeak.org (commits@source.squeak.org) Date: Thu Dec 31 22:05:32 2015 Subject: [squeak-dev] The Inbox: Collections-cmm.673.mcz Message-ID: Chris Muller uploaded a new version of Collections to project The Inbox: http://source.squeak.org/inbox/Collections-cmm.673.mcz ==================== Summary ==================== Name: Collections-cmm.673 Author: cmm Time: 31 December 2015, 4:05:21.536 pm UUID: f9f9498e-b548-443b-87fe-af21ba6e8f52 Ancestors: Collections-eem.672 Do not let FinalizationDependents become a strongly-referencing Array, and let it grow exponentially instead of linearly, because that is the best general growth strategy when one doesn't know how many will need to be created. =============== Diff against Collections-eem.672 =============== Item was changed: ----- Method: WeakArray class>>addWeakDependent: (in category 'accessing') ----- addWeakDependent: anObject FinalizationLock critical: [ | emptySlotIndex | emptySlotIndex := FinalizationDependents identityIndexOf: nil ifAbsent: [ | newIndex | newIndex := FinalizationDependents size + 1. + FinalizationDependents := (FinalizationDependents grownBy: FinalizationDependents size) as: WeakArray. - "Grow linearly" - FinalizationDependents := FinalizationDependents grownBy: 10. newIndex ]. FinalizationDependents at: emptySlotIndex put: anObject ] ifError: [ :msg :rcvr | rcvr error: msg ]! From commits at source.squeak.org Thu Dec 31 23:14:02 2015 From: commits at source.squeak.org (commits@source.squeak.org) Date: Thu Dec 31 23:14:04 2015 Subject: [squeak-dev] The Inbox: Compression-jdr.48.mcz Message-ID: A new version of Compression was added to project The Inbox: http://source.squeak.org/inbox/Compression-jdr.48.mcz ==================== Summary ==================== Name: Compression-jdr.48 Author: jdr Time: 31 December 2015, 6:13:59.974 pm UUID: 7798d7c6-8f39-45cc-a3cc-e0887130a62d Ancestors: Compression-dtl.47 It solves a problem compressing a zipfile with many files (> 200), previously all files to be processed remained open, now only the file being processed is opened. =============== Diff against Compression-dtl.47 =============== Item was changed: ----- Method: ZipArchive>>writeTo: (in category 'writing') ----- writeTo: stream stream binary. members do: [ :member | member writeTo: stream. member endRead. + member close ]. writeCentralDirectoryOffset := stream position. self writeCentralDirectoryTo: stream. ! Item was changed: ----- Method: ZipArchive>>writeTo:prepending: (in category 'writing') ----- writeTo: stream prepending: aString stream binary. stream nextPutAll: aString. members do: [ :member | member writeTo: stream. member endRead. + member close ]. writeCentralDirectoryOffset := stream position. self writeCentralDirectoryTo: stream. ! Item was changed: ----- Method: ZipNewFileMember>>from: (in category 'initialization') ----- from: aFileName | entry | compressionMethod := CompressionStored. "Now get the size, attributes, and timestamps, and see if the file exists" stream := StandardFileStream readOnlyFileNamed: aFileName. self localFileName: (externalFileName := stream name). entry := stream directoryEntry. compressedSize := uncompressedSize := entry fileSize. desiredCompressionMethod := compressedSize > 0 ifTrue: [ CompressionDeflated ] ifFalse: [ CompressionStored ]. + self setLastModFileDateTimeFrom: entry modificationTime. + stream close - self setLastModFileDateTimeFrom: entry modificationTime ! Item was changed: ----- Method: ZipNewFileMember>>rewindData (in category 'private-writing') ----- rewindData super rewindData. + stream ensureOpen. readDataRemaining := stream size. stream position: 0.!