[squeak-dev] Metacello Tonel Support Proposal

John Pfersich smalltalker2 at mac.com
Fri May 22 05:59:38 UTC 2020


This post would make a great addition to the Squeak Wiki. Just sayin’.

/—————————————————————/
For encrypted mail use jgpfersich at protonmail.com - Free account at ProtonMail.com
Web: https://objectnets.net and https://objectnets.org
https://datascilv.com https://datascilv.org


> On May 20, 2020, at 05:24, Beckmann, Tom <Tom.Beckmann at student.hpi.uni-potsdam.de> wrote:
> 
> Hi everyone.
> 
> We are currently working on improving the support for different code-serialization formats in Squeak. This includes Tonel [2], which is used by developers who are working on their projects in the Pharo environment and storing code artifacts on GitHub such as PolyMath [6]. While those projects can be virtually compatible with Squeak, support for the Tonel format is required to actually load the source code -- be it via Monticello, Metacello, or another front-end.
> 
> We wrote briefly about our intentions in the rather long discussion about performance on the vm-dev mailing list [4]. You can find the pull request for the necessary changes to Metacello on GitHub [5]. A version of Tonel with Squeak-specific compatibility changes is already available [2].
> 
> Please try it out. You can support our efforts by helping us find bugs. 
> 
> We encourage all of you to load your favorite projects not only in a fresh Trunk image [7] but also in the image you are using anyway -- after making a backup copy :). Please report debuggers, performance drops, or other issues. Of course, do also mention any ideas you have on the topic.
> 
> Here is how to test your projects: (1) install our pull request for Metacello and (2) load your project through Metacello.
> 
> **Step 1 - Install Metacello from our pull request**
> 
> *Note that, in a fresh Squeak image, `Metacello new` will first install the official Metacello version. After that, the following code snippet will update that version with our pull request. That’s why you need an extra `get` and `load.`.*
> 
> ```Smalltalk
> Metacello new
>    baseline: 'Metacello';
>    repository: 'github://tom95/metacello:squeak-download-repo-split/repository';
>    get; load: #default.
> ```
> 
> **Step 2 - Load your project via Metacello**
> 
> We can load PolyMath [6] in the following example. Note, however, that not all PolyMath classes can be loaded in Squeak because of several missing classes, which are specific to Pharo’s standard library. Yet, loading starts fine, which indicates a working support of the Tonel format in Squeak. Again, don’t forget to backup your production images.
> 
> *Note that loading an already installed project will not trigger the new code paths, we added to Metacello. So, remove those projects from the Metacello registry first or use a fresh Squeak image instead.*
> 
> ```Smalltalk
> Metacello new
>    repository: 'github://PolyMathOrg/PolyMath/src';
>    baseline: 'PolyMath';
>    load.
> ```
> 
> **Further advice**
> 
> When you try to load a repository that uses Tonel for the first time, the installer will ask you whether to install Tonel support for Squeak. If you deny this request, you will have to do it three times because of Metacello's internal retry logic.
> 
> **Emergency brake:** Should anything out of the ordinary happen, you might easily get back to Metacello’s current loading logic by simply removing the checks for `type = ’github’` and `type=’bitbucket’` at the end of `MetacelloSqueakPlatform>>#createRepository:`.
> 
> We -- Marcel (mt), Fabio (fn), Jakob (jr), and Tom (tobe) -- thank you for your support.
> 
> Happy testing! :)
> 
> -----------------------
> 
> **More Bits on Tonel**
> 
> Pharo switched to Tonel as their default format a while ago, making it more challenging to maintain cross-dialect support for repositories. There are some projects that strive for compatibility -- such as [1] --, but that entails an extra maintenance effort. Only brave people are willing to go that extra mile from what we've seen.
> 
> Tonel is responsible for writing Smalltalk source code into the file system. Entire classes are stored in files, rather than methods as in filetree [8], which also improves performance when storing or loading code artifacts. Considering Git support, metadata (e.g. method timestamps) is not part of the serialization format, which fosters clean merge requests and a reduction of noise throughout the (external/git) commit history. Consequently, Tonel puts less pressure on the operating system’s file system, which can especially be beneficial on Windows-based platforms.
> 
> However, since Tonel-based project repositories are omitting Squeak’s method timestamps, existing code-browsing tools would have to be extended to fetch that metadata from the particular Git repository. Otherwise, both authorship and time of last edit wouldn’t be available in Squeak, even though they would be updated through a Git commit/push process -- that is, if Tonel is used in combination with Git, which is not mandatory, considering Monticello’s (not Metacello’s) connection to SqueakSource and the local file system. So, Tonel poses a tricky compatibility challenge on Squeak’s tools. We plan to add computation of timestamps based on the git history to Squot, our Git infrastructure, in the foreseeable future (which not only benefits Tonel, but should also allow us to recover the full version history in metadataless filetree repositories).
> 
> A second drawback of Tonel might be limited support for custom parsers, which has been mentioned as a valuable use case by  the Squeak community, if I understand correctly [9].
> 
> Our proposed implementation [2] employs Metacello’s platform-specific package concept. While the core of Metacello is shared across Smalltalk dialects, there are packages for Squeak or Pharo. One of the key aspects for each dialect is code loading. We decouple **transport** (e.g., downloading from a git repository) from **deserialization** (e.g., parsing files from Tonel or FileTree and creating Monticello’s MCDefinitions). For this, we propose the `MCFilesystemFetchOnlyRepository`, which couples common fetching logic, as a wrapper around `MCFiletreeRepository` and `TonelRepository`, which specialize parsing respectively.
> 
> In the long term, we want to enable and support Squeak compatibility upstream. In the spirit of common open source practice, we therefore forked the original Tonel repository as squeak-tonel [2] (as well as its dependency on Filesystem [10]). Having the pull request for Metacello as described above, this enables us to quickly address compatibility issues in Squeak. However, our goal is to push our contributions upstream. Consequently, any compatibility fixes will be opened as pull requests for review.
> 
> Lastly, note that our efforts to support Tonel in Squeak do also entail the support of the Smalltalk Object Notation (STON) for Squeak [3]. STON is vital to identify whether a repository is using Tonel or not when loading projects through Metacello. Any heuristic approach would likely be too error-prone considering Metacello’s expected reliability. Therefore, we added STON as a *mandatory dependency* to Metacello in our pull request. Like squeak-tonel, we want to enable and support Squeak compatibility upstream for squeak-ston [3] in the long term.
> 
> [1] https://github.com/ronsaldo/abstract-gpu 
> [2] https://github.com/squeak-smalltalk/squeak-tonel 
> [3] https://github.com/squeak-smalltalk/squeak-ston
> [4] http://forum.world.st/Squeak-and-Pharo-speed-differences-td5116635.html
> [5] https://github.com/Metacello/metacello/pull/515
> [6] https://github.com/PolyMathOrg/PolyMath
> [7] http://files.squeak.org/6.0alpha/ 
> [8] https://github.com/dalehenrich/filetree 
> [9] http://forum.world.st/Squeak-and-Tonel-td5095284.html#a5095317 
> [10] https://github.com/squeak-smalltalk/squeak-filesystem
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20200521/5047754c/attachment.html>


More information about the Squeak-dev mailing list