<div dir="ltr">Hi all, <div>Esteban, </div><div> one thing we can do is to add a job to the Pharo CI to try to perform the bootstrap and testing of it using the latest VM.</div><div>This of course will no replace manual testing but it can give us the result of running a quite complex process using the latest VM. </div><div>Even more, the last problems with stability were detected when trying to execute a complete bootstrap.</div><div><br></div><div>Today I will make a pull request (to never be integrated) using the latest VM instead of the stable one.</div><div>Later we can migrate it to be an independent job, that is always there to validate the new latest VM.</div><div>Moreover, if the VM is published using a kind of semantic versioning, we can refer to an specific VM in the bootstrap process.</div><div>This will allow us to check new VMs just with a PR (taking advance of all the automatic process that now is working) and also to reproduce the building using the exact VM used during the generation of an image, this is a weak point we have today, where we all always using Stable, and not a defined version.</div><div><br></div><div><br></div><div>Cheers,</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Sat, Mar 31, 2018 at 3:03 PM, Esteban Lorenzano <span dir="ltr"><<a href="mailto:estebanlm@gmail.com" target="_blank">estebanlm@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> <br><div style="word-wrap:break-word;line-break:after-white-space"><br><div><br><blockquote type="cite"><div>On 30 Mar 2018, at 23:56, Nicolas Cellier <<a href="mailto:nicolas.cellier.aka.nice@gmail.com" target="_blank">nicolas.cellier.aka.nice@<wbr>gmail.com</a>> wrote:</div><br class="m_129758517362692669Apple-interchange-newline"><div><div dir="ltr"><div><div><div><div><br><div class="gmail_extra"><br><div class="gmail_quote">2018-03-24 11:11 GMT+01:00 Esteban Lorenzano <span dir="ltr"><<a href="mailto:estebanlm@gmail.com" target="_blank">estebanlm@gmail.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br>
hi,<br>
<br>
> On 24 Mar 2018, at 09:50, Cyril Ferlicot D. <<a href="mailto:cyril.ferlicot@gmail.com" target="_blank">cyril.ferlicot@gmail.com</a>> wrote:<br>
><br>
> Le 23/03/2018 à 21:52, Eliot Miranda a écrit :<br>
>> Hi Damien,<br>
>><br>
>> Indeed the image is corrupt at start-up. See below.<br>
>><br>
>><br>
>> Right. This VM is prior to the bug fixes in VMMaker.oscog-eem.2320:<br>
>><br>
>> Spur:<br>
>> Fix a bad bug in SpurPlnningCompactor.<br>
>> unmarkObjectsFromFirstFreeObje<wbr>ct, used when the compactor requires more<br>
>> than one pass due to insufficient savedFirstFieldsSpace, expects the<br>
>> corpse of a moved object to be unmarked, but<br>
>> copyAndUnmarkObject:to:bytes:f<wbr>irstField: only unmarked the target.<br>
>> Unmarking the corpse before the copy unmarks both. This fixes a crash<br>
>> with ReleaseBuilder class>>saveAsNewRelease when non-use of cacheDuring:<br>
>> creates lots of files, enough to push the system into the multi-pass regime.<br>
>><br>
>><br>
>> Pharo urgently needs to upgrade the VM to one more up to date than 2017<br>
>> 08 27 (in fact more up-to-date than opensmalltalk/vm commit<br>
>> 0fe1e1ea108e53501a0e7287360480<wbr>62c83a66ce, Fri Jan 19 13:17:57 2018<br>
>> -0800). The bug that VMMaker.oscog-eem.2320 fixes can result in image<br>
>> corruption in large images, and can occur (as it has here) at start-up,<br>
>> causing one's work to be irretrievably lost.<br>
>><br>
><br>
> Hi Eliot,<br>
><br>
> I think that there is a lot of people who would like to get a newer<br>
> stable vm for Pharo 6.1 and 7. The problem is that it is hard to know<br>
> which VM are stable enough to be promoted as stable.<br>
><br>
> Some weeks ago Esteban tried to promote a VM as stable and he had to<br>
> revert it the same day because a regression occurred in the VM.<br>
><br>
> If you're able to tell us which vms are stable in those present at<br>
> <a href="http://files.pharo.org/vm/pharo-spur32/" rel="noreferrer" target="_blank">http://files.pharo.org/vm/phar<wbr>o-spur32/</a> and<br>
> <a href="http://files.pharo.org/vm/pharo-spur64/" rel="noreferrer" target="_blank">http://files.pharo.org/vm/phar<wbr>o-spur64/</a> it would be a great help.<br>
><br>
> Even better would be for the pharo community to have a way to know which<br>
> vms are stable or not without having to ask you.<br>
<br>
there is no “stable” branch in Cog, and that’s a problem.<br>
“released” versions (the version you can find as stable) are not working for Pharo :(<br>
<br>
I tried to promote versions from end feb and that crashed.<br>
<br>
next week I will try again, maybe now they are stable enough… one thing is true: the versions that we consider stable (from oct/17) present problems that are already solved on latest.<br>
<br>
Esteban<br>
<br>
<br>
><br>
> Have a nice day.<br>
><br>
<span class="m_129758517362692669m_2136362129077557755HOEnZb"><font color="#888888">>><br>
>> --<br>
>> _,,,^..^,,,_<br>
>> best, Eliot<br>
> --<br>
> Cyril Ferlicot<br>
> <a href="https://ferlicot.fr/" rel="noreferrer" target="_blank">https://ferlicot.fr</a><br>
><br>
<br>
</font></span></blockquote></div><br></div><div class="gmail_extra">Hi,<br></div><div class="gmail_extra">Several problems are mixed here, let's try and decouple:<br></div><div class="gmail_extra">- 1) there are ongoing development in the core of VM that may introduce some instability<br></div><div class="gmail_extra">- 2) there are ongoing development in some plugins also<br></div><div class="gmail_extra">- 3) there are infrastructure problems preventing to produce artifacts whatever the intrinsic stability of the VM<br></div></div></div></div></div></div></div></blockquote><div><br></div><div>you are right in all points, but for me this is a problem of process. </div><div><br></div><div>- we have no defined milestones so nobody knows if they can jump to help.</div><div>- plugin development happens “by his own” and nobody knows what happens, why happens and how it happens.</div><div>- infrastructure is not bad and a lot of efforts has been made to make it work. But code sources are scattered around the world and the only thing that reunites them is the hand of the one who generates the C sources.</div><div><br></div><div>IMHO, is this “disconnection” what causes most of the problems. </div><div><br></div><div>cheers,</div><div>Esteban</div><br><blockquote type="cite"><div><div dir="ltr"><div><div><div><div><div class="gmail_extra"><br></div><div class="gmail_extra">For 1) development happens in VMMaker, and we have to be relying on experts. Today that is Eliot and Clement.<br></div><div class="gmail_extra">We all want 64bits VM, improved GC, improved become:, write barrier, ephemerons, threaded FFI calls and adaptive optimization.<br></div><div class="gmail_extra">Pharo is relying on these progress, they are vital.<br></div><div class="gmail_extra">IMO, we are reaching a good level of confidence, and I hope to see some VMMaker version blessed as stable pretty soon.<br></div><div class="gmail_extra"><br></div><div class="gmail_extra">Instead of whining, the best we can do for reaching this state is help them by providing accurate bug reports and even better reproducible cases.<br></div><div class="gmail_extra">Thanks to all who are working in this direction.<br></div><div class="gmail_extra"><br></div><div class="gmail_extra">For 2) we had a few problems, but again this is for improving important features (SSL...)<br></div><div class="gmail_extra">Much of the development happens in feature branches already.<br></div><div class="gmail_extra">But since we are targetting so many platforms, and don't have automated tests that scale yet, we still need beta testers.<br></div><div class="gmail_extra">We can discuss about the introduction of such beta features wrt release cycles, that will be a good thing.<br></div><div class="gmail_extra">Ideally we should tend toward continuous integration and have very short cycles, but we're not yet there.<br></div><div class="gmail_extra"><br>For 3) we had a lot of problems, like staled links, invalid credentials, evolution of the version of tools at automated build site, etc...<br><br></div><div class="gmail_extra">If we don't build the artifacts, then we can't even have a chance to test the stability of 1) and 2)<br>We have to understand that 3) is absolutely vital.<br><br>May I remind that for a very long period last year, the build were broken due to lack of work at Pharo side.<br>Fortunately, this has changed in 2018.<br>Fabio has been working REALLY hard to improve 3), and without the help of Esteban,I don't think he could have reached the holy green build status.<div class="gmail_extra">We will never thank them enough for that. This also shows that cooperation may pay.<br></div><br>But this is still very fragile.</div>If we want to make progress, we should ask why it is so.<br></div>We could analyze the regressions, and decide if the complexity is sustainable, or eventually drop some drag.<br></div>We are chasing many hares by building the VM for Newspeak/Pharo/Squeak i386/x86_64/ARM Spur/Stack/V3 Sista/lowcode linux/Macosx/Windows ...<br></div>If it happens that a fix vital for Pharo/Squeak does break Newspeak tests, then it slows down the progress...<br></div>Maybe we would want to decouple a bit more the problems there too (they may come from some image side weakness).<br><div><div><div><div><div><div><div class="gmail_extra"><br></div><div class="gmail_extra">Last two years I've also observed some work exclusively done in the Pharo fork of the opensmalltalk VM.<br></div><div class="gmail_extra">This was counter productive. Work must be produced upstream, or it's wasted.<br></div></div></div></div></div></div></div></div></div></blockquote><div><br></div><div>This happened just once or twice. And it was because people were ignorant of “joint" so they continued contributing as before (and people were pointed to right place when we had the opportunity).</div><div>And I disagree this was counterproductive because I took the effort to merge the changes into osvm. This worked fine until I stopped to do that job, but well… just one PR got stalled there for months and Alistair integrated it recently. </div><div><br></div><div>What *did happen* and I’m still not ready to let it go is a lot of the small changes that we presented to be rejected (or ignored) without further consideration. But well, let’s keep it positive and not enter to sterile discussions, I just think you are wrong with this argument.</div><div><br></div><div>cheers,</div><div>Esteban</div><br><blockquote type="cite"><div><div dir="ltr"><div><div><div><div><div><div><div class="gmail_extra">I once thought that the Pharo fork could be the place for the pharo team to manage official stable versions.<br></div><div class="gmail_extra">But I agree that this is too much duplicated work and would be very happy to see the work happen upstream too.<br><br></div><div class="gmail_extra">If you have constructive ideas that will help decoupling all these problems, we are all ear.<br><br></div><div class="gmail_extra">PS: i did not post this answer for avoiding sterile discussion, but since Phil asked...<br></div><div class="gmail_extra"><br></div></div></div></div></div></div></div></div>
</div></blockquote></div><br></div><br></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature" data-smartmail="gmail_signature">Pablo Tesone.<br><a href="mailto:tesonep@gmail.com" target="_blank">tesonep@gmail.com</a></div>
</div>