<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class="">Hi, </div><div class=""><br class=""></div><div class="">I think one problem we have now is that we cannot know if the build will work or not, then sometimes we generate binaries (which are published on bintray and in the case of pharo on <a href="http://files.pharo.org" class="">files.pharo.org</a>) and we don’t know if they will work at all. </div><div class="">this is because development branch and stable branch are the same (which basically means we do not have a stable branch, heh). </div><div class=""><br class=""></div><div class="">let me explain how I would like to handle this instead: </div><div class=""><br class=""></div><div class="">1) We need a stable branch, let’s say is Cog</div><div class="">2) We also need a development branch, let’s call it CogDev</div><div class=""><br class=""></div><div class="">All commits and contributions should be in CogDev. Then when Eliot decides a build is good enough for people to use, he should merge changes to stable (Cog). </div><div class="">Before you think is too much work, is as easy as doing: </div><div class=""><br class=""></div><div class="">$ git checkout Cog</div><div class="">$ git merge CogDev</div><div class="">$ git push</div><div class="">$ git checkout CogDev</div><div class=""><br class=""></div><div class="">(which can be easily put in a script called “promote-stable” or whatever).</div><div class=""><br class=""></div><div class="">this way we have the benefits of both worlds. </div><div class=""><br class=""></div><div class="">In any case, in the “pharo side” of things, we keep a parallel CI process who does this: </div><div class=""><br class=""></div><div class="">1) there is a development branch that keep all sources we need: the osvm *and* all monticello packages (filetree format) acting as a mirror that merges changes and executes all building process (including generating sources) and testing (we run all pharo tests as a way to test the VM). This is done just for being sure the CI is green and no artifact is created. This is how I have early reported problem in VM generation before.</div><div class="">2) in parallel the build follows the “normal” process on opensmalltalk-vm and everything is build there. This generates a “latest” build which is published in pharo servers so people can test (but this binary could be wrong as is not something declared as “stable”</div><div class="">3) time to time, when Eliot says “this VM is good”, I manually promote the latest vm to stable vm. This is the step that I could avoid if we have two separated branchs :)</div><div class=""><br class=""></div><div class="">well, that’s my 2c</div><div class=""><br class=""></div><div class="">Esteban</div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><br class=""><div><blockquote type="cite" class=""><div class="">On 30 May 2017, at 05:43, Ben Coman <<a href="mailto:btc@openInWorld.com" class="">btc@openInWorld.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class="gmail_extra" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><br class="Apple-interchange-newline"><br class=""><div class="gmail_quote">On Tue, May 30, 2017 at 3:25 AM, Eliot Miranda<span class="Apple-converted-space"> </span><span dir="ltr" class=""><<a href="mailto:eliot.miranda@gmail.com" target="_blank" class="">eliot.miranda@gmail.com</a>></span><span class="Apple-converted-space"> </span>wrote:<br class=""><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex;"> <br class=""><div dir="ltr" class="">Hi Ben,<div class="gmail_extra"><br class=""><div class="gmail_quote">On Sun, May 28, 2017 at 6:46 AM, Ben Coman<span class="Apple-converted-space"> </span><span dir="ltr" class=""><<a href="mailto:btc@openinworld.com" target="_blank" class="">btc@openinworld.com</a>></span><span class="Apple-converted-space"> </span>wrote:<br class=""><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex;"> <br class=""><div dir="ltr" class=""><div class=""><br class=""></div>Maybe I'm speaking out of turn from the sidelines, but just curious to ask a leading question... <div class=""><br class=""></div><div class="">What is the purpose of the Continuous Integration system?</div></div></blockquote><div class=""><br class=""></div><div class="">One thing for me is to identify when I've made a mistake Yes it is selfish of me<span class="Apple-converted-space"> </span></div></div></div></div></blockquote><div class=""><br class=""></div><div class="">Thats fine :). We need you to keep to productive ;)</div><div class="">I'm trying to understand your perspective and needs to see if the workflow can be tuned without slowing you down too much. </div><div class=""><br class=""></div><div class=""> </div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex;"><div dir="ltr" class=""><div class="gmail_extra"><div class="gmail_quote"><div class="">to want to break a build, but in some way those of us who are active VM developers must be able to commit and see what fails, especially when we have a large platform and VM variant spread.</div></div></div></div></blockquote><div class=""><br class=""></div><div class="">Creating a PR automatically runs the CI tests. So you'll get that.</div><div class=""><br class=""></div><div class="">I'd propose your continuous work would occur in a single Cog-eem branch. Pulling from the server-Cog branch would be the same as your current workflow. Pushing to the server as a PR would automatically the CI tests and automatically integrate into the server-Cog branch upon success. There is nothing stopping your work continuing in your Cog-eem branch while the CI tests are running, just the same as you currently continue working in your local-Cog branch. </div><div class=""><br class=""></div><div class="">We can probably devise an efficient way for you to issue the pull request from the command line - something like...<br class=""></div><div class=""><a href="https://github.com/github/hub/issues/1219" class="">https://github.com/github/hub/issues/1219</a><br class=""></div><div class=""><a href="https://stackoverflow.com/questions/4037928/can-you-issue-pull-requests-from-the-command-line-on-github" class="">https://stackoverflow.com/questions/4037928/can-you-issue-pull-requests-from-the-command-line-on-github</a><br class=""></div><div class=""><br class=""></div><div class=""> <br class=""></div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex;"><div dir="ltr" class=""><div class="gmail_extra"><div class="gmail_quote"><div class=""><br class=""></div><div class="">IMO what needs to be added is a way of marking commits as good. I think we should continue to be allowed to break the build (something we do unintentionally). </div></div></div></div></blockquote><div class=""><br class=""></div><div class=""><div class="">In my proposal, you break builds in your own Cog-eem branch, and when the CI build succeeds its automatically integrated into the mainline server-Cog branch. If explicit no-ff merge points are maintained, then that may be sufficient to mark a good-build. </div></div><div class=""><br class=""></div><div class=""> </div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex;"><div dir="ltr" class=""><div class="gmail_extra"><div class="gmail_quote"><div class="">But we need a set of criteria to mark a build as fit for release and have a download URL from which we obtain the latest releasable VM. </div></div></div></div></blockquote><div class=""><br class=""></div><div class="">Marking a good-build as a release-build would be a further and separate step. <br class=""></div><div class="">btw, I thought "master" branch was going to track "release-builds" ? </div><div class=""> <br class=""></div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex;"><div dir="ltr" class=""><div class="gmail_extra"><div class="gmail_quote"><div class=""><br class=""></div><div class="">I agree with Fabio that if we want the CI server to be as green as possible marking builds which have been failing for a long time as "allowed to fail" is a good idea. But it also has to be brought to people's attention that there are such builds.</div></div></div></div></blockquote><div class=""><br class=""></div><div class="">I'm not sure the best way to do this. Perhaps upon "travis_sucess" would it be possible to scan the log to report allowed-failures of experimental builds?</div><div class=""><br class=""></div><div class=""> </div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex;"><div dir="ltr" class=""><div class="gmail_extra"><div class="gmail_quote"><div class=""><br class=""></div><div class="">As far as the Sista and Lowcode VMs go these are experimental, as would a threaded FFI VM, something we may have later this year. <span class="Apple-converted-space"> </span></div></div></div></div></blockquote><div class=""><br class=""></div><div class="">Maybe these should be separate feature-branches with their own Travis build, which first only tests the feature specific build for quick turn around, and then triggers the production builds for further information.</div><div class=""><br class=""></div><div class=""> <br class=""></div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex;"><div dir="ltr" class=""><div class="gmail_extra"><div class="gmail_quote"><div class="">It would be nice to segregate the standard VMs that are already in production from these experimental builds so that failures within the experimental set don't affect the status of the production VMs. I'd rather see that segregation than mark certain builds as allowed to fail.</div></div></div></div></blockquote><div class=""><br class=""></div><div class="">The segregation might be done as a dependent build, so it is only started if the production build succeeds. <br class=""></div><div class="">But I would guess the later is quicker to implement, so can we do that first? </div><div class="">Lets get the builds green asap, then move from there.</div><div class=""><br class=""></div><div class="">@Fabio, what is the actual Travis change that would effect this?</div><div class=""><br class=""></div><div class=""> <br class=""></div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex;"><div dir="ltr" class=""><div class="gmail_extra"><div class="gmail_quote"><div class=""></div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex;"><div dir="ltr" class=""><div class="">I notice Travis has not had a green build for nearly 120 days<br class=""></div><div class=""> <a href="https://travis-ci.org/OpenSmalltalk/opensmalltalk-vm/builds" target="_blank" class="">https://travis-ci.org/OpenSma<wbr class="">lltalk/opensmalltalk-vm/builds</a><br class=""></div><div class="">since Build #603...<br class=""></div><div class=""> <a href="https://travis-ci.org/OpenSmalltalk/opensmalltalk-vm/builds/200671152" target="_blank" class="">https://travis-ci.org/OpenSma<wbr class="">lltalk/opensmalltalk-vm/builds<wbr class="">/200671152</a><br class=""></div><div class=""><br class=""></div><div class="">I'd guess that makes it harder to identify which "new" commits introduce "new" defects.</div><div class="">It was tough for me trying to categorise the historical failures to understand what might be required to make them green. </div><div class=""><br class=""></div><div class="">For example, lately the usual failure has been just a single job... </div><div class=""><div class=""> <span class="Apple-converted-space"> </span>macos32x64 squeak.sista.spur.</div></div><div class="">which last worked 22 days ago in Build #713 </div><div class=""> <a href="https://travis-ci.org/OpenSmalltalk/opensmalltalk-vm/builds/228902233" target="_blank" class="">https://travis-ci.org/OpenSma<wbr class="">lltalk/opensmalltalk-vm/builds<wbr class="">/228902233</a><br class=""></div><div class="">but there are other failures in builds that obscure that fact, way back to #603.</div><div class="">Only an exhaustive search through all builds reveals this.</div><div class=""><br class=""></div><div class="">For example, recent Build #748 has macos32x64 squeak.sista.spur as its only failure </div><div class=""> <a href="https://travis-ci.org/OpenSmalltalk/opensmalltalk-vm/builds/236010112" target="_blank" class="">https://travis-ci.org/OpenSma<wbr class="">lltalk/opensmalltalk-vm/builds<wbr class="">/236010112</a><br class=""></div><div class="">but then #750,#751, #752, #753 introduce other failures.</div><div class=""><br class=""></div><div class="">Perhaps a contributing factor is commits being pushed directly to the server Cog branch, </div><div class="">with CI tests only running after they are integrated. I guess that was done to keep the workflow</div><div class="">similar for those moving from subversion to git. However it seems to lose some value of CI</div><div class="">in "preventing" build failures from entering the repository. After a year of using git maybe </div><div class="">this workflow could be reconsidered? Perhaps turn turn on branch protection for administrators</div><div class="">and "force" all commits to the server Cog branch to first pass CI testing? </div></div></blockquote><div class=""><br class=""></div><div class="">That's not the reason. then reason is so that we can see if a commit is good across the entire matrix. If there's no way to discover what elements of the matrix are affected by a build then that will reduce my productivity.</div></div></div></div></blockquote><div class=""><br class=""></div><div class="">You'll get that with a PR. But you'll get test-results before your code is integrated into the server-Cog branch.</div><div class=""><br class=""></div><div class=""> </div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex;"><div dir="ltr" class=""><div class="gmail_extra"><div class="gmail_quote"><div class="">Another thing that would reduce productivity would be having to commit to a clone of the CI and then only being able to push to the Cog branch when the clone succeeds, as this would introduce hours of delay. <br class=""></div></div></div></div></blockquote><div class=""><br class=""></div><div class="">Could you expand on this? I'm not sure what you mean by "clone of the CI".</div><div class="">If you are continuously working in a Cog-eem branch there is no reason to stop work while the CI tests run. Any further commits pushed in your Cog-eem branch automatically update an open PR and the CI testing automatically runs again and automatically integrates into the server-Cog branch upon success. </div><div class=""><br class=""></div><div class=""> <br class=""></div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex;"><div dir="ltr" class=""><div class="gmail_extra"><div class="gmail_quote"><div class="">Much better is a system of tests that then selects good builds, and something that identifies a good build as one in which the full set of production VMs have passed their tests. And of course one wants to be able to see the converse.</div><div class=""> </div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex;"><div dir="ltr" class=""><div class="">Of course needing to submit everything via PRs adds workflow overhead, but some workflows might minimise the impact.<br class=""></div><div class="">So I can't argue whether the benefit is worth it, since I'm not working in the VM every day. </div><div class="">I'm just bumping the status quo to table the question for discussion. </div></div></blockquote><div class=""><br class=""></div><div class="">I don't see the benefit of submitting everything via PR when the need is to define the latest releasable VM, which needs additional testing and packaging, not just green builds.</div></div></div></div></blockquote><div class=""> <br class=""></div><div class="">Having green incremental commit-builds is a separate issue from having a releasable build. The former provides tighter tracking of breakages. For example, when someone else's commits break the build, this confounds the test results of your own commits and I guess introduces at least some minor delay to work out. </div><div class=""><br class=""></div><div class="">Just because there is a higher-level need, doesn't mean the lesser commit-builds are unimportant. They are a pre-requisite for release-builds. Can we do the easier step first. </div><div class=""><br class=""></div><div class=""> <br class=""></div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex;"><div dir="ltr" class=""><div class="gmail_extra"><div class="gmail_quote"><div class=""><br class=""></div><div class=""> </div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex;"><div dir="ltr" class=""><div class=""><br class=""></div><div class="">cheers -ben</div><div class=""><br class=""></div><div class="">P.S. Should macos32x64 squeak.sista.spur be fixed, or temporarily removed from the build matrix?</div><div class="">A continually failing build seems to serve no purpose, whereas green builds should be a great help to noticing new failures. </div></div></blockquote><div class=""><br class=""></div><div class="">I hope it can be both fixed and put in an experimental matrix. </div></div><br class=""><div class="gmail-m_-4321734963970311453gmail_signature"><div dir="ltr" class=""><div class=""><span style="font-size: small; border-collapse: separate;" class=""><div class="">_,,,^..^,,,_<br class=""></div><div class="">best, Eliot</div></span></div></div></div></div></div><br class=""></blockquote><div class=""><br class=""></div><div class="">cheers -ben</div><div class=""> </div></div><br class=""></div><div class="gmail_extra" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><div class="">P.S just bumped into an interesting (short) article "a few conceptual difficulties really stand out."</div><div class=""><a href="https://stevebennett.me/2014/02/26/git-what-they-didnt-tell-you/" class="">https://stevebennett.me/2014/02/26/git-what-they-didnt-tell-you/</a><br class=""></div><div class=""><br class=""></div><div class=""><br class=""></div></div><div class="gmail_extra" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><div class="">P.P.S. I think in a really ideal system, once Travis vetted the final commit of the PR and was ready to merge, it would go back and test each interposing commit and squash it if its build failed, so that every commit in the repo was a green build. But I guess that is just dreaming.</div></div></div></blockquote></div><br class=""></div></body></html>