<div dir="ltr">Hi Ben,<div class="gmail_extra"><br><div class="gmail_quote">On Sun, May 28, 2017 at 6:46 AM, Ben Coman <span dir="ltr"><<a href="mailto:btc@openinworld.com" target="_blank">btc@openinworld.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> <br><div dir="ltr"><div><br></div>Maybe I'm speaking out of turn from the sidelines, but just curious to ask a leading question... <div><br></div><div>What is the purpose of the Continuous Integration system?</div></div></blockquote><div><br></div><div>One thing for me is to identify when I've made a mistake Yes it is selfish of me to want to break a build, but in some way those of us who are active VM developers must be able to commit and see what fails, especially when we have a large platform and VM variant spread.</div><div><br></div><div>IMO what needs to be added is a way of marking commits as good. I think we should continue to be allowed to break the build (something we do unintentionally). But we need a set of criteria to mark a build as fit for release and have a download URL from which we obtain the latest releasable VM.</div><div><br></div><div>I agree with Fabio that if we want the CI server to be as green as possible marking builds which have been failing for a long time as "allowed to fail" is a good idea. But it also has to be brought to people's attention that there are such builds.</div><div><br></div><div>As far as the Sista and Locoed VMs go these are experimental, as would a threaded FFI VM, something we may have later this year. It would be nice to segregate the standard VMs that are already in production from these experimental builds so that failures within the experimental set don't affect the status of the production VMs. I'd rather se that segregation than mark certain builds as allowed to fail.</div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><br></div><div><br></div><div><br></div><div>I notice Travis has not had a green build for nearly 120 days</div><div> <a href="https://travis-ci.org/OpenSmalltalk/opensmalltalk-vm/builds" target="_blank">https://travis-ci.org/OpenSma<wbr>lltalk/opensmalltalk-vm/builds</a><br></div><div>since Build #603...<br></div><div> <a href="https://travis-ci.org/OpenSmalltalk/opensmalltalk-vm/builds/200671152" target="_blank">https://travis-ci.org/OpenSma<wbr>lltalk/opensmalltalk-vm/<wbr>builds/200671152</a><br></div><div><br></div><div>I'd guess that makes it harder to identify which "new" commits introduce "new" defects.</div><div>It was tough for me trying to categorise the historical failures to understand what might be required to make them green. </div><div><br></div><div>For example, lately the usual failure has been just a single job... </div><div><div> macos32x64 squeak.sista.spur.</div></div><div>which last worked 22 days ago in Build #713 </div><div> <a href="https://travis-ci.org/OpenSmalltalk/opensmalltalk-vm/builds/228902233" target="_blank">https://travis-ci.org/OpenSma<wbr>lltalk/opensmalltalk-vm/<wbr>builds/228902233</a><br></div><div>but there are other failures in builds that obscure that fact, way back to #603.</div><div>Only an exhaustive search through all builds reveals this.</div><div><br></div><div>For example, recent Build #748 has macos32x64 squeak.sista.spur as its only failure </div><div> <a href="https://travis-ci.org/OpenSmalltalk/opensmalltalk-vm/builds/236010112" target="_blank">https://travis-ci.org/OpenSma<wbr>lltalk/opensmalltalk-vm/<wbr>builds/236010112</a><br></div><div>but then #750,#751, #752, #753 introduce other failures.</div><div><br></div><div>Perhaps a contributing factor is commits being pushed directly to the server Cog branch, </div><div>with CI tests only running after they are integrated. I guess that was done to keep the workflow</div><div>similar for those moving from subversion to git. However it seems to lose some value of CI</div><div>in "preventing" build failures from entering the repository. After a year of using git maybe </div><div>this workflow could be reconsidered? Perhaps turn turn on branch protection for administrators</div><div>and "force" all commits to the server Cog branch to first pass CI testing? </div></div></blockquote><div><br></div><div>That's not the reason. then reason is so that we can see if a commit is good across the entire matrix. If there's no way to discover what elements of the matrix are affected by a build then that will reduce my productivity.</div><div><br></div><div>Another thing that would reduce productivity would be having to commit to a clone of the CI and then only being able to push to the Cog branch when the clone succeeds, as this would introduce hours of delay. Much better is a system of tests that then selects good builds, and something that identifies a good build as one in which the full set of production VMs have passed their tests. And of course one wants to be able to see the converse.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>Of course needing to submit everything via PRs adds workflow overhead, but some workflows might minimise the impact.<br></div><div>So I can't argue whether the benefit is worth it, since I'm not working in the VM every day. </div><div>I'm just bumping the status quo to table the question for discussion. </div></div></blockquote><div><br></div><div>I don't see the benefit of submitting everything via PR when the need is to define the latest releasable VM, which needs additional testing and packaging, not just green builds.</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><br></div><div>cheers -ben</div><div><br></div><div>P.S. Should macos32x64 squeak.sista.spur be fixed, or temporarily removed from the build matrix?</div><div>A continually failing build seems to serve no purpose, whereas green builds should be a great help to noticing new failures. </div></div></blockquote><div><br></div><div>I hope it can be both fixed and put in an experimental matrix. </div></div><br><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><span style="font-size:small;border-collapse:separate"><div>_,,,^..^,,,_<br></div><div>best, Eliot</div></span></div></div></div>
</div></div>