+1

If you try to look up since when something fails you should not be fooled
by skipped tests.


Thiede, Christoph <Christoph.Thiede at student.hpi.uni-potsdam.de> schrieb am
Sa., 18. Apr. 2020, 15:22:

> Thanks for your arguments! Yeah, we should really take CI more seriously,
> a red bar shouldn't be something you see every day.
>
> > Note that we're already excluding some test cases:
> >
> https://github.com/squeak-smalltalk/squeak-app/blob/b063518c61711ab7a0ca89ffa3d2494925fac823/smalltalk-ci/Squeak64-trunk.ston#L8-L12
>
> Hm, for this purpose of "expected failures, don't run them at all" I think
> it could be helpful if smalltalkCI printed these tests into the output log.
> Example:
>
> #########################
> # 4 tests did not pass: #
> #########################
>
> SocketTest
>  ✗ #testSocketReuse (215ms)
>  ✗ #testUDP (10007ms)
> ...
>
> #########################
> # 4 tests were skipped: #
> #########################
>
> AllocationTest
>  ⛔ #testOneGigAllocation
>  ⛔ #testOneMegAllocation
>  ⛔ #testOutOfMemorySignal
> ...
>
>
> What do you think? :-)
>
> Best,
> Christoph
>
> ------------------------------
> *Von:* Squeak-dev <squeak-dev-bounces at lists.squeakfoundation.org> im
> Auftrag von Fabio Niephaus <lists at fniephaus.com>
> *Gesendet:* Freitag, 17. April 2020 17:58 Uhr
> *An:* The general-purpose Squeak developers list
> *Betreff:* Re: [squeak-dev] [CRON] Fixed:
> squeak-smalltalk/squeak-app#1648 (squeak-trunk - b063518)
>
> Note that we're already excluding some test cases:
>
> https://github.com/squeak-smalltalk/squeak-app/blob/b063518c61711ab7a0ca89ffa3d2494925fac823/smalltalk-ci/Squeak64-trunk.ston#L8-L12
>
> On Fri, Apr 17, 2020 at 5:57 PM Fabio Niephaus <lists at fniephaus.com>
> wrote:
>
>> On Fri, Apr 17, 2020 at 5:53 PM Thiede, Christoph <
>> Christoph.Thiede at student.hpi.uni-potsdam.de> wrote:
>>
>>> > Hmmm... in any case, we want to new bundles to appear on
>>> files.squeak.org/trunk right?
>>>
>>>
>>> Of course, but that does not mean that we have to handle the build as
>>> successful, does it? We can also upload the new bundles and then let the
>>> build fail.
>>>
>> Or we could fix (or ignore) the four failing tests and move on:
>>
>> [image: image.png]
>>
>> Fabio
>>
>>>
>>> Best,
>>>
>>> Christoph
>>> ------------------------------
>>> *Von:* Squeak-dev <squeak-dev-bounces at lists.squeakfoundation.org> im
>>> Auftrag von Taeumel, Marcel
>>> *Gesendet:* Freitag, 17. April 2020 17:27:29
>>> *An:* gettimothy via Squeak-dev
>>> *Betreff:* Re: [squeak-dev] [CRON] Fixed:
>>> squeak-smalltalk/squeak-app#1648 (squeak-trunk - b063518)
>>>
>>> Hmmm... in any case, we want to new bundles to appear on
>>> files.squeak.org/trunk right? That's why it is called "ALPHA". :-)
>>>
>>> Best,
>>> Marcel
>>>
>>> Am 17.04.2020 17:22:16 schrieb Thiede, Christoph <
>>> christoph.thiede at student.hpi.uni-potsdam.de>:
>>>
>>> Sorry for the imprecise question. What I find a bit confusing is the
>>> following: Not all tests are passing, but the Travis job still ends up with
>>> exit code 0 so we get the mail "build was fixed", and the badge of the repo
>>> becomes green. Wouldn't it be more intuitive if the mails sent to the list
>>> would keep reporting "still failing" until we have a green bar in the Trunk?
>>>
>>>
>>> This is the relevant place in the code (in the function
>>> check_test_status):
>>>
>>> squeak-app/prepare_image.sh
>>> <https://github.com/squeak-smalltalk/squeak-app/blob/b063518c61711ab7a0ca89ffa3d2494925fac823/prepare_image.sh#L90-L94>
>>>
>>> Lines 90 to 94 in b063518
>>> <https://github.com/squeak-smalltalk/squeak-app/commit/b063518c61711ab7a0ca89ffa3d2494925fac823>
>>> # Temporarily disable test status check for trunk builds. Remove this
>>> check as
>>> # soon as all tests are running in trunk (hopefully soon).
>>> if is_trunk; then
>>> return 0
>>> fi
>>>
>>>
>>> Couldn't we remove this edge case and run check_test_status *after* the
>>> deployment stuff in prepare.sh?
>>>
>>>
>>> Best,
>>>
>>> Christoph
>>>
>>>
>>> <http://www.hpi.de/>
>>> ------------------------------
>>> *Von:* Squeak-dev <squeak-dev-bounces at lists.squeakfoundation.org> im
>>> Auftrag von Fabio Niephaus <lists at fniephaus.com>
>>> *Gesendet:* Freitag, 17. April 2020 16:25:40
>>> *An:* The general-purpose Squeak developers list
>>> *Betreff:* Re: [squeak-dev] [CRON] Fixed:
>>> squeak-smalltalk/squeak-app#1648 (squeak-trunk - b063518)
>>>
>>> On Fri, Apr 17, 2020 at 4:10 PM Thiede, Christoph <
>>> Christoph.Thiede at student.hpi.uni-potsdam.de> wrote:
>>>
>>>> Do we really want to get exit code 0 before every single test passes?
>>>>
>>>
>>> Sorry, but I don't understand your question. Maybe because not all tests
>>> were passing [1]?
>>>
>>> [1]
>>> https://travis-ci.org/github/squeak-smalltalk/squeak-app/jobs/676189826#L789
>>>
>>>
>>>> ------------------------------
>>>> *Von:* Squeak-dev <squeak-dev-bounces at lists.squeakfoundation.org> im
>>>> Auftrag von Travis CI <builds at travis-ci.org>
>>>> *Gesendet:* Freitag, 17. April 2020 15:41:44
>>>> *An:* squeak-dev at lists.squeakfoundation.org
>>>> *Betreff:* [squeak-dev] [CRON] Fixed: squeak-smalltalk/squeak-app#1648
>>>> (squeak-trunk - b063518)
>>>>
>>>>
>>>> squeak-smalltalk
>>>>
>>>> /
>>>>
>>>> squeak-app
>>>>
>>>> <https://travis-ci.org/github/squeak-smalltalk/squeak-app?utm_medium=notification&utm_source=email>
>>>>
>>>> [image: branch icon]squeak-trunk
>>>> <https://github.com/squeak-smalltalk/squeak-app/tree/squeak-trunk>
>>>> [image: build has passed]
>>>> Build #1648 was fixed
>>>> <https://travis-ci.org/github/squeak-smalltalk/squeak-app/builds/676189824?utm_medium=notification&utm_source=email>
>>>> [image: arrow to build time]
>>>> [image: clock icon]20 mins and 26 secs
>>>>
>>>> [image: Fabio Niephaus avatar]Fabio Niephaus
>>>> b063518 CHANGESET →
>>>> <https://github.com/squeak-smalltalk/squeak-app/compare/76ccf898d33362c1a577fb2cccdcb3dd4ae9b253...b063518c61711ab7a0ca89ffa3d2494925fac823>
>>>>
>>>> Send email notifications to squeak-dev [ci skip]
>>>>
>>>> Want to know about upcoming build environment updates?
>>>>
>>>> Would you like to stay up-to-date with the upcoming Travis CI build
>>>> environment updates? We set up a mailing list for you!
>>>> SIGN UP HERE <http://eepurl.com/9OCsP>
>>>>
>>>> [image: book icon]
>>>>
>>>> Documentation <https://docs.travis-ci.com/> about Travis CI
>>>> Have any questions? We're here to help. <support at travis-ci.com>
>>>> Unsubscribe
>>>> <https://travis-ci.org/account/preferences/unsubscribe?repository=8901856&utm_medium=notification&utm_source=email>
>>>> from build emails from the squeak-smalltalk/squeak-app repository.
>>>> To unsubscribe from *all* build emails, please update your settings
>>>> <https://travis-ci.org/account/preferences/unsubscribe?utm_medium=notification&utm_source=email>.
>>>>
>>>> [image: black and white travis ci logo] <https://travis-ci.com>
>>>>
>>>> Travis CI GmbH, Rigaer Str. 8, 10427 Berlin, Germany | GF/CEO: Randy
>>>> Jacops | Contact: contact at travis-ci.com | Amtsgericht Charlottenburg,
>>>> Berlin, HRB 140133 B | Umsatzsteuer-ID gemäß §27 a Umsatzsteuergesetz:
>>>> DE282002648
>>>>
>>>>
>>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20200419/12d5e644/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 19029 bytes
Desc: not available
URL: <http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20200419/12d5e644/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 19029 bytes
Desc: not available
URL: <http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20200419/12d5e644/attachment-0003.png>