[squeak-dev] The Inbox: SUnit-cmm.116.mcz

Chris Muller asqueaker at gmail.com
Wed May 29 20:46:57 UTC 2019


> if an unexpected pass would be treated as a failure, *you would encourage* tests like this:
>
> testSomething
>    self assert: 5 / 0.
>
> expectedFailures
>    ^ #(testSomething)

It appears that has already happened already even without this change.
But, nobody noticed.  Check out UTF8EdgeCaseTest>>#testOverlongNUL.  Is
this a case of misunderstanding what expectedFaillures was for?  Or
did it eventually get working and so the test now passes, but we
forgot to remove it?  That would mean the "documentation" has been
wrong for a long time.  If only we could have gotten some sort of
"notification" whenever an expectedFailure would start working again,
hmmm....  ;)

> A programmer's goal should always be to write passing tests. An expected failure should only be temporary. If that expected failure passes at some point without the programmer noticing and updating #expectedFailures, that's not a bad thing. It's progress. The programmer will update #expectedFailures eventually.
>
> The list of expected failures is just an add-on to your test suite to document not-yet-working stuff so that fellow programmers do not treat those failing tests as some new regression.

What do fellow programmers do with those expectedFailures then?  The
way you make it sound, they're supposed to ignore them.  That's
the trouble, they've muscled their way into the tool, disrupting the
lists and tainting the *real* output status fellow programmers
actually care about.  These are core SUnit outputs!

The way we're using expectedFailures now is so limited, yet so
intrusvie.  We could "document not-yet-working stuff" anywhere...
comments, or an external project-plan document even..

OR, we could tweak SUnit so they integrate seamlessly:

   - as a tucked-away documentation list of expected failures,
   - with users still reminded of its existence by the totals output ("4
expected failures"),
   - and yet, not "in-the-way" of the real work,
   - and getting proper green/yellow/red status restored.

Best,
  Chris


More information about the Squeak-dev mailing list