Hi all! :-)
I was just working a bit with Squeak by Example (the book) and found out that resumable assertions as described in the book (see chapter SUnit, Advanced, Continuing after a failure) do not any longer work in a trunk image. If you write a test case with some resumable assertions, turn on logging, and run the test, only the first failure will be logged, not the rest, too.
(Methods to reproduce:
isLogging
^ true "You will need to open a Transcript"
testResumable
#(1 2 3 4 5) do: [:i |
self assert: i even description: ['{1} is not even' format: {i}] resumable: true]
)
A second issue I experienced frequently when I felt the desire to work with resumable assertions was that they are basically only available for two selectors: #assert:description:resumable: and #deny:description:identical:. I often would like to insert a resumable into any test without being that verbose, and nearly redoubling the number of assertion selectors by adding a resumable argument to each of them did not feel right, either.
Please find the attached changeset, which fixes both issues.
* In TestResult >> #runCase:, resumable assertions are identified and - finally - resumed, after marking the test as failed. Unfortunately, other test runners (yes, I'm thinking of smalltalkCI) will need to reimplement this feature unless they reuse the TestResult implementation. * In TestCase, I refactored some methods that raise assertion failures and added the most basic #signalFailure:resumable: as the single source of signalization. Instead of setting the resumable bit manually for each assertion, you now can also make the whole test execution resumable by: * overriding #isResumable, * setting a resumable flag using #resumable:, * setting a resumable flag temporarily using #beResumable:during:, or * adding a <resumable> or <resumable: true> pragma to the test method. * In addition, I tested the added and fixed behavior in ResumableTestFailureTestCase.
Looking forward to your review! :-)
Best,
Christoph
PS: I ran all tests in a fresh trunk image before and after loading the changeset, there were no regressions.
http://www.hpi.de/ ________________________________ Von: Squeak-dev squeak-dev-bounces@lists.squeakfoundation.org im Auftrag von Thiede, Christoph Gesendet: Dienstag, 29. September 2020 01:18:07 An: Squeak Dev Betreff: [squeak-dev] ResumableTestFailure
Hi all! :-)
I was just working a bit with Squeak by Example (the book) and found out that resumable assertions as described in the book (see chapter SUnit, Advanced, Continuing after a failure) do not any longer work in a trunk image. If you write a test case with some resumable assertions, turn on logging, and run the test, only the first failure will be logged, not the rest, too.
(Methods to reproduce:
isLogging
^ true "You will need to open a Transcript"
testResumable
#(1 2 3 4 5) do: [:i |
self assert: i even description: ['{1} is not even' format: {i}] resumable: true]
)
A second issue I experienced frequently when I felt the desire to work with resumable assertions was that they are basically only available for two selectors: #assert:description:resumable: and #deny:description:identical:. I often would like to insert a resumable into any test without being that verbose, and nearly redoubling the number of assertion selectors by adding a resumable argument to each of them did not feel right, either.
Please find the attached changeset, which fixes both issues.
* In TestResult >> #runCase:, resumable assertions are identified and - finally - resumed, after marking the test as failed. Unfortunately, other test runners (yes, I'm thinking of smalltalkCI) will need to reimplement this feature unless they reuse the TestResult implementation. * In TestCase, I refactored some methods that raise assertion failures and added the most basic #signalFailure:resumable: as the single source of signalization. Instead of setting the resumable bit manually for each assertion, you now can also make the whole test execution resumable by: * overriding #isResumable, * setting a resumable flag using #resumable:, * setting a resumable flag temporarily using #beResumable:during:, or * adding a <resumable> or <resumable: true> pragma to the test method. * In addition, I tested the added and fixed behavior in ResumableTestFailureTestCase.
Looking forward to your review! :-)
Best,
Christoph
Hi all,
would anyone object if I merged this patch into the Trunk? tl;dr: Revives/extends the mechanism for resumable failures on assertions. I'd love to have this in Trunk, then in the next step we could add support for it in smalltalkCI! :-)
Best, Christoph
--- Sent from Squeak Inbox Talk
On 2020-09-28T23:39:21+00:00, christoph.thiede@student.hpi.uni-potsdam.de wrote:
PS: I ran all tests in a fresh trunk image before and after loading the changeset, there were no regressions.
http://www.hpi.de/ ________________________________ Von: Squeak-dev <squeak-dev-bounces at lists.squeakfoundation.org> im Auftrag von Thiede, Christoph Gesendet: Dienstag, 29. September 2020 01:18:07 An: Squeak Dev Betreff: [squeak-dev] ResumableTestFailure
Hi all! :-)
I was just working a bit with Squeak by Example (the book) and found out that resumable assertions as described in the book (see chapter SUnit, Advanced, Continuing after a failure) do not any longer work in a trunk image. If you write a test case with some resumable assertions, turn on logging, and run the test, only the first failure will be logged, not the rest, too.
(Methods to reproduce:
isLogging
^ true "You will need to open a Transcript"testResumable
#(1 2 3 4 5) do: [:i | self assert: i even description: ['{1} is not even' format: {i}] resumable: true])
A second issue I experienced frequently when I felt the desire to work with resumable assertions was that they are basically only available for two selectors: #assert:description:resumable: and #deny:description:identical:. I often would like to insert a resumable into any test without being that verbose, and nearly redoubling the number of assertion selectors by adding a resumable argument to each of them did not feel right, either.
Please find the attached changeset, which fixes both issues.
- In TestResult >> #runCase:, resumable assertions are identified and - finally - resumed, after marking the test as failed. Unfortunately, other test runners (yes, I'm thinking of smalltalkCI) will need to reimplement this feature unless they reuse the TestResult implementation.
- In TestCase, I refactored some methods that raise assertion failures and added the most basic #signalFailure:resumable: as the single source of signalization. Instead of setting the resumable bit manually for each assertion, you now can also make the whole test execution resumable by:
- overriding #isResumable,
- setting a resumable flag using #resumable:,
- setting a resumable flag temporarily using #beResumable:during:, or
- adding a <resumable> or <resumable: true> pragma to the test method.
- In addition, I tested the added and fixed behavior in ResumableTestFailureTestCase.
Looking forward to your review! :-)
Best,
Christoph
squeak-dev@lists.squeakfoundation.org
-
christoph.thiede＠student.hpi.uni-potsdam.de -
Thiede, Christoph