-
Notifications
You must be signed in to change notification settings - Fork 9
Description
Initially asked on Discord:
Codewars runner supports some languages and testing frameworks which do not fit well into the BDD pattern and do not have constructs which map well into <DESCRIBE::>
and <IT::>
markers used by test output panel. As a consequence, some reporters emit artificially created <IT::>
tags, with titles derived from some additional context data because authors cannot assign any title to them. This often results in an awkward output with confusing or misplaced titles (see #239 , #241 , codewars/FactCheck.jl#3 ), or which spam <IT::>
beyond reason.
My proposal would be to get rid of artificial <IT::>
's from all reporters for frameworks which do not have a corresponding construct, and add support for assertions which are not in scope of any <IT::>
. Maybe it would be good enough to allow for assertions inside of <DESCRIBE::>
, or maybe a better solution would be to provide some new marker, which would be used for frameworks which do not map well into BDD and use simpler concept of a test and a context, for exampel something like:
<TEST::>
<TEST::> (if nesting is allowed)
<ASSERTIONS_CONTEXT::> (optional)
<ASSERTIONS_CONTEXT::> (if nesting is possible)
<ASSERTION::> (probably <PASSED::> or <FAILED::>)
<ASSERTION::> (if not stop on first failure)
<COMPLETEDIN::>
<COMPLETEDIN::>
<COMPLETEDIN::>
<COMPLETEDIN::>
With such setup, Julia reporter would emit <TEST::>
for @facts
(which is optional), <ASSERTIONS_CONTEXT::>
for @context
(which is optional), and <PASSED::>
or <FAILED::>
for @fact
. For Clojure, we'd get <TEST::>
for deftest
(whici is mandatory), <ASSERTIONS_CONTEXT::>
for testing
(which is optional), and <PASSED::>
or <FAILED::>
for is
.