sbt.testing
Type members
Classlikes
Indicates that classes or modules with a specific annotation, either on at least one top level method or on the class or module itself, should be discovered as test classes.
Indicates that classes or modules with a specific annotation, either on at least one top level method or on the class or module itself, should be discovered as test classes.
An event fired by the test framework during a run.
An event fired by the test framework during a run.
Interface implemented by clients that handle events fired by the test framework during a run.
Interface implemented by clients that handle events fired by the test framework during a run.
An event handler is passed to the test framework via the
execute
method of Task
s.
A way to identify test classes and/or modules that should be discovered when the client performs discovery.
A way to identify test classes and/or modules that should be discovered when the client performs discovery.
Scala.js: Implementations may not rely on the identity of Fingerprints, since they are serialized between JS / JVM.
Interface implemented by test frameworks.
Interface implemented by test frameworks.
A logger through which to provide feedback to the user about a run.
A logger through which to provide feedback to the user about a run.
The difference between the event handler and the logger is that the event handler is for events intended to be consumed by the client software whereas the logger is for messages intended to be consumed by the client user (i.e., a human).
Implementations of this interface must be thread-safe.
Information in addition to a test class name that identifies a nested suite about which an event was fired.
Information in addition to a test class name that identifies a nested suite about which an event was fired.
Information in addition to a test class name that identifies a test in a nested suite about which an event was fired.
Information in addition to a test class name that identifies a test in a nested suite about which an event was fired.
Represents one run of a suite of tests.
Represents one run of a suite of tests.
The run represented by a Runner
has a lifecycle. The run begins
when the Runner
is instantiated by the framework and returned
to the client during a Framework.runner
invocation. The run
continues until the client invokes done
on the
Runner
. Before invoking done
, the client can
invoke the tasks
method as many times at it wants, but once
done
has been invoked, the Runner
enters "spent"
mode. Any subsequent invocations of tasks
will be met with an
IllegalStateException
.
Information in addition to a test class name that identifies the suite or test about which an event was fired.
Information in addition to a test class name that identifies the suite or test about which an event was fired.
This class has five subtypes:
SuiteSelector
- indicates an event is about an entire suite of tests whose class was reported asfullyQualifiedName
in theEvent
TestSelector
- indicates an event is about a single test directly contained in the suite whose class was reported asfullyQualifiedName
in theEvent
NestedSuiteSelector
- indicates an event is about an entire nested suite of tests whose top-level, "nesting" class was reported asfullyQualifiedName
in theEvent
NestedTestSelector
- indicates an event is about a single test contained in a nested suite whose top-level, "nesting" class was reported asfullyQualifiedName
in theEvent
TestWildcardSelector
- indicates an event is about zero to many tests directly contained in the suite whose class was reported asfullyQualifiedName
in theEvent
Represents the status of running a test.
Represents the status of running a test.
Test frameworks can decided which of these to use and what they mean, but in general, the intended meanings are:
- Success - a test succeeded
- Error - an "error" occurred during a test
- Failure - an "failure" during a test
- Skipped - a test was skipped for any reason
- Ignored - a test was ignored, i.e., temporarily disabled with the intention of fixing it later
- Canceled - a test was canceled, i.e., not able to be completed because of some unmet pre-condition, such as a database being offline that the test requires
- Pending - a test was declared as pending, i.e., with test code and/or production code as yet unimplemented
The difference between errors and failures, if any, is determined by the test frameworks. JUnit and specs2 differentiate between errors and failures. ScalaTest reports everything (both assertion failures and unexpected errors) as failures. JUnit and ScalaTest support ignored tests. ScalaTest and specs2 support a notion of pending tests. ScalaTest differentiates between ignored and canceled tests, whereas specs2 only supports skipped tests, which are implemented like ScalaTest's canceled tests. TestNG uses "skipped" to report tests that were not executed because of failures in dependencies, which is also similar to canceled tests in ScalaTest.
Indicates that classes (and possibly modules) that extend a particular superclass, or mix in a particular supertrait, should be discovered as test classes.
Indicates that classes (and possibly modules) that extend a particular superclass, or mix in a particular supertrait, should be discovered as test classes.
Indicates an event was about the entire suite whose class had the fully
qualified name specified as the fullyQualifiedName
attribute
the event.
Indicates an event was about the entire suite whose class had the fully
qualified name specified as the fullyQualifiedName
attribute
the event.
A task to execute.
A task to execute.
The client may decide when or how to execute the task based on its tags. A task can be any job, but is primarily intended for running tests and/or supplying more tasks to the client. A framework can supply more tasks to the client in the returned an array of Tasks (which can be empty if there's no more work to do.)
A bundle of information used to request a Task
from a test
framework.
A bundle of information used to request a Task
from a test
framework.
An array of TaskDef
is passed to Runner
's tasks
method,
which returns an array of Tasks
. Each returned task, when
executed, will run tests and suites determined by the test class name,
fingerprints, "explicitly specified" field, and selectors of one of the
passed TaskDef
s.
The "Explicitly specified" field means the user supplied a complete fully qualified test name, such as with the command:
> test-only com.mycompany.myproject.WholeNameSpec
as opposed to commands like:
> test-only *WholeNameSpec
or simply:
> test
The explicitlySpecified
field will be true for in the first
case, and false in the last two cases, because only in the first case was
the fully qualified test class name completely specified by the user. The
test framework can use this information to decide whether to ignore an
annotation requesting a class not be discovered.
The fingerprint
parameter indicates how the test suite was
identified as a test suite. This tasks
method may be called
with TaskDef
s containing the same value for
testClassName
but different fingerprints. For example, if both
a class and its companion object were test classes, the tasks
method could be passed an array containing TaskDef
s with the
same name but with a different value for fingerprint.isModule
.
A test framework may "reject" a requested task by returning no
Task
for that TaskDef
.
- Value Params
- explicitlySpecified
indicates whether the test class was explicitly specified by user.
- fingerprint
indicates how the test suite was identified as a test suite
- fullyQualifiedName
the fully qualified name of the test class to be run by the requested task
- selectors
a possibly empty array of
Selectors
determining suites and tests to run
Information in addition to a test class name that identifies a test directly
contained in the suite whose class had the fully qualified name specified as
the fullyQualifiedName
attribute passed to the event.
Information in addition to a test class name that identifies a test directly
contained in the suite whose class had the fully qualified name specified as
the fullyQualifiedName
attribute passed to the event.
Information that identifies zero to many tests directly contained in a test class.
Information that identifies zero to many tests directly contained in a test class.
The testWildcard
is a simple string, i.e., not a glob
or regular expression. Any test whose name includes the
testWildcard
string as a substring will be selected.