Package org.testng
Class TestNG
java.lang.Object
org.testng.TestNG
This class is the main entry point for running tests in the TestNG framework. Users can create
their own TestNG object and invoke it in many different ways:
- On an existing testng.xml
- On a synthetic testng.xml, created entirely from Java
- By directly setting the test classes
The command line parameters are:
- -d
outputdir
: specify the output directory - -testclass
class_name
: specifies one or several class names - -testjar
jar_name
: specifies the jar containing the tests - -sourcedir
src1;src2
: ; separated list of source directories (used only when javadoc annotations are used) - -target
- -groups
- -testrunfactory
- -listener
Please consult documentation for more details.
FIXME: should support more than simple paths for suite xmls
- See Also:
-
usage()
-
Field Summary
FieldsModifier and TypeFieldDescriptionstatic final String
The default name for a suite launched from the command linestatic final String
The default name for a test launched from the command linestatic final String
The default name of the result's output directory (keep public, used by Eclipse).static final Integer
protected long
protected long
protected boolean
-
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionvoid
addClassLoader
(ClassLoader loader) Add a class loader to the searchable loaders.void
addListener
(Object listener) Deprecated.Use addListener(ITestNGListener) insteadvoid
addListener
(ITestNGListener listener) void
addMethodSelector
(String className, int priority) void
addMethodSelector
(XmlMethodSelector selector) void
alwaysRunListeners
(boolean alwaysRun) void
Deprecated.use new configure(CommandLineArgs) methodprotected void
configure
(CommandLineArgs cla) Configure the TestNG instance based on the command line parameters.Returns the configuration failure policy.protected org.testng.internal.IConfiguration
static TestNG
Deprecated.since 5.1protected long
getEnd()
boolean
protected long
getStart()
int
boolean
boolean
boolean
hasSkip()
void
Invoked by the remote runner.void
boolean
boolean
static void
The TestNG entry point for command line execution.static TestNG
privateMain
(String[] argv, ITestListener listener) Note: this method is not part of the public API and is meant for internal usage only.void
void
run()
Run TestNG.Run the test suites.This needs to be public for maven2, for now..At least until an alternative mechanism is found.void
setCommandLineSuite
(XmlSuite suite) void
setConfigFailurePolicy
(XmlSuite.FailurePolicy failurePolicy) Sets the policy for whether or not to ever invoke a configuration method again after it has failed once.void
setDataProviderThreadCount
(int count) void
setDefaultSuiteName
(String defaultSuiteName) void
setDefaultTestName
(String defaultTestName) void
setExcludedGroups
(String groups) Define which groups will be excluded from this run.void
void
setGenerateResultsPerSuite
(boolean generateResultsPerSuite) void
setGroupByInstances
(boolean b) void
Define which groups will be included from this run.void
setInjectorFactory
(IInjectorFactory factory) void
setListenerClasses
(List<Class<? extends ITestNGListener>> classes) Define which listeners to user for this run.void
setListenerComparator
(ListenerComparator listenerComparator) void
setListenerFactory
(ITestNGListenerFactory factory) void
setListenersToSkipFromBeingWiredInViaServiceLoaders
(String... listeners) void
setMethodInterceptor
(IMethodInterceptor methodInterceptor) void
setObjectFactory
(Class<? extends ITestObjectFactory> c) void
setObjectFactory
(ITestObjectFactory factory) void
setOutputDirectory
(String outputdir) Sets the output directory where the reports will be created.void
setParallel
(String parallel) Deprecated.Use #setParallel(XmlSuite.ParallelMode) insteadvoid
setParallel
(XmlSuite.ParallelMode parallel) void
setPreserveOrder
(boolean b) void
setRandomizeSuites
(boolean randomizeSuites) void
setReportAllDataDrivenTestsAsSkipped
(boolean reportAllDataDrivenTestsAsSkipped) void
void
void
setSourcePath
(String path) Deprecated.void
setSuiteThreadPoolSize
(Integer suiteThreadPoolSize) void
setTestClasses
(Class[] classes) Set the test classes to be run by this TestNG object.void
setTestJar
(String jarPath) Sets a jar containing a testng.xml file.void
setTestNames
(List<String> testNames) protected void
void
setTestSuites
(List<String> suites) Set the suites file names to be run by this TestNG object.void
setThreadCount
(int threadCount) void
setUseDefaultListeners
(boolean useDefaultListeners) void
setVerbose
(int verbose) Sets the level of verbosity.void
setXmlPathInJar
(String xmlPathInJar) void
setXmlSuites
(List<XmlSuite> suites) Specifies the XmlSuite objects to run.void
shareThreadPoolForDataProviders
(boolean flag) void
shouldUseGlobalThreadPool
(boolean flag) void
toggleFailureIfAllTestsWereSkipped
(boolean failIfAllTestsSkipped) boolean
protected static void
Double check that the command line parameters are valid.
-
Field Details
-
DEFAULT_COMMAND_LINE_SUITE_NAME
The default name for a suite launched from the command line- See Also:
-
DEFAULT_COMMAND_LINE_TEST_NAME
The default name for a test launched from the command line- See Also:
-
DEFAULT_OUTPUTDIR
The default name of the result's output directory (keep public, used by Eclipse).- See Also:
-
m_suites
-
m_useDefaultListeners
protected boolean m_useDefaultListeners -
DEFAULT_VERBOSE
-
m_end
protected long m_end -
m_start
protected long m_start
-
-
Constructor Details
-
TestNG
public TestNG()Default constructor. Setting also usage of default listeners/reporters. -
TestNG
public TestNG(boolean useDefaultListeners) Used by maven2 to have 0 output of any kind come out of testng.- Parameters:
useDefaultListeners
- Whether or not any default reports should be added to tests.
-
-
Method Details
-
toggleFailureIfAllTestsWereSkipped
public void toggleFailureIfAllTestsWereSkipped(boolean failIfAllTestsSkipped) - Parameters:
failIfAllTestsSkipped
- - Whether TestNG should enable/disable failing when all the tests were skipped and nothing was run (Mostly when a test is powered by a data provider and when the data provider itself fails causing all tests to skip).
-
setListenersToSkipFromBeingWiredInViaServiceLoaders
- Parameters:
listeners
- - An array of fully qualified class names that should be skipped from being wired in via service loaders.
-
getStatus
public int getStatus() -
setOutputDirectory
Sets the output directory where the reports will be created.- Parameters:
outputdir
- The directory.
-
setUseDefaultListeners
public void setUseDefaultListeners(boolean useDefaultListeners) - Parameters:
useDefaultListeners
- If true before run(), the default listeners will not be used.- org.testng.reporters.TestHTMLReporter
- org.testng.reporters.JUnitXMLReporter
- org.testng.reporters.XMLReporter
- See Also:
-
setListenerComparator
-
getListenerComparator
-
setTestJar
Sets a jar containing a testng.xml file.- Parameters:
jarPath
- - Path of the jar
-
setXmlPathInJar
- Parameters:
xmlPathInJar
- Sets the path to the XML file in the test jar file.
-
initializeSuitesAndJarFile
public void initializeSuitesAndJarFile() -
setThreadCount
public void setThreadCount(int threadCount) - Parameters:
threadCount
- Define the number of threads in the thread pool.
-
setParallel
Deprecated.Use #setParallel(XmlSuite.ParallelMode) instead- Parameters:
parallel
- Define whether this run will be run in parallel mode.
-
setParallel
-
setCommandLineSuite
-
setTestClasses
Set the test classes to be run by this TestNG object. This method will create a dummy suite that will wrap these classes called "Command Line Test".If used together with threadCount, parallel, groups, excludedGroups than this one must be set first.
- Parameters:
classes
- An array of classes that contain TestNG annotations.
-
addMethodSelector
-
addMethodSelector
-
setReportAllDataDrivenTestsAsSkipped
public void setReportAllDataDrivenTestsAsSkipped(boolean reportAllDataDrivenTestsAsSkipped) -
getReportAllDataDrivenTestsAsSkipped
public boolean getReportAllDataDrivenTestsAsSkipped() -
propagateDataProviderFailureAsTestFailure
public void propagateDataProviderFailureAsTestFailure() -
isPropagateDataProviderFailureAsTestFailure
public boolean isPropagateDataProviderFailureAsTestFailure() -
useGlobalThreadPool
public boolean useGlobalThreadPool() -
shouldUseGlobalThreadPool
public void shouldUseGlobalThreadPool(boolean flag) -
setTestSuites
Set the suites file names to be run by this TestNG object. This method tries to load and parse the specified TestNG suite xml files. If a file is missing, it is ignored.- Parameters:
suites
- A list of paths to one more XML files defining the tests. For example:TestNG tng = new TestNG(); List<String> suites = Lists.newArrayList(); suites.add("c:/tests/testng1.xml"); suites.add("c:/tests/testng2.xml"); tng.setTestSuites(suites); tng.run();
-
setXmlSuites
Specifies the XmlSuite objects to run. -
setExcludedGroups
Define which groups will be excluded from this run.- Parameters:
groups
- A list of group names separated by a comma.
-
setGroups
Define which groups will be included from this run.- Parameters:
groups
- A list of group names separated by a comma.
-
setTestRunnerFactory
-
setObjectFactory
-
setObjectFactory
-
setListenerClasses
Define which listeners to user for this run.- Parameters:
classes
- A list of classes, which must be either ISuiteListener, ITestListener or IReporter
-
addListener
Deprecated.Use addListener(ITestNGListener) instead- Parameters:
listener
- The listener to add
-
addListener
-
getReporters
-
getTestListeners
-
getSuiteListeners
-
setVerbose
public void setVerbose(int verbose) Sets the level of verbosity. This value will override the value specified in the test suites.- Parameters:
verbose
- the verbosity level (0 to 10 where 10 is most detailed) Actually, this is a lie: you can specify -1 and this will put TestNG in debug mode (no longer slicing off stack traces and all).
-
setExecutorServiceFactory
-
setListenerFactory
-
setGenerateResultsPerSuite
public void setGenerateResultsPerSuite(boolean generateResultsPerSuite) -
initializeEverything
public void initializeEverything()Invoked by the remote runner. -
run
public void run()Run TestNG. -
runSuites
Run the test suites.This method can be overridden by subclass.
For example, DistributedTestNG to run in master/slave mode according to commandline args.- Returns:
- - List of suites that were run as
ISuite
objects. - Since:
- 6.9.11 when moving distributed/remote classes out into separate project
-
runSuitesLocally
This needs to be public for maven2, for now..At least until an alternative mechanism is found.- Returns:
- The locally run suites
-
getConfiguration
protected org.testng.internal.IConfiguration getConfiguration() -
main
The TestNG entry point for command line execution.- Parameters:
argv
- the TestNG command line parameters.
-
privateMain
Note: this method is not part of the public API and is meant for internal usage only.- Parameters:
argv
- The param argumentslistener
- The listener- Returns:
- The TestNG instance
-
configure
Configure the TestNG instance based on the command line parameters.- Parameters:
cla
- The command line parameters
-
setSuiteThreadPoolSize
-
getSuiteThreadPoolSize
-
setRandomizeSuites
public void setRandomizeSuites(boolean randomizeSuites) -
alwaysRunListeners
public void alwaysRunListeners(boolean alwaysRun) -
setSourcePath
Deprecated.This method is invoked by Maven's Surefire, only remove it once Surefire has been modified to no longer call it.- Parameters:
path
- The path
-
configure
Deprecated.use new configure(CommandLineArgs) methodThis method is invoked by Maven's Surefire to configure the runner, do not remove unless you know for sure that Surefire has been updated to use the new configure(CommandLineArgs) method.- Parameters:
cmdLineArgs
- The command line
-
setTestNames
- Parameters:
testNames
- Only run the specified tests from the suite.
-
setSkipFailedInvocationCounts
-
validateCommandLineParameters
Double check that the command line parameters are valid.- Parameters:
args
- The command line to check
-
hasFailure
public boolean hasFailure()- Returns:
- true if at least one test failed.
-
hasFailureWithinSuccessPercentage
public boolean hasFailureWithinSuccessPercentage()- Returns:
- true if at least one test failed within success percentage.
-
hasSkip
public boolean hasSkip()- Returns:
- true if at least one test was skipped.
-
getOutputDirectory
-
getAnnotationTransformer
-
getDefaultSuiteName
- Returns:
- the defaultSuiteName
-
setDefaultSuiteName
- Parameters:
defaultSuiteName
- the defaultSuiteName to set
-
getDefaultTestName
- Returns:
- the defaultTestName
-
setDefaultTestName
- Parameters:
defaultTestName
- the defaultTestName to set
-
setConfigFailurePolicy
Sets the policy for whether or not to ever invoke a configuration method again after it has failed once. Possible values are defined inXmlSuite
. The default value isXmlSuite.FailurePolicy.SKIP
- Parameters:
failurePolicy
- the configuration failure policy
-
getConfigFailurePolicy
Returns the configuration failure policy.- Returns:
- config failure policy
-
getDefault
Deprecated.since 5.1- Returns:
- The default instance
-
setMethodInterceptor
-
setDataProviderThreadCount
public void setDataProviderThreadCount(int count) -
addClassLoader
Add a class loader to the searchable loaders.- Parameters:
loader
- The class loader to add
-
setPreserveOrder
public void setPreserveOrder(boolean b) -
getStart
protected long getStart() -
getEnd
protected long getEnd() -
setGroupByInstances
public void setGroupByInstances(boolean b) -
setServiceLoaderClassLoader
-
getServiceLoaderListeners
-
setInjectorFactory
-