Parent for Spock specs that test Jenkins pipeline code.

Such specifications need all of the Pipeline symbols that might exist at runtime (e.g. node, sh, echo, ws, stash, etc) to exist, or else the pipeline code won't run.

This class ensures that all pipeline extension points exist as Spock Mock objects so that the calls will succeed and that interactions can be inspected, stubbed, and verified. Just extend this instead of the regular Specification and test your pipeline scripts. You can access a Spock mock for any pipeline step that would exist by using getPipelineMock(String).

1. Testing Groovy Functions
2. Testing Pipeline Scripts
3. Mock Pipeline Steps
4. Using Pipeline Variables
   1. Mock GlobalVariables
   2. Defining Pipeline Variables
5. Mock Shared Library Variables
6. Preparing Additional Objects
7. Mocking Additional Pipeline Steps
8. Mocking Additional Pipeline Variables
9. Mock Jenkins
10. Mock Pipeline Execution
11. Metaprogramming
12. Define Custom Classpath For Script Loading

Testing Groovy Functions

For example, the following Groovy class:

class MyJenkinsPipelineHelper {
	public Map helloNode(String _label, Closure _body) {
		return node( _label ) {
			echo( "Hello from a [${_label}] node!" )
			_body()
			echo( "Goodbye from a [${_label}] node!" )
		}
	}
}
 

can be tested like this:

public class MyJenkinsPipelineHelperSpec extends JenkinsPipelineSpecification {
	def "helloNode" () {
		given:
			MyJenkinsPipelineHelper myVar = new MyJenkinsPipelineHelper()
		when:
			myVar.helloNode( "nodeType" ) {
				echo( "inside node" )
			}
		then:
			1 * getPipelineMock( "node" )("nodeType", _)
			1 * getPipelineMock( "echo" )("Hello from a [nodeType] node!")
			1 * getPipelineMock( "echo" )("Goodbye from a [nodeType] node!")
			1 * getPipelineMock( "echo" )("inside node")
	}
}
 

Testing Pipeline Scripts

loadPipelineScriptForTest(java.lang.String)

If you have pipeline scripts (i.e. whole Jenkinsfiles) in src/main/resources, you can load them as Scripts and run them during unit tests with loadPipelineScriptForTest(java.lang.String). Just call Script.run on the returned object, e.g.

node( "legacy" ) {
	echo( "hello world" )
}
 

def "Jenkinsfile"() {
	setup:
		def Jenkinsfile = loadPipelineScriptForTest("com/homeaway/CoolJenkinsfile.groovy")
	when:
		Jenkinsfile.run()
	then:
		1 * getPipelineMock("node")("legacy", _)
		1 * getPipelineMock("echo")("hello world")
}
 

Mock Pipeline Steps

Mock pipeline steps are available at getPipelineMock("stepName"). You can verify interactions with them and stub them:

...
then:
	1 * getPipelineMock( "echo" )( "hello world" )
	1 * getPipelineMock( "sh" )( [returnStdout: true, script: "echo hi"] ) >> "hi"
...
 

Step Bodies

Pipeline steps with body blocks will have their bodies executed after the invocation is registered with their mock. Any call on a pipeline step where the last argument is a Groovy Closure will result in that Closure being executed. This is true even for Pipeline steps that don't normally accept body blocks - be careful!

when:
	node( 'some-label' ) {
		echo( "hello" )
	}
then:
	1 * getPipelineMock("node")("some-label)
	1 * getPipelineMock("echo")("hello")
 

The parallel(...) step is special-cased and will execute all closures that are found as values of the Map provided to the step. The closures will be executed serially in an unspecified order (dependent on the platform's Set implementation used for the Map).

when:
	parallel(
		"stream 1" : {
			echo( "hello 1" )
		},
		"stream 2" : {
			echo( "hello 2" )
		}
	)
then:
	1 * getPipelineMock("echo")("hello 1")
	1 * getPipelineMock("echo")("hello 2")
 

Argument Capture

A quirk of Groovy's implementation of Closures and the choice of Closures as the interface to mock to provide pipeline step mocks changes Spock's argument-capture idiom: When a pipeline step is called with more than one argument, captured arguments are wrapped in an extra layer of Object[] and must be unwrapped.

def "single-argument capture" () {
	when:
		getPipelineMock("echo")("hello")
	then:
		1 * getPipelineMock("echo")(_) >> { _arguments ->
			assert "hello" == _arguments[0]
		}
}

def "multi-argument capture" () {
	when:
		getPipelineMock("stage")("label") {
			echo("body")
		}
	then:
		1 * getPipelineMock("stage")(_) >> { _arguments ->
			def args = _arguments[0]
			assert "label" == args[0] 
		}
}
 

Using Pipeline Variables

There are two ways that Jenkins can make a globally-accessible variable available to your Pipeline scripts:

1. Defining it before your script runs
2. A class that uses the GlobalVariable extension point

Mock GlobalVariables

Method calls on GlobalVariables are available as mocks at getPipelineMock("VariableName.methodName"). Because it can be impossible to tell which methods will be valid on a given variable at runtime all method calls on variables are allowed during tests and captured by a Spock Mock.

You can expect and stub interactions with these mocks normally:

...
then:
	1 * getPipelineMock( "docker.inside" )( "maven:latest" )
	1 * getPipelineMock( "JenkinsPluginSapling.hello" )( [who: "bob", greeting: "hi"] ) >> "hi bob" 
...
 

Implementation Note: If you access getPipelineMock("VariableName") you will find a PipelineVariableImpersonator that will create mocks on-the-fly (if necessary) for every method called on it. You shouldn't need to interact directly with these objects.

Properties of Mock Pipeline Variables

All property access attempts on pipeline variables will be forwarded to getPipelineMock("VariableName.getProperty")(propertyName). You can expect and stub these accesses normally:

...
when:
	String someEnvvar = env.someEnvvar
	String otherEnvvar = env.otherEnvvar
then:
	1 * getPipelineMock( "env.getProperty" )( "someEnvvar" )
	1 * getPipelineMock( "env.getProperty" )( "otherEnvvar" ) >> "expected"
expect:
	"expected" == otherEnvvar
...
 

Exception: You cannot stub .toString() of the Mock Pipeline Variable itself; getPipelineMock("someVar.toString") >> "hello" won't work, meaning if you've written String s = "${someVar} something" in your code-under-test, that can't be correctly stubbed. The immediate cause is that a useful PipelineVariableImpersonator.toString is already defined on PipelineVariableImpersonator, for understanding its presence in log messages. However, due to GROOVY-3493, we can't "fix" this with metaprogramming at test-time either.

Defining Pipeline Variables

There are some globally-accessible variables in pipeline scripts that Jenkins sets automatically. These variables do not need to be mocked. In fact, they cannot and should not be mocked! Just set dummy values when setting up your test. If the variable is used in a pipeline script that's being tested, set the variable on the script's Binding:

setup:
	def Jenkinsfile = loadPipelineScriptForTest("Jenkinsfile")
	Jenkinsfile.getBinding().setVariable( "BRANCH_NAME", "master" )
	Jenkinsfile.getBinding().setVariable( "env", [
		"foo": "bar",
		"fee": "fie"
	])
 

Mock Shared Library Variables

Any Groovy scripts on the classpath at /vars will be treated as Pipeline Shared Library Global Variables, and mocked as described above.

Unit-Testing PSL Variables

Pipeline Shared Library Global Variables are just pipeline scripts on the classpath, so they can be unit-tested with loadPipelineScriptForTest(java.lang.String):

setup:
	def MyFunction = loadPipelineScriptForTest("vars/MyFunction.groovy")
when:
	MyFunction("...")
then:
	1 * getPipelineMock("echo")("Hello World")
 

PSL Variable Mocks

The most common way to use Pipeline Shared Library Global Variables is to implicitly .call(...) them, e.g.

stage( "Do Something" )  {
	MyFunction("...")
}
 

To expect or stub that interaction, use getPipelineMock("FileName.call")(...), e.g.

setup:
	getPipelineMock("MyFunction.call")("Hello") >> "Hello World"
when:
	Jenkinsfile.run()
then:
	1 * getPipelineMock("echo")("Hello World")
 

when:
	Jenkinsfile.run()
then:
	1 * getPipelineMock("MyFunction.call")(_)
 

Preparing Additional Objects

By default, all non-interface classes in your project's source directories are instrumented to be able to call pipeline steps during each Specification test suite (see setupSpec() and setup()), along with the class that the specification is for (determined by class name, e.g. MyClassSpec is "for" MyClass). If you need to instrument additional objects, you can use addPipelineMocksToObjects(java.lang.Object). If you provide a Class object, all subsequent instances of that class will be instrumented.

Warning: Attempting to instrument an Interface class may have frustratingly permanent side-effects. Do not do that. Instead, read GROOVY-3493.

Mocking Additional Pipeline Steps

explicitlyMockPipelineStep(java.lang.String,java.lang.String)

If for some reason you need to mock a pipeline step that does not come from a detectable plugin, you can use explicitlyMockPipelineStep(java.lang.String,java.lang.String) to add the ability to call that pipeline step and get a Mock object to all currently-instrumented classes and objects.

Mocking Additional Pipeline Variables

explicitlyMockPipelineVariable(java.lang.String)

If for some reason you need to capture interactions with a variable that is not detected on the classpath and automatically mocked, you can use explicitlyMockPipelineVariable(java.lang.String) to create and return a PipelineVariableImpersonator that will intercept all subsequent interactions with that variable and forward them to appropriately-named Spock mocks, creating the mocks on-the-fly if necessary.

Mock Jenkins

1. Jenkins.HOLDER is implemented to return the mock Jenkins.
2. Jenkins.getInstance is stubbed to return the mock Jenkins
3. Jenkins.getInstanceOrNull is stubbed to return the mock Jenkins
4. Jenkins.getExtensionList is stubbed to return instances of the @Extensions of the provided type (actually, only the ExtensionList.iterator method is stubbed, so use a for/each loop).

Mock Static Jenkins

Before any tests run, jenkins-spock scans the classpath(s) for Jenkins Extensions and determines their names. The canonical name of a Jenkins Pipeline Step extension is reported by the StepDescriptor#getFunctionName()#getFunctionName() instance method. Therefore, the descriptor must classloaded and then instantiated in order to get the right name.

Some Jenkins extensions try to access the Jenkins singelton in static { ... } setup, or in their Descriptor's constructor. The mock Jenkins that is automatically created for every test case won't help here because no test cases are running yet: jenkins-spock is still setting up.

In case a test suite involves classes that behave this way, jenkins-spock automatically creates a separate, static Spock mock of the Jenkins singelton and injects it into the Jenkins class before any Extensions are classloaded and before any Descriptors are instantiated. This mock cannot be stubbed and its interactions cannot be verified because Spock mocks do not fully work outside a specification. All this mock can do is return null for every method call, but that may be enough for some Extensions.

Stubbing Static Jenkins

If an Extension on a test suite's classpath not only interacts with the Jenkins singleton at classload- or Descriptor-instantiation-time, but also tries to interact with return values from methods called on the Jenkins singleton, the test suite must stub interactions with the static Jenkins access. Jenkins-spock provides an override-able makeStaticJenkins() method to allow a test-suite to define its own Jenkins singleton object to use outside, and only outside the Spock test cases. You cannot stub a Spock mock of Jenkins here. Try Mockito.

makeStaticJenkins() will be called at most once by jenkins-spock; the result will be stored and re-used. Test-suites may call getStaticJenkins() to access the static Jenkins singleton. Spock test features should never call either of these methods.

Mock Pipeline Execution

A Mock of the pipeline execution's Binding will be available at getPipelineMock("getBinding"). Code-under-test might access this mock by calling CpsScript.getBinding. Usually, this is done by GlobalVariable implementations.

There is a Spock Spy of type CpsScript available at getPipelineMock("CpsScript"). It represents the cps-transformed execution that would exist if Jenkins were running your code for real. You should never have to interact directly with this object.

Architecture Note: CpsScript overrides CpsScript.invokeMethod and tries to invoke a CPS-transformed equivalent. Spock mocks won't override this particular method, and so any method invocations on a mock CpsScript will result in the real invokeMethod being called and CPS-transformed execution attempted... But during unit tests, we haven't CPS-transformed anything so this will fail unless we manually override invokeMethod on a "real" CpsScript object and redirect those method calls back to some "normal" GroovyObject.invokeMethod.

Metaprogramming

Groovy's metaprogramming capabilities are used heavily to make this class work. A methodMissing and propertyMissing method is dynamically created for every class that needs to be able to call pipeline steps. What happens if that class already had one of those methods defined? Trouble, that's what.

This class will detect such a situation, and call the other class' appropriate "missing" handler first. If that handler throws either a MissingMethodException or MissingPropertyException, then this class will proceed with the regular "maybe it was a pipeline step" logic. However, a class that "chomps" all missing events, like so:

def methodMissing(String _name, _args) {
	LOG.warn( "Called a missing method: ${_name}(${_args.toString()})" )
}
 

simply cannot be enabled to call pipeline steps during tests. Avoid writing classes like that.

Define Custom Classpath For Script Loading

When your project has resources in locations other than Maven's default locations, you need to set the paths for the loading of your scripts manually. For example, to you want to load a pipeline script that is located in `test/resources`:

 
class PipelineTest extends JenkinsPipelineSpecification {
	def setup() {
		scriptClassPath = ["test/resources"] //Note that this is a collection and you can define multiple paths.
	}

	def "Some test) {
		def pipeline = loadPipelineScriptForTest("Jenkinsfile") // Will test/resources/Jenkinsfile
		[...]
	}
}
 

• src/main/resources
• src/test/resources
• target/classes
• target/test-classes

Authors:

awitt

mld-ger