Utility wrapper around assertArrayEquals that resolves the types
Utility wrapper around assertArrayEquals that resolves the types
Create an input stream for the provided input sequence.
Create an input stream for the provided input sequence. This is done using TestInputStream as queueStream's are not checkpointable.
Runs the streams set up in ssc
on manual clock for numBatches
batches and
returns the collected output.
Runs the streams set up in ssc
on manual clock for numBatches
batches and
returns the collected output. It will wait until numExpectedOutput
number of
output data has been collected or timeout (set by maxWaitTimeMillis
) is reached.
Returns a sequence of items for each RDD.
Set up required DStreams to test the binary operation using the sequence of input collections.
Set up required DStreams to test the binary operation using the sequence of input collections.
Set up required DStreams to test the DStream operation using the two sequences of input collections.
Set up required DStreams to test the DStream operation using the two sequences of input collections.
Test unary DStream operation with a list of inputs
Test unary DStream operation with a list of inputs
Sequence of input collections
Binary DStream operation to be applied to the 2 inputs
Sequence of expected output collections
Number of batches to run the operation for
Compare the output values with the expected output values as sets (order matters) or as lists (order does not matter)
Test unary DStream operation with a list of inputs, with number of batches to run same as the number of expected output values
Verify whether the output values after running a DStream operation is same as the expected output values, by comparing the output collections either as lists (order matters) or sets (order does not matter)
Run a block of code with the given StreamingContext and automatically stop the context when the block completes or when an exception is thrown.
Run a block of code with the given StreamingContext and automatically stop the context when the block completes or when an exception is thrown.
This is the base trait for Spark Streaming testsuites. This provides basic functionality to run user-defined set of input on user-defined stream operations, and verify the output. This implementation is designer to work with JUnit for java users Note: this always uses the manual clock