org.nd4j.autodiff.validation

Class OpValidation

java.lang.Object

org.nd4j.autodiff.validation.OpValidation

public class OpValidation
extends Object

Main test case runner for validating ops used in SameDiff.
This OpValidation class also collects test coverage information, to determine the op test coverage, for both op outputs and gradients/backprop.

Two types of test cases are supported:
1. TestCase - Can be used to check op outputs, as well as gradients
2. OpTestCase - Used to check the output(s) of a single op only

NOTE: For the op coverage information to work properly for ND4J tests, we need the op validation to be run as part of the OpValidationSuite test suite. * Otherwise, we could report coverage information before all test have run - underestimating coverage.

SINGLE OP TEST: OpValidation.validate(new OpTestCase(op).expectedOutputs(0, )) - OpTestCase checks the output values of a single op, no backprop/gradients
- Returns an error message if op failed, or NULL if op passed
SAMEDIFF TEST: OpValidation.validate(new TestCase(sameDiff).gradientCheck(true).expectedOutput("someVar", ))
- These tests can be used to check both gradients AND expected output, collecting coverage as required
- Returns an error message if op failed, or NULL if op passed
- Note gradientCheck(true) is the default
- Expected outputs are optional
- You can specify a function for validating the correctness of each output using TestCase.expected(String, Function)

Author:

Alex Black

Constructor Summary

Constructors

Constructor and Description
OpValidation()

Method Summary

Modifier and Type	Method and Description
static void	checkDeserializedEquality(SameDiff original, ByteBuffer bbSerialized, TestCase tc)
static void	collectTensorflowImportCoverage(SameDiff graph)
protected static void	equalConsideringNull(List<String> l1, List<String> l2, String msg, Object... args)
static void	logCoverageInformation(boolean logAdequatelyTested, boolean logInadequate, boolean logUnmappedLibnd4jOps, boolean logUntestedTFImport, boolean logUnmappedTFOps) Log the coverage information
static String	validate(OpTestCase testCase) Validate the outputs of a single op
static String	validate(TestCase testCase) Run test case
static String	validate(TestCase testCase, boolean exceptionsAsErrorMsg)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

OpValidation

public OpValidation()

Method Detail

validate

public static String validate(TestCase testCase)

Run test case

Parameters:

testCase - Test case to run

Returns:

NULL if test passes, or error message otherwise

validate

public static String validate(TestCase testCase,
                              boolean exceptionsAsErrorMsg)

checkDeserializedEquality

public static void checkDeserializedEquality(SameDiff original,
                                             ByteBuffer bbSerialized,
                                             TestCase tc)

equalConsideringNull

protected static void equalConsideringNull(List<String> l1,
                                           List<String> l2,
                                           String msg,
                                           Object... args)

validate

public static String validate(OpTestCase testCase)

Validate the outputs of a single op

Parameters:

testCase - Op test case to run

Returns:

NULL if test is OK, or an error message otherwise

collectTensorflowImportCoverage

public static void collectTensorflowImportCoverage(SameDiff graph)

logCoverageInformation

public static void logCoverageInformation(boolean logAdequatelyTested,
                                          boolean logInadequate,
                                          boolean logUnmappedLibnd4jOps,
                                          boolean logUntestedTFImport,
                                          boolean logUnmappedTFOps)

Log the coverage information

Parameters:

logAdequatelyTested - If true: log details of each op that has both forward and (if appropriate) backward tests

logInadequate - If false: log details of each op that does NOT have both forward and (if appropriate) backward tests