org.nd4j.imports.graphmapper.onnx

Class OnnxGraphMapper

java.lang.Object

org.nd4j.imports.graphmapper.BaseGraphMapper<OnnxProto3.GraphProto,OnnxProto3.NodeProto,OnnxProto3.AttributeProto,OnnxProto3.TypeProto.Tensor>

org.nd4j.imports.graphmapper.onnx.OnnxGraphMapper

All Implemented Interfaces:

GraphMapper<OnnxProto3.GraphProto,OnnxProto3.NodeProto,OnnxProto3.AttributeProto,OnnxProto3.TypeProto.Tensor>

public class OnnxGraphMapper
extends BaseGraphMapper<OnnxProto3.GraphProto,OnnxProto3.NodeProto,OnnxProto3.AttributeProto,OnnxProto3.TypeProto.Tensor>

A mapper for onnx graphs to SameDiff instances.

Author:

Adam Gibson

Constructor Summary

Constructors

Constructor and Description
OnnxGraphMapper()

Method Summary

Modifier and Type	Method and Description
protected void	addDummyTensor(String name, Map<String,OnnxProto3.TypeProto.Tensor> to)
boolean	alreadySeen(OnnxProto3.NodeProto nodeProto)
DataType	dataTypeForTensor(OnnxProto3.TypeProto.Tensor tensorProto, int outputNum)
void	dumpBinaryProtoAsText(File inputFile, File outputFile) Dump a binary proto file representation as a plain string in to the target text file
void	dumpBinaryProtoAsText(InputStream inputFile, File outputFile) Dump a binary proto file representation as a plain string in to the target text file
INDArray	getArrayFrom(OnnxProto3.NodeProto nodeProto, OnnxProto3.GraphProto graph)
Map<String,OnnxProto3.AttributeProto>	getAttrMap(OnnxProto3.NodeProto nodeProto) Get the attribute map for given node
String	getAttrValueFromNode(OnnxProto3.NodeProto nodeProto, String key)
List<String>	getControlDependencies(OnnxProto3.NodeProto node) Get the list of control dependencies for the current node (or null if none exist)
String	getInputFromNode(OnnxProto3.NodeProto node, int index) Get the input node for the given node
static OnnxGraphMapper	getInstance()
DifferentialFunction	getMappedOp(String name) Get the mapped op name for a given op relative to the type of node being mapped.
String	getName(OnnxProto3.NodeProto nodeProto) Get the name of the node
INDArray	getNDArrayFromTensor(String tensorName, OnnxProto3.TypeProto.Tensor tensorProto, OnnxProto3.GraphProto graph)
com.github.os72.protobuf351.Message.Builder	getNewGraphBuilder() Returns a graph builder for initial definition and parsing.
List<OnnxProto3.NodeProto>	getNodeList(OnnxProto3.GraphProto graphProto)
OnnxProto3.NodeProto	getNodeWithNameFromGraph(OnnxProto3.GraphProto graph, String name) Get the node from the graph
String	getOpType(OnnxProto3.NodeProto nodeProto)
long[]	getShape(OnnxProto3.NodeProto nodeProto)
long[]	getShapeFromAttr(OnnxProto3.AttributeProto attr) Get the shape of the attribute value
long[]	getShapeFromAttribute(OnnxProto3.AttributeProto attributeProto)
long[]	getShapeFromTensor(OnnxProto3.TensorProto tensorProto) Get the shape from a tensor proto.
long[]	getShapeFromTensor(OnnxProto3.TypeProto.Tensor tensorProto) Get the shape for the given tensor type
String	getTargetMappingForOp(DifferentialFunction function, OnnxProto3.NodeProto node) Get the target mapping key (usually based on the node name) for the given function
boolean	hasShape(OnnxProto3.NodeProto nodeProto)
void	initFunctionFromProperties(String mappedTfName, DifferentialFunction on, Map<String,OnnxProto3.AttributeProto> attributesForNode, OnnxProto3.NodeProto node, OnnxProto3.GraphProto graph) Init a function's attributes
boolean	isConstant(OnnxProto3.TypeProto.Tensor nodeType) Returns true if the given node is a constant
boolean	isOpIgnoreException(OnnxProto3.NodeProto node) Returns true if this node is a special case (maybe because of name or other scenarios) that should override GraphMapper.opsToIgnore() in certain circumstances
boolean	isPlaceHolder(OnnxProto3.TypeProto.Tensor nodeType) Returns true if the given node is a place holder type (think a yet to be determined shape)_
boolean	isPlaceHolderNode(OnnxProto3.TypeProto.Tensor node) Returns true if the given node is a place holder
boolean	isStringType(OnnxProto3.TypeProto.Tensor tensor)
boolean	isVariableNode(OnnxProto3.NodeProto nodeProto)
void	mapNodeType(OnnxProto3.NodeProto tfNode, ImportState<OnnxProto3.GraphProto,OnnxProto3.TypeProto.Tensor> importState, OpImportOverride<OnnxProto3.GraphProto,OnnxProto3.NodeProto,OnnxProto3.AttributeProto> opImportOverride, OpImportFilter<OnnxProto3.GraphProto,OnnxProto3.NodeProto,OnnxProto3.AttributeProto> opFilter) Map a node in to the import state covering the SameDiff instance
void	mapProperty(String name, DifferentialFunction on, OnnxProto3.NodeProto node, OnnxProto3.GraphProto graph, SameDiff sameDiff, Map<String,Map<String,PropertyMapping>> propertyMappingsForFunction)
INDArray	mapTensorProto(OnnxProto3.TensorProto tensor)
DataType	nd4jTypeFromOnnxType(OnnxProto3.TensorProto.DataType dataType) Convert an onnx type to the proper nd4j type
int	numInputsFor(OnnxProto3.NodeProto nodeProto) Get the number of inputs for a node.
Set<String>	opsToIgnore() Ops to ignore for mapping
OnnxProto3.GraphProto	parseGraphFrom(byte[] inputStream) Parse a graph from an input stream
OnnxProto3.GraphProto	parseGraphFrom(InputStream inputStream) Parse a graph from an input stream
boolean	shouldSkip(OnnxProto3.NodeProto opType)
String	translateToSameDiffName(String name, OnnxProto3.NodeProto node)
Map<String,OnnxProto3.TypeProto.Tensor>	variablesForGraph(OnnxProto3.GraphProto graphProto) Get the variables for the given graph

Methods inherited from class org.nd4j.imports.graphmapper.BaseGraphMapper

importGraph, importGraph, importGraph, importGraph, importGraph, importGraph, initOutputVariables, mapProperties, nameIndexForGraph, nodesByName, opTypeForNode, readGraph, validateGraphStructure, validTensorDataType

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

OnnxGraphMapper

public OnnxGraphMapper()

Method Detail

getInstance

public static OnnxGraphMapper getInstance()

dumpBinaryProtoAsText

public void dumpBinaryProtoAsText(InputStream inputFile,
                                  File outputFile)

Description copied from interface: GraphMapper

Dump a binary proto file representation as a plain string in to the target text file

initFunctionFromProperties

public void initFunctionFromProperties(String mappedTfName,
                                       DifferentialFunction on,
                                       Map<String,OnnxProto3.AttributeProto> attributesForNode,
                                       OnnxProto3.NodeProto node,
                                       OnnxProto3.GraphProto graph)

Init a function's attributes

Parameters:

mappedTfName - the onnx name to pick (sometimes ops have multiple names

on - the function to map

attributesForNode - the attributes for the node

node -

graph -

isOpIgnoreException

public boolean isOpIgnoreException(OnnxProto3.NodeProto node)

Description copied from interface: GraphMapper

Returns true if this node is a special case (maybe because of name or other scenarios) that should override GraphMapper.opsToIgnore() in certain circumstances

Parameters:

node - the node to check

Returns:

true if this node is an exception false otherwise

getTargetMappingForOp

public String getTargetMappingForOp(DifferentialFunction function,
                                    OnnxProto3.NodeProto node)

Description copied from interface: GraphMapper

Get the target mapping key (usually based on the node name) for the given function

Parameters:

function - the function

node - the node to derive the target mapping from

Returns:

mapProperty

public void mapProperty(String name,
                        DifferentialFunction on,
                        OnnxProto3.NodeProto node,
                        OnnxProto3.GraphProto graph,
                        SameDiff sameDiff,
                        Map<String,Map<String,PropertyMapping>> propertyMappingsForFunction)

getNodeWithNameFromGraph

public OnnxProto3.NodeProto getNodeWithNameFromGraph(OnnxProto3.GraphProto graph,
                                                     String name)

Description copied from interface: GraphMapper

Get the node from the graph

Parameters:

graph - the graph to get the node from

name - the name of the node to get from the graph

Returns:

isPlaceHolderNode

public boolean isPlaceHolderNode(OnnxProto3.TypeProto.Tensor node)

Description copied from interface: GraphMapper

Returns true if the given node is a place holder

Parameters:

node - the node to check

Returns:

true if the node is a place holder or not

getControlDependencies

public List<String> getControlDependencies(OnnxProto3.NodeProto node)

Description copied from interface: GraphMapper

Get the list of control dependencies for the current node (or null if none exist)

Parameters:

node - Node to get the control dependencies (if any) for

Returns:

dumpBinaryProtoAsText

public void dumpBinaryProtoAsText(File inputFile,
                                  File outputFile)

Description copied from interface: GraphMapper

Dump a binary proto file representation as a plain string in to the target text file

getMappedOp

public DifferentialFunction getMappedOp(String name)

Description copied from interface: GraphMapper

Get the mapped op name for a given op relative to the type of node being mapped. The input name should be based on a tensorflow type or onnx type, not the nd4j name

Parameters:

name - the tensorflow or onnx name

Returns:

variablesForGraph

public Map<String,OnnxProto3.TypeProto.Tensor> variablesForGraph(OnnxProto3.GraphProto graphProto)

Description copied from interface: GraphMapper

Get the variables for the given graph

Parameters:

graphProto - the graph to get the variables for

Returns:

a map of variable name to tensor

translateToSameDiffName

public String translateToSameDiffName(String name,
                                      OnnxProto3.NodeProto node)

Returns:

addDummyTensor

protected void addDummyTensor(String name,
                              Map<String,OnnxProto3.TypeProto.Tensor> to)

getNewGraphBuilder

public com.github.os72.protobuf351.Message.Builder getNewGraphBuilder()

Description copied from interface: GraphMapper

Returns a graph builder for initial definition and parsing.

Returns:

parseGraphFrom

public OnnxProto3.GraphProto parseGraphFrom(byte[] inputStream)
                                     throws IOException

Description copied from interface: GraphMapper

Parse a graph from an input stream

Parameters:

inputStream - the input stream to load from

Returns:

Throws:

IOException

parseGraphFrom

public OnnxProto3.GraphProto parseGraphFrom(InputStream inputStream)
                                     throws IOException

Description copied from interface: GraphMapper

Parse a graph from an input stream

Parameters:

inputStream - the input stream to load from

Returns:

Throws:

IOException

mapNodeType

public void mapNodeType(OnnxProto3.NodeProto tfNode,
                        ImportState<OnnxProto3.GraphProto,OnnxProto3.TypeProto.Tensor> importState,
                        OpImportOverride<OnnxProto3.GraphProto,OnnxProto3.NodeProto,OnnxProto3.AttributeProto> opImportOverride,
                        OpImportFilter<OnnxProto3.GraphProto,OnnxProto3.NodeProto,OnnxProto3.AttributeProto> opFilter)

Description copied from interface: GraphMapper

Map a node in to the import state covering the SameDiff instance

Parameters:

tfNode - the node to map

importState - the current import state

opFilter - Optional filter for skipping operations

dataTypeForTensor

public DataType dataTypeForTensor(OnnxProto3.TypeProto.Tensor tensorProto,
                                  int outputNum)

Returns:

isStringType

public boolean isStringType(OnnxProto3.TypeProto.Tensor tensor)

nd4jTypeFromOnnxType

public DataType nd4jTypeFromOnnxType(OnnxProto3.TensorProto.DataType dataType)

Convert an onnx type to the proper nd4j type

Parameters:

dataType - the data type to convert

Returns:

the nd4j type for the onnx type

getAttrValueFromNode

public String getAttrValueFromNode(OnnxProto3.NodeProto nodeProto,
                                   String key)

Returns:

getShapeFromAttribute

public long[] getShapeFromAttribute(OnnxProto3.AttributeProto attributeProto)

Returns:

isPlaceHolder

public boolean isPlaceHolder(OnnxProto3.TypeProto.Tensor nodeType)

Description copied from interface: GraphMapper

Returns true if the given node is a place holder type (think a yet to be determined shape)_

Returns:

isConstant

public boolean isConstant(OnnxProto3.TypeProto.Tensor nodeType)

Description copied from interface: GraphMapper

Returns true if the given node is a constant

Returns:

getNDArrayFromTensor

public INDArray getNDArrayFromTensor(String tensorName,
                                     OnnxProto3.TypeProto.Tensor tensorProto,
                                     OnnxProto3.GraphProto graph)

Returns:

mapTensorProto

public INDArray mapTensorProto(OnnxProto3.TensorProto tensor)

getShapeFromTensor

public long[] getShapeFromTensor(OnnxProto3.TypeProto.Tensor tensorProto)

Description copied from interface: GraphMapper

Get the shape for the given tensor type

Returns:

getShapeFromTensor

public long[] getShapeFromTensor(OnnxProto3.TensorProto tensorProto)

Get the shape from a tensor proto. Note that this is different from getShapeFromTensor(OnnxProto3.TensorProto)

Parameters:

tensorProto - the tensor to get the shape from

Returns:

opsToIgnore

public Set<String> opsToIgnore()

Description copied from interface: GraphMapper

Ops to ignore for mapping

Returns:

getInputFromNode

public String getInputFromNode(OnnxProto3.NodeProto node,
                               int index)

Description copied from interface: GraphMapper

Get the input node for the given node

Parameters:

node - the node

index - hte index

Returns:

numInputsFor

public int numInputsFor(OnnxProto3.NodeProto nodeProto)

Description copied from interface: GraphMapper

Get the number of inputs for a node.

Parameters:

nodeProto - the node to get the number of inputs for

Returns:

getShapeFromAttr

public long[] getShapeFromAttr(OnnxProto3.AttributeProto attr)

Description copied from interface: GraphMapper

Get the shape of the attribute value

Parameters:

attr - the attribute value

Returns:

the shape of the attribute if any or null

getAttrMap

public Map<String,OnnxProto3.AttributeProto> getAttrMap(OnnxProto3.NodeProto nodeProto)

Description copied from interface: GraphMapper

Get the attribute map for given node

Parameters:

nodeProto - the node

Returns:

the attribute map for the attribute

getName

public String getName(OnnxProto3.NodeProto nodeProto)

Description copied from interface: GraphMapper

Get the name of the node

Parameters:

nodeProto - the node to get the name for

Returns:

alreadySeen

public boolean alreadySeen(OnnxProto3.NodeProto nodeProto)

Returns:

isVariableNode

public boolean isVariableNode(OnnxProto3.NodeProto nodeProto)

Returns:

shouldSkip

public boolean shouldSkip(OnnxProto3.NodeProto opType)

Returns:

hasShape

public boolean hasShape(OnnxProto3.NodeProto nodeProto)

Returns:

getShape

public long[] getShape(OnnxProto3.NodeProto nodeProto)

Returns:

getArrayFrom

public INDArray getArrayFrom(OnnxProto3.NodeProto nodeProto,
                             OnnxProto3.GraphProto graph)

Returns:

getOpType

public String getOpType(OnnxProto3.NodeProto nodeProto)

getNodeList

public List<OnnxProto3.NodeProto> getNodeList(OnnxProto3.GraphProto graphProto)

Returns: