org.nd4j.imports.graphmapper

Interface GraphMapper<GRAPH_TYPE,NODE_TYPE,ATTR_TYPE,TENSOR_TYPE>

Type Parameters:

GRAPH_TYPE - the proto type for the graph

NODE_TYPE - the proto type for the node

ATTR_TYPE - the proto type for the attribute

TENSOR_TYPE - the proto type for the tensor

All Known Implementing Classes:

BaseGraphMapper, OnnxGraphMapper, TFGraphMapper

public interface GraphMapper<GRAPH_TYPE,NODE_TYPE,ATTR_TYPE,TENSOR_TYPE>

Map graph proto types to SameDiff instances

Author:

Adam Gibson

Method Summary

Modifier and Type	Method and Description
boolean	alreadySeen(NODE_TYPE nodeType)
DataType	dataTypeForTensor(TENSOR_TYPE tensorType, int outputNum)
void	dumpBinaryProtoAsText(File inputFile, File outputFile) Dump a binary proto file representation as a plain string in to the target text file
void	dumpBinaryProtoAsText(InputStream inputFile, File outputFile) Dump a binary proto file representation as a plain string in to the target text file
INDArray	getArrayFrom(NODE_TYPE nodeType, GRAPH_TYPE graph)
Map<String,ATTR_TYPE>	getAttrMap(NODE_TYPE nodeType) Get the attribute map for given node
String	getAttrValueFromNode(NODE_TYPE nodeType, String key)
List<String>	getControlDependencies(NODE_TYPE node) Get the list of control dependencies for the current node (or null if none exist)
String	getInputFromNode(NODE_TYPE node, int index) Get the input node for the given node
DifferentialFunction	getMappedOp(String name) Get the mapped op name for a given op relative to the type of node being mapped.
String	getName(NODE_TYPE nodeType) Get the name of the node
INDArray	getNDArrayFromTensor(String tensorName, TENSOR_TYPE tensorType, GRAPH_TYPE graph)
org.nd4j.shade.protobuf.Message.Builder	getNewGraphBuilder() Returns a graph builder for initial definition and parsing.
List<NODE_TYPE>	getNodeList(GRAPH_TYPE graphType)
NODE_TYPE	getNodeWithNameFromGraph(GRAPH_TYPE graph, String name) Get the node from the graph
String	getOpType(NODE_TYPE nodeType)
long[]	getShape(NODE_TYPE nodeType)
long[]	getShapeFromAttr(ATTR_TYPE attr) Get the shape of the attribute value
long[]	getShapeFromAttribute(ATTR_TYPE attrType)
long[]	getShapeFromTensor(TENSOR_TYPE tensorType) Get the shape for the given tensor type
String	getTargetMappingForOp(DifferentialFunction function, NODE_TYPE node) Get the target mapping key (usually based on the node name) for the given function
boolean	hasShape(NODE_TYPE nodeType)
SameDiff	importGraph(File graphFile) Import a graph as SameDiff from the given file
SameDiff	importGraph(File graphFile, Map<String,? extends OpImportOverride<GRAPH_TYPE,NODE_TYPE,ATTR_TYPE>> opImportOverrides, OpImportFilter<GRAPH_TYPE,NODE_TYPE,ATTR_TYPE> opFilter) Import a graph as SameDiff from the given file, with optional op import overrides. The OpImportOverride instances allow the operation import to be overridden - useful for importing ops that have not been mapped for import in SameDiff yet, and also for non-standard/user-defined functions.
SameDiff	importGraph(GRAPH_TYPE graph) This method converts given graph type (in its native format) to SameDiff
SameDiff	importGraph(GRAPH_TYPE graph, Map<String,? extends OpImportOverride<GRAPH_TYPE,NODE_TYPE,ATTR_TYPE>> opImportOverrides, OpImportFilter<GRAPH_TYPE,NODE_TYPE,ATTR_TYPE> opFilter) This method converts given graph type (in its native format) to SameDiff The OpImportOverride instances allow the operation import to be overridden - useful for importing ops that have not been mapped for import in SameDiff yet, and also for non-standard/user-defined functions.
SameDiff	importGraph(InputStream graphFile) Import a graph as SameDiff from the given file
SameDiff	importGraph(InputStream graphFile, Map<String,? extends OpImportOverride<GRAPH_TYPE,NODE_TYPE,ATTR_TYPE>> opImportOverrides, OpImportFilter<GRAPH_TYPE,NODE_TYPE,ATTR_TYPE> opFilter)
boolean	isConstant(TENSOR_TYPE nodeType) Returns true if the given node is a constant
boolean	isOpIgnoreException(NODE_TYPE node) Returns true if this node is a special case (maybe because of name or other scenarios) that should override opsToIgnore() in certain circumstances
boolean	isPlaceHolder(TENSOR_TYPE nodeType) Returns true if the given node is a place holder type (think a yet to be determined shape)_
boolean	isPlaceHolderNode(TENSOR_TYPE node) Returns true if the given node is a place holder
boolean	isStringType(TENSOR_TYPE tensor)
boolean	isVariableNode(NODE_TYPE nodeType)
void	mapNodeType(NODE_TYPE tfNode, ImportState<GRAPH_TYPE,TENSOR_TYPE> importState, OpImportOverride<GRAPH_TYPE,NODE_TYPE,ATTR_TYPE> opImportOverride, OpImportFilter<GRAPH_TYPE,NODE_TYPE,ATTR_TYPE> opFilter) Map a node in to the import state covering the SameDiff instance
void	mapProperties(DifferentialFunction on, NODE_TYPE node, GRAPH_TYPE graph, SameDiff sameDiff, Map<String,Map<String,PropertyMapping>> propertyMappings)
void	mapProperty(String name, DifferentialFunction on, NODE_TYPE node, GRAPH_TYPE graph, SameDiff sameDiff, Map<String,Map<String,PropertyMapping>> propertyMappingsForFunction)
Map<String,NODE_TYPE>	nameIndexForGraph(GRAPH_TYPE graph)
Map<String,NODE_TYPE>	nodesByName(GRAPH_TYPE graph) Get the nodes sorted by n ame from a given graph
int	numInputsFor(NODE_TYPE nodeType) Get the number of inputs for a node.
Set<String>	opsToIgnore() Ops to ignore for mapping
Op.Type	opTypeForNode(NODE_TYPE nodeType) Returns an op type for the given input node
GRAPH_TYPE	parseGraphFrom(byte[] inputStream) Parse a graph from an input stream
GRAPH_TYPE	parseGraphFrom(InputStream inputStream) Parse a graph from an input stream
boolean	shouldSkip(NODE_TYPE opType)
String	translateToSameDiffName(String name, NODE_TYPE node)
boolean	validTensorDataType(TENSOR_TYPE tensorType) Whether the data type for the tensor is valid for creating an INDArray
Map<String,TENSOR_TYPE>	variablesForGraph(GRAPH_TYPE graphType) Get the variables for the given graph

Method Detail

importGraph

SameDiff importGraph(InputStream graphFile)

Import a graph as SameDiff from the given file

Parameters:

graphFile - Input stream pointing to graph file to import

Returns:

Imported graph

importGraph

SameDiff importGraph(InputStream graphFile,
                     Map<String,? extends OpImportOverride<GRAPH_TYPE,NODE_TYPE,ATTR_TYPE>> opImportOverrides,
                     OpImportFilter<GRAPH_TYPE,NODE_TYPE,ATTR_TYPE> opFilter)

importGraph

SameDiff importGraph(File graphFile)

Import a graph as SameDiff from the given file

Parameters:

graphFile - Graph file to import

Returns:

Imported graph

See Also:

#importGraph(File, Map)

importGraph

SameDiff importGraph(File graphFile,
                     Map<String,? extends OpImportOverride<GRAPH_TYPE,NODE_TYPE,ATTR_TYPE>> opImportOverrides,
                     OpImportFilter<GRAPH_TYPE,NODE_TYPE,ATTR_TYPE> opFilter)

Import a graph as SameDiff from the given file, with optional op import overrides.
The OpImportOverride instances allow the operation import to be overridden - useful for importing ops that have not been mapped for import in SameDiff yet, and also for non-standard/user-defined functions.

Parameters:

graphFile - Graph file to import

opImportOverrides - May be null. If non-null: used to import the specified operations. Key is the name of the operation to import, value is the object used to import it

Returns:

Imported graph

importGraph

SameDiff importGraph(GRAPH_TYPE graph)

This method converts given graph type (in its native format) to SameDiff

Parameters:

graph - Graph to import

Returns:

Imported graph

importGraph

SameDiff importGraph(GRAPH_TYPE graph,
                     Map<String,? extends OpImportOverride<GRAPH_TYPE,NODE_TYPE,ATTR_TYPE>> opImportOverrides,
                     OpImportFilter<GRAPH_TYPE,NODE_TYPE,ATTR_TYPE> opFilter)

This method converts given graph type (in its native format) to SameDiff
The OpImportOverride instances allow the operation import to be overridden - useful for importing ops that have not been mapped for import in SameDiff yet, and also for non-standard/user-defined functions.

Parameters:

graph - Graph to import

Returns:

Imported graph

isOpIgnoreException

boolean isOpIgnoreException(NODE_TYPE node)

Returns true if this node is a special case (maybe because of name or other scenarios) that should override opsToIgnore() in certain circumstances

Parameters:

node - the node to check

Returns:

true if this node is an exception false otherwise

nodesByName

Map<String,NODE_TYPE> nodesByName(GRAPH_TYPE graph)

Get the nodes sorted by n ame from a given graph

Parameters:

graph - the graph to get the nodes for

Returns:

the map of the nodes by name for a given graph

getTargetMappingForOp

String getTargetMappingForOp(DifferentialFunction function,
                             NODE_TYPE node)

Get the target mapping key (usually based on the node name) for the given function

Parameters:

function - the function

node - the node to derive the target mapping from

Returns:

mapProperties

void mapProperties(DifferentialFunction on,
                   NODE_TYPE node,
                   GRAPH_TYPE graph,
                   SameDiff sameDiff,
                   Map<String,Map<String,PropertyMapping>> propertyMappings)

Parameters:

on -

node -

graph -

sameDiff -

propertyMappings -

mapProperty

void mapProperty(String name,
                 DifferentialFunction on,
                 NODE_TYPE node,
                 GRAPH_TYPE graph,
                 SameDiff sameDiff,
                 Map<String,Map<String,PropertyMapping>> propertyMappingsForFunction)

Parameters:

name -

on -

node -

graph -

sameDiff -

propertyMappingsForFunction -

getNodeWithNameFromGraph

NODE_TYPE getNodeWithNameFromGraph(GRAPH_TYPE graph,
                                   String name)

Get the node from the graph

Parameters:

graph - the graph to get the node from

name - the name of the node to get from the graph

Returns:

isPlaceHolderNode

boolean isPlaceHolderNode(TENSOR_TYPE node)

Returns true if the given node is a place holder

Parameters:

node - the node to check

Returns:

true if the node is a place holder or not

getControlDependencies

List<String> getControlDependencies(NODE_TYPE node)

Get the list of control dependencies for the current node (or null if none exist)

Parameters:

node - Node to get the control dependencies (if any) for

Returns:

dumpBinaryProtoAsText

void dumpBinaryProtoAsText(File inputFile,
                           File outputFile)

Dump a binary proto file representation as a plain string in to the target text file

Parameters:

inputFile -

outputFile -

dumpBinaryProtoAsText

void dumpBinaryProtoAsText(InputStream inputFile,
                           File outputFile)

Dump a binary proto file representation as a plain string in to the target text file

Parameters:

inputFile -

outputFile -

getMappedOp

DifferentialFunction getMappedOp(String name)

Get the mapped op name for a given op relative to the type of node being mapped. The input name should be based on a tensorflow type or onnx type, not the nd4j name

Parameters:

name - the tensorflow or onnx name

Returns:

the function based on the values in DifferentialFunctionClassHolder

variablesForGraph

Map<String,TENSOR_TYPE> variablesForGraph(GRAPH_TYPE graphType)

Get the variables for the given graph

Parameters:

graphType - the graph to get the variables for

Returns:

a map of variable name to tensor

translateToSameDiffName

String translateToSameDiffName(String name,
                               NODE_TYPE node)

Parameters:

name -

node -

Returns:

nameIndexForGraph

Map<String,NODE_TYPE> nameIndexForGraph(GRAPH_TYPE graph)

Parameters:

graph -

Returns:

opTypeForNode

Op.Type opTypeForNode(NODE_TYPE nodeType)

Returns an op type for the given input node

Parameters:

nodeType - the node to use

Returns:

the optype for the given node

getNewGraphBuilder

org.nd4j.shade.protobuf.Message.Builder getNewGraphBuilder()

Returns a graph builder for initial definition and parsing.

Returns:

parseGraphFrom

GRAPH_TYPE parseGraphFrom(byte[] inputStream)
                   throws IOException

Parse a graph from an input stream

Parameters:

inputStream - the input stream to load from

Returns:

Throws:

IOException

parseGraphFrom

GRAPH_TYPE parseGraphFrom(InputStream inputStream)
                   throws IOException

Parse a graph from an input stream

Parameters:

inputStream - the input stream to load from

Returns:

Throws:

IOException

mapNodeType

void mapNodeType(NODE_TYPE tfNode,
                 ImportState<GRAPH_TYPE,TENSOR_TYPE> importState,
                 OpImportOverride<GRAPH_TYPE,NODE_TYPE,ATTR_TYPE> opImportOverride,
                 OpImportFilter<GRAPH_TYPE,NODE_TYPE,ATTR_TYPE> opFilter)

Map a node in to the import state covering the SameDiff instance

Parameters:

tfNode - the node to map

importState - the current import state

opFilter - Optional filter for skipping operations

dataTypeForTensor

DataType dataTypeForTensor(TENSOR_TYPE tensorType,
                           int outputNum)

Parameters:

tensorType -

outputNum -

Returns:

isStringType

boolean isStringType(TENSOR_TYPE tensor)

getAttrValueFromNode

String getAttrValueFromNode(NODE_TYPE nodeType,
                            String key)

Parameters:

nodeType -

key -

Returns:

getShapeFromAttribute

long[] getShapeFromAttribute(ATTR_TYPE attrType)

Parameters:

attrType -

Returns:

isPlaceHolder

boolean isPlaceHolder(TENSOR_TYPE nodeType)

Returns true if the given node is a place holder type (think a yet to be determined shape)_

Parameters:

nodeType -

Returns:

isConstant

boolean isConstant(TENSOR_TYPE nodeType)

Returns true if the given node is a constant

Parameters:

nodeType -

Returns:

getNDArrayFromTensor

INDArray getNDArrayFromTensor(String tensorName,
                              TENSOR_TYPE tensorType,
                              GRAPH_TYPE graph)

Parameters:

tensorName -

tensorType -

graph -

Returns:

getShapeFromTensor

long[] getShapeFromTensor(TENSOR_TYPE tensorType)

Get the shape for the given tensor type

Parameters:

tensorType -

Returns:

opsToIgnore

Set<String> opsToIgnore()

Ops to ignore for mapping

Returns:

getInputFromNode

String getInputFromNode(NODE_TYPE node,
                        int index)

Get the input node for the given node

Parameters:

node - the node

index - hte index

Returns:

numInputsFor

int numInputsFor(NODE_TYPE nodeType)

Get the number of inputs for a node.

Parameters:

nodeType - the node to get the number of inputs for

Returns:

validTensorDataType

boolean validTensorDataType(TENSOR_TYPE tensorType)

Whether the data type for the tensor is valid for creating an INDArray

Parameters:

tensorType - the tensor proto to test

Returns:

getShapeFromAttr

long[] getShapeFromAttr(ATTR_TYPE attr)

Get the shape of the attribute value

Parameters:

attr - the attribute value

Returns:

the shape of the attribute if any or null

getAttrMap

Map<String,ATTR_TYPE> getAttrMap(NODE_TYPE nodeType)

Get the attribute map for given node

Parameters:

nodeType - the node

Returns:

the attribute map for the attribute

getName

String getName(NODE_TYPE nodeType)

Get the name of the node

Parameters:

nodeType - the node to get the name for

Returns:

alreadySeen

boolean alreadySeen(NODE_TYPE nodeType)

Parameters:

nodeType -

Returns:

isVariableNode

boolean isVariableNode(NODE_TYPE nodeType)

Parameters:

nodeType -

Returns:

shouldSkip

boolean shouldSkip(NODE_TYPE opType)

Parameters:

opType -

Returns:

hasShape

boolean hasShape(NODE_TYPE nodeType)

Parameters:

nodeType -

Returns:

getShape

long[] getShape(NODE_TYPE nodeType)

Parameters:

nodeType -

Returns:

getArrayFrom

INDArray getArrayFrom(NODE_TYPE nodeType,
                      GRAPH_TYPE graph)

Parameters:

nodeType -

graph -

Returns:

getOpType

String getOpType(NODE_TYPE nodeType)

getNodeList

List<NODE_TYPE> getNodeList(GRAPH_TYPE graphType)

Parameters:

graphType -

Returns: