public class InferenceSession extends AbstractSession<INDArray,SameDiffOp>
SessionMemMgr
instance (for allocations/deallocations) and
also IdentityDependencyTracker
to track where arrays are actually used. The IdentityDependencyTracker tells
us when the array is no longer needed (i.e., has been "fully consumed" by all ops depending on it) accounting for the
fact that some operations, such as identity, enter, exit, etc, are "zero copy" for performance reasons.Modifier and Type | Class and Description |
---|---|
protected static class |
InferenceSession.ConstantDep |
static class |
InferenceSession.Dep |
protected static class |
InferenceSession.ExecDoneDep |
static class |
InferenceSession.OpDep |
protected static class |
InferenceSession.PlaceholderDep |
protected static class |
InferenceSession.ReqOutputDep |
protected static class |
InferenceSession.VariableDep |
AbstractSession.ExecStep, AbstractSession.ExecStepPredicate, AbstractSession.ExecType, AbstractSession.FrameIter, AbstractSession.VarId
Modifier and Type | Field and Description |
---|---|
protected static String |
KERAS_TRAIN_TEST |
dt, nodeOutputs, OUTER_FRAME, sameDiff, subgraph, subgraphOps, tensorArrays, zeroInputOpsInSubgraph
Constructor and Description |
---|
InferenceSession(SameDiff sameDiff) |
Modifier and Type | Method and Description |
---|---|
INDArray[] |
doExec(DifferentialFunction op,
AbstractSession.FrameIter outputFrameIter,
Set<AbstractSession.VarId> opInputs,
Set<AbstractSession.VarId> allIterInputs,
Set<String> constAndPhInputs) |
SameDiffOp |
getAndParameterizeOp(String opName,
AbstractSession.FrameIter frameIter,
Set<AbstractSession.VarId> opInputs,
Set<AbstractSession.VarId> allIterInputs,
Set<String> constAndPhInputs,
Map<String,INDArray> placeholderValues,
Set<String> allReqVariables)
Get the parameterized op to execute - for example, the op/DifferentialFunction with all inputs set
|
protected INDArray |
getArray(SDVariable sdv,
Collection<AbstractSession.VarId> opInputs,
Collection<AbstractSession.VarId> allIterInputs) |
INDArray |
getConstantOrVariable(String variableName)
Get the constant or variable output - for example, constant array or constant shape.
|
INDArray[] |
getOutputs(SameDiffOp op,
AbstractSession.FrameIter outputFrameIter,
Set<AbstractSession.VarId> opInputs,
Set<AbstractSession.VarId> allIterInputs,
Set<String> constAndPhInputs,
List<Listener> listeners,
At at,
MultiDataSet batch,
Set<String> allReqVariables)
Execute the op - calculate INDArrays, or shape info, etc
|
INDArray[] |
getOutputsHelperTensorArrayOps(DifferentialFunction op,
AbstractSession.FrameIter outputFrameIter,
Set<AbstractSession.VarId> opInputs,
Set<AbstractSession.VarId> allIterInputs)
Forward pass for TensorArray ops
|
protected Map<String,INDArray> |
postProcessOutput(Map<String,INDArray> output)
Post process the session output values, if required.
|
protected Map<String,INDArray> |
preprocessPlaceholders(Map<String,INDArray> placeholders,
At at)
Preprocess the placeholder values, if required.
|
addDependenciesForOp, addVarControlDeps, contains, execFailed, get, get, getExecStepForVar, initSubgraph, lookup, lookup, output, updateDescendantDeps
protected static final String KERAS_TRAIN_TEST
public InferenceSession(@NonNull SameDiff sameDiff)
protected Map<String,INDArray> preprocessPlaceholders(Map<String,INDArray> placeholders, At at)
AbstractSession
preprocessPlaceholders
in class AbstractSession<INDArray,SameDiffOp>
placeholders
- Placeholders to preprocess.protected Map<String,INDArray> postProcessOutput(Map<String,INDArray> output)
AbstractSession
postProcessOutput
in class AbstractSession<INDArray,SameDiffOp>
output
- Output to be returned to the userpublic INDArray[] getOutputs(SameDiffOp op, AbstractSession.FrameIter outputFrameIter, Set<AbstractSession.VarId> opInputs, Set<AbstractSession.VarId> allIterInputs, Set<String> constAndPhInputs, List<Listener> listeners, At at, MultiDataSet batch, Set<String> allReqVariables)
AbstractSession
getOutputs
in class AbstractSession<INDArray,SameDiffOp>
op
- Operation to exit. This should be parameterized (i.e., all inputs set)outputFrameIter
- The frame and iteration of the outputsopInputs
- The specific input arrays for the opallReqVariables
- All required variables requested for the current session execution (not just the current op outputs)public INDArray[] doExec(DifferentialFunction op, AbstractSession.FrameIter outputFrameIter, Set<AbstractSession.VarId> opInputs, Set<AbstractSession.VarId> allIterInputs, Set<String> constAndPhInputs)
public INDArray[] getOutputsHelperTensorArrayOps(DifferentialFunction op, AbstractSession.FrameIter outputFrameIter, Set<AbstractSession.VarId> opInputs, Set<AbstractSession.VarId> allIterInputs)
public INDArray getConstantOrVariable(String variableName)
AbstractSession
getConstantOrVariable
in class AbstractSession<INDArray,SameDiffOp>
variableName
- The name of the variable to get the constant forpublic SameDiffOp getAndParameterizeOp(String opName, AbstractSession.FrameIter frameIter, Set<AbstractSession.VarId> opInputs, Set<AbstractSession.VarId> allIterInputs, Set<String> constAndPhInputs, Map<String,INDArray> placeholderValues, Set<String> allReqVariables)
AbstractSession
getAndParameterizeOp
in class AbstractSession<INDArray,SameDiffOp>
opName
- Name of the opframeIter
- The frame and iteration of the op outputsopInputs
- The inputs to the op (excluding constants/placeholders) - for the specific frame + iterationallIterInputs
- The inputs - those that are not iteration-specific (mainly Enter op vars, which might be used in all iterations but are only executed once on iter 0)constAndPhInputs
- The constant and placeholder inputs - used for all frames/iterationsallReqVariables
- All required variables requested for the current session execution (not just the current op outputs)protected INDArray getArray(SDVariable sdv, Collection<AbstractSession.VarId> opInputs, Collection<AbstractSession.VarId> allIterInputs)
Copyright © 2019. All rights reserved.