InferenceModel

Instance Constructors

new InferenceModel(autoScalingEnabled: Boolean, concurrentNum: Int)

create an InferenceModel with specified autoScalingEnabled, supportedConcurrentNum and maxConcurrentNum
create an InferenceModel with specified autoScalingEnabled, supportedConcurrentNum and maxConcurrentNum
autoScalingEnabled
if auto-scaling is enabled
concurrentNum
the concurrentNum of the InferenceModel
returns
a specified InferenceModel
new InferenceModel(concurrentNum: Int)

create an auto-scaling disabled InferenceModel with supportedConcurrentNum
create an auto-scaling disabled InferenceModel with supportedConcurrentNum
concurrentNum
the concurrentNum of the InferenceModel
returns
an auto-scaling disabled InferenceModel
new InferenceModel()

default constructor, will create a InferenceModel with auto-scaling enabled.
default constructor, will create a InferenceModel with auto-scaling enabled.
returns
an auto-scaling enabled InferenceModel
new InferenceModel(autoScalingEnabled: Boolean = true, concurrentNum: Int = 20, originalModel: AbstractModel = null, modelQueue: LinkedBlockingQueue[AbstractModel] = null)

Value Members

final def !=(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def ##(): Int

Definition Classes
AnyRef → Any
final def ==(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def asInstanceOf[T0]: T0

Definition Classes
Any
def clearWeightBias(model: Module[Float]): Unit

Definition Classes
InferenceSupportive
def clone(): AnyRef

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( ... )
def doLoad(modelPath: String, weightPath: String = null): Unit

loads a bigdl, analytics-zoo model
loads a bigdl, analytics-zoo model
modelPath
the file path of the model
weightPath
the file path of the weights
def doLoadCaffe(modelPath: String, weightPath: String): Unit

loads a caffe model
loads a caffe model
modelPath
the path of the prototxt file
weightPath
the path of the caffemodel file
def doLoadOpenVINO(modelPath: String, weightPath: String): Unit

loads a openvino IR
loads a openvino IR
modelPath
the path of openvino ir xml file
weightPath
the path of openvino ir bin file
def doLoadOpenVINOInt8(modelPath: String, weightPath: String, batchSize: Int): Unit

loads a openvino IR Int8
loads a openvino IR Int8
modelPath
the path of openvino ir xml file
weightPath
the path of openvino ir bin file
def doLoadTF(modelPath: String, imageClassificationModelType: String, checkpointPath: String, inputShape: Array[Int], ifReverseInputChannels: Boolean, meanValues: Array[Float], scale: Float): Unit

load TF model as OpenVINO IR
load TF model as OpenVINO IR
modelPath
the path of the tensorflow model
imageClassificationModelType
the type of the tensorflow model
checkpointPath
the path of the tensorflow checkpoint file
inputShape
input shape that should be fed to an input node(s) of the model
ifReverseInputChannels
the boolean value of if need reverse input channels. switch the input channels order from RGB to BGR (or vice versa).
meanValues
all input values coming from original network inputs will be divided by this value.
scale
the scale value, to be used for the input image per channel.
def doLoadTF(modelPath: String, objectDetectionModelType: String, pipelineConfigPath: String, extensionsConfigPath: String): Unit

loads a TF model as OpenVINO
loads a TF model as OpenVINO
modelPath
the path of the tensorflow model
objectDetectionModelType
the type of the tensorflow model, please refer to ModelType e.g. faster_rcnn_resnet101_coco, mask_rcnn_inception_v2_coco, rfcn_resnet101_coco, ssd_inception_v2_coco
pipelineConfigPath
the path of the pipeline configure file
extensionsConfigPath
the path of the extensions configure file
def doLoadTF(modelPath: String, pipelineConfigPath: String, extensionsConfigPath: String): Unit

loads a TF model as OpenVINO
loads a TF model as OpenVINO
modelPath
the path of the tensorflow model
pipelineConfigPath
the path of the pipeline configure file
extensionsConfigPath
the path of the extensions configure file
def doLoadTF(modelPath: String, objectDetectionModelType: String): Unit

loads a TF model as OpenVINO
loads a TF model as OpenVINO
modelPath
the path of the tensorflow model
objectDetectionModelType
the type of the tensorflow model, please refer to ModelType e.g. faster_rcnn_resnet101_coco, mask_rcnn_inception_v2_coco, rfcn_resnet101_coco, ssd_inception_v2_coco
def doLoadTF(modelPath: String, intraOpParallelismThreads: Int, interOpParallelismThreads: Int, usePerSessionThreads: Boolean): Unit

loads a TF model as TFNet
loads a TF model as TFNet
modelPath
the path of the tensorflow model
intraOpParallelismThreads
the num of intraOpParallelismThreads
interOpParallelismThreads
the num of interOpParallelismThreads
usePerSessionThreads
whether to perSessionThreads
def doLoadTF(modelPath: String): Unit

loads a TF model as TFNet
loads a TF model as TFNet
modelPath
the path of the tensorflow model file
def doLoadTFAsCalibratedOpenVINO(modelPath: String, modelType: String, checkpointPath: String, inputShape: Array[Int], ifReverseInputChannels: Boolean, meanValues: Array[Float], scale: Float, networkType: String, validationFilePath: String, subset: Int, opencvLibPath: String): Unit

load TF model as Calibrated OpenVINO IR
load TF model as Calibrated OpenVINO IR
modelPath
the path of the tensorflow model
modelType
the type of the tensorflow model
checkpointPath
the path of the tensorflow checkpoint file
inputShape
input shape that should be fed to an input node(s) of the model
ifReverseInputChannels
the boolean value of if need reverse input channels. switch the input channels order from RGB to BGR (or vice versa).
meanValues
all input values coming from original network inputs will be divided by this value.
scale
the scale value, to be used for the input image per channel.
networkType
Type of an inferred network, "C" to calibrate Classification, "OD" to calibrate Object Detection, "RawC" to collect only statistics for Classification, "RawOD" to collect only statistics for Object Detection
validationFilePath
Path to a file with validation images
subset
Number of pictures from the whole validation set to create the calibration dataset.
opencvLibPath
the lib path whwere libopencv_imgcodecs.so.4.0, libopencv_core.so.4.0 and libopencv_imgproc.so.4.0 can be found
def doPredict(inputActivity: Activity): Activity

predicts the inference result
predicts the inference result
inputActivity
the input activity
returns
the output activity
def doPredict(inputs: List[List[JTensor]]): List[List[JTensor]]

predicts the inference result
predicts the inference result
inputs
the input tensor with batch
returns
the output tensor with batch
def doPredictInt8(inputActivity: Activity): Activity

predicts int8 the inference result
predicts int8 the inference result
inputActivity
the input activity
returns
the output activity
def doPredictInt8(inputs: List[List[JTensor]]): List[List[JTensor]]

predicts int8 the inference result
predicts int8 the inference result
inputs
the input tensor with batch
returns
the output tensor with batch
def doReload(modelPath: String, weightPath: String): Unit

reloads the bigdl, analytics-zoo model
reloads the bigdl, analytics-zoo model
modelPath
the file path of the model
weightPath
the file path of the weights
final def eq(arg0: AnyRef): Boolean

Definition Classes
AnyRef
def equals(arg0: Any): Boolean

Definition Classes
AnyRef → Any
def finalize(): Unit

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( classOf[java.lang.Throwable] )
final def getClass(): Class[_]

Definition Classes
AnyRef → Any
def getOriginalModel: AbstractModel
def hashCode(): Int

Definition Classes
AnyRef → Any
final def isInstanceOf[T0]: Boolean

Definition Classes
Any
def makeMetaModel(original: AbstractModule[Activity, Activity, Float]): AbstractModule[Activity, Activity, Float]

Definition Classes
InferenceSupportive
def makeUpModel(clonedModel: Module[Float], weightBias: Array[Tensor[Float]]): AbstractModule[Activity, Activity, Float]

Definition Classes
InferenceSupportive
final def ne(arg0: AnyRef): Boolean

Definition Classes
AnyRef
final def notify(): Unit

Definition Classes
AnyRef
final def notifyAll(): Unit

Definition Classes
AnyRef
def release(): Unit

release original model and all the cloned ones in the queue
def releaseWeightBias(model: Module[Float]): Unit

Definition Classes
InferenceSupportive
final def synchronized[T0](arg0: ⇒ T0): T0

Definition Classes
AnyRef
def timing[T](name: String)(f: ⇒ T): T

Definition Classes
InferenceSupportive
def toString(): String

Definition Classes
InferenceModel → AnyRef → Any
def transferBatchTableToJListOfJListOfJTensor(batchTable: Table, batchSize: Int): List[List[JTensor]]

Definition Classes
InferenceSupportive
def transferBatchTensorToJListOfJListOfJTensor(batchTensor: Tensor[Float], batchSize: Int): List[List[JTensor]]

Definition Classes
InferenceSupportive
def transferListOfActivityToActivityOfBatch(inputs: List[List[JTensor]], batchSize: Int): Activity

Definition Classes
InferenceSupportive
def transferTensorToJTensor(input: Tensor[Float]): JTensor

Definition Classes
InferenceSupportive
def transferTensorsToTensorOfBatch(tensors: Array[JTensor]): Tensor[Float]

Definition Classes
InferenceSupportive
final def wait(): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long, arg1: Int): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )

Deprecated Value Members

def doPredict(input: List[Float], shape: List[Integer]): List[Float]

Annotations
@deprecated
Deprecated

Related Docs: object InferenceModel | package inference

class InferenceModel extends InferenceSupportive with Serializable

Instance Constructors

new InferenceModel(autoScalingEnabled: Boolean, concurrentNum: Int)

new InferenceModel(concurrentNum: Int)

new InferenceModel()

new InferenceModel(autoScalingEnabled: Boolean = true, concurrentNum: Int = 20, originalModel: AbstractModel = null, modelQueue: LinkedBlockingQueue[AbstractModel] = null)

Value Members

final def !=(arg0: Any): Boolean

final def ##(): Int

final def ==(arg0: Any): Boolean

final def asInstanceOf[T0]: T0

def clearWeightBias(model: Module[Float]): Unit

def clone(): AnyRef

def doLoad(modelPath: String, weightPath: String = null): Unit

def doLoadCaffe(modelPath: String, weightPath: String): Unit

def doLoadOpenVINO(modelPath: String, weightPath: String): Unit

def doLoadOpenVINOInt8(modelPath: String, weightPath: String, batchSize: Int): Unit

def doLoadTF(modelPath: String, imageClassificationModelType: String, checkpointPath: String, inputShape: Array[Int], ifReverseInputChannels: Boolean, meanValues: Array[Float], scale: Float): Unit

def doLoadTF(modelPath: String, objectDetectionModelType: String, pipelineConfigPath: String, extensionsConfigPath: String): Unit

def doLoadTF(modelPath: String, pipelineConfigPath: String, extensionsConfigPath: String): Unit

def doLoadTF(modelPath: String, objectDetectionModelType: String): Unit

def doLoadTF(modelPath: String, intraOpParallelismThreads: Int, interOpParallelismThreads: Int, usePerSessionThreads: Boolean): Unit

def doLoadTF(modelPath: String): Unit

def doLoadTFAsCalibratedOpenVINO(modelPath: String, modelType: String, checkpointPath: String, inputShape: Array[Int], ifReverseInputChannels: Boolean, meanValues: Array[Float], scale: Float, networkType: String, validationFilePath: String, subset: Int, opencvLibPath: String): Unit

def doPredict(inputActivity: Activity): Activity

def doPredict(inputs: List[List[JTensor]]): List[List[JTensor]]

def doPredictInt8(inputActivity: Activity): Activity

def doPredictInt8(inputs: List[List[JTensor]]): List[List[JTensor]]

def doReload(modelPath: String, weightPath: String): Unit

final def eq(arg0: AnyRef): Boolean

def equals(arg0: Any): Boolean

def finalize(): Unit

final def getClass(): Class[_]

def getOriginalModel: AbstractModel

def hashCode(): Int

final def isInstanceOf[T0]: Boolean

def makeMetaModel(original: AbstractModule[Activity, Activity, Float]): AbstractModule[Activity, Activity, Float]

def makeUpModel(clonedModel: Module[Float], weightBias: Array[Tensor[Float]]): AbstractModule[Activity, Activity, Float]

final def ne(arg0: AnyRef): Boolean

final def notify(): Unit

final def notifyAll(): Unit

def release(): Unit

def releaseWeightBias(model: Module[Float]): Unit

final def synchronized[T0](arg0: ⇒ T0): T0

def timing[T](name: String)(f: ⇒ T): T

def toString(): String

def transferBatchTableToJListOfJListOfJTensor(batchTable: Table, batchSize: Int): List[List[JTensor]]

def transferBatchTensorToJListOfJListOfJTensor(batchTensor: Tensor[Float], batchSize: Int): List[List[JTensor]]

def transferListOfActivityToActivityOfBatch(inputs: List[List[JTensor]], batchSize: Int): Activity

def transferTensorToJTensor(input: Tensor[Float]): JTensor

def transferTensorsToTensorOfBatch(tensors: Array[JTensor]): Tensor[Float]

final def wait(): Unit

final def wait(arg0: Long, arg1: Int): Unit

final def wait(arg0: Long): Unit

Deprecated Value Members

def doPredict(input: List[Float], shape: List[Integer]): List[Float]

Inherited from Serializable

Inherited from Serializable

Inherited from InferenceSupportive

Inherited from AnyRef

Inherited from Any

Ungrouped