ai.onnxruntime

Class OrtSession.SessionOptions

java.lang.Object

ai.onnxruntime.OrtSession.SessionOptions

All Implemented Interfaces:

java.lang.AutoCloseable

Enclosing class:

OrtSession

public static class OrtSession.SessionOptions
extends java.lang.Object
implements java.lang.AutoCloseable

Represents the options used to construct this session.

Used to set the number of threads, optimisation level, computation backend and other options.

Modifying this after the session has been constructed will have no effect.

The SessionOptions object must not be closed until all sessions which use it are closed, as otherwise it could release resources that are in use.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	OrtSession.SessionOptions.ExecutionMode The execution mode to use.
static class	OrtSession.SessionOptions.OptLevel The optimisation level to use.

Constructor Summary

Constructors

Constructor and Description
SessionOptions() Create an empty session options.

Method Summary

Modifier and Type	Method and Description
void	addACL(boolean useArena) Adds the ARM Compute Library as an execution backend.
void	addConfigEntry(java.lang.String configKey, java.lang.String configValue) Adds a single session configuration entry as a pair of strings.
void	addCPU(boolean useArena) Adds the CPU as an execution backend, using the arena allocator if desired.
void	addCUDA() Add CUDA as an execution backend, using device 0.
void	addCUDA(int deviceNum) Add CUDA as an execution backend, using the specified CUDA device id.
void	addDirectML(int deviceId) Adds DirectML as an execution backend.
void	addDnnl(boolean useArena) Adds Intel's Deep Neural Network Library as an execution backend.
void	addNnapi() Adds Android's NNAPI as an execution backend.
void	addNuphar(boolean allowUnalignedBuffers, java.lang.String settings) Adds Nuphar as an execution backend.
void	addOpenVINO(java.lang.String deviceId) Adds OpenVINO as an execution backend.
void	addTensorrt(int deviceNum) Adds Nvidia's TensorRT as an execution backend.
void	close() Closes the session options, releasing any memory acquired.
void	disableProfiling() Disables profiling in sessions using this SessionOptions.
void	enableProfiling(java.lang.String filePath) Enables profiling in sessions using this SessionOptions.
java.util.Map<java.lang.String,java.lang.String>	getConfigEntries() Returns an unmodifiable view of the map contains all session configuration entries.
void	registerCustomOpLibrary(java.lang.String path) Registers a library of custom ops for use with OrtSessions using this SessionOptions.
void	setCPUArenaAllocator(boolean useArena) Sets the CPU to use an arena memory allocator.
void	setExecutionMode(OrtSession.SessionOptions.ExecutionMode mode) Sets the execution mode of this options object, overriding the old setting.
void	setInterOpNumThreads(int numThreads) Sets the size of the CPU thread pool used for executing multiple request concurrently, if executing on a CPU.
void	setIntraOpNumThreads(int numThreads) Sets the size of the CPU thread pool used for executing a single graph, if executing on a CPU.
void	setLoggerId(java.lang.String loggerId) Sets the logger id to use.
void	setMemoryPatternOptimization(boolean memoryPatternOptimization) Turns on memory pattern optimizations, where memory is preallocated if all shapes are known.
void	setOptimizationLevel(OrtSession.SessionOptions.OptLevel level) Sets the optimization level of this options object, overriding the old setting.
void	setOptimizedModelFilePath(java.lang.String outputPath) Sets the output path for the optimized model.
void	setSessionLogLevel(OrtLoggingLevel logLevel) Sets the Session's logging level.
void	setSessionLogVerbosityLevel(int logLevel) Sets the Session's logging verbosity level.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SessionOptions

public SessionOptions()

Create an empty session options.

Method Detail

close

public void close()

Closes the session options, releasing any memory acquired.

Specified by:

close in interface java.lang.AutoCloseable

setExecutionMode

public void setExecutionMode(OrtSession.SessionOptions.ExecutionMode mode)
                      throws OrtException

Sets the execution mode of this options object, overriding the old setting.

Parameters:

mode - The execution mode to use.

Throws:

OrtException - If there was an error in native code.

setOptimizationLevel

public void setOptimizationLevel(OrtSession.SessionOptions.OptLevel level)
                          throws OrtException

Sets the optimization level of this options object, overriding the old setting.

Parameters:

level - The optimization level to use.

Throws:

OrtException - If there was an error in native code.

setInterOpNumThreads

public void setInterOpNumThreads(int numThreads)
                          throws OrtException

Sets the size of the CPU thread pool used for executing multiple request concurrently, if executing on a CPU.

Parameters:

numThreads - The number of threads to use.

Throws:

OrtException - If there was an error in native code.

setIntraOpNumThreads

public void setIntraOpNumThreads(int numThreads)
                          throws OrtException

Sets the size of the CPU thread pool used for executing a single graph, if executing on a CPU.

Parameters:

numThreads - The number of threads to use.

Throws:

OrtException - If there was an error in native code.

setOptimizedModelFilePath

public void setOptimizedModelFilePath(java.lang.String outputPath)
                               throws OrtException

Sets the output path for the optimized model.

Parameters:

outputPath - The output path to write the model to.

Throws:

OrtException - If there was an error in native code.

setLoggerId

public void setLoggerId(java.lang.String loggerId)
                 throws OrtException

Sets the logger id to use.

Parameters:

loggerId - The logger id string.

Throws:

OrtException - If there was an error in native code.

enableProfiling

public void enableProfiling(java.lang.String filePath)
                     throws OrtException

Enables profiling in sessions using this SessionOptions.

Parameters:

filePath - The file to write profile information to.

Throws:

OrtException - If there was an error in native code.

disableProfiling

public void disableProfiling()
                      throws OrtException

Disables profiling in sessions using this SessionOptions.

Throws:

OrtException - If there was an error in native code.

setMemoryPatternOptimization

public void setMemoryPatternOptimization(boolean memoryPatternOptimization)
                                  throws OrtException

Turns on memory pattern optimizations, where memory is preallocated if all shapes are known.

Parameters:

memoryPatternOptimization - If true enable memory pattern optimizations.

Throws:

OrtException - If there was an error in native code.

setCPUArenaAllocator

public void setCPUArenaAllocator(boolean useArena)
                          throws OrtException

Sets the CPU to use an arena memory allocator.

Parameters:

useArena - If true use an arena memory allocator for the CPU execution provider.

Throws:

OrtException - If there was an error in native code.

setSessionLogLevel

public void setSessionLogLevel(OrtLoggingLevel logLevel)
                        throws OrtException

Sets the Session's logging level.

Parameters:

logLevel - The log level to use.

Throws:

OrtException - If there was an error in native code.

setSessionLogVerbosityLevel

public void setSessionLogVerbosityLevel(int logLevel)
                                 throws OrtException

Sets the Session's logging verbosity level.

Parameters:

logLevel - The logging verbosity to use.

Throws:

OrtException - If there was an error in native code.

registerCustomOpLibrary

public void registerCustomOpLibrary(java.lang.String path)
                             throws OrtException

Registers a library of custom ops for use with OrtSessions using this SessionOptions.

Parameters:

path - The path to the library on disk.

Throws:

OrtException - If there was an error loading the library.

addConfigEntry

public void addConfigEntry(java.lang.String configKey,
                           java.lang.String configValue)
                    throws OrtException

Adds a single session configuration entry as a pair of strings.

Parameters:

configKey - The config key string.

configValue - The config value string.

Throws:

OrtException - If there was an error in native code.

getConfigEntries

public java.util.Map<java.lang.String,java.lang.String> getConfigEntries()

Returns an unmodifiable view of the map contains all session configuration entries.

Returns:

All session configuration entries

addCUDA

public void addCUDA()
             throws OrtException

Add CUDA as an execution backend, using device 0.

Throws:

OrtException - If there was an error in native code.

addCUDA

public void addCUDA(int deviceNum)
             throws OrtException

Add CUDA as an execution backend, using the specified CUDA device id.

Parameters:

deviceNum - The CUDA device id.

Throws:

OrtException - If there was an error in native code.

addCPU

public void addCPU(boolean useArena)
            throws OrtException

Adds the CPU as an execution backend, using the arena allocator if desired.

By default this backend is used, but if other backends are requested, it should be requested last.

Parameters:

useArena - If true use the arena memory allocator.

Throws:

OrtException - If there was an error in native code.

addDnnl

public void addDnnl(boolean useArena)
             throws OrtException

Adds Intel's Deep Neural Network Library as an execution backend.

Parameters:

useArena - If true use the arena memory allocator.

Throws:

OrtException - If there was an error in native code.

addOpenVINO

public void addOpenVINO(java.lang.String deviceId)
                 throws OrtException

Adds OpenVINO as an execution backend.

Parameters:

deviceId - The id of the OpenVINO execution device.

Throws:

OrtException - If there was an error in native code.

addTensorrt

public void addTensorrt(int deviceNum)
                 throws OrtException

Adds Nvidia's TensorRT as an execution backend.

Parameters:

deviceNum - The id of the CUDA device.

Throws:

OrtException - If there was an error in native code.

addNnapi

public void addNnapi()
              throws OrtException

Adds Android's NNAPI as an execution backend.

Throws:

OrtException - If there was an error in native code.

addNuphar

public void addNuphar(boolean allowUnalignedBuffers,
                      java.lang.String settings)
               throws OrtException

Adds Nuphar as an execution backend.

Parameters:

allowUnalignedBuffers - Allow unaligned memory buffers.

settings - See the documentation for valid settings strings.

Throws:

OrtException - If there was an error in native code.

addDirectML

public void addDirectML(int deviceId)
                 throws OrtException

Adds DirectML as an execution backend.

Parameters:

deviceId - The id of the DirectML device.

Throws:

OrtException - If there was an error in native code.

addACL

public void addACL(boolean useArena)
            throws OrtException

Adds the ARM Compute Library as an execution backend.

Parameters:

useArena - If true use the arena memory allocator.

Throws:

OrtException - If there was an error in native code.