public static class OrtSession.SessionOptions extends Object implements AutoCloseable
Used to set the number of threads, optimisation level, computation backend and other options.
Modifying this after the session has been constructed will have no effect.
The SessionOptions object must not be closed until all sessions which use it are closed, as otherwise it could release resources that are in use.
| Modifier and Type | Class and Description |
|---|---|
static class |
OrtSession.SessionOptions.ExecutionMode
The execution mode to use.
|
static class |
OrtSession.SessionOptions.OptLevel
The optimisation level to use.
|
| Constructor and Description |
|---|
SessionOptions()
Create an empty session options.
|
| Modifier and Type | Method and Description |
|---|---|
void |
addACL(boolean enableFastMath)
Adds the ARM Compute Library as an execution backend.
|
void |
addArmNN(boolean useArena)
Adds the ARM Neural Net library as an execution backend.
|
void |
addConfigEntry(String configKey,
String configValue)
Adds a single session configuration entry as a pair of strings.
|
void |
addCoreML()
Adds Apple's CoreML as an execution backend.
|
void |
addCoreML(EnumSet<CoreMLFlags> flags)
Adds Apple's CoreML as an execution backend.
|
void |
addCoreML(Map<String,String> providerOptions)
Adds CoreML as an execution backend.
|
void |
addCPU(boolean useArena)
Adds the CPU as an execution backend, using the arena allocator if desired.
|
void |
addCUDA()
Add CUDA as an execution backend, using device 0.
|
void |
addCUDA(int deviceNum)
Add CUDA as an execution backend, using the specified CUDA device id.
|
void |
addCUDA(OrtCUDAProviderOptions cudaOpts)
Adds CUDA as an execution backend, using the specified CUDA options.
|
void |
addDirectML(int deviceId)
Adds DirectML as an execution backend.
|
void |
addDnnl(boolean useArena)
Adds Intel's Deep Neural Network Library as an execution backend.
|
void |
addExternalInitializers(Map<String,OnnxTensorLike> initializers)
Adds in the supplied externally loaded initializers.
|
void |
addInitializer(String name,
OnnxTensorLike initializer)
Adds an initializer to override one from the ONNX model.
|
void |
addNnapi()
Adds Android's NNAPI as an execution backend.
|
void |
addNnapi(EnumSet<NNAPIFlags> flags)
Adds Android's NNAPI as an execution backend.
|
void |
addOpenVINO(String deviceId)
Adds OpenVINO as an execution backend.
|
void |
addQnn(Map<String,String> providerOptions)
Adds QNN as an execution backend.
|
void |
addROCM()
Add ROCM as an execution backend, using device 0.
|
void |
addROCM(int deviceNum)
Add ROCM as an execution backend, using the specified ROCM device id.
|
void |
addTensorrt(int deviceNum)
Adds Nvidia's TensorRT as an execution backend.
|
void |
addTensorrt(OrtTensorRTProviderOptions tensorRTOpts)
Adds Nvidia's TensorRT as an execution backend.
|
void |
addTvm(String settings)
Adds TVM as an execution backend.
|
void |
addWebGPU(Map<String,String> providerOptions)
Adds WebGPU as an execution backend.
|
void |
addXnnpack(Map<String,String> providerOptions)
Adds XNNPACK as an execution backend.
|
void |
close()
Closes the session options, releasing any memory acquired.
|
void |
disablePerSessionThreads()
Disables the per session thread pools.
|
void |
disableProfiling()
Disables profiling in sessions using this SessionOptions.
|
void |
enableProfiling(String filePath)
Enables profiling in sessions using this SessionOptions.
|
Map<String,String> |
getConfigEntries()
Returns an unmodifiable view of the map contains all session configuration entries.
|
void |
registerCustomOpLibrary(String path)
Registers a library of custom ops for use with
OrtSessions using this SessionOptions. |
void |
registerCustomOpsUsingFunction(String registrationFuncName)
Registers custom ops for use with
OrtSessions using this SessionOptions by calling
the specified native function name. |
void |
setCPUArenaAllocator(boolean useArena)
Sets the CPU to use an arena memory allocator.
|
void |
setDeterministicCompute(boolean value)
Set whether to use deterministic compute.
|
void |
setExecutionMode(OrtSession.SessionOptions.ExecutionMode mode)
Sets the execution mode of this options object, overriding the old setting.
|
void |
setInterOpNumThreads(int numThreads)
Sets the size of the CPU thread pool used for executing multiple request concurrently, if
executing on a CPU.
|
void |
setIntraOpNumThreads(int numThreads)
Sets the size of the CPU thread pool used for executing a single graph, if executing on a
CPU.
|
void |
setLoggerId(String loggerId)
Sets the logger id to use.
|
void |
setMemoryPatternOptimization(boolean memoryPatternOptimization)
Turns on memory pattern optimizations, where memory is preallocated if all shapes are known.
|
void |
setOptimizationLevel(OrtSession.SessionOptions.OptLevel level)
Sets the optimization level of this options object, overriding the old setting.
|
void |
setOptimizedModelFilePath(String outputPath)
Sets the output path for the optimized model.
|
void |
setSessionLogLevel(OrtLoggingLevel logLevel)
Sets the Session's logging level.
|
void |
setSessionLogVerbosityLevel(int logLevel)
Sets the Session's logging verbosity level.
|
void |
setSymbolicDimensionValue(String dimensionName,
long dimensionValue)
Sets the value of a symbolic dimension.
|
public void close()
close in interface AutoCloseablepublic void setExecutionMode(OrtSession.SessionOptions.ExecutionMode mode) throws OrtException
mode - The execution mode to use.OrtException - If there was an error in native code.public void setOptimizationLevel(OrtSession.SessionOptions.OptLevel level) throws OrtException
level - The optimization level to use.OrtException - If there was an error in native code.public void setInterOpNumThreads(int numThreads)
throws OrtException
numThreads - The number of threads to use.OrtException - If there was an error in native code.public void setIntraOpNumThreads(int numThreads)
throws OrtException
numThreads - The number of threads to use.OrtException - If there was an error in native code.public void setOptimizedModelFilePath(String outputPath) throws OrtException
outputPath - The output path to write the model to.OrtException - If there was an error in native code.public void setLoggerId(String loggerId) throws OrtException
loggerId - The logger id string.OrtException - If there was an error in native code.public void enableProfiling(String filePath) throws OrtException
filePath - The file to write profile information to.OrtException - If there was an error in native code.public void disableProfiling()
throws OrtException
OrtException - If there was an error in native code.public void setMemoryPatternOptimization(boolean memoryPatternOptimization)
throws OrtException
memoryPatternOptimization - If true enable memory pattern optimizations.OrtException - If there was an error in native code.public void setCPUArenaAllocator(boolean useArena)
throws OrtException
useArena - If true use an arena memory allocator for the CPU execution provider.OrtException - If there was an error in native code.public void setSessionLogLevel(OrtLoggingLevel logLevel) throws OrtException
logLevel - The log level to use.OrtException - If there was an error in native code.public void setSessionLogVerbosityLevel(int logLevel)
throws OrtException
logLevel - The logging verbosity to use.OrtException - If there was an error in native code.public void registerCustomOpLibrary(String path) throws OrtException
OrtSessions using this SessionOptions.path - The path to the library on disk.OrtException - If there was an error loading the library.public void registerCustomOpsUsingFunction(String registrationFuncName) throws OrtException
OrtSessions using this SessionOptions by calling
the specified native function name. The custom ops library must either be linked against, or
have previously been loaded by the user.
The registration function must have the signature:
OrtStatus* (*fn)(OrtSessionOptions* options, const OrtApiBase* api);
See https://onnxruntime.ai/docs/reference/operators/add-custom-op.html for more information on custom ops. See https://github.com/microsoft/onnxruntime/blob/342a5bf2b756d1a1fc6fdc582cfeac15182632fe/onnxruntime/test/testdata/custom_op_library/custom_op_library.cc#L115 for an example of a custom op library registration function.
registrationFuncName - The name of the registration function to call.OrtException - If there was an error finding or calling the registration function.public void setSymbolicDimensionValue(String dimensionName, long dimensionValue) throws OrtException
dimensionName - The name of the symbolic dimension.dimensionValue - The value to set that dimension to.OrtException - If there was an error in native code.public void setDeterministicCompute(boolean value)
throws OrtException
Default is false. If set to true, this will enable deterministic compute for GPU kernels where possible. Note that this most likely will have a performance cost.
value - Should the compute be deterministic?OrtException - If there was an error in native code.public void disablePerSessionThreads()
throws OrtException
OrtException - If there was an error in native code.public void addConfigEntry(String configKey, String configValue) throws OrtException
configKey - The config key string.configValue - The config value string.OrtException - If there was an error in native code.public Map<String,String> getConfigEntries()
public void addExternalInitializers(Map<String,OnnxTensorLike> initializers) throws OrtException
Note the initializers are copied into the session once it has been created, and the native
references are removed from this SessionOptions. Once the session has been created
those initializers can be closed. This is a different lifetime to initializers added via
addInitializer(String, OnnxTensorLike). The initializers must be created from Buffer objects.
initializers - The map of names to initializers.OrtException - If the initializers could not be loaded.public void addInitializer(String name, OnnxTensorLike initializer) throws OrtException
Note the initializer lifetime must outlive the session and session options. This is a
different lifetime to initializers added via addExternalInitializers(Map). The
initializers must be created from Buffer objects.
name - The initializer name.initializer - The initializer value.OrtException - If the initializer could not be loaded into the session options.public void addCUDA()
throws OrtException
OrtException - If there was an error in native code.public void addCUDA(int deviceNum)
throws OrtException
deviceNum - The CUDA device id.OrtException - If there was an error in native code.public void addCUDA(OrtCUDAProviderOptions cudaOpts) throws OrtException
cudaOpts - The CUDA execution provider options.OrtException - If there was an error in the native code.public void addROCM()
throws OrtException
OrtException - If there was an error in native code.public void addROCM(int deviceNum)
throws OrtException
deviceNum - The ROCM device id.OrtException - If there was an error in native code.public void addCPU(boolean useArena)
throws OrtException
By default this backend is used, but if other backends are requested, it should be requested last.
useArena - If true use the arena memory allocator.OrtException - If there was an error in native code.public void addDnnl(boolean useArena)
throws OrtException
useArena - If true use the arena memory allocator.OrtException - If there was an error in native code.public void addOpenVINO(String deviceId) throws OrtException
deviceId - The id of the OpenVINO execution device.OrtException - If there was an error in native code.public void addTensorrt(int deviceNum)
throws OrtException
deviceNum - The id of the CUDA device.OrtException - If there was an error in native code.public void addTensorrt(OrtTensorRTProviderOptions tensorRTOpts) throws OrtException
tensorRTOpts - The configuration parameters for TensorRT.OrtException - If there was an error in native code.public void addNnapi()
throws OrtException
OrtException - If there was an error in native code.public void addNnapi(EnumSet<NNAPIFlags> flags) throws OrtException
flags - The flags which control the NNAPI configuration.OrtException - If there was an error in native code.public void addTvm(String settings) throws OrtException
settings - See the documentation for valid settings strings.OrtException - If there was an error in native code.public void addDirectML(int deviceId)
throws OrtException
deviceId - The id of the DirectML device.OrtException - If there was an error in native code.public void addACL(boolean enableFastMath)
throws OrtException
enableFastMath - Enable fast math mode in ACL.OrtException - If there was an error in native code.public void addArmNN(boolean useArena)
throws OrtException
useArena - If true use the arena memory allocator.OrtException - If there was an error in native code.public void addCoreML()
throws OrtException
OrtException - If there was an error in native code.public void addCoreML(EnumSet<CoreMLFlags> flags) throws OrtException
flags - The flags which control the CoreML configuration.OrtException - If there was an error in native code.public void addXnnpack(Map<String,String> providerOptions) throws OrtException
providerOptions - Configuration options for the XNNPACK backend. Refer to the XNNPACK
execution provider's documentation.OrtException - If there was an error in native code.public void addQnn(Map<String,String> providerOptions) throws OrtException
providerOptions - Configuration options for the QNN backend. Refer to the QNN execution
provider's documentation.OrtException - If there was an error in native code.public void addCoreML(Map<String,String> providerOptions) throws OrtException
providerOptions - Configuration options for the CoreML backend. Refer to the CoreML
execution provider's documentation.OrtException - If there was an error in native code.public void addWebGPU(Map<String,String> providerOptions) throws OrtException
providerOptions - Configuration options for the WebGPU backend. Refer to the WebGPU
execution provider's documentation.OrtException - If there was an error in native code.Copyright © 2025. All rights reserved.