org.bytedeco.tensorrt.nvinfer

Class IBuilder

java.lang.Object

org.bytedeco.javacpp.Pointer

org.bytedeco.tensorrt.nvinfer.INoCopy

org.bytedeco.tensorrt.nvinfer.IBuilder

All Implemented Interfaces:

AutoCloseable

@Namespace(value="nvinfer1")
 @NoOffset
 @Properties(inherit=nvinfer.class)
public class IBuilder
extends INoCopy

\class IBuilder \brief Builds an engine from a network definition. \warning Do not inherit from this class, as doing so will break forward-compatibility of the API and ABI.

Nested Class Summary

Nested classes/interfaces inherited from class org.bytedeco.javacpp.Pointer

Pointer.CustomDeallocator, Pointer.Deallocator, Pointer.NativeDeallocator, Pointer.ReferenceCounter

Field Summary

Fields inherited from class org.bytedeco.javacpp.Pointer

address, capacity, limit, position

Constructor Summary

Constructors

Constructor and Description
IBuilder(Pointer p) Pointer cast constructor.

Method Summary

Modifier and Type	Method and Description
ICudaEngine	buildEngineWithConfig(INetworkDefinition network, IBuilderConfig config) \brief Builds a network for the given INetworkDefinition and IBuilderConfig.
IHostMemory	buildSerializedNetwork(INetworkDefinition network, IBuilderConfig config) \brief Builds and serializes a network for the given INetworkDefinition and IBuilderConfig.
IHostMemory	buildSerializedNetwork(INetworkDefinition network, IBuilderConfig config, IHostMemory kernelText) \brief Extended form of buildSerializedNetwork that optionally permits getting the kernelText.
boolean	buildSerializedNetworkToStream(INetworkDefinition network, IBuilderConfig config, IStreamWriter writer) \brief Builds and serializes a network into stream for the given INetworkDefinition and IBuilderConfig.
IBuilderConfig	createBuilderConfig() \brief Create a builder configuration object.
INetworkDefinition	createNetworkV2(int flags) \brief Create a network definition object Creates a network definition object with immutable properties specified using the flags parameter.
IOptimizationProfile	createOptimizationProfile() \brief Create a new optimization profile.
IErrorRecorder	getErrorRecorder() \brief get the ErrorRecorder assigned to this interface.
ILogger	getLogger() \brief get the logger with which the builder was created
int	getMaxDLABatchSize() \brief Get the maximum batch size DLA can support.
int	getMaxThreads() \brief get the maximum number of threads that can be used by the builder.
int	getNbDLACores() \brief Return the number of DLA engines available to this builder.
IPluginRegistry	getPluginRegistry() \brief get the local plugin registry that can be used by the builder.
boolean	isNetworkSupported(INetworkDefinition network, IBuilderConfig config) \brief Checks that a network is within the scope of the IBuilderConfig settings.
void	reset() \brief Resets the builder state to default values.
void	setErrorRecorder(IErrorRecorder recorder) \brief Set the ErrorRecorder for this interface Assigns the ErrorRecorder to this interface.
void	setGpuAllocator(IGpuAllocator allocator) \brief Set the GPU allocator.
boolean	setMaxThreads(int maxThreads) \brief Set the maximum number of threads.

Methods inherited from class org.bytedeco.javacpp.Pointer

address, asBuffer, asByteBuffer, availablePhysicalBytes, calloc, capacity, capacity, close, deallocate, deallocate, deallocateReferences, deallocator, deallocator, equals, fill, formatBytes, free, getDirectBufferAddress, getPointer, getPointer, getPointer, getPointer, hashCode, interruptDeallocatorThread, isNull, isNull, limit, limit, malloc, maxBytes, maxPhysicalBytes, memchr, memcmp, memcpy, memmove, memset, offsetAddress, offsetof, offsetof, parseBytes, physicalBytes, physicalBytesInaccurate, position, position, put, realloc, referenceCount, releaseReference, retainReference, setNull, sizeof, sizeof, toString, totalBytes, totalCount, totalPhysicalBytes, trimMemory, withDeallocator, zero

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

IBuilder

public IBuilder(Pointer p)

Pointer cast constructor. Invokes Pointer(Pointer).

Method Detail

getMaxDLABatchSize

@NoException(value=true)
public int getMaxDLABatchSize()

\brief Get the maximum batch size DLA can support. For any tensor the total volume of index dimensions combined(dimensions other than CHW) with the requested batch size should not exceed the value returned by this function. \warning getMaxDLABatchSize does not work with dynamic shapes.

getNbDLACores

@NoException(value=true)
public int getNbDLACores()

\brief Return the number of DLA engines available to this builder.

setGpuAllocator

@NoException(value=true)
public void setGpuAllocator(IGpuAllocator allocator)

\brief Set the GPU allocator.

Parameters:

allocator - Set the GPU allocator to be used by the builder. All GPU memory acquired will use this allocator. If NULL is passed, the default allocator will be used. Default: allocateAsync uses cudaMallocAsync if cudaDevAttrMemoryPoolsSupported returns true, otherwise falls back to cudaMalloc. allocate always uses cudaMalloc. \note This allocator will be passed to any engines created via the builder; thus the lifetime of the allocator must span the lifetime of those engines as well as that of the builder. If nullptr is passed, the default allocator will be used.

createBuilderConfig

@NoException(value=true)
public IBuilderConfig createBuilderConfig()

\brief Create a builder configuration object. The caller owns the new IBuilderConfig, which must be destroyed with operator delete before this IBuilder is destroyed. Destroying this IBuilder before destroying the IBuilderConfig causes undefined behavior.

See Also:

IBuilderConfig

createNetworkV2

@NoException(value=true)
public INetworkDefinition createNetworkV2(@Cast(value="nvinfer1::NetworkDefinitionCreationFlags")
                                                                    int flags)

\brief Create a network definition object Creates a network definition object with immutable properties specified using the flags parameter. createNetworkV2 supports creating network with properties from NetworkDefinitionCreationFlags. CreateNetworkV2 supports dynamic shapes and explicit batch dimensions by default. The network is always strongly typed: tensor data types are inferred from network input types and operator type specification. The kSTRONGLY_TYPED flag is deprecated and ignored. The caller owns the new INetworkDefinition, which must be destroyed with operator delete before this IBuilder is destroyed. Destroying this IBuilder before destroying the INetworkDefinition causes undefined behavior.

Parameters:

flags - Bitset of NetworkDefinitionCreationFlags specifying network properties combined with bitwise OR, e.g., 1U << NetworkDefinitionCreationFlag::kPREFER_JIT_PYTHON_PLUGINS.

See Also:

INetworkDefinition, NetworkDefinitionCreationFlags

createOptimizationProfile

@NoException(value=true)
public IOptimizationProfile createOptimizationProfile()

\brief Create a new optimization profile. If the network has any dynamic input tensors, the appropriate calls to setDimensions() must be made. Likewise, if there are any shape input tensors, the appropriate calls to \ref IOptimizationProfile::setShapeValuesV2() are required. The builder retains ownership of the created optimization profile and returns a raw pointer, i.e. the users must not attempt to delete the returned pointer.

See Also:

IOptimizationProfile

setErrorRecorder

@NoException(value=true)
public void setErrorRecorder(IErrorRecorder recorder)

\brief Set the ErrorRecorder for this interface Assigns the ErrorRecorder to this interface. The ErrorRecorder will track all errors during execution. This function will call incRefCount of the registered ErrorRecorder at least once. Setting recorder to nullptr unregisters the recorder with the interface, resulting in a call to decRefCount if a recorder has been registered. If an error recorder is not set, messages will be sent to the global log stream.

Parameters:

recorder - The error recorder to register with this interface.

See Also:

getErrorRecorder()

getErrorRecorder

@NoException(value=true)
public IErrorRecorder getErrorRecorder()

\brief get the ErrorRecorder assigned to this interface. Retrieves the assigned error recorder object for the given class. A nullptr will be returned if setErrorRecorder has not been called.

Returns:

A pointer to the IErrorRecorder object that has been registered.

See Also:

setErrorRecorder()

reset

@NoException(value=true)
public void reset()

\brief Resets the builder state to default values.

buildSerializedNetwork

@NoException(value=true)
public IHostMemory buildSerializedNetwork(@ByRef
                                                                    INetworkDefinition network,
                                                                    @ByRef
                                                                    IBuilderConfig config)

\brief Builds and serializes a network for the given INetworkDefinition and IBuilderConfig. This function allows building and serialization of a network without creating an engine.

Parameters:

network - Network definition.

config - Builder configuration.

Returns:

A pointer to a IHostMemory object that contains a serialized network. \note This function will synchronize the CUDA stream returned by \p config.getProfileStream() before returning.

See Also:

INetworkDefinition, IBuilderConfig, IHostMemory

buildSerializedNetworkToStream

@Cast(value="bool")
 @NoException(value=true)
public boolean buildSerializedNetworkToStream(@ByRef
                                                                                              INetworkDefinition network,
                                                                                              @ByRef
                                                                                              IBuilderConfig config,
                                                                                              @ByRef
                                                                                              IStreamWriter writer)

\brief Builds and serializes a network into stream for the given INetworkDefinition and IBuilderConfig. This function allows building and serialization of a network without creating an engine. The engine is finally serialized into the writer stream.

Parameters:

network - Network definition.

config - Builder configuration.

writer - Output writer stream.

Returns:

true if build succeed, otherwise false. \note This function will synchronize the CUDA stream returned by \p config.getProfileStream() before returning.

See Also:

INetworkDefinition, IBuilderConfig, IStreamWriter

buildSerializedNetwork

@NoException(value=true)
public IHostMemory buildSerializedNetwork(@ByRef
                                                                    INetworkDefinition network,
                                                                    @ByRef
                                                                    IBuilderConfig config,
                                                                    @ByPtrRef
                                                                    IHostMemory kernelText)

\brief Extended form of buildSerializedNetwork that optionally permits getting the kernelText. Similar to two-argument form, except that if an engine with safe capability is successfully built and there are kernels, sets kernelText to ..... Otherwise sets kernelText=nullptr. This function allows building and serialization of a network without creating an engine.

Parameters:

network - Network definition.

config - Builder configuration.

kernelText - A reference to a pointer to a IHostMemory object that will be set to the kernel CPP code text

Returns:

A pointer to a IHostMemory object that contains a serialized network. \note This function will synchronize the CUDA stream returned by \p config.getProfileStream() before returning.

See Also:

INetworkDefinition, IBuilderConfig, IHostMemory

buildEngineWithConfig

@NoException(value=true)
public ICudaEngine buildEngineWithConfig(@ByRef
                                                                   INetworkDefinition network,
                                                                   @ByRef
                                                                   IBuilderConfig config)

\brief Builds a network for the given INetworkDefinition and IBuilderConfig.

Parameters:

network - Network definition.

config - Builder configuration.

Returns:

A pointer to a ICudaEngine object that contains an engine. \note This function will synchronize the CUDA stream returned by \p config.getProfileStream() before returning. \note This function does not support \p BuilderFlag::kVERSION_COMPATIBLE. Please use \p buildSerializedNetwork to get a version compatible engine.

See Also:

INetworkDefinition, IBuilderConfig, ICudaEngine

isNetworkSupported

@Cast(value="bool")
 @NoException(value=true)
public boolean isNetworkSupported(@Const @ByRef
                                                                                  INetworkDefinition network,
                                                                                  @Const @ByRef
                                                                                  IBuilderConfig config)

\brief Checks that a network is within the scope of the IBuilderConfig settings.

Parameters:

network - The network definition to check for configuration compliance.

config - The configuration of the builder to use when checking \p network. Given an INetworkDefinition, \p network, and an IBuilderConfig, \p config, check if the network falls within the constraints of the builder configuration based on the EngineCapability, BuilderFlag, and DeviceType. If the network is within the constraints, then the function returns true, and false if a violation occurs. This function reports the conditions that are violated to the registered ErrorRecorder.

Returns:

True if network is within the scope of the restrictions specified by the builder config, false otherwise. \note A true return value does not guarantee that engine building will succeed, as backends may reject it for reasons not detectable with this fast validation. To definitively check whether a network can be built with a given config, use \p buildEngineWithConfig or \p buildSerializedNetwork (depending on the engine capability). \note This function will synchronize the CUDA stream returned by \p config.getProfileStream() before returning.

getLogger

@NoException(value=true)
public ILogger getLogger()

\brief get the logger with which the builder was created

Returns:

the logger

setMaxThreads

@Cast(value="bool")
 @NoException(value=true)
public boolean setMaxThreads(int maxThreads)

\brief Set the maximum number of threads.

Parameters:

maxThreads - The maximum number of threads that can be used by the builder.

Returns:

True if successful, false otherwise. The default value is 1 and includes the current thread. A value greater than 1 permits TensorRT to use multi-threaded algorithms. A value less than 1 triggers a kINVALID_ARGUMENT error.

getMaxThreads

@NoException(value=true)
public int getMaxThreads()

\brief get the maximum number of threads that can be used by the builder. Retrieves the maximum number of threads that can be used by the builder.

Returns:

The maximum number of threads that can be used by the builder.

See Also:

setMaxThreads()

getPluginRegistry

@ByRef
 @NoException(value=true)
public IPluginRegistry getPluginRegistry()

\brief get the local plugin registry that can be used by the builder.

Returns:

The local plugin registry that can be used by the builder.