org.bytedeco.tensorrt.nvinfer

Class IOutputAllocator

java.lang.Object

org.bytedeco.javacpp.Pointer

org.bytedeco.tensorrt.nvinfer.IVersionedInterface

org.bytedeco.tensorrt.nvinfer.IOutputAllocator

All Implemented Interfaces:

AutoCloseable

@Namespace(value="nvinfer1::v_1_0")
 @Properties(inherit=nvinfer.class)
public class IOutputAllocator
extends IVersionedInterface

Nested Class Summary

Nested classes/interfaces inherited from class org.bytedeco.javacpp.Pointer

Pointer.CustomDeallocator, Pointer.Deallocator, Pointer.NativeDeallocator, Pointer.ReferenceCounter

Field Summary

Fields inherited from class org.bytedeco.javacpp.Pointer

address, capacity, limit, position

Constructor Summary

Constructors

Constructor and Description
IOutputAllocator(Pointer p) Pointer cast constructor.

Method Summary

Modifier and Type	Method and Description
InterfaceInfo	getInterfaceInfo() \brief Return version information associated with this interface.
void	notifyShape(BytePointer tensorName, Dims64 dims)
void	notifyShape(String tensorName, Dims64 dims) \brief Called by TensorRT when the shape of the output tensor is known.
Pointer	reallocateOutput(BytePointer tensorName, Pointer currentMemory, long size, long alignment) Deprecated.
Pointer	reallocateOutput(String tensorName, Pointer currentMemory, long size, long alignment) Deprecated. Deprecated in TensorRT 10.0. Superseded by reallocateOutputAsync with cudaStream_t argument
Pointer	reallocateOutputAsync(BytePointer tensorName, Pointer currentMemory, long size, long alignment, org.bytedeco.cuda.cudart.CUstream_st arg4)
Pointer	reallocateOutputAsync(String tensorName, Pointer currentMemory, long size, long alignment, org.bytedeco.cuda.cudart.CUstream_st arg4) \brief Return a pointer to memory for an output tensor, or nullptr if memory cannot be allocated.

Methods inherited from class org.bytedeco.tensorrt.nvinfer.IVersionedInterface

getAPILanguage

Methods inherited from class org.bytedeco.javacpp.Pointer

address, asBuffer, asByteBuffer, availablePhysicalBytes, calloc, capacity, capacity, close, deallocate, deallocate, deallocateReferences, deallocator, deallocator, equals, fill, formatBytes, free, getDirectBufferAddress, getPointer, getPointer, getPointer, getPointer, hashCode, interruptDeallocatorThread, isNull, isNull, limit, limit, malloc, maxBytes, maxPhysicalBytes, memchr, memcmp, memcpy, memmove, memset, offsetAddress, offsetof, offsetof, parseBytes, physicalBytes, physicalBytesInaccurate, position, position, put, realloc, referenceCount, releaseReference, retainReference, setNull, sizeof, sizeof, toString, totalBytes, totalCount, totalPhysicalBytes, withDeallocator, zero

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

IOutputAllocator

public IOutputAllocator(Pointer p)

Pointer cast constructor. Invokes Pointer(Pointer).

Method Detail

getInterfaceInfo

@ByVal
 @NoException(value=true)
public InterfaceInfo getInterfaceInfo()

\brief Return version information associated with this interface. Applications must not override this method.

Overrides:

getInterfaceInfo in class IVersionedInterface

reallocateOutput

@Deprecated
 @NoException(value=true)
public Pointer reallocateOutput(String tensorName,
                                                                      Pointer currentMemory,
                                                                      @Cast(value="uint64_t")
                                                                      long size,
                                                                      @Cast(value="uint64_t")
                                                                      long alignment)

Deprecated. Deprecated in TensorRT 10.0. Superseded by reallocateOutputAsync with cudaStream_t argument

\brief Return a pointer to memory for an output tensor, or nullptr if memory cannot be allocated. If the requested memory size exceeds the currentMemory size, the currentMemory can be freed as well. If currentMemory is known to be big enough, one option is to return currentMemory.

Parameters:

tensorName - name of the output tensor.

currentMemory - points to the address set by IExecutionContext::setTensorAddress.

size - number of bytes required. Always positive, even for an empty tensor.

alignment - required alignment of the allocation.

Returns:

A pointer to memory to use for the output tensor or nullptr. To preallocate memory and have the engine fail if the preallocation is not big enough, use IExecutionContext::setTensorAddress to set a pointer to the preallocated memory, and have reallocateOutput return nullptr if that memory is not big enough.

reallocateOutput

@Deprecated
 @NoException(value=true)
public Pointer reallocateOutput(@Cast(value="const char*")
                                                                      BytePointer tensorName,
                                                                      Pointer currentMemory,
                                                                      @Cast(value="uint64_t")
                                                                      long size,
                                                                      @Cast(value="uint64_t")
                                                                      long alignment)

Deprecated.

reallocateOutputAsync

public Pointer reallocateOutputAsync(String tensorName,
                                     Pointer currentMemory,
                                     @Cast(value="uint64_t")
                                     long size,
                                     @Cast(value="uint64_t")
                                     long alignment,
                                     org.bytedeco.cuda.cudart.CUstream_st arg4)

\brief Return a pointer to memory for an output tensor, or nullptr if memory cannot be allocated. If the requested memory size exceeds the currentMemory size, the currentMemory can be freed as well. If currentMemory is known to be big enough, one option is to return currentMemory.

Parameters:

tensorName - name of the output tensor.

currentMemory - points to the address set by IExecutionContext::setTensorAddress.

size - number of bytes required. Always positive, even for an empty tensor.

alignment - required alignment of the allocation.

stream - The stream in which to execute the kernels.

Returns:

A pointer to memory to use for the output tensor or nullptr. To preallocate memory and have the engine fail if the preallocation is not big enough, use IExecutionContext::setTensorAddress to set a pointer to the preallocated memory, and have reallocateOutputAsync return nullptr if that memory is not big enough. The default definition exists for sake of backward compatibility with earlier versions of TensorRT. Eventually this method will become a pure virtual method that requires an override, and method reallocateOutput() will disappear. Code moving away from TensorRT 9.x should override method reallocateOutputAsync() and NOT override method reallocateOutput().

reallocateOutputAsync

public Pointer reallocateOutputAsync(@Cast(value="const char*")
                                     BytePointer tensorName,
                                     Pointer currentMemory,
                                     @Cast(value="uint64_t")
                                     long size,
                                     @Cast(value="uint64_t")
                                     long alignment,
                                     org.bytedeco.cuda.cudart.CUstream_st arg4)

notifyShape

@NoException(value=true)
public void notifyShape(String tensorName,
                                                 @Cast(value="const nvinfer1::Dims*") @ByRef
                                                 Dims64 dims)

\brief Called by TensorRT when the shape of the output tensor is known. Called by TensorRT sometime between when it calls reallocateOutput and enqueueV3 returns.

Parameters:

dims - dimensions of the output

tensorName - name of the tensor

notifyShape

@NoException(value=true)
public void notifyShape(@Cast(value="const char*")
                                                 BytePointer tensorName,
                                                 @Cast(value="const nvinfer1::Dims*") @ByRef
                                                 Dims64 dims)