public class nvinfer extends nvinfer
Modifier and Type | Class and Description |
---|---|
static class |
nvinfer.ActivationType
\enum ActivationType
\brief Enumerates the types of activation to perform in an activation layer.
|
static class |
nvinfer.AllocatorFlag
Maximum number of elements in TensorFormat enum.
|
static class |
nvinfer.BoundingBoxFormat
\brief Representation of bounding box data used for the Boxes input tensor in INMSLayer
|
static class |
nvinfer.BuilderFlag
\enum BuilderFlag
\brief List of valid modes that the builder can enable when creating an engine from a network definition.
|
static class |
nvinfer.CalibrationAlgoType
enum CalibrationAlgoType
\brief Version of calibration algorithm to use.
|
static class |
nvinfer.DataType
\enum DataType
\brief The type of weights and tensors.
|
static class |
nvinfer.DeviceType
\enum DeviceType
\brief The device that this layer/network will execute on.
|
static class |
nvinfer.DimensionOperation
\enum DimensionOperation
\brief An operation on two IDimensionExpr, which represent integer expressions used in dimension computations.
|
static class |
nvinfer.ElementWiseOperation
\enum ElementWiseOperation
\brief Enumerates the binary operations that may be performed by an ElementWise layer.
|
static class |
nvinfer.EngineCapability
\enum EngineCapability
\brief List of supported engine capability flows.
|
static class |
nvinfer.ErrorCode
\enum ErrorCode
\brief Error codes that can be returned by TensorRT during execution.
|
static class |
nvinfer.FillOperation
\enum FillOperation
\brief Enumerates the tensor fill operations that may performed by a fill layer.
|
static class |
nvinfer.GatherMode
\brief Control form of IGatherLayer
|
static class |
nvinfer.HardwareCompatibilityLevel
Describes requirements of compatibility with GPU architectures other than that of the GPU on which the engine was
built.
|
static class |
nvinfer.InterpolationMode
\enum InterpolationMode
\brief Enumerates various modes of interpolation
|
static class |
nvinfer.LayerInformationFormat
\enum LayerInformationFormat
\brief The format in which the IEngineInspector prints the layer information.
|
static class |
nvinfer.LayerType
\enum LayerType
\brief The type values of layer classes.
|
static class |
nvinfer.LoopOutput
Enum that describes kinds of loop outputs.
|
static class |
nvinfer.MatrixOperation
\enum MatrixOperation
\brief Enumerates the operations that may be performed on a tensor
by IMatrixMultiplyLayer before multiplication.
|
static class |
nvinfer.MemoryPoolType
\enum MemoryPoolType
\brief The type for memory pools used by TensorRT.
|
static class |
nvinfer.NetworkDefinitionCreationFlag
\enum NetworkDefinitionCreationFlag
\brief List of immutable network properties expressed at network creation time.
|
static class |
nvinfer.OptProfileSelector
\enum OptProfileSelector
\brief When setting or querying optimization profile parameters (such as shape tensor inputs or dynamic dimensions),
select whether we are interested in the minimum, optimum, or maximum values for these parameters.
|
static class |
nvinfer.PaddingMode
\enum PaddingMode
\brief Enumerates the modes of padding to perform in convolution, deconvolution and pooling layer,
padding mode takes precedence if setPaddingMode() and setPrePadding() are also used.
|
static class |
nvinfer.PluginFieldType
\enum PluginFieldType
\brief The possible field types for custom layer.
|
static class |
nvinfer.PluginVersion
\struct PluginVersion
\brief Definition of plugin versions.
|
static class |
nvinfer.PoolingType
\enum PoolingType
\brief The type of pooling to perform in a pooling layer.
|
static class |
nvinfer.PreviewFeature
\enum PreviewFeature
\brief Define preview features
Preview Features have been fully tested but are not yet as stable as other features in TensorRT.
|
static class |
nvinfer.ProfilingVerbosity
\enum ProfilingVerbosity
\brief List of verbosity levels of layer information exposed in NVTX annotations and in IEngineInspector.
|
static class |
nvinfer.QuantizationFlag
\enum QuantizationFlag
\brief List of valid flags for quantizing the network to int8
|
static class |
nvinfer.ReduceOperation
\enum ReduceOperation
\brief Enumerates the reduce operations that may be performed by a Reduce layer.
|
static class |
nvinfer.ResizeCoordinateTransformation
\enum ResizeCoordinateTransformation
\brief The resize coordinate transformation function.
|
static class |
nvinfer.ResizeRoundMode
\enum ResizeRoundMode
\brief The rounding mode for nearest neighbor resize.
|
static class |
nvinfer.ResizeSelector
\enum ResizeSelector
\brief The coordinate selector when resize to single pixel output.
|
static class |
nvinfer.RNNDirection
\enum RNNDirection
\brief Enumerates the RNN direction that may be performed by an RNN layer.
|
static class |
nvinfer.RNNGateType
\enum RNNGateType
\brief Identifies an individual gate within an RNN cell.
|
static class |
nvinfer.RNNInputMode
\enum RNNInputMode
\brief Enumerates the RNN input modes that may occur with an RNN layer.
|
static class |
nvinfer.RNNOperation
\enum RNNOperation
\brief Enumerates the RNN operations that may be performed by an RNN layer.
|
static class |
nvinfer.SampleMode
\brief Controls how ISliceLayer and IGridSample handle out-of-bounds coordinates.
|
static class |
nvinfer.ScaleMode
\brief Controls how shift, scale and power are applied in a Scale layer.
|
static class |
nvinfer.ScatterMode
\brief Control form of IScatterLayer
|
static class |
nvinfer.TacticSource
\enum TacticSource
\brief List of tactic sources for TensorRT.
|
static class |
nvinfer.TempfileControlFlag
\enum TempfileControlFlag
\brief Flags used to control TensorRT's behavior when creating executable temporary files.
|
static class |
nvinfer.TensorFormat
\enum TensorFormat
\brief Format of the input/output tensors.
|
static class |
nvinfer.TensorIOMode
\enum TensorIOMode
\brief Definition of tensor IO Mode.
|
static class |
nvinfer.TensorLocation
\enum TensorLocation
\brief The location for tensor data storage, device or host.
|
static class |
nvinfer.TopKOperation
\enum TopKOperation
\brief Enumerates the operations that may be performed by a TopK layer.
|
static class |
nvinfer.TripLimit
Enum that describes kinds of trip limits.
|
static class |
nvinfer.UnaryOperation
\enum UnaryOperation
\brief Enumerates the unary operations that may be performed by a Unary layer.
|
static class |
nvinfer.WeightsRole
\enum WeightsRole
\brief How a layer uses particular Weights.
|
Modifier and Type | Field and Description |
---|---|
static int |
kNV_TENSORRT_VERSION_IMPL |
static int |
NV_TENSORRT_BUILD
TensorRT build number.
|
static int |
NV_TENSORRT_LWS_MAJOR
TensorRT LWS major version.
|
static int |
NV_TENSORRT_LWS_MINOR
TensorRT LWS minor version.
|
static int |
NV_TENSORRT_LWS_PATCH
TensorRT LWS patch version.
|
static int |
NV_TENSORRT_MAJOR
TensorRT major version.
|
static int |
NV_TENSORRT_MINOR
TensorRT minor version.
|
static int |
NV_TENSORRT_PATCH
TensorRT patch version.
|
static int |
NV_TENSORRT_RELEASE_TYPE
TensorRT release type
|
static int |
NV_TENSORRT_RELEASE_TYPE_EARLY_ACCESS
An early access release
|
static int |
NV_TENSORRT_RELEASE_TYPE_GENERAL_AVAILABILITY
A final release
|
static int |
NV_TENSORRT_RELEASE_TYPE_RELEASE_CANDIDATE
A release candidate
|
static int |
NV_TENSORRT_SONAME_MAJOR
Shared object library major version number.
|
static int |
NV_TENSORRT_SONAME_MINOR
Shared object library minor version number.
|
static int |
NV_TENSORRT_SONAME_PATCH
Shared object library patch version number.
|
static int |
NV_TENSORRT_VERSION |
Constructor and Description |
---|
nvinfer() |
Modifier and Type | Method and Description |
---|---|
static Pointer |
createInferBuilder_INTERNAL(Pointer logger,
int version)
Internal C entry point for creating IBuilder.
|
static IBuilder |
createInferBuilder(ILogger logger)
\brief Create an instance of an IBuilder class.
|
static Pointer |
createInferRefitter_INTERNAL(Pointer engine,
Pointer logger,
int version)
Internal C entry point for creating IRefitter.
|
static IRefitter |
createInferRefitter(ICudaEngine engine,
ILogger logger)
\brief Create an instance of an IRefitter class.
|
static Pointer |
createInferRuntime_INTERNAL(Pointer logger,
int version)
Internal C entry point for creating IRuntime.
|
static IRuntime |
createInferRuntime(ILogger logger)
\brief Create an instance of an IRuntime class.
|
static IPluginRegistry |
getBuilderPluginRegistry(int capability) |
static IPluginRegistry |
getBuilderPluginRegistry(nvinfer.EngineCapability capability)
\brief Return the plugin registry for building a Standard engine, or nullptr if no registry exists.
|
static SafeIPluginRegistry |
getBuilderSafePluginRegistry(int capability) |
static SafeIPluginRegistry |
getBuilderSafePluginRegistry(nvinfer.EngineCapability capability)
\brief Return the plugin registry for building a Safety engine, or nullptr if no registry exists.
|
static int |
getInferLibVersion()
\brief Return the library version number.
|
static ILogger |
getLogger()
\brief Return the logger object.
|
static IPluginRegistry |
getPluginRegistry()
\brief Return the plugin registry
|
static int |
kNV_TENSORRT_VERSION_IMPL()
\namespace nvinfer1
\brief The TensorRT API version 1 namespace.
|
static int |
NV_TENSORRT_VERSION()
\file NvInferRuntimeBase.h
This file contains common definitions, data structures and interfaces shared between the standard and safe runtime.
|
static boolean |
reorderSubBuffers(Pointer input,
int[] order,
int num,
int size)
Deprecated.
|
static boolean |
reorderSubBuffers(Pointer input,
IntBuffer order,
int num,
int size)
Deprecated.
|
static boolean |
reorderSubBuffers(Pointer input,
IntPointer order,
int num,
int size)
Deprecated.
Deprecated in TensorRT 8.0.
\warning This file will be removed in TensorRT 10.0.
|
static boolean |
reshapeWeights(Weights input,
int[] shape,
int[] shapeOrder,
Pointer data,
int nbDims)
Deprecated.
|
static boolean |
reshapeWeights(Weights input,
IntBuffer shape,
IntBuffer shapeOrder,
Pointer data,
int nbDims)
Deprecated.
|
static boolean |
reshapeWeights(Weights input,
IntPointer shape,
IntPointer shapeOrder,
Pointer data,
int nbDims)
Deprecated.
Deprecated in TensorRT 8.0.
\warning This file will be removed in TensorRT 10.0.
|
static boolean |
transposeSubBuffers(Pointer input,
int type,
int num,
int height,
int width)
Deprecated.
|
static boolean |
transposeSubBuffers(Pointer input,
nvinfer.DataType type,
int num,
int height,
int width)
Deprecated.
Deprecated in TensorRT 8.0.
\warning This file will be removed in TensorRT 10.0.
|
public static final int NV_TENSORRT_MAJOR
public static final int NV_TENSORRT_MINOR
public static final int NV_TENSORRT_PATCH
public static final int NV_TENSORRT_BUILD
public static final int NV_TENSORRT_LWS_MAJOR
public static final int NV_TENSORRT_LWS_MINOR
public static final int NV_TENSORRT_LWS_PATCH
public static final int NV_TENSORRT_SONAME_MAJOR
public static final int NV_TENSORRT_SONAME_MINOR
public static final int NV_TENSORRT_SONAME_PATCH
public static final int NV_TENSORRT_RELEASE_TYPE_EARLY_ACCESS
public static final int NV_TENSORRT_RELEASE_TYPE_RELEASE_CANDIDATE
public static final int NV_TENSORRT_RELEASE_TYPE_GENERAL_AVAILABILITY
public static final int NV_TENSORRT_RELEASE_TYPE
public static final int NV_TENSORRT_VERSION
public static final int kNV_TENSORRT_VERSION_IMPL
@MemberGetter public static int NV_TENSORRT_VERSION()
@Namespace(value="nvinfer1") @MemberGetter public static int kNV_TENSORRT_VERSION_IMPL()
@NoException(value=true) public static int getInferLibVersion()
@NoException(value=true) public static Pointer createInferRuntime_INTERNAL(Pointer logger, int version)
@NoException(value=true) public static Pointer createInferRefitter_INTERNAL(Pointer engine, Pointer logger, int version)
@NoException(value=true) public static IPluginRegistry getPluginRegistry()
@NoException(value=true) public static ILogger getLogger()
@Namespace(value="nvinfer1") @NoException(value=true) public static IRuntime createInferRuntime(@ByRef ILogger logger)
logger
- The logging class for the runtime.@Namespace(value="nvinfer1") @NoException(value=true) public static IRefitter createInferRefitter(@ByRef ICudaEngine engine, @ByRef ILogger logger)
logger
- The logging class for the refitter.@NoException(value=true) public static Pointer createInferBuilder_INTERNAL(Pointer logger, int version)
@Namespace(value="nvinfer1") @NoException(value=true) public static IBuilder createInferBuilder(@ByRef ILogger logger)
logger
- The logging class for the builder.
unnamed namespace avoids linkage surprises when linking objects built with different versions of this header.@Namespace(value="nvinfer1") @NoException(value=true) public static IPluginRegistry getBuilderPluginRegistry(nvinfer.EngineCapability capability)
@Namespace(value="nvinfer1") @NoException(value=true) public static IPluginRegistry getBuilderPluginRegistry(@Cast(value="nvinfer1::EngineCapability") int capability)
@Namespace(value="nvinfer1") @NoException(value=true) public static SafeIPluginRegistry getBuilderSafePluginRegistry(nvinfer.EngineCapability capability)
@Namespace(value="nvinfer1") @NoException(value=true) public static SafeIPluginRegistry getBuilderSafePluginRegistry(@Cast(value="nvinfer1::EngineCapability") int capability)
@Namespace(value="nvinfer1::utils") @Cast(value="bool") @Deprecated @NoException(value=true) public static boolean reshapeWeights(@Const @ByRef Weights input, @Const IntPointer shape, @Const IntPointer shapeOrder, Pointer data, int nbDims)
input
- The input weights to reshape.shape
- The shape of the weights.shapeOrder
- The order of the dimensions to process for the output.data
- The location where the output data is placed.nbDims
- The number of dimensions to process.
\brief Reformat the input weights of the given shape based on the new
order of dimensions.
Take the weights specified by \p input with the dimensions specified by
\p shape and re-order the weights based on the new dimensions specified
by \p shapeOrder. The size of each dimension and the input data is not
modified. The output volume pointed to by \p data must be the same as
he \p input volume.
Example usage:
float *out = new float[N*C*H*W];
Weights input{DataType::kFLOAT, {0 ... N*C*H*W-1}, N*C*H*W size};
int32_t order[4]{1, 0, 3, 2};
int32_t shape[4]{C, N, W, H};
reshapeWeights(input, shape, order, out, 4);
Weights reshaped{input.type, out, input.count};
Input Matrix{3, 2, 3, 2}:
{ 0 1}, { 2 3}, { 4 5} <-- {0, 0, *, *}
{ 6 7}, { 8 9}, {10 11} <-- {0, 1, *, *}
{12 13}, {14 15}, {16 17} <-- {1, 0, *, *}
{18 19}, {20 21}, {22 23} <-- {1, 1, *, *}
{24 25}, {26 27}, {28 29} <-- {2, 0, *, *}
{30 31}, {32 33}, {34 35} <-- {2, 1, *, *}
Output Matrix{2, 3, 2, 3}:
{ 0 2 4}, { 1 3 5} <-- {0, 0, *, *}
{12 14 16}, {13 15 17} <-- {0, 1, *, *}
{24 26 28}, {25 27 29} <-- {0, 2, *, *}
{ 6 8 10}, { 7 9 11} <-- {1, 0, *, *}
{18 20 22}, {19 21 23} <-- {1, 1, *, *}
{30 32 34}, {31 33 35} <-- {1, 2, *, *}@Namespace(value="nvinfer1::utils") @Cast(value="bool") @Deprecated @NoException(value=true) public static boolean reshapeWeights(@Const @ByRef Weights input, @Const IntBuffer shape, @Const IntBuffer shapeOrder, Pointer data, int nbDims)
@Namespace(value="nvinfer1::utils") @Cast(value="bool") @Deprecated @NoException(value=true) public static boolean reshapeWeights(@Const @ByRef Weights input, @Const int[] shape, @Const int[] shapeOrder, Pointer data, int nbDims)
@Namespace(value="nvinfer1::utils") @Cast(value="bool") @Deprecated @NoException(value=true) public static boolean reorderSubBuffers(Pointer input, @Const IntPointer order, int num, int size)
input
- The input data to re-order.order
- The new order of the data sub-buffers.num
- The number of data sub-buffers to re-order.size
- The size of each data sub-buffer in bytes.
\brief Takes an input stream and re-orders \p num chunks of the data
given the \p size and \p order.
In some frameworks, the ordering of the sub-buffers within a dimension
is different than the way that TensorRT expects them.
TensorRT expects the gate/bias sub-buffers for LSTM's to be in fico order.
TensorFlow however formats the sub-buffers in icfo order.
This helper function solves this in a generic fashion.
Example usage output of reshapeWeights above:
int32_t indir[1]{1, 0}
int32_t stride = W*H;
for (int32_t x = 0, y = N*C; x < y; ++x)
reorderSubBuffers(out + x * stride, indir, H, W);
Input Matrix{2, 3, 2, 3}:
{ 0 2 4}, { 1 3 5} <-- {0, 0, *, *}
{12 14 16}, {13 15 17} <-- {0, 1, *, *}
{24 26 28}, {25 27 29} <-- {0, 2, *, *}
{ 6 8 10}, { 7 9 11} <-- {1, 0, *, *}
{18 20 22}, {19 21 23} <-- {1, 1, *, *}
{30 32 34}, {31 33 35} <-- {1, 2, *, *}
Output Matrix{2, 3, 2, 3}:
{ 1 3 5}, { 0 2 4} <-- {0, 0, *, *}
{13 15 17}, {12 14 16} <-- {0, 1, *, *}
{25 27 29}, {24 26 28} <-- {0, 2, *, *}
{ 7 9 11}, { 6 8 10} <-- {1, 0, *, *}
{19 21 23}, {18 20 22} <-- {1, 1, *, *}
{31 33 35}, {30 32 34} <-- {1, 2, *, *}reshapeWeights()
@Namespace(value="nvinfer1::utils") @Cast(value="bool") @Deprecated @NoException(value=true) public static boolean reorderSubBuffers(Pointer input, @Const IntBuffer order, int num, int size)
@Namespace(value="nvinfer1::utils") @Cast(value="bool") @Deprecated @NoException(value=true) public static boolean reorderSubBuffers(Pointer input, @Const int[] order, int num, int size)
@Namespace(value="nvinfer1::utils") @Cast(value="bool") @Deprecated @NoException(value=true) public static boolean transposeSubBuffers(Pointer input, nvinfer.DataType type, int num, int height, int width)
input
- The input data to transpose.type
- The type of the data to transpose.num
- The number of data sub-buffers to transpose.height
- The size of the height dimension to transpose.width
- The size of the width dimension to transpose.
\brief Transpose \p num sub-buffers of \p height * \p width.@Namespace(value="nvinfer1::utils") @Cast(value="bool") @Deprecated @NoException(value=true) public static boolean transposeSubBuffers(Pointer input, @Cast(value="nvinfer1::DataType") int type, int num, int height, int width)
Copyright © 2024. All rights reserved.