org.bytedeco.tensorrt.nvinfer

Class IDequantizeLayer

java.lang.Object

org.bytedeco.javacpp.Pointer

org.bytedeco.tensorrt.nvinfer.INoCopy

org.bytedeco.tensorrt.nvinfer.ILayer

org.bytedeco.tensorrt.nvinfer.IDequantizeLayer

All Implemented Interfaces:

AutoCloseable

@Namespace(value="nvinfer1")
 @NoOffset
 @Properties(inherit=nvinfer.class)
public class IDequantizeLayer
extends ILayer

\class IDequantizeLayer \brief A Dequantize layer in a network definition. This layer accepts a quantized type input tensor, and uses the configured scale and zeroPt inputs to dequantize the input according to: \p output = (\p input - \p zeroPt) * \p scale The first input (index 0) is the tensor to be dequantized. The second (index 1) and third (index 2) are the scale and zero point respectively. \p scale and \p zeroPt should have identical dimensions, and a rank that is lower or equal to 2. The \p zeroPt tensor is optional, and if not set, will be assumed to be zero. Its data type must be identical to the input's data type. \p zeroPt must only contain zero-valued coefficients, because only symmetric quantization is supported. The \p scale value must be a scalar for per-tensor quantization, a 1D tensor for per-channel quantization, or the same rank as the input tensor for block quantization. All \p scale coefficients must have strictly positive values. The size of the 1D \p scale tensor must match the size of the quantization axis. For block quantization, the shape of \p scale tensor must match the shape of the input, except for one dimension (the last or second to last dimension) in which blocking occurs. The size of \p zeroPt must match the size of \p scale. The subgraph which terminates with the \p zeroPt tensor must be a build-time constant containing only zeros. The output type, if constrained, must be constrained to DataType::kFLOAT, DataType::kHALF, or DataType::kBF16. The input type, if constrained, must be constrained to DataType::kINT8, DataType::kFP8, DataType::kINT4 or DataType::kFP4. The output size is the same as the input size. The quantization axis is in reference to the input tensor's dimensions. IDequantizeLayer supports DataType::kINT8 (default), DataType::kFP8, DataType::kINT4 or DataType::kFP4. For strongly typed networks, \p input data type must be the same as \p zeroPt data type. IDequantizeLayer supports DataType::kFLOAT, DataType::kHALF, or DataType::kBF16 output. The output data type must be configured explicitly using \p setToType. As an example of the operation of this layer, imagine a 4D NCHW activation input which can be quantized using a single scale coefficient (referred to as per-tensor quantization): For each n in N: For each c in C: For each h in H: For each w in W: output[n,c,h,w] = (\p input[n,c,h,w] - \p zeroPt) * \p scale Per-channel dequantization is supported only for input that is rooted at an IConstantLayer (i.e. weights). Activations cannot be quantized per-channel. As an example of per-channel operation, imagine a 4D KCRS weights input and K (dimension 0) as the quantization axis. The scale is an array of coefficients, which is the same size as the quantization axis. For each k in K: For each c in C: For each r in R: For each s in S: output[k,c,r,s] = (\p input[k,c,r,s] - \p zeroPt[k]) * \p scale[k] Block dequantization is supported for input types DataType::kFP4, DataType::kFP8 and DataType::kINT4. As an example of blocked operation, imagine a 2D RS input with R (dimension 0) as the blocking axis and B as the block size. The scale is a 2D array of coefficients, with dimensions (R//B, S). For each r in R: For each s in S: output[r,s] = (\p input[r,s] - \p zeroPt[r//B, s]) * \p scale[r//B, s] \note Only symmetric quantization is supported. \note Currently the only allowed build-time constant \p zeroPt subgraphs are: 1. Constant -> Quantize 2. Constant -> Cast -> Quantize \note The input tensor for this layer must not be a scalar. \warning Do not inherit from this class, as doing so will break forward-compatibility of the API and ABI.

Nested Class Summary

Nested classes/interfaces inherited from class org.bytedeco.javacpp.Pointer

Pointer.CustomDeallocator, Pointer.Deallocator, Pointer.NativeDeallocator, Pointer.ReferenceCounter

Field Summary

Fields inherited from class org.bytedeco.javacpp.Pointer

address, capacity, limit, position

Constructor Summary

Constructors

Constructor and Description
IDequantizeLayer(Pointer p) Pointer cast constructor.

Method Summary

Modifier and Type	Method and Description
int	getAxis() \brief Get the quantization axis.
Dims64	getBlockShape() \brief Get the shape of the quantization block.
nvinfer.DataType	getToType() \brief Return the Dequantize layer output type.
void	setAxis(int axis) \brief Set the quantization axis.
boolean	setBlockShape(Dims64 blockShape) \brief Set the shape of the quantization block.
void	setToType(int toType)
void	setToType(nvinfer.DataType toType) \brief Set the Dequantize layer output type.

Methods inherited from class org.bytedeco.tensorrt.nvinfer.ILayer

getInput, getMetadata, getName, getNbInputs, getNbOutputs, getOutput, getOutputType, getPrecision, getType, outputTypeIsSet, precisionIsSet, resetOutputType, resetPrecision, setInput, setMetadata, setMetadata, setName, setName, setOutputType, setOutputType, setPrecision, setPrecision

Methods inherited from class org.bytedeco.javacpp.Pointer

address, asBuffer, asByteBuffer, availablePhysicalBytes, calloc, capacity, capacity, close, deallocate, deallocate, deallocateReferences, deallocator, deallocator, equals, fill, formatBytes, free, getDirectBufferAddress, getPointer, getPointer, getPointer, getPointer, hashCode, interruptDeallocatorThread, isNull, isNull, limit, limit, malloc, maxBytes, maxPhysicalBytes, memchr, memcmp, memcpy, memmove, memset, offsetAddress, offsetof, offsetof, parseBytes, physicalBytes, physicalBytesInaccurate, position, position, put, realloc, referenceCount, releaseReference, retainReference, setNull, sizeof, sizeof, toString, totalBytes, totalCount, totalPhysicalBytes, trimMemory, withDeallocator, zero

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

IDequantizeLayer

public IDequantizeLayer(Pointer p)

Pointer cast constructor. Invokes Pointer(Pointer).

Method Detail

getAxis

@NoException(value=true)
public int getAxis()

\brief Get the quantization axis.

Returns:

axis parameter set by setAxis(). The return value is the index of the quantization axis in the input tensor's dimensions. A value of -1 indicates per-tensor quantization. The default value is -1.

setAxis

@NoException(value=true)
public void setAxis(int axis)

\brief Set the quantization axis. Set the index of the quantization axis (with reference to the input tensor's dimensions). The axis must be a valid axis if the scale tensor has more than one coefficient. The axis value will be ignored if the scale tensor has exactly one coefficient (per-tensor quantization).

setBlockShape

@Cast(value="bool")
 @NoException(value=true)
public boolean setBlockShape(@Cast(value="const nvinfer1::Dims*") @ByRef
                                                                           Dims64 blockShape)

\brief Set the shape of the quantization block.

Parameters:

blockShape - The shape of the quantization block. Set the shape of the quantization block. Allowed values are positive values and -1 which denotes a fully blocked dimension. Returns true if the block shape was set successfully, false if the block shape is invalid. The default value is empty Dims.

See Also:

getBlockShape()

getBlockShape

@ByVal
 @Cast(value="nvinfer1::Dims*")
 @NoException(value=true)
public Dims64 getBlockShape()

\brief Get the shape of the quantization block. The default value is empty Dims.

See Also:

setBlockShape()

setToType

@NoException(value=true)
public void setToType(nvinfer.DataType toType)

\brief Set the Dequantize layer output type.

Parameters:

toType - The DataType of the output tensor. Set the output type of the dequantize layer. Valid values are DataType::kFLOAT, DataType::kHALF and DataType::kBF16. If the network is strongly typed, setToType must be used to set the output type, and use of setOutputType is an error. Otherwise, types passed to setOutputType and setToType must be the same.

See Also:

NetworkDefinitionCreationFlag::kSTRONGLY_TYPED

setToType

@NoException(value=true)
public void setToType(@Cast(value="nvinfer1::DataType")
                                               int toType)

getToType

@NoException(value=true)
public nvinfer.DataType getToType()

\brief Return the Dequantize layer output type.

Returns:

toType parameter set during layer creation or by setToType(). The return value is the output type of the quantize layer. The default value is DataType::kFLOAT.