public final class Interpreter extends Object implements AutoCloseable
A Interpreter
encapsulates a pre-trained TensorFlow Lite model, in which operations
are executed for model inference.
For example, if a model takes only one input and returns only one output:
try (Interpreter interpreter = new Interpreter(file_of_a_tensorflowlite_model)) {
interpreter.run(input, output);
}
If a model takes multiple inputs or outputs:
Object[] inputs = {input0, input1, ...};
Map<Integer, Object> map_of_indices_to_outputs = new HashMap<>();
ByteBuffer ith_output = ByteBuffer.allocateDirect(3 * 2 * 4 * 4); // Float tensor, shape 3x2x4.
ith_output.order(ByteOrder.nativeOrder());
map_of_indices_to_outputs.put(i, ith_output);
try (Interpreter interpreter = new Interpreter(file_of_a_tensorflowlite_model)) {
interpreter.runForMultipleInputsOutputs(inputs, map_of_indices_to_outputs);
}
If a model takes or produces string tensors:
String[] input = {"foo", "bar"}; // Input tensor shape is [2].
String[] output = new String[3][2]; // Output tensor shape is [3, 2].
try (Interpreter interpreter = new Interpreter(file_of_a_tensorflowlite_model)) {
interpreter.runForMultipleInputsOutputs(input, output);
}
Orders of inputs and outputs are determined when converting TensorFlow model to TensorFlowLite model with Toco, as are the default shapes of the inputs.
When inputs are provided as (multi-dimensional) arrays, the corresponding input tensor(s) will
be implicitly resized according to that array's shape. When inputs are provided as ByteBuffer
types, no implicit resizing is done; the caller must ensure that the ByteBuffer
byte size either matches that of the corresponding tensor, or that they first resize
the tensor via #resizeInput()
. Tensor shape and type information can be obtained via the
Tensor
class, available via getInputTensor(int)
and getOutputTensor(int)
.
WARNING:Instances of a Interpreter
is not thread-safe. A Interpreter
owns resources that must be explicitly freed by invoking close()
Modifier and Type | Class and Description |
---|---|
static class |
Interpreter.Options
An options class for controlling runtime interpreter behavior.
|
Constructor and Description |
---|
Interpreter(@NonNull ByteBuffer byteBuffer)
Initializes a
Interpreter with a ByteBuffer of a model file. |
Interpreter(@NonNull ByteBuffer byteBuffer,
int numThreads)
Deprecated.
Prefer using the
Interpreter(ByteBuffer,Options) constructor. This method
will be removed in a future release. |
Interpreter(@NonNull ByteBuffer byteBuffer,
Interpreter.Options options)
Initializes a
Interpreter with a ByteBuffer of a model file and a set of custom
#Options . |
Interpreter(@NonNull File modelFile)
Initializes a
Interpreter |
Interpreter(@NonNull File modelFile,
int numThreads)
Deprecated.
Prefer using the
Interpreter(File,Options) constructor. This method will
be removed in a future release. |
Interpreter(@NonNull File modelFile,
Interpreter.Options options)
Initializes a
Interpreter and specifies the number of threads used for inference. |
Interpreter(@NonNull MappedByteBuffer mappedByteBuffer)
Deprecated.
Prefer using the
Interpreter(ByteBuffer,Options) constructor. This method
will be removed in a future release. |
Modifier and Type | Method and Description |
---|---|
void |
close()
Release resources associated with the
Interpreter . |
protected void |
finalize() |
int |
getInputIndex(String opName)
Gets index of an input given the op name of the input.
|
Tensor |
getInputTensor(int inputIndex)
Gets the Tensor associated with the provdied input index.
|
int |
getInputTensorCount()
Gets the number of input tensors.
|
Long |
getLastNativeInferenceDurationNanoseconds()
Returns native inference timing.
|
int |
getOutputIndex(String opName)
Gets index of an output given the op name of the output.
|
Tensor |
getOutputTensor(int outputIndex)
Gets the Tensor associated with the provdied output index.
|
int |
getOutputTensorCount()
Gets the number of output Tensors.
|
void |
modifyGraphWithDelegate(Delegate delegate)
Advanced: Modifies the graph with the provided
Delegate . |
void |
resetVariableTensors()
Advanced: Resets all variable tensors to the default value.
|
void |
resizeInput(int idx,
@NonNull int[] dims)
Resizes idx-th input of the native model to the given dims.
|
void |
run(Object input,
Object output)
Runs model inference if the model takes only one input, and provides only one output.
|
void |
runForMultipleInputsOutputs(@NonNull Object[] inputs,
@NonNull Map<Integer,Object> outputs)
Runs model inference if the model takes multiple inputs, or returns multiple outputs.
|
void |
setNumThreads(int numThreads)
Deprecated.
Prefer using
Interpreter.Options.setNumThreads(int) directly for controlling thread
multi-threading. This method will be removed in a future release. |
void |
setUseNNAPI(boolean useNNAPI)
Deprecated.
Prefer using
Interpreter.Options.setUseNNAPI(boolean) directly for enabling NN API.
This method will be removed in a future release. |
public Interpreter(@NonNull File modelFile)
Interpreter
modelFile:
- a File of a pre-trained TF Lite model.@Deprecated public Interpreter(@NonNull File modelFile, int numThreads)
Interpreter(File,Options)
constructor. This method will
be removed in a future release.Interpreter
and specifies the number of threads used for inference.modelFile:
- a file of a pre-trained TF Lite modelnumThreads:
- number of threads to use for inferencepublic Interpreter(@NonNull File modelFile, Interpreter.Options options)
Interpreter
and specifies the number of threads used for inference.modelFile:
- a file of a pre-trained TF Lite modeloptions:
- a set of options for customizing interpreter behaviorpublic Interpreter(@NonNull ByteBuffer byteBuffer)
Interpreter
with a ByteBuffer
of a model file.
The ByteBuffer should not be modified after the construction of a Interpreter
. The
ByteBuffer
can be either a MappedByteBuffer
that memory-maps a model file, or a
direct ByteBuffer
of nativeOrder() that contains the bytes content of a model.
@Deprecated public Interpreter(@NonNull ByteBuffer byteBuffer, int numThreads)
Interpreter(ByteBuffer,Options)
constructor. This method
will be removed in a future release.Interpreter
with a ByteBuffer
of a model file and specifies the
number of threads used for inference.
The ByteBuffer should not be modified after the construction of a Interpreter
. The
ByteBuffer
can be either a MappedByteBuffer
that memory-maps a model file, or a
direct ByteBuffer
of nativeOrder() that contains the bytes content of a model.
@Deprecated public Interpreter(@NonNull MappedByteBuffer mappedByteBuffer)
Interpreter(ByteBuffer,Options)
constructor. This method
will be removed in a future release.Interpreter
with a MappedByteBuffer
to the model file.
The MappedByteBuffer
should remain unchanged after the construction of a Interpreter
.
public Interpreter(@NonNull ByteBuffer byteBuffer, Interpreter.Options options)
Interpreter
with a ByteBuffer
of a model file and a set of custom
#Options
.
The ByteBuffer should not be modified after the construction of a Interpreter
. The
ByteBuffer
can be either a MappedByteBuffer
that memory-maps a model file, or a
direct ByteBuffer
of nativeOrder() that contains the bytes content of a model.
public void run(Object input, Object output)
Warning: The API runs much faster if ByteBuffer
is used as input data type. Please
consider using ByteBuffer
to feed primitive input data for better performance.
input
- an array or multidimensional array, or a ByteBuffer
of primitive types
including int, float, long, and byte. ByteBuffer
is the preferred way to pass large
input data for primitive types, whereas string types require using the (multi-dimensional)
array input path. When ByteBuffer
is used, its content should remain unchanged
until model inference is done. A null
value is allowed only if the caller is using
a Delegate
that allows buffer handle interop, and such a buffer has been bound to
the input Tensor
.output
- a multidimensional array of output data, or a ByteBuffer
of primitive
types including int, float, long, and byte. A null value is allowed only if the caller is
using a Delegate
that allows buffer handle interop, and such a buffer has been
bound to the output Tensor
. See also Options#setAllowBufferHandleOutput()
.public void runForMultipleInputsOutputs(@NonNull Object[] inputs, @NonNull Map<Integer,Object> outputs)
Warning: The API runs much faster if ByteBuffer
is used as input data type. Please
consider using ByteBuffer
to feed primitive input data for better performance.
Note: null
values for invididual elements of inputs
and outputs
is
allowed only if the caller is using a Delegate
that allows buffer handle interop, and
such a buffer has been bound to the corresponding input or output Tensor
(s).
inputs
- an array of input data. The inputs should be in the same order as inputs of the
model. Each input can be an array or multidimensional array, or a ByteBuffer
of
primitive types including int, float, long, and byte. ByteBuffer
is the preferred
way to pass large input data, whereas string types require using the (multi-dimensional)
array input path. When ByteBuffer
is used, its content should remain unchanged
until model inference is done.outputs
- a map mapping output indices to multidimensional arrays of output data or ByteBuffer
s of primitive types including int, float, long, and byte. It only needs to keep
entries for the outputs to be used.public void resizeInput(int idx, @NonNull int[] dims)
IllegalArgumentException will be thrown if it fails to resize.
public int getInputTensorCount()
public int getInputIndex(String opName)
IllegalArgumentException will be thrown if the op name does not exist in the model file used
to initialize the Interpreter
.
public Tensor getInputTensor(int inputIndex)
IllegalArgumentException will be thrown if the provided index is invalid.
public int getOutputTensorCount()
public int getOutputIndex(String opName)
IllegalArgumentException will be thrown if the op name does not exist in the model file used
to initialize the Interpreter
.
public Tensor getOutputTensor(int outputIndex)
IllegalArgumentException will be thrown if the provided index is invalid.
public Long getLastNativeInferenceDurationNanoseconds()
IllegalArgumentException will be thrown if the model is not initialized by the Interpreter
.
@Deprecated public void setUseNNAPI(boolean useNNAPI)
Interpreter.Options.setUseNNAPI(boolean)
directly for enabling NN API.
This method will be removed in a future release.@Deprecated public void setNumThreads(int numThreads)
Interpreter.Options.setNumThreads(int)
directly for controlling thread
multi-threading. This method will be removed in a future release.public void modifyGraphWithDelegate(Delegate delegate)
Delegate
.
Note: The typical path for providing delegates is via Interpreter.Options.addDelegate(org.tensorflow.lite.Delegate)
, at
creation time. This path should only be used when a delegate might require coordinated
interaction between Interpeter creation and delegate application.
WARNING: This is an experimental API and subject to change.
public void resetVariableTensors()
If a variable tensor doesn't have an associated buffer, it will be reset to zero.
WARNING: This is an experimental API and subject to change.
public void close()
Interpreter
.close
in interface AutoCloseable
Copyright © 2022. All rights reserved.