@Operator public final class Batch extends PrimitiveOp
When many instances of this Op are being run concurrently with the same container/shared_name in the same device, some will output zero-shaped Tensors and others will output Tensors of size up to max_batch_size.
All Tensors in in_tensors are batched together (so, for example, labels and features should be batched with a single instance of this operation.
Each invocation of batch emits an `id` scalar which will be used to identify this particular invocation when doing unbatch or its gradient.
Each op which emits a non-empty batch will also emit a non-empty batch_index Tensor, which, is a [K, 3] matrix where each row contains the invocation's id, start, and length of elements of each set of Tensors present in batched_tensors.
Batched tensors are concatenated along the first dimension, and all tensors in in_tensors must have the first dimension of the same size.
in_tensors: The tensors to be batched. num_batch_threads: Number of scheduling threads for processing batches of work. Determines the number of batches processed in parallel. max_batch_size: Batch sizes will never be bigger than this. batch_timeout_micros: Maximum number of microseconds to wait before outputting an incomplete batch. allowed_batch_sizes: Optional list of allowed batch sizes. If left empty, does nothing. Otherwise, supplies a list of batch sizes, causing the op to pad batches up to one of those sizes. The entries must increase monotonically, and the final entry must equal max_batch_size. grad_timeout_micros: The timeout to use for the gradient. See Unbatch. batched_tensors: Either empty tensors or a batch of concatenated Tensors. batch_index: If out_tensors is non-empty, has information to invert it. container: Controls the scope of sharing of this batch. id: always contains a scalar with a unique ID for this invocation of Batch. shared_name: Concurrently running instances of batch in the same device with the same container and shared_name will batch their elements together. If left empty, the op name will be used as the shared name. T: the types of tensors to be batched.
Modifier and Type | Class and Description |
---|---|
static class |
Batch.Options
Optional attributes for
Batch |
operation
Modifier and Type | Method and Description |
---|---|
static Batch.Options |
allowedBatchSizes(List<Long> allowedBatchSizes) |
List<Output<?>> |
batchedTensors() |
Output<Long> |
batchIndex() |
static Batch.Options |
batchingQueue(String batchingQueue) |
static Batch.Options |
container(String container) |
static Batch |
create(Scope scope,
Iterable<Operand<?>> inTensors,
Long numBatchThreads,
Long maxBatchSize,
Long batchTimeoutMicros,
Long gradTimeoutMicros,
Batch.Options... options)
Factory method to create a class wrapping a new Batch operation.
|
Output<Long> |
id() |
static Batch.Options |
maxEnqueuedBatches(Long maxEnqueuedBatches) |
static Batch.Options |
sharedName(String sharedName) |
equals, hashCode, op, toString
public static Batch create(Scope scope, Iterable<Operand<?>> inTensors, Long numBatchThreads, Long maxBatchSize, Long batchTimeoutMicros, Long gradTimeoutMicros, Batch.Options... options)
scope
- current scopeinTensors
- numBatchThreads
- maxBatchSize
- batchTimeoutMicros
- gradTimeoutMicros
- options
- carries optional attributes valuespublic static Batch.Options maxEnqueuedBatches(Long maxEnqueuedBatches)
maxEnqueuedBatches
- public static Batch.Options allowedBatchSizes(List<Long> allowedBatchSizes)
allowedBatchSizes
- public static Batch.Options container(String container)
container
- public static Batch.Options sharedName(String sharedName)
sharedName
- public static Batch.Options batchingQueue(String batchingQueue)
batchingQueue
- Copyright © 2022. All rights reserved.