@Operator public final class ResourceApplyAdamWithAmsgrad extends PrimitiveOp
$$lr_t := \text{learning\_rate} * \sqrt{1 - beta_2^t} / (1 - beta_1^t)$$ $$m_t := beta_1 * m_{t-1} + (1 - beta_1) * g$$ $$v_t := beta_2 * v_{t-1} + (1 - beta_2) * g * g$$ $$vhat_t := max{vhat_{t-1}, v_t}$$ $$variable := variable - lr_t * m_t / (\sqrt{vhat_t} + \epsilon)$$
Modifier and Type | Class and Description |
---|---|
static class |
ResourceApplyAdamWithAmsgrad.Options
Optional attributes for
ResourceApplyAdamWithAmsgrad |
operation
Modifier and Type | Method and Description |
---|---|
static <T> ResourceApplyAdamWithAmsgrad |
create(Scope scope,
Operand<?> var,
Operand<?> m,
Operand<?> v,
Operand<?> vhat,
Operand<T> beta1Power,
Operand<T> beta2Power,
Operand<T> lr,
Operand<T> beta1,
Operand<T> beta2,
Operand<T> epsilon,
Operand<T> grad,
ResourceApplyAdamWithAmsgrad.Options... options)
Factory method to create a class wrapping a new ResourceApplyAdamWithAmsgrad operation.
|
static ResourceApplyAdamWithAmsgrad.Options |
useLocking(Boolean useLocking) |
equals, hashCode, op, toString
public static <T> ResourceApplyAdamWithAmsgrad create(Scope scope, Operand<?> var, Operand<?> m, Operand<?> v, Operand<?> vhat, Operand<T> beta1Power, Operand<T> beta2Power, Operand<T> lr, Operand<T> beta1, Operand<T> beta2, Operand<T> epsilon, Operand<T> grad, ResourceApplyAdamWithAmsgrad.Options... options)
scope
- current scopevar
- Should be from a Variable().m
- Should be from a Variable().v
- Should be from a Variable().vhat
- Should be from a Variable().beta1Power
- Must be a scalar.beta2Power
- Must be a scalar.lr
- Scaling factor. Must be a scalar.beta1
- Momentum factor. Must be a scalar.beta2
- Momentum factor. Must be a scalar.epsilon
- Ridge term. Must be a scalar.grad
- The gradient.options
- carries optional attributes valuespublic static ResourceApplyAdamWithAmsgrad.Options useLocking(Boolean useLocking)
useLocking
- If `True`, updating of the var, m, and v tensors will be protected
by a lock; otherwise the behavior is undefined, but may exhibit less
contention.Copyright © 2022. All rights reserved.