mx.nd.ftml.update
¶
Description¶
The FTML optimizer described in FTML - Follow the Moving Leader in Deep Learning, available at http://proceedings.mlr.press/v70/zheng17a/zheng17a.pdf.
Arguments¶
Argument |
Description |
---|---|
|
NDArray-or-Symbol. Weight |
|
NDArray-or-Symbol. Gradient |
|
NDArray-or-Symbol. Internal state |
|
NDArray-or-Symbol. Internal state |
|
NDArray-or-Symbol. Internal state |
|
float, required. Learning rate. |
|
float, optional, default=0.600000024. Generally close to 0.5. |
|
float, optional, default=0.999000013. Generally close to 1. |
|
double, optional, default=9.9999999392252903e-09. Epsilon to prevent div 0. |
|
int, required. Number of update. |
|
float, optional, default=0. Weight decay augments the objective function with a regularization term that penalizes large weights. The penalty scales with the square of the magnitude of each weight. |
|
float, optional, default=1. Rescale gradient to grad = rescale_grad*grad. |
|
float, optional, default=-1. Clip gradient to the range of [-clip_gradient, clip_gradient] If clip_gradient <= 0, gradient clipping is turned off. grad = max(min(grad, clip_gradient), -clip_gradient). |
Value¶
out
The result mx.ndarray
Link to Source Code: http://github.com/apache/incubator-mxnet/blob/1.6.0/src/operator/optimizer_op.cc#L640