mx.symbol.signum_update
¶
Description¶
SIGN momentUM (Signum) optimizer.
- It updates the weights using::
state = momentum * state + (1-momentum) * gradient weight = weight - learning_rate * sign(state)
Where the parameter
momentum
is the decay rate of momentum estimates at each epoch.
Note
sparse ndarray not supported for this optimizer yet.
Usage¶
mx.symbol.signum_update(...)
Arguments¶
Argument |
Description |
---|---|
|
NDArray-or-Symbol. Weight |
|
NDArray-or-Symbol. Gradient |
|
NDArray-or-Symbol. Momentum |
|
float, required. Learning rate |
|
float, optional, default=0. The decay rate of momentum estimates at each epoch. |
|
float, optional, default=0. Weight decay augments the objective function with a regularization term that penalizes large weights. The penalty scales with the square of the magnitude of each weight. |
|
float, optional, default=1. Rescale gradient to grad = rescale_grad*grad. |
|
float, optional, default=-1. Clip gradient to the range of [-clip_gradient, clip_gradient] If clip_gradient <= 0, gradient clipping is turned off. grad = max(min(grad, clip_gradient), -clip_gradient). |
|
float, optional, default=0. The amount of weight decay that does not go into gradient/momentum calculationsotherwise do weight decay algorithmically only. |
|
string, optional. Name of the resulting symbol. |
Value¶
out
The result mx.symbol
Link to Source Code: http://github.com/apache/incubator-mxnet/blob/1.6.0/src/operator/optimizer_op.cc#L92