mx.nd.preloaded.multi.mp.sgd.update

Description

Update function for multi-precision Stochastic Gradient Descent (SDG) optimizer.

It updates the weights using:

weight = weight - learning_rate * (gradient + wd * weight)

Arguments

Argument

Description

data

NDArray-or-Symbol[].

Weights, gradients, learning rates and weight decays

rescale.grad

float, optional, default=1.

Rescale gradient to grad = rescale_grad*grad.

clip.gradient

float, optional, default=-1.

Clip gradient to the range of [-clip_gradient, clip_gradient] If clip_gradient <= 0, gradient clipping is turned off. grad = max(min(grad, clip_gradient), -clip_gradient).

num.weights

int, optional, default=’1’.

Number of updated weights.