Optimization

Initialize and update model weights during training

Optimizers

mx.opt.adadelta

Create an AdaDelta optimizer with respective parameters

mx.opt.adagrad

Create an AdaGrad optimizer with respective parameters

mx.opt.adam

Create an Adam optimizer with respective parameters

mx.opt.create

Create an optimizer by name and parameters

mx.opt.get.updater

Get an updater closure that can take list of weight and gradient and return updated list of weight

mx.opt.nag

Create a Nesterov Accelerated SGD( NAG) optimizer

mx.opt.rmsprop

Create an RMSProp optimizer with respective parameters

mx.opt.sgd

Create an SGD optimizer with respective parameters

Initialization

mx.init.Xavier

Xavier initializer

mx.init.create

Create initialization of argument like arg.array

mx.init.internal.default

Internal default value initialization scheme

mx.init.normal

Create a initializer that initialize the weight with normal(0, sd)

mx.init.uniform

Create a initializer that initialize the weight with uniform [-scale, scale]

mx.model.init.params

Parameter initialization

Learning rate schedule

mx.lr_scheduler.FactorScheduler

Learning rate scheduler

mx.lr_scheduler.MultiFactorScheduler

Multifactor learning rate scheduler

Optimizer updates (NDArray)

mx.nd.adam.update

Update function for Adam optimizer

mx.nd.all.finite

Check if all the float numbers in the array are finite (used for AMP)

mx.nd.ftml.update

The FTML optimizer described in FTML - Follow the Moving Leader in Deep Learning, available at http://proceedings.mlr.press/v70/zheng17a/zheng17a.pdf

mx.nd.ftrl.update

Update function for Ftrl optimizer

mx.nd.lamb.update.phase1

Phase I of lamb update it performs the following operations and returns g:

mx.nd.lamb.update.phase2

Phase II of lamb update it performs the following operations and updates grad

mx.nd.mp.lamb.update.phase1

Mixed Precision version of Phase I of lamb update it performs the following operations and returns g:

mx.nd.mp.lamb.update.phase2

Mixed Precision version Phase II of lamb update it performs the following operations and updates grad

mx.nd.mp.nag.mom.update

Update function for multi-precision Nesterov Accelerated Gradient( NAG) optimizer

mx.nd.mp.sgd.mom.update

Updater function for multi-precision sgd optimizer

mx.nd.mp.sgd.update

Updater function for multi-precision sgd optimizer

mx.nd.multi.all.finite

Check if all the float numbers in all the arrays are finite (used for AMP)

mx.nd.multi.mp.sgd.mom.update

Momentum update function for multi-precision Stochastic Gradient Descent (SGD) optimizer

mx.nd.multi.mp.sgd.update

Update function for multi-precision Stochastic Gradient Descent (SDG) optimizer

mx.nd.multi.sgd.mom.update

Momentum update function for Stochastic Gradient Descent (SGD) optimizer

mx.nd.multi.sgd.update

Update function for Stochastic Gradient Descent (SDG) optimizer

mx.nd.nag.mom.update

Update function for Nesterov Accelerated Gradient( NAG) optimizer

mx.nd.preloaded.multi.mp.sgd.mom.update

Momentum update function for multi-precision Stochastic Gradient Descent (SGD) optimizer

mx.nd.preloaded.multi.mp.sgd.update

Update function for multi-precision Stochastic Gradient Descent (SDG) optimizer

mx.nd.preloaded.multi.sgd.mom.update

Momentum update function for Stochastic Gradient Descent (SGD) optimizer

mx.nd.preloaded.multi.sgd.update

Update function for Stochastic Gradient Descent (SDG) optimizer

mx.nd.rmsprop.update

Update function for RMSProp optimizer

mx.nd.rmspropalex.update

Update function for RMSPropAlex optimizer

mx.nd.sgd.mom.update

Momentum update function for Stochastic Gradient Descent (SGD) optimizer

mx.nd.sgd.update

Update function for Stochastic Gradient Descent (SGD) optimizer

mx.nd.signsgd.update

Update function for SignSGD optimizer

mx.nd.signum.update

SIGN momentUM (Signum) optimizer

Optimizer updates (Symbol)

mx.symbol.adam_update

Update function for Adam optimizer

mx.symbol.all_finite

Check if all the float numbers in the array are finite (used for AMP)

mx.symbol.ftml_update

The FTML optimizer described in FTML - Follow the Moving Leader in Deep Learning, available at http://proceedings.mlr.press/v70/zheng17a/zheng17a.pdf

mx.symbol.ftrl_update

Update function for Ftrl optimizer

mx.symbol.mp_sgd_mom_update

Updater function for multi-precision sgd optimizer

mx.symbol.mp_sgd_update

Updater function for multi-precision sgd optimizer

mx.symbol.multi_all_finite

Check if all the float numbers in all the arrays are finite (used for AMP)

mx.symbol.multi_mp_sgd_mom_update

Momentum update function for multi-precision Stochastic Gradient Descent (SGD) optimizer

mx.symbol.multi_mp_sgd_update

Update function for multi-precision Stochastic Gradient Descent (SDG) optimizer

mx.symbol.multi_sgd_mom_update

Momentum update function for Stochastic Gradient Descent (SGD) optimizer

mx.symbol.multi_sgd_update

Update function for Stochastic Gradient Descent (SDG) optimizer

mx.symbol.preloaded_multi_mp_sgd_mom_update

Momentum update function for multi-precision Stochastic Gradient Descent (SGD) optimizer

mx.symbol.preloaded_multi_mp_sgd_update

Update function for multi-precision Stochastic Gradient Descent (SDG) optimizer

mx.symbol.preloaded_multi_sgd_mom_update

Momentum update function for Stochastic Gradient Descent (SGD) optimizer

mx.symbol.preloaded_multi_sgd_update

Update function for Stochastic Gradient Descent (SDG) optimizer

mx.symbol.sgd_mom_update

Momentum update function for Stochastic Gradient Descent (SGD) optimizer

mx.symbol.sgd_update

Update function for Stochastic Gradient Descent (SGD) optimizer

mx.symbol.signsgd_update

Update function for SignSGD optimizer

mx.symbol.signum_update

SIGN momentUM (Signum) optimizer