gluon.Parameter

class Parameter(name='weight', grad_req='write', shape=None, dtype=<class 'numpy.float32'>, lr_mult=1.0, wd_mult=1.0, init=None, allow_deferred_init=False, differentiable=True, stype='default', grad_stype='default')[source]

Bases: object

A Container holding parameters (weights) of Blocks.

Parameter holds a copy of the parameter on each Device after it is initialized with Parameter.initialize(...). If grad_req is not 'null', it will also hold a gradient array on each Device:

device = mx.gpu(0)
x = mx.np.zeros((16, 100), device=device)
w = mx.gluon.Parameter('fc_weight', shape=(64, 100), init=mx.init.Xavier())
b = mx.gluon.Parameter('fc_bias', shape=(64,), init=mx.init.Zero())
w.initialize(device=device)
b.initialize(device=device)
out = mx.npx.fully_connected(x, w.data(device), b.data(device), num_hidden=64)

Methods

cast(dtype)

Cast data and gradient of this Parameter to a new data type.

data([device])

Returns a copy of this parameter on one device.

grad([device])

Returns a gradient buffer for this parameter on one device.

initialize([init, device, default_init, …])

Initializes parameter and gradient arrays.

list_ctx()

This function has been deprecated.

list_data()

Returns copies of this parameter on all devices, in the same order as creation.

list_device()

Returns a list of devices this parameter is initialized on.

list_grad()

Returns gradient buffers on all devices, in the same order as values().

list_row_sparse_data(row_id)

Returns copies of the ‘row_sparse’ parameter on all devices, in the same order as creation.

reset_ctx(ctx)

This function has been deprecated.

reset_device(device)

Re-assign Parameter to other devices.

row_sparse_data(row_id)

Returns a copy of the ‘row_sparse’ parameter on the same device as row_id’s.

set_data(data)

Sets this parameter’s value on all devices.

var()

Returns a symbol representing this parameter.

zero_grad()

Sets gradient buffer on all devices to 0.

Attributes

dtype

The type of the parameter.

shape

The shape of the parameter.

Parameters
  • name (str, default 'weight') – Name of this parameter. It decides the corresponding default initializer.

  • grad_req ({'write', 'add', 'null'}, default 'write') –

    Specifies how to update gradient to grad arrays.

    • 'write' means everytime gradient is written to grad NDArray.

    • 'add' means everytime gradient is added to the grad NDArray. You need to manually call zero_grad() to clear the gradient buffer before each iteration when using this option.

    • ’null’ means gradient is not requested for this parameter. gradient arrays will not be allocated.

  • shape (int or tuple of int, default None) – Shape of this parameter. By default shape is not specified. Parameter with unknown shape can be used for Symbol API, but init will throw an error when using NDArray API.

  • dtype (numpy.dtype or str, default 'float32') – Data type of this parameter. For example, numpy.float32 or 'float32'.

  • lr_mult (float, default 1.0) – Learning rate multiplier. Learning rate will be multiplied by lr_mult when updating this parameter with optimizer.

  • wd_mult (float, default 1.0) – Weight decay multiplier (L2 regularizer coefficient). Works similar to lr_mult.

  • init (Initializer, default None) – Initializer of this parameter. Will use the global initializer by default.

  • stype ({'default', 'row_sparse', 'csr'}, defaults to 'default'.) – The storage type of the parameter.

  • grad_stype ({'default', 'row_sparse', 'csr'}, defaults to 'default'.) – The storage type of the parameter’s gradient.

grad_req

This can be set before or after initialization. Setting grad_req to 'null' with x.grad_req = 'null' saves memory and computation when you don’t need gradient w.r.t x.

Type

{‘write’, ‘add’, ‘null’}

lr_mult

Local learning rate multiplier for this Parameter. The actual learning rate is calculated with learning_rate * lr_mult. You can set it with param.lr_mult = 2.0

Type

float

wd_mult

Local weight decay multiplier for this Parameter.

Type

float

cast(dtype)[source]

Cast data and gradient of this Parameter to a new data type.

Parameters

dtype (str or numpy.dtype) – The new data type.

data(device=None)[source]

Returns a copy of this parameter on one device. Must have been initialized on this device before. For sparse parameters, use Parameter.row_sparse_data() instead.

Parameters

device (Device) – Desired device.

Returns

Return type

NDArray on device

property dtype

The type of the parameter.

Setting the dtype value is equivalent to casting the value of the parameter

grad(device=None)[source]

Returns a gradient buffer for this parameter on one device.

Parameters

device (Device) – Desired device.

initialize(init=None, device=None, default_init=<mxnet.initializer.Uniform object>, force_reinit=False)[source]

Initializes parameter and gradient arrays. Only used for NDArray API.

Parameters
  • init (Initializer) – The initializer to use. Overrides Parameter.init() and default_init.

  • device (Device or list of Device, default device.current_device().) –

    Assign Parameter to given device. If device is a list of Device, a copy will be made for each device.

    Note

    Copies are independent arrays. User is responsible for keeping their values consistent when updating. Normally gluon.Trainer does this for you.

  • default_init (Initializer) – Default initializer is used when both init() and Parameter.init() are None.

  • force_reinit (bool, default False) – Whether to force re-initialization if parameter is already initialized.

Examples

>>> weight = mx.gluon.Parameter('weight', shape=(2, 2))
>>> weight.initialize(device=mx.cpu(0))
>>> weight.data()
[[-0.01068833  0.01729892]
 [ 0.02042518 -0.01618656]]
<NDArray 2x2 @cpu(0)>
>>> weight.grad()
[[ 0.  0.]
 [ 0.  0.]]
<NDArray 2x2 @cpu(0)>
>>> weight.initialize(device=[mx.gpu(0), mx.gpu(1)])
>>> weight.data(mx.gpu(0))
[[-0.00873779 -0.02834515]
 [ 0.05484822 -0.06206018]]
<NDArray 2x2 @gpu(0)>
>>> weight.data(mx.gpu(1))
[[-0.00873779 -0.02834515]
 [ 0.05484822 -0.06206018]]
<NDArray 2x2 @gpu(1)>
list_ctx()[source]

This function has been deprecated. Please refer to Parameter.list_device.

list_data()[source]

Returns copies of this parameter on all devices, in the same order as creation. For sparse parameters, use Parameter.list_row_sparse_data() instead.

Returns

Return type

list of NDArrays

list_device()[source]

Returns a list of devices this parameter is initialized on.

list_grad()[source]

Returns gradient buffers on all devices, in the same order as values().

list_row_sparse_data(row_id)[source]

Returns copies of the ‘row_sparse’ parameter on all devices, in the same order as creation. The copy only retains rows whose ids occur in provided row ids. The parameter must have been initialized before.

Parameters

row_id (NDArray) – Row ids to retain for the ‘row_sparse’ parameter.

Returns

Return type

list of NDArrays

reset_ctx(ctx)[source]

This function has been deprecated. Please refer to Parameter.reset_device.

reset_device(device)[source]

Re-assign Parameter to other devices.

Parameters

device (Device or list of Device, default device.current_device().) – Assign Parameter to given device. If device is a list of Device, a copy will be made for each device.

row_sparse_data(row_id)[source]

Returns a copy of the ‘row_sparse’ parameter on the same device as row_id’s. The copy only retains rows whose ids occur in provided row ids. The parameter must have been initialized on this device before.

Parameters

row_id (NDArray) – Row ids to retain for the ‘row_sparse’ parameter.

Returns

Return type

NDArray on row_id’s device

set_data(data)[source]

Sets this parameter’s value on all devices.

property shape

The shape of the parameter.

By default, an unknown dimension size is 0. However, when the NumPy semantic is turned on, unknown dimension size is -1.

var()[source]

Returns a symbol representing this parameter.

zero_grad()[source]

Sets gradient buffer on all devices to 0. No action is taken if parameter is uninitialized or doesn’t require gradient.