# Symbol API¶

## Overview¶

This document lists the routines of the symbolic expression package:

 mxnet.symbol Symbol API of MXNet.

The Symbol API, defined in the symbol (or simply sym) package, provides neural network graphs and auto-differentiation. A symbol represents a multi-output symbolic expression. They are composited by operators, such as simple matrix operations (e.g. “+”), or a neural network layer (e.g. convolution layer). An operator can take several input variables, produce more than one output variables, and have internal state variables. A variable can be either free, which we can bind with value later, or an output of another symbol.

>>> a = mx.sym.Variable('a')
>>> b = mx.sym.Variable('b')
>>> c = 2 * a + b
>>> type(c)

>>> e = c.bind(mx.cpu(), {'a': mx.nd.array([1,2]), 'b':mx.nd.array([2,3])})
>>> y = e.forward()
>>> y
[]
>>> y[0].asnumpy()
array([ 4.,  7.], dtype=float32)


A detailed tutorial is available at Symbol - Neural network graphs and auto-differentiation.

Note

most operators provided in symbol are similar to those in ndarray although there are few differences:

• symbol adopts declarative programming. In other words, we need to first compose the computations, and then feed it with data for execution whereas ndarray adopts imperative programming.
• Most binary operators in symbol such as + and > don’t broadcast. We need to call the broadcast version of the operator such as broadcast_plus explicitly.

In the rest of this document, we first overview the methods provided by the symbol.Symbol class, and then list other routines provided by the symbol package.

## The Symbol class¶

### Composition¶

Composite multiple symbols into a new one by an operator.

 Symbol.__call__ Composes symbol using inputs.

#### Arithmetic operations¶

 Symbol.__add__ x.__add__(y) <=> x+y Symbol.__sub__ x.__sub__(y) <=> x-y Symbol.__rsub__ x.__rsub__(y) <=> y-x Symbol.__neg__ x.__neg__() <=> -x Symbol.__mul__ x.__mul__(y) <=> x*y Symbol.__div__ x.__div__(y) <=> x/y Symbol.__rdiv__ x.__rdiv__(y) <=> y/x Symbol.__mod__ x.__mod__(y) <=> x%y Symbol.__rmod__ x.__rmod__(y) <=> y%x Symbol.__pow__ x.__pow__(y) <=> x**y

#### Trigonometric functions¶

 Symbol.sin Convenience fluent method for sin(). Symbol.cos Convenience fluent method for cos(). Symbol.tan Convenience fluent method for tan(). Symbol.arcsin Convenience fluent method for arcsin(). Symbol.arccos Convenience fluent method for arccos(). Symbol.arctan Convenience fluent method for arctan(). Symbol.degrees Convenience fluent method for degrees(). Symbol.radians Convenience fluent method for radians().

#### Hyperbolic functions¶

 Symbol.sinh Convenience fluent method for sinh(). Symbol.cosh Convenience fluent method for cosh(). Symbol.tanh Convenience fluent method for tanh(). Symbol.arcsinh Convenience fluent method for arcsinh(). Symbol.arccosh Convenience fluent method for arccosh(). Symbol.arctanh Convenience fluent method for arctanh().

#### Exponents and logarithms¶

 Symbol.exp Convenience fluent method for exp(). Symbol.expm1 Convenience fluent method for expm1(). Symbol.log Convenience fluent method for log(). Symbol.log10 Convenience fluent method for log10(). Symbol.log2 Convenience fluent method for log2(). Symbol.log1p Convenience fluent method for log1p().

#### Powers¶

 Symbol.sqrt Convenience fluent method for sqrt(). Symbol.rsqrt Convenience fluent method for rsqrt(). Symbol.cbrt Convenience fluent method for cbrt(). Symbol.rcbrt Convenience fluent method for rcbrt(). Symbol.square Convenience fluent method for square().

## Basic neural network functions¶

 Symbol.relu Convenience fluent method for relu(). Symbol.sigmoid Convenience fluent method for sigmoid(). Symbol.softmax Convenience fluent method for softmax(). Symbol.log_softmax Convenience fluent method for log_softmax().

### Comparison operators¶

 Symbol.__lt__ x.__lt__(y) <=> x Symbol.__le__ x.__le__(y) <=> x<=y Symbol.__gt__ x.__gt__(y) <=> x>y Symbol.__ge__ x.__ge__(y) <=> x>=y Symbol.__eq__ x.__eq__(y) <=> x==y Symbol.__ne__ x.__ne__(y) <=> x!=y

### Symbol creation¶

 Symbol.zeros_like Convenience fluent method for zeros_like(). Symbol.ones_like Convenience fluent method for ones_like().

### Changing shape and type¶

 Symbol.astype Convenience fluent method for cast(). Symbol.reshape Convenience fluent method for reshape(). Symbol.reshape_like Convenience fluent method for reshape_like(). Symbol.flatten Convenience fluent method for flatten(). Symbol.expand_dims Convenience fluent method for expand_dims().

### Expanding elements¶

 Symbol.broadcast_to Convenience fluent method for broadcast_to(). Symbol.broadcast_axes Convenience fluent method for broadcast_axes(). Symbol.tile Convenience fluent method for tile(). Symbol.pad Convenience fluent method for pad().

### Rearranging elements¶

 Symbol.transpose Convenience fluent method for transpose(). Symbol.swapaxes Convenience fluent method for swapaxes(). Symbol.flip Convenience fluent method for flip().

### Reduce functions¶

 Symbol.sum Convenience fluent method for sum(). Symbol.nansum Convenience fluent method for nansum(). Symbol.prod Convenience fluent method for prod(). Symbol.nanprod Convenience fluent method for nanprod(). Symbol.mean Convenience fluent method for mean(). Symbol.max Convenience fluent method for max(). Symbol.min Convenience fluent method for min(). Symbol.norm Convenience fluent method for norm().

### Rounding¶

 Symbol.round Convenience fluent method for round(). Symbol.rint Convenience fluent method for rint(). Symbol.fix Convenience fluent method for fix(). Symbol.floor Convenience fluent method for floor(). Symbol.ceil Convenience fluent method for ceil(). Symbol.trunc Convenience fluent method for trunc().

### Sorting and searching¶

 Symbol.sort Convenience fluent method for sort(). Symbol.argsort Convenience fluent method for argsort(). Symbol.topk Convenience fluent method for topk(). Symbol.argmax Convenience fluent method for argmax(). Symbol.argmin Convenience fluent method for argmin(). Symbol.argmax_channel Convenience fluent method for argmax_channel().

### Query information¶

 Symbol.name Gets name string from the symbol, this function only works for non-grouped symbol. Symbol.list_arguments Lists all the arguments in the symbol. Symbol.list_outputs Lists all the outputs in the symbol. Symbol.list_auxiliary_states Lists all the auxiliary states in the symbol. Symbol.list_attr Gets all attributes from the symbol. Symbol.attr Returns the attribute string for corresponding input key from the symbol. Symbol.attr_dict Recursively gets all attributes from the symbol and its children.

### Indexing¶

 Symbol.slice Convenience fluent method for slice(). Symbol.slice_axis Convenience fluent method for slice_axis(). Symbol.take Convenience fluent method for take(). Symbol.one_hot Convenience fluent method for one_hot(). Symbol.pick Convenience fluent method for pick().

### Get internal and output symbol¶

 Symbol.__getitem__ x.__getitem__(i) <=> x[i] Symbol.__iter__ Returns a generator object of symbol. Symbol.get_internals Gets a new grouped symbol sgroup. Symbol.get_children Gets a new grouped symbol whose output contains inputs to output nodes of the original symbol.

### Inference type and shape¶

 Symbol.infer_type Infers the type of all arguments and all outputs, given the known types for some arguments. Symbol.infer_shape Infers the shapes of all arguments and all outputs given the known shapes of some arguments. Symbol.infer_shape_partial Infers the shape partially.

### Bind¶

 Symbol.bind Binds the current symbol to an executor and returns it. Symbol.simple_bind Bind current symbol to get an executor, allocate all the arguments needed.

### Save¶

 Symbol.save Saves symbol to a file. Symbol.tojson Saves symbol to a JSON string. Symbol.debug_str Gets a debug string of symbol.

### Miscellaneous¶

 Symbol.clip Convenience fluent method for clip(). Symbol.sign Convenience fluent method for sign().

## Symbol creation routines¶

 var Creates a symbolic variable with specified name. zeros Returns a new symbol of given shape and type, filled with zeros. zeros_like Return an array of zeros with the same shape and type as the input array. ones Returns a new symbol of given shape and type, filled with ones. ones_like Return an array of ones with the same shape and type as the input array. arange Returns evenly spaced values within a given interval.

## Symbol manipulation routines¶

### Changing shape and type¶

 cast Casts all elements of the input to a new type. reshape Reshapes the input array. reshape_like Reshape lhs to have the same shape as rhs. flatten Flattens the input array into a 2-D array by collapsing the higher dimensions. expand_dims Inserts a new axis of size 1 into the array shape

### Expanding elements¶

 broadcast_to Broadcasts the input array to a new shape. broadcast_axes Broadcasts the input array over particular axes. repeat Repeats elements of an array. tile Repeats the whole array multiple times. pad Pads an input array with a constant or edge values of the array.

### Rearranging elements¶

 transpose Permutes the dimensions of an array. swapaxes Interchanges two axes of an array. flip Reverses the order of elements along given axis while preserving array shape.

### Joining and splitting symbols¶

 concat Joins input arrays along a given axis. split Splits an array along a particular axis into multiple sub-arrays. stack Join a sequence of arrays along a new axis.

### Indexing routines¶

 slice Slices a region of the array. slice_axis Slices along a given axis. take Takes elements from an input array along the given axis. batch_take Takes elements from a data batch. one_hot Returns a one-hot array. pick Picks elements from an input array according to the input indices along the given axis. where Given three ndarrays, condition, x, and y, return an ndarray with the elements from x or y, depending on the elements from condition are true or false. gather_nd Gather elements or slices from data and store to a tensor whose shape is defined by indices. scatter_nd Scatters data into a new tensor according to indices.

## Mathematical functions¶

### Arithmetic operations¶

 broadcast_add Returns element-wise sum of the input arrays with broadcasting. broadcast_sub Returns element-wise difference of the input arrays with broadcasting. broadcast_mul Returns element-wise product of the input arrays with broadcasting. broadcast_div Returns element-wise division of the input arrays with broadcasting. broadcast_mod Returns element-wise modulo of the input arrays with broadcasting. negative Numerical negative of the argument, element-wise. dot Dot product of two arrays. batch_dot Batchwise dot product. add_n Adds all input arguments element-wise.

### Trigonometric functions¶

 sin Computes the element-wise sine of the input array. cos Computes the element-wise cosine of the input array. tan Computes the element-wise tangent of the input array. arcsin Returns element-wise inverse sine of the input array. arccos Returns element-wise inverse cosine of the input array. arctan Returns element-wise inverse tangent of the input array. hypot Given the “legs” of a right triangle, returns its hypotenuse. broadcast_hypot Returns the hypotenuse of a right angled triangle, given its “legs” with broadcasting. degrees Converts each element of the input array from radians to degrees. radians Converts each element of the input array from degrees to radians.

### Hyperbolic functions¶

 sinh Returns the hyperbolic sine of the input array, computed element-wise. cosh Returns the hyperbolic cosine of the input array, computed element-wise. tanh Returns the hyperbolic tangent of the input array, computed element-wise. arcsinh Returns the element-wise inverse hyperbolic sine of the input array, computed element-wise. arccosh Returns the element-wise inverse hyperbolic cosine of the input array, computed element-wise. arctanh Returns the element-wise inverse hyperbolic tangent of the input array, computed element-wise.

### Reduce functions¶

 sum Computes the sum of array elements over given axes. nansum Computes the sum of array elements over given axes treating Not a Numbers (NaN) as zero. prod Computes the product of array elements over given axes. nanprod Computes the product of array elements over given axes treating Not a Numbers (NaN) as one. mean Computes the mean of array elements over given axes. max Computes the max of array elements over given axes. min Computes the min of array elements over given axes. norm Flattens the input array and then computes the l2 norm.

### Rounding¶

 round Returns element-wise rounded value to the nearest integer of the input. rint Returns element-wise rounded value to the nearest integer of the input. fix Returns element-wise rounded value to the nearest integer towards zero of the input. floor Returns element-wise floor of the input. ceil Returns element-wise ceiling of the input. trunc Return the element-wise truncated value of the input.

### Exponents and logarithms¶

 exp Returns element-wise exponential value of the input. expm1 Returns exp(x) - 1 computed element-wise on the input. log Returns element-wise Natural logarithmic value of the input. log10 Returns element-wise Base-10 logarithmic value of the input. log2 Returns element-wise Base-2 logarithmic value of the input. log1p Returns element-wise log(1 + x) value of the input.

### Powers¶

 broadcast_power Returns result of first array elements raised to powers from second array, element-wise with broadcasting. sqrt Returns element-wise square-root value of the input. rsqrt Returns element-wise inverse square-root value of the input. cbrt Returns element-wise cube-root value of the input. rcbrt Returns element-wise inverse cube-root value of the input. square Returns element-wise squared value of the input. reciprocal Returns the reciprocal of the argument, element-wise.

### Comparison¶

 broadcast_equal Returns the result of element-wise equal to (==) comparison operation with broadcasting. broadcast_not_equal Returns the result of element-wise not equal to (!=) comparison operation with broadcasting. broadcast_greater Returns the result of element-wise greater than (>) comparison operation with broadcasting. broadcast_greater_equal Returns the result of element-wise greater than or equal to (>=) comparison operation with broadcasting. broadcast_lesser Returns the result of element-wise lesser than (<) comparison operation with broadcasting. broadcast_lesser_equal Returns the result of element-wise lesser than or equal to (<=) comparison operation with broadcasting.

### Random sampling¶

 mxnet.symbol.random.uniform Draw random samples from a uniform distribution. mxnet.symbol.random.normal Draw random samples from a normal (Gaussian) distribution. mxnet.symbol.random.gamma Draw random samples from a gamma distribution. mxnet.symbol.random.exponential Draw samples from an exponential distribution. mxnet.symbol.random.poisson Draw random samples from a Poisson distribution. mxnet.symbol.random.negative_binomial Draw random samples from a negative binomial distribution. mxnet.symbol.random.generalized_negative_binomial Draw random samples from a generalized negative binomial distribution. mxnet.random.seed Seeds the random number generators in MXNet.

### Sorting and searching¶

 sort Returns a sorted copy of an input array along the given axis. topk Returns the top k elements in an input array along the given axis. argsort Returns the indices that would sort an input array along the given axis. argmax Returns indices of the maximum values along an axis. argmin Returns indices of the minimum values along an axis.

### Sequence operation¶

 SequenceLast Takes the last element of a sequence. SequenceMask Sets all elements outside the sequence to a constant value. SequenceReverse Reverses the elements of each sequence.

### Miscellaneous¶

 maximum Returns element-wise maximum of the input elements. minimum Returns element-wise minimum of the input elements. broadcast_maximum Returns element-wise maximum of the input arrays with broadcasting. broadcast_minimum Returns element-wise minimum of the input arrays with broadcasting. clip Clips (limits) the values in an array. abs Returns element-wise absolute value of the input. sign Returns element-wise sign of the input. gamma Returns the gamma function (extension of the factorial function to the reals), computed element-wise on the input array. gammaln Returns element-wise log of the absolute value of the gamma function of the input.

## Neural network¶

### Basic¶

 FullyConnected Applies a linear transformation: $$Y = XW^T + b$$. Convolution Compute N-D convolution on (N+2)-D input. Activation Applies an activation function element-wise to the input. BatchNorm Batch normalization. Pooling Performs pooling on the input. SoftmaxOutput Computes the gradient of cross entropy loss with respect to softmax output. softmax Applies the softmax function. log_softmax Computes the log softmax of the input. relu Computes rectified linear. sigmoid Computes sigmoid of x element-wise.

### More¶

 Correlation Applies correlation to inputs. Deconvolution Computes 1D or 2D transposed convolution (aka fractionally strided convolution) of the input tensor. RNN Applies a recurrent layer to input. Embedding Maps integer indices to vector representations (embeddings). LeakyReLU Applies Leaky rectified linear unit activation element-wise to the input. InstanceNorm Applies instance normalization to the n-dimensional input array. L2Normalization Normalize the input array using the L2 norm. LRN Applies local response normalization to the input. ROIPooling Performs region of interest(ROI) pooling on the input array. SoftmaxActivation Applies softmax activation to input. Dropout Applies dropout operation to input array. BilinearSampler Applies bilinear sampling to input feature map. GridGenerator Generates 2D sampling grid for bilinear sampling. UpSampling Performs nearest neighbor/bilinear up sampling to inputs. SpatialTransformer Applies a spatial transformer to input feature map. LinearRegressionOutput Computes and optimizes for squared loss during backward propagation. LogisticRegressionOutput Applies a logistic function to the input. MAERegressionOutput Computes mean absolute error of the input. SVMOutput Computes support vector machine based transformation of the input. softmax_cross_entropy Calculate cross entropy of softmax output and one-hot label. smooth_l1 Calculate Smooth L1 Loss(lhs, scalar) by summing IdentityAttachKLSparseReg Apply a sparse regularization to the output a sigmoid activation function. MakeLoss Make your own loss function in network construction. BlockGrad Stops gradient computation. Custom Apply a custom operator implemented in a frontend language (like Python).

## API Reference¶

class mxnet.symbol.Symbol(handle)[source]

Symbol is symbolic graph of the mxnet.

__repr__()[source]

Gets a string representation of the symbol.

__iter__()[source]

Returns a generator object of symbol.

One can loop through the returned object list to get outputs.

Example

>>> a = mx.sym.Variable('a')
>>> b = mx.sym.Variable('b')
>>> c = a+b
>>> d = mx.sym.Variable('d')
>>> e = d+c
>>> out = e.get_children()
>>> out

>>> for i in out:
...     i
...


__add__(other)[source]

__sub__(other)[source]

x.__sub__(y) <=> x-y

__rsub__(other)[source]

x.__rsub__(y) <=> y-x

Only NDArray is supported for now.

Example

>>> x = mx.nd.ones((2,3))*3
>>> y = mx.nd.ones((2,3))
>>> x.__rsub__(y).asnumpy()
array([[-2., -2., -2.],
[-2., -2., -2.]], dtype=float32)

__mul__(other)[source]

x.__mul__(y) <=> x*y

__div__(other)[source]

x.__div__(y) <=> x/y

__rdiv__(other)[source]

x.__rdiv__(y) <=> y/x

Only NDArray is supported for now.

Example

>>> x = mx.nd.ones((2,3))*3
>>> y = mx.nd.ones((2,3))
>>> x.__rdiv__(y).asnumpy()
array([[ 0.33333334,  0.33333334,  0.33333334],
[ 0.33333334,  0.33333334,  0.33333334]], dtype=float32)

__mod__(other)[source]

x.__mod__(y) <=> x%y

__rmod__(other)[source]

x.__rmod__(y) <=> y%x

Only NDArray is supported for now.

Example

>>> x = mx.nd.ones((2,3))*3
>>> y = mx.nd.ones((2,3))
>>> x.__rmod__(y).asnumpy()
array([[ 1.,  1.,  1.,
[ 1.,  1.,  1., dtype=float32)

__pow__(other)[source]

x.__pow__(y) <=> x**y

__neg__()[source]

x.__neg__() <=> -x

Numerical negative, element-wise.

Example

>>> a = mx.sym.Variable('a')
>>> a

>>> -a

>>> a_neg = a.__neg__()
>>> c = a_neg*b
>>> ex = c.eval(ctx=mx.cpu(), a=mx.nd.ones([2,3]), b=mx.nd.ones([2,3]))
>>> ex[0].asnumpy()
array([[-1., -1., -1.],
[-1., -1., -1.]], dtype=float32)

__deepcopy__(_)[source]

Returns a deep copy of the input object.

This function returns a deep copy of the input object including the current state of all its parameters such as weights, biases, etc.

Any changes made to the deep copy do not reflect in the original object.

Example

>>> import copy
>>> data = mx.sym.Variable('data')
>>> data_1 = copy.deepcopy(data)
>>> data_1 = 2*data
>>> data_1.tojson()
>>> data_1 is data    # Data got modified
False

__eq__(other)[source]

x.__eq__(y) <=> x==y

__ne__(other)[source]

x.__ne__(y) <=> x!=y

__gt__(other)[source]

x.__gt__(y) <=> x>y

__ge__(other)[source]

x.__ge__(y) <=> x>=y

__lt__(other)[source]

x.__lt__(y) <=> x

__le__(other)[source]

x.__le__(y) <=> x<=y

__call__(*args, **kwargs)[source]

Composes symbol using inputs.

x.__call__(y, z) <=> x(y,z)

This function internally calls _compose to compose the symbol and returns the composed symbol.

Example

>>> data = mx.symbol.Variable('data')
>>> net1 = mx.symbol.FullyConnected(data=data, name='fc1', num_hidden=10)
>>> net2 = mx.symbol.FullyConnected(name='fc3', num_hidden=10)
>>> composed = net2(fc3_data=net1, name='composed')
>>> composed

>>> called = net2.__call__(fc3_data=net1, name='composed')
>>> called


Parameters: args – Positional arguments. kwargs – Keyword arguments. The resulting symbol.
__getitem__(index)[source]

x.__getitem__(i) <=> x[i]

Returns a sliced view of the input symbol.

Example

>>> a = mx.sym.var('a')
>>> a.__getitem__(0)

>>> a[0]


Parameters: index (int or str) – Indexing key
name

Gets name string from the symbol, this function only works for non-grouped symbol.

Returns: value – The name of this symbol, returns None for grouped symbol. str
attr(key)[source]

Returns the attribute string for corresponding input key from the symbol.

This function only works for non-grouped symbols.

Example

>>> data = mx.sym.Variable('data', attr={'mood': 'angry'})
>>> data.attr('mood')
'angry'

Parameters: key (str) – The key corresponding to the desired attribute. value – The desired attribute value, returns None if the attribute does not exist. str
list_attr(recursive=False)[source]

Gets all attributes from the symbol.

Example

>>> data = mx.sym.Variable('data', attr={'mood': 'angry'})
>>> data.list_attr()
{'mood': 'angry'}

Returns: ret – A dictionary mapping attribute keys to values. Dict of str to str
attr_dict()[source]

Recursively gets all attributes from the symbol and its children.

Example

>>> a = mx.sym.Variable('a', attr={'a1':'a2'})
>>> b = mx.sym.Variable('b', attr={'b1':'b2'})
>>> c = a+b
>>> c.attr_dict()
{'a': {'a1': 'a2'}, 'b': {'b1': 'b2'}}

Returns: ret – There is a key in the returned dict for every child with non-empty attribute set. For each symbol, the name of the symbol is its key in the dict and the correspond value is that symbol’s attribute list (itself a dictionary). Dict of str to dict
get_internals()[source]

Gets a new grouped symbol sgroup. The output of sgroup is a list of outputs of all of the internal nodes.

Consider the following code:

Example

>>> a = mx.sym.var('a')
>>> b = mx.sym.var('b')
>>> c = a + b
>>> d = c.get_internals()
>>> d

>>> d.list_outputs()
['a', 'b', '_plus4_output']

Returns: sgroup – A symbol group containing all internal and leaf nodes of the computation graph used to compute the symbol. Symbol
get_children()[source]

Gets a new grouped symbol whose output contains inputs to output nodes of the original symbol.

Example

>>> x = mx.sym.Variable('x')
>>> y = mx.sym.Variable('y')
>>> z = mx.sym.Variable('z')
>>> a = y+z
>>> b = x+a
>>> b.get_children()

>>> b.get_children().list_outputs()
['x', '_plus10_output']
>>> b.get_children().get_children().list_outputs()
['y', 'z']

Returns: sgroup – The children of the head node. If the symbol has no inputs then None will be returned. Symbol or None
list_arguments()[source]

Lists all the arguments in the symbol.

Example

>>> a = mx.sym.var('a')
>>> b = mx.sym.var('b')
>>> c = a + b
>>> c.list_arguments
['a', 'b']

Returns: args – List containing the names of all the arguments required to compute the symbol. list of string
list_outputs()[source]

Lists all the outputs in the symbol.

Example

>>> a = mx.sym.var('a')
>>> b = mx.sym.var('b')
>>> c = a + b
>>> c.list_outputs()
['_plus12_output']

Returns: List of all the outputs. For most symbols, this list contains only the name of this symbol. For symbol groups, this is a list with the names of all symbols in the group. list of str
__len__()[source]

Get number of outputs for the symbol.

Example

>>> a = mx.sym.var('a')
>>> b = mx.sym.var('b')
>>> c = a + b
>>> len(c)

Returns: len(self) – Number of outputs Number of outputs
list_auxiliary_states()[source]

Lists all the auxiliary states in the symbol.

Example

>>> a = mx.sym.var('a')
>>> b = mx.sym.var('b')
>>> c = a + b
>>> c.list_auxiliary_states()
[]


Example of auxiliary states in BatchNorm.

>>> data = mx.symbol.Variable('data')
>>> weight = mx.sym.Variable(name='fc1_weight')
>>> fc1  = mx.symbol.FullyConnected(data = data, weight=weight, name='fc1', num_hidden=128)
>>> fc2 = mx.symbol.BatchNorm(fc1, name='batchnorm0')
>>> fc2.list_auxiliary_states()
['batchnorm0_moving_mean', 'batchnorm0_moving_var']

Returns: aux_states – List of the auxiliary states in input symbol. list of str

Notes

Auxiliary states are special states of symbols that do not correspond to an argument, and are not updated by gradient descent. Common examples of auxiliary states include the moving_mean and moving_variance in BatchNorm. Most operators do not have auxiliary states.

list_inputs()[source]

Lists all arguments and auxiliary states of this Symbol.

Returns: inputs – List of all inputs. list of str

Examples

>>> bn = mx.sym.BatchNorm(name='bn')
>>> bn.list_arguments()
['bn_data', 'bn_gamma', 'bn_beta']
>>> bn.list_auxiliary_states()
['bn_moving_mean', 'bn_moving_var']
>>> bn.list_inputs()
['bn_data', 'bn_gamma', 'bn_beta', 'bn_moving_mean', 'bn_moving_var']

infer_type(*args, **kwargs)[source]

Infers the type of all arguments and all outputs, given the known types for some arguments.

This function takes the known types of some arguments in either positional way or keyword argument way as input. It returns a tuple of None values if there is not enough information to deduce the missing types.

Inconsistencies in the known types will cause an error to be raised.

Example

>>> a = mx.sym.var('a')
>>> b = mx.sym.var('b')
>>> c = a + b
>>> arg_types, out_types, aux_types = c.infer_type(a='float32')
>>> arg_types
[, ]
>>> out_types
[]
>>> aux_types
[]

Parameters: *args – Type of known arguments in a positional way. Unknown type can be marked as None. **kwargs – Keyword arguments of known types. arg_types (list of numpy.dtype or None) – List of argument types. The order is same as the order of list_arguments(). out_types (list of numpy.dtype or None) – List of output types. The order is same as the order of list_outputs(). aux_types (list of numpy.dtype or None) – List of auxiliary state types. The order is same as the order of list_auxiliary_states().
infer_shape(*args, **kwargs)[source]

Infers the shapes of all arguments and all outputs given the known shapes of some arguments.

This function takes the known shapes of some arguments in either positional way or keyword argument way as input. It returns a tuple of None values if there is not enough information to deduce the missing shapes.

Example

>>> a = mx.sym.var('a')
>>> b = mx.sym.var('b')
>>> c = a + b
>>> arg_shapes, out_shapes, aux_shapes = c.infer_shape(a=(3,3))
>>> arg_shapes
[(3L, 3L), (3L, 3L)]
>>> out_shapes
[(3L, 3L)]
>>> aux_shapes
[]
>>> c.infer_shape(a=(0,3)) # 0s in shape means unknown dimensions. So, returns None.
(None, None, None)


Inconsistencies in the known shapes will cause an error to be raised. See the following example:

>>> data = mx.sym.Variable('data')
>>> out = mx.sym.FullyConnected(data=data, name='fc1', num_hidden=1000)
>>> out = mx.sym.Activation(data=out, act_type='relu')
>>> out = mx.sym.FullyConnected(data=out, name='fc2', num_hidden=10)
>>> weight_shape= (1, 100)
>>> data_shape = (100, 100)
>>> out.infer_shape(data=data_shape, fc1_weight=weight_shape)
Error in operator fc1: Shape inconsistent, Provided=(1,100), inferred shape=(1000,100)

Parameters: *args – Shape of arguments in a positional way. Unknown shape can be marked as None. **kwargs – Keyword arguments of the known shapes. arg_shapes (list of tuple or None) – List of argument shapes. The order is same as the order of list_arguments(). out_shapes (list of tuple or None) – List of output shapes. The order is same as the order of list_outputs(). aux_shapes (list of tuple or None) – List of auxiliary state shapes. The order is same as the order of list_auxiliary_states().
infer_shape_partial(*args, **kwargs)[source]

Infers the shape partially.

This functions works the same way as infer_shape, except that this function can return partial results.

In the following example, information about fc2 is not available. So, infer_shape will return a tuple of None values but infer_shape_partial will return partial values.

Example

>>> data = mx.sym.Variable('data')
>>> prev = mx.sym.Variable('prev')
>>> fc1  = mx.sym.FullyConnected(data=data, name='fc1', num_hidden=128)
>>> fc2  = mx.sym.FullyConnected(data=prev, name='fc2', num_hidden=128)
>>> out  = mx.sym.Activation(data=mx.sym.elemwise_add(fc1, fc2), act_type='relu')
>>> out.list_arguments()
['data', 'fc1_weight', 'fc1_bias', 'prev', 'fc2_weight', 'fc2_bias']
>>> out.infer_shape(data=(10,64))
(None, None, None)
>>> out.infer_shape_partial(data=(10,64))
([(10L, 64L), (128L, 64L), (128L,), (), (), ()], [(10L, 128L)], [])
>>> # infers shape if you give information about fc2
>>> out.infer_shape(data=(10,64), prev=(10,128))
([(10L, 64L), (128L, 64L), (128L,), (10L, 128L), (128L, 128L), (128L,)], [(10L, 128L)], [])

Parameters: *args – Shape of arguments in a positional way. Unknown shape can be marked as None **kwargs – Keyword arguments of known shapes. arg_shapes (list of tuple or None) – List of argument shapes. The order is same as the order of list_arguments(). out_shapes (list of tuple or None) – List of output shapes. The order is same as the order of list_outputs(). aux_shapes (list of tuple or None) – List of auxiliary state shapes. The order is same as the order of list_auxiliary_states().
debug_str()[source]

Gets a debug string of symbol.

It contains Symbol output, variables and operators in the computation graph with their inputs, variables and attributes.

Returns: Debug string of the symbol. string

Examples

>>> a = mx.sym.Variable('a')
>>> b = mx.sym.sin(a)
>>> c = 2 * a + b
>>> d = mx.sym.FullyConnected(data=c, num_hidden=10)
>>> d.debug_str()
>>> print d.debug_str()
Symbol Outputs:
output[0]=fullyconnected0(0)
Variable:a
--------------------
Op:_mul_scalar, Name=_mulscalar0
Inputs:
arg[0]=a(0) version=0
Attrs:
scalar=2
--------------------
Op:sin, Name=sin0
Inputs:
arg[0]=a(0) version=0
--------------------
Inputs:
arg[0]=_mulscalar0(0)
arg[1]=sin0(0)
Variable:fullyconnected0_weight
Variable:fullyconnected0_bias
--------------------
Op:FullyConnected, Name=fullyconnected0
Inputs:
arg[0]=_plus0(0)
arg[1]=fullyconnected0_weight(0) version=0
arg[2]=fullyconnected0_bias(0) version=0
Attrs:
num_hidden=10

save(fname)[source]

Saves symbol to a file.

You can also use pickle to do the job if you only work on python. The advantage of load/save functions is that the file contents are language agnostic. This means the model saved by one language binding can be loaded by a different language binding of MXNet. You also get the benefit of being able to directly load/save from cloud storage(S3, HDFS).

Parameters: fname (str) – The name of the file. “s3://my-bucket/path/my-s3-symbol” “hdfs://my-bucket/path/my-hdfs-symbol” “/path-to/my-local-symbol”

symbol.load()
Used to load symbol from file.
tojson()[source]

Saves symbol to a JSON string.

symbol.load_json()
Used to load symbol from JSON string.
simple_bind(ctx, grad_req='write', type_dict=None, stype_dict=None, group2ctx=None, shared_arg_names=None, shared_exec=None, shared_buffer=None, **kwargs)[source]

Bind current symbol to get an executor, allocate all the arguments needed. Allows specifying data types.

This function simplifies the binding procedure. You need to specify only input data shapes. Before binding the executor, the function allocates arguments and auxiliary states that were not explicitly specified. Allows specifying data types.

Example

>>> x = mx.sym.Variable('x')
>>> y = mx.sym.FullyConnected(x, num_hidden=4)
>>> exe = y.simple_bind(mx.cpu(), x=(5,4), grad_req='null')
>>> exe.forward()
[]
>>> exe.outputs[0].asnumpy()
array([[ 0.,  0.,  0.,  0.],
[ 0.,  0.,  0.,  0.],
[ 0.,  0.,  0.,  0.],
[ 0.,  0.,  0.,  0.],
[ 0.,  0.,  0.,  0.]], dtype=float32)
>>> exe.arg_arrays
[, , ]
[, , ]

Parameters: ctx (Context) – The device context the generated executor to run on. grad_req (string) – {‘write’, ‘add’, ‘null’}, or list of str or dict of str to str, optional To specify how we should update the gradient to the args_grad. ‘write’ means every time gradient is written to specified args_grad NDArray. ‘add’ means every time gradient is added to the specified NDArray. ‘null’ means no action is taken, the gradient may not be calculated. type_dict (Dict of str->numpy.dtype) – Input type dictionary, name->dtype stype_dict (Dict of str->str) – Input storage type dictionary, name->storage_type group2ctx (Dict of string to mx.Context) – The dict mapping the ctx_group attribute to the context assignment. shared_arg_names (List of string) – The argument names whose NDArray of shared_exec can be reused for initializing the current executor. shared_exec (Executor) – The executor whose arg_arrays, arg_arrays, grad_arrays, and aux_arrays can be reused for initializing the current executor. shared_buffer (Dict of string to NDArray) – The dict mapping argument names to the NDArray that can be reused for initializing the current executor. This buffer will be checked for reuse if one argument name of the current executor is not found in shared_arg_names. The NDArrays are expected have default storage type. kwargs (Dict of str->shape) – Input shape dictionary, name->shape executor – The generated executor mxnet.Executor
bind(ctx, args, args_grad=None, grad_req='write', aux_states=None, group2ctx=None, shared_exec=None)[source]

Binds the current symbol to an executor and returns it.

We first declare the computation and then bind to the data to run. This function returns an executor which provides method forward() method for evaluation and a outputs() method to get all the results.

Example

>>> a = mx.sym.Variable('a')
>>> b = mx.sym.Variable('b')
>>> c = a + b

>>> ex = c.bind(ctx=mx.cpu(), args={'a' : mx.nd.ones([2,3]), 'b' : mx.nd.ones([2,3])})
>>> ex.forward()
[]
>>> ex.outputs[0].asnumpy()
[[ 2.  2.  2.]
[ 2.  2.  2.]]


Notes

Auxiliary states are the special states of symbols that do not correspond to an argument, and do not have gradient but are still useful for the specific operations. Common examples of auxiliary states include the moving_mean and moving_variance states in BatchNorm. Most operators do not have auxiliary states and in those cases, this parameter can be safely ignored.

One can give up gradient by using a dict in args_grad and only specify gradient they interested in.

gradient(wrt)[source]

Gets the autodiff of current symbol.

This function can only be used if current symbol is a loss function.

Note

This function is currently not implemented.

Parameters: wrt (Array of String) – keyword arguments of the symbol that the gradients are taken. grad – A gradient Symbol with returns to be the corresponding gradients. Symbol
eval(ctx=None, **kwargs)[source]

Evaluates a symbol given arguments.

The eval method combines a call to bind (which returns an executor) with a call to forward (executor method). For the common use case, where you might repeatedly evaluate with same arguments, eval is slow. In that case, you should call bind once and then repeatedly call forward. This function allows simpler syntax for less cumbersome introspection.

Example

>>> a = mx.sym.Variable('a')
>>> b = mx.sym.Variable('b')
>>> c = a + b
>>> ex = c.eval(ctx = mx.cpu(), a = mx.nd.ones([2,3]), b = mx.nd.ones([2,3]))
>>> ex
[]
>>> ex[0].asnumpy()
array([[ 2.,  2.,  2.],
[ 2.,  2.,  2.]], dtype=float32)

Parameters: ctx (Context) – The device context the generated executor to run on. kwargs (Keyword arguments of type NDArray) – Input arguments to the symbol. All the arguments must be provided. result (a list of NDArrays corresponding to the values taken by each symbol when) evaluated on given args. When called on a single symbol (not a group), the result will be a list with one element.
reshape(*args, **kwargs)[source]

Convenience fluent method for reshape().

The arguments are the same as for reshape(), with this array as data.

reshape_like(*args, **kwargs)[source]

Convenience fluent method for reshape_like().

The arguments are the same as for reshape_like(), with this array as data.

astype(*args, **kwargs)[source]

Convenience fluent method for cast().

The arguments are the same as for cast(), with this array as data.

zeros_like(*args, **kwargs)[source]

Convenience fluent method for zeros_like().

The arguments are the same as for zeros_like(), with this array as data.

ones_like(*args, **kwargs)[source]

Convenience fluent method for ones_like().

The arguments are the same as for ones_like(), with this array as data.

broadcast_axes(*args, **kwargs)[source]

Convenience fluent method for broadcast_axes().

The arguments are the same as for broadcast_axes(), with this array as data.

repeat(*args, **kwargs)[source]

Convenience fluent method for repeat().

The arguments are the same as for repeat(), with this array as data.

pad(*args, **kwargs)[source]

Convenience fluent method for pad().

The arguments are the same as for pad(), with this array as data.

swapaxes(*args, **kwargs)[source]

Convenience fluent method for swapaxes().

The arguments are the same as for swapaxes(), with this array as data.

split(*args, **kwargs)[source]

Convenience fluent method for split().

The arguments are the same as for split(), with this array as data.

slice(*args, **kwargs)[source]

Convenience fluent method for slice().

The arguments are the same as for slice(), with this array as data.

slice_axis(*args, **kwargs)[source]

Convenience fluent method for slice_axis().

The arguments are the same as for slice_axis(), with this array as data.

take(*args, **kwargs)[source]

Convenience fluent method for take().

The arguments are the same as for take(), with this array as data.

one_hot(*args, **kwargs)[source]

Convenience fluent method for one_hot().

The arguments are the same as for one_hot(), with this array as data.

pick(*args, **kwargs)[source]

Convenience fluent method for pick().

The arguments are the same as for pick(), with this array as data.

sort(*args, **kwargs)[source]

Convenience fluent method for sort().

The arguments are the same as for sort(), with this array as data.

topk(*args, **kwargs)[source]

Convenience fluent method for topk().

The arguments are the same as for topk(), with this array as data.

argsort(*args, **kwargs)[source]

Convenience fluent method for argsort().

The arguments are the same as for argsort(), with this array as data.

argmax(*args, **kwargs)[source]

Convenience fluent method for argmax().

The arguments are the same as for argmax(), with this array as data.

argmax_channel(*args, **kwargs)[source]

Convenience fluent method for argmax_channel().

The arguments are the same as for argmax_channel(), with this array as data.

argmin(*args, **kwargs)[source]

Convenience fluent method for argmin().

The arguments are the same as for argmin(), with this array as data.

clip(*args, **kwargs)[source]

Convenience fluent method for clip().

The arguments are the same as for clip(), with this array as data.

abs(*args, **kwargs)[source]

Convenience fluent method for abs().

The arguments are the same as for abs(), with this array as data.

sign(*args, **kwargs)[source]

Convenience fluent method for sign().

The arguments are the same as for sign(), with this array as data.

flatten(*args, **kwargs)[source]

Convenience fluent method for flatten().

The arguments are the same as for flatten(), with this array as data.

expand_dims(*args, **kwargs)[source]

Convenience fluent method for expand_dims().

The arguments are the same as for expand_dims(), with this array as data.

broadcast_to(*args, **kwargs)[source]

Convenience fluent method for broadcast_to().

The arguments are the same as for broadcast_to(), with this array as data.

tile(*args, **kwargs)[source]

Convenience fluent method for tile().

The arguments are the same as for tile(), with this array as data.

transpose(*args, **kwargs)[source]

Convenience fluent method for transpose().

The arguments are the same as for transpose(), with this array as data.

flip(*args, **kwargs)[source]

Convenience fluent method for flip().

The arguments are the same as for flip(), with this array as data.

sum(*args, **kwargs)[source]

Convenience fluent method for sum().

The arguments are the same as for sum(), with this array as data.

nansum(*args, **kwargs)[source]

Convenience fluent method for nansum().

The arguments are the same as for nansum(), with this array as data.

prod(*args, **kwargs)[source]

Convenience fluent method for prod().

The arguments are the same as for prod(), with this array as data.

nanprod(*args, **kwargs)[source]

Convenience fluent method for nanprod().

The arguments are the same as for nanprod(), with this array as data.

mean(*args, **kwargs)[source]

Convenience fluent method for mean().

The arguments are the same as for mean(), with this array as data.

max(*args, **kwargs)[source]

Convenience fluent method for max().

The arguments are the same as for max(), with this array as data.

min(*args, **kwargs)[source]

Convenience fluent method for min().

The arguments are the same as for min(), with this array as data.

norm(*args, **kwargs)[source]

Convenience fluent method for norm().

The arguments are the same as for norm(), with this array as data.

round(*args, **kwargs)[source]

Convenience fluent method for round().

The arguments are the same as for round(), with this array as data.

rint(*args, **kwargs)[source]

Convenience fluent method for rint().

The arguments are the same as for rint(), with this array as data.

fix(*args, **kwargs)[source]

Convenience fluent method for fix().

The arguments are the same as for fix(), with this array as data.

floor(*args, **kwargs)[source]

Convenience fluent method for floor().

The arguments are the same as for floor(), with this array as data.

ceil(*args, **kwargs)[source]

Convenience fluent method for ceil().

The arguments are the same as for ceil(), with this array as data.

trunc(*args, **kwargs)[source]

Convenience fluent method for trunc().

The arguments are the same as for trunc(), with this array as data.

sin(*args, **kwargs)[source]

Convenience fluent method for sin().

The arguments are the same as for sin(), with this array as data.

cos(*args, **kwargs)[source]

Convenience fluent method for cos().

The arguments are the same as for cos(), with this array as data.

tan(*args, **kwargs)[source]

Convenience fluent method for tan().

The arguments are the same as for tan(), with this array as data.

arcsin(*args, **kwargs)[source]

Convenience fluent method for arcsin().

The arguments are the same as for arcsin(), with this array as data.

arccos(*args, **kwargs)[source]

Convenience fluent method for arccos().

The arguments are the same as for arccos(), with this array as data.

arctan(*args, **kwargs)[source]

Convenience fluent method for arctan().

The arguments are the same as for arctan(), with this array as data.

degrees(*args, **kwargs)[source]

Convenience fluent method for degrees().

The arguments are the same as for degrees(), with this array as data.

radians(*args, **kwargs)[source]

Convenience fluent method for radians().

The arguments are the same as for radians(), with this array as data.

sinh(*args, **kwargs)[source]

Convenience fluent method for sinh().

The arguments are the same as for sinh(), with this array as data.

cosh(*args, **kwargs)[source]

Convenience fluent method for cosh().

The arguments are the same as for cosh(), with this array as data.

tanh(*args, **kwargs)[source]

Convenience fluent method for tanh().

The arguments are the same as for tanh(), with this array as data.

arcsinh(*args, **kwargs)[source]

Convenience fluent method for arcsinh().

The arguments are the same as for arcsinh(), with this array as data.

arccosh(*args, **kwargs)[source]

Convenience fluent method for arccosh().

The arguments are the same as for arccosh(), with this array as data.

arctanh(*args, **kwargs)[source]

Convenience fluent method for arctanh().

The arguments are the same as for arctanh(), with this array as data.

exp(*args, **kwargs)[source]

Convenience fluent method for exp().

The arguments are the same as for exp(), with this array as data.

expm1(*args, **kwargs)[source]

Convenience fluent method for expm1().

The arguments are the same as for expm1(), with this array as data.

log(*args, **kwargs)[source]

Convenience fluent method for log().

The arguments are the same as for log(), with this array as data.

log10(*args, **kwargs)[source]

Convenience fluent method for log10().

The arguments are the same as for log10(), with this array as data.

log2(*args, **kwargs)[source]

Convenience fluent method for log2().

The arguments are the same as for log2(), with this array as data.

log1p(*args, **kwargs)[source]

Convenience fluent method for log1p().

The arguments are the same as for log1p(), with this array as data.

sqrt(*args, **kwargs)[source]

Convenience fluent method for sqrt().

The arguments are the same as for sqrt(), with this array as data.

rsqrt(*args, **kwargs)[source]

Convenience fluent method for rsqrt().

The arguments are the same as for rsqrt(), with this array as data.

cbrt(*args, **kwargs)[source]

Convenience fluent method for cbrt().

The arguments are the same as for cbrt(), with this array as data.

rcbrt(*args, **kwargs)[source]

Convenience fluent method for rcbrt().

The arguments are the same as for rcbrt(), with this array as data.

square(*args, **kwargs)[source]

Convenience fluent method for square().

The arguments are the same as for square(), with this array as data.

reciprocal(*args, **kwargs)[source]

Convenience fluent method for reciprocal().

The arguments are the same as for reciprocal(), with this array as data.

relu(*args, **kwargs)[source]

Convenience fluent method for relu().

The arguments are the same as for relu(), with this array as data.

sigmoid(*args, **kwargs)[source]

Convenience fluent method for sigmoid().

The arguments are the same as for sigmoid(), with this array as data.

softmax(*args, **kwargs)[source]

Convenience fluent method for softmax().

The arguments are the same as for softmax(), with this array as data.

log_softmax(*args, **kwargs)[source]

Convenience fluent method for log_softmax().

The arguments are the same as for log_softmax(), with this array as data.

Symbol API of MXNet.

mxnet.symbol.Activation(data=None, act_type=_Null, name=None, attr=None, out=None, **kwargs)

Applies an activation function element-wise to the input.

The following activation functions are supported:

• relu: Rectified Linear Unit, $$y = max(x, 0)$$
• sigmoid: $$y = \frac{1}{1 + exp(-x)}$$
• tanh: Hyperbolic tangent, $$y = \frac{exp(x) - exp(-x)}{exp(x) + exp(-x)}$$
• softrelu: Soft ReLU, or SoftPlus, $$y = log(1 + exp(x))$$

Defined in src/operator/nn/activation.cc:L92

Parameters: data (Symbol) – Input array to activation function. act_type ({'relu', 'sigmoid', 'softrelu', 'tanh'}, required) – Activation function to be applied. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol

Examples

A one-hidden-layer MLP with ReLU activation:

>>> data = Variable('data')
>>> mlp = FullyConnected(data=data, num_hidden=128, name='proj')
>>> mlp = Activation(data=mlp, act_type='relu', name='activation')
>>> mlp = FullyConnected(data=mlp, num_hidden=10, name='mlp')
>>> mlp



ReLU activation

>>> test_suites = [
... ('relu', lambda x: np.maximum(x, 0)),
... ('sigmoid', lambda x: 1 / (1 + np.exp(-x))),
... ('tanh', lambda x: np.tanh(x)),
... ('softrelu', lambda x: np.log(1 + np.exp(x)))
... ]
>>> x = test_utils.random_arrays((2, 3, 4))
>>> for act_type, numpy_impl in test_suites:
... op = Activation(act_type=act_type, name='act')
... y = test_utils.simple_forward(op, act_data=x)
... y_np = numpy_impl(x)
... print('%s: %s' % (act_type, test_utils.almost_equal(y, y_np)))
relu: True
sigmoid: True
tanh: True
softrelu: True

mxnet.symbol.BatchNorm(data=None, gamma=None, beta=None, moving_mean=None, moving_var=None, eps=_Null, momentum=_Null, fix_gamma=_Null, use_global_stats=_Null, output_mean_var=_Null, axis=_Null, cudnn_off=_Null, name=None, attr=None, out=None, **kwargs)

Batch normalization.

Normalizes a data batch by mean and variance, and applies a scale gamma as well as offset beta.

Assume the input has more than one dimension and we normalize along axis 1. We first compute the mean and variance along this axis:

$\begin{split}data\_mean[i] = mean(data[:,i,:,...]) \\ data\_var[i] = var(data[:,i,:,...])\end{split}$

Then compute the normalized output, which has the same shape as input, as following:

$out[:,i,:,...] = \frac{data[:,i,:,...] - data\_mean[i]}{\sqrt{data\_var[i]+\epsilon}} * gamma[i] + beta[i]$

Both mean and var returns a scalar by treating the input as a vector.

Assume the input has size k on axis 1, then both gamma and beta have shape (k,). If output_mean_var is set to be true, then outputs both data_mean and data_var as well, which are needed for the backward pass.

Besides the inputs and the outputs, this operator accepts two auxiliary states, moving_mean and moving_var, which are k-length vectors. They are global statistics for the whole dataset, which are updated by:

moving_mean = moving_mean * momentum + data_mean * (1 - momentum)
moving_var = moving_var * momentum + data_var * (1 - momentum)


If use_global_stats is set to be true, then moving_mean and moving_var are used instead of data_mean and data_var to compute the output. It is often used during inference.

The parameter axis specifies which axis of the input shape denotes the ‘channel’ (separately normalized groups). The default is 1. Specifying -1 sets the channel axis to be the last item in the input shape.

Both gamma and beta are learnable parameters. But if fix_gamma is true, then set gamma to 1 and its gradient to 0.

Defined in src/operator/nn/batch_norm.cc:L400

Parameters: data (Symbol) – Input data to batch normalization gamma (Symbol) – gamma array beta (Symbol) – beta array moving_mean (Symbol) – running mean of input moving_var (Symbol) – running variance of input eps (double, optional, default=0.001) – Epsilon to prevent div 0. Must be no less than CUDNN_BN_MIN_EPSILON defined in cudnn.h when using cudnn (usually 1e-5) momentum (float, optional, default=0.9) – Momentum for moving average fix_gamma (boolean, optional, default=1) – Fix gamma while training use_global_stats (boolean, optional, default=0) – Whether use global moving statistics instead of local batch-norm. This will force change batch-norm into a scale shift operator. output_mean_var (boolean, optional, default=0) – Output All,normal mean and var axis (int, optional, default='1') – Specify which shape axis the channel is specified cudnn_off (boolean, optional, default=0) – Do not select CUDNN operator, if available name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.BatchNorm_v1(data=None, gamma=None, beta=None, eps=_Null, momentum=_Null, fix_gamma=_Null, use_global_stats=_Null, output_mean_var=_Null, name=None, attr=None, out=None, **kwargs)

Batch normalization.

Normalizes a data batch by mean and variance, and applies a scale gamma as well as offset beta.

Assume the input has more than one dimension and we normalize along axis 1. We first compute the mean and variance along this axis:

$\begin{split}data\_mean[i] = mean(data[:,i,:,...]) \\ data\_var[i] = var(data[:,i,:,...])\end{split}$

Then compute the normalized output, which has the same shape as input, as following:

$out[:,i,:,...] = \frac{data[:,i,:,...] - data\_mean[i]}{\sqrt{data\_var[i]+\epsilon}} * gamma[i] + beta[i]$

Both mean and var returns a scalar by treating the input as a vector.

Assume the input has size k on axis 1, then both gamma and beta have shape (k,). If output_mean_var is set to be true, then outputs both data_mean and data_var as well, which are needed for the backward pass.

Besides the inputs and the outputs, this operator accepts two auxiliary states, moving_mean and moving_var, which are k-length vectors. They are global statistics for the whole dataset, which are updated by:

moving_mean = moving_mean * momentum + data_mean * (1 - momentum)
moving_var = moving_var * momentum + data_var * (1 - momentum)


If use_global_stats is set to be true, then moving_mean and moving_var are used instead of data_mean and data_var to compute the output. It is often used during inference.

Both gamma and beta are learnable parameters. But if fix_gamma is true, then set gamma to 1 and its gradient to 0.

Defined in src/operator/batch_norm_v1.cc:L90

Parameters: data (Symbol) – Input data to batch normalization gamma (Symbol) – gamma array beta (Symbol) – beta array eps (float, optional, default=0.001) – Epsilon to prevent div 0 momentum (float, optional, default=0.9) – Momentum for moving average fix_gamma (boolean, optional, default=1) – Fix gamma while training use_global_stats (boolean, optional, default=0) – Whether use global moving statistics instead of local batch-norm. This will force change batch-norm into a scale shift operator. output_mean_var (boolean, optional, default=0) – Output All,normal mean and var name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.BilinearSampler(data=None, grid=None, name=None, attr=None, out=None, **kwargs)

Applies bilinear sampling to input feature map.

Bilinear Sampling is the key of [NIPS2015] “Spatial Transformer Networks”. The usage of the operator is very similar to remap function in OpenCV, except that the operator has the backward pass.

Given $$data$$ and $$grid$$, then the output is computed by

$\begin{split}x_{src} = grid[batch, 0, y_{dst}, x_{dst}] \\ y_{src} = grid[batch, 1, y_{dst}, x_{dst}] \\ output[batch, channel, y_{dst}, x_{dst}] = G(data[batch, channel, y_{src}, x_{src})\end{split}$

$$x_{dst}$$, $$y_{dst}$$ enumerate all spatial locations in $$output$$, and $$G()$$ denotes the bilinear interpolation kernel. The out-boundary points will be padded with zeros.The shape of the output will be (data.shape[0], data.shape[1], grid.shape[2], grid.shape[3]).

The operator assumes that $$data$$ has ‘NCHW’ layout and $$grid$$ has been normalized to [-1, 1].

BilinearSampler often cooperates with GridGenerator which generates sampling grids for BilinearSampler. GridGenerator supports two kinds of transformation: affine and warp. If users want to design a CustomOp to manipulate $$grid$$, please firstly refer to the code of GridGenerator.

Example 1:

## Zoom out data two times
data = array([[[[1, 4, 3, 6],
[1, 8, 8, 9],
[0, 4, 1, 5],
[1, 0, 1, 3]]]])

affine_matrix = array([[2, 0, 0],
[0, 2, 0]])

affine_matrix = reshape(affine_matrix, shape=(1, 6))

grid = GridGenerator(data=affine_matrix, transform_type='affine', target_shape=(4, 4))

out = BilinearSampler(data, grid)

out
[[[[ 0,   0,     0,   0],
[ 0,   3.5,   6.5, 0],
[ 0,   1.25,  2.5, 0],
[ 0,   0,     0,   0]]]


Example 2:

## shift data horizontally by -1 pixel

data = array([[[[1, 4, 3, 6],
[1, 8, 8, 9],
[0, 4, 1, 5],
[1, 0, 1, 3]]]])

warp_maxtrix = array([[[[1, 1, 1, 1],
[1, 1, 1, 1],
[1, 1, 1, 1],
[1, 1, 1, 1]],
[[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0]]]])

grid = GridGenerator(data=warp_matrix, transform_type='warp')
out = BilinearSampler(data, grid)

out
[[[[ 4,  3,  6,  0],
[ 8,  8,  9,  0],
[ 4,  1,  5,  0],
[ 0,  1,  3,  0]]]


Defined in src/operator/bilinear_sampler.cc:L245

Parameters: data (Symbol) – Input data to the BilinearsamplerOp. grid (Symbol) – Input grid to the BilinearsamplerOp.grid has two channels: x_src, y_src name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.BlockGrad(data=None, name=None, attr=None, out=None, **kwargs)

Stops the accumulated gradient of the inputs from flowing through this operator in the backward direction. In other words, this operator prevents the contribution of its inputs to be taken into account for computing gradients.

Example:

v1 = [1, 2]
v2 = [0, 1]
a = Variable('a')
b = Variable('b')

executor = loss.simple_bind(ctx=cpu(), a=(1,2), b=(1,2))
executor.forward(is_train=True, a=v1, b=v2)
executor.outputs
[ 1.  5.]

executor.backward()
[ 0.  0.]
[ 1.  1.]


Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L166

Parameters: data (Symbol) – The input array. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.Cast(data=None, dtype=_Null, name=None, attr=None, out=None, **kwargs)

Casts all elements of the input to a new type.

Note

Cast is deprecated. Use cast instead.

Example:

cast([0.9, 1.3], dtype='int32') = [0, 1]
cast([1e20, 11.1], dtype='float16') = [inf, 11.09375]
cast([300, 11.1, 10.9, -1, -3], dtype='uint8') = [44, 11, 10, 255, 253]


Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L310

Parameters: data (Symbol) – The input. dtype ({'float16', 'float32', 'float64', 'int32', 'uint8'}, required) – Output data type. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.Concat(*data, **kwargs)

Joins input arrays along a given axis.

Note

Concat is deprecated. Use concat instead.

The dimensions of the input arrays should be the same except the axis along which they will be concatenated. The dimension of the output array along the concatenated axis will be equal to the sum of the corresponding dimensions of the input arrays.

Example:

x = [[1,1],[2,2]]
y = [[3,3],[4,4],[5,5]]
z = [[6,6], [7,7],[8,8]]

concat(x,y,z,dim=0) = [[ 1.,  1.],
[ 2.,  2.],
[ 3.,  3.],
[ 4.,  4.],
[ 5.,  5.],
[ 6.,  6.],
[ 7.,  7.],
[ 8.,  8.]]

Note that you cannot concat x,y,z along dimension 1 since dimension
0 is not the same for all the input arrays.

concat(y,z,dim=1) = [[ 3.,  3.,  6.,  6.],
[ 4.,  4.,  7.,  7.],
[ 5.,  5.,  8.,  8.]]


Defined in src/operator/concat.cc:L104 This function support variable length of positional input.

Parameters: data (Symbol[]) – List of arrays to concatenate dim (int, optional, default='1') – the dimension to be concated. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol

Examples

Concat two (or more) inputs along a specific dimension:

>>> a = Variable('a')
>>> b = Variable('b')
>>> c = Concat(a, b, dim=1, name='my-concat')
>>> c

>>> SymbolDoc.get_output_shape(c, a=(128, 10, 3, 3), b=(128, 15, 3, 3))
{'my-concat_output': (128L, 25L, 3L, 3L)}


Note the shape should be the same except on the dimension that is being concatenated.

mxnet.symbol.Convolution(data=None, weight=None, bias=None, kernel=_Null, stride=_Null, dilate=_Null, pad=_Null, num_filter=_Null, num_group=_Null, workspace=_Null, no_bias=_Null, cudnn_tune=_Null, cudnn_off=_Null, layout=_Null, name=None, attr=None, out=None, **kwargs)

Compute N-D convolution on (N+2)-D input.

In the 2-D convolution, given input data with shape (batch_size, channel, height, width), the output is computed by

$out[n,i,:,:] = bias[i] + \sum_{j=0}^{channel} data[n,j,:,:] \star weight[i,j,:,:]$

where $$\star$$ is the 2-D cross-correlation operator.

For general 2-D convolution, the shapes are

• data: (batch_size, channel, height, width)
• weight: (num_filter, channel, kernel[0], kernel[1])
• bias: (num_filter,)
• out: (batch_size, num_filter, out_height, out_width).

Define:

f(x,k,p,s,d) = floor((x+2*p-d*(k-1)-1)/s)+1


then we have:

out_height=f(height, kernel[0], pad[0], stride[0], dilate[0])


If no_bias is set to be true, then the bias term is ignored.

The default data layout is NCHW, namely (batch_size, channel, height, width). We can choose other layouts such as NHWC.

If num_group is larger than 1, denoted by g, then split the input data evenly into g parts along the channel axis, and also evenly split weight along the first dimension. Next compute the convolution on the i-th part of the data with the i-th weight part. The output is obtained by concatenating all the g results.

1-D convolution does not have height dimension but only width in space.

• data: (batch_size, channel, width)
• weight: (num_filter, channel, kernel[0])
• bias: (num_filter,)
• out: (batch_size, num_filter, out_width).

3-D convolution adds an additional depth dimension besides height and width. The shapes are

• data: (batch_size, channel, depth, height, width)
• weight: (num_filter, channel, kernel[0], kernel[1], kernel[2])
• bias: (num_filter,)
• out: (batch_size, num_filter, out_depth, out_height, out_width).

Both weight and bias are learnable parameters.

There are other options to tune the performance.

• cudnn_tune: enable this option leads to higher startup time but may give faster speed. Options are
• off: no tuning
• limited_workspace:run test and pick the fastest algorithm that doesn’t exceed workspace limit.
• fastest: pick the fastest algorithm and ignore workspace limit.
• None (default): the behavior is determined by environment variable MXNET_CUDNN_AUTOTUNE_DEFAULT. 0 for off, 1 for limited workspace (default), 2 for fastest.
• workspace: A large number leads to more (GPU) memory usage but may improve the performance.

Defined in src/operator/nn/convolution.cc:L170

Parameters: data (Symbol) – Input data to the ConvolutionOp. weight (Symbol) – Weight matrix. bias (Symbol) – Bias parameter. kernel (Shape(tuple), required) – Convolution kernel size: (w,), (h, w) or (d, h, w) stride (Shape(tuple), optional, default=[]) – Convolution stride: (w,), (h, w) or (d, h, w). Defaults to 1 for each dimension. dilate (Shape(tuple), optional, default=[]) – Convolution dilate: (w,), (h, w) or (d, h, w). Defaults to 1 for each dimension. pad (Shape(tuple), optional, default=[]) – Zero pad for convolution: (w,), (h, w) or (d, h, w). Defaults to no padding. num_filter (int (non-negative), required) – Convolution filter(channel) number num_group (int (non-negative), optional, default=1) – Number of group partitions. workspace (long (non-negative), optional, default=1024) – Maximum temporary workspace allowed for convolution (MB). no_bias (boolean, optional, default=0) – Whether to disable bias parameter. cudnn_tune ({None, 'fastest', 'limited_workspace', 'off'},optional, default='None') – Whether to pick convolution algo by running performance test. cudnn_off (boolean, optional, default=0) – Turn off cudnn for this layer. layout ({None, 'NCDHW', 'NCHW', 'NCW', 'NDHWC', 'NHWC'},optional, default='None') – Set layout for input, output and weight. Empty for default layout: NCW for 1d, NCHW for 2d and NCDHW for 3d. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.Convolution_v1(data=None, weight=None, bias=None, kernel=_Null, stride=_Null, dilate=_Null, pad=_Null, num_filter=_Null, num_group=_Null, workspace=_Null, no_bias=_Null, cudnn_tune=_Null, cudnn_off=_Null, layout=_Null, name=None, attr=None, out=None, **kwargs)

This operator is DEPRECATED. Apply convolution to input then add a bias.

Parameters: data (Symbol) – Input data to the ConvolutionV1Op. weight (Symbol) – Weight matrix. bias (Symbol) – Bias parameter. kernel (Shape(tuple), required) – convolution kernel size: (h, w) or (d, h, w) stride (Shape(tuple), optional, default=[]) – convolution stride: (h, w) or (d, h, w) dilate (Shape(tuple), optional, default=[]) – convolution dilate: (h, w) or (d, h, w) pad (Shape(tuple), optional, default=[]) – pad for convolution: (h, w) or (d, h, w) num_filter (int (non-negative), required) – convolution filter(channel) number num_group (int (non-negative), optional, default=1) – Number of group partitions. Equivalent to slicing input into num_group partitions, apply convolution on each, then concatenate the results workspace (long (non-negative), optional, default=1024) – Maximum tmp workspace allowed for convolution (MB). no_bias (boolean, optional, default=0) – Whether to disable bias parameter. cudnn_tune ({None, 'fastest', 'limited_workspace', 'off'},optional, default='None') – Whether to pick convolution algo by running performance test. Leads to higher startup time but may give faster speed. Options are: ‘off’: no tuning ‘limited_workspace’: run test and pick the fastest algorithm that doesn’t exceed workspace limit. ‘fastest’: pick the fastest algorithm and ignore workspace limit. If set to None (default), behavior is determined by environment variable MXNET_CUDNN_AUTOTUNE_DEFAULT: 0 for off, 1 for limited workspace (default), 2 for fastest. cudnn_off (boolean, optional, default=0) – Turn off cudnn for this layer. layout ({None, 'NCDHW', 'NCHW', 'NDHWC', 'NHWC'},optional, default='None') – Set layout for input, output and weight. Empty for default layout: NCHW for 2d and NCDHW for 3d. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.Correlation(data1=None, data2=None, kernel_size=_Null, max_displacement=_Null, stride1=_Null, stride2=_Null, pad_size=_Null, is_multiply=_Null, name=None, attr=None, out=None, **kwargs)

Applies correlation to inputs.

The correlation layer performs multiplicative patch comparisons between two feature maps.

Given two multi-channel feature maps $$f_{1}, f_{2}$$, with $$w$$, $$h$$, and $$c$$ being their width, height, and number of channels, the correlation layer lets the network compare each patch from $$f_{1}$$ with each patch from $$f_{2}$$.

For now we consider only a single comparison of two patches. The ‘correlation’ of two patches centered at $$x_{1}$$ in the first map and $$x_{2}$$ in the second map is then defined as:

$c(x_{1}, x_{2}) = \sum_{o \in [-k,k] \times [-k,k]}$

for a square patch of size $$K:=2k+1$$.

Note that the equation above is identical to one step of a convolution in neural networks, but instead of convolving data with a filter, it convolves data with other data. For this reason, it has no training weights.

Computing $$c(x_{1}, x_{2})$$ involves $$c * K^{2}$$ multiplications. Comparing all patch combinations involves $$w^{2}*h^{2}$$ such computations.

Given a maximum displacement $$d$$, for each location $$x_{1}$$ it computes correlations $$c(x_{1}, x_{2})$$ only in a neighborhood of size $$D:=2d+1$$, by limiting the range of $$x_{2}$$. We use strides $$s_{1}, s_{2}$$, to quantize $$x_{1}$$ globally and to quantize $$x_{2}$$ within the neighborhood centered around $$x_{1}$$.

The final output is defined by the following expression:

$out[n, q, i, j] = c(x_{i, j}, x_{q})$

where $$i$$ and $$j$$ enumerate spatial locations in $$f_{1}$$, and $$q$$ denotes the $$q^{th}$$ neighborhood of $$x_{i,j}$$.

Defined in src/operator/correlation.cc:L192

Parameters: data1 (Symbol) – Input data1 to the correlation. data2 (Symbol) – Input data2 to the correlation. kernel_size (int (non-negative), optional, default=1) – kernel size for Correlation must be an odd number max_displacement (int (non-negative), optional, default=1) – Max displacement of Correlation stride1 (int (non-negative), optional, default=1) – stride1 quantize data1 globally stride2 (int (non-negative), optional, default=1) – stride2 quantize data2 within the neighborhood centered around data1 pad_size (int (non-negative), optional, default=0) – pad for Correlation is_multiply (boolean, optional, default=1) – operation type is either multiplication or subduction name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.Crop(*data, **kwargs)

Note

Crop is deprecated. Use slice instead.

Crop the 2nd and 3rd dim of input data, with the corresponding size of h_w or with width and height of the second input symbol, i.e., with one input, we need h_w to specify the crop height and width, otherwise the second input symbol’s size will be used

Defined in src/operator/crop.cc:L50 This function support variable length of positional input.

Parameters: data (Symbol or Symbol[]) – Tensor or List of Tensors, the second input will be used as crop_like shape reference offset (Shape(tuple), optional, default=[0,0]) – crop offset coordinate: (y, x) h_w (Shape(tuple), optional, default=[0,0]) – crop height and width: (h, w) center_crop (boolean, optional, default=0) – If set to true, then it will use be the center_crop,or it will crop using the shape of crop_like name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.Custom(*data, **kwargs)

Apply a custom operator implemented in a frontend language (like Python).

Custom operators should override required methods like forward and backward. The custom operator must be registered before it can be used. Please check the tutorial here: https://mxnet.incubator.apache.org/versions/1.1.0/faq/new_op.html.

Defined in src/operator/custom/custom.cc:L369

Parameters: data (Symbol[]) – Input data for the custom operator. op_type (string) – Name of the custom operator. This is the name that is passed to mx.operator.register to register the operator. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.Deconvolution(data=None, weight=None, bias=None, kernel=_Null, stride=_Null, dilate=_Null, pad=_Null, adj=_Null, target_shape=_Null, num_filter=_Null, num_group=_Null, workspace=_Null, no_bias=_Null, cudnn_tune=_Null, cudnn_off=_Null, layout=_Null, name=None, attr=None, out=None, **kwargs)

Computes 1D or 2D transposed convolution (aka fractionally strided convolution) of the input tensor. This operation can be seen as the gradient of Convolution operation with respect to its input. Convolution usually reduces the size of the input. Transposed convolution works the other way, going from a smaller input to a larger output while preserving the connectivity pattern.

Parameters: data (Symbol) – Input tensor to the deconvolution operation. weight (Symbol) – Weights representing the kernel. bias (Symbol) – Bias added to the result after the deconvolution operation. kernel (Shape(tuple), required) – Deconvolution kernel size: (w,), (h, w) or (d, h, w). This is same as the kernel size used for the corresponding convolution stride (Shape(tuple), optional, default=[]) – The stride used for the corresponding convolution: (w,), (h, w) or (d, h, w). Defaults to 1 for each dimension. dilate (Shape(tuple), optional, default=[]) – Dilation factor for each dimension of the input: (w,), (h, w) or (d, h, w). Defaults to 1 for each dimension. pad (Shape(tuple), optional, default=[]) – The amount of implicit zero padding added during convolution for each dimension of the input: (w,), (h, w) or (d, h, w). (kernel-1)/2 is usually a good choice. If target_shape is set, pad will be ignored and a padding that will generate the target shape will be used. Defaults to no padding. adj (Shape(tuple), optional, default=[]) – Adjustment for output shape: (w,), (h, w) or (d, h, w). If target_shape is set, adj will be ignored and computed accordingly. target_shape (Shape(tuple), optional, default=[]) – Shape of the output tensor: (w,), (h, w) or (d, h, w). num_filter (int (non-negative), required) – Number of output filters. num_group (int (non-negative), optional, default=1) – Number of groups partition. workspace (long (non-negative), optional, default=512) – Maximum temporal workspace allowed for deconvolution (MB). no_bias (boolean, optional, default=1) – Whether to disable bias parameter. cudnn_tune ({None, 'fastest', 'limited_workspace', 'off'},optional, default='None') – Whether to pick convolution algorithm by running performance test. cudnn_off (boolean, optional, default=0) – Turn off cudnn for this layer. layout ({None, 'NCDHW', 'NCHW', 'NCW', 'NDHWC', 'NHWC'},optional, default='None') – Set layout for input, output and weight. Empty for default layout, NCW for 1d, NCHW for 2d and NCDHW for 3d. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.Dropout(data=None, p=_Null, mode=_Null, name=None, attr=None, out=None, **kwargs)

Applies dropout operation to input array.

• During training, each element of the input is set to zero with probability p. The whole array is rescaled by $$1/(1-p)$$ to keep the expected sum of the input unchanged.
• During testing, this operator does not change the input if mode is ‘training’. If mode is ‘always’, the same computaion as during training will be applied.

Example:

random.seed(998)
input_array = array([[3., 0.5,  -0.5,  2., 7.],
[2., -0.4,   7.,  3., 0.2]])
a = symbol.Variable('a')
dropout = symbol.Dropout(a, p = 0.2)
executor = dropout.simple_bind(a = input_array.shape)

## If training
executor.forward(is_train = True, a = input_array)
executor.outputs
[[ 3.75   0.625 -0.     2.5    8.75 ]
[ 2.5   -0.5    8.75   3.75   0.   ]]

## If testing
executor.forward(is_train = False, a = input_array)
executor.outputs
[[ 3.     0.5   -0.5    2.     7.   ]
[ 2.    -0.4    7.     3.     0.2  ]]


Defined in src/operator/nn/dropout.cc:L79

Parameters: data (Symbol) – Input array to which dropout will be applied. p (float, optional, default=0.5) – Fraction of the input that gets dropped out during training time. mode ({'always', 'training'},optional, default='training') – Whether to only turn on dropout during training or to also turn on for inference. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol

Examples

Apply dropout to corrupt input as zero with probability 0.2:

>>> data = Variable('data')
>>> data_dp = Dropout(data=data, p=0.2)

>>> shape = (100, 100)  # take larger shapes to be more statistical stable
>>> x = np.ones(shape)
>>> op = Dropout(p=0.5, name='dp')
>>> # dropout is identity during testing
>>> y = test_utils.simple_forward(op, dp_data=x, is_train=False)
>>> test_utils.almost_equal(x, y)
True
>>> y = test_utils.simple_forward(op, dp_data=x, is_train=True)
>>> # expectation is (approximately) unchanged
>>> np.abs(x.mean() - y.mean()) < 0.1
True
>>> set(np.unique(y)) == set([0, 2])
True

mxnet.symbol.ElementWiseSum(*args, **kwargs)

$add\_n(a_1, a_2, ..., a_n) = a_1 + a_2 + ... + a_n$

add_n is potentially more efficient than calling add by n times.

The storage type of add_n output depends on storage types of inputs

• add_n(row_sparse, row_sparse, ..) = row_sparse
• otherwise, add_n generates output with default storage

Defined in src/operator/tensor/elemwise_sum.cc:L123 This function support variable length of positional input.

Parameters: args (Symbol[]) – Positional input arguments name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.Embedding(data=None, weight=None, input_dim=_Null, output_dim=_Null, dtype=_Null, name=None, attr=None, out=None, **kwargs)

Maps integer indices to vector representations (embeddings).

This operator maps words to real-valued vectors in a high-dimensional space, called word embeddings. These embeddings can capture semantic and syntactic properties of the words. For example, it has been noted that in the learned embedding spaces, similar words tend to be close to each other and dissimilar words far apart.

For an input array of shape (d1, ..., dK), the shape of an output array is (d1, ..., dK, output_dim). All the input values should be integers in the range [0, input_dim).

If the input_dim is ip0 and output_dim is op0, then shape of the embedding weight matrix must be (ip0, op0).

By default, if any index mentioned is too large, it is replaced by the index that addresses the last vector in an embedding matrix.

Examples:

input_dim = 4
output_dim = 5

// Each row in weight matrix y represents a word. So, y = (w0,w1,w2,w3)
y = [[  0.,   1.,   2.,   3.,   4.],
[  5.,   6.,   7.,   8.,   9.],
[ 10.,  11.,  12.,  13.,  14.],
[ 15.,  16.,  17.,  18.,  19.]]

// Input array x represents n-grams(2-gram). So, x = [(w1,w3), (w0,w2)]
x = [[ 1.,  3.],
[ 0.,  2.]]

// Mapped input x to its vector representation y.
Embedding(x, y, 4, 5) = [[[  5.,   6.,   7.,   8.,   9.],
[ 15.,  16.,  17.,  18.,  19.]],

[[  0.,   1.,   2.,   3.,   4.],
[ 10.,  11.,  12.,  13.,  14.]]]


Defined in src/operator/tensor/indexing_op.cc:L225

Parameters: data (Symbol) – The input array to the embedding operator. weight (Symbol) – The embedding weight matrix. input_dim (int, required) – Vocabulary size of the input indices. output_dim (int, required) – Dimension of the embedding vectors. dtype ({'float16', 'float32', 'float64', 'int32', 'uint8'},optional, default='float32') – Data type of weight. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol

Examples

Assume we want to map the 26 English alphabet letters to 16-dimensional vectorial representations.

>>> vocabulary_size = 26
>>> embed_dim = 16
>>> seq_len, batch_size = (10, 64)
>>> input = Variable('letters')
>>> op = Embedding(data=input, input_dim=vocabulary_size, output_dim=embed_dim,
...name='embed')
>>> SymbolDoc.get_output_shape(op, letters=(seq_len, batch_size))
{'embed_output': (10L, 64L, 16L)}

>>> vocab_size, embed_dim = (26, 16)
>>> batch_size = 12
>>> word_vecs = test_utils.random_arrays((vocab_size, embed_dim))
>>> op = Embedding(name='embed', input_dim=vocab_size, output_dim=embed_dim)
>>> x = np.random.choice(vocab_size, batch_size)
>>> y = test_utils.simple_forward(op, embed_data=x, embed_weight=word_vecs)
>>> y_np = word_vecs[x]
>>> test_utils.almost_equal(y, y_np)
True

mxnet.symbol.Flatten(data=None, name=None, attr=None, out=None, **kwargs)

Flattens the input array into a 2-D array by collapsing the higher dimensions.

Note

Flatten is deprecated. Use flatten instead.

For an input array with shape (d1, d2, ..., dk), flatten operation reshapes the input array into an output array of shape (d1, d2*...*dk).

Example:

x = [[
[1,2,3],
[4,5,6],
[7,8,9]
],
[    [1,2,3],
[4,5,6],
[7,8,9]
]],

flatten(x) = [[ 1.,  2.,  3.,  4.,  5.,  6.,  7.,  8.,  9.],
[ 1.,  2.,  3.,  4.,  5.,  6.,  7.,  8.,  9.]]


Defined in src/operator/tensor/matrix_op.cc:L208

Parameters: data (Symbol) – Input array. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol

Examples

Flatten is usually applied before FullyConnected, to reshape the 4D tensor produced by convolutional layers to 2D matrix:

>>> data = Variable('data')  # say this is 4D from some conv/pool
>>> flatten = Flatten(data=data, name='flat')  # now this is 2D
>>> SymbolDoc.get_output_shape(flatten, data=(2, 3, 4, 5))
{'flat_output': (2L, 60L)}

>>> test_dims = [(2, 3, 4, 5), (2, 3), (2,)]
>>> op = Flatten(name='flat')
>>> for dims in test_dims:
... x = test_utils.random_arrays(dims)
... y = test_utils.simple_forward(op, flat_data=x)
... y_np = x.reshape((dims[0], np.prod(dims[1:]).astype('int32')))
... print('%s: %s' % (dims, test_utils.almost_equal(y, y_np)))
(2, 3, 4, 5): True
(2, 3): True
(2,): True

mxnet.symbol.FullyConnected(data=None, weight=None, bias=None, num_hidden=_Null, no_bias=_Null, flatten=_Null, name=None, attr=None, out=None, **kwargs)

Applies a linear transformation: $$Y = XW^T + b$$.

If flatten is set to be true, then the shapes are:

• data: (batch_size, x1, x2, ..., xn)
• weight: (num_hidden, x1 * x2 * ... * xn)
• bias: (num_hidden,)
• out: (batch_size, num_hidden)

If flatten is set to be false, then the shapes are:

• data: (x1, x2, ..., xn, input_dim)
• weight: (num_hidden, input_dim)
• bias: (num_hidden,)
• out: (x1, x2, ..., xn, num_hidden)

The learnable parameters include both weight and bias.

If no_bias is set to be true, then the bias term is ignored.

Defined in src/operator/nn/fully_connected.cc:L98

Parameters: data (Symbol) – Input data. weight (Symbol) – Weight matrix. bias (Symbol) – Bias parameter. num_hidden (int, required) – Number of hidden nodes of the output. no_bias (boolean, optional, default=0) – Whether to disable bias parameter. flatten (boolean, optional, default=1) – Whether to collapse all but the first axis of the input data tensor. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol

Examples

Construct a fully connected operator with target dimension 512.

>>> data = Variable('data')  # or some constructed NN
>>> op = FullyConnected(data=data,
... num_hidden=512,
... name='FC1')
>>> op

>>> SymbolDoc.get_output_shape(op, data=(128, 100))
{'FC1_output': (128L, 512L)}


A simple 3-layer MLP with ReLU activation:

>>> net = Variable('data')
>>> for i, dim in enumerate([128, 64]):
... net = FullyConnected(data=net, num_hidden=dim, name='FC%d' % i)
... net = Activation(data=net, act_type='relu', name='ReLU%d' % i)
>>> # 10-class predictor (e.g. MNIST)
>>> net = FullyConnected(data=net, num_hidden=10, name='pred')
>>> net


>>> dim_in, dim_out = (3, 4)
>>> x, w, b = test_utils.random_arrays((10, dim_in), (dim_out, dim_in), (dim_out,))
>>> op = FullyConnected(num_hidden=dim_out, name='FC')
>>> out = test_utils.simple_forward(op, FC_data=x, FC_weight=w, FC_bias=b)
>>> # numpy implementation of FullyConnected
>>> out_np = np.dot(x, w.T) + b
>>> test_utils.almost_equal(out, out_np)
True

mxnet.symbol.GridGenerator(data=None, transform_type=_Null, target_shape=_Null, name=None, attr=None, out=None, **kwargs)

Generates 2D sampling grid for bilinear sampling.

Parameters: data (Symbol) – Input data to the function. transform_type ({'affine', 'warp'}, required) – The type of transformation. For affine, input data should be an affine matrix of size (batch, 6). For warp, input data should be an optical flow of size (batch, 2, h, w). target_shape (Shape(tuple), optional, default=[0,0]) – Specifies the output shape (H, W). This is required if transformation type is affine. If transformation type is warp, this parameter is ignored. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.IdentityAttachKLSparseReg(data=None, sparseness_target=_Null, penalty=_Null, momentum=_Null, name=None, attr=None, out=None, **kwargs)

Apply a sparse regularization to the output a sigmoid activation function.

Parameters: data (Symbol) – Input data. sparseness_target (float, optional, default=0.1) – The sparseness target penalty (float, optional, default=0.001) – The tradeoff parameter for the sparseness penalty momentum (float, optional, default=0.9) – The momentum for running average name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.InstanceNorm(data=None, gamma=None, beta=None, eps=_Null, name=None, attr=None, out=None, **kwargs)

Applies instance normalization to the n-dimensional input array.

This operator takes an n-dimensional input array where (n>2) and normalizes the input using the following formula:

$out = \frac{x - mean[data]}{ \sqrt{Var[data]} + \epsilon} * gamma + beta$

This layer is similar to batch normalization layer (BatchNorm) with two differences: first, the normalization is carried out per example (instance), not over a batch. Second, the same normalization is applied both at test and train time. This operation is also known as contrast normalization.

If the input data is of shape [batch, channel, spacial_dim1, spacial_dim2, ...], gamma and beta parameters must be vectors of shape [channel].

This implementation is based on paper:

 [1] Instance Normalization: The Missing Ingredient for Fast Stylization, D. Ulyanov, A. Vedaldi, V. Lempitsky, 2016 (arXiv:1607.08022v2).

Examples:

// Input of shape (2,1,2)
x = [[[ 1.1,  2.2]],
[[ 3.3,  4.4]]]

// gamma parameter of length 1
gamma = [1.5]

// beta parameter of length 1
beta = [0.5]

// Instance normalization is calculated with the above formula
InstanceNorm(x,gamma,beta) = [[[-0.997527  ,  1.99752665]],
[[-0.99752653,  1.99752724]]]


Defined in src/operator/instance_norm.cc:L95

Parameters: data (Symbol) – An n-dimensional input array (n > 2) of the form [batch, channel, spatial_dim1, spatial_dim2, ...]. gamma (Symbol) – A vector of length ‘channel’, which multiplies the normalized input. beta (Symbol) – A vector of length ‘channel’, which is added to the product of the normalized input and the weight. eps (float, optional, default=0.001) – An epsilon parameter to prevent division by 0. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.L2Normalization(data=None, eps=_Null, mode=_Null, name=None, attr=None, out=None, **kwargs)

Normalize the input array using the L2 norm.

For 1-D NDArray, it computes:

out = data / sqrt(sum(data ** 2) + eps)


For N-D NDArray, if the input array has shape (N, N, ..., N),

with mode = instance, it normalizes each instance in the multidimensional array by its L2 norm.:

for i in 0...N
out[i,:,:,...,:] = data[i,:,:,...,:] / sqrt(sum(data[i,:,:,...,:] ** 2) + eps)


with mode = channel, it normalizes each channel in the array by its L2 norm.:

for i in 0...N
out[:,i,:,...,:] = data[:,i,:,...,:] / sqrt(sum(data[:,i,:,...,:] ** 2) + eps)


with mode = spatial, it normalizes the cross channel norm for each position in the array by its L2 norm.:

for dim in 2...N
for i in 0...N
out[.....,i,...] = take(out, indices=i, axis=dim) / sqrt(sum(take(out, indices=i, axis=dim) ** 2) + eps)
-dim-


Example:

x = [[[1,2],
[3,4]],
[[2,2],
[5,6]]]

L2Normalization(x, mode='instance')
=[[[ 0.18257418  0.36514837]
[ 0.54772252  0.73029673]]
[[ 0.24077171  0.24077171]
[ 0.60192931  0.72231513]]]

L2Normalization(x, mode='channel')
=[[[ 0.31622776  0.44721359]
[ 0.94868326  0.89442718]]
[[ 0.37139067  0.31622776]
[ 0.92847669  0.94868326]]]

L2Normalization(x, mode='spatial')
=[[[ 0.44721359  0.89442718]
[ 0.60000002  0.80000001]]
[[ 0.70710677  0.70710677]
[ 0.6401844   0.76822126]]]


Defined in src/operator/l2_normalization.cc:L93

Parameters: data (Symbol) – Input array to normalize. eps (float, optional, default=1e-10) – A small constant for numerical stability. mode ({'channel', 'instance', 'spatial'},optional, default='instance') – Specify the dimension along which to compute L2 norm. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.LRN(data=None, alpha=_Null, beta=_Null, knorm=_Null, nsize=_Null, name=None, attr=None, out=None, **kwargs)

Applies local response normalization to the input.

The local response normalization layer performs “lateral inhibition” by normalizing over local input regions.

If $$a_{x,y}^{i}$$ is the activity of a neuron computed by applying kernel $$i$$ at position $$(x, y)$$ and then applying the ReLU nonlinearity, the response-normalized activity $$b_{x,y}^{i}$$ is given by the expression:

$b_{x,y}^{i} = \frac{a_{x,y}^{i}}{\Bigg({k + \alpha \sum_{j=max(0, i-\frac{n}{2})}^{min(N-1, i+\frac{n}{2})} (a_{x,y}^{j})^{2}}\Bigg)^{\beta}}$

where the sum runs over $$n$$ “adjacent” kernel maps at the same spatial position, and $$N$$ is the total number of kernels in the layer.

Defined in src/operator/lrn.cc:L73

Parameters: data (Symbol) – Input data. alpha (float, optional, default=0.0001) – The variance scaling parameter $$lpha$$ in the LRN expression. beta (float, optional, default=0.75) – The power parameter $$eta$$ in the LRN expression. knorm (float, optional, default=2) – The parameter $$k$$ in the LRN expression. nsize (int (non-negative), required) – normalization window width in elements. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.LeakyReLU(data=None, act_type=_Null, slope=_Null, lower_bound=_Null, upper_bound=_Null, name=None, attr=None, out=None, **kwargs)

Applies Leaky rectified linear unit activation element-wise to the input.

Leaky ReLUs attempt to fix the “dying ReLU” problem by allowing a small slope when the input is negative and has a slope of one when input is positive.

The following modified ReLU Activation functions are supported:

• elu: Exponential Linear Unit. y = x > 0 ? x : slope * (exp(x)-1)
• leaky: Leaky ReLU. y = x > 0 ? x : slope * x
• prelu: Parametric ReLU. This is same as leaky except that slope is learnt during training.
• rrelu: Randomized ReLU. same as leaky but the slope is uniformly and randomly chosen from [lower_bound, upper_bound) for training, while fixed to be (lower_bound+upper_bound)/2 for inference.

Defined in src/operator/leaky_relu.cc:L58

Parameters: data (Symbol) – Input data to activation function. act_type ({'elu', 'leaky', 'prelu', 'rrelu'},optional, default='leaky') – Activation function to be applied. slope (float, optional, default=0.25) – Init slope for the activation. (For leaky and elu only) lower_bound (float, optional, default=0.125) – Lower bound of random slope. (For rrelu only) upper_bound (float, optional, default=0.334) – Upper bound of random slope. (For rrelu only) name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.LinearRegressionOutput(data=None, label=None, grad_scale=_Null, name=None, attr=None, out=None, **kwargs)

Computes and optimizes for squared loss during backward propagation. Just outputs data during forward propagation.

If $$\hat{y}_i$$ is the predicted value of the i-th sample, and $$y_i$$ is the corresponding target value, then the squared loss estimated over $$n$$ samples is defined as

$$\text{SquaredLoss}(\textbf{Y}, \hat{\textbf{Y}} ) = \frac{1}{n} \sum_{i=0}^{n-1} \lVert \textbf{y}_i - \hat{\textbf{y}}_i \rVert_2$$

Note

Use the LinearRegressionOutput as the final output layer of a net.

By default, gradients of this loss function are scaled by factor 1/m, where m is the number of regression outputs of a training example. The parameter grad_scale can be used to change this scale to grad_scale/m.

Defined in src/operator/regression_output.cc:L80

Parameters: data (Symbol) – Input data to the function. label (Symbol) – Input label to the function. grad_scale (float, optional, default=1) – Scale the gradient by a float factor name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.LogisticRegressionOutput(data=None, label=None, grad_scale=_Null, name=None, attr=None, out=None, **kwargs)

Applies a logistic function to the input.

The logistic function, also known as the sigmoid function, is computed as $$\frac{1}{1+exp(-\textbf{x})}$$.

Commonly, the sigmoid is used to squash the real-valued output of a linear model :math:wTx+b into the [0,1] range so that it can be interpreted as a probability. It is suitable for binary classification or probability prediction tasks.

Note

Use the LogisticRegressionOutput as the final output layer of a net.

By default, gradients of this loss function are scaled by factor 1/m, where m is the number of regression outputs of a training example. The parameter grad_scale can be used to change this scale to grad_scale/m.

Defined in src/operator/regression_output.cc:L122

Parameters: data (Symbol) – Input data to the function. label (Symbol) – Input label to the function. grad_scale (float, optional, default=1) – Scale the gradient by a float factor name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.MAERegressionOutput(data=None, label=None, grad_scale=_Null, name=None, attr=None, out=None, **kwargs)

Computes mean absolute error of the input.

MAE is a risk metric corresponding to the expected value of the absolute error.

If $$\hat{y}_i$$ is the predicted value of the i-th sample, and $$y_i$$ is the corresponding target value, then the mean absolute error (MAE) estimated over $$n$$ samples is defined as

$$\text{MAE}(\textbf{Y}, \hat{\textbf{Y}} ) = \frac{1}{n} \sum_{i=0}^{n-1} \lVert \textbf{y}_i - \hat{\textbf{y}}_i \rVert_1$$

Note

Use the MAERegressionOutput as the final output layer of a net.

By default, gradients of this loss function are scaled by factor 1/m, where m is the number of regression outputs of a training example. The parameter grad_scale can be used to change this scale to grad_scale/m.

Defined in src/operator/regression_output.cc:L101

Parameters: data (Symbol) – Input data to the function. label (Symbol) – Input label to the function. grad_scale (float, optional, default=1) – Scale the gradient by a float factor name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.MakeLoss(data=None, grad_scale=_Null, valid_thresh=_Null, normalization=_Null, name=None, attr=None, out=None, **kwargs)

Make your own loss function in network construction.

This operator accepts a customized loss function symbol as a terminal loss and the symbol should be an operator with no backward dependency. The output of this function is the gradient of loss with respect to the input data.

For example, if you are a making a cross entropy loss function. Assume out is the predicted output and label is the true label, then the cross entropy can be defined as:

cross_entropy = label * log(out) + (1 - label) * log(1 - out)
loss = MakeLoss(cross_entropy)


We will need to use MakeLoss when we are creating our own loss function or we want to combine multiple loss functions. Also we may want to stop some variables’ gradients from backpropagation. See more detail in BlockGrad or stop_gradient.

In addition, we can give a scale to the loss by setting grad_scale, so that the gradient of the loss will be rescaled in the backpropagation.

Note

This operator should be used as a Symbol instead of NDArray.

Defined in src/operator/make_loss.cc:L71

Parameters: data (Symbol) – Input array. grad_scale (float, optional, default=1) – Gradient scale as a supplement to unary and binary operators valid_thresh (float, optional, default=0) – clip each element in the array to 0 when it is less than valid_thresh. This is used when normalization is set to 'valid'. normalization ({'batch', 'null', 'valid'},optional, default='null') – If this is set to null, the output gradient will not be normalized. If this is set to batch, the output gradient will be divided by the batch size. If this is set to valid, the output gradient will be divided by the number of valid input elements. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.Pad(data=None, mode=_Null, pad_width=_Null, constant_value=_Null, name=None, attr=None, out=None, **kwargs)

Pads an input array with a constant or edge values of the array.

Note

Note

Current implementation only supports 4D and 5D input arrays with padding applied only on axes 1, 2 and 3. Expects axes 4 and 5 in pad_width to be zero.

This operation pads an input array with either a constant_value or edge values along each axis of the input array. The amount of padding is specified by pad_width.

pad_width is a tuple of integer padding widths for each axis of the format (before_1, after_1, ... , before_N, after_N). The pad_width should be of length 2*N where N is the number of dimensions of the array.

For dimension N of the input array, before_N and after_N indicates how many values to add before and after the elements of the array along dimension N. The widths of the higher two dimensions before_1, after_1, before_2, after_2 must be 0.

Example:

x = [[[[  1.   2.   3.]
[  4.   5.   6.]]

[[  7.   8.   9.]
[ 10.  11.  12.]]]

[[[ 11.  12.  13.]
[ 14.  15.  16.]]

[[ 17.  18.  19.]
[ 20.  21.  22.]]]]

[[[[  1.   1.   2.   3.   3.]
[  1.   1.   2.   3.   3.]
[  4.   4.   5.   6.   6.]
[  4.   4.   5.   6.   6.]]

[[  7.   7.   8.   9.   9.]
[  7.   7.   8.   9.   9.]
[ 10.  10.  11.  12.  12.]
[ 10.  10.  11.  12.  12.]]]

[[[ 11.  11.  12.  13.  13.]
[ 11.  11.  12.  13.  13.]
[ 14.  14.  15.  16.  16.]
[ 14.  14.  15.  16.  16.]]

[[ 17.  17.  18.  19.  19.]
[ 17.  17.  18.  19.  19.]
[ 20.  20.  21.  22.  22.]
[ 20.  20.  21.  22.  22.]]]]

[[[[  0.   0.   0.   0.   0.]
[  0.   1.   2.   3.   0.]
[  0.   4.   5.   6.   0.]
[  0.   0.   0.   0.   0.]]

[[  0.   0.   0.   0.   0.]
[  0.   7.   8.   9.   0.]
[  0.  10.  11.  12.   0.]
[  0.   0.   0.   0.   0.]]]

[[[  0.   0.   0.   0.   0.]
[  0.  11.  12.  13.   0.]
[  0.  14.  15.  16.   0.]
[  0.   0.   0.   0.   0.]]

[[  0.   0.   0.   0.   0.]
[  0.  17.  18.  19.   0.]
[  0.  20.  21.  22.   0.]
[  0.   0.   0.   0.   0.]]]]


Parameters: data (Symbol) – An n-dimensional input array. mode ({'constant', 'edge', 'reflect'}, required) – Padding type to use. “constant” pads with constant_value “edge” pads using the edge values of the input array “reflect” pads by reflecting values with respect to the edges. pad_width (Shape(tuple), required) – Widths of the padding regions applied to the edges of each axis. It is a tuple of integer padding widths for each axis of the format (before_1, after_1, ... , before_N, after_N). It should be of length 2*N where N is the number of dimensions of the array.This is equivalent to pad_width in numpy.pad, but flattened. constant_value (double, optional, default=0) – The value used for padding when mode is “constant”. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.Pooling(data=None, global_pool=_Null, cudnn_off=_Null, kernel=_Null, pool_type=_Null, pooling_convention=_Null, stride=_Null, pad=_Null, name=None, attr=None, out=None, **kwargs)

Performs pooling on the input.

The shapes for 1-D pooling are

• data: (batch_size, channel, width),
• out: (batch_size, num_filter, out_width).

The shapes for 2-D pooling are

• data: (batch_size, channel, height, width)

• out: (batch_size, num_filter, out_height, out_width), with:

out_height = f(height, kernel[0], pad[0], stride[0])
out_width = f(width, kernel[1], pad[1], stride[1])


The definition of f depends on pooling_convention, which has two options:

• valid (default):

f(x, k, p, s) = floor((x+2*p-k)/s)+1

• full, which is compatible with Caffe:

f(x, k, p, s) = ceil((x+2*p-k)/s)+1


But global_pool is set to be true, then do a global pooling, namely reset kernel=(height, width).

Three pooling options are supported by pool_type:

• avg: average pooling
• max: max pooling
• sum: sum pooling

For 3-D pooling, an additional depth dimension is added before height. Namely the input data will have shape (batch_size, channel, depth, height, width).

Defined in src/operator/nn/pooling.cc:L133

Parameters: data (Symbol) – Input data to the pooling operator. global_pool (boolean, optional, default=0) – Ignore kernel size, do global pooling based on current input feature map. cudnn_off (boolean, optional, default=0) – Turn off cudnn pooling and use MXNet pooling operator. kernel (Shape(tuple), required) – Pooling kernel size: (y, x) or (d, y, x) pool_type ({'avg', 'max', 'sum'}, required) – Pooling type to be applied. pooling_convention ({'full', 'valid'},optional, default='valid') – Pooling convention to be applied. stride (Shape(tuple), optional, default=[]) – Stride: for pooling (y, x) or (d, y, x). Defaults to 1 for each dimension. pad (Shape(tuple), optional, default=[]) – Pad for pooling: (y, x) or (d, y, x). Defaults to no padding. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.Pooling_v1(data=None, global_pool=_Null, kernel=_Null, pool_type=_Null, pooling_convention=_Null, stride=_Null, pad=_Null, name=None, attr=None, out=None, **kwargs)

This operator is DEPRECATED. Perform pooling on the input.

The shapes for 2-D pooling is

• data: (batch_size, channel, height, width)

• out: (batch_size, num_filter, out_height, out_width), with:

out_height = f(height, kernel[0], pad[0], stride[0])
out_width = f(width, kernel[1], pad[1], stride[1])


The definition of f depends on pooling_convention, which has two options:

• valid (default):

f(x, k, p, s) = floor((x+2*p-k)/s)+1

• full, which is compatible with Caffe:

f(x, k, p, s) = ceil((x+2*p-k)/s)+1


But global_pool is set to be true, then do a global pooling, namely reset kernel=(height, width).

Three pooling options are supported by pool_type:

• avg: average pooling
• max: max pooling
• sum: sum pooling

1-D pooling is special case of 2-D pooling with weight=1 and kernel[1]=1.

For 3-D pooling, an additional depth dimension is added before height. Namely the input data will have shape (batch_size, channel, depth, height, width).

Defined in src/operator/pooling_v1.cc:L104

Parameters: data (Symbol) – Input data to the pooling operator. global_pool (boolean, optional, default=0) – Ignore kernel size, do global pooling based on current input feature map. kernel (Shape(tuple), required) – pooling kernel size: (y, x) or (d, y, x) pool_type ({'avg', 'max', 'sum'}, required) – Pooling type to be applied. pooling_convention ({'full', 'valid'},optional, default='valid') – Pooling convention to be applied. stride (Shape(tuple), optional, default=[]) – stride: for pooling (y, x) or (d, y, x) pad (Shape(tuple), optional, default=[]) – pad for pooling: (y, x) or (d, y, x) name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.RNN(data=None, parameters=None, state=None, state_cell=None, state_size=_Null, num_layers=_Null, bidirectional=_Null, mode=_Null, p=_Null, state_outputs=_Null, name=None, attr=None, out=None, **kwargs)

Applies a recurrent layer to input.

Parameters: data (Symbol) – Input data to RNN parameters (Symbol) – Vector of all RNN trainable parameters concatenated state (Symbol) – initial hidden state of the RNN state_cell (Symbol) – initial cell state for LSTM networks (only for LSTM) state_size (int (non-negative), required) – size of the state for each layer num_layers (int (non-negative), required) – number of stacked layers bidirectional (boolean, optional, default=0) – whether to use bidirectional recurrent layers mode ({'gru', 'lstm', 'rnn_relu', 'rnn_tanh'}, required) – the type of RNN to compute p (float, optional, default=0) – Dropout probability, fraction of the input that gets dropped out at training time state_outputs (boolean, optional, default=0) – Whether to have the states as symbol outputs. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.ROIPooling(data=None, rois=None, pooled_size=_Null, spatial_scale=_Null, name=None, attr=None, out=None, **kwargs)

Performs region of interest(ROI) pooling on the input array.

ROI pooling is a variant of a max pooling layer, in which the output size is fixed and region of interest is a parameter. Its purpose is to perform max pooling on the inputs of non-uniform sizes to obtain fixed-size feature maps. ROI pooling is a neural-net layer mostly used in training a Fast R-CNN network for object detection.

This operator takes a 4D feature map as an input array and region proposals as rois, then it pools over sub-regions of input and produces a fixed-sized output array regardless of the ROI size.

To crop the feature map accordingly, you can resize the bounding box coordinates by changing the parameters rois and spatial_scale.

The cropped feature maps are pooled by standard max pooling operation to a fixed size output indicated by a pooled_size parameter. batch_size will change to the number of region bounding boxes after ROIPooling.

The size of each region of interest doesn’t have to be perfectly divisible by the number of pooling sections(pooled_size).

Example:

x = [[[[  0.,   1.,   2.,   3.,   4.,   5.],
[  6.,   7.,   8.,   9.,  10.,  11.],
[ 12.,  13.,  14.,  15.,  16.,  17.],
[ 18.,  19.,  20.,  21.,  22.,  23.],
[ 24.,  25.,  26.,  27.,  28.,  29.],
[ 30.,  31.,  32.,  33.,  34.,  35.],
[ 36.,  37.,  38.,  39.,  40.,  41.],
[ 42.,  43.,  44.,  45.,  46.,  47.]]]]

// region of interest i.e. bounding box coordinates.
y = [[0,0,0,4,4]]

// returns array of shape (2,2) according to the given roi with max pooling.
ROIPooling(x, y, (2,2), 1.0) = [[[[ 14.,  16.],
[ 26.,  28.]]]]

// region of interest is changed due to the change in spacial_scale parameter.
ROIPooling(x, y, (2,2), 0.7) = [[[[  7.,   9.],
[ 19.,  21.]]]]


Defined in src/operator/roi_pooling.cc:L287

Parameters: data (Symbol) – The input array to the pooling operator, a 4D Feature maps rois (Symbol) – Bounding box coordinates, a 2D array of [[batch_index, x1, y1, x2, y2]], where (x1, y1) and (x2, y2) are top left and bottom right corners of designated region of interest. batch_index indicates the index of corresponding image in the input array pooled_size (Shape(tuple), required) – ROI pooling output shape (h,w) spatial_scale (float, required) – Ratio of input feature map height (or w) to raw image height (or w). Equals the reciprocal of total stride in convolutional layers name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.Reshape(data=None, shape=_Null, reverse=_Null, target_shape=_Null, keep_highest=_Null, name=None, attr=None, out=None, **kwargs)

Reshapes the input array.

Note

Reshape is deprecated, use reshape

Given an array and a shape, this function returns a copy of the array in the new shape. The shape is a tuple of integers such as (2,3,4).The size of the new shape should be same as the size of the input array.

Example:

reshape([1,2,3,4], shape=(2,2)) = [[1,2], [3,4]]


Some dimensions of the shape can take special values from the set {0, -1, -2, -3, -4}. The significance of each is explained below:

• 0 copy this dimension from the input to the output shape.

Example:

- input shape = (2,3,4), shape = (4,0,2), output shape = (4,3,2)
- input shape = (2,3,4), shape = (2,0,0), output shape = (2,3,4)

• -1 infers the dimension of the output shape by using the remainder of the input dimensions keeping the size of the new array same as that of the input array. At most one dimension of shape can be -1.

Example:

- input shape = (2,3,4), shape = (6,1,-1), output shape = (6,1,4)
- input shape = (2,3,4), shape = (3,-1,8), output shape = (3,1,8)
- input shape = (2,3,4), shape=(-1,), output shape = (24,)

• -2 copy all/remainder of the input dimensions to the output shape.

Example:

- input shape = (2,3,4), shape = (-2,), output shape = (2,3,4)
- input shape = (2,3,4), shape = (2,-2), output shape = (2,3,4)
- input shape = (2,3,4), shape = (-2,1,1), output shape = (2,3,4,1,1)

• -3 use the product of two consecutive dimensions of the input shape as the output dimension.

Example:

- input shape = (2,3,4), shape = (-3,4), output shape = (6,4)
- input shape = (2,3,4,5), shape = (-3,-3), output shape = (6,20)
- input shape = (2,3,4), shape = (0,-3), output shape = (2,12)
- input shape = (2,3,4), shape = (-3,-2), output shape = (6,4)

• -4 split one dimension of the input into two dimensions passed subsequent to -4 in shape (can contain -1).

Example:

- input shape = (2,3,4), shape = (-4,1,2,-2), output shape =(1,2,3,4)
- input shape = (2,3,4), shape = (2,-4,-1,3,-2), output shape = (2,1,3,4)


If the argument reverse is set to 1, then the special values are inferred from right to left.

Example:

- without reverse=1, for input shape = (10,5,4), shape = (-1,0), output shape would be (40,5)
- with reverse=1, output shape will be (50,4).


Defined in src/operator/tensor/matrix_op.cc:L164

Parameters: data (Symbol) – Input data to reshape. shape (Shape(tuple), optional, default=[]) – The target shape reverse (boolean, optional, default=0) – If true then the special values are inferred from right to left target_shape (Shape(tuple), optional, default=[]) – (Deprecated! Use shape instead.) Target new shape. One and only one dim can be 0, in which case it will be inferred from the rest of dims keep_highest (boolean, optional, default=0) – (Deprecated! Use shape instead.) Whether keep the highest dim unchanged.If set to true, then the first dim in target_shape is ignored,and always fixed as input name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.SVMOutput(data=None, label=None, margin=_Null, regularization_coefficient=_Null, use_linear=_Null, name=None, attr=None, out=None, **kwargs)

Computes support vector machine based transformation of the input.

This tutorial demonstrates using SVM as output layer for classification instead of softmax: https://github.com/dmlc/mxnet/tree/master/example/svm_mnist.

Parameters: data (Symbol) – Input data for SVM transformation. label (Symbol) – Class label for the input data. margin (float, optional, default=1) – The loss function penalizes outputs that lie outside this margin. Default margin is 1. regularization_coefficient (float, optional, default=1) – Regularization parameter for the SVM. This balances the tradeoff between coefficient size and error. use_linear (boolean, optional, default=0) – Whether to use L1-SVM objective. L2-SVM objective is used by default. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.SequenceLast(data=None, sequence_length=None, use_sequence_length=_Null, axis=_Null, name=None, attr=None, out=None, **kwargs)

Takes the last element of a sequence.

This function takes an n-dimensional input array of the form [max_sequence_length, batch_size, other_feature_dims] and returns a (n-1)-dimensional array of the form [batch_size, other_feature_dims].

Parameter sequence_length is used to handle variable-length sequences. sequence_length should be an input array of positive ints of dimension [batch_size]. To use this parameter, set use_sequence_length to True, otherwise each example in the batch is assumed to have the max sequence length.

Note

Alternatively, you can also use take operator.

Example:

x = [[[  1.,   2.,   3.],
[  4.,   5.,   6.],
[  7.,   8.,   9.]],

[[ 10.,   11.,   12.],
[ 13.,   14.,   15.],
[ 16.,   17.,   18.]],

[[  19.,   20.,   21.],
[  22.,   23.,   24.],
[  25.,   26.,   27.]]]

// returns last sequence when sequence_length parameter is not used
SequenceLast(x) = [[  19.,   20.,   21.],
[  22.,   23.,   24.],
[  25.,   26.,   27.]]

// sequence_length is used
SequenceLast(x, sequence_length=[1,1,1], use_sequence_length=True) =
[[  1.,   2.,   3.],
[  4.,   5.,   6.],
[  7.,   8.,   9.]]

// sequence_length is used
SequenceLast(x, sequence_length=[1,2,3], use_sequence_length=True) =
[[  1.,    2.,   3.],
[  13.,  14.,  15.],
[  25.,  26.,  27.]]


Defined in src/operator/sequence_last.cc:L92

Parameters: data (Symbol) – n-dimensional input array of the form [max_sequence_length, batch_size, other_feature_dims] where n>2 sequence_length (Symbol) – vector of sequence lengths of the form [batch_size] use_sequence_length (boolean, optional, default=0) – If set to true, this layer takes in an extra input parameter sequence_length to specify variable length sequence axis (int, optional, default='0') – The sequence axis. Only values of 0 and 1 are currently supported. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.SequenceMask(data=None, sequence_length=None, use_sequence_length=_Null, value=_Null, axis=_Null, name=None, attr=None, out=None, **kwargs)

Sets all elements outside the sequence to a constant value.

This function takes an n-dimensional input array of the form [max_sequence_length, batch_size, other_feature_dims] and returns an array of the same shape.

Parameter sequence_length is used to handle variable-length sequences. sequence_length should be an input array of positive ints of dimension [batch_size]. To use this parameter, set use_sequence_length to True, otherwise each example in the batch is assumed to have the max sequence length and this operator works as the identity operator.

Example:

x = [[[  1.,   2.,   3.],
[  4.,   5.,   6.]],

[[  7.,   8.,   9.],
[ 10.,  11.,  12.]],

[[ 13.,  14.,   15.],
[ 16.,  17.,   18.]]]

// Batch 1
B1 = [[  1.,   2.,   3.],
[  7.,   8.,   9.],
[ 13.,  14.,  15.]]

// Batch 2
B2 = [[  4.,   5.,   6.],
[ 10.,  11.,  12.],
[ 16.,  17.,  18.]]

// works as identity operator when sequence_length parameter is not used
SequenceMask(x) = [[[  1.,   2.,   3.],
[  4.,   5.,   6.]],

[[  7.,   8.,   9.],
[ 10.,  11.,  12.]],

[[ 13.,  14.,   15.],
[ 16.,  17.,   18.]]]

// sequence_length [1,1] means 1 of each batch will be kept
// and other rows are masked with default mask value = 0
[[[  1.,   2.,   3.],
[  4.,   5.,   6.]],

[[  0.,   0.,   0.],
[  0.,   0.,   0.]],

[[  0.,   0.,   0.],
[  0.,   0.,   0.]]]

// sequence_length [2,3] means 2 of batch B1 and 3 of batch B2 will be kept
// and other rows are masked with value = 1
[[[  1.,   2.,   3.],
[  4.,   5.,   6.]],

[[  7.,   8.,   9.],
[  10.,  11.,  12.]],

[[   1.,   1.,   1.],
[  16.,  17.,  18.]]]


Parameters: data (Symbol) – n-dimensional input array of the form [max_sequence_length, batch_size, other_feature_dims] where n>2 sequence_length (Symbol) – vector of sequence lengths of the form [batch_size] use_sequence_length (boolean, optional, default=0) – If set to true, this layer takes in an extra input parameter sequence_length to specify variable length sequence value (float, optional, default=0) – The value to be used as a mask. axis (int, optional, default='0') – The sequence axis. Only values of 0 and 1 are currently supported. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.SequenceReverse(data=None, sequence_length=None, use_sequence_length=_Null, axis=_Null, name=None, attr=None, out=None, **kwargs)

Reverses the elements of each sequence.

This function takes an n-dimensional input array of the form [max_sequence_length, batch_size, other_feature_dims] and returns an array of the same shape.

Parameter sequence_length is used to handle variable-length sequences. sequence_length should be an input array of positive ints of dimension [batch_size]. To use this parameter, set use_sequence_length to True, otherwise each example in the batch is assumed to have the max sequence length.

Example:

x = [[[  1.,   2.,   3.],
[  4.,   5.,   6.]],

[[  7.,   8.,   9.],
[ 10.,  11.,  12.]],

[[ 13.,  14.,   15.],
[ 16.,  17.,   18.]]]

// Batch 1
B1 = [[  1.,   2.,   3.],
[  7.,   8.,   9.],
[ 13.,  14.,  15.]]

// Batch 2
B2 = [[  4.,   5.,   6.],
[ 10.,  11.,  12.],
[ 16.,  17.,  18.]]

// returns reverse sequence when sequence_length parameter is not used
SequenceReverse(x) = [[[ 13.,  14.,   15.],
[ 16.,  17.,   18.]],

[[  7.,   8.,   9.],
[ 10.,  11.,  12.]],

[[  1.,   2.,   3.],
[  4.,   5.,   6.]]]

// sequence_length [2,2] means 2 rows of
// both batch B1 and B2 will be reversed.
SequenceReverse(x, sequence_length=[2,2], use_sequence_length=True) =
[[[  7.,   8.,   9.],
[ 10.,  11.,  12.]],

[[  1.,   2.,   3.],
[  4.,   5.,   6.]],

[[ 13.,  14.,   15.],
[ 16.,  17.,   18.]]]

// sequence_length [2,3] means 2 of batch B2 and 3 of batch B3
// will be reversed.
SequenceReverse(x, sequence_length=[2,3], use_sequence_length=True) =
[[[  7.,   8.,   9.],
[ 16.,  17.,  18.]],

[[  1.,   2.,   3.],
[ 10.,  11.,  12.]],

[[ 13.,  14,   15.],
[  4.,   5.,   6.]]]


Defined in src/operator/sequence_reverse.cc:L113

Parameters: data (Symbol) – n-dimensional input array of the form [max_sequence_length, batch_size, other dims] where n>2 sequence_length (Symbol) – vector of sequence lengths of the form [batch_size] use_sequence_length (boolean, optional, default=0) – If set to true, this layer takes in an extra input parameter sequence_length to specify variable length sequence axis (int, optional, default='0') – The sequence axis. Only 0 is currently supported. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.SliceChannel(data=None, num_outputs=_Null, axis=_Null, squeeze_axis=_Null, name=None, attr=None, out=None, **kwargs)

Splits an array along a particular axis into multiple sub-arrays.

Note

SliceChannel is deprecated. Use split instead.

Note that num_outputs should evenly divide the length of the axis along which to split the array.

Example:

x  = [[[ 1.]
[ 2.]]
[[ 3.]
[ 4.]]
[[ 5.]
[ 6.]]]
x.shape = (3, 2, 1)

y = split(x, axis=1, num_outputs=2) // a list of 2 arrays with shape (3, 1, 1)
y = [[[ 1.]]
[[ 3.]]
[[ 5.]]]

[[[ 2.]]
[[ 4.]]
[[ 6.]]]

y[0].shape = (3, 1, 1)

z = split(x, axis=0, num_outputs=3) // a list of 3 arrays with shape (1, 2, 1)
z = [[[ 1.]
[ 2.]]]

[[[ 3.]
[ 4.]]]

[[[ 5.]
[ 6.]]]

z[0].shape = (1, 2, 1)


squeeze_axis=1 removes the axis with length 1 from the shapes of the output arrays. Note that setting squeeze_axis to 1 removes axis with length 1 only along the axis which it is split. Also squeeze_axis can be set to true only if input.shape[axis] == num_outputs.

Example:

z = split(x, axis=0, num_outputs=3, squeeze_axis=1) // a list of 3 arrays with shape (2, 1)
z = [[ 1.]
[ 2.]]

[[ 3.]
[ 4.]]

[[ 5.]
[ 6.]]
z[0].shape = (2 ,1 )


Defined in src/operator/slice_channel.cc:L107

Parameters: data (Symbol) – The input num_outputs (int, required) – Number of splits. Note that this should evenly divide the length of the axis. axis (int, optional, default='1') – Axis along which to split. squeeze_axis (boolean, optional, default=0) – If true, Removes the axis with length 1 from the shapes of the output arrays. Note that setting squeeze_axis to true removes axis with length 1 only along the axis which it is split. Also squeeze_axis can be set to true only if input.shape[axis] == num_outputs. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.Softmax(data=None, grad_scale=_Null, ignore_label=_Null, multi_output=_Null, use_ignore=_Null, preserve_shape=_Null, normalization=_Null, out_grad=_Null, smooth_alpha=_Null, name=None, attr=None, out=None, **kwargs)

Note

This operator has been renamed to SoftmaxOutput, which computes the gradient of cross-entropy loss w.r.t softmax output. To just compute softmax output, use the softmax operator.

Defined in src/operator/softmax_output.cc:L138

Parameters: data (Symbol) – Input array. grad_scale (float, optional, default=1) – Scales the gradient by a float factor. ignore_label (float, optional, default=-1) – The instances whose labels == ignore_label will be ignored during backward, if use_ignore is set to true). multi_output (boolean, optional, default=0) – If set to true, the softmax function will be computed along axis 1. This is applied when the shape of input array differs from the shape of label array. use_ignore (boolean, optional, default=0) – If set to true, the ignore_label value will not contribute to the backward gradient. preserve_shape (boolean, optional, default=0) – If set to true, the softmax function will be computed along the last axis (-1). normalization ({'batch', 'null', 'valid'},optional, default='null') – Normalizes the gradient. out_grad (boolean, optional, default=0) – Multiplies gradient with output gradient element-wise. smooth_alpha (float, optional, default=0) – Constant for computing a label smoothed version of cross-entropyfor the backwards pass. This constant gets subtracted from theone-hot encoding of the gold label and distributed uniformly toall other labels. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.SoftmaxActivation(data=None, mode=_Null, name=None, attr=None, out=None, **kwargs)

Applies softmax activation to input. This is intended for internal layers.

Note

This operator has been deprecated, please use softmax.

If mode = instance, this operator will compute a softmax for each instance in the batch. This is the default mode.

If mode = channel, this operator will compute a k-class softmax at each position of each instance, where k = num_channel. This mode can only be used when the input array has at least 3 dimensions. This can be used for fully convolutional network, image segmentation, etc.

Example:

>>> input_array = mx.nd.array([[3., 0.5, -0.5, 2., 7.],
>>>                            [2., -.4, 7.,   3., 0.2]])
>>> softmax_act = mx.nd.SoftmaxActivation(input_array)
>>> print softmax_act.asnumpy()
[[  1.78322066e-02   1.46375655e-03   5.38485940e-04   6.56010211e-03   9.73605454e-01]
[  6.56221947e-03   5.95310994e-04   9.73919690e-01   1.78379621e-02   1.08472735e-03]]


Defined in src/operator/nn/softmax_activation.cc:L67

Parameters: data (Symbol) – Input array to activation function. mode ({'channel', 'instance'},optional, default='instance') – Specifies how to compute the softmax. If set to instance, it computes softmax for each instance. If set to channel, It computes cross channel softmax for each position of each instance. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.SoftmaxOutput(data=None, label=None, grad_scale=_Null, ignore_label=_Null, multi_output=_Null, use_ignore=_Null, preserve_shape=_Null, normalization=_Null, out_grad=_Null, smooth_alpha=_Null, name=None, attr=None, out=None, **kwargs)

Computes the gradient of cross entropy loss with respect to softmax output.

• This operator computes the gradient in two steps. The cross entropy loss does not actually need to be computed.

• Applies softmax function on the input array.
• Computes and returns the gradient of cross entropy loss w.r.t. the softmax output.
• The softmax function, cross entropy loss and gradient is given by:

• Softmax Function:

$\text{softmax}(x)_i = \frac{exp(x_i)}{\sum_j exp(x_j)}$
• Cross Entropy Function:

$\text{CE(label, output)} = - \sum_i \text{label}_i \log(\text{output}_i)$
• The gradient of cross entropy loss w.r.t softmax output:

$\text{gradient} = \text{output} - \text{label}$
• During forward propagation, the softmax function is computed for each instance in the input array.

For general N-D input arrays with shape $$(d_1, d_2, ..., d_n)$$. The size is $$s=d_1 \cdot d_2 \cdot \cdot \cdot d_n$$. We can use the parameters preserve_shape and multi_output to specify the way to compute softmax:

• By default, preserve_shape is false. This operator will reshape the input array into a 2-D array with shape $$(d_1, \frac{s}{d_1})$$ and then compute the softmax function for each row in the reshaped array, and afterwards reshape it back to the original shape $$(d_1, d_2, ..., d_n)$$.
• If preserve_shape is true, the softmax function will be computed along the last axis (axis = -1).
• If multi_output is true, the softmax function will be computed along the second axis (axis = 1).
• During backward propagation, the gradient of cross-entropy loss w.r.t softmax output array is computed. The provided label can be a one-hot label array or a probability label array.

• If the parameter use_ignore is true, ignore_label can specify input instances with a particular label to be ignored during backward propagation. This has no effect when softmax output has same shape as label.

Example:

data = [[1,2,3,4],[2,2,2,2],[3,3,3,3],[4,4,4,4]]
label = [1,0,2,3]
ignore_label = 1
SoftmaxOutput(data=data, label = label,\
multi_output=true, use_ignore=true,\
ignore_label=ignore_label)
## forward softmax output
[[ 0.0320586   0.08714432  0.23688284  0.64391428]
[ 0.25        0.25        0.25        0.25      ]
[ 0.25        0.25        0.25        0.25      ]
[ 0.25        0.25        0.25        0.25      ]]
[[ 0.    0.    0.    0.  ]
[-0.75  0.25  0.25  0.25]
[ 0.25  0.25 -0.75  0.25]
[ 0.25  0.25  0.25 -0.75]]
## notice that the first row is all 0 because label[0] is 1, which is equal to ignore_label.

• The parameter grad_scale can be used to rescale the gradient, which is often used to give each loss function different weights.

• This operator also supports various ways to normalize the gradient by normalization, The normalization is applied if softmax output has different shape than the labels. The normalization mode can be set to the followings:

• 'null': do nothing.
• 'batch': divide the gradient by the batch size.
• 'valid': divide the gradient by the number of instances which are not ignored.

Defined in src/operator/softmax_output.cc:L123

Parameters: data (Symbol) – Input array. label (Symbol) – Ground truth label. grad_scale (float, optional, default=1) – Scales the gradient by a float factor. ignore_label (float, optional, default=-1) – The instances whose labels == ignore_label will be ignored during backward, if use_ignore is set to true). multi_output (boolean, optional, default=0) – If set to true, the softmax function will be computed along axis 1. This is applied when the shape of input array differs from the shape of label array. use_ignore (boolean, optional, default=0) – If set to true, the ignore_label value will not contribute to the backward gradient. preserve_shape (boolean, optional, default=0) – If set to true, the softmax function will be computed along the last axis (-1). normalization ({'batch', 'null', 'valid'},optional, default='null') – Normalizes the gradient. out_grad (boolean, optional, default=0) – Multiplies gradient with output gradient element-wise. smooth_alpha (float, optional, default=0) – Constant for computing a label smoothed version of cross-entropyfor the backwards pass. This constant gets subtracted from theone-hot encoding of the gold label and distributed uniformly toall other labels. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.SpatialTransformer(data=None, loc=None, target_shape=_Null, transform_type=_Null, sampler_type=_Null, name=None, attr=None, out=None, **kwargs)

Applies a spatial transformer to input feature map.

Parameters: data (Symbol) – Input data to the SpatialTransformerOp. loc (Symbol) – localisation net, the output dim should be 6 when transform_type is affine. You shold initialize the weight and bias with identity tranform. target_shape (Shape(tuple), optional, default=[0,0]) – output shape(h, w) of spatial transformer: (y, x) transform_type ({'affine'}, required) – transformation type sampler_type ({'bilinear'}, required) – sampling type name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.SwapAxis(data=None, dim1=_Null, dim2=_Null, name=None, attr=None, out=None, **kwargs)

Interchanges two axes of an array.

Examples:

 x = [[1, 2, 3]])
swapaxes(x, 0, 1) = [[ 1],
[ 2],
[ 3]]

x = [[[ 0, 1],
[ 2, 3]],
[[ 4, 5],
[ 6, 7]]]  // (2,2,2) array

swapaxes(x, 0, 2) = [[[ 0, 4],
[ 2, 6]],
[[ 1, 5],
[ 3, 7]]]


Defined in src/operator/swapaxis.cc:L70

Parameters: data (Symbol) – Input array. dim1 (int (non-negative), optional, default=0) – the first axis to be swapped. dim2 (int (non-negative), optional, default=0) – the second axis to be swapped. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.UpSampling(*data, **kwargs)

Performs nearest neighbor/bilinear up sampling to inputs. This function support variable length of positional input.

Parameters: data (Symbol[]) – Array of tensors to upsample scale (int (non-negative), required) – Up sampling scale num_filter (int (non-negative), optional, default=0) – Input filter. Only used by bilinear sample_type. sample_type ({'bilinear', 'nearest'}, required) – upsampling method multi_input_mode ({'concat', 'sum'},optional, default='concat') – How to handle multiple input. concat means concatenate upsampled images along the channel dimension. sum means add all images together, only available for nearest neighbor upsampling. workspace (long (non-negative), optional, default=512) – Tmp workspace for deconvolution (MB) name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.abs(data=None, name=None, attr=None, out=None, **kwargs)

Returns element-wise absolute value of the input.

Example:

abs([-2, 0, 3]) = [2, 0, 3]


The storage type of abs output depends upon the input storage type:

• abs(default) = default
• abs(row_sparse) = row_sparse

Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L385

Parameters: data (Symbol) – The input array. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.adam_update(weight=None, grad=None, mean=None, var=None, lr=_Null, beta1=_Null, beta2=_Null, epsilon=_Null, wd=_Null, rescale_grad=_Null, clip_gradient=_Null, name=None, attr=None, out=None, **kwargs)

Adam update consists of the following steps, where g represents gradient and m, v are 1st and 2nd order moment estimates (mean and variance).

$\begin{split}g_t = \nabla J(W_{t-1})\\ m_t = \beta_1 m_{t-1} + (1 - \beta_1) g_t\\ v_t = \beta_2 v_{t-1} + (1 - \beta_2) g_t^2\\ W_t = W_{t-1} - \alpha \frac{ m_t }{ \sqrt{ v_t } + \epsilon }\end{split}$

m = beta1*m + (1-beta1)*grad
w += - learning_rate * m / (sqrt(v) + epsilon)


If w, m and v are all of row_sparse storage type, only the row slices whose indices appear in grad.indices are updated (for w, m and v):

for row in grad.indices:
w[row] += - learning_rate * m[row] / (sqrt(v[row]) + epsilon)


Defined in src/operator/optimizer_op.cc:L383

mxnet.symbol.add_n(*args, **kwargs)

$add\_n(a_1, a_2, ..., a_n) = a_1 + a_2 + ... + a_n$

add_n is potentially more efficient than calling add by n times.

The storage type of add_n output depends on storage types of inputs

• add_n(row_sparse, row_sparse, ..) = row_sparse
• otherwise, add_n generates output with default storage

Defined in src/operator/tensor/elemwise_sum.cc:L123 This function support variable length of positional input.

Parameters: args (Symbol[]) – Positional input arguments name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.arccos(data=None, name=None, attr=None, out=None, **kwargs)

Returns element-wise inverse cosine of the input array.

The input should be in range [-1, 1]. The output is in the closed interval $$[0, \pi]$$

$arccos([-1, -.707, 0, .707, 1]) = [\pi, 3\pi/4, \pi/2, \pi/4, 0]$

The storage type of arccos output is always dense

Defined in src/operator/tensor/elemwise_unary_op_trig.cc:L123

Parameters: data (Symbol) – The input array. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.arccosh(data=None, name=None, attr=None, out=None, **kwargs)

Returns the element-wise inverse hyperbolic cosine of the input array, computed element-wise.

The storage type of arccosh output is always dense

Defined in src/operator/tensor/elemwise_unary_op_trig.cc:L264

Parameters: data (Symbol) – The input array. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.arcsin(data=None, name=None, attr=None, out=None, **kwargs)

Returns element-wise inverse sine of the input array.

The input should be in the range [-1, 1]. The output is in the closed interval of [$$-\pi/2$$, $$\pi/2$$].

$arcsin([-1, -.707, 0, .707, 1]) = [-\pi/2, -\pi/4, 0, \pi/4, \pi/2]$

The storage type of arcsin output depends upon the input storage type:

• arcsin(default) = default
• arcsin(row_sparse) = row_sparse

Defined in src/operator/tensor/elemwise_unary_op_trig.cc:L104

Parameters: data (Symbol) – The input array. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.arcsinh(data=None, name=None, attr=None, out=None, **kwargs)

Returns the element-wise inverse hyperbolic sine of the input array, computed element-wise.

The storage type of arcsinh output depends upon the input storage type:

• arcsinh(default) = default
• arcsinh(row_sparse) = row_sparse

Defined in src/operator/tensor/elemwise_unary_op_trig.cc:L250

Parameters: data (Symbol) – The input array. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.arctan(data=None, name=None, attr=None, out=None, **kwargs)

Returns element-wise inverse tangent of the input array.

The output is in the closed interval $$[-\pi/2, \pi/2]$$

$arctan([-1, 0, 1]) = [-\pi/4, 0, \pi/4]$

The storage type of arctan output depends upon the input storage type:

• arctan(default) = default
• arctan(row_sparse) = row_sparse

Defined in src/operator/tensor/elemwise_unary_op_trig.cc:L144

Parameters: data (Symbol) – The input array. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.arctanh(data=None, name=None, attr=None, out=None, **kwargs)

Returns the element-wise inverse hyperbolic tangent of the input array, computed element-wise.

The storage type of arctanh output depends upon the input storage type:

• arctanh(default) = default
• arctanh(row_sparse) = row_sparse

Defined in src/operator/tensor/elemwise_unary_op_trig.cc:L281

Parameters: data (Symbol) – The input array. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.argmax(data=None, axis=_Null, keepdims=_Null, name=None, attr=None, out=None, **kwargs)

Returns indices of the maximum values along an axis.

In the case of multiple occurrences of maximum values, the indices corresponding to the first occurrence are returned.

Examples:

x = [[ 0.,  1.,  2.],
[ 3.,  4.,  5.]]

// argmax along axis 0
argmax(x, axis=0) = [ 1.,  1.,  1.]

// argmax along axis 1
argmax(x, axis=1) = [ 2.,  2.]

// argmax along axis 1 keeping same dims as an input array
argmax(x, axis=1, keepdims=True) = [[ 2.],
[ 2.]]


Parameters: data (Symbol) – The input axis (int or None, optional, default='None') – The axis along which to perform the reduction. Negative values means indexing from right to left. Requires axis to be set as int, because global reduction is not supported yet. keepdims (boolean, optional, default=0) – If this is set to True, the reduced axis is left in the result as dimension with size one. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.argmax_channel(data=None, name=None, attr=None, out=None, **kwargs)

Returns argmax indices of each channel from the input array.

The result will be an NDArray of shape (num_channel,).

In case of multiple occurrences of the maximum values, the indices corresponding to the first occurrence are returned.

Examples:

x = [[ 0.,  1.,  2.],
[ 3.,  4.,  5.]]

argmax_channel(x) = [ 2.,  2.]


Parameters: data (Symbol) – The input array name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.argmin(data=None, axis=_Null, keepdims=_Null, name=None, attr=None, out=None, **kwargs)

Returns indices of the minimum values along an axis.

In the case of multiple occurrences of minimum values, the indices corresponding to the first occurrence are returned.

Examples:

x = [[ 0.,  1.,  2.],
[ 3.,  4.,  5.]]

// argmin along axis 0
argmin(x, axis=0) = [ 0.,  0.,  0.]

// argmin along axis 1
argmin(x, axis=1) = [ 0.,  0.]

// argmin along axis 1 keeping same dims as an input array
argmin(x, axis=1, keepdims=True) = [[ 0.],
[ 0.]]


Parameters: data (Symbol) – The input axis (int or None, optional, default='None') – The axis along which to perform the reduction. Negative values means indexing from right to left. Requires axis to be set as int, because global reduction is not supported yet. keepdims (boolean, optional, default=0) – If this is set to True, the reduced axis is left in the result as dimension with size one. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.argsort(data=None, axis=_Null, is_ascend=_Null, name=None, attr=None, out=None, **kwargs)

Returns the indices that would sort an input array along the given axis.

This function performs sorting along the given axis and returns an array of indices having same shape as an input array that index data in sorted order.

Examples:

x = [[ 0.3,  0.2,  0.4],
[ 0.1,  0.3,  0.2]]

// sort along axis -1
argsort(x) = [[ 1.,  0.,  2.],
[ 0.,  2.,  1.]]

// sort along axis 0
argsort(x, axis=0) = [[ 1.,  0.,  1.]
[ 0.,  1.,  0.]]

// flatten and then sort
argsort(x) = [ 3.,  1.,  5.,  0.,  4.,  2.]


Defined in src/operator/tensor/ordering_op.cc:L176

Parameters: data (Symbol) – The input array axis (int or None, optional, default='-1') – Axis along which to sort the input tensor. If not given, the flattened array is used. Default is -1. is_ascend (boolean, optional, default=1) – Whether to sort in ascending or descending order. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.batch_dot(lhs=None, rhs=None, transpose_a=_Null, transpose_b=_Null, name=None, attr=None, out=None, **kwargs)

Batchwise dot product.

batch_dot is used to compute dot product of x and y when x and y are data in batch, namely 3D arrays in shape of (batch_size, :, :).

For example, given x with shape (batch_size, n, m) and y with shape (batch_size, m, k), the result array will have shape (batch_size, n, k), which is computed by:

batch_dot(x,y)[i,:,:] = dot(x[i,:,:], y[i,:,:])


Defined in src/operator/tensor/dot.cc:L110

Parameters: lhs (Symbol) – The first input rhs (Symbol) – The second input transpose_a (boolean, optional, default=0) – If true then transpose the first input before dot. transpose_b (boolean, optional, default=0) – If true then transpose the second input before dot. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.batch_take(a=None, indices=None, name=None, attr=None, out=None, **kwargs)

Takes elements from a data batch.

Note

batch_take is deprecated. Use pick instead.

Given an input array of shape (d0, d1) and indices of shape (i0,), the result will be an output array of shape (i0,) with:

output[i] = input[i, indices[i]]


Examples:

x = [[ 1.,  2.],
[ 3.,  4.],
[ 5.,  6.]]

// takes elements with specified indices
batch_take(x, [0,1,0]) = [ 1.  4.  5.]


Defined in src/operator/tensor/indexing_op.cc:L422

Parameters: a (Symbol) – The input array indices (Symbol) – The index array name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.broadcast_add(lhs=None, rhs=None, name=None, attr=None, out=None, **kwargs)

Returns element-wise sum of the input arrays with broadcasting.

Example:

x = [[ 1.,  1.,  1.],
[ 1.,  1.,  1.]]

y = [[ 0.],
[ 1.]]

[ 2.,  2.,  2.]]

broadcast_plus(x, y) = [[ 1.,  1.,  1.],
[ 2.,  2.,  2.]]


Parameters: lhs (Symbol) – First input to the function rhs (Symbol) – Second input to the function name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.broadcast_axes(data=None, axis=_Null, size=_Null, name=None, attr=None, out=None, **kwargs)

Broadcasts the input array over particular axes.

Broadcasting is allowed on axes with size 1, such as from (2,1,3,1) to (2,8,3,9). Elements will be duplicated on the broadcasted axes.

Example:

// given x of shape (1,2,1)
x = [[[ 1.],
[ 2.]]]

// broadcast x on on axis 2
broadcast_axis(x, axis=2, size=3) = [[[ 1.,  1.,  1.],
[ 2.,  2.,  2.]]]
// broadcast x on on axes 0 and 2
broadcast_axis(x, axis=(0,2), size=(2,3)) = [[[ 1.,  1.,  1.],
[ 2.,  2.,  2.]],
[[ 1.,  1.,  1.],
[ 2.,  2.,  2.]]]


Parameters: data (Symbol) – The input axis (Shape(tuple), optional, default=[]) – The axes to perform the broadcasting. size (Shape(tuple), optional, default=[]) – Target sizes of the broadcasting axes. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.broadcast_axis(data=None, axis=_Null, size=_Null, name=None, attr=None, out=None, **kwargs)

Broadcasts the input array over particular axes.

Broadcasting is allowed on axes with size 1, such as from (2,1,3,1) to (2,8,3,9). Elements will be duplicated on the broadcasted axes.

Example:

// given x of shape (1,2,1)
x = [[[ 1.],
[ 2.]]]

// broadcast x on on axis 2
broadcast_axis(x, axis=2, size=3) = [[[ 1.,  1.,  1.],
[ 2.,  2.,  2.]]]
// broadcast x on on axes 0 and 2
broadcast_axis(x, axis=(0,2), size=(2,3)) = [[[ 1.,  1.,  1.],
[ 2.,  2.,  2.]],
[[ 1.,  1.,  1.],
[ 2.,  2.,  2.]]]


Parameters: data (Symbol) – The input axis (Shape(tuple), optional, default=[]) – The axes to perform the broadcasting. size (Shape(tuple), optional, default=[]) – Target sizes of the broadcasting axes. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.broadcast_div(lhs=None, rhs=None, name=None, attr=None, out=None, **kwargs)

Returns element-wise division of the input arrays with broadcasting.

Example:

x = [[ 6.,  6.,  6.],
[ 6.,  6.,  6.]]

y = [[ 2.],
[ 3.]]

broadcast_div(x, y) = [[ 3.,  3.,  3.],
[ 2.,  2.,  2.]]


Parameters: lhs (Symbol) – First input to the function rhs (Symbol) – Second input to the function name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.broadcast_equal(lhs=None, rhs=None, name=None, attr=None, out=None, **kwargs)

Returns the result of element-wise equal to (==) comparison operation with broadcasting.

Example:

x = [[ 1.,  1.,  1.],
[ 1.,  1.,  1.]]

y = [[ 0.],
[ 1.]]

broadcast_equal(x, y) = [[ 0.,  0.,  0.],
[ 1.,  1.,  1.]]


Parameters: lhs (Symbol) – First input to the function rhs (Symbol) – Second input to the function name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.broadcast_greater(lhs=None, rhs=None, name=None, attr=None, out=None, **kwargs)

Returns the result of element-wise greater than (>) comparison operation with broadcasting.

Example:

x = [[ 1.,  1.,  1.],
[ 1.,  1.,  1.]]

y = [[ 0.],
[ 1.]]

broadcast_greater(x, y) = [[ 1.,  1.,  1.],
[ 0.,  0.,  0.]]


Parameters: lhs (Symbol) – First input to the function rhs (Symbol) – Second input to the function name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.broadcast_greater_equal(lhs=None, rhs=None, name=None, attr=None, out=None, **kwargs)

Returns the result of element-wise greater than or equal to (>=) comparison operation with broadcasting.

Example:

x = [[ 1.,  1.,  1.],
[ 1.,  1.,  1.]]

y = [[ 0.],
[ 1.]]

broadcast_greater_equal(x, y) = [[ 1.,  1.,  1.],
[ 1.,  1.,  1.]]


Parameters: lhs (Symbol) – First input to the function rhs (Symbol) – Second input to the function name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.broadcast_hypot(lhs=None, rhs=None, name=None, attr=None, out=None, **kwargs)

Returns the hypotenuse of a right angled triangle, given its “legs” with broadcasting.

It is equivalent to doing $$sqrt(x_1^2 + x_2^2)$$.

Example:

x = [[ 3.,  3.,  3.]]

y = [[ 4.],
[ 4.]]

broadcast_hypot(x, y) = [[ 5.,  5.,  5.],
[ 5.,  5.,  5.]]

z = [[ 0.],
[ 4.]]

broadcast_hypot(x, z) = [[ 3.,  3.,  3.],
[ 5.,  5.,  5.]]


Parameters: lhs (Symbol) – First input to the function rhs (Symbol) – Second input to the function name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.broadcast_lesser(lhs=None, rhs=None, name=None, attr=None, out=None, **kwargs)

Returns the result of element-wise lesser than (<) comparison operation with broadcasting.

Example:

x = [[ 1.,  1.,  1.],
[ 1.,  1.,  1.]]

y = [[ 0.],
[ 1.]]

broadcast_lesser(x, y) = [[ 0.,  0.,  0.],
[ 0.,  0.,  0.]]


Parameters: lhs (Symbol) – First input to the function rhs (Symbol) – Second input to the function name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.broadcast_lesser_equal(lhs=None, rhs=None, name=None, attr=None, out=None, **kwargs)

Returns the result of element-wise lesser than or equal to (<=) comparison operation with broadcasting.

Example:

x = [[ 1.,  1.,  1.],
[ 1.,  1.,  1.]]

y = [[ 0.],
[ 1.]]

broadcast_lesser_equal(x, y) = [[ 0.,  0.,  0.],
[ 1.,  1.,  1.]]


Parameters: lhs (Symbol) – First input to the function rhs (Symbol) – Second input to the function name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.broadcast_maximum(lhs=None, rhs=None, name=None, attr=None, out=None, **kwargs)

Returns element-wise maximum of the input arrays with broadcasting.

This function compares two input arrays and returns a new array having the element-wise maxima.

Example:

x = [[ 1.,  1.,  1.],
[ 1.,  1.,  1.]]

y = [[ 0.],
[ 1.]]

broadcast_maximum(x, y) = [[ 1.,  1.,  1.],
[ 1.,  1.,  1.]]


Parameters: lhs (Symbol) – First input to the function rhs (Symbol) – Second input to the function name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.broadcast_minimum(lhs=None, rhs=None, name=None, attr=None, out=None, **kwargs)

Returns element-wise minimum of the input arrays with broadcasting.

This function compares two input arrays and returns a new array having the element-wise minima.

Example:

x = [[ 1.,  1.,  1.],
[ 1.,  1.,  1.]]

y = [[ 0.],
[ 1.]]

broadcast_maximum(x, y) = [[ 0.,  0.,  0.],
[ 1.,  1.,  1.]]


Parameters: lhs (Symbol) – First input to the function rhs (Symbol) – Second input to the function name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.broadcast_minus(lhs=None, rhs=None, name=None, attr=None, out=None, **kwargs)

Returns element-wise difference of the input arrays with broadcasting.

Example:

x = [[ 1.,  1.,  1.],
[ 1.,  1.,  1.]]

y = [[ 0.],
[ 1.]]

broadcast_sub(x, y) = [[ 1.,  1.,  1.],
[ 0.,  0.,  0.]]

broadcast_minus(x, y) = [[ 1.,  1.,  1.],
[ 0.,  0.,  0.]]


Parameters: lhs (Symbol) – First input to the function rhs (Symbol) – Second input to the function name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.broadcast_mod(lhs=None, rhs=None, name=None, attr=None, out=None, **kwargs)

Returns element-wise modulo of the input arrays with broadcasting.

Example:

x = [[ 8.,  8.,  8.],
[ 8.,  8.,  8.]]

y = [[ 2.],
[ 3.]]

broadcast_mod(x, y) = [[ 0.,  0.,  0.],
[ 2.,  2.,  2.]]


Parameters: lhs (Symbol) – First input to the function rhs (Symbol) – Second input to the function name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.broadcast_mul(lhs=None, rhs=None, name=None, attr=None, out=None, **kwargs)

Returns element-wise product of the input arrays with broadcasting.

Example:

x = [[ 1.,  1.,  1.],
[ 1.,  1.,  1.]]

y = [[ 0.],
[ 1.]]

broadcast_mul(x, y) = [[ 0.,  0.,  0.],
[ 1.,  1.,  1.]]


Parameters: lhs (Symbol) – First input to the function rhs (Symbol) – Second input to the function name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.broadcast_not_equal(lhs=None, rhs=None, name=None, attr=None, out=None, **kwargs)

Returns the result of element-wise not equal to (!=) comparison operation with broadcasting.

Example:

x = [[ 1.,  1.,  1.],
[ 1.,  1.,  1.]]

y = [[ 0.],
[ 1.]]

broadcast_not_equal(x, y) = [[ 1.,  1.,  1.],
[ 0.,  0.,  0.]]


Parameters: lhs (Symbol) – First input to the function rhs (Symbol) – Second input to the function name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.broadcast_plus(lhs=None, rhs=None, name=None, attr=None, out=None, **kwargs)

Returns element-wise sum of the input arrays with broadcasting.

Example:

x = [[ 1.,  1.,  1.],
[ 1.,  1.,  1.]]

y = [[ 0.],
[ 1.]]

[ 2.,  2.,  2.]]

broadcast_plus(x, y) = [[ 1.,  1.,  1.],
[ 2.,  2.,  2.]]


Parameters: lhs (Symbol) – First input to the function rhs (Symbol) – Second input to the function name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.broadcast_power(lhs=None, rhs=None, name=None, attr=None, out=None, **kwargs)

Returns result of first array elements raised to powers from second array, element-wise with broadcasting.

Example:

x = [[ 1.,  1.,  1.],
[ 1.,  1.,  1.]]

y = [[ 0.],
[ 1.]]

broadcast_power(x, y) = [[ 2.,  2.,  2.],
[ 4.,  4.,  4.]]


Parameters: lhs (Symbol) – First input to the function rhs (Symbol) – Second input to the function name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.broadcast_sub(lhs=None, rhs=None, name=None, attr=None, out=None, **kwargs)

Returns element-wise difference of the input arrays with broadcasting.

Example:

x = [[ 1.,  1.,  1.],
[ 1.,  1.,  1.]]

y = [[ 0.],
[ 1.]]

broadcast_sub(x, y) = [[ 1.,  1.,  1.],
[ 0.,  0.,  0.]]

broadcast_minus(x, y) = [[ 1.,  1.,  1.],
[ 0.,  0.,  0.]]


Parameters: lhs (Symbol) – First input to the function rhs (Symbol) – Second input to the function name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.broadcast_to(data=None, shape=_Null, name=None, attr=None, out=None, **kwargs)

Broadcasts the input array to a new shape.

Broadcasting is a mechanism that allows NDArrays to perform arithmetic operations with arrays of different shapes efficiently without creating multiple copies of arrays. Also see, Broadcasting for more explanation.

Broadcasting is allowed on axes with size 1, such as from (2,1,3,1) to (2,8,3,9). Elements will be duplicated on the broadcasted axes.

For example:

broadcast_to([[1,2,3]], shape=(2,3)) = [[ 1.,  2.,  3.],
[ 1.,  2.,  3.]])


The dimension which you do not want to change can also be kept as 0 which means copy the original value. So with shape=(2,0), we will obtain the same result as in the above example.

Parameters: data (Symbol) – The input shape (Shape(tuple), optional, default=[]) – The shape of the desired array. We can set the dim to zero if it’s same as the original. E.g A = broadcast_to(B, shape=(10, 0, 0)) has the same meaning as A = broadcast_axis(B, axis=0, size=10). name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.cast(data=None, dtype=_Null, name=None, attr=None, out=None, **kwargs)

Casts all elements of the input to a new type.

Note

Cast is deprecated. Use cast instead.

Example:

cast([0.9, 1.3], dtype='int32') = [0, 1]
cast([1e20, 11.1], dtype='float16') = [inf, 11.09375]
cast([300, 11.1, 10.9, -1, -3], dtype='uint8') = [44, 11, 10, 255, 253]


Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L310

Parameters: data (Symbol) – The input. dtype ({'float16', 'float32', 'float64', 'int32', 'uint8'}, required) – Output data type. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.cast_storage(data=None, stype=_Null, name=None, attr=None, out=None, **kwargs)

Casts tensor storage type to the new type.

When an NDArray with default storage type is cast to csr or row_sparse storage, the result is compact, which means:

• for csr, zero values will not be retained
• for row_sparse, row slices of all zeros will not be retained

The storage type of cast_storage output depends on stype parameter:

• cast_storage(csr, ‘default’) = default
• cast_storage(row_sparse, ‘default’) = default
• cast_storage(default, ‘csr’) = csr
• cast_storage(default, ‘row_sparse’) = row_sparse

Example:

dense = [[ 0.,  1.,  0.],
[ 2.,  0.,  3.],
[ 0.,  0.,  0.],
[ 0.,  0.,  0.]]

# cast to row_sparse storage type
rsp = cast_storage(dense, 'row_sparse')
rsp.indices = [0, 1]
rsp.values = [[ 0.,  1.,  0.],
[ 2.,  0.,  3.]]

# cast to csr storage type
csr = cast_storage(dense, 'csr')
csr.indices = [1, 0, 2]
csr.values = [ 1.,  2.,  3.]
csr.indptr = [0, 1, 3, 3, 3]


Defined in src/operator/tensor/cast_storage.cc:L69

Parameters: data (Symbol) – The input. stype ({'csr', 'default', 'row_sparse'}, required) – Output storage type. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.cbrt(data=None, name=None, attr=None, out=None, **kwargs)

Returns element-wise cube-root value of the input.

$cbrt(x) = \sqrt[3]{x}$

Example:

cbrt([1, 8, -125]) = [1, 2, -5]


Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L601

Parameters: data (Symbol) – The input array. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.ceil(data=None, name=None, attr=None, out=None, **kwargs)

Returns element-wise ceiling of the input.

The ceil of the scalar x is the smallest integer i, such that i >= x.

Example:

ceil([-2.1, -1.9, 1.5, 1.9, 2.1]) = [-2., -1.,  2.,  2.,  3.]


The storage type of ceil output depends upon the input storage type:

• ceil(default) = default
• ceil(row_sparse) = row_sparse

Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L463

Parameters: data (Symbol) – The input array. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.choose_element_0index(lhs=None, rhs=None, name=None, attr=None, out=None, **kwargs)

Choose one element from each line(row for python, column for R/Julia) in lhs according to index indicated by rhs. This function assume rhs uses 0-based index.

Parameters: lhs (NDArray) – Left operand to the function. rhs (NDArray) – Right operand to the function. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.clip(data=None, a_min=_Null, a_max=_Null, name=None, attr=None, out=None, **kwargs)

Clips (limits) the values in an array.

Given an interval, values outside the interval are clipped to the interval edges. Clipping x between a_min and a_x would be:

clip(x, a_min, a_max) = max(min(x, a_max), a_min))


Example:

x = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

clip(x,1,8) = [ 1.,  1.,  2.,  3.,  4.,  5.,  6.,  7.,  8.,  8.]


The storage type of clip output depends on storage types of inputs and the a_min, a_max parameter values:

• clip(default) = default
• clip(row_sparse, a_min <= 0, a_max >= 0) = row_sparse
• clip(csr, a_min <= 0, a_max >= 0) = csr
• clip(row_sparse, a_min < 0, a_max < 0) = default
• clip(row_sparse, a_min > 0, a_max > 0) = default
• clip(csr, a_min < 0, a_max < 0) = csr
• clip(csr, a_min > 0, a_max > 0) = csr

Defined in src/operator/tensor/matrix_op.cc:L486

Parameters: data (Symbol) – Input array. a_min (float, required) – Minimum value a_max (float, required) – Maximum value name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.concat(*data, **kwargs)

Joins input arrays along a given axis.

Note

Concat is deprecated. Use concat instead.

The dimensions of the input arrays should be the same except the axis along which they will be concatenated. The dimension of the output array along the concatenated axis will be equal to the sum of the corresponding dimensions of the input arrays.

Example:

x = [[1,1],[2,2]]
y = [[3,3],[4,4],[5,5]]
z = [[6,6], [7,7],[8,8]]

concat(x,y,z,dim=0) = [[ 1.,  1.],
[ 2.,  2.],
[ 3.,  3.],
[ 4.,  4.],
[ 5.,  5.],
[ 6.,  6.],
[ 7.,  7.],
[ 8.,  8.]]

Note that you cannot concat x,y,z along dimension 1 since dimension
0 is not the same for all the input arrays.

concat(y,z,dim=1) = [[ 3.,  3.,  6.,  6.],
[ 4.,  4.,  7.,  7.],
[ 5.,  5.,  8.,  8.]]


Defined in src/operator/concat.cc:L104 This function support variable length of positional input.

Parameters: data (Symbol[]) – List of arrays to concatenate dim (int, optional, default='1') – the dimension to be concated. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.cos(data=None, name=None, attr=None, out=None, **kwargs)

Computes the element-wise cosine of the input array.

The input should be in radians ($$2\pi$$ rad equals 360 degrees).

$cos([0, \pi/4, \pi/2]) = [1, 0.707, 0]$

The storage type of cos output is always dense

Defined in src/operator/tensor/elemwise_unary_op_trig.cc:L63

Parameters: data (Symbol) – The input array. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.cosh(data=None, name=None, attr=None, out=None, **kwargs)

Returns the hyperbolic cosine of the input array, computed element-wise.

$cosh(x) = 0.5\times(exp(x) + exp(-x))$

The storage type of cosh output is always dense

Defined in src/operator/tensor/elemwise_unary_op_trig.cc:L216

Parameters: data (Symbol) – The input array. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.crop(data=None, begin=_Null, end=_Null, step=_Null, name=None, attr=None, out=None, **kwargs)

Slices a region of the array.

Note

crop is deprecated. Use slice instead.

This function returns a sliced array between the indices given by begin and end with the corresponding step.

For an input array of shape=(d_0, d_1, ..., d_n-1), slice operation with begin=(b_0, b_1...b_m-1), end=(e_0, e_1, ..., e_m-1), and step=(s_0, s_1, ..., s_m-1), where m <= n, results in an array with the shape (|e_0-b_0|/|s_0|, ..., |e_m-1-b_m-1|/|s_m-1|, d_m, ..., d_n-1).

The resulting array’s k-th dimension contains elements from the k-th dimension of the input array starting from index b_k (inclusive) with step s_k until reaching e_k (exclusive).

If the k-th elements are None in the sequence of begin, end, and step, the following rule will be used to set default values. If s_k is None, set s_k=1. If s_k > 0, set b_k=0, e_k=d_k; else, set b_k=d_k-1, e_k=-1.

The storage type of slice output depends on storage types of inputs

• slice(csr) = csr
• otherwise, slice generates output with default storage

Note

When input data storage type is csr, it only supports

step=(), or step=(None,), or step=(1,) to generate a csr output. For other step parameter values, it falls back to slicing a dense tensor.

Example:

x = [[  1.,   2.,   3.,   4.],
[  5.,   6.,   7.,   8.],
[  9.,  10.,  11.,  12.]]

slice(x, begin=(0,1), end=(2,4)) = [[ 2.,  3.,  4.],
[ 6.,  7.,  8.]]
slice(x, begin=(None, 0), end=(None, 3), step=(-1, 2)) = [[9., 11.],
[5.,  7.],
[1.,  3.]]


Defined in src/operator/tensor/matrix_op.cc:L355

Parameters: data (Symbol) – Source input begin (Shape(tuple), required) – starting indices for the slice operation, supports negative indices. end (Shape(tuple), required) – ending indices for the slice operation, supports negative indices. step (Shape(tuple), optional, default=[]) – step for the slice operation, supports negative values. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.degrees(data=None, name=None, attr=None, out=None, **kwargs)

Converts each element of the input array from radians to degrees.

$degrees([0, \pi/2, \pi, 3\pi/2, 2\pi]) = [0, 90, 180, 270, 360]$

The storage type of degrees output depends upon the input storage type:

• degrees(default) = default
• degrees(row_sparse) = row_sparse

Defined in src/operator/tensor/elemwise_unary_op_trig.cc:L163

Parameters: data (Symbol) – The input array. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.dot(lhs=None, rhs=None, transpose_a=_Null, transpose_b=_Null, name=None, attr=None, out=None, **kwargs)

Dot product of two arrays.

dot‘s behavior depends on the input array dimensions:

• 1-D arrays: inner product of vectors

• 2-D arrays: matrix multiplication

• N-D arrays: a sum product over the last axis of the first input and the first axis of the second input

For example, given 3-D x with shape (n,m,k) and y with shape (k,r,s), the result array will have shape (n,m,r,s). It is computed by:

dot(x,y)[i,j,a,b] = sum(x[i,j,:]*y[:,a,b])


Example:

x = reshape([0,1,2,3,4,5,6,7], shape=(2,2,2))
y = reshape([7,6,5,4,3,2,1,0], shape=(2,2,2))
dot(x,y)[0,0,1,1] = 0
sum(x[0,0,:]*y[:,1,1]) = 0


The storage type of dot output depends on storage types of inputs and transpose options:

• dot(csr, default) = default
• dot(csr.T, default) = row_sparse
• dot(csr, row_sparse) = default
• dot(default, csr) = csr
• otherwise, dot generates output with default storage

Defined in src/operator/tensor/dot.cc:L62

Parameters: lhs (Symbol) – The first input rhs (Symbol) – The second input transpose_a (boolean, optional, default=0) – If true then transpose the first input before dot. transpose_b (boolean, optional, default=0) – If true then transpose the second input before dot. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.elemwise_add(lhs=None, rhs=None, name=None, attr=None, out=None, **kwargs)

The storage type of elemwise_add output depends on storage types of inputs

• otherwise, elemwise_add generates output with default storage
Parameters: lhs (Symbol) – first input rhs (Symbol) – second input name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.elemwise_div(lhs=None, rhs=None, name=None, attr=None, out=None, **kwargs)

Divides arguments element-wise.

The storage type of elemwise_div output is always dense

Parameters: lhs (Symbol) – first input rhs (Symbol) – second input name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.elemwise_mul(lhs=None, rhs=None, name=None, attr=None, out=None, **kwargs)

Multiplies arguments element-wise.

The storage type of elemwise_mul output depends on storage types of inputs

• elemwise_mul(default, default) = default
• elemwise_mul(row_sparse, row_sparse) = row_sparse
• elemwise_mul(default, row_sparse) = default
• elemwise_mul(row_sparse, default) = default
• elemwise_mul(csr, csr) = csr
• otherwise, elemwise_mul generates output with default storage
Parameters: lhs (Symbol) – first input rhs (Symbol) – second input name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.elemwise_sub(lhs=None, rhs=None, name=None, attr=None, out=None, **kwargs)

Subtracts arguments element-wise.

The storage type of elemwise_sub output depends on storage types of inputs

• elemwise_sub(row_sparse, row_sparse) = row_sparse
• elemwise_sub(csr, csr) = csr
• otherwise, elemwise_sub generates output with default storage
Parameters: lhs (Symbol) – first input rhs (Symbol) – second input name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.exp(data=None, name=None, attr=None, out=None, **kwargs)

Returns element-wise exponential value of the input.

$exp(x) = e^x \approx 2.718^x$

Example:

exp([0, 1, 2]) = [1., 2.71828175, 7.38905621]


The storage type of exp output is always dense

Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L641

Parameters: data (Symbol) – The input array. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.expand_dims(data=None, axis=_Null, name=None, attr=None, out=None, **kwargs)

Inserts a new axis of size 1 into the array shape

For example, given x with shape (2,3,4), then expand_dims(x, axis=1) will return a new array with shape (2,1,3,4).

Defined in src/operator/tensor/matrix_op.cc:L289

Parameters: data (Symbol) – Source input axis (int, required) – Position where new axis is to be inserted. Suppose that the input NDArray‘s dimension is ndim, the range of the inserted axis is [-ndim, ndim] name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.expm1(data=None, name=None, attr=None, out=None, **kwargs)

Returns exp(x) - 1 computed element-wise on the input.

This function provides greater precision than exp(x) - 1 for small values of x.

The storage type of expm1 output depends upon the input storage type:

• expm1(default) = default
• expm1(row_sparse) = row_sparse

Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L720

Parameters: data (Symbol) – The input array. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.fill_element_0index(lhs=None, mhs=None, rhs=None, name=None, attr=None, out=None, **kwargs)

Fill one element of each line(row for python, column for R/Julia) in lhs according to index indicated by rhs and values indicated by mhs. This function assume rhs uses 0-based index.

Parameters: lhs (NDArray) – Left operand to the function. mhs (NDArray) – Middle operand to the function. rhs (NDArray) – Right operand to the function. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.fix(data=None, name=None, attr=None, out=None, **kwargs)

Returns element-wise rounded value to the nearest integer towards zero of the input.

Example:

fix([-2.1, -1.9, 1.9, 2.1]) = [-2., -1.,  1., 2.]


The storage type of fix output depends upon the input storage type:

• fix(default) = default
• fix(row_sparse) = row_sparse

Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L520

Parameters: data (Symbol) – The input array. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.flatten(data=None, name=None, attr=None, out=None, **kwargs)

Flattens the input array into a 2-D array by collapsing the higher dimensions.

Note

Flatten is deprecated. Use flatten instead.

For an input array with shape (d1, d2, ..., dk), flatten operation reshapes the input array into an output array of shape (d1, d2*...*dk).

Example:

x = [[
[1,2,3],
[4,5,6],
[7,8,9]
],
[    [1,2,3],
[4,5,6],
[7,8,9]
]],

flatten(x) = [[ 1.,  2.,  3.,  4.,  5.,  6.,  7.,  8.,  9.],
[ 1.,  2.,  3.,  4.,  5.,  6.,  7.,  8.,  9.]]


Defined in src/operator/tensor/matrix_op.cc:L208

Parameters: data (Symbol) – Input array. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.flip(data=None, axis=_Null, name=None, attr=None, out=None, **kwargs)

Reverses the order of elements along given axis while preserving array shape.

Note: reverse and flip are equivalent. We use reverse in the following examples.

Examples:

x = [[ 0.,  1.,  2.,  3.,  4.],
[ 5.,  6.,  7.,  8.,  9.]]

reverse(x, axis=0) = [[ 5.,  6.,  7.,  8.,  9.],
[ 0.,  1.,  2.,  3.,  4.]]

reverse(x, axis=1) = [[ 4.,  3.,  2.,  1.,  0.],
[ 9.,  8.,  7.,  6.,  5.]]


Defined in src/operator/tensor/matrix_op.cc:L661

Parameters: data (Symbol) – Input data array axis (Shape(tuple), required) – The axis which to reverse elements. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.floor(data=None, name=None, attr=None, out=None, **kwargs)

Returns element-wise floor of the input.

The floor of the scalar x is the largest integer i, such that i <= x.

Example:

floor([-2.1, -1.9, 1.5, 1.9, 2.1]) = [-3., -2.,  1.,  1.,  2.]


The storage type of floor output depends upon the input storage type:

• floor(default) = default
• floor(row_sparse) = row_sparse

Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L482

Parameters: data (Symbol) – The input array. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.ftml_update(weight=None, grad=None, d=None, v=None, z=None, lr=_Null, beta1=_Null, beta2=_Null, epsilon=_Null, wd=_Null, rescale_grad=_Null, clip_gradient=_Null, name=None, attr=None, out=None, **kwargs)

The FTML optimizer described in FTML - Follow the Moving Leader in Deep Learning, available at http://proceedings.mlr.press/v70/zheng17a/zheng17a.pdf.

$\begin{split}g_t = \nabla J(W_{t-1})\\ v_t = \beta_2 v_{t-1} + (1 - \beta_2) g_t^2\\ d_t = \frac{ (1 - \beta_1^t) }{ \eta_t } (\sqrt{ \frac{ v_t }{ 1 - \beta_2^t } } + \epsilon) \sigma_t = d_t - \beta_1 d_{t-1} z_t = \beta_1 z_{ t-1 } + (1 - \beta_1^t) g_t - \sigma_t W_{t-1} W_t = - \frac{ z_t }{ d_t }\end{split}$

Defined in src/operator/optimizer_op.cc:L336

Parameters: weight (Symbol) – Weight grad (Symbol) – Gradient d (Symbol) – Internal state d_t v (Symbol) – Internal state v_t z (Symbol) – Internal state z_t lr (float, required) – Learning rate beta1 (float, optional, default=0.9) – The decay rate for the 1st moment estimates. beta2 (float, optional, default=0.999) – The decay rate for the 2nd moment estimates. epsilon (float, optional, default=1e-08) – A small constant for numerical stability. wd (float, optional, default=0) – Weight decay augments the objective function with a regularization term that penalizes large weights. The penalty scales with the square of the magnitude of each weight. rescale_grad (float, optional, default=1) – Rescale gradient to grad = rescale_grad*grad. clip_gradient (float, optional, default=-1) – Clip gradient to the range of [-clip_gradient, clip_gradient] If clip_gradient <= 0, gradient clipping is turned off. grad = max(min(grad, clip_gradient), -clip_gradient). name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.ftrl_update(weight=None, grad=None, z=None, n=None, lr=_Null, lamda1=_Null, beta=_Null, wd=_Null, rescale_grad=_Null, clip_gradient=_Null, name=None, attr=None, out=None, **kwargs)

Update function for Ftrl optimizer. Referenced from Ad Click Prediction: a View from the Trenches, available at http://dl.acm.org/citation.cfm?id=2488200.

rescaled_grad = clip(grad * rescale_grad, clip_gradient)
z += rescaled_grad - (sqrt(n + rescaled_grad**2) - sqrt(n)) * weight / learning_rate
w = (sign(z) * lamda1 - z) / ((beta + sqrt(n)) / learning_rate + wd) * (abs(z) > lamda1)


If w, z and n are all of row_sparse storage type, only the row slices whose indices appear in grad.indices are updated (for w, z and n):

for row in grad.indices:
z[row] += rescaled_grad[row] - (sqrt(n[row] + rescaled_grad[row]**2) - sqrt(n[row])) * weight[row] / learning_rate
w[row] = (sign(z[row]) * lamda1 - z[row]) / ((beta + sqrt(n[row])) / learning_rate + wd) * (abs(z[row]) > lamda1)


Defined in src/operator/optimizer_op.cc:L520

mxnet.symbol.gamma(data=None, name=None, attr=None, out=None, **kwargs)

Returns the gamma function (extension of the factorial function to the reals), computed element-wise on the input array.

The storage type of gamma output is always dense

Parameters: data (Symbol) – The input array. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.gammaln(data=None, name=None, attr=None, out=None, **kwargs)

Returns element-wise log of the absolute value of the gamma function of the input.

The storage type of gammaln output is always dense

Parameters: data (Symbol) – The input array. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.gather_nd(data=None, indices=None, name=None, attr=None, out=None, **kwargs)

Gather elements or slices from data and store to a tensor whose shape is defined by indices.

Given data with shape (X_0, X_1, ..., X_{N-1}) and indices with shape (M, Y_0, ..., Y_{K-1}), the output will have shape (Y_0, ..., Y_{K-1}, X_M, ..., X_{N-1}), where M <= N. If M == N, output shape will simply be (Y_0, ..., Y_{K-1}).

The elements in output is defined as follows:

output[y_0, ..., y_{K-1}, x_M, ..., x_{N-1}] = data[indices[0, y_0, ..., y_{K-1}],
...,
indices[M-1, y_0, ..., y_{K-1}],
x_M, ..., x_{N-1}]


Examples:

data = [[0, 1], [2, 3]]
indices = [[1, 1, 0], [0, 1, 0]]
gather_nd(data, indices) = [2, 3, 0]

Parameters: data (Symbol) – data indices (Symbol) – indices name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.identity(data=None, name=None, attr=None, out=None, **kwargs)

Returns a copy of the input.

From:src/operator/tensor/elemwise_unary_op_basic.cc:111

Parameters: data (Symbol) – The input array. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.khatri_rao(*args, **kwargs)

Computes the Khatri-Rao product of the input matrices.

Given a collection of $$n$$ input matrices,

$A_1 \in \mathbb{R}^{M_1 \times M}, \ldots, A_n \in \mathbb{R}^{M_n \times N},$

the (column-wise) Khatri-Rao product is defined as the matrix,

$X = A_1 \otimes \cdots \otimes A_n \in \mathbb{R}^{(M_1 \cdots M_n) \times N},$

where the $$kth column is equal to the column-wise outer product :math:{A_1}_k \otimes \cdots \otimes {A_n}_k$$ where $${A_i}_k$$ is the kth column of the ith matrix.

Example:

>>> A = mx.nd.array([[1, -1],
>>>                  [2, -3]])
>>> B = mx.nd.array([[1, 4],
>>>                  [2, 5],
>>>                  [3, 6]])
>>> C = mx.nd.khatri_rao(A, B)
>>> print(C.asnumpy())
[[  1.  -4.]
[  2.  -5.]
[  3.  -6.]
[  2. -12.]
[  4. -15.]
[  6. -18.]]


Defined in src/operator/contrib/krprod.cc:L108 This function support variable length of positional input.

Parameters: args (Symbol[]) – Positional input matrices name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.linalg_gelqf(A=None, name=None, attr=None, out=None, **kwargs)

LQ factorization for general matrix. Input is a tensor A of dimension n >= 2.

If n=2, we compute the LQ factorization (LAPACK gelqf, followed by orglq). A must have shape (x, y) with x <= y, and must have full rank =x. The LQ factorization consists of L with shape (x, x) and Q with shape (x, y), so that:

A = L * Q

Here, L is lower triangular (upper triangle equal to zero) with nonzero diagonal, and Q is row-orthonormal, meaning that

Q * QT

is equal to the identity matrix of shape (x, x).

If n>2, gelqf is performed separately on the trailing two dimensions for all inputs (batch mode).

Note

The operator supports float32 and float64 data types only.

Examples:

// Single LQ factorization
A = [[1., 2., 3.], [4., 5., 6.]]
Q, L = gelqf(A)
Q = [[-0.26726124, -0.53452248, -0.80178373],
[0.87287156, 0.21821789, -0.43643578]]
L = [[-3.74165739, 0.],
[-8.55235974, 1.96396101]]

// Batch LQ factorization
A = [[[1., 2., 3.], [4., 5., 6.]],
[[7., 8., 9.], [10., 11., 12.]]]
Q, L = gelqf(A)
Q = [[[-0.26726124, -0.53452248, -0.80178373],
[0.87287156, 0.21821789, -0.43643578]],
[[-0.50257071, -0.57436653, -0.64616234],
[0.7620735, 0.05862104, -0.64483142]]]
L = [[[-3.74165739, 0.],
[-8.55235974, 1.96396101]],
[[-13.92838828, 0.],
[-19.09768702, 0.52758934]]]


Defined in src/operator/tensor/la_op.cc:L529

Parameters: A (Symbol) – Tensor of input matrices to be factorized name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.linalg_gemm(A=None, B=None, C=None, transpose_a=_Null, transpose_b=_Null, alpha=_Null, beta=_Null, name=None, attr=None, out=None, **kwargs)

Performs general matrix multiplication and accumulation. Input are tensors A, B, C, each of dimension n >= 2 and having the same shape on the leading n-2 dimensions.

If n=2, the BLAS3 function gemm is performed:

out = alpha * op(A) * op(B) + beta * C

Here, alpha and beta are scalar parameters, and op() is either the identity or matrix transposition (depending on transpose_a, transpose_b).

If n>2, gemm is performed separately on the trailing two dimensions for all inputs (batch mode).

Note

The operator supports float32 and float64 data types only.

Examples:

// Single matrix multiply-add
A = [[1.0, 1.0], [1.0, 1.0]]
B = [[1.0, 1.0], [1.0, 1.0], [1.0, 1.0]]
C = [[1.0, 1.0, 1.0], [1.0, 1.0, 1.0]]
gemm(A, B, C, transpose_b=True, alpha=2.0, beta=10.0)
= [[14.0, 14.0, 14.0], [14.0, 14.0, 14.0]]

A = [[[1.0, 1.0]], [[0.1, 0.1]]]
B = [[[1.0, 1.0]], [[0.1, 0.1]]]
C = [[[10.0]], [[0.01]]]
gemm(A, B, C, transpose_b=True, alpha=2.0 , beta=10.0)
= [[[104.0]], [[0.14]]]


Defined in src/operator/tensor/la_op.cc:L69

Parameters: A (Symbol) – Tensor of input matrices B (Symbol) – Tensor of input matrices C (Symbol) – Tensor of input matrices transpose_a (boolean, optional, default=0) – Multiply with transposed of first input (A). transpose_b (boolean, optional, default=0) – Multiply with transposed of second input (B). alpha (double, optional, default=1) – Scalar factor multiplied with A*B. beta (double, optional, default=1) – Scalar factor multiplied with C. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.linalg_gemm2(A=None, B=None, transpose_a=_Null, transpose_b=_Null, alpha=_Null, name=None, attr=None, out=None, **kwargs)

Performs general matrix multiplication. Input are tensors A, B, each of dimension n >= 2 and having the same shape on the leading n-2 dimensions.

If n=2, the BLAS3 function gemm is performed:

out = alpha * op(A) * op(B)

Here alpha is a scalar parameter and op() is either the identity or the matrix transposition (depending on transpose_a, transpose_b).

If n>2, gemm is performed separately on the trailing two dimensions for all inputs (batch mode).

Note

The operator supports float32 and float64 data types only.

Examples:

// Single matrix multiply
A = [[1.0, 1.0], [1.0, 1.0]]
B = [[1.0, 1.0], [1.0, 1.0], [1.0, 1.0]]
gemm2(A, B, transpose_b=True, alpha=2.0)
= [[4.0, 4.0, 4.0], [4.0, 4.0, 4.0]]

// Batch matrix multiply
A = [[[1.0, 1.0]], [[0.1, 0.1]]]
B = [[[1.0, 1.0]], [[0.1, 0.1]]]
gemm2(A, B, transpose_b=True, alpha=2.0)
= [[[4.0]], [[0.04 ]]]


Defined in src/operator/tensor/la_op.cc:L128

Parameters: A (Symbol) – Tensor of input matrices B (Symbol) – Tensor of input matrices transpose_a (boolean, optional, default=0) – Multiply with transposed of first input (A). transpose_b (boolean, optional, default=0) – Multiply with transposed of second input (B). alpha (double, optional, default=1) – Scalar factor multiplied with A*B. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.linalg_potrf(A=None, name=None, attr=None, out=None, **kwargs)

Performs Cholesky factorization of a symmetric positive-definite matrix. Input is a tensor A of dimension n >= 2.

If n=2, the Cholesky factor L of the symmetric, positive definite matrix A is computed. L is lower triangular (entries of upper triangle are all zero), has positive diagonal entries, and:

A = L * LT

If n>2, potrf is performed separately on the trailing two dimensions for all inputs (batch mode).

Note

The operator supports float32 and float64 data types only.

Examples:

// Single matrix factorization
A = [[4.0, 1.0], [1.0, 4.25]]
potrf(A) = [[2.0, 0], [0.5, 2.0]]

// Batch matrix factorization
A = [[[4.0, 1.0], [1.0, 4.25]], [[16.0, 4.0], [4.0, 17.0]]]
potrf(A) = [[[2.0, 0], [0.5, 2.0]], [[4.0, 0], [1.0, 4.0]]]


Defined in src/operator/tensor/la_op.cc:L178

Parameters: A (Symbol) – Tensor of input matrices to be decomposed name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.linalg_potri(A=None, name=None, attr=None, out=None, **kwargs)

Performs matrix inversion from a Cholesky factorization. Input is a tensor A of dimension n >= 2.

If n=2, A is a lower triangular matrix (entries of upper triangle are all zero) with positive diagonal. We compute:

out = A-T * A-1

In other words, if A is the Cholesky factor of a symmetric positive definite matrix B (obtained by potrf), then

out = B-1

If n>2, potri is performed separately on the trailing two dimensions for all inputs (batch mode).

Note

The operator supports float32 and float64 data types only.

Note

Use this operator only if you are certain you need the inverse of B, and cannot use the Cholesky factor A (potrf), together with backsubstitution (trsm). The latter is numerically much safer, and also cheaper.

Examples:

// Single matrix inverse
A = [[2.0, 0], [0.5, 2.0]]
potri(A) = [[0.26563, -0.0625], [-0.0625, 0.25]]

// Batch matrix inverse
A = [[[2.0, 0], [0.5, 2.0]], [[4.0, 0], [1.0, 4.0]]]
potri(A) = [[[0.26563, -0.0625], [-0.0625, 0.25]],
[[0.06641, -0.01562], [-0.01562, 0,0625]]]


Defined in src/operator/tensor/la_op.cc:L236

Parameters: A (Symbol) – Tensor of lower triangular matrices name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.linalg_sumlogdiag(A=None, name=None, attr=None, out=None, **kwargs)

Computes the sum of the logarithms of the diagonal elements of a square matrix. Input is a tensor A of dimension n >= 2.

If n=2, A must be square with positive diagonal entries. We sum the natural logarithms of the diagonal elements, the result has shape (1,).

If n>2, sumlogdiag is performed separately on the trailing two dimensions for all inputs (batch mode).

Note

The operator supports float32 and float64 data types only.

Examples:

// Single matrix reduction
A = [[1.0, 1.0], [1.0, 7.0]]
sumlogdiag(A) = [1.9459]

// Batch matrix reduction
A = [[[1.0, 1.0], [1.0, 7.0]], [[3.0, 0], [0, 17.0]]]
sumlogdiag(A) = [1.9459, 3.9318]


Defined in src/operator/tensor/la_op.cc:L405

Parameters: A (Symbol) – Tensor of square matrices name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.linalg_syrk(A=None, transpose=_Null, alpha=_Null, name=None, attr=None, out=None, **kwargs)

Multiplication of matrix with its transpose. Input is a tensor A of dimension n >= 2.

If n=2, the operator performs the BLAS3 function syrk:

out = alpha * A * AT

if transpose=False, or

out = alpha * AT * A

if transpose=True.

If n>2, syrk is performed separately on the trailing two dimensions for all inputs (batch mode).

Note

The operator supports float32 and float64 data types only.

Examples:

// Single matrix multiply
A = [[1., 2., 3.], [4., 5., 6.]]
syrk(A, alpha=1., transpose=False)
= [[14., 32.],
[32., 77.]]
syrk(A, alpha=1., transpose=True)
= [[17., 22., 27.],
[22., 29., 36.],
[27., 36., 45.]]

// Batch matrix multiply
A = [[[1., 1.]], [[0.1, 0.1]]]
syrk(A, alpha=2., transpose=False) = [[[4.]], [[0.04]]]


Defined in src/operator/tensor/la_op.cc:L461

Parameters: A (Symbol) – Tensor of input matrices transpose (boolean, optional, default=0) – Use transpose of input matrix. alpha (double, optional, default=1) – Scalar factor to be applied to the result. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.linalg_trmm(A=None, B=None, transpose=_Null, rightside=_Null, alpha=_Null, name=None, attr=None, out=None, **kwargs)

Performs multiplication with a lower triangular matrix. Input are tensors A, B, each of dimension n >= 2 and having the same shape on the leading n-2 dimensions.

If n=2, A must be lower triangular. The operator performs the BLAS3 function trmm:

out = alpha * op(A) * B

if rightside=False, or

out = alpha * B * op(A)

if rightside=True. Here, alpha is a scalar parameter, and op() is either the identity or the matrix transposition (depending on transpose).

If n>2, trmm is performed separately on the trailing two dimensions for all inputs (batch mode).

Note

The operator supports float32 and float64 data types only.

Examples:

// Single triangular matrix multiply
A = [[1.0, 0], [1.0, 1.0]]
B = [[1.0, 1.0, 1.0], [1.0, 1.0, 1.0]]
trmm(A, B, alpha=2.0) = [[2.0, 2.0, 2.0], [4.0, 4.0, 4.0]]

// Batch triangular matrix multiply
A = [[[1.0, 0], [1.0, 1.0]], [[1.0, 0], [1.0, 1.0]]]
B = [[[1.0, 1.0, 1.0], [1.0, 1.0, 1.0]], [[0.5, 0.5, 0.5], [0.5, 0.5, 0.5]]]
trmm(A, B, alpha=2.0) = [[[2.0, 2.0, 2.0], [4.0, 4.0, 4.0]],
[[1.0, 1.0, 1.0], [2.0, 2.0, 2.0]]]


Defined in src/operator/tensor/la_op.cc:L293

Parameters: A (Symbol) – Tensor of lower triangular matrices B (Symbol) – Tensor of matrices transpose (boolean, optional, default=0) – Use transposed of the triangular matrix rightside (boolean, optional, default=0) – Multiply triangular matrix from the right to non-triangular one. alpha (double, optional, default=1) – Scalar factor to be applied to the result. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.linalg_trsm(A=None, B=None, transpose=_Null, rightside=_Null, alpha=_Null, name=None, attr=None, out=None, **kwargs)

Solves matrix equation involving a lower triangular matrix. Input are tensors A, B, each of dimension n >= 2 and having the same shape on the leading n-2 dimensions.

If n=2, A must be lower triangular. The operator performs the BLAS3 function trsm, solving for out in:

op(A) * out = alpha * B

if rightside=False, or

out * op(A) = alpha * B

if rightside=True. Here, alpha is a scalar parameter, and op() is either the identity or the matrix transposition (depending on transpose).

If n>2, trsm is performed separately on the trailing two dimensions for all inputs (batch mode).

Note

The operator supports float32 and float64 data types only.

Examples:

// Single matrix solve
A = [[1.0, 0], [1.0, 1.0]]
B = [[2.0, 2.0, 2.0], [4.0, 4.0, 4.0]]
trsm(A, B, alpha=0.5) = [[1.0, 1.0, 1.0], [1.0, 1.0, 1.0]]

// Batch matrix solve
A = [[[1.0, 0], [1.0, 1.0]], [[1.0, 0], [1.0, 1.0]]]
B = [[[2.0, 2.0, 2.0], [4.0, 4.0, 4.0]],
[[4.0, 4.0, 4.0], [8.0, 8.0, 8.0]]]
trsm(A, B, alpha=0.5) = [[[1.0, 1.0, 1.0], [1.0, 1.0, 1.0]],
[[2.0, 2.0, 2.0], [2.0, 2.0, 2.0]]]


Defined in src/operator/tensor/la_op.cc:L356

Parameters: A (Symbol) – Tensor of lower triangular matrices B (Symbol) – Tensor of matrices transpose (boolean, optional, default=0) – Use transposed of the triangular matrix rightside (boolean, optional, default=0) – Multiply triangular matrix from the right to non-triangular one. alpha (double, optional, default=1) – Scalar factor to be applied to the result. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.log(data=None, name=None, attr=None, out=None, **kwargs)

Returns element-wise Natural logarithmic value of the input.

The natural logarithm is logarithm in base e, so that log(exp(x)) = x

The storage type of log output is always dense

Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L653

Parameters: data (Symbol) – The input array. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.log10(data=None, name=None, attr=None, out=None, **kwargs)

Returns element-wise Base-10 logarithmic value of the input.

10**log10(x) = x

The storage type of log10 output is always dense

Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L665

Parameters: data (Symbol) – The input array. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.log1p(data=None, name=None, attr=None, out=None, **kwargs)

Returns element-wise log(1 + x) value of the input.

This function is more accurate than log(1 + x) for small x so that $$1+x\approx 1$$

The storage type of log1p output depends upon the input storage type:

• log1p(default) = default
• log1p(row_sparse) = row_sparse

Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L702

Parameters: data (Symbol) – The input array. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.log2(data=None, name=None, attr=None, out=None, **kwargs)

Returns element-wise Base-2 logarithmic value of the input.

2**log2(x) = x

The storage type of log2 output is always dense

Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L677

Parameters: data (Symbol) – The input array. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.log_softmax(data=None, axis=_Null, name=None, attr=None, out=None, **kwargs)

Computes the log softmax of the input. This is equivalent to computing softmax followed by log.

Examples:

>>> x = mx.nd.array([1, 2, .1])
>>> mx.nd.log_softmax(x).asnumpy()
array([-1.41702998, -0.41702995, -2.31702995], dtype=float32)

>>> x = mx.nd.array( [[1, 2, .1],[.1, 2, 1]] )
>>> mx.nd.log_softmax(x, axis=0).asnumpy()
array([[-0.34115392, -0.69314718, -1.24115396],
[-1.24115396, -0.69314718, -0.34115392]], dtype=float32)

Parameters: data (Symbol) – The input array. axis (int, optional, default='-1') – The axis along which to compute softmax. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.make_loss(data=None, name=None, attr=None, out=None, **kwargs)

Make your own loss function in network construction.

This operator accepts a customized loss function symbol as a terminal loss and the symbol should be an operator with no backward dependency. The output of this function is the gradient of loss with respect to the input data.

For example, if you are a making a cross entropy loss function. Assume out is the predicted output and label is the true label, then the cross entropy can be defined as:

cross_entropy = label * log(out) + (1 - label) * log(1 - out)
loss = make_loss(cross_entropy)


We will need to use make_loss when we are creating our own loss function or we want to combine multiple loss functions. Also we may want to stop some variables’ gradients from backpropagation. See more detail in BlockGrad or stop_gradient.

The storage type of make_loss output depends upon the input storage type:

• make_loss(default) = default
• make_loss(row_sparse) = row_sparse

Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L199

Parameters: data (Symbol) – The input array. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.max(data=None, axis=_Null, keepdims=_Null, exclude=_Null, name=None, attr=None, out=None, **kwargs)

Computes the max of array elements over given axes.

Parameters: data (Symbol) – The input axis (Shape(tuple), optional, default=[]) – The axis or axes along which to perform the reduction. The default, axis=(), will compute over all elements into a scalar array with shape (1,).If axis is int, a reduction is performed on a particular axis. If axis is a tuple of ints, a reduction is performed on all the axes specified in the tuple. If exclude is true, reduction will be performed on the axes that are NOT in axis instead. Negative values means indexing from right to left. keepdims (boolean, optional, default=0) – If this is set to True, the reduced axes are left in the result as dimension with size one. exclude (boolean, optional, default=0) – Whether to perform reduction on axis that are NOT in axis instead. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.max_axis(data=None, axis=_Null, keepdims=_Null, exclude=_Null, name=None, attr=None, out=None, **kwargs)

Computes the max of array elements over given axes.

Parameters: data (Symbol) – The input axis (Shape(tuple), optional, default=[]) – The axis or axes along which to perform the reduction. The default, axis=(), will compute over all elements into a scalar array with shape (1,).If axis is int, a reduction is performed on a particular axis. If axis is a tuple of ints, a reduction is performed on all the axes specified in the tuple. If exclude is true, reduction will be performed on the axes that are NOT in axis instead. Negative values means indexing from right to left. keepdims (boolean, optional, default=0) – If this is set to True, the reduced axes are left in the result as dimension with size one. exclude (boolean, optional, default=0) – Whether to perform reduction on axis that are NOT in axis instead. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.mean(data=None, axis=_Null, keepdims=_Null, exclude=_Null, name=None, attr=None, out=None, **kwargs)

Computes the mean of array elements over given axes.

Parameters: data (Symbol) – The input axis (Shape(tuple), optional, default=[]) – The axis or axes along which to perform the reduction. The default, axis=(), will compute over all elements into a scalar array with shape (1,).If axis is int, a reduction is performed on a particular axis. If axis is a tuple of ints, a reduction is performed on all the axes specified in the tuple. If exclude is true, reduction will be performed on the axes that are NOT in axis instead. Negative values means indexing from right to left. keepdims (boolean, optional, default=0) – If this is set to True, the reduced axes are left in the result as dimension with size one. exclude (boolean, optional, default=0) – Whether to perform reduction on axis that are NOT in axis instead. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.min(data=None, axis=_Null, keepdims=_Null, exclude=_Null, name=None, attr=None, out=None, **kwargs)

Computes the min of array elements over given axes.

Parameters: data (Symbol) – The input axis (Shape(tuple), optional, default=[]) – The axis or axes along which to perform the reduction. The default, axis=(), will compute over all elements into a scalar array with shape (1,).If axis is int, a reduction is performed on a particular axis. If axis is a tuple of ints, a reduction is performed on all the axes specified in the tuple. If exclude is true, reduction will be performed on the axes that are NOT in axis instead. Negative values means indexing from right to left. keepdims (boolean, optional, default=0) – If this is set to True, the reduced axes are left in the result as dimension with size one. exclude (boolean, optional, default=0) – Whether to perform reduction on axis that are NOT in axis instead. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.min_axis(data=None, axis=_Null, keepdims=_Null, exclude=_Null, name=None, attr=None, out=None, **kwargs)

Computes the min of array elements over given axes.

Parameters: data (Symbol) – The input axis (Shape(tuple), optional, default=[]) – The axis or axes along which to perform the reduction. The default, axis=(), will compute over all elements into a scalar array with shape (1,).If axis is int, a reduction is performed on a particular axis. If axis is a tuple of ints, a reduction is performed on all the axes specified in the tuple. If exclude is true, reduction will be performed on the axes that are NOT in axis instead. Negative values means indexing from right to left. keepdims (boolean, optional, default=0) – If this is set to True, the reduced axes are left in the result as dimension with size one. exclude (boolean, optional, default=0) – Whether to perform reduction on axis that are NOT in axis instead. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.mp_sgd_mom_update(weight=None, grad=None, mom=None, weight32=None, lr=_Null, momentum=_Null, wd=_Null, rescale_grad=_Null, clip_gradient=_Null, name=None, attr=None, out=None, **kwargs)

Updater function for multi-precision sgd optimizer

mxnet.symbol.mp_sgd_update(weight=None, grad=None, weight32=None, lr=_Null, wd=_Null, rescale_grad=_Null, clip_gradient=_Null, name=None, attr=None, out=None, **kwargs)

Updater function for multi-precision sgd optimizer

mxnet.symbol.nanprod(data=None, axis=_Null, keepdims=_Null, exclude=_Null, name=None, attr=None, out=None, **kwargs)

Computes the product of array elements over given axes treating Not a Numbers (NaN) as one.

Parameters: data (Symbol) – The input axis (Shape(tuple), optional, default=[]) – The axis or axes along which to perform the reduction. The default, axis=(), will compute over all elements into a scalar array with shape (1,).If axis is int, a reduction is performed on a particular axis. If axis is a tuple of ints, a reduction is performed on all the axes specified in the tuple. If exclude is true, reduction will be performed on the axes that are NOT in axis instead. Negative values means indexing from right to left. keepdims (boolean, optional, default=0) – If this is set to True, the reduced axes are left in the result as dimension with size one. exclude (boolean, optional, default=0) – Whether to perform reduction on axis that are NOT in axis instead. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.nansum(data=None, axis=_Null, keepdims=_Null, exclude=_Null, name=None, attr=None, out=None, **kwargs)

Computes the sum of array elements over given axes treating Not a Numbers (NaN) as zero.

Parameters: data (Symbol) – The input axis (Shape(tuple), optional, default=[]) – The axis or axes along which to perform the reduction. The default, axis=(), will compute over all elements into a scalar array with shape (1,).If axis is int, a reduction is performed on a particular axis. If axis is a tuple of ints, a reduction is performed on all the axes specified in the tuple. If exclude is true, reduction will be performed on the axes that are NOT in axis instead. Negative values means indexing from right to left. keepdims (boolean, optional, default=0) – If this is set to True, the reduced axes are left in the result as dimension with size one. exclude (boolean, optional, default=0) – Whether to perform reduction on axis that are NOT in axis instead. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.negative(data=None, name=None, attr=None, out=None, **kwargs)

Numerical negative of the argument, element-wise.

The storage type of negative output depends upon the input storage type:

• negative(default) = default
• negative(row_sparse) = row_sparse
• negative(csr) = csr
Parameters: data (Symbol) – The input array. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.norm(data=None, name=None, attr=None, out=None, **kwargs)

Flattens the input array and then computes the l2 norm.

Examples:

x = [[1, 2],
[3, 4]]

norm(x) = [5.47722578]

rsp = x.cast_storage('row_sparse')

norm(rsp) = [5.47722578]

csr = x.cast_storage('csr')

norm(csr) = [5.47722578]


Parameters: data (Symbol) – Source input name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.normal(loc=_Null, scale=_Null, shape=_Null, ctx=_Null, dtype=_Null, name=None, attr=None, out=None, **kwargs)

Draw random samples from a normal (Gaussian) distribution.

Note

The existing alias normal is deprecated.

Samples are distributed according to a normal distribution parametrized by loc (mean) and scale (standard deviation).

Example:

normal(loc=0, scale=1, shape=(2,2)) = [[ 1.89171135, -1.16881478],
[-1.23474145,  1.55807114]]


Defined in src/operator/random/sample_op.cc:L85

Parameters: loc (float, optional, default=0) – Mean of the distribution. scale (float, optional, default=1) – Standard deviation of the distribution. shape (Shape(tuple), optional, default=[]) – Shape of the output. ctx (string, optional, default='') – Context of output, in format [cpu|gpu|cpu_pinned](n). Only used for imperative calls. dtype ({'None', 'float16', 'float32', 'float64'},optional, default='None') – DType of the output in case this can’t be inferred. Defaults to float32 if not defined (dtype=None). name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.one_hot(indices=None, depth=_Null, on_value=_Null, off_value=_Null, dtype=_Null, name=None, attr=None, out=None, **kwargs)

Returns a one-hot array.

The locations represented by indices take value on_value, while all other locations take value off_value.

one_hot operation with indices of shape (i0, i1) and depth of d would result in an output array of shape (i0, i1, d) with:

output[i,j,:] = off_value
output[i,j,indices[i,j]] = on_value


Examples:

one_hot([1,0,2,0], 3) = [[ 0.  1.  0.]
[ 1.  0.  0.]
[ 0.  0.  1.]
[ 1.  0.  0.]]

one_hot([1,0,2,0], 3, on_value=8, off_value=1,
dtype='int32') = [[1 8 1]
[8 1 1]
[1 1 8]
[8 1 1]]

one_hot([[1,0],[1,0],[2,0]], 3) = [[[ 0.  1.  0.]
[ 1.  0.  0.]]

[[ 0.  1.  0.]
[ 1.  0.  0.]]

[[ 0.  0.  1.]
[ 1.  0.  0.]]]


Defined in src/operator/tensor/indexing_op.cc:L468

Parameters: indices (Symbol) – array of locations where to set on_value depth (int, required) – Depth of the one hot dimension. on_value (double, optional, default=1) – The value assigned to the locations represented by indices. off_value (double, optional, default=0) – The value assigned to the locations not represented by indices. dtype ({'float16', 'float32', 'float64', 'int32', 'uint8'},optional, default='float32') – DType of the output name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.ones_like(data=None, name=None, attr=None, out=None, **kwargs)

Return an array of ones with the same shape and type as the input array.

Examples:

x = [[ 0.,  0.,  0.],
[ 0.,  0.,  0.]]

ones_like(x) = [[ 1.,  1.,  1.],
[ 1.,  1.,  1.]]

Parameters: data (Symbol) – The input name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.pad(data=None, mode=_Null, pad_width=_Null, constant_value=_Null, name=None, attr=None, out=None, **kwargs)

Pads an input array with a constant or edge values of the array.

Note

Note

Current implementation only supports 4D and 5D input arrays with padding applied only on axes 1, 2 and 3. Expects axes 4 and 5 in pad_width to be zero.

This operation pads an input array with either a constant_value or edge values along each axis of the input array. The amount of padding is specified by pad_width.

pad_width is a tuple of integer padding widths for each axis of the format (before_1, after_1, ... , before_N, after_N). The pad_width should be of length 2*N where N is the number of dimensions of the array.

For dimension N of the input array, before_N and after_N indicates how many values to add before and after the elements of the array along dimension N. The widths of the higher two dimensions before_1, after_1, before_2, after_2 must be 0.

Example:

x = [[[[  1.   2.   3.]
[  4.   5.   6.]]

[[  7.   8.   9.]
[ 10.  11.  12.]]]

[[[ 11.  12.  13.]
[ 14.  15.  16.]]

[[ 17.  18.  19.]
[ 20.  21.  22.]]]]

[[[[  1.   1.   2.   3.   3.]
[  1.   1.   2.   3.   3.]
[  4.   4.   5.   6.   6.]
[  4.   4.   5.   6.   6.]]

[[  7.   7.   8.   9.   9.]
[  7.   7.   8.   9.   9.]
[ 10.  10.  11.  12.  12.]
[ 10.  10.  11.  12.  12.]]]

[[[ 11.  11.  12.  13.  13.]
[ 11.  11.  12.  13.  13.]
[ 14.  14.  15.  16.  16.]
[ 14.  14.  15.  16.  16.]]

[[ 17.  17.  18.  19.  19.]
[ 17.  17.  18.  19.  19.]
[ 20.  20.  21.  22.  22.]
[ 20.  20.  21.  22.  22.]]]]

[[[[  0.   0.   0.   0.   0.]
[  0.   1.   2.   3.   0.]
[  0.   4.   5.   6.   0.]
[  0.   0.   0.   0.   0.]]

[[  0.   0.   0.   0.   0.]
[  0.   7.   8.   9.   0.]
[  0.  10.  11.  12.   0.]
[  0.   0.   0.   0.   0.]]]

[[[  0.   0.   0.   0.   0.]
[  0.  11.  12.  13.   0.]
[  0.  14.  15.  16.   0.]
[  0.   0.   0.   0.   0.]]

[[  0.   0.   0.   0.   0.]
[  0.  17.  18.  19.   0.]
[  0.  20.  21.  22.   0.]
[  0.   0.   0.   0.   0.]]]]


Parameters: data (Symbol) – An n-dimensional input array. mode ({'constant', 'edge', 'reflect'}, required) – Padding type to use. “constant” pads with constant_value “edge” pads using the edge values of the input array “reflect” pads by reflecting values with respect to the edges. pad_width (Shape(tuple), required) – Widths of the padding regions applied to the edges of each axis. It is a tuple of integer padding widths for each axis of the format (before_1, after_1, ... , before_N, after_N). It should be of length 2*N where N is the number of dimensions of the array.This is equivalent to pad_width in numpy.pad, but flattened. constant_value (double, optional, default=0) – The value used for padding when mode is “constant”. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.pick(data=None, index=None, axis=_Null, keepdims=_Null, name=None, attr=None, out=None, **kwargs)

Picks elements from an input array according to the input indices along the given axis.

Given an input array of shape (d0, d1) and indices of shape (i0,), the result will be an output array of shape (i0,) with:

output[i] = input[i, indices[i]]


By default, if any index mentioned is too large, it is replaced by the index that addresses the last element along an axis (the clip mode).

This function supports n-dimensional input and (n-1)-dimensional indices arrays.

Examples:

x = [[ 1.,  2.],
[ 3.,  4.],
[ 5.,  6.]]

// picks elements with specified indices along axis 0
pick(x, y=[0,1], 0) = [ 1.,  4.]

// picks elements with specified indices along axis 1
pick(x, y=[0,1,0], 1) = [ 1.,  4.,  5.]

y = [[ 1.],
[ 0.],
[ 2.]]

// picks elements with specified indices along axis 1 and dims are maintained
pick(x,y, 1, keepdims=True) = [[ 2.],
[ 3.],
[ 6.]]


Parameters: data (Symbol) – The input array index (Symbol) – The index array axis (int or None, optional, default='None') – The axis along which to perform the reduction. Negative values means indexing from right to left. Requires axis to be set as int, because global reduction is not supported yet. keepdims (boolean, optional, default=0) – If this is set to True, the reduced axis is left in the result as dimension with size one. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.prod(data=None, axis=_Null, keepdims=_Null, exclude=_Null, name=None, attr=None, out=None, **kwargs)

Computes the product of array elements over given axes.

Parameters: data (Symbol) – The input axis (Shape(tuple), optional, default=[]) – The axis or axes along which to perform the reduction. The default, axis=(), will compute over all elements into a scalar array with shape (1,).If axis is int, a reduction is performed on a particular axis. If axis is a tuple of ints, a reduction is performed on all the axes specified in the tuple. If exclude is true, reduction will be performed on the axes that are NOT in axis instead. Negative values means indexing from right to left. keepdims (boolean, optional, default=0) – If this is set to True, the reduced axes are left in the result as dimension with size one. exclude (boolean, optional, default=0) – Whether to perform reduction on axis that are NOT in axis instead. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.radians(data=None, name=None, attr=None, out=None, **kwargs)

Converts each element of the input array from degrees to radians.

$radians([0, 90, 180, 270, 360]) = [0, \pi/2, \pi, 3\pi/2, 2\pi]$

The storage type of radians output depends upon the input storage type:

Defined in src/operator/tensor/elemwise_unary_op_trig.cc:L182

Parameters: data (Symbol) – The input array. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.random_exponential(lam=_Null, shape=_Null, ctx=_Null, dtype=_Null, name=None, attr=None, out=None, **kwargs)

Draw random samples from an exponential distribution.

Samples are distributed according to an exponential distribution parametrized by lambda (rate).

Example:

exponential(lam=4, shape=(2,2)) = [[ 0.0097189 ,  0.08999364],
[ 0.04146638,  0.31715935]]


Defined in src/operator/random/sample_op.cc:L115

Parameters: lam (float, optional, default=1) – Lambda parameter (rate) of the exponential distribution. shape (Shape(tuple), optional, default=[]) – Shape of the output. ctx (string, optional, default='') – Context of output, in format [cpu|gpu|cpu_pinned](n). Only used for imperative calls. dtype ({'None', 'float16', 'float32', 'float64'},optional, default='None') – DType of the output in case this can’t be inferred. Defaults to float32 if not defined (dtype=None). name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.random_gamma(alpha=_Null, beta=_Null, shape=_Null, ctx=_Null, dtype=_Null, name=None, attr=None, out=None, **kwargs)

Draw random samples from a gamma distribution.

Samples are distributed according to a gamma distribution parametrized by alpha (shape) and beta (scale).

Example:

gamma(alpha=9, beta=0.5, shape=(2,2)) = [[ 7.10486984,  3.37695289],
[ 3.91697288,  3.65933681]]


Defined in src/operator/random/sample_op.cc:L100

Parameters: alpha (float, optional, default=1) – Alpha parameter (shape) of the gamma distribution. beta (float, optional, default=1) – Beta parameter (scale) of the gamma distribution. shape (Shape(tuple), optional, default=[]) – Shape of the output. ctx (string, optional, default='') – Context of output, in format [cpu|gpu|cpu_pinned](n). Only used for imperative calls. dtype ({'None', 'float16', 'float32', 'float64'},optional, default='None') – DType of the output in case this can’t be inferred. Defaults to float32 if not defined (dtype=None). name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.random_generalized_negative_binomial(mu=_Null, alpha=_Null, shape=_Null, ctx=_Null, dtype=_Null, name=None, attr=None, out=None, **kwargs)

Draw random samples from a generalized negative binomial distribution.

Samples are distributed according to a generalized negative binomial distribution parametrized by mu (mean) and alpha (dispersion). alpha is defined as 1/k where k is the failure limit of the number of unsuccessful experiments (generalized to real numbers). Samples will always be returned as a floating point data type.

Example:

generalized_negative_binomial(mu=2.0, alpha=0.3, shape=(2,2)) = [[ 2.,  1.],
[ 6.,  4.]]


Defined in src/operator/random/sample_op.cc:L168

Parameters: mu (float, optional, default=1) – Mean of the negative binomial distribution. alpha (float, optional, default=1) – Alpha (dispersion) parameter of the negative binomial distribution. shape (Shape(tuple), optional, default=[]) – Shape of the output. ctx (string, optional, default='') – Context of output, in format [cpu|gpu|cpu_pinned](n). Only used for imperative calls. dtype ({'None', 'float16', 'float32', 'float64'},optional, default='None') – DType of the output in case this can’t be inferred. Defaults to float32 if not defined (dtype=None). name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.random_negative_binomial(k=_Null, p=_Null, shape=_Null, ctx=_Null, dtype=_Null, name=None, attr=None, out=None, **kwargs)

Draw random samples from a negative binomial distribution.

Samples are distributed according to a negative binomial distribution parametrized by k (limit of unsuccessful experiments) and p (failure probability in each experiment). Samples will always be returned as a floating point data type.

Example:

negative_binomial(k=3, p=0.4, shape=(2,2)) = [[ 4.,  7.],
[ 2.,  5.]]


Defined in src/operator/random/sample_op.cc:L149

Parameters: k (int, optional, default='1') – Limit of unsuccessful experiments. p (float, optional, default=1) – Failure probability in each experiment. shape (Shape(tuple), optional, default=[]) – Shape of the output. ctx (string, optional, default='') – Context of output, in format [cpu|gpu|cpu_pinned](n). Only used for imperative calls. dtype ({'None', 'float16', 'float32', 'float64'},optional, default='None') – DType of the output in case this can’t be inferred. Defaults to float32 if not defined (dtype=None). name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.random_normal(loc=_Null, scale=_Null, shape=_Null, ctx=_Null, dtype=_Null, name=None, attr=None, out=None, **kwargs)

Draw random samples from a normal (Gaussian) distribution.

Note

The existing alias normal is deprecated.

Samples are distributed according to a normal distribution parametrized by loc (mean) and scale (standard deviation).

Example:

normal(loc=0, scale=1, shape=(2,2)) = [[ 1.89171135, -1.16881478],
[-1.23474145,  1.55807114]]


Defined in src/operator/random/sample_op.cc:L85

Parameters: loc (float, optional, default=0) – Mean of the distribution. scale (float, optional, default=1) – Standard deviation of the distribution. shape (Shape(tuple), optional, default=[]) – Shape of the output. ctx (string, optional, default='') – Context of output, in format [cpu|gpu|cpu_pinned](n). Only used for imperative calls. dtype ({'None', 'float16', 'float32', 'float64'},optional, default='None') – DType of the output in case this can’t be inferred. Defaults to float32 if not defined (dtype=None). name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.random_poisson(lam=_Null, shape=_Null, ctx=_Null, dtype=_Null, name=None, attr=None, out=None, **kwargs)

Draw random samples from a Poisson distribution.

Samples are distributed according to a Poisson distribution parametrized by lambda (rate). Samples will always be returned as a floating point data type.

Example:

poisson(lam=4, shape=(2,2)) = [[ 5.,  2.],
[ 4.,  6.]]


Defined in src/operator/random/sample_op.cc:L132

Parameters: lam (float, optional, default=1) – Lambda parameter (rate) of the Poisson distribution. shape (Shape(tuple), optional, default=[]) – Shape of the output. ctx (string, optional, default='') – Context of output, in format [cpu|gpu|cpu_pinned](n). Only used for imperative calls. dtype ({'None', 'float16', 'float32', 'float64'},optional, default='None') – DType of the output in case this can’t be inferred. Defaults to float32 if not defined (dtype=None). name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.random_uniform(low=_Null, high=_Null, shape=_Null, ctx=_Null, dtype=_Null, name=None, attr=None, out=None, **kwargs)

Draw random samples from a uniform distribution.

Note

The existing alias uniform is deprecated.

Samples are uniformly distributed over the half-open interval [low, high) (includes low, but excludes high).

Example:

uniform(low=0, high=1, shape=(2,2)) = [[ 0.60276335,  0.85794562],
[ 0.54488319,  0.84725171]]


Defined in src/operator/random/sample_op.cc:L66

Parameters: low (float, optional, default=0) – Lower bound of the distribution. high (float, optional, default=1) – Upper bound of the distribution. shape (Shape(tuple), optional, default=[]) – Shape of the output. ctx (string, optional, default='') – Context of output, in format [cpu|gpu|cpu_pinned](n). Only used for imperative calls. dtype ({'None', 'float16', 'float32', 'float64'},optional, default='None') – DType of the output in case this can’t be inferred. Defaults to float32 if not defined (dtype=None). name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.rcbrt(data=None, name=None, attr=None, out=None, **kwargs)

Returns element-wise inverse cube-root value of the input.

$rcbrt(x) = 1/\sqrt[3]{x}$

Example:

rcbrt([1,8,-125]) = [1.0, 0.5, -0.2]


Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L618

Parameters: data (Symbol) – The input array. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.reciprocal(data=None, name=None, attr=None, out=None, **kwargs)

Returns the reciprocal of the argument, element-wise.

Calculates 1/x.

Example:

reciprocal([-2, 1, 3, 1.6, 0.2]) = [-0.5, 1.0, 0.33333334, 0.625, 5.0]


Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L363

Parameters: data (Symbol) – The input array. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.relu(data=None, name=None, attr=None, out=None, **kwargs)

Computes rectified linear.

$max(features, 0)$

The storage type of relu output depends upon the input storage type:

• relu(default) = default
• relu(row_sparse) = row_sparse

Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L83

Parameters: data (Symbol) – The input array. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.repeat(data=None, repeats=_Null, axis=_Null, name=None, attr=None, out=None, **kwargs)

Repeats elements of an array.

By default, repeat flattens the input array into 1-D and then repeats the elements:

x = [[ 1, 2],
[ 3, 4]]

repeat(x, repeats=2) = [ 1.,  1.,  2.,  2.,  3.,  3.,  4.,  4.]


The parameter axis specifies the axis along which to perform repeat:

repeat(x, repeats=2, axis=1) = [[ 1.,  1.,  2.,  2.],
[ 3.,  3.,  4.,  4.]]

repeat(x, repeats=2, axis=0) = [[ 1.,  2.],
[ 1.,  2.],
[ 3.,  4.],
[ 3.,  4.]]

repeat(x, repeats=2, axis=-1) = [[ 1.,  1.,  2.,  2.],
[ 3.,  3.,  4.,  4.]]


Defined in src/operator/tensor/matrix_op.cc:L559

Parameters: data (Symbol) – Input data array repeats (int, required) – The number of repetitions for each element. axis (int or None, optional, default='None') – The axis along which to repeat values. The negative numbers are interpreted counting from the backward. By default, use the flattened input array, and return a flat output array. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.reshape(data=None, shape=_Null, reverse=_Null, target_shape=_Null, keep_highest=_Null, name=None, attr=None, out=None, **kwargs)

Reshapes the input array.

Note

Reshape is deprecated, use reshape

Given an array and a shape, this function returns a copy of the array in the new shape. The shape is a tuple of integers such as (2,3,4).The size of the new shape should be same as the size of the input array.

Example:

reshape([1,2,3,4], shape=(2,2)) = [[1,2], [3,4]]


Some dimensions of the shape can take special values from the set {0, -1, -2, -3, -4}. The significance of each is explained below:

• 0 copy this dimension from the input to the output shape.

Example:

- input shape = (2,3,4), shape = (4,0,2), output shape = (4,3,2)
- input shape = (2,3,4), shape = (2,0,0), output shape = (2,3,4)

• -1 infers the dimension of the output shape by using the remainder of the input dimensions keeping the size of the new array same as that of the input array. At most one dimension of shape can be -1.

Example:

- input shape = (2,3,4), shape = (6,1,-1), output shape = (6,1,4)
- input shape = (2,3,4), shape = (3,-1,8), output shape = (3,1,8)
- input shape = (2,3,4), shape=(-1,), output shape = (24,)

• -2 copy all/remainder of the input dimensions to the output shape.

Example:

- input shape = (2,3,4), shape = (-2,), output shape = (2,3,4)
- input shape = (2,3,4), shape = (2,-2), output shape = (2,3,4)
- input shape = (2,3,4), shape = (-2,1,1), output shape = (2,3,4,1,1)

• -3 use the product of two consecutive dimensions of the input shape as the output dimension.

Example:

- input shape = (2,3,4), shape = (-3,4), output shape = (6,4)
- input shape = (2,3,4,5), shape = (-3,-3), output shape = (6,20)
- input shape = (2,3,4), shape = (0,-3), output shape = (2,12)
- input shape = (2,3,4), shape = (-3,-2), output shape = (6,4)

• -4 split one dimension of the input into two dimensions passed subsequent to -4 in shape (can contain -1).

Example:

- input shape = (2,3,4), shape = (-4,1,2,-2), output shape =(1,2,3,4)
- input shape = (2,3,4), shape = (2,-4,-1,3,-2), output shape = (2,1,3,4)


If the argument reverse is set to 1, then the special values are inferred from right to left.

Example:

- without reverse=1, for input shape = (10,5,4), shape = (-1,0), output shape would be (40,5)
- with reverse=1, output shape will be (50,4).


Defined in src/operator/tensor/matrix_op.cc:L164

Parameters: data (Symbol) – Input data to reshape. shape (Shape(tuple), optional, default=[]) – The target shape reverse (boolean, optional, default=0) – If true then the special values are inferred from right to left target_shape (Shape(tuple), optional, default=[]) – (Deprecated! Use shape instead.) Target new shape. One and only one dim can be 0, in which case it will be inferred from the rest of dims keep_highest (boolean, optional, default=0) – (Deprecated! Use shape instead.) Whether keep the highest dim unchanged.If set to true, then the first dim in target_shape is ignored,and always fixed as input name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.reshape_like(lhs=None, rhs=None, name=None, attr=None, out=None, **kwargs)

Reshape lhs to have the same shape as rhs.

Parameters: lhs (Symbol) – First input. rhs (Symbol) – Second input. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.reverse(data=None, axis=_Null, name=None, attr=None, out=None, **kwargs)

Reverses the order of elements along given axis while preserving array shape.

Note: reverse and flip are equivalent. We use reverse in the following examples.

Examples:

x = [[ 0.,  1.,  2.,  3.,  4.],
[ 5.,  6.,  7.,  8.,  9.]]

reverse(x, axis=0) = [[ 5.,  6.,  7.,  8.,  9.],
[ 0.,  1.,  2.,  3.,  4.]]

reverse(x, axis=1) = [[ 4.,  3.,  2.,  1.,  0.],
[ 9.,  8.,  7.,  6.,  5.]]


Defined in src/operator/tensor/matrix_op.cc:L661

Parameters: data (Symbol) – Input data array axis (Shape(tuple), required) – The axis which to reverse elements. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.rint(data=None, name=None, attr=None, out=None, **kwargs)

Returns element-wise rounded value to the nearest integer of the input.

Note

• For input n.5 rint returns n while round returns n+1.
• For input -n.5 both rint and round returns -n-1.

Example:

rint([-1.5, 1.5, -1.9, 1.9, 2.1]) = [-2.,  1., -2.,  2.,  2.]


The storage type of rint output depends upon the input storage type:

• rint(default) = default
• rint(row_sparse) = row_sparse

Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L444

Parameters: data (Symbol) – The input array. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.rmsprop_update(weight=None, grad=None, n=None, lr=_Null, gamma1=_Null, epsilon=_Null, wd=_Null, rescale_grad=_Null, clip_gradient=_Null, clip_weights=_Null, name=None, attr=None, out=None, **kwargs)

Update function for RMSProp optimizer.

RMSprop is a variant of stochastic gradient descent where the gradients are divided by a cache which grows with the sum of squares of recent gradients?

RMSProp is similar to AdaGrad, a popular variant of SGD which adaptively tunes the learning rate of each parameter. AdaGrad lowers the learning rate for each parameter monotonically over the course of training. While this is analytically motivated for convex optimizations, it may not be ideal for non-convex problems. RMSProp deals with this heuristically by allowing the learning rates to rebound as the denominator decays over time.

Define the Root Mean Square (RMS) error criterion of the gradient as $$RMS[g]_t = \sqrt{E[g^2]_t + \epsilon}$$, where $$g$$ represents gradient and $$E[g^2]_t$$ is the decaying average over past squared gradient.

The $$E[g^2]_t$$ is given by:

$E[g^2]_t = \gamma * E[g^2]_{t-1} + (1-\gamma) * g_t^2$

The update step is

$\theta_{t+1} = \theta_t - \frac{\eta}{RMS[g]_t} g_t$

The RMSProp code follows the version in http://www.cs.toronto.edu/~tijmen/csc321/slides/lecture_slides_lec6.pdf Tieleman & Hinton, 2012.

Hinton suggests the momentum term $$\gamma$$ to be 0.9 and the learning rate $$\eta$$ to be 0.001.

Defined in src/operator/optimizer_op.cc:L441

mxnet.symbol.rmspropalex_update(weight=None, grad=None, n=None, g=None, delta=None, lr=_Null, gamma1=_Null, gamma2=_Null, epsilon=_Null, wd=_Null, rescale_grad=_Null, clip_gradient=_Null, clip_weights=_Null, name=None, attr=None, out=None, **kwargs)

Update function for RMSPropAlex optimizer.

RMSPropAlex is non-centered version of RMSProp.

Define $$E[g^2]_t$$ is the decaying average over past squared gradient and $$E[g]_t$$ is the decaying average over past gradient.

$\begin{split}E[g^2]_t = \gamma_1 * E[g^2]_{t-1} + (1 - \gamma_1) * g_t^2\\ E[g]_t = \gamma_1 * E[g]_{t-1} + (1 - \gamma_1) * g_t\\ \Delta_t = \gamma_2 * \Delta_{t-1} - \frac{\eta}{\sqrt{E[g^2]_t - E[g]_t^2 + \epsilon}} g_t\\\end{split}$

The update step is

$\theta_{t+1} = \theta_t + \Delta_t$

The RMSPropAlex code follows the version in http://arxiv.org/pdf/1308.0850v5.pdf Eq(38) - Eq(45) by Alex Graves, 2013.

Graves suggests the momentum term $$\gamma_1$$ to be 0.95, $$\gamma_2$$ to be 0.9 and the learning rate $$\eta$$ to be 0.0001.

Defined in src/operator/optimizer_op.cc:L480

mxnet.symbol.round(data=None, name=None, attr=None, out=None, **kwargs)

Returns element-wise rounded value to the nearest integer of the input.

Example:

round([-1.5, 1.5, -1.9, 1.9, 2.1]) = [-2.,  2., -2.,  2.,  2.]


The storage type of round output depends upon the input storage type:

• round(default) = default
• round(row_sparse) = row_sparse

Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L423

Parameters: data (Symbol) – The input array. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.rsqrt(data=None, name=None, attr=None, out=None, **kwargs)

Returns element-wise inverse square-root value of the input.

$rsqrt(x) = 1/\sqrt{x}$

Example:

rsqrt([4,9,16]) = [0.5, 0.33333334, 0.25]


The storage type of rsqrt output is always dense

Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L584

Parameters: data (Symbol) – The input array. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.sample_exponential(lam=None, shape=_Null, dtype=_Null, name=None, attr=None, out=None, **kwargs)

Concurrent sampling from multiple exponential distributions with parameters lambda (rate).

The parameters of the distributions are provided as an input array. Let [s] be the shape of the input array, n be the dimension of [s], [t] be the shape specified as the parameter of the operator, and m be the dimension of [t]. Then the output will be a (n+m)-dimensional array with shape [s]x[t].

For any valid n-dimensional index i with respect to the input array, output[i] will be an m-dimensional array that holds randomly drawn samples from the distribution which is parameterized by the input value at index i. If the shape parameter of the operator is not set, then one sample will be drawn per distribution and the output array has the same shape as the input array.

Examples:

lam = [ 1.0, 8.5 ]

// Draw a single sample for each distribution
sample_exponential(lam) = [ 0.51837951,  0.09994757]

// Draw a vector containing two samples for each distribution
sample_exponential(lam, shape=(2)) = [[ 0.51837951,  0.19866663],
[ 0.09994757,  0.50447971]]


Defined in src/operator/random/multisample_op.cc:L284

Parameters: lam (Symbol) – Lambda (rate) parameters of the distributions. shape (Shape(tuple), optional, default=[]) – Shape to be sampled from each random distribution. dtype ({'None', 'float16', 'float32', 'float64'},optional, default='None') – DType of the output in case this can’t be inferred. Defaults to float32 if not defined (dtype=None). name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.sample_gamma(alpha=None, beta=None, shape=_Null, dtype=_Null, name=None, attr=None, out=None, **kwargs)

Concurrent sampling from multiple gamma distributions with parameters alpha (shape) and beta (scale).

The parameters of the distributions are provided as input arrays. Let [s] be the shape of the input arrays, n be the dimension of [s], [t] be the shape specified as the parameter of the operator, and m be the dimension of [t]. Then the output will be a (n+m)-dimensional array with shape [s]x[t].

For any valid n-dimensional index i with respect to the input arrays, output[i] will be an m-dimensional array that holds randomly drawn samples from the distribution which is parameterized by the input values at index i. If the shape parameter of the operator is not set, then one sample will be drawn per distribution and the output array has the same shape as the input arrays.

Examples:

alpha = [ 0.0, 2.5 ]
beta = [ 1.0, 0.7 ]

// Draw a single sample for each distribution
sample_gamma(alpha, beta) = [ 0.        ,  2.25797319]

// Draw a vector containing two samples for each distribution
sample_gamma(alpha, beta, shape=(2)) = [[ 0.        ,  0.        ],
[ 2.25797319,  1.70734084]]


Defined in src/operator/random/multisample_op.cc:L282

Parameters: alpha (Symbol) – Alpha (shape) parameters of the distributions. shape (Shape(tuple), optional, default=[]) – Shape to be sampled from each random distribution. dtype ({'None', 'float16', 'float32', 'float64'},optional, default='None') – DType of the output in case this can’t be inferred. Defaults to float32 if not defined (dtype=None). beta (Symbol) – Beta (scale) parameters of the distributions. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.sample_generalized_negative_binomial(mu=None, alpha=None, shape=_Null, dtype=_Null, name=None, attr=None, out=None, **kwargs)

Concurrent sampling from multiple generalized negative binomial distributions with parameters mu (mean) and alpha (dispersion).

The parameters of the distributions are provided as input arrays. Let [s] be the shape of the input arrays, n be the dimension of [s], [t] be the shape specified as the parameter of the operator, and m be the dimension of [t]. Then the output will be a (n+m)-dimensional array with shape [s]x[t].

For any valid n-dimensional index i with respect to the input arrays, output[i] will be an m-dimensional array that holds randomly drawn samples from the distribution which is parameterized by the input values at index i. If the shape parameter of the operator is not set, then one sample will be drawn per distribution and the output array has the same shape as the input arrays.

Samples will always be returned as a floating point data type.

Examples:

mu = [ 2.0, 2.5 ]
alpha = [ 1.0, 0.1 ]

// Draw a single sample for each distribution
sample_generalized_negative_binomial(mu, alpha) = [ 0.,  3.]

// Draw a vector containing two samples for each distribution
sample_generalized_negative_binomial(mu, alpha, shape=(2)) = [[ 0.,  3.],
[ 3.,  1.]]


Defined in src/operator/random/multisample_op.cc:L293

Parameters: mu (Symbol) – Means of the distributions. shape (Shape(tuple), optional, default=[]) – Shape to be sampled from each random distribution. dtype ({'None', 'float16', 'float32', 'float64'},optional, default='None') – DType of the output in case this can’t be inferred. Defaults to float32 if not defined (dtype=None). alpha (Symbol) – Alpha (dispersion) parameters of the distributions. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.sample_multinomial(data=None, shape=_Null, get_prob=_Null, dtype=_Null, name=None, attr=None, out=None, **kwargs)

Concurrent sampling from multiple multinomial distributions.

data is an n dimensional array whose last dimension has length k, where k is the number of possible outcomes of each multinomial distribution. This operator will draw shape samples from each distribution. If shape is empty one sample will be drawn from each distribution.

If get_prob is true, a second array containing log likelihood of the drawn samples will also be returned. This is usually used for reinforcement learning where you can provide reward as head gradient for this array to estimate gradient.

Note that the input distribution must be normalized, i.e. data must sum to 1 along its last axis.

Examples:

probs = [[0, 0.1, 0.2, 0.3, 0.4], [0.4, 0.3, 0.2, 0.1, 0]]

// Draw a single sample for each distribution
sample_multinomial(probs) = [3, 0]

// Draw a vector containing two samples for each distribution
sample_multinomial(probs, shape=(2)) = [[4, 2],
[0, 0]]

// requests log likelihood
sample_multinomial(probs, get_prob=True) = [2, 1], [0.2, 0.3]

Parameters: data (Symbol) – Distribution probabilities. Must sum to one on the last axis. shape (Shape(tuple), optional, default=[]) – Shape to be sampled from each random distribution. get_prob (boolean, optional, default=0) – Whether to also return the log probability of sampled result. This is usually used for differentiating through stochastic variables, e.g. in reinforcement learning. dtype ({'int32'},optional, default='int32') – DType of the output in case this can’t be inferred. Only support int32 for now. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.sample_negative_binomial(k=None, p=None, shape=_Null, dtype=_Null, name=None, attr=None, out=None, **kwargs)

Concurrent sampling from multiple negative binomial distributions with parameters k (failure limit) and p (failure probability).

The parameters of the distributions are provided as input arrays. Let [s] be the shape of the input arrays, n be the dimension of [s], [t] be the shape specified as the parameter of the operator, and m be the dimension of [t]. Then the output will be a (n+m)-dimensional array with shape [s]x[t].

For any valid n-dimensional index i with respect to the input arrays, output[i] will be an m-dimensional array that holds randomly drawn samples from the distribution which is parameterized by the input values at index i. If the shape parameter of the operator is not set, then one sample will be drawn per distribution and the output array has the same shape as the input arrays.

Samples will always be returned as a floating point data type.

Examples:

k = [ 20, 49 ]
p = [ 0.4 , 0.77 ]

// Draw a single sample for each distribution
sample_negative_binomial(k, p) = [ 15.,  16.]

// Draw a vector containing two samples for each distribution
sample_negative_binomial(k, p, shape=(2)) = [[ 15.,  50.],
[ 16.,  12.]]


Defined in src/operator/random/multisample_op.cc:L289

Parameters: k (Symbol) – Limits of unsuccessful experiments. shape (Shape(tuple), optional, default=[]) – Shape to be sampled from each random distribution. dtype ({'None', 'float16', 'float32', 'float64'},optional, default='None') – DType of the output in case this can’t be inferred. Defaults to float32 if not defined (dtype=None). p (Symbol) – Failure probabilities in each experiment. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.sample_normal(mu=None, sigma=None, shape=_Null, dtype=_Null, name=None, attr=None, out=None, **kwargs)

Concurrent sampling from multiple normal distributions with parameters mu (mean) and sigma (standard deviation).

The parameters of the distributions are provided as input arrays. Let [s] be the shape of the input arrays, n be the dimension of [s], [t] be the shape specified as the parameter of the operator, and m be the dimension of [t]. Then the output will be a (n+m)-dimensional array with shape [s]x[t].

For any valid n-dimensional index i with respect to the input arrays, output[i] will be an m-dimensional array that holds randomly drawn samples from the distribution which is parameterized by the input values at index i. If the shape parameter of the operator is not set, then one sample will be drawn per distribution and the output array has the same shape as the input arrays.

Examples:

mu = [ 0.0, 2.5 ]
sigma = [ 1.0, 3.7 ]

// Draw a single sample for each distribution
sample_normal(mu, sigma) = [-0.56410581,  0.95934606]

// Draw a vector containing two samples for each distribution
sample_normal(mu, sigma, shape=(2)) = [[-0.56410581,  0.2928229 ],
[ 0.95934606,  4.48287058]]


Defined in src/operator/random/multisample_op.cc:L279

Parameters: mu (Symbol) – Means of the distributions. shape (Shape(tuple), optional, default=[]) – Shape to be sampled from each random distribution. dtype ({'None', 'float16', 'float32', 'float64'},optional, default='None') – DType of the output in case this can’t be inferred. Defaults to float32 if not defined (dtype=None). sigma (Symbol) – Standard deviations of the distributions. name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.sample_poisson(lam=None, shape=_Null, dtype=_Null, name=None, attr=None, out=None, **kwargs)

Concurrent sampling from multiple Poisson distributions with parameters lambda (rate).

The parameters of the distributions are provided as an input array. Let [s] be the shape of the input array, n be the dimension of [s], [t] be the shape specified as the parameter of the operator, and m be the dimension of [t]. Then the output will be a (n+m)-dimensional array with shape [s]x[t].

For any valid n-dimensional index i with respect to the input array, output[i] will be an m-dimensional array that holds randomly drawn samples from the distribution which is parameterized by the input value at index i. If the shape parameter of the operator is not set, then one sample will be drawn per distribution and the output array has the same shape as the input array.

Samples will always be returned as a floating point data type.

Examples:

lam = [ 1.0, 8.5 ]

// Draw a single sample for each distribution
sample_poisson(lam) = [  0.,  13.]

// Draw a vector containing two samples for each distribution
sample_poisson(lam, shape=(2)) = [[  0.,   4.],
[ 13.,   8.]]


Defined in src/operator/random/multisample_op.cc:L286

Parameters: lam (Symbol) – Lambda (rate) parameters of the distributions. shape (Shape(tuple), optional, default=[]) – Shape to be sampled from each random distribution. dtype ({'None', 'float16', 'float32', 'float64'},optional, default='None') – DType of the output in case this can’t be inferred. Defaults to float32 if not defined (dtype=None). name (string, optional.) – Name of the resulting symbol. The result symbol. Symbol
mxnet.symbol.sample_uniform(low=None, high=None, shape=_Null, dtype=_Null, name=None, attr=None, out=None, **kwargs)

Concurrent sampling from multiple uniform distributions on the intervals given by [low,high).

The parameters of the distributions are provided as input arrays. Let [s] be the shape of the input arrays, n be the dimension of [s], [t] be the shape specified as the parameter of the operator, and m be the dimension of [t]. Then the output will be a (n+m)-dimensional array with shape [s]x[t].

For any valid n-dimensional index i with respect to the input arrays, output[i] will be an m-dimensional array that holds randomly drawn samples from the distribution which is parameterized by the input values at index i. If the shape parameter of the operator is not set, then one sample will be drawn per distribution and the output array has the same shape as the input arrays.

Examples:

low = [ 0.0, 2.5 ]
high = [ 1.0, 3.7 ]

// Draw a single sample for each distribution
sample_uniform(low, high) = [ 0.40451524,  3.18687344]

// Draw a vector containing two samples for each distribution
sample_uniform(low, high, shape=(2)) = [[ 0.40451524,  0.18017688],
[ 3.18687344,  3.68352246]]


Defined in src/operator/random/multisample_op.cc:L277

Parameters: low (Symbol) – Lower bounds of the distributions. shape (Shape(tuple), optional, default=[]) – Shape to be sampled from each random distribution. dtype ({'None', 'float16', 'float32', 'float64'},optional, default='None') – DType of the output in case this can’t be inferred. Defaults to float32 if not defined (dtype=None). high (