Applies an activation function elementwise to the input.
The following activation functions are supported:
 relu
: Rectified Linear Unit, :math:y = max(x, 0)
 sigmoid
: :math:y = \frac{1}{1 + exp(x)}
 tanh
: Hyperbolic tangent, :math:y = \frac{exp(x)  exp(x)}{exp(x) + exp(x)}
 softrelu
: Soft ReLU, or SoftPlus, :math:y = log(1 + exp(x))
 softsign
: :math:y = \frac{x}{1 + abs(x)}
Defined in src/operator/nn/activation.cc:L176
Applies an activation function elementwise to the input.
The following activation functions are supported:
 relu
: Rectified Linear Unit, :math:y = max(x, 0)
 sigmoid
: :math:y = \frac{1}{1 + exp(x)}
 tanh
: Hyperbolic tangent, :math:y = \frac{exp(x)  exp(x)}{exp(x) + exp(x)}
 softrelu
: Soft ReLU, or SoftPlus, :math:y = log(1 + exp(x))
 softsign
: :math:y = \frac{x}{1 + abs(x)}
Defined in src/operator/nn/activation.cc:L176
The input array.
Activation function to be applied.
org.apache.mxnet.Symbol
Batch normalization.
Normalizes a data batch by mean and variance, and applies a scale
asgamma
well as offset
.beta
Assume the input has more than one dimension and we normalize along axis 1.
We first compute the mean and variance along this axis:
..
Batch normalization.
Normalizes a data batch by mean and variance, and applies a scale
asgamma
well as offset
.beta
Assume the input has more than one dimension and we normalize along axis 1.
We first compute the mean and variance along this axis:
.. math::
data\_mean[i] = mean(data[:,i,:,...]) \\
data\_var[i] = var(data[:,i,:,...])
Then compute the normalized output, which has the same shape as input, as following:
.. math::
out[:,i,:,...] = \frac{data[:,i,:,...]  data\_mean[i]}{\sqrt{data\_var[i]+\epsilon}} * gamma[i] + beta[i]
Both *mean* and *var* returns a scalar by treating the input as a vector.
Assume the input has size *k* on axis 1, then both
and gamma
beta
have shape *(k,)*. If
is set to be true, then outputs both output_mean_var
anddata_mean
the inverse of
, which are needed for the backward pass. Note that gradient of these data_var
two outputs are blocked.
Besides the inputs and the outputs, this operator accepts two auxiliary
states,
and moving_mean
, which are *k*lengthmoving_var
vectors. They are global statistics for the whole dataset, which are updated
by::
moving_mean = moving_mean * momentum + data_mean * (1  momentum)
moving_var = moving_var * momentum + data_var * (1  momentum)
If
is set to be true, then use_global_stats
andmoving_mean
are used instead of moving_var
and data_mean
to computedata_var
the output. It is often used during inference.
The parameter
specifies which axis of the input shape denotesaxis
the 'channel' (separately normalized groups). The default is 1. Specifying 1 sets the channel
axis to be the last item in the input shape.
Both
and gamma
are learnable parameters. But if beta
is true,fix_gamma
then set
to 1 and its gradient to 0.gamma
Note::
When fix_gamma is set to True, no sparse support is provided. If fix_gamma is set to False,
the sparse tensors will fallback.
Defined in src/operator/nn/batch_norm.cc:L571
Input data to batch normalization
gamma array
beta array
running mean of input
running variance of input
Epsilon to prevent div 0. Must be no less than CUDNN_BN_MIN_EPSILON defined in cudnn.h when using cudnn (usually 1e5)
Momentum for moving average
Fix gamma while training
Whether use global moving statistics instead of local batchnorm. This will force change batchnorm into a scale shift operator.
Output the mean and inverse std
Specify which shape axis the channel is specified
Do not select CUDNN operator, if available
org.apache.mxnet.Symbol
Batch normalization.
This operator is DEPRECATED.
Batch normalization.
This operator is DEPRECATED. Perform BatchNorm on the input.
Normalizes a data batch by mean and variance, and applies a scale
asgamma
well as offset
.beta
Assume the input has more than one dimension and we normalize along axis 1.
We first compute the mean and variance along this axis:
.. math::
data\_mean[i] = mean(data[:,i,:,...]) \\
data\_var[i] = var(data[:,i,:,...])
Then compute the normalized output, which has the same shape as input, as following:
.. math::
out[:,i,:,...] = \frac{data[:,i,:,...]  data\_mean[i]}{\sqrt{data\_var[i]+\epsilon}} * gamma[i] + beta[i]
Both *mean* and *var* returns a scalar by treating the input as a vector.
Assume the input has size *k* on axis 1, then both
and gamma
beta
have shape *(k,)*. If
is set to be true, then outputs both output_mean_var
anddata_mean
as well, which are needed for the backward pass.data_var
Besides the inputs and the outputs, this operator accepts two auxiliary
states,
and moving_mean
, which are *k*lengthmoving_var
vectors. They are global statistics for the whole dataset, which are updated
by::
moving_mean = moving_mean * momentum + data_mean * (1  momentum)
moving_var = moving_var * momentum + data_var * (1  momentum)
If
is set to be true, then use_global_stats
andmoving_mean
are used instead of moving_var
and data_mean
to computedata_var
the output. It is often used during inference.
Both
and gamma
are learnable parameters. But if beta
is true,fix_gamma
then set
to 1 and its gradient to 0.gamma
There's no sparse support for this operator, and it will exhibit problematic behavior if used with
sparse tensors.
Defined in src/operator/batch_norm_v1.cc:L95
Input data to batch normalization
gamma array
beta array
Epsilon to prevent div 0
Momentum for moving average
Fix gamma while training
Whether use global moving statistics instead of local batchnorm. This will force change batchnorm into a scale shift operator.
Output All,normal mean and var
org.apache.mxnet.Symbol
Applies bilinear sampling to input feature map.
Bilinear Sampling is the key of [NIPS2015] \"Spatial Transformer Networks\".
Applies bilinear sampling to input feature map.
Bilinear Sampling is the key of [NIPS2015] \"Spatial Transformer Networks\". The usage of the operator is very similar to remap function in OpenCV,
except that the operator has the backward pass.
Given :math:data
and :math:grid
, then the output is computed by
.. math::
x_{src} = grid[batch, 0, y_{dst}, x_{dst}] \\
y_{src} = grid[batch, 1, y_{dst}, x_{dst}] \\
output[batch, channel, y_{dst}, x_{dst}] = G(data[batch, channel, y_{src}, x_{src})
:math:x_{dst}
, :math:y_{dst}
enumerate all spatial locations in :math:output
, and :math:G()
denotes the bilinear interpolation kernel.
The outboundary points will be padded with zeros.The shape of the output will be (data.shape[0], data.shape[1], grid.shape[2], grid.shape[3]).
The operator assumes that :math:data
has 'NCHW' layout and :math:grid
has been normalized to [1, 1].
BilinearSampler often cooperates with GridGenerator which generates sampling grids for BilinearSampler.
GridGenerator supports two kinds of transformation:
and affine
.warp
If users want to design a CustomOp to manipulate :math:grid
, please firstly refer to the code of GridGenerator.
Example 1::
## Zoom out data two times
data = array(4, 3, 6],
[1, 8, 8, 9],
[0, 4, 1, 5],
[1, 0, 1, 3)
affine_matrix = array(0, 0],
[0, 2, 0)
affine_matrix = reshape(affine_matrix, shape=(1, 6))
grid = GridGenerator(data=affine_matrix, transform_type='affine', target_shape=(4, 4))
out = BilinearSampler(data, grid)
out
0, 0, 0, 0],
[ 0, 3.5, 6.5, 0],
[ 0, 1.25, 2.5, 0],
[ 0, 0, 0, 0]]]
Example 2::
## shift data horizontally by 1 pixel
data = array(4, 3, 6],
[1, 8, 8, 9],
[0, 4, 1, 5],
[1, 0, 1, 3)
warp_maxtrix = array(1, 1, 1],
[1, 1, 1, 1],
[1, 1, 1, 1],
[1, 1, 1, 1]],
0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0]])
grid = GridGenerator(data=warp_matrix, transform_type='warp')
out = BilinearSampler(data, grid)
out
4, 3, 6, 0],
[ 8, 8, 9, 0],
[ 4, 1, 5, 0],
[ 0, 1, 3, 0]]]
Defined in src/operator/bilinear_sampler.cc:L245
Input data to the BilinearsamplerOp.
Input grid to the BilinearsamplerOp.grid has two channels: x_src, y_src
org.apache.mxnet.Symbol
Stops gradient computation.
Stops the accumulated gradient of the inputs from flowing through this operator
in the backward direction.
Stops gradient computation.
Stops the accumulated gradient of the inputs from flowing through this operator
in the backward direction. In other words, this operator prevents the contribution
of its inputs to be taken into account for computing gradients.
Example::
v1 = [1, 2]
v2 = [0, 1]
a = Variable('a')
b = Variable('b')
b_stop_grad = stop_gradient(3 * b)
loss = MakeLoss(b_stop_grad + a)
executor = loss.simple_bind(ctx=cpu(), a=(1,2), b=(1,2))
executor.forward(is_train=True, a=v1, b=v2)
executor.outputs
[ 1. 5.]
executor.backward()
executor.grad_arrays
[ 0. 0.]
[ 1. 1.]
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L265
The input array.
org.apache.mxnet.Symbol
Casts all elements of the input to a new type.
..
Casts all elements of the input to a new type.
.. note::
is deprecated. Use Cast
instead.cast
Example::
cast([0.9, 1.3], dtype='int32') = [0, 1]
cast([1e20, 11.1], dtype='float16') = [inf, 11.09375]
cast([300, 11.1, 10.9, 1, 3], dtype='uint8') = [44, 11, 10, 255, 253]
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L504
The input.
Output data type.
org.apache.mxnet.Symbol
Joins input arrays along a given axis.
..
Joins input arrays along a given axis.
.. note:: Concat
is deprecated. Use concat
instead.
The dimensions of the input arrays should be the same except the axis along
which they will be concatenated.
The dimension of the output array along the concatenated axis will be equal
to the sum of the corresponding dimensions of the input arrays.
The storage type of
output depends on storage types of inputsconcat
 concat(csr, csr, ..., csr, dim=0) = csr
 otherwise,
generates output with default storageconcat
Example::
x = 1,1],[2,2
y = 3,3],[4,4],[5,5
z = [7,7],[8,8
concat(x,y,z,dim=0) = 1., 1.],
[ 2., 2.],
[ 3., 3.],
[ 4., 4.],
[ 5., 5.],
[ 6., 6.],
[ 7., 7.],
[ 8., 8.
Note that you cannot concat x,y,z along dimension 1 since dimension
0 is not the same for all the input arrays.
concat(y,z,dim=1) = 3., 3., 6., 6.],
[ 4., 4., 7., 7.],
[ 5., 5., 8., 8.
Defined in src/operator/nn/concat.cc:L270
List of arrays to concatenate
Number of inputs to be concated.
the dimension to be concated.
org.apache.mxnet.Symbol
Compute *N*D convolution on *(N+2)*D input.
In the 2D convolution, given input data with shape *(batch_size,
channel, height, width)*, the output is computed by
..
Compute *N*D convolution on *(N+2)*D input.
In the 2D convolution, given input data with shape *(batch_size,
channel, height, width)*, the output is computed by
.. math::
out[n,i,:,:] = bias[i] + \sum_{j=0}^{channel} data[n,j,:,:] \star
weight[i,j,:,:]
where :math:\star
is the 2D crosscorrelation operator.
For general 2D convolution, the shapes are
 **data**: *(batch_size, channel, height, width)*
 **weight**: *(num_filter, channel, kernel[0], kernel[1])*
 **bias**: *(num_filter,)*
 **out**: *(batch_size, num_filter, out_height, out_width)*.
Define::
f(x,k,p,s,d) = floor((x+2*pd*(k1)1)/s)+1
then we have::
out_height=f(height, kernel[0], pad[0], stride[0], dilate[0])
out_width=f(width, kernel[1], pad[1], stride[1], dilate[1])
If
is set to be true, then the no_bias
term is ignored.bias
The default data
is *NCHW*, namely *(batch_size, channel, height,layout
width)*. We can choose other layouts such as *NHWC*.
If
is larger than 1, denoted by *g*, then split the input num_group
data
evenly into *g* parts along the channel axis, and also evenly split weight
along the first dimension. Next compute the convolution on the *i*th part of
the data with the *i*th weight part. The output is obtained by concatenating all
the *g* results.
1D convolution does not have *height* dimension but only *width* in space.
 **data**: *(batch_size, channel, width)*
 **weight**: *(num_filter, channel, kernel[0])*
 **bias**: *(num_filter,)*
 **out**: *(batch_size, num_filter, out_width)*.
3D convolution adds an additional *depth* dimension besides *height* and
*width*. The shapes are
 **data**: *(batch_size, channel, depth, height, width)*
 **weight**: *(num_filter, channel, kernel[0], kernel[1], kernel[2])*
 **bias**: *(num_filter,)*
 **out**: *(batch_size, num_filter, out_depth, out_height, out_width)*.
Both
and weight
are learnable parameters.bias
There are other options to tune the performance.
 **cudnn_tune**: enable this option leads to higher startup time but may give
faster speed. Options are
MXNET_CUDNN_AUTOTUNE_DEFAULT
. 0 for off, 1 for limited workspaceInput data to the ConvolutionOp.
Weight matrix.
Bias parameter.
Convolution kernel size: (w,), (h, w) or (d, h, w)
Convolution stride: (w,), (h, w) or (d, h, w). Defaults to 1 for each dimension.
Convolution dilate: (w,), (h, w) or (d, h, w). Defaults to 1 for each dimension.
Zero pad for convolution: (w,), (h, w) or (d, h, w). Defaults to no padding.
Convolution filter(channel) number
Number of group partitions.
Maximum temporary workspace allowed (MB) in convolution.This parameter has two usages. When CUDNN is not used, it determines the effective batch size of the convolution kernel. When CUDNN is used, it controls the maximum temporary storage used for tuning the best CUDNN kernel when limited_workspace
strategy is used.
Whether to disable bias parameter.
Whether to pick convolution algo by running performance test.
Turn off cudnn for this layer.
Set layout for input, output and weight. Empty for default layout: NCW for 1d, NCHW for 2d and NCDHW for 3d.
org.apache.mxnet.Symbol
This operator is DEPRECATED.
This operator is DEPRECATED. Apply convolution to input then add a bias.
Input data to the ConvolutionV1Op.
Weight matrix.
Bias parameter.
convolution kernel size: (h, w) or (d, h, w)
convolution stride: (h, w) or (d, h, w)
convolution dilate: (h, w) or (d, h, w)
pad for convolution: (h, w) or (d, h, w)
convolution filter(channel) number
Number of group partitions. Equivalent to slicing input into num_group partitions, apply convolution on each, then concatenate the results
Maximum temporary workspace allowed for convolution (MB).This parameter determines the effective batch size of the convolution kernel, which may be smaller than the given batch size. Also, the workspace will be automatically enlarged to make sure that we can run the kernel with batch_size=1
Whether to disable bias parameter.
Whether to pick convolution algo by running performance test. Leads to higher startup time but may give faster speed. Options are: 'off': no tuning 'limited_workspace': run test and pick the fastest algorithm that doesn't exceed workspace limit. 'fastest': pick the fastest algorithm and ignore workspace limit. If set to None (default), behavior is determined by environment variable MXNET_CUDNN_AUTOTUNE_DEFAULT: 0 for off, 1 for limited workspace (default), 2 for fastest.
Turn off cudnn for this layer.
Set layout for input, output and weight. Empty for default layout: NCHW for 2d and NCDHW for 3d.
org.apache.mxnet.Symbol
Applies correlation to inputs.
The correlation layer performs multiplicative patch comparisons between two feature maps.
Given two multichannel feature maps :math:f_{1}, f_{2}
, with :math:w
, :math:h
, and :math:c
being their width, height, and number of channels,
the correlation layer lets the network compare each patch from :math:f_{1}
with each patch from :math:f_{2}
.
For now we consider only a single comparison of two patches.
Applies correlation to inputs.
The correlation layer performs multiplicative patch comparisons between two feature maps.
Given two multichannel feature maps :math:f_{1}, f_{2}
, with :math:w
, :math:h
, and :math:c
being their width, height, and number of channels,
the correlation layer lets the network compare each patch from :math:f_{1}
with each patch from :math:f_{2}
.
For now we consider only a single comparison of two patches. The 'correlation' of two patches centered at :math:x_{1}
in the first map and
:math:x_{2}
in the second map is then defined as:
.. math::
c(x_{1}, x_{2}) = \sum_{o \in [k,k] \times [k,k]} <f_{1}(x_{1} + o), f_{2}(x_{2} + o)>
for a square patch of size :math:K:=2k+1
.
Note that the equation above is identical to one step of a convolution in neural networks, but instead of convolving data with a filter, it convolves data with other
data. For this reason, it has no training weights.
Computing :math:c(x_{1}, x_{2})
involves :math:c * K^{{2} multiplications. Comparing all patch combinations involves :math:w}{2}*h^{{2} such computations.
Given a maximum displacement :math:d, for each location :math:x_{1} it computes correlations :math:c(x_{1}, x_{2}) only in a neighborhood of size :math:D:=2d+1,
by limiting the range of :math:x_{2}. We use strides :math:s_{1}, s_{2}, to quantize :math:x_{1} globally and to quantize :math:x_{2} within the neighborhood
centered around :math:x_{1}.
The final output is defined by the following expression:
.. math::
out[n, q, i, j] = c(x_{i, j}, x_{q})
where :math:i and :math:j enumerate spatial locations in :math:f_{1}, and :math:q denotes the :math:q}{th}
neighborhood of :math:x_{i,j}
.
Defined in src/operator/correlation.cc:L198
Input data1 to the correlation.
Input data2 to the correlation.
kernel size for Correlation must be an odd number
Max displacement of Correlation
stride1 quantize data1 globally
stride2 quantize data2 within the neighborhood centered around data1
pad for Correlation
operation type is either multiplication or subduction
org.apache.mxnet.Symbol
..
.. note:: Crop
is deprecated. Use slice
instead.
Crop the 2nd and 3rd dim of input data, with the corresponding size of h_w or
with width and height of the second input symbol, i.e., with one input, we need h_w to
specify the crop height and width, otherwise the second input symbol's size will be used
Defined in src/operator/crop.cc:L50
Tensor or List of Tensors, the second input will be used as crop_like shape reference
Number of inputs for crop, if equals one, then we will use the h_wfor crop height and width, else if equals two, then we will use the heightand width of the second input symbol, we name crop_like here
crop offset coordinate: (y, x)
crop height and width: (h, w)
If set to true, then it will use be the center_crop,or it will crop using the shape of crop_like
org.apache.mxnet.Symbol
Computes 1D or 2D transposed convolution (aka fractionally strided convolution) of the input tensor.
Computes 1D or 2D transposed convolution (aka fractionally strided convolution) of the input tensor. This operation can be seen as the gradient of Convolution operation with respect to its input. Convolution usually reduces the size of the input. Transposed convolution works the other way, going from a smaller input to a larger output while preserving the connectivity pattern.
Input tensor to the deconvolution operation.
Weights representing the kernel.
Bias added to the result after the deconvolution operation.
Deconvolution kernel size: (w,), (h, w) or (d, h, w). This is same as the kernel size used for the corresponding convolution
The stride used for the corresponding convolution: (w,), (h, w) or (d, h, w). Defaults to 1 for each dimension.
Dilation factor for each dimension of the input: (w,), (h, w) or (d, h, w). Defaults to 1 for each dimension.
The amount of implicit zero padding added during convolution for each dimension of the input: (w,), (h, w) or (d, h, w).
is usually a good choice. If (kernel1)/2
target_shape
is set, pad
will be ignored and a padding that will generate the target shape will be used. Defaults to no padding.
Adjustment for output shape: (w,), (h, w) or (d, h, w). If target_shape
is set, adj
will be ignored and computed accordingly.
Shape of the output tensor: (w,), (h, w) or (d, h, w).
Number of output filters.
Number of groups partition.
Maximum temporary workspace allowed (MB) in deconvolution.This parameter has two usages. When CUDNN is not used, it determines the effective batch size of the deconvolution kernel. When CUDNN is used, it controls the maximum temporary storage used for tuning the best CUDNN kernel when limited_workspace
strategy is used.
Whether to disable bias parameter.
Whether to pick convolution algorithm by running performance test.
Turn off cudnn for this layer.
Set layout for input, output and weight. Empty for default layout, NCW for 1d, NCHW for 2d and NCDHW for 3d.
org.apache.mxnet.Symbol
Applies dropout operation to input array.
 During training, each element of the input is set to zero with probability p.
The whole array is rescaled by :math:1/(1p)
to keep the expected
sum of the input unchanged.
 During testing, this operator does not change the input if mode is 'training'.
If mode is 'always', the same computaion as during training will be applied.
Example::
random.seed(998)
input_array = array(0.5, 0.5, 2., 7.],
[2., 0.4, 7., 3., 0.2)
a = symbol.Variable('a')
dropout = symbol.Dropout(a, p = 0.2)
executor = dropout.simple_bind(a = input_array.shape)
## If training
executor.forward(is_train = True, a = input_array)
executor.outputs
3.75 0.625 0. 2.5 8.75 ]
[ 2.5 0.5 8.75 3.75 0.
## If testing
executor.forward(is_train = False, a = input_array)
executor.outputs
3. 0.5 0.5 2. 7. ]
[ 2. 0.4 7. 3. 0.2
Defined in src/operator/nn/dropout.cc:L76
Applies dropout operation to input array.
 During training, each element of the input is set to zero with probability p.
The whole array is rescaled by :math:1/(1p)
to keep the expected
sum of the input unchanged.
 During testing, this operator does not change the input if mode is 'training'.
If mode is 'always', the same computaion as during training will be applied.
Example::
random.seed(998)
input_array = array(0.5, 0.5, 2., 7.],
[2., 0.4, 7., 3., 0.2)
a = symbol.Variable('a')
dropout = symbol.Dropout(a, p = 0.2)
executor = dropout.simple_bind(a = input_array.shape)
## If training
executor.forward(is_train = True, a = input_array)
executor.outputs
3.75 0.625 0. 2.5 8.75 ]
[ 2.5 0.5 8.75 3.75 0.
## If testing
executor.forward(is_train = False, a = input_array)
executor.outputs
3. 0.5 0.5 2. 7. ]
[ 2. 0.4 7. 3. 0.2
Defined in src/operator/nn/dropout.cc:L76
Input array to which dropout will be applied.
Fraction of the input that gets dropped out during training time.
Whether to only turn on dropout during training or to also turn on for inference.
Axes for variational dropout kernel.
org.apache.mxnet.Symbol
Adds all input arguments elementwise.
..
Adds all input arguments elementwise.
.. math::
add\_n(a_1, a_2, ..., a_n) = a_1 + a_2 + ... + a_n
is potentially more efficient than calling add_n
by add
n
times.
The storage type of
output depends on storage types of inputsadd_n
 add_n(row_sparse, row_sparse, ..) = row_sparse
 add_n(default, csr, default) = default
 add_n(any input combinations longer than 4 (>4) with at least one default type) = default
 otherwise,
falls all inputs back to default storage and generates default storageadd_n
Defined in src/operator/tensor/elemwise_sum.cc:L156
Positional input arguments
org.apache.mxnet.Symbol
Maps integer indices to vector representations (embeddings).
This operator maps words to realvalued vectors in a highdimensional space,
called word embeddings.
Maps integer indices to vector representations (embeddings).
This operator maps words to realvalued vectors in a highdimensional space,
called word embeddings. These embeddings can capture semantic and syntactic properties of the words.
For example, it has been noted that in the learned embedding spaces, similar words tend
to be close to each other and dissimilar words far apart.
For an input array of shape (d1, ..., dK),
the shape of an output array is (d1, ..., dK, output_dim).
All the input values should be integers in the range [0, input_dim).
If the input_dim is ip0 and output_dim is op0, then shape of the embedding weight matrix must be
(ip0, op0).
By default, if any index mentioned is too large, it is replaced by the index that addresses
the last vector in an embedding matrix.
Examples::
input_dim = 4
output_dim = 5
// Each row in weight matrix y represents a word. So, y = (w0,w1,w2,w3)
y = 0., 1., 2., 3., 4.],
[ 5., 6., 7., 8., 9.],
[ 10., 11., 12., 13., 14.],
[ 15., 16., 17., 18., 19.
// Input array x represents ngrams(2gram). So, x = [(w1,w3), (w0,w2)]
x = 1., 3.],
[ 0., 2.
// Mapped input x to its vector representation y.
Embedding(x, y, 4, 5) = 5., 6., 7., 8., 9.],
[ 15., 16., 17., 18., 19.]],
0., 1., 2., 3., 4.],
[ 10., 11., 12., 13., 14.]
The storage type of weight can be either row_sparse or default.
.. Note::
If "sparse_grad" is set to True, the storage type of gradient w.r.t weights will be
"row_sparse". Only a subset of optimizers support sparse gradients, including SGD, AdaGrad
and Adam. Note that by default lazy updates is turned on, which may perform differently
from standard updates. For more details, please check the Optimization API at:
https://mxnet.incubator.apache.org/api/python/optimization/optimization.html
Defined in src/operator/tensor/indexing_op.cc:L239
The input array to the embedding operator.
The embedding weight matrix.
Vocabulary size of the input indices.
Dimension of the embedding vectors.
Data type of weight.
Compute row sparse gradient in the backward calculation. If set to True, the grad's storage type is row_sparse.
org.apache.mxnet.Symbol
Flattens the input array into a 2D array by collapsing the higher dimensions.
..
Flattens the input array into a 2D array by collapsing the higher dimensions.
.. note:: Flatten
is deprecated. Use flatten
instead.
For an input array with shape
, (d1, d2, ..., dk)
flatten
operation reshapes
the input array into an output array of shape
.(d1, d2*...*dk)
Note that the bahavior of this function is different from numpy.ndarray.flatten,
which behaves similar to mxnet.ndarray.reshape((1,)).
Example::
x = [1,2,3],
[4,5,6],
[7,8,9]
],
[ [1,2,3],
[4,5,6],
[7,8,9]
,
flatten(x) = 1., 2., 3., 4., 5., 6., 7., 8., 9.],
[ 1., 2., 3., 4., 5., 6., 7., 8., 9.
Defined in src/operator/tensor/matrix_op.cc:L258
Input array.
org.apache.mxnet.Symbol
Applies a linear transformation: :math:Y = XW^T + b
.
If flatten
is set to be true, then the shapes are:
(batch_size, x1, x2, ..., xn)
 **data**:
(num_hidden, x1 * x2 * ... * xn)
 **weight**:
(num_hidden,)
 **bias**:
(batch_size, num_hidden)
 **out**:
If flatten
is set to be false, then the shapes are:
(x1, x2, ..., xn, input_dim)
 **data**:
(num_hidden, input_dim)
 **weight**:
(num_hidden,)
 **bias**:
(x1, x2, ..., xn, num_hidden)
 **out**:
The learnable parameters include both weight
and
bias
.
If no_bias
is set to be true, then the
bias
term is ignored.
row_sparse
Note that the operator also supports forward computation with weight and bias,
weight.indices
where the length of and
bias.indices must be equal to
num_hidden.
row_sparse
This could be used for model inference with weights trained with
SparseEmbedding.
Defined in src/operator/nn/fully_connected.cc:L257
Applies a linear transformation: :math:Y = XW^T + b
.
If flatten
is set to be true, then the shapes are:
(batch_size, x1, x2, ..., xn)
 **data**:
(num_hidden, x1 * x2 * ... * xn)
 **weight**:
(num_hidden,)
 **bias**:
(batch_size, num_hidden)
 **out**:
If flatten
is set to be false, then the shapes are:
(x1, x2, ..., xn, input_dim)
 **data**:
(num_hidden, input_dim)
 **weight**:
(num_hidden,)
 **bias**:
(x1, x2, ..., xn, num_hidden)
 **out**:
The learnable parameters include both weight
and
bias
.
If no_bias
is set to be true, then the
bias
term is ignored.
row_sparse
Note that the operator also supports forward computation with weight and bias,
weight.indices
where the length of and
bias.indices must be equal to
num_hidden.
row_sparse
This could be used for model inference with weights trained with
SparseEmbedding.
Defined in src/operator/nn/fully_connected.cc:L257
Input data.
Weight matrix.
Bias parameter.
Number of hidden nodes of the output.
Whether to disable bias parameter.
Whether to collapse all but the first axis of the input data tensor.
org.apache.mxnet.Symbol
Generates 2D sampling grid for bilinear sampling.
Generates 2D sampling grid for bilinear sampling.
Input data to the function.
The type of transformation. For affine
, input data should be an affine matrix of size (batch, 6). For warp
, input data should be an optical flow of size (batch, 2, h, w).
Specifies the output shape (H, W). This is required if transformation type is affine
. If transformation type is warp
, this parameter is ignored.
org.apache.mxnet.Symbol
Apply a sparse regularization to the output a sigmoid activation function.
Apply a sparse regularization to the output a sigmoid activation function.
Input data.
The sparseness target
The tradeoff parameter for the sparseness penalty
The momentum for running average
org.apache.mxnet.Symbol
Applies instance normalization to the ndimensional input array.
This operator takes an ndimensional input array where (n>2) and normalizes
the input using the following formula:
..
Applies instance normalization to the ndimensional input array.
This operator takes an ndimensional input array where (n>2) and normalizes
the input using the following formula:
.. math::
out = \frac{x  mean[data]}{ \sqrt{Var[data]} + \epsilon} * gamma + beta
This layer is similar to batch normalization layer (BatchNorm
)
with two differences: first, the normalization is
carried out per example (instance), not over a batch. Second, the
same normalization is applied both at test and train time. This
operation is also known as contrast normalization
.
If the input data is of shape [batch, channel, spacial_dim1, spacial_dim2, ...],
gamma
and beta
parameters must be vectors of shape [channel].
This implementation is based on paper:
.. [1] Instance Normalization: The Missing Ingredient for Fast Stylization,
D. Ulyanov, A. Vedaldi, V. Lempitsky, 2016 (arXiv:1607.08022v2).
Examples::
// Input of shape (2,1,2)
x = 1.1, 2.2]],
3.3, 4.4]
// gamma parameter of length 1
gamma = [1.5]
// beta parameter of length 1
beta = [0.5]
// Instance normalization is calculated with the above formula
InstanceNorm(x,gamma,beta) = , 1.99752665]],
1.99752724]
Defined in src/operator/instance_norm.cc:L95
An ndimensional input array (n > 2) of the form [batch, channel, spatial_dim1, spatial_dim2, ...].
A vector of length 'channel', which multiplies the normalized input.
A vector of length 'channel', which is added to the product of the normalized input and the weight.
An epsilon
parameter to prevent division by 0.
org.apache.mxnet.Symbol
Normalize the input array using the L2 norm.
For 1D NDArray, it computes::
out = data / sqrt(sum(data ** 2) + eps)
For ND NDArray, if the input array has shape (N, N, ..., N),
with
= mode
, it normalizes each instance in the multidimensionalinstance
array by its L2 norm.::
for i in 0...N
out[i,:,:,...,:] = data[i,:,:,...,:] / sqrt(sum(data[i,:,:,...,:] ** 2) + eps)
with
= mode
, it normalizes each channel in the array by its L2 norm.::channel
for i in 0...N
out[:,i,:,...,:] = data[:,i,:,...,:] / sqrt(sum(data[:,i,:,...,:] ** 2) + eps)
with
= mode
, it normalizes the cross channel norm for each positionspatial
in the array by its L2 norm.::
for dim in 2...N
for i in 0...N
out[.....,i,...] = take(out, indices=i, axis=dim) / sqrt(sum(take(out, indices=i, axis=dim) ** 2) + eps)
dim
Example::
x = [3,4]],
[5,6]
L2Normalization(x, mode='instance')
Normalize the input array using the L2 norm.
For 1D NDArray, it computes::
out = data / sqrt(sum(data ** 2) + eps)
For ND NDArray, if the input array has shape (N, N, ..., N),
with
= mode
, it normalizes each instance in the multidimensionalinstance
array by its L2 norm.::
for i in 0...N
out[i,:,:,...,:] = data[i,:,:,...,:] / sqrt(sum(data[i,:,:,...,:] ** 2) + eps)
with
= mode
, it normalizes each channel in the array by its L2 norm.::channel
for i in 0...N
out[:,i,:,...,:] = data[:,i,:,...,:] / sqrt(sum(data[:,i,:,...,:] ** 2) + eps)
with
= mode
, it normalizes the cross channel norm for each positionspatial
in the array by its L2 norm.::
for dim in 2...N
for i in 0...N
out[.....,i,...] = take(out, indices=i, axis=dim) / sqrt(sum(take(out, indices=i, axis=dim) ** 2) + eps)
dim
Example::
x = [3,4]],
[5,6]
L2Normalization(x, mode='instance')
Input array to normalize.
A small constant for numerical stability.
Specify the dimension along which to compute L2 norm.
org.apache.mxnet.Symbol
Applies local response normalization to the input.
The local response normalization layer performs "lateral inhibition" by normalizing
over local input regions.
If :math:a_{x,y}^{{i} is the activity of a neuron computed by applying kernel :math:i at position
:math:(x, y) and then applying the ReLU nonlinearity, the responsenormalized
activity :math:b_{x,y}}{i}
is given by the expression:
..
Applies local response normalization to the input.
The local response normalization layer performs "lateral inhibition" by normalizing
over local input regions.
If :math:a_{x,y}^{{i} is the activity of a neuron computed by applying kernel :math:i at position
:math:(x, y) and then applying the ReLU nonlinearity, the responsenormalized
activity :math:b_{x,y}}{i}
is given by the expression:
.. math::
b_{x,y}^{{i} = \frac{a_{x,y}}{i}}{\Bigg({k + \frac{\alpha}{n} \sum_{j=max(0, i\frac{n}{2})}^{{min(N1, i+\frac{n}{2})} (a_{x,y}}{j})^{{2}}\Bigg)}{\beta}}
where the sum runs over :math:n
"adjacent" kernel maps at the same spatial position, and :math:N
is the total
number of kernels in the layer.
Defined in src/operator/nn/lrn.cc:L175
Input data to LRN
The variance scaling parameter :math:lpha
in the LRN expression.
The power parameter :math:eta
in the LRN expression.
The parameter :math:k
in the LRN expression.
normalization window width in elements.
org.apache.mxnet.Symbol
Layer normalization.
Normalizes the channels of the input tensor by mean and variance, and applies a scale
asgamma
well as offset
.beta
Assume the input has more than one dimension and we normalize along axis 1.
We first compute the mean and variance along this axis and then
compute the normalized output, which has the same shape as input, as following:
..
Layer normalization.
Normalizes the channels of the input tensor by mean and variance, and applies a scale
asgamma
well as offset
.beta
Assume the input has more than one dimension and we normalize along axis 1.
We first compute the mean and variance along this axis and then
compute the normalized output, which has the same shape as input, as following:
.. math::
out = \frac{data  mean(data, axis)}{\sqrt{var(data, axis) + \epsilon}} * gamma + beta
Both
and gamma
are learnable parameters.beta
Unlike BatchNorm and InstanceNorm, the *mean* and *var* are computed along the channel dimension.
Assume the input has size *k* on axis 1, then both
and gamma
beta
have shape *(k,)*. If
is set to be true, then outputs both output_mean_var
anddata_mean
. Note that no gradient will be passed through these two outputs.data_std
The parameter
specifies which axis of the input shape denotesaxis
the 'channel' (separately normalized groups). The default is 1, which sets the channel
axis to be the last item in the input shape.
Defined in src/operator/nn/layer_norm.cc:L94
Input data to layer normalization
gamma array
beta array
The axis to perform layer normalization. Usually, this should be be axis of the channel dimension. Negative values means indexing from right to left.
An epsilon
parameter to prevent division by 0.
Output the mean and std calculated along the given axis.
org.apache.mxnet.Symbol
Applies Leaky rectified linear unit activation elementwise to the input.
Leaky ReLUs attempt to fix the "dying ReLU" problem by allowing a small slope
when the input is negative and has a slope of one when input is positive.
The following modified ReLU Activation functions are supported:
 *elu*: Exponential Linear Unit.
Applies Leaky rectified linear unit activation elementwise to the input.
Leaky ReLUs attempt to fix the "dying ReLU" problem by allowing a small slope
when the input is negative and has a slope of one when input is positive.
The following modified ReLU Activation functions are supported:
 *elu*: Exponential Linear Unit. y = x > 0 ? x : slope * (exp(x)1)
 *leaky*: Leaky ReLU. y = x > 0 ? x : slope * x
 *prelu*: Parametric ReLU. This is same as *leaky* except that slope
is learnt during training.
 *rrelu*: Randomized ReLU. same as *leaky* but the slope
is uniformly and randomly chosen from
*[lower_bound, upper_bound)* for training, while fixed to be
*(lower_bound+upper_bound)/2* for inference.
Defined in src/operator/leaky_relu.cc:L63
Input data to activation function.
Slope parameter for PReLU. Only required when act_type is 'prelu'. It should be either a vector of size 1, or the same size as the second dimension of data.
Activation function to be applied.
Init slope for the activation. (For leaky and elu only)
Lower bound of random slope. (For rrelu only)
Upper bound of random slope. (For rrelu only)
org.apache.mxnet.Symbol
Computes and optimizes for squared loss during backward propagation.
Just outputs
during forward propagation.data
If :math:\hat{y}_i
is the predicted value of the ith sample, and :math:y_i
is the corresponding target value,
then the squared loss estimated over :math:n
samples is defined as
:math:\text{SquaredLoss}(\textbf{Y}, \hat{\textbf{Y}} ) = \frac{1}{n} \sum_{i=0}^{n1} \lVert \textbf{y}_i  \hat{\textbf{y}}_i \rVert_2
.. note::
Use the LinearRegressionOutput as the final output layer of a net.
The storage type of label
can be
default
or
csr
1/m
 LinearRegressionOutput(default, default) = default
 LinearRegressionOutput(default, csr) = default
By default, gradients of this loss function are scaled by factor , where m is the number of regression outputs of a training example.
grad_scale
The parameter can be used to change this scale to
grad_scale/m.
Defined in src/operator/regression_output.cc:L92
Computes and optimizes for squared loss during backward propagation.
Just outputs
during forward propagation.data
If :math:\hat{y}_i
is the predicted value of the ith sample, and :math:y_i
is the corresponding target value,
then the squared loss estimated over :math:n
samples is defined as
:math:\text{SquaredLoss}(\textbf{Y}, \hat{\textbf{Y}} ) = \frac{1}{n} \sum_{i=0}^{n1} \lVert \textbf{y}_i  \hat{\textbf{y}}_i \rVert_2
.. note::
Use the LinearRegressionOutput as the final output layer of a net.
The storage type of label
can be
default
or
csr
1/m
 LinearRegressionOutput(default, default) = default
 LinearRegressionOutput(default, csr) = default
By default, gradients of this loss function are scaled by factor , where m is the number of regression outputs of a training example.
grad_scale
The parameter can be used to change this scale to
grad_scale/m.
Defined in src/operator/regression_output.cc:L92
Input data to the function.
Input label to the function.
Scale the gradient by a float factor
org.apache.mxnet.Symbol
Applies a logistic function to the input.
The logistic function, also known as the sigmoid function, is computed as
:math:\frac{1}{1+exp(\textbf{x})}
.
Commonly, the sigmoid is used to squash the realvalued output of a linear model
:math:wTx+b
into the [0,1] range so that it can be interpreted as a probability.
It is suitable for binary classification or probability prediction tasks.
..
Applies a logistic function to the input.
The logistic function, also known as the sigmoid function, is computed as
:math:\frac{1}{1+exp(\textbf{x})}
.
Commonly, the sigmoid is used to squash the realvalued output of a linear model
:math:wTx+b
into the [0,1] range so that it can be interpreted as a probability.
It is suitable for binary classification or probability prediction tasks.
.. note::
Use the LogisticRegressionOutput as the final output layer of a net.
The storage type of
can be label
or default
csr
 LogisticRegressionOutput(default, default) = default
 LogisticRegressionOutput(default, csr) = default
By default, gradients of this loss function are scaled by factor 1/m
, where m is the number of regression outputs of a training example.
The parameter grad_scale
can be used to change this scale to grad_scale/m
.
Defined in src/operator/regression_output.cc:L148
Input data to the function.
Input label to the function.
Scale the gradient by a float factor
org.apache.mxnet.Symbol
Computes mean absolute error of the input.
MAE is a risk metric corresponding to the expected value of the absolute error.
If :math:\hat{y}_i
is the predicted value of the ith sample, and :math:y_i
is the corresponding target value,
then the mean absolute error (MAE) estimated over :math:n
samples is defined as
:math:\text{MAE}(\textbf{Y}, \hat{\textbf{Y}} ) = \frac{1}{n} \sum_{i=0}^{n1} \lVert \textbf{y}_i  \hat{\textbf{y}}_i \rVert_1
.. note::
Use the MAERegressionOutput as the final output layer of a net.
The storage type of label
can be
default
or
csr
1/m
 MAERegressionOutput(default, default) = default
 MAERegressionOutput(default, csr) = default
By default, gradients of this loss function are scaled by factor , where m is the number of regression outputs of a training example.
grad_scale
The parameter can be used to change this scale to
grad_scale/m.
Defined in src/operator/regression_output.cc:L120
Computes mean absolute error of the input.
MAE is a risk metric corresponding to the expected value of the absolute error.
If :math:\hat{y}_i
is the predicted value of the ith sample, and :math:y_i
is the corresponding target value,
then the mean absolute error (MAE) estimated over :math:n
samples is defined as
:math:\text{MAE}(\textbf{Y}, \hat{\textbf{Y}} ) = \frac{1}{n} \sum_{i=0}^{n1} \lVert \textbf{y}_i  \hat{\textbf{y}}_i \rVert_1
.. note::
Use the MAERegressionOutput as the final output layer of a net.
The storage type of label
can be
default
or
csr
1/m
 MAERegressionOutput(default, default) = default
 MAERegressionOutput(default, csr) = default
By default, gradients of this loss function are scaled by factor , where m is the number of regression outputs of a training example.
grad_scale
The parameter can be used to change this scale to
grad_scale/m.
Defined in src/operator/regression_output.cc:L120
Input data to the function.
Input label to the function.
Scale the gradient by a float factor
org.apache.mxnet.Symbol
Make your own loss function in network construction.
This operator accepts a customized loss function symbol as a terminal loss and
the symbol should be an operator with no backward dependency.
The output of this function is the gradient of loss with respect to the input data.
For example, if you are a making a cross entropy loss function.
Make your own loss function in network construction.
This operator accepts a customized loss function symbol as a terminal loss and
the symbol should be an operator with no backward dependency.
The output of this function is the gradient of loss with respect to the input data.
For example, if you are a making a cross entropy loss function. Assume
is theout
predicted output and
is the true label, then the cross entropy can be defined as::label
cross_entropy = label * log(out) + (1  label) * log(1  out)
loss = MakeLoss(cross_entropy)
We will need to use
when we are creating our own loss function or we want toMakeLoss
combine multiple loss functions. Also we may want to stop some variables' gradients
from backpropagation. See more detail in
or BlockGrad
.stop_gradient
In addition, we can give a scale to the loss by setting
,grad_scale
so that the gradient of the loss will be rescaled in the backpropagation.
.. note:: This operator should be used as a Symbol instead of NDArray.
Defined in src/operator/make_loss.cc:L71
Input array.
Gradient scale as a supplement to unary and binary operators
clip each element in the array to 0 when it is less than
. This is used when valid_thresh
is set to normalization
.'valid'
If this is set to null, the output gradient will not be normalized. If this is set to batch, the output gradient will be divided by the batch size. If this is set to valid, the output gradient will be divided by the number of valid input elements.
org.apache.mxnet.Symbol
Pads an input array with a constant or edge values of the array.
..
Pads an input array with a constant or edge values of the array.
.. note:: Pad
is deprecated. Use pad
instead.
.. note:: Current implementation only supports 4D and 5D input arrays with padding applied
only on axes 1, 2 and 3. Expects axes 4 and 5 in pad_width
to be zero.
This operation pads an input array with either a constant_value
or edge values
along each axis of the input array. The amount of padding is specified by pad_width
.
pad_width
is a tuple of integer padding widths for each axis of the format
. The (before_1, after_1, ... , before_N, after_N)
pad_width
should be of length 2*N
where
is the number of dimensions of the array.N
For dimension
of the input array, N
and before_N
indicates how many valuesafter_N
to add before and after the elements of the array along dimension
.N
The widths of the higher two dimensions
, before_1
, after_1
,before_2
must be 0.after_2
Example::
x = 1. 2. 3.]
[ 4. 5. 6.]]
7. 8. 9.]
[ 10. 11. 12.]
11. 12. 13.]
[ 14. 15. 16.]]
17. 18. 19.]
[ 20. 21. 22.]]
pad(x,mode="edge", pad_width=(0,0,0,0,1,1,1,1)) =
1. 1. 2. 3. 3.]
[ 1. 1. 2. 3. 3.]
[ 4. 4. 5. 6. 6.]
[ 4. 4. 5. 6. 6.]]
7. 7. 8. 9. 9.]
[ 7. 7. 8. 9. 9.]
[ 10. 10. 11. 12. 12.]
[ 10. 10. 11. 12. 12.]
11. 11. 12. 13. 13.]
[ 11. 11. 12. 13. 13.]
[ 14. 14. 15. 16. 16.]
[ 14. 14. 15. 16. 16.]]
17. 17. 18. 19. 19.]
[ 17. 17. 18. 19. 19.]
[ 20. 20. 21. 22. 22.]
[ 20. 20. 21. 22. 22.]]
pad(x, mode="constant", constant_value=0, pad_width=(0,0,0,0,1,1,1,1)) =
0. 0. 0. 0. 0.]
[ 0. 1. 2. 3. 0.]
[ 0. 4. 5. 6. 0.]
[ 0. 0. 0. 0. 0.]]
0. 0. 0. 0. 0.]
[ 0. 7. 8. 9. 0.]
[ 0. 10. 11. 12. 0.]
[ 0. 0. 0. 0. 0.]
0. 0. 0. 0. 0.]
[ 0. 11. 12. 13. 0.]
[ 0. 14. 15. 16. 0.]
[ 0. 0. 0. 0. 0.]]
0. 0. 0. 0. 0.]
[ 0. 17. 18. 19. 0.]
[ 0. 20. 21. 22. 0.]
[ 0. 0. 0. 0. 0.]]
Defined in src/operator/pad.cc:L766
An ndimensional input array.
Padding type to use. "constant" pads with constant_value
"edge" pads using the edge values of the input array "reflect" pads by reflecting values with respect to the edges.
Widths of the padding regions applied to the edges of each axis. It is a tuple of integer padding widths for each axis of the format
. It should be of length (before_1, after_1, ... , before_N, after_N)
where 2*N
is the number of dimensions of the array.This is equivalent to pad_width in numpy.pad, but flattened.N
The value used for padding when mode
is "constant".
org.apache.mxnet.Symbol
Performs pooling on the input.
The shapes for 1D pooling are
 **data**: *(batch_size, channel, width)*,
 **out**: *(batch_size, num_filter, out_width)*.
The shapes for 2D pooling are
 **data**: *(batch_size, channel, height, width)*
 **out**: *(batch_size, num_filter, out_height, out_width)*, with::
out_height = f(height, kernel[0], pad[0], stride[0])
out_width = f(width, kernel[1], pad[1], stride[1])
The definition of *f* depends on
, which has two options:pooling_convention
 **valid** (default)::
f(x, k, p, s) = floor((x+2*pk)/s)+1
 **full**, which is compatible with Caffe::
f(x, k, p, s) = ceil((x+2*pk)/s)+1
But
is set to be true, then do a global pooling, namely resetglobal_pool
.kernel=(height, width)
Three pooling options are supported by
:pool_type
 **avg**: average pooling
 **max**: max pooling
 **sum**: sum pooling
 **lp**: Lp pooling
For 3D pooling, an additional *depth* dimension is added before
*height*.
Performs pooling on the input.
The shapes for 1D pooling are
 **data**: *(batch_size, channel, width)*,
 **out**: *(batch_size, num_filter, out_width)*.
The shapes for 2D pooling are
 **data**: *(batch_size, channel, height, width)*
 **out**: *(batch_size, num_filter, out_height, out_width)*, with::
out_height = f(height, kernel[0], pad[0], stride[0])
out_width = f(width, kernel[1], pad[1], stride[1])
The definition of *f* depends on
, which has two options:pooling_convention
 **valid** (default)::
f(x, k, p, s) = floor((x+2*pk)/s)+1
 **full**, which is compatible with Caffe::
f(x, k, p, s) = ceil((x+2*pk)/s)+1
But
is set to be true, then do a global pooling, namely resetglobal_pool
.kernel=(height, width)
Three pooling options are supported by
:pool_type
 **avg**: average pooling
 **max**: max pooling
 **sum**: sum pooling
 **lp**: Lp pooling
For 3D pooling, an additional *depth* dimension is added before
*height*. Namely the input data will have shape *(batch_size, channel, depth,
height, width)*.
Notes on Lp pooling:
Lp pooling was first introduced by this paper: https://arxiv.org/pdf/1204.3968.pdf.
L1 pooling is simply sum pooling, while Linf pooling is simply max pooling.
We can see that Lp pooling stands between those two, in practice the most common value for p is 2.
For each window
, the mathematical expression for Lp pooling is:X
..math::
f(X) = \sqrt{p}{\sum\limits_{x \in X} x^p}
Defined in src/operator/nn/pooling.cc:L383
Input data to the pooling operator.
Pooling kernel size: (y, x) or (d, y, x)
Pooling type to be applied.
Ignore kernel size, do global pooling based on current input feature map.
Turn off cudnn pooling and use MXNet pooling operator.
Pooling convention to be applied.
Stride: for pooling (y, x) or (d, y, x). Defaults to 1 for each dimension.
Pad for pooling: (y, x) or (d, y, x). Defaults to no padding.
Value of p for Lp pooling, can be 1 or 2, required for Lp Pooling.
Only used for AvgPool, specify whether to count padding elements for averagecalculation. For example, with a 5*5 kernel on a 3*3 corner of a image,the sum of the 9 valid elements will be divided by 25 if this is set to true,or it will be divided by 9 if this is set to false. Defaults to true.
org.apache.mxnet.Symbol
This operator is DEPRECATED.
Perform pooling on the input.
The shapes for 2D pooling is
 **data**: *(batch_size, channel, height, width)*
 **out**: *(batch_size, num_filter, out_height, out_width)*, with::
out_height = f(height, kernel[0], pad[0], stride[0])
out_width = f(width, kernel[1], pad[1], stride[1])
The definition of *f* depends on
, which has two options:pooling_convention
 **valid** (default)::
f(x, k, p, s) = floor((x+2*pk)/s)+1
 **full**, which is compatible with Caffe::
f(x, k, p, s) = ceil((x+2*pk)/s)+1
But
is set to be true, then do a global pooling, namely resetglobal_pool
.kernel=(height, width)
Three pooling options are supported by
:pool_type
 **avg**: average pooling
 **max**: max pooling
 **sum**: sum pooling
1D pooling is special case of 2D pooling with *weight=1* and
*kernel[1]=1*.
For 3D pooling, an additional *depth* dimension is added before
*height*.
This operator is DEPRECATED.
Perform pooling on the input.
The shapes for 2D pooling is
 **data**: *(batch_size, channel, height, width)*
 **out**: *(batch_size, num_filter, out_height, out_width)*, with::
out_height = f(height, kernel[0], pad[0], stride[0])
out_width = f(width, kernel[1], pad[1], stride[1])
The definition of *f* depends on
, which has two options:pooling_convention
 **valid** (default)::
f(x, k, p, s) = floor((x+2*pk)/s)+1
 **full**, which is compatible with Caffe::
f(x, k, p, s) = ceil((x+2*pk)/s)+1
But
is set to be true, then do a global pooling, namely resetglobal_pool
.kernel=(height, width)
Three pooling options are supported by
:pool_type
 **avg**: average pooling
 **max**: max pooling
 **sum**: sum pooling
1D pooling is special case of 2D pooling with *weight=1* and
*kernel[1]=1*.
For 3D pooling, an additional *depth* dimension is added before
*height*. Namely the input data will have shape *(batch_size, channel, depth,
height, width)*.
Defined in src/operator/pooling_v1.cc:L104
Input data to the pooling operator.
pooling kernel size: (y, x) or (d, y, x)
Pooling type to be applied.
Ignore kernel size, do global pooling based on current input feature map.
Pooling convention to be applied.
stride: for pooling (y, x) or (d, y, x)
pad for pooling: (y, x) or (d, y, x)
org.apache.mxnet.Symbol
Applies recurrent layers to input data.
Applies recurrent layers to input data. Currently, vanilla RNN, LSTM and GRU are
implemented, with both multilayer and bidirectional support.
**Vanilla RNN**
Applies a singlegate recurrent layer to input X. Two kinds of activation function are supported:
ReLU and Tanh.
With ReLU activation function:
.. math::
h_t = relu(W_{ih} * x_t + b_{ih} + W_{hh} * h_{(t1)} + b_{hh})
With Tanh activtion function:
.. math::
h_t = \tanh(W_{ih} * x_t + b_{ih} + W_{hh} * h_{(t1)} + b_{hh})
Reference paper: Finding structure in time  Elman, 1988.
https://crl.ucsd.edu/~elman/Papers/fsit.pdf
**LSTM**
Long ShortTerm Memory  Hochreiter, 1997. http://www.bioinf.jku.at/publications/older/2604.pdf
.. math::
\begin{array}{ll}
i_t = \mathrm{sigmoid}(W_{ii} x_t + b_{ii} + W_{hi} h_{(t1)} + b_{hi}) \\
f_t = \mathrm{sigmoid}(W_{if} x_t + b_{if} + W_{hf} h_{(t1)} + b_{hf}) \\
g_t = \tanh(W_{ig} x_t + b_{ig} + W_{hc} h_{(t1)} + b_{hg}) \\
o_t = \mathrm{sigmoid}(W_{io} x_t + b_{io} + W_{ho} h_{(t1)} + b_{ho}) \\
c_t = f_t * c_{(t1)} + i_t * g_t \\
h_t = o_t * \tanh(c_t)
\end{array}
**GRU**
Gated Recurrent Unit  Cho et al. 2014. http://arxiv.org/abs/1406.1078
The definition of GRU here is slightly different from paper but compatible with CUDNN.
.. math::
\begin{array}{ll}
r_t = \mathrm{sigmoid}(W_{ir} x_t + b_{ir} + W_{hr} h_{(t1)} + b_{hr}) \\
z_t = \mathrm{sigmoid}(W_{iz} x_t + b_{iz} + W_{hz} h_{(t1)} + b_{hz}) \\
n_t = \tanh(W_{in} x_t + b_{in} + r_t * (W_{hn} h_{(t1)}+ b_{hn})) \\
h_t = (1  z_t) * n_t + z_t * h_{(t1)} \\
\end{array}
Input data to RNN
Vector of all RNN trainable parameters concatenated
initial hidden state of the RNN
initial cell state for LSTM networks (only for LSTM)
size of the state for each layer
number of stacked layers
whether to use bidirectional recurrent layers
the type of RNN to compute
Dropout probability, fraction of the input that gets dropped out at training time
Whether to have the states as symbol outputs.
org.apache.mxnet.Symbol
Performs region of interest(ROI) pooling on the input array.
ROI pooling is a variant of a max pooling layer, in which the output size is fixed and
region of interest is a parameter.
Performs region of interest(ROI) pooling on the input array.
ROI pooling is a variant of a max pooling layer, in which the output size is fixed and
region of interest is a parameter. Its purpose is to perform max pooling on the inputs
of nonuniform sizes to obtain fixedsize feature maps. ROI pooling is a neuralnet
layer mostly used in training a Fast RCNN
network for object detection.
This operator takes a 4D feature map as an input array and region proposals as rois
,
then it pools over subregions of input and produces a fixedsized output array
regardless of the ROI size.
To crop the feature map accordingly, you can resize the bounding box coordinates
by changing the parameters rois
and spatial_scale
.
The cropped feature maps are pooled by standard max pooling operation to a fixed size output
indicated by a pooled_size
parameter. batch_size will change to the number of region
bounding boxes after ROIPooling
.
The size of each region of interest doesn't have to be perfectly divisible by
the number of pooling sections(pooled_size
).
Example::
x = 0., 1., 2., 3., 4., 5.],
[ 6., 7., 8., 9., 10., 11.],
[ 12., 13., 14., 15., 16., 17.],
[ 18., 19., 20., 21., 22., 23.],
[ 24., 25., 26., 27., 28., 29.],
[ 30., 31., 32., 33., 34., 35.],
[ 36., 37., 38., 39., 40., 41.],
[ 42., 43., 44., 45., 46., 47.
// region of interest i.e. bounding box coordinates.
y = 0,0,0,4,4
// returns array of shape (2,2) according to the given roi with max pooling.
ROIPooling(x, y, (2,2), 1.0) = 14., 16.],
[ 26., 28.
// region of interest is changed due to the change in spacial_scale
parameter.
ROIPooling(x, y, (2,2), 0.7) = 7., 9.],
[ 19., 21.
Defined in src/operator/roi_pooling.cc:L295
The input array to the pooling operator, a 4D Feature maps
Bounding box coordinates, a 2D array of x1, y1, x2, y2, where (x1, y1) and (x2, y2) are top left and bottom right corners of designated region of interest. batch_index
indicates the index of corresponding image in the input array
ROI pooling output shape (h,w)
Ratio of input feature map height (or w) to raw image height (or w). Equals the reciprocal of total stride in convolutional layers
org.apache.mxnet.Symbol
Reshapes the input array.
..
Reshapes the input array.
.. note::
is deprecated, use Reshape
reshape
Given an array and a shape, this function returns a copy of the array in the new shape.
The shape is a tuple of integers such as (2,3,4). The size of the new shape should be same as the size of the input array.
Example::
reshape([1,2,3,4], shape=(2,2)) = [3,4
Some dimensions of the shape can take special values from the set {0, 1, 2, 3, 4}. The significance of each is explained below:

copy this dimension from the input to the output shape.0
Example::
1
infers the dimension of the output shape by using the remainder of the input dimensions2
copy all/remainder of the input dimensions to the output shape.3
use the product of two consecutive dimensions of the input shape as the output dimension.4
split one dimension of the input into two dimensions passed subsequent to 4 in shape (can contain 1).reverse
is set to 1, then the special values are inferred from right to left.Input data to reshape.
The target shape
If true then the special values are inferred from right to left
(Deprecated! Use
instead.) Target new shape. One and only one dim can be 0, in which case it will be inferred from the rest of dimsshape
(Deprecated! Use
instead.) Whether keep the highest dim unchanged.If set to true, then the first dim in target_shape is ignored,and always fixed as inputshape
org.apache.mxnet.Symbol
Computes support vector machine based transformation of the input.
This tutorial demonstrates using SVM as output layer for classification instead of softmax:
https://github.com/dmlc/mxnet/tree/master/example/svm_mnist.
Computes support vector machine based transformation of the input.
This tutorial demonstrates using SVM as output layer for classification instead of softmax:
https://github.com/dmlc/mxnet/tree/master/example/svm_mnist.
Input data for SVM transformation.
Class label for the input data.
The loss function penalizes outputs that lie outside this margin. Default margin is 1.
Regularization parameter for the SVM. This balances the tradeoff between coefficient size and error.
Whether to use L1SVM objective. L2SVM objective is used by default.
org.apache.mxnet.Symbol
Takes the last element of a sequence.
This function takes an ndimensional input array of the form
[max_sequence_length, batch_size, other_feature_dims] and returns a (n1)dimensional array
of the form [batch_size, other_feature_dims].
Parameter sequence_length
is used to handle variablelength sequences.
Takes the last element of a sequence.
This function takes an ndimensional input array of the form
[max_sequence_length, batch_size, other_feature_dims] and returns a (n1)dimensional array
of the form [batch_size, other_feature_dims].
Parameter sequence_length
is used to handle variablelength sequences. sequence_length
should be
an input array of positive ints of dimension [batch_size]. To use this parameter,
set use_sequence_length
to True
, otherwise each example in the batch is assumed
to have the max sequence length.
.. note:: Alternatively, you can also use take
operator.
Example::
x = 1., 2., 3.],
[ 4., 5., 6.],
[ 7., 8., 9.]],
10., 11., 12.],
[ 13., 14., 15.],
[ 16., 17., 18.,
19., 20., 21.],
[ 22., 23., 24.],
[ 25., 26., 27.]
// returns last sequence when sequence_length parameter is not used
SequenceLast(x) = 19., 20., 21.],
[ 22., 23., 24.],
[ 25., 26., 27.
// sequence_length is used
SequenceLast(x, sequence_length=[1,1,1], use_sequence_length=True) =
1., 2., 3.],
[ 4., 5., 6.],
[ 7., 8., 9.
// sequence_length is used
SequenceLast(x, sequence_length=[1,2,3], use_sequence_length=True) =
1., 2., 3.],
[ 13., 14., 15.],
[ 25., 26., 27.
Defined in src/operator/sequence_last.cc:L92
ndimensional input array of the form [max_sequence_length, batch_size, other_feature_dims] where n>2
vector of sequence lengths of the form [batch_size]
If set to true, this layer takes in an extra input parameter sequence_length
to specify variable length sequence
The sequence axis. Only values of 0 and 1 are currently supported.
org.apache.mxnet.Symbol
Sets all elements outside the sequence to a constant value.
This function takes an ndimensional input array of the form
[max_sequence_length, batch_size, other_feature_dims] and returns an array of the same shape.
Parameter sequence_length
is used to handle variablelength sequences.
Sets all elements outside the sequence to a constant value.
This function takes an ndimensional input array of the form
[max_sequence_length, batch_size, other_feature_dims] and returns an array of the same shape.
Parameter sequence_length
is used to handle variablelength sequences. sequence_length
should be an input array of positive ints of dimension [batch_size].
To use this parameter, set use_sequence_length
to True
,
otherwise each example in the batch is assumed to have the max sequence length and
this operator works as the identity
operator.
Example::
x = 1., 2., 3.],
[ 4., 5., 6.]],
7., 8., 9.],
[ 10., 11., 12.,
13., 14., 15.],
[ 16., 17., 18.]
// Batch 1
B1 = 1., 2., 3.],
[ 7., 8., 9.],
[ 13., 14., 15.
// Batch 2
B2 = 4., 5., 6.],
[ 10., 11., 12.],
[ 16., 17., 18.
// works as identity operator when sequence_length parameter is not used
SequenceMask(x) = 1., 2., 3.],
[ 4., 5., 6.]],
7., 8., 9.],
[ 10., 11., 12.,
13., 14., 15.],
[ 16., 17., 18.]
// sequence_length [1,1] means 1 of each batch will be kept
// and other rows are masked with default mask value = 0
SequenceMask(x, sequence_length=[1,1], use_sequence_length=True) =
1., 2., 3.],
[ 4., 5., 6.]],
0., 0., 0.],
[ 0., 0., 0.,
0., 0., 0.],
[ 0., 0., 0.]
// sequence_length [2,3] means 2 of batch B1 and 3 of batch B2 will be kept
// and other rows are masked with value = 1
SequenceMask(x, sequence_length=[2,3], use_sequence_length=True, value=1) =
1., 2., 3.],
[ 4., 5., 6.]],
7., 8., 9.],
[ 10., 11., 12.,
1., 1., 1.],
[ 16., 17., 18.]
Defined in src/operator/sequence_mask.cc:L114
ndimensional input array of the form [max_sequence_length, batch_size, other_feature_dims] where n>2
vector of sequence lengths of the form [batch_size]
If set to true, this layer takes in an extra input parameter sequence_length
to specify variable length sequence
The value to be used as a mask.
The sequence axis. Only values of 0 and 1 are currently supported.
org.apache.mxnet.Symbol
Reverses the elements of each sequence.
This function takes an ndimensional input array of the form [max_sequence_length, batch_size, other_feature_dims]
and returns an array of the same shape.
Parameter sequence_length
is used to handle variablelength sequences.
sequence_length
should be an input array of positive ints of dimension [batch_size].
To use this parameter, set use_sequence_length
to True
,
otherwise each example in the batch is assumed to have the max sequence length.
Example::
x = 1., 2., 3.],
[ 4., 5., 6.]],
7., 8., 9.],
[ 10., 11., 12.,
13., 14., 15.],
[ 16., 17., 18.]
// Batch 1
B1 = 1., 2., 3.],
[ 7., 8., 9.],
[ 13., 14., 15.
// Batch 2
B2 = 4., 5., 6.],
[ 10., 11., 12.],
[ 16., 17., 18.
// returns reverse sequence when sequence_length parameter is not used
SequenceReverse(x) = 13., 14., 15.],
[ 16., 17., 18.]],
7., 8., 9.],
[ 10., 11., 12.,
1., 2., 3.],
[ 4., 5., 6.]
// sequence_length [2,2] means 2 rows of
// both batch B1 and B2 will be reversed.
SequenceReverse(x, sequence_length=[2,2], use_sequence_length=True) =
7., 8., 9.],
[ 10., 11., 12.]],
1., 2., 3.],
[ 4., 5., 6.,
13., 14., 15.],
[ 16., 17., 18.]
// sequence_length [2,3] means 2 of batch B2 and 3 of batch B3
// will be reversed.
SequenceReverse(x, sequence_length=[2,3], use_sequence_length=True) =
7., 8., 9.],
[ 16., 17., 18.]],
1., 2., 3.],
[ 10., 11., 12.,
13., 14, 15.],
[ 4., 5., 6.]
Defined in src/operator/sequence_reverse.cc:L113
Reverses the elements of each sequence.
This function takes an ndimensional input array of the form [max_sequence_length, batch_size, other_feature_dims]
and returns an array of the same shape.
Parameter sequence_length
is used to handle variablelength sequences.
sequence_length
should be an input array of positive ints of dimension [batch_size].
To use this parameter, set use_sequence_length
to True
,
otherwise each example in the batch is assumed to have the max sequence length.
Example::
x = 1., 2., 3.],
[ 4., 5., 6.]],
7., 8., 9.],
[ 10., 11., 12.,
13., 14., 15.],
[ 16., 17., 18.]
// Batch 1
B1 = 1., 2., 3.],
[ 7., 8., 9.],
[ 13., 14., 15.
// Batch 2
B2 = 4., 5., 6.],
[ 10., 11., 12.],
[ 16., 17., 18.
// returns reverse sequence when sequence_length parameter is not used
SequenceReverse(x) = 13., 14., 15.],
[ 16., 17., 18.]],
7., 8., 9.],
[ 10., 11., 12.,
1., 2., 3.],
[ 4., 5., 6.]
// sequence_length [2,2] means 2 rows of
// both batch B1 and B2 will be reversed.
SequenceReverse(x, sequence_length=[2,2], use_sequence_length=True) =
7., 8., 9.],
[ 10., 11., 12.]],
1., 2., 3.],
[ 4., 5., 6.,
13., 14., 15.],
[ 16., 17., 18.]
// sequence_length [2,3] means 2 of batch B2 and 3 of batch B3
// will be reversed.
SequenceReverse(x, sequence_length=[2,3], use_sequence_length=True) =
7., 8., 9.],
[ 16., 17., 18.]],
1., 2., 3.],
[ 10., 11., 12.,
13., 14, 15.],
[ 4., 5., 6.]
Defined in src/operator/sequence_reverse.cc:L113
ndimensional input array of the form [max_sequence_length, batch_size, other dims] where n>2
vector of sequence lengths of the form [batch_size]
If set to true, this layer takes in an extra input parameter sequence_length
to specify variable length sequence
The sequence axis. Only 0 is currently supported.
org.apache.mxnet.Symbol
Splits an array along a particular axis into multiple subarrays.
..
Splits an array along a particular axis into multiple subarrays.
.. note::
is deprecated. Use SliceChannel
instead.split
**Note** that num_outputs
should evenly divide the length of the axis
along which to split the array.
Example::
x = 1.]
[ 2.]]
3.]
[ 4.
5.]
[ 6.]
x.shape = (3, 2, 1)
y = split(x, axis=1, num_outputs=2) // a list of 2 arrays with shape (3, 1, 1)
y = 1.]]
3.
5.]
2.]]
4.
6.]
y[0].shape = (3, 1, 1)
z = split(x, axis=0, num_outputs=3) // a list of 3 arrays with shape (1, 2, 1)
z = 1.]
[ 2.
3.]
[ 4.
5.]
[ 6.
z[0].shape = (1, 2, 1)
squeeze_axis=1
removes the axis with length 1 from the shapes of the output arrays.
**Note** that setting squeeze_axis
to
removes axis with length 1 only1
along the axis
which it is split.
Also squeeze_axis
can be set to true only if
.input.shape[axis] == num_outputs
Example::
z = split(x, axis=0, num_outputs=3, squeeze_axis=1) // a list of 3 arrays with shape (2, 1)
z = 1.]
[ 2.
3.]
[ 4.
5.]
[ 6.
z[0].shape = (2 ,1 )
Defined in src/operator/slice_channel.cc:L107
The input
Number of splits. Note that this should evenly divide the length of the axis
.
Axis along which to split.
If true, Removes the axis with length 1 from the shapes of the output arrays. **Note** that setting squeeze_axis
to
removes axis with length 1 only along the true
axis
which it is split. Also squeeze_axis
can be set to
only if true
.input.shape[axis] == num_outputs
org.apache.mxnet.Symbol
Please use SoftmaxOutput
.
..
Please use SoftmaxOutput
.
.. note::
This operator has been renamed to SoftmaxOutput
, which
computes the gradient of crossentropy loss w.r.t softmax output.
To just compute softmax output, use the softmax
operator.
Defined in src/operator/softmax_output.cc:L138
Input array.
Scales the gradient by a float factor.
The instances whose labels
== ignore_label
will be ignored during backward, if use_ignore
is set to
).true
If set to
, the softmax function will be computed along axis true
. This is applied when the shape of input array differs from the shape of label array.1
If set to
, the true
ignore_label
value will not contribute to the backward gradient.
If set to
, the softmax function will be computed along the last axis (true
).1
Normalizes the gradient.
Multiplies gradient with output gradient elementwise.
Constant for computing a label smoothed version of crossentropyfor the backwards pass. This constant gets subtracted from theonehot encoding of the gold label and distributed uniformly toall other labels.
org.apache.mxnet.Symbol
Applies softmax activation to input.
Applies softmax activation to input. This is intended for internal layers.
.. note::
This operator has been deprecated, please use softmax
.
If mode
=
, this operator will compute a softmax for each instance in the batch.instance
This is the default mode.
If mode
=
, this operator will compute a kclass softmax at each positionchannel
of each instance, where k
=
. This mode can only be used when the input arraynum_channel
has at least 3 dimensions.
This can be used for fully convolutional network
, image segmentation
, etc.
Example::
>>> input_array = mx.nd.array(0.5, 0.5, 2., 7.],
>>> [2., .4, 7., 3., 0.2)
>>> softmax_act = mx.nd.SoftmaxActivation(input_array)
>>> print softmax_act.asnumpy()
1.78322066e02 1.46375655e03 5.38485940e04 6.56010211e03 9.73605454e01]
[ 6.56221947e03 5.95310994e04 9.73919690e01 1.78379621e02 1.08472735e03
Defined in src/operator/nn/softmax_activation.cc:L59
The input array.
Specifies how to compute the softmax. If set to
, it computes softmax for each instance. If set to instance
, It computes cross channel softmax for each position of each instance.channel
org.apache.mxnet.Symbol
Computes the gradient of cross entropy loss with respect to softmax output.
 This operator computes the gradient in two steps.
The cross entropy loss does not actually need to be computed.
Computes the gradient of cross entropy loss with respect to softmax output.
 This operator computes the gradient in two steps.
The cross entropy loss does not actually need to be computed.
(d_1, d_2, ..., d_n)
. The size iss=d_1 \cdot d_2 \cdot \cdot \cdot d_n
. We can use the parameters preserve_shape
multi_output
to specify the way to compute softmax:preserve_shape
is false
. This operator will reshape the input array(d_1, \frac{s}{d_1})
and then compute the softmax function for(d_1, d_2, ..., d_n)
.preserve_shape
is true
, the softmax function will be computed alongaxis
= 1
).multi_output
is true
, the softmax function will be computed alongaxis
= 1
).use_ignore
is true
, ignore_label
can specify input instancesoutput
has same shape as label
**.grad_scale
can be used to rescale the gradient, which is often used tonormalization
,normalization
is applied if softmax output has different shape than the labels.normalization
mode can be set to the followings:'null'
: do nothing.'batch'
: divide the gradient by the batch size.'valid'
: divide the gradient by the number of instances which are not ignored.Input array.
Ground truth label.
Scales the gradient by a float factor.
The instances whose labels
== ignore_label
will be ignored during backward, if use_ignore
is set to
).true
If set to
, the softmax function will be computed along axis true
. This is applied when the shape of input array differs from the shape of label array.1
If set to
, the true
ignore_label
value will not contribute to the backward gradient.
If set to
, the softmax function will be computed along the last axis (true
).1
Normalizes the gradient.
Multiplies gradient with output gradient elementwise.
Constant for computing a label smoothed version of crossentropyfor the backwards pass. This constant gets subtracted from theonehot encoding of the gold label and distributed uniformly toall other labels.
org.apache.mxnet.Symbol
Applies a spatial transformer to input feature map.
Applies a spatial transformer to input feature map.
Input data to the SpatialTransformerOp.
localisation net, the output dim should be 6 when transform_type is affine. You shold initialize the weight and bias with identity tranform.
output shape(h, w) of spatial transformer: (y, x)
transformation type
sampling type
org.apache.mxnet.Symbol
Interchanges two axes of an array.
Examples::
x = 2, 3)
swapaxes(x, 0, 1) = 1],
[ 2],
[ 3
x = 0, 1],
[ 2, 3]],
4, 5],
[ 6, 7] // (2,2,2) array
swapaxes(x, 0, 2) = 0, 4],
[ 2, 6]],
1, 5],
[ 3, 7]
Defined in src/operator/swapaxis.cc:L70
Interchanges two axes of an array.
Examples::
x = 2, 3)
swapaxes(x, 0, 1) = 1],
[ 2],
[ 3
x = 0, 1],
[ 2, 3]],
4, 5],
[ 6, 7] // (2,2,2) array
swapaxes(x, 0, 2) = 0, 4],
[ 2, 6]],
1, 5],
[ 3, 7]
Defined in src/operator/swapaxis.cc:L70
Input array.
the first axis to be swapped.
the second axis to be swapped.
org.apache.mxnet.Symbol
Performs nearest neighbor/bilinear up sampling to inputs.
Performs nearest neighbor/bilinear up sampling to inputs.
Array of tensors to upsample
Up sampling scale
Input filter. Only used by bilinear sample_type.
upsampling method
How to handle multiple input. concat means concatenate upsampled images along the channel dimension. sum means add all images together, only available for nearest neighbor upsampling.
Number of inputs to be upsampled. For nearest neighbor upsampling, this can be 1N; the size of output will be(scale*h_0,scale*w_0) and all other inputs will be upsampled to thesame size. For bilinear upsampling this must be 2; 1 input and 1 weight.
Tmp workspace for deconvolution (MB)
org.apache.mxnet.Symbol
Returns elementwise absolute value of the input.
Example::
abs([2, 0, 3]) = [2, 0, 3]
The storage type of
output depends upon the input storage type:abs
Returns elementwise absolute value of the input.
Example::
abs([2, 0, 3]) = [2, 0, 3]
The storage type of
output depends upon the input storage type:abs
The input array.
org.apache.mxnet.Symbol
Update function for Adam optimizer.
Update function for Adam optimizer. Adam is seen as a generalization
of AdaGrad.
Adam update consists of the following steps, where g represents gradient and m, v
are 1st and 2nd order moment estimates (mean and variance).
.. math::
g_t = \nabla J(W_{t1})\\
m_t = \beta_1 m_{t1} + (1  \beta_1) g_t\\
v_t = \beta_2 v_{t1} + (1  \beta_2) g_t^2\\
W_t = W_{t1}  \alpha \frac{ m_t }{ \sqrt{ v_t } + \epsilon }
It updates the weights using::
m = beta1*m + (1beta1)*grad
v = beta2*v + (1beta2)*(grad**2)
w +=  learning_rate * m / (sqrt(v) + epsilon)
However, if grad's storage type is
, row_sparse
is True and the storagelazy_update
type of weight is the same as those of m and v,
only the row slices whose indices appear in grad.indices are updated (for w, m and v)::
for row in grad.indices:
m[row] = beta1*m[row] + (1beta1)*grad[row]
v[row] = beta2*v[row] + (1beta2)*(grad[row]**2)
w[row] +=  learning_rate * m[row] / (sqrt(v[row]) + epsilon)
Defined in src/operator/optimizer_op.cc:L495
Weight
Gradient
Moving mean
Moving variance
Learning rate
The decay rate for the 1st moment estimates.
The decay rate for the 2nd moment estimates.
A small constant for numerical stability.
Weight decay augments the objective function with a regularization term that penalizes large weights. The penalty scales with the square of the magnitude of each weight.
Rescale gradient to grad = rescale_grad*grad.
Clip gradient to the range of [clip_gradient, clip_gradient] If clip_gradient <= 0, gradient clipping is turned off. grad = max(min(grad, clip_gradient), clip_gradient).
If true, lazy updates are applied if gradient's stype is row_sparse and all of w, m and v have the same stype
org.apache.mxnet.Symbol
Adds all input arguments elementwise.
..
Adds all input arguments elementwise.
.. math::
add\_n(a_1, a_2, ..., a_n) = a_1 + a_2 + ... + a_n
is potentially more efficient than calling add_n
by add
n
times.
The storage type of
output depends on storage types of inputsadd_n
 add_n(row_sparse, row_sparse, ..) = row_sparse
 add_n(default, csr, default) = default
 add_n(any input combinations longer than 4 (>4) with at least one default type) = default
 otherwise,
falls all inputs back to default storage and generates default storageadd_n
Defined in src/operator/tensor/elemwise_sum.cc:L156
Positional input arguments
org.apache.mxnet.Symbol
Returns elementwise inverse cosine of the input array.
The input should be in range [1, 1]
.
The output is in the closed interval :math:[0, \pi]
..
Returns elementwise inverse cosine of the input array.
The input should be in range [1, 1]
.
The output is in the closed interval :math:[0, \pi]
.. math::
arccos([1, .707, 0, .707, 1]) = [\pi, 3\pi/4, \pi/2, \pi/4, 0]
The storage type of
output is always densearccos
Defined in src/operator/tensor/elemwise_unary_op_trig.cc:L123
The input array.
org.apache.mxnet.Symbol
Returns the elementwise inverse hyperbolic cosine of the input array, \
computed elementwise.
The storage type of
output is always densearccosh
Defined in src/operator/tensor/elemwise_unary_op_trig.cc:L264
Returns the elementwise inverse hyperbolic cosine of the input array, \
computed elementwise.
The storage type of
output is always densearccosh
Defined in src/operator/tensor/elemwise_unary_op_trig.cc:L264
The input array.
org.apache.mxnet.Symbol
Returns elementwise inverse sine of the input array.
The input should be in the range [1, 1]
.
The output is in the closed interval of [:math:\pi/2
, :math:\pi/2
].
..
Returns elementwise inverse sine of the input array.
The input should be in the range [1, 1]
.
The output is in the closed interval of [:math:\pi/2
, :math:\pi/2
].
.. math::
arcsin([1, .707, 0, .707, 1]) = [\pi/2, \pi/4, 0, \pi/4, \pi/2]
The storage type of
output depends upon the input storage type:arcsin
The input array.
org.apache.mxnet.Symbol
Returns the elementwise inverse hyperbolic sine of the input array, \
computed elementwise.
The storage type of
output depends upon the input storage type:arcsinh
Returns the elementwise inverse hyperbolic sine of the input array, \
computed elementwise.
The storage type of
output depends upon the input storage type:arcsinh
The input array.
org.apache.mxnet.Symbol
Returns elementwise inverse tangent of the input array.
The output is in the closed interval :math:[\pi/2, \pi/2]
..
Returns elementwise inverse tangent of the input array.
The output is in the closed interval :math:[\pi/2, \pi/2]
.. math::
arctan([1, 0, 1]) = [\pi/4, 0, \pi/4]
The storage type of
output depends upon the input storage type:arctan
The input array.
org.apache.mxnet.Symbol
Returns the elementwise inverse hyperbolic tangent of the input array, \
computed elementwise.
The storage type of
output depends upon the input storage type:arctanh
Returns the elementwise inverse hyperbolic tangent of the input array, \
computed elementwise.
The storage type of
output depends upon the input storage type:arctanh
The input array.
org.apache.mxnet.Symbol
Returns indices of the maximum values along an axis.
In the case of multiple occurrences of maximum values, the indices corresponding to the first occurrence
are returned.
Examples::
x = 0., 1., 2.],
[ 3., 4., 5.
// argmax along axis 0
argmax(x, axis=0) = [ 1., 1., 1.]
// argmax along axis 1
argmax(x, axis=1) = [ 2., 2.]
// argmax along axis 1 keeping same dims as an input array
argmax(x, axis=1, keepdims=True) = 2.],
[ 2.
Defined in src/operator/tensor/broadcast_reduce_op_index.cc:L52
Returns indices of the maximum values along an axis.
In the case of multiple occurrences of maximum values, the indices corresponding to the first occurrence
are returned.
Examples::
x = 0., 1., 2.],
[ 3., 4., 5.
// argmax along axis 0
argmax(x, axis=0) = [ 1., 1., 1.]
// argmax along axis 1
argmax(x, axis=1) = [ 2., 2.]
// argmax along axis 1 keeping same dims as an input array
argmax(x, axis=1, keepdims=True) = 2.],
[ 2.
Defined in src/operator/tensor/broadcast_reduce_op_index.cc:L52
The input
The axis along which to perform the reduction. Negative values means indexing from right to left. Requires axis to be set as int, because global reduction is not supported yet.
If this is set to True
, the reduced axis is left in the result as dimension with size one.
org.apache.mxnet.Symbol
Returns argmax indices of each channel from the input array.
The result will be an NDArray of shape (num_channel,).
In case of multiple occurrences of the maximum values, the indices corresponding to the first occurrence
are returned.
Examples::
x = 0., 1., 2.],
[ 3., 4., 5.
argmax_channel(x) = [ 2., 2.]
Defined in src/operator/tensor/broadcast_reduce_op_index.cc:L97
Returns argmax indices of each channel from the input array.
The result will be an NDArray of shape (num_channel,).
In case of multiple occurrences of the maximum values, the indices corresponding to the first occurrence
are returned.
Examples::
x = 0., 1., 2.],
[ 3., 4., 5.
argmax_channel(x) = [ 2., 2.]
Defined in src/operator/tensor/broadcast_reduce_op_index.cc:L97
The input array
org.apache.mxnet.Symbol
Returns indices of the minimum values along an axis.
In the case of multiple occurrences of minimum values, the indices corresponding to the first occurrence
are returned.
Examples::
x = 0., 1., 2.],
[ 3., 4., 5.
// argmin along axis 0
argmin(x, axis=0) = [ 0., 0., 0.]
// argmin along axis 1
argmin(x, axis=1) = [ 0., 0.]
// argmin along axis 1 keeping same dims as an input array
argmin(x, axis=1, keepdims=True) = 0.],
[ 0.
Defined in src/operator/tensor/broadcast_reduce_op_index.cc:L77
Returns indices of the minimum values along an axis.
In the case of multiple occurrences of minimum values, the indices corresponding to the first occurrence
are returned.
Examples::
x = 0., 1., 2.],
[ 3., 4., 5.
// argmin along axis 0
argmin(x, axis=0) = [ 0., 0., 0.]
// argmin along axis 1
argmin(x, axis=1) = [ 0., 0.]
// argmin along axis 1 keeping same dims as an input array
argmin(x, axis=1, keepdims=True) = 0.],
[ 0.
Defined in src/operator/tensor/broadcast_reduce_op_index.cc:L77
The input
The axis along which to perform the reduction. Negative values means indexing from right to left. Requires axis to be set as int, because global reduction is not supported yet.
If this is set to True
, the reduced axis is left in the result as dimension with size one.
org.apache.mxnet.Symbol
Returns the indices that would sort an input array along the given axis.
This function performs sorting along the given axis and returns an array of indices having same shape
as an input array that index data in sorted order.
Examples::
x = 0.3, 0.2, 0.4],
[ 0.1, 0.3, 0.2
// sort along axis 1
argsort(x) = 1., 0., 2.],
[ 0., 2., 1.
// sort along axis 0
argsort(x, axis=0) = 1., 0., 1.]
[ 0., 1., 0.
// flatten and then sort
argsort(x) = [ 3., 1., 5., 0., 4., 2.]
Defined in src/operator/tensor/ordering_op.cc:L176
Returns the indices that would sort an input array along the given axis.
This function performs sorting along the given axis and returns an array of indices having same shape
as an input array that index data in sorted order.
Examples::
x = 0.3, 0.2, 0.4],
[ 0.1, 0.3, 0.2
// sort along axis 1
argsort(x) = 1., 0., 2.],
[ 0., 2., 1.
// sort along axis 0
argsort(x, axis=0) = 1., 0., 1.]
[ 0., 1., 0.
// flatten and then sort
argsort(x) = [ 3., 1., 5., 0., 4., 2.]
Defined in src/operator/tensor/ordering_op.cc:L176
The input array
Axis along which to sort the input tensor. If not given, the flattened array is used. Default is 1.
Whether to sort in ascending or descending order.
org.apache.mxnet.Symbol
Batchwise dot product.
is used to compute dot product of batch_dot
and x
when y
andx
are data in batch, namely 3D arrays in shape of y
(batch_size, :, :)
.
For example, given
with shape x
(batch_size, n, m)
and
with shapey
(batch_size, m, k)
, the result array will have shape (batch_size, n, k)
,
which is computed by::
batch_dot(x,y)[i,:,:] = dot(x[i,:,:], y[i,:,:])
Defined in src/operator/tensor/dot.cc:L125
Batchwise dot product.
is used to compute dot product of batch_dot
and x
when y
andx
are data in batch, namely 3D arrays in shape of y
(batch_size, :, :)
.
For example, given
with shape x
(batch_size, n, m)
and
with shapey
(batch_size, m, k)
, the result array will have shape (batch_size, n, k)
,
which is computed by::
batch_dot(x,y)[i,:,:] = dot(x[i,:,:], y[i,:,:])
Defined in src/operator/tensor/dot.cc:L125
The first input
The second input
If true then transpose the first input before dot.
If true then transpose the second input before dot.
The desired storage type of the forward output given by user, if thecombination of input storage types and this hint does not matchany implemented ones, the dot operator will perform fallback operationand still produce an output of the desired storage type.
org.apache.mxnet.Symbol
Takes elements from a data batch.
..
Takes elements from a data batch.
.. note::
batch_take
is deprecated. Use pick
instead.
Given an input array of shape
and indices of shape (d0, d1)
, the result will be(i0,)
an output array of shape
with::(i0,)
output[i] = input[i, indices[i]]
Examples::
x = 1., 2.],
[ 3., 4.],
[ 5., 6.
// takes elements with specified indices
batch_take(x, [0,1,0]) = [ 1. 4. 5.]
Defined in src/operator/tensor/indexing_op.cc:L462
The input array
The index array
org.apache.mxnet.Symbol
Returns elementwise sum of the input arrays with broadcasting.
broadcast_plus
is an alias to the function broadcast_add
.
Example::
x = 1., 1., 1.],
[ 1., 1., 1.
y = 0.],
[ 1.
broadcast_add(x, y) = 1., 1., 1.],
[ 2., 2., 2.
broadcast_plus(x, y) = 1., 1., 1.],
[ 2., 2., 2.
Supported sparse operations:
broadcast_add(csr, dense(1D)) = dense
broadcast_add(dense(1D), csr) = dense
Defined in src/operator/tensor/elemwise_binary_broadcast_op_basic.cc:L58
Returns elementwise sum of the input arrays with broadcasting.
broadcast_plus
is an alias to the function broadcast_add
.
Example::
x = 1., 1., 1.],
[ 1., 1., 1.
y = 0.],
[ 1.
broadcast_add(x, y) = 1., 1., 1.],
[ 2., 2., 2.
broadcast_plus(x, y) = 1., 1., 1.],
[ 2., 2., 2.
Supported sparse operations:
broadcast_add(csr, dense(1D)) = dense
broadcast_add(dense(1D), csr) = dense
Defined in src/operator/tensor/elemwise_binary_broadcast_op_basic.cc:L58
First input to the function
Second input to the function
org.apache.mxnet.Symbol
Broadcasts the input array over particular axes.
Broadcasting is allowed on axes with size 1, such as from (2,1,3,1)
to
(2,8,3,9)
.
Broadcasts the input array over particular axes.
Broadcasting is allowed on axes with size 1, such as from (2,1,3,1)
to
(2,8,3,9)
. Elements will be duplicated on the broadcasted axes.
Example::
// given x of shape (1,2,1)
x = 1.],
[ 2.
// broadcast x on on axis 2
broadcast_axis(x, axis=2, size=3) = 1., 1., 1.],
[ 2., 2., 2.
// broadcast x on on axes 0 and 2
broadcast_axis(x, axis=(0,2), size=(2,3)) = 1., 1., 1.],
[ 2., 2., 2.]],
1., 1., 1.],
[ 2., 2., 2.]
Defined in src/operator/tensor/broadcast_reduce_op_value.cc:L237
The input
The axes to perform the broadcasting.
Target sizes of the broadcasting axes.
org.apache.mxnet.Symbol
Broadcasts the input array over particular axes.
Broadcasting is allowed on axes with size 1, such as from (2,1,3,1)
to
(2,8,3,9)
.
Broadcasts the input array over particular axes.
Broadcasting is allowed on axes with size 1, such as from (2,1,3,1)
to
(2,8,3,9)
. Elements will be duplicated on the broadcasted axes.
Example::
// given x of shape (1,2,1)
x = 1.],
[ 2.
// broadcast x on on axis 2
broadcast_axis(x, axis=2, size=3) = 1., 1., 1.],
[ 2., 2., 2.
// broadcast x on on axes 0 and 2
broadcast_axis(x, axis=(0,2), size=(2,3)) = 1., 1., 1.],
[ 2., 2., 2.]],
1., 1., 1.],
[ 2., 2., 2.]
Defined in src/operator/tensor/broadcast_reduce_op_value.cc:L237
The input
The axes to perform the broadcasting.
Target sizes of the broadcasting axes.
org.apache.mxnet.Symbol
Returns elementwise division of the input arrays with broadcasting.
Example::
x = 6., 6., 6.],
[ 6., 6., 6.
y = 2.],
[ 3.
broadcast_div(x, y) = 3., 3., 3.],
[ 2., 2., 2.
Supported sparse operations:
broadcast_div(csr, dense(1D)) = csr
Defined in src/operator/tensor/elemwise_binary_broadcast_op_basic.cc:L187
Returns elementwise division of the input arrays with broadcasting.
Example::
x = 6., 6., 6.],
[ 6., 6., 6.
y = 2.],
[ 3.
broadcast_div(x, y) = 3., 3., 3.],
[ 2., 2., 2.
Supported sparse operations:
broadcast_div(csr, dense(1D)) = csr
Defined in src/operator/tensor/elemwise_binary_broadcast_op_basic.cc:L187
First input to the function
Second input to the function
org.apache.mxnet.Symbol
Returns the result of elementwise **equal to** (==) comparison operation with broadcasting.
Example::
x = 1., 1., 1.],
[ 1., 1., 1.
y = 0.],
[ 1.
broadcast_equal(x, y) = 0., 0., 0.],
[ 1., 1., 1.
Defined in src/operator/tensor/elemwise_binary_broadcast_op_logic.cc:L46
Returns the result of elementwise **equal to** (==) comparison operation with broadcasting.
Example::
x = 1., 1., 1.],
[ 1., 1., 1.
y = 0.],
[ 1.
broadcast_equal(x, y) = 0., 0., 0.],
[ 1., 1., 1.
Defined in src/operator/tensor/elemwise_binary_broadcast_op_logic.cc:L46
First input to the function
Second input to the function
org.apache.mxnet.Symbol
Returns the result of elementwise **greater than** (>) comparison operation with broadcasting.
Example::
x = 1., 1., 1.],
[ 1., 1., 1.
y = 0.],
[ 1.
broadcast_greater(x, y) = 1., 1., 1.],
[ 0., 0., 0.
Defined in src/operator/tensor/elemwise_binary_broadcast_op_logic.cc:L82
Returns the result of elementwise **greater than** (>) comparison operation with broadcasting.
Example::
x = 1., 1., 1.],
[ 1., 1., 1.
y = 0.],
[ 1.
broadcast_greater(x, y) = 1., 1., 1.],
[ 0., 0., 0.
Defined in src/operator/tensor/elemwise_binary_broadcast_op_logic.cc:L82
First input to the function
Second input to the function
org.apache.mxnet.Symbol
Returns the result of elementwise **greater than or equal to** (>=) comparison operation with broadcasting.
Example::
x = 1., 1., 1.],
[ 1., 1., 1.
y = 0.],
[ 1.
broadcast_greater_equal(x, y) = 1., 1., 1.],
[ 1., 1., 1.
Defined in src/operator/tensor/elemwise_binary_broadcast_op_logic.cc:L100
Returns the result of elementwise **greater than or equal to** (>=) comparison operation with broadcasting.
Example::
x = 1., 1., 1.],
[ 1., 1., 1.
y = 0.],
[ 1.
broadcast_greater_equal(x, y) = 1., 1., 1.],
[ 1., 1., 1.
Defined in src/operator/tensor/elemwise_binary_broadcast_op_logic.cc:L100
First input to the function
Second input to the function
org.apache.mxnet.Symbol
Returns the hypotenuse of a right angled triangle, given its "legs"
with broadcasting.
It is equivalent to doing :math:sqrt(x_1^{2 + x_2}2)
.
Example::
x = 3., 3., 3.
y = 4.],
[ 4.
broadcast_hypot(x, y) = 5., 5., 5.],
[ 5., 5., 5.
z = 0.],
[ 4.
broadcast_hypot(x, z) = 3., 3., 3.],
[ 5., 5., 5.
Defined in src/operator/tensor/elemwise_binary_broadcast_op_extended.cc:L156
Returns the hypotenuse of a right angled triangle, given its "legs"
with broadcasting.
It is equivalent to doing :math:sqrt(x_1^{2 + x_2}2)
.
Example::
x = 3., 3., 3.
y = 4.],
[ 4.
broadcast_hypot(x, y) = 5., 5., 5.],
[ 5., 5., 5.
z = 0.],
[ 4.
broadcast_hypot(x, z) = 3., 3., 3.],
[ 5., 5., 5.
Defined in src/operator/tensor/elemwise_binary_broadcast_op_extended.cc:L156
First input to the function
Second input to the function
org.apache.mxnet.Symbol
Returns the result of elementwise **lesser than** (<) comparison operation with broadcasting.
Example::
x = 1., 1., 1.],
[ 1., 1., 1.
y = 0.],
[ 1.
broadcast_lesser(x, y) = 0., 0., 0.],
[ 0., 0., 0.
Defined in src/operator/tensor/elemwise_binary_broadcast_op_logic.cc:L118
Returns the result of elementwise **lesser than** (<) comparison operation with broadcasting.
Example::
x = 1., 1., 1.],
[ 1., 1., 1.
y = 0.],
[ 1.
broadcast_lesser(x, y) = 0., 0., 0.],
[ 0., 0., 0.
Defined in src/operator/tensor/elemwise_binary_broadcast_op_logic.cc:L118
First input to the function
Second input to the function
org.apache.mxnet.Symbol
Returns the result of elementwise **lesser than or equal to** (<=) comparison operation with broadcasting.
Example::
x = 1., 1., 1.],
[ 1., 1., 1.
y = 0.],
[ 1.
broadcast_lesser_equal(x, y) = 0., 0., 0.],
[ 1., 1., 1.
Defined in src/operator/tensor/elemwise_binary_broadcast_op_logic.cc:L136
Returns the result of elementwise **lesser than or equal to** (<=) comparison operation with broadcasting.
Example::
x = 1., 1., 1.],
[ 1., 1., 1.
y = 0.],
[ 1.
broadcast_lesser_equal(x, y) = 0., 0., 0.],
[ 1., 1., 1.
Defined in src/operator/tensor/elemwise_binary_broadcast_op_logic.cc:L136
First input to the function
Second input to the function
org.apache.mxnet.Symbol
Broadcasts lhs to have the same shape as rhs.
Broadcasting is a mechanism that allows NDArrays to perform arithmetic operations
with arrays of different shapes efficiently without creating multiple copies of arrays.
Also see, Broadcasting <https://docs.scipy.org/doc/numpy/user/basics.broadcasting.html>
_ for more explanation.
Broadcasting is allowed on axes with size 1, such as from (2,1,3,1)
to
(2,8,3,9)
.
Broadcasts lhs to have the same shape as rhs.
Broadcasting is a mechanism that allows NDArrays to perform arithmetic operations
with arrays of different shapes efficiently without creating multiple copies of arrays.
Also see, Broadcasting <https://docs.scipy.org/doc/numpy/user/basics.broadcasting.html>
_ for more explanation.
Broadcasting is allowed on axes with size 1, such as from (2,1,3,1)
to
(2,8,3,9)
. Elements will be duplicated on the broadcasted axes.
For example::
broadcast_like(1,2,3, 5,6,7],[7,8,9) = 1., 2., 3.],
[ 1., 2., 3.)
Defined in src/operator/tensor/broadcast_reduce_op_value.cc:L312
First input.
Second input.
org.apache.mxnet.Symbol
Returns the result of elementwise **logical and** with broadcasting.
Example::
x = 1., 1., 1.],
[ 1., 1., 1.
y = 0.],
[ 1.
broadcast_logical_and(x, y) = 0., 0., 0.],
[ 1., 1., 1.
Defined in src/operator/tensor/elemwise_binary_broadcast_op_logic.cc:L154
Returns the result of elementwise **logical and** with broadcasting.
Example::
x = 1., 1., 1.],
[ 1., 1., 1.
y = 0.],
[ 1.
broadcast_logical_and(x, y) = 0., 0., 0.],
[ 1., 1., 1.
Defined in src/operator/tensor/elemwise_binary_broadcast_op_logic.cc:L154
First input to the function
Second input to the function
org.apache.mxnet.Symbol
Returns the result of elementwise **logical or** with broadcasting.
Example::
x = 1., 1., 0.],
[ 1., 1., 0.
y = 1.],
[ 0.
broadcast_logical_or(x, y) = 1., 1., 1.],
[ 1., 1., 0.
Defined in src/operator/tensor/elemwise_binary_broadcast_op_logic.cc:L172
Returns the result of elementwise **logical or** with broadcasting.
Example::
x = 1., 1., 0.],
[ 1., 1., 0.
y = 1.],
[ 0.
broadcast_logical_or(x, y) = 1., 1., 1.],
[ 1., 1., 0.
Defined in src/operator/tensor/elemwise_binary_broadcast_op_logic.cc:L172
First input to the function
Second input to the function
org.apache.mxnet.Symbol
Returns the result of elementwise **logical xor** with broadcasting.
Example::
x = 1., 1., 0.],
[ 1., 1., 0.
y = 1.],
[ 0.
broadcast_logical_xor(x, y) = 0., 0., 1.],
[ 1., 1., 0.
Defined in src/operator/tensor/elemwise_binary_broadcast_op_logic.cc:L190
Returns the result of elementwise **logical xor** with broadcasting.
Example::
x = 1., 1., 0.],
[ 1., 1., 0.
y = 1.],
[ 0.
broadcast_logical_xor(x, y) = 0., 0., 1.],
[ 1., 1., 0.
Defined in src/operator/tensor/elemwise_binary_broadcast_op_logic.cc:L190
First input to the function
Second input to the function
org.apache.mxnet.Symbol
Returns elementwise maximum of the input arrays with broadcasting.
This function compares two input arrays and returns a new array having the elementwise maxima.
Example::
x = 1., 1., 1.],
[ 1., 1., 1.
y = 0.],
[ 1.
broadcast_maximum(x, y) = 1., 1., 1.],
[ 1., 1., 1.
Defined in src/operator/tensor/elemwise_binary_broadcast_op_extended.cc:L80
Returns elementwise maximum of the input arrays with broadcasting.
This function compares two input arrays and returns a new array having the elementwise maxima.
Example::
x = 1., 1., 1.],
[ 1., 1., 1.
y = 0.],
[ 1.
broadcast_maximum(x, y) = 1., 1., 1.],
[ 1., 1., 1.
Defined in src/operator/tensor/elemwise_binary_broadcast_op_extended.cc:L80
First input to the function
Second input to the function
org.apache.mxnet.Symbol
Returns elementwise minimum of the input arrays with broadcasting.
This function compares two input arrays and returns a new array having the elementwise minima.
Example::
x = 1., 1., 1.],
[ 1., 1., 1.
y = 0.],
[ 1.
broadcast_maximum(x, y) = 0., 0., 0.],
[ 1., 1., 1.
Defined in src/operator/tensor/elemwise_binary_broadcast_op_extended.cc:L115
Returns elementwise minimum of the input arrays with broadcasting.
This function compares two input arrays and returns a new array having the elementwise minima.
Example::
x = 1., 1., 1.],
[ 1., 1., 1.
y = 0.],
[ 1.
broadcast_maximum(x, y) = 0., 0., 0.],
[ 1., 1., 1.
Defined in src/operator/tensor/elemwise_binary_broadcast_op_extended.cc:L115
First input to the function
Second input to the function
org.apache.mxnet.Symbol
Returns elementwise difference of the input arrays with broadcasting.
broadcast_minus
is an alias to the function broadcast_sub
.
Example::
x = 1., 1., 1.],
[ 1., 1., 1.
y = 0.],
[ 1.
broadcast_sub(x, y) = 1., 1., 1.],
[ 0., 0., 0.
broadcast_minus(x, y) = 1., 1., 1.],
[ 0., 0., 0.
Supported sparse operations:
broadcast_sub/minus(csr, dense(1D)) = dense
broadcast_sub/minus(dense(1D), csr) = dense
Defined in src/operator/tensor/elemwise_binary_broadcast_op_basic.cc:L106
Returns elementwise difference of the input arrays with broadcasting.
broadcast_minus
is an alias to the function broadcast_sub
.
Example::
x = 1., 1., 1.],
[ 1., 1., 1.
y = 0.],
[ 1.
broadcast_sub(x, y) = 1., 1., 1.],
[ 0., 0., 0.
broadcast_minus(x, y) = 1., 1., 1.],
[ 0., 0., 0.
Supported sparse operations:
broadcast_sub/minus(csr, dense(1D)) = dense
broadcast_sub/minus(dense(1D), csr) = dense
Defined in src/operator/tensor/elemwise_binary_broadcast_op_basic.cc:L106
First input to the function
Second input to the function
org.apache.mxnet.Symbol
Returns elementwise modulo of the input arrays with broadcasting.
Example::
x = 8., 8., 8.],
[ 8., 8., 8.
y = 2.],
[ 3.
broadcast_mod(x, y) = 0., 0., 0.],
[ 2., 2., 2.
Defined in src/operator/tensor/elemwise_binary_broadcast_op_basic.cc:L222
Returns elementwise modulo of the input arrays with broadcasting.
Example::
x = 8., 8., 8.],
[ 8., 8., 8.
y = 2.],
[ 3.
broadcast_mod(x, y) = 0., 0., 0.],
[ 2., 2., 2.
Defined in src/operator/tensor/elemwise_binary_broadcast_op_basic.cc:L222
First input to the function
Second input to the function
org.apache.mxnet.Symbol
Returns elementwise product of the input arrays with broadcasting.
Example::
x = 1., 1., 1.],
[ 1., 1., 1.
y = 0.],
[ 1.
broadcast_mul(x, y) = 0., 0., 0.],
[ 1., 1., 1.
Supported sparse operations:
broadcast_mul(csr, dense(1D)) = csr
Defined in src/operator/tensor/elemwise_binary_broadcast_op_basic.cc:L146
Returns elementwise product of the input arrays with broadcasting.
Example::
x = 1., 1., 1.],
[ 1., 1., 1.
y = 0.],
[ 1.
broadcast_mul(x, y) = 0., 0., 0.],
[ 1., 1., 1.
Supported sparse operations:
broadcast_mul(csr, dense(1D)) = csr
Defined in src/operator/tensor/elemwise_binary_broadcast_op_basic.cc:L146
First input to the function
Second input to the function
org.apache.mxnet.Symbol
Returns the result of elementwise **not equal to** (!=) comparison operation with broadcasting.
Example::
x = 1., 1., 1.],
[ 1., 1., 1.
y = 0.],
[ 1.
broadcast_not_equal(x, y) = 1., 1., 1.],
[ 0., 0., 0.
Defined in src/operator/tensor/elemwise_binary_broadcast_op_logic.cc:L64
Returns the result of elementwise **not equal to** (!=) comparison operation with broadcasting.
Example::
x = 1., 1., 1.],
[ 1., 1., 1.
y = 0.],
[ 1.
broadcast_not_equal(x, y) = 1., 1., 1.],
[ 0., 0., 0.
Defined in src/operator/tensor/elemwise_binary_broadcast_op_logic.cc:L64
First input to the function
Second input to the function
org.apache.mxnet.Symbol
Returns elementwise sum of the input arrays with broadcasting.
broadcast_plus
is an alias to the function broadcast_add
.
Example::
x = 1., 1., 1.],
[ 1., 1., 1.
y = 0.],
[ 1.
broadcast_add(x, y) = 1., 1., 1.],
[ 2., 2., 2.
broadcast_plus(x, y) = 1., 1., 1.],
[ 2., 2., 2.
Supported sparse operations:
broadcast_add(csr, dense(1D)) = dense
broadcast_add(dense(1D), csr) = dense
Defined in src/operator/tensor/elemwise_binary_broadcast_op_basic.cc:L58
Returns elementwise sum of the input arrays with broadcasting.
broadcast_plus
is an alias to the function broadcast_add
.
Example::
x = 1., 1., 1.],
[ 1., 1., 1.
y = 0.],
[ 1.
broadcast_add(x, y) = 1., 1., 1.],
[ 2., 2., 2.
broadcast_plus(x, y) = 1., 1., 1.],
[ 2., 2., 2.
Supported sparse operations:
broadcast_add(csr, dense(1D)) = dense
broadcast_add(dense(1D), csr) = dense
Defined in src/operator/tensor/elemwise_binary_broadcast_op_basic.cc:L58
First input to the function
Second input to the function
org.apache.mxnet.Symbol
Returns result of first array elements raised to powers from second array, elementwise with broadcasting.
Example::
x = 1., 1., 1.],
[ 1., 1., 1.
y = 0.],
[ 1.
broadcast_power(x, y) = 2., 2., 2.],
[ 4., 4., 4.
Defined in src/operator/tensor/elemwise_binary_broadcast_op_extended.cc:L45
Returns result of first array elements raised to powers from second array, elementwise with broadcasting.
Example::
x = 1., 1., 1.],
[ 1., 1., 1.
y = 0.],
[ 1.
broadcast_power(x, y) = 2., 2., 2.],
[ 4., 4., 4.
Defined in src/operator/tensor/elemwise_binary_broadcast_op_extended.cc:L45
First input to the function
Second input to the function
org.apache.mxnet.Symbol
Returns elementwise difference of the input arrays with broadcasting.
broadcast_minus
is an alias to the function broadcast_sub
.
Example::
x = 1., 1., 1.],
[ 1., 1., 1.
y = 0.],
[ 1.
broadcast_sub(x, y) = 1., 1., 1.],
[ 0., 0., 0.
broadcast_minus(x, y) = 1., 1., 1.],
[ 0., 0., 0.
Supported sparse operations:
broadcast_sub/minus(csr, dense(1D)) = dense
broadcast_sub/minus(dense(1D), csr) = dense
Defined in src/operator/tensor/elemwise_binary_broadcast_op_basic.cc:L106
Returns elementwise difference of the input arrays with broadcasting.
broadcast_minus
is an alias to the function broadcast_sub
.
Example::
x = 1., 1., 1.],
[ 1., 1., 1.
y = 0.],
[ 1.
broadcast_sub(x, y) = 1., 1., 1.],
[ 0., 0., 0.
broadcast_minus(x, y) = 1., 1., 1.],
[ 0., 0., 0.
Supported sparse operations:
broadcast_sub/minus(csr, dense(1D)) = dense
broadcast_sub/minus(dense(1D), csr) = dense
Defined in src/operator/tensor/elemwise_binary_broadcast_op_basic.cc:L106
First input to the function
Second input to the function
org.apache.mxnet.Symbol
Broadcasts the input array to a new shape.
Broadcasting is a mechanism that allows NDArrays to perform arithmetic operations
with arrays of different shapes efficiently without creating multiple copies of arrays.
Also see, Broadcasting <https://docs.scipy.org/doc/numpy/user/basics.broadcasting.html>
_ for more explanation.
Broadcasting is allowed on axes with size 1, such as from (2,1,3,1)
to
(2,8,3,9)
.
Broadcasts the input array to a new shape.
Broadcasting is a mechanism that allows NDArrays to perform arithmetic operations
with arrays of different shapes efficiently without creating multiple copies of arrays.
Also see, Broadcasting <https://docs.scipy.org/doc/numpy/user/basics.broadcasting.html>
_ for more explanation.
Broadcasting is allowed on axes with size 1, such as from (2,1,3,1)
to
(2,8,3,9)
. Elements will be duplicated on the broadcasted axes.
For example::
broadcast_to(1,2,3, shape=(2,3)) = 1., 2., 3.],
[ 1., 2., 3.)
The dimension which you do not want to change can also be kept as 0
which means copy the original value.
So with shape=(2,0)
, we will obtain the same result as in the above example.
Defined in src/operator/tensor/broadcast_reduce_op_value.cc:L261
The input
The shape of the desired array. We can set the dim to zero if it's same as the original. E.g A = broadcast_to(B, shape=(10, 0, 0))
has the same meaning as A = broadcast_axis(B, axis=0, size=10)
.
org.apache.mxnet.Symbol
Casts all elements of the input to a new type.
..
Casts all elements of the input to a new type.
.. note::
is deprecated. Use Cast
instead.cast
Example::
cast([0.9, 1.3], dtype='int32') = [0, 1]
cast([1e20, 11.1], dtype='float16') = [inf, 11.09375]
cast([300, 11.1, 10.9, 1, 3], dtype='uint8') = [44, 11, 10, 255, 253]
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L504
The input.
Output data type.
org.apache.mxnet.Symbol
Casts tensor storage type to the new type.
When an NDArray with default storage type is cast to csr or row_sparse storage,
the result is compact, which means:
 for csr, zero values will not be retained
 for row_sparse, row slices of all zeros will not be retained
The storage type of
output depends on stype parameter:cast_storage
 cast_storage(csr, 'default') = default
 cast_storage(row_sparse, 'default') = default
 cast_storage(default, 'csr') = csr
 cast_storage(default, 'row_sparse') = row_sparse
 cast_storage(csr, 'csr') = csr
 cast_storage(row_sparse, 'row_sparse') = row_sparse
Example::
dense = 0., 1., 0.],
[ 2., 0., 3.],
[ 0., 0., 0.],
[ 0., 0., 0.
# cast to row_sparse storage type
rsp = cast_storage(dense, 'row_sparse')
rsp.indices = [0, 1]
rsp.values = 0., 1., 0.],
[ 2., 0., 3.
# cast to csr storage type
csr = cast_storage(dense, 'csr')
csr.indices = [1, 0, 2]
csr.values = [ 1., 2., 3.]
csr.indptr = [0, 1, 3, 3, 3]
Defined in src/operator/tensor/cast_storage.cc:L71
Casts tensor storage type to the new type.
When an NDArray with default storage type is cast to csr or row_sparse storage,
the result is compact, which means:
 for csr, zero values will not be retained
 for row_sparse, row slices of all zeros will not be retained
The storage type of
output depends on stype parameter:cast_storage
 cast_storage(csr, 'default') = default
 cast_storage(row_sparse, 'default') = default
 cast_storage(default, 'csr') = csr
 cast_storage(default, 'row_sparse') = row_sparse
 cast_storage(csr, 'csr') = csr
 cast_storage(row_sparse, 'row_sparse') = row_sparse
Example::
dense = 0., 1., 0.],
[ 2., 0., 3.],
[ 0., 0., 0.],
[ 0., 0., 0.
# cast to row_sparse storage type
rsp = cast_storage(dense, 'row_sparse')
rsp.indices = [0, 1]
rsp.values = 0., 1., 0.],
[ 2., 0., 3.
# cast to csr storage type
csr = cast_storage(dense, 'csr')
csr.indices = [1, 0, 2]
csr.values = [ 1., 2., 3.]
csr.indptr = [0, 1, 3, 3, 3]
Defined in src/operator/tensor/cast_storage.cc:L71
The input.
Output storage type.
org.apache.mxnet.Symbol
Returns elementwise cuberoot value of the input.
..
Returns elementwise cuberoot value of the input.
.. math::
cbrt(x) = \sqrt[3]{x}
Example::
cbrt([1, 8, 125]) = [1, 2, 5]
The storage type of
output depends upon the input storage type:cbrt
The input array.
org.apache.mxnet.Symbol
Returns elementwise ceiling of the input.
The ceil of the scalar x is the smallest integer i, such that i >= x.
Example::
ceil([2.1, 1.9, 1.5, 1.9, 2.1]) = [2., 1., 2., 2., 3.]
The storage type of
output depends upon the input storage type:ceil
Returns elementwise ceiling of the input.
The ceil of the scalar x is the smallest integer i, such that i >= x.
Example::
ceil([2.1, 1.9, 1.5, 1.9, 2.1]) = [2., 1., 2., 2., 3.]
The storage type of
output depends upon the input storage type:ceil
The input array.
org.apache.mxnet.Symbol
Choose one element from each line(row for python, column for R/Julia) in lhs according to index indicated by rhs.
Choose one element from each line(row for python, column for R/Julia) in lhs according to index indicated by rhs. This function assume rhs uses 0based index.
Left operand to the function.
Right operand to the function.
org.apache.mxnet.Symbol
Clips (limits) the values in an array.
Given an interval, values outside the interval are clipped to the interval edges.
Clipping
between x
a_min
and a_x
would be::
clip(x, a_min, a_max) = max(min(x, a_max), a_min))
Example::
x = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
clip(x,1,8) = [ 1., 1., 2., 3., 4., 5., 6., 7., 8., 8.]
The storage type of
output depends on storage types of inputs and the a_min, a_max \clip
parameter values:
Clips (limits) the values in an array.
Given an interval, values outside the interval are clipped to the interval edges.
Clipping
between x
a_min
and a_x
would be::
clip(x, a_min, a_max) = max(min(x, a_max), a_min))
Example::
x = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
clip(x,1,8) = [ 1., 1., 2., 3., 4., 5., 6., 7., 8., 8.]
The storage type of
output depends on storage types of inputs and the a_min, a_max \clip
parameter values:
Input array.
Minimum value
Maximum value
org.apache.mxnet.Symbol
Joins input arrays along a given axis.
..
Joins input arrays along a given axis.
.. note:: Concat
is deprecated. Use concat
instead.
The dimensions of the input arrays should be the same except the axis along
which they will be concatenated.
The dimension of the output array along the concatenated axis will be equal
to the sum of the corresponding dimensions of the input arrays.
The storage type of
output depends on storage types of inputsconcat
 concat(csr, csr, ..., csr, dim=0) = csr
 otherwise,
generates output with default storageconcat
Example::
x = 1,1],[2,2
y = 3,3],[4,4],[5,5
z = [7,7],[8,8
concat(x,y,z,dim=0) = 1., 1.],
[ 2., 2.],
[ 3., 3.],
[ 4., 4.],
[ 5., 5.],
[ 6., 6.],
[ 7., 7.],
[ 8., 8.
Note that you cannot concat x,y,z along dimension 1 since dimension
0 is not the same for all the input arrays.
concat(y,z,dim=1) = 3., 3., 6., 6.],
[ 4., 4., 7., 7.],
[ 5., 5., 8., 8.
Defined in src/operator/nn/concat.cc:L270
List of arrays to concatenate
Number of inputs to be concated.
the dimension to be concated.
org.apache.mxnet.Symbol
Computes the elementwise cosine of the input array.
The input should be in radians (:math:2\pi
rad equals 360 degrees).
..
Computes the elementwise cosine of the input array.
The input should be in radians (:math:2\pi
rad equals 360 degrees).
.. math::
cos([0, \pi/4, \pi/2]) = [1, 0.707, 0]
The storage type of
output is always densecos
Defined in src/operator/tensor/elemwise_unary_op_trig.cc:L63
The input array.
org.apache.mxnet.Symbol
Returns the hyperbolic cosine of the input array, computed elementwise.
..
Returns the hyperbolic cosine of the input array, computed elementwise.
.. math::
cosh(x) = 0.5\times(exp(x) + exp(x))
The storage type of
output is always densecosh
Defined in src/operator/tensor/elemwise_unary_op_trig.cc:L216
The input array.
org.apache.mxnet.Symbol
Slices a region of the array.
..
Slices a region of the array.
.. note::
is deprecated. Use crop
instead.slice
This function returns a sliced array between the indices given
by begin
and end
with the corresponding step
.
For an input array of
,shape=(d_0, d_1, ..., d_n1)
slice operation with
,begin=(b_0, b_1...b_m1)
, and end=(e_0, e_1, ..., e_m1)
,step=(s_0, s_1, ..., s_m1)
where m <= n, results in an array with the shape
.(e_0b_0/s_0, ..., e_m1b_m1/s_m1, d_m, ..., d_n1)
The resulting array's *k*th dimension contains elements
from the *k*th dimension of the input array starting
from index
(inclusive) with step b_k
s_k
until reaching
(exclusive).e_k
If the *k*th elements are None
in the sequence of begin
, end
,
and step
, the following rule will be used to set default values.
If s_k
is None
, set s_k=1
. If s_k > 0
, set b_k=0
, e_k=d_k
;
else, set b_k=d_k1
, e_k=1
.
The storage type of
output depends on storage types of inputsslice
 slice(csr) = csr
 otherwise,
generates output with default storageslice
.. note:: When input data storage type is csr, it only supports
step=(), or step=(None,), or step=(1,) to generate a csr output.
For other step parameter values, it falls back to slicing
a dense tensor.
Example::
x = 1., 2., 3., 4.],
[ 5., 6., 7., 8.],
[ 9., 10., 11., 12.
slice(x, begin=(0,1), end=(2,4)) = 2., 3., 4.],
[ 6., 7., 8.
slice(x, begin=(None, 0), end=(None, 3), step=(1, 2)) = 11.],
[5., 7.],
[1., 3.
Defined in src/operator/tensor/matrix_op.cc:L412
Source input
starting indices for the slice operation, supports negative indices.
ending indices for the slice operation, supports negative indices.
step for the slice operation, supports negative values.
org.apache.mxnet.Symbol
Converts each element of the input array from radians to degrees.
..
Converts each element of the input array from radians to degrees.
.. math::
degrees([0, \pi/2, \pi, 3\pi/2, 2\pi]) = [0, 90, 180, 270, 360]
The storage type of
output depends upon the input storage type:degrees
The input array.
org.apache.mxnet.Symbol
Extracts a diagonal or constructs a diagonal array.
's behavior depends on the input array dimensions:diag
 1D arrays: constructs a 2D array with the input as its diagonal, all other elements are zero
 2D arrays: returns elements in the diagonal as a new 1D array
 ND arrays: not supported yet
Examples::
x = 2, 3],
[4, 5, 6
diag(x) = [1, 5]
diag(x, k=1) = [2, 6]
diag(x, k=1) = [4]
x = [1, 2, 3]
diag(x) = 0, 0],
[0, 2, 0],
[0, 0, 3
diag(x, k=1) = 1, 0],
[0, 0, 2],
[0, 0, 0
diag(x, k=1) = 0, 0],
[1, 0, 0],
[0, 2, 0
Defined in src/operator/tensor/diag_op.cc:L68
Extracts a diagonal or constructs a diagonal array.
's behavior depends on the input array dimensions:diag
 1D arrays: constructs a 2D array with the input as its diagonal, all other elements are zero
 2D arrays: returns elements in the diagonal as a new 1D array
 ND arrays: not supported yet
Examples::
x = 2, 3],
[4, 5, 6
diag(x) = [1, 5]
diag(x, k=1) = [2, 6]
diag(x, k=1) = [4]
x = [1, 2, 3]
diag(x) = 0, 0],
[0, 2, 0],
[0, 0, 3
diag(x, k=1) = 1, 0],
[0, 0, 2],
[0, 0, 0
diag(x, k=1) = 0, 0],
[1, 0, 0],
[0, 2, 0
Defined in src/operator/tensor/diag_op.cc:L68
Input ndarray
Diagonal in question. The default is 0. Use k>0 for diagonals above the main diagonal, and k<0 for diagonals below the main diagonal. If input has shape (S0 S1) k must be between S0 and S1
org.apache.mxnet.Symbol
Dot product of two arrays.
's behavior depends on the input array dimensions:dot
 1D arrays: inner product of vectors
 2D arrays: matrix multiplication
 ND arrays: a sum product over the last axis of the first input and the first
axis of the second input
For example, given 3D
with shape x
(n,m,k)
and
with shape y
(k,r,s)
, the
result array will have shape (n,m,r,s)
.
Dot product of two arrays.
's behavior depends on the input array dimensions:dot
 1D arrays: inner product of vectors
 2D arrays: matrix multiplication
 ND arrays: a sum product over the last axis of the first input and the first
axis of the second input
For example, given 3D
with shape x
(n,m,k)
and
with shape y
(k,r,s)
, the
result array will have shape (n,m,r,s)
. It is computed by::
dot(x,y)[i,j,a,b] = sum(x[i,j,:]*y[:,a,b])
Example::
x = reshape([0,1,2,3,4,5,6,7], shape=(2,2,2))
y = reshape([7,6,5,4,3,2,1,0], shape=(2,2,2))
dot(x,y)[0,0,1,1] = 0
sum(x[0,0,:]*y[:,1,1]) = 0
The storage type of
output depends on storage types of inputs, transpose option anddot
forward_stype option for output storage type. Implemented sparse operations include:
 dot(default, default, transpose_a=True/False, transpose_b=True/False) = default
 dot(csr, default, transpose_a=True) = default
 dot(csr, default, transpose_a=True) = row_sparse
 dot(csr, default) = default
 dot(csr, row_sparse) = default
 dot(default, csr) = csr (CPU only)
 dot(default, csr, forward_stype='default') = default
 dot(default, csr, transpose_b=True, forward_stype='default') = default
If the combination of input storage types and forward_stype does not match any of the
above patterns,
will fallback and generate output with default storage.dot
.. Note::
If the storage type of the lhs is "csr", the storage type of gradient w.r.t rhs will be
"row_sparse". Only a subset of optimizers support sparse gradients, including SGD, AdaGrad
and Adam. Note that by default lazy updates is turned on, which may perform differently
from standard updates. For more details, please check the Optimization API at:
https://mxnet.incubator.apache.org/api/python/optimization/optimization.html
Defined in src/operator/tensor/dot.cc:L77
The first input
The second input
If true then transpose the first input before dot.
If true then transpose the second input before dot.
The desired storage type of the forward output given by user, if thecombination of input storage types and this hint does not matchany implemented ones, the dot operator will perform fallback operationand still produce an output of the desired storage type.
org.apache.mxnet.Symbol
Adds arguments elementwise.
The storage type of
output depends on storage types of inputselemwise_add
Adds arguments elementwise.
The storage type of
output depends on storage types of inputselemwise_add
elemwise_add
generates output with default storagefirst input
second input
org.apache.mxnet.Symbol
Divides arguments elementwise.
The storage type of
output is always denseelemwise_div
Divides arguments elementwise.
The storage type of
output is always denseelemwise_div
first input
second input
org.apache.mxnet.Symbol
Multiplies arguments elementwise.
The storage type of
output depends on storage types of inputselemwise_mul
Multiplies arguments elementwise.
The storage type of
output depends on storage types of inputselemwise_mul
elemwise_mul
generates output with default storagefirst input
second input
org.apache.mxnet.Symbol
Subtracts arguments elementwise.
The storage type of
output depends on storage types of inputselemwise_sub
Subtracts arguments elementwise.
The storage type of
output depends on storage types of inputselemwise_sub
elemwise_sub
generates output with default storagefirst input
second input
org.apache.mxnet.Symbol
Returns elementwise exponential value of the input.
..
Returns elementwise exponential value of the input.
.. math::
exp(x) = e^{x \approx 2.718}x
Example::
exp([0, 1, 2]) = [1., 2.71828175, 7.38905621]
The storage type of
output is always denseexp
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L839
The input array.
org.apache.mxnet.Symbol
Inserts a new axis of size 1 into the array shape
For example, given
with shape x
, then (2,3,4)
expand_dims(x, axis=1)
will return a new array with shape
.(2,1,3,4)
Defined in src/operator/tensor/matrix_op.cc:L346
Inserts a new axis of size 1 into the array shape
For example, given
with shape x
, then (2,3,4)
expand_dims(x, axis=1)
will return a new array with shape
.(2,1,3,4)
Defined in src/operator/tensor/matrix_op.cc:L346
Source input
Position where new axis is to be inserted. Suppose that the input NDArray
's dimension is ndim
, the range of the inserted axis is [ndim, ndim]
org.apache.mxnet.Symbol
Returns
computed elementwise on the input.exp(x)  1
This function provides greater precision than
for small values of exp(x)  1
.x
The storage type of
output depends upon the input storage type:expm1
Returns
computed elementwise on the input.exp(x)  1
This function provides greater precision than
for small values of exp(x)  1
.x
The storage type of
output depends upon the input storage type:expm1
The input array.
org.apache.mxnet.Symbol
Fill one element of each line(row for python, column for R/Julia) in lhs according to index indicated by rhs and values indicated by mhs.
Fill one element of each line(row for python, column for R/Julia) in lhs according to index indicated by rhs and values indicated by mhs. This function assume rhs uses 0based index.
Left operand to the function.
Middle operand to the function.
Right operand to the function.
org.apache.mxnet.Symbol
Returns elementwise rounded value to the nearest \
integer towards zero of the input.
Example::
fix([2.1, 1.9, 1.9, 2.1]) = [2., 1., 1., 2.]
The storage type of
output depends upon the input storage type:fix
Returns elementwise rounded value to the nearest \
integer towards zero of the input.
Example::
fix([2.1, 1.9, 1.9, 2.1]) = [2., 1., 1., 2.]
The storage type of
output depends upon the input storage type:fix
The input array.
org.apache.mxnet.Symbol
Flattens the input array into a 2D array by collapsing the higher dimensions.
..
Flattens the input array into a 2D array by collapsing the higher dimensions.
.. note:: Flatten
is deprecated. Use flatten
instead.
For an input array with shape
, (d1, d2, ..., dk)
flatten
operation reshapes
the input array into an output array of shape
.(d1, d2*...*dk)
Note that the bahavior of this function is different from numpy.ndarray.flatten,
which behaves similar to mxnet.ndarray.reshape((1,)).
Example::
x = [1,2,3],
[4,5,6],
[7,8,9]
],
[ [1,2,3],
[4,5,6],
[7,8,9]
,
flatten(x) = 1., 2., 3., 4., 5., 6., 7., 8., 9.],
[ 1., 2., 3., 4., 5., 6., 7., 8., 9.
Defined in src/operator/tensor/matrix_op.cc:L258
Input array.
org.apache.mxnet.Symbol
Reverses the order of elements along given axis while preserving array shape.
Note: reverse and flip are equivalent.
Reverses the order of elements along given axis while preserving array shape.
Note: reverse and flip are equivalent. We use reverse in the following examples.
Examples::
x = 0., 1., 2., 3., 4.],
[ 5., 6., 7., 8., 9.
reverse(x, axis=0) = 5., 6., 7., 8., 9.],
[ 0., 1., 2., 3., 4.
reverse(x, axis=1) = 4., 3., 2., 1., 0.],
[ 9., 8., 7., 6., 5.
Defined in src/operator/tensor/matrix_op.cc:L792
Input data array
The axis which to reverse elements.
org.apache.mxnet.Symbol
Returns elementwise floor of the input.
The floor of the scalar x is the largest integer i, such that i <= x.
Example::
floor([2.1, 1.9, 1.5, 1.9, 2.1]) = [3., 2., 1., 1., 2.]
The storage type of
output depends upon the input storage type:floor
Returns elementwise floor of the input.
The floor of the scalar x is the largest integer i, such that i <= x.
Example::
floor([2.1, 1.9, 1.5, 1.9, 2.1]) = [3., 2., 1., 1., 2.]
The storage type of
output depends upon the input storage type:floor
The input array.
org.apache.mxnet.Symbol
The FTML optimizer described in
*FTML  Follow the Moving Leader in Deep Learning*,
available at http://proceedings.mlr.press/v70/zheng17a/zheng17a.pdf.
..
The FTML optimizer described in
*FTML  Follow the Moving Leader in Deep Learning*,
available at http://proceedings.mlr.press/v70/zheng17a/zheng17a.pdf.
.. math::
g_t = \nabla J(W_{t1})\\
v_t = \beta_2 v_{t1} + (1  \beta_2) g_t^{2\\
d_t = \frac{ 1  \beta_1}t }{ \eta_t } (\sqrt{ \frac{ v_t }{ 1  \beta_2^{t } } + \epsilon)
\sigma_t = d_t  \beta_1 d_{t1}
z_t = \beta_1 z_{ t1 } + (1  \beta_1}t) g_t  \sigma_t W_{t1}
W_t =  \frac{ z_t }{ d_t }
Defined in src/operator/optimizer_op.cc:L447
Weight
Gradient
Internal state d_t
Internal state v_t
Internal state z_t
Learning rate.
Generally close to 0.5.
Generally close to 1.
Epsilon to prevent div 0.
Number of update.
Weight decay augments the objective function with a regularization term that penalizes large weights. The penalty scales with the square of the magnitude of each weight.
Rescale gradient to grad = rescale_grad*grad.
Clip gradient to the range of [clip_gradient, clip_gradient] If clip_gradient <= 0, gradient clipping is turned off. grad = max(min(grad, clip_gradient), clip_gradient).
org.apache.mxnet.Symbol
Update function for Ftrl optimizer.
Referenced from *Ad Click Prediction: a View from the Trenches*, available at
http://dl.acm.org/citation.cfm?id=2488200.
It updates the weights using::
rescaled_grad = clip(grad * rescale_grad, clip_gradient)
z += rescaled_grad  (sqrt(n + rescaled_grad**2)  sqrt(n)) * weight / learning_rate
n += rescaled_grad**2
w = (sign(z) * lamda1  z) / ((beta + sqrt(n)) / learning_rate + wd) * (abs(z) > lamda1)
If w, z and n are all of
storage type,row_sparse
only the row slices whose indices appear in grad.indices are updated (for w, z and n)::
for row in grad.indices:
rescaled_grad[row] = clip(grad[row] * rescale_grad, clip_gradient)
z[row] += rescaled_grad[row]  (sqrt(n[row] + rescaled_grad[row]**2)  sqrt(n[row])) * weight[row] / learning_rate
n[row] += rescaled_grad[row]**2
w[row] = (sign(z[row]) * lamda1  z[row]) / ((beta + sqrt(n[row])) / learning_rate + wd) * (abs(z[row]) > lamda1)
Defined in src/operator/optimizer_op.cc:L632
Update function for Ftrl optimizer.
Referenced from *Ad Click Prediction: a View from the Trenches*, available at
http://dl.acm.org/citation.cfm?id=2488200.
It updates the weights using::
rescaled_grad = clip(grad * rescale_grad, clip_gradient)
z += rescaled_grad  (sqrt(n + rescaled_grad**2)  sqrt(n)) * weight / learning_rate
n += rescaled_grad**2
w = (sign(z) * lamda1  z) / ((beta + sqrt(n)) / learning_rate + wd) * (abs(z) > lamda1)
If w, z and n are all of
storage type,row_sparse
only the row slices whose indices appear in grad.indices are updated (for w, z and n)::
for row in grad.indices:
rescaled_grad[row] = clip(grad[row] * rescale_grad, clip_gradient)
z[row] += rescaled_grad[row]  (sqrt(n[row] + rescaled_grad[row]**2)  sqrt(n[row])) * weight[row] / learning_rate
n[row] += rescaled_grad[row]**2
w[row] = (sign(z[row]) * lamda1  z[row]) / ((beta + sqrt(n[row])) / learning_rate + wd) * (abs(z[row]) > lamda1)
Defined in src/operator/optimizer_op.cc:L632
Weight
Gradient
z
Square of grad
Learning rate
The L1 regularization coefficient.
PerCoordinate Learning Rate beta.
Weight decay augments the objective function with a regularization term that penalizes large weights. The penalty scales with the square of the magnitude of each weight.
Rescale gradient to grad = rescale_grad*grad.
Clip gradient to the range of [clip_gradient, clip_gradient] If clip_gradient <= 0, gradient clipping is turned off. grad = max(min(grad, clip_gradient), clip_gradient).
org.apache.mxnet.Symbol
Returns the gamma function (extension of the factorial function \
to the reals), computed elementwise on the input array.
The storage type of
output is always densegamma
Returns the gamma function (extension of the factorial function \
to the reals), computed elementwise on the input array.
The storage type of
output is always densegamma
The input array.
org.apache.mxnet.Symbol
Returns elementwise log of the absolute value of the gamma function \
of the input.
The storage type of
output is always densegammaln
Returns elementwise log of the absolute value of the gamma function \
of the input.
The storage type of
output is always densegammaln
The input array.
org.apache.mxnet.Symbol
Gather elements or slices from data
and store to a tensor whose
shape is defined by indices
.
Given data
with shape (X_0, X_1, ..., X_{N1})
and indices with shape
(M, Y_0, ..., Y_{K1})
, the output will have shape (Y_0, ..., Y_{K1}, X_M, ..., X_{N1})
,
where M <= N
.
Gather elements or slices from data
and store to a tensor whose
shape is defined by indices
.
Given data
with shape (X_0, X_1, ..., X_{N1})
and indices with shape
(M, Y_0, ..., Y_{K1})
, the output will have shape (Y_0, ..., Y_{K1}, X_M, ..., X_{N1})
,
where M <= N
. If M == N
, output shape will simply be (Y_0, ..., Y_{K1})
.
The elements in output is defined as follows::
output[y_0, ..., y_{K1}, x_M, ..., x_{N1}] = data[indices[0, y_0, ..., y_{K1}],
...,
indices[M1, y_0, ..., y_{K1}],
x_M, ..., x_{N1}]
Examples::
data = 1], [2, 3
indices = 1, 0], [0, 1, 0
gather_nd(data, indices) = [2, 3, 0]
data
indices
org.apache.mxnet.Symbol
Computes hard sigmoid of x elementwise.
..
Computes hard sigmoid of x elementwise.
.. math::
y = max(0, min(1, alpha * x + beta))
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L115
The input array.
Slope of hard sigmoid
Bias of hard sigmoid.
org.apache.mxnet.Symbol
Returns a copy of the input.
From:src/operator/tensor/elemwise_unary_op_basic.cc:200
Returns a copy of the input.
From:src/operator/tensor/elemwise_unary_op_basic.cc:200
The input array.
org.apache.mxnet.Symbol
Computes the KhatriRao product of the input matrices.
Given a collection of :math:n
input matrices,
..
Computes the KhatriRao product of the input matrices.
Given a collection of :math:n
input matrices,
.. math::
A_1 \in \mathbb{R}^{{M_1 \times M}, \ldots, A_n \in \mathbb{R}}{M_n \times N},
the (columnwise) KhatriRao product is defined as the matrix,
.. math::
X = A_1 \otimes \cdots \otimes A_n \in \mathbb{R}^{(M_1 \cdots M_n) \times N},
where the :math:k
th column is equal to the columnwise outer product
:math:{A_1}_k \otimes \cdots \otimes {A_n}_k
where :math:{A_i}_k
is the kth
column of the ith matrix.
Example::
>>> A = mx.nd.array(1],
>>> [2, 3)
>>> B = mx.nd.array(4],
>>> [2, 5],
>>> [3, 6)
>>> C = mx.nd.khatri_rao(A, B)
>>> print(C.asnumpy())
1. 4.]
[ 2. 5.]
[ 3. 6.]
[ 2. 12.]
[ 4. 15.]
[ 6. 18.
Defined in src/operator/contrib/krprod.cc:L108
Positional input matrices
org.apache.mxnet.Symbol
LQ factorization for general matrix.
Input is a tensor *A* of dimension *n >= 2*.
If *n=2*, we compute the LQ factorization (LAPACK *gelqf*, followed by *orglq*).
LQ factorization for general matrix.
Input is a tensor *A* of dimension *n >= 2*.
If *n=2*, we compute the LQ factorization (LAPACK *gelqf*, followed by *orglq*). *A*
must have shape *(x, y)* with *x <= y*, and must have full rank *=x*. The LQ
factorization consists of *L* with shape *(x, x)* and *Q* with shape *(x, y)*, so
that:
*A* = *L* \* *Q*
Here, *L* is lower triangular (upper triangle equal to zero) with nonzero diagonal,
and *Q* is roworthonormal, meaning that
*Q* \* *Q*\ :sup:T
is equal to the identity matrix of shape *(x, x)*.
If *n>2*, *gelqf* is performed separately on the trailing two dimensions for all
inputs (batch mode).
.. note:: The operator supports float32 and float64 data types only.
Examples::
// Single LQ factorization
A = 2., 3.], [4., 5., 6.
Q, L = gelqf(A)
Q = 0.53452248, 0.80178373],
[0.87287156, 0.21821789, 0.43643578
L = 0.],
[8.55235974, 1.96396101
// Batch LQ factorization
A = 2., 3.], [4., 5., 6.]],
8., 9.], [10., 11., 12.]
Q, L = gelqf(A)
Q = 0.53452248, 0.80178373],
[0.87287156, 0.21821789, 0.43643578]],
0.57436653, 0.64616234],
[0.7620735, 0.05862104, 0.64483142]
L = 0.],
[8.55235974, 1.96396101]],
0.],
[19.09768702, 0.52758934]
Defined in src/operator/tensor/la_op.cc:L552
Tensor of input matrices to be factorized
org.apache.mxnet.Symbol
Performs general matrix multiplication and accumulation.
Input are tensors *A*, *B*, *C*, each of dimension *n >= 2* and having the same shape
on the leading *n2* dimensions.
If *n=2*, the BLAS3 function *gemm* is performed:
*out* = *alpha* \* *op*\ (*A*) \* *op*\ (*B*) + *beta* \* *C*
Here, *alpha* and *beta* are scalar parameters, and *op()* is either the identity or
matrix transposition (depending on *transpose_a*, *transpose_b*).
If *n>2*, *gemm* is performed separately for a batch of matrices.
Performs general matrix multiplication and accumulation.
Input are tensors *A*, *B*, *C*, each of dimension *n >= 2* and having the same shape
on the leading *n2* dimensions.
If *n=2*, the BLAS3 function *gemm* is performed:
*out* = *alpha* \* *op*\ (*A*) \* *op*\ (*B*) + *beta* \* *C*
Here, *alpha* and *beta* are scalar parameters, and *op()* is either the identity or
matrix transposition (depending on *transpose_a*, *transpose_b*).
If *n>2*, *gemm* is performed separately for a batch of matrices. The column indices of the matrices
are given by the last dimensions of the tensors, the row indices by the axis specified with the *axis*
parameter. By default, the trailing two dimensions will be used for matrix encoding.
For a nondefault axis parameter, the operation performed is equivalent to a series of swapaxes/gemm/swapaxes
calls. For example let *A*, *B*, *C* be 5 dimensional tensors. Then gemm(*A*, *B*, *C*, axis=1) is equivalent to
A1 = swapaxes(A, dim1=1, dim2=3)
B1 = swapaxes(B, dim1=1, dim2=3)
C = swapaxes(C, dim1=1, dim2=3)
C = gemm(A1, B1, C)
C = swapaxis(C, dim1=1, dim2=3)
without the overhead of the additional swapaxis operations.
.. note:: The operator supports float32 and float64 data types only.
Examples::
// Single matrix multiplyadd
A = 1.0], [1.0, 1.0
B = 1.0], [1.0, 1.0], [1.0, 1.0
C = 1.0, 1.0], [1.0, 1.0, 1.0
gemm(A, B, C, transpose_b=True, alpha=2.0, beta=10.0)
B = 1.0]], 0.1]
C = 0.01]
gemm(A, B, C, transpose_b=True, alpha=2.0 , beta=10.0)
Tensor of input matrices
Tensor of input matrices
Tensor of input matrices
Multiply with transposed of first input (A).
Multiply with transposed of second input (B).
Scalar factor multiplied with A*B.
Scalar factor multiplied with C.
Axis corresponding to the matrix rows.
org.apache.mxnet.Symbol
Performs general matrix multiplication.
Input are tensors *A*, *B*, each of dimension *n >= 2* and having the same shape
on the leading *n2* dimensions.
If *n=2*, the BLAS3 function *gemm* is performed:
*out* = *alpha* \* *op*\ (*A*) \* *op*\ (*B*)
Here *alpha* is a scalar parameter and *op()* is either the identity or the matrix
transposition (depending on *transpose_a*, *transpose_b*).
If *n>2*, *gemm* is performed separately for a batch of matrices.
Performs general matrix multiplication.
Input are tensors *A*, *B*, each of dimension *n >= 2* and having the same shape
on the leading *n2* dimensions.
If *n=2*, the BLAS3 function *gemm* is performed:
*out* = *alpha* \* *op*\ (*A*) \* *op*\ (*B*)
Here *alpha* is a scalar parameter and *op()* is either the identity or the matrix
transposition (depending on *transpose_a*, *transpose_b*).
If *n>2*, *gemm* is performed separately for a batch of matrices. The column indices of the matrices
are given by the last dimensions of the tensors, the row indices by the axis specified with the *axis*
parameter. By default, the trailing two dimensions will be used for matrix encoding.
For a nondefault axis parameter, the operation performed is equivalent to a series of swapaxes/gemm/swapaxes
calls. For example let *A*, *B* be 5 dimensional tensors. Then gemm(*A*, *B*, axis=1) is equivalent to
A1 = swapaxes(A, dim1=1, dim2=3)
B1 = swapaxes(B, dim1=1, dim2=3)
C = gemm2(A1, B1)
C = swapaxis(C, dim1=1, dim2=3)
without the overhead of the additional swapaxis operations.
.. note:: The operator supports float32 and float64 data types only.
Examples::
// Single matrix multiply
A = 1.0], [1.0, 1.0
B = 1.0], [1.0, 1.0], [1.0, 1.0
gemm2(A, B, transpose_b=True, alpha=2.0)
B = 1.0]], 0.1]
gemm2(A, B, transpose_b=True, alpha=2.0)
Tensor of input matrices
Tensor of input matrices
Multiply with transposed of first input (A).
Multiply with transposed of second input (B).
Scalar factor multiplied with A*B.
Axis corresponding to the matrix row indices.
org.apache.mxnet.Symbol
Performs Cholesky factorization of a symmetric positivedefinite matrix.
Input is a tensor *A* of dimension *n >= 2*.
If *n=2*, the Cholesky factor *L* of the symmetric, positive definite matrix *A* is
computed.
Performs Cholesky factorization of a symmetric positivedefinite matrix.
Input is a tensor *A* of dimension *n >= 2*.
If *n=2*, the Cholesky factor *L* of the symmetric, positive definite matrix *A* is
computed. *L* is lower triangular (entries of upper triangle are all zero), has
positive diagonal entries, and:
*A* = *L* \* *L*\ :sup:T
If *n>2*, *potrf* is performed separately on the trailing two dimensions for all inputs
(batch mode).
.. note:: The operator supports float32 and float64 data types only.
Examples::
// Single matrix factorization
A = 1.0], [1.0, 4.25
potrf(A) = 0], [0.5, 2.0
// Batch matrix factorization
A = 1.0], [1.0, 4.25]], 4.0], [4.0, 17.0]
potrf(A) = 0], [0.5, 2.0]], 0], [1.0, 4.0]
Defined in src/operator/tensor/la_op.cc:L201
Tensor of input matrices to be decomposed
org.apache.mxnet.Symbol
Performs matrix inversion from a Cholesky factorization.
Input is a tensor *A* of dimension *n >= 2*.
If *n=2*, *A* is a lower triangular matrix (entries of upper triangle are all zero)
with positive diagonal.
Performs matrix inversion from a Cholesky factorization.
Input is a tensor *A* of dimension *n >= 2*.
If *n=2*, *A* is a lower triangular matrix (entries of upper triangle are all zero)
with positive diagonal. We compute:
*out* = *A*\ :sup:T
\* *A*\ :sup:1
In other words, if *A* is the Cholesky factor of a symmetric positive definite matrix
*B* (obtained by *potrf*), then
*out* = *B*\ :sup:1
If *n>2*, *potri* is performed separately on the trailing two dimensions for all inputs
(batch mode).
.. note:: The operator supports float32 and float64 data types only.
.. note:: Use this operator only if you are certain you need the inverse of *B*, and
cannot use the Cholesky factor *A* (*potrf*), together with backsubstitution
(*trsm*). The latter is numerically much safer, and also cheaper.
Examples::
// Single matrix inverse
A = 0], [0.5, 2.0
potri(A) = 0.0625], [0.0625, 0.25
// Batch matrix inverse
A = 0], [0.5, 2.0]], 0], [1.0, 4.0]
potri(A) = 0.0625], [0.0625, 0.25]],
0.01562], [0.01562, 0,0625]
Defined in src/operator/tensor/la_op.cc:L259
Tensor of lower triangular matrices
org.apache.mxnet.Symbol
Computes the sum of the logarithms of the diagonal elements of a square matrix.
Input is a tensor *A* of dimension *n >= 2*.
If *n=2*, *A* must be square with positive diagonal entries.
Computes the sum of the logarithms of the diagonal elements of a square matrix.
Input is a tensor *A* of dimension *n >= 2*.
If *n=2*, *A* must be square with positive diagonal entries. We sum the natural
logarithms of the diagonal elements, the result has shape (1,).
If *n>2*, *sumlogdiag* is performed separately on the trailing two dimensions for all
inputs (batch mode).
.. note:: The operator supports float32 and float64 data types only.
Examples::
// Single matrix reduction
A = 1.0], [1.0, 7.0
sumlogdiag(A) = [1.9459]
// Batch matrix reduction
A = 1.0], [1.0, 7.0]], 0], [0, 17.0]
sumlogdiag(A) = [1.9459, 3.9318]
Defined in src/operator/tensor/la_op.cc:L428
Tensor of square matrices
org.apache.mxnet.Symbol
Multiplication of matrix with its transpose.
Input is a tensor *A* of dimension *n >= 2*.
If *n=2*, the operator performs the BLAS3 function *syrk*:
*out* = *alpha* \* *A* \* *A*\ :sup:T
if *transpose=False*, or
*out* = *alpha* \* *A*\ :sup:T
\ \* *A*
if *transpose=True*.
If *n>2*, *syrk* is performed separately on the trailing two dimensions for all
inputs (batch mode).
..
Multiplication of matrix with its transpose.
Input is a tensor *A* of dimension *n >= 2*.
If *n=2*, the operator performs the BLAS3 function *syrk*:
*out* = *alpha* \* *A* \* *A*\ :sup:T
if *transpose=False*, or
*out* = *alpha* \* *A*\ :sup:T
\ \* *A*
if *transpose=True*.
If *n>2*, *syrk* is performed separately on the trailing two dimensions for all
inputs (batch mode).
.. note:: The operator supports float32 and float64 data types only.
Examples::
// Single matrix multiply
A = 2., 3.], [4., 5., 6.
syrk(A, alpha=1., transpose=False)
syrk(A, alpha=2., transpose=False) = 0.04]
Defined in src/operator/tensor/la_op.cc:L484
Tensor of input matrices
Use transpose of input matrix.
Scalar factor to be applied to the result.
org.apache.mxnet.Symbol
Performs multiplication with a lower triangular matrix.
Input are tensors *A*, *B*, each of dimension *n >= 2* and having the same shape
on the leading *n2* dimensions.
If *n=2*, *A* must be lower triangular.
Performs multiplication with a lower triangular matrix.
Input are tensors *A*, *B*, each of dimension *n >= 2* and having the same shape
on the leading *n2* dimensions.
If *n=2*, *A* must be lower triangular. The operator performs the BLAS3 function
*trmm*:
*out* = *alpha* \* *op*\ (*A*) \* *B*
if *rightside=False*, or
*out* = *alpha* \* *B* \* *op*\ (*A*)
if *rightside=True*. Here, *alpha* is a scalar parameter, and *op()* is either the
identity or the matrix transposition (depending on *transpose*).
If *n>2*, *trmm* is performed separately on the trailing two dimensions for all inputs
(batch mode).
.. note:: The operator supports float32 and float64 data types only.
Examples::
// Single triangular matrix multiply
A = 0], [1.0, 1.0
B = 1.0, 1.0], [1.0, 1.0, 1.0
trmm(A, B, alpha=2.0) = 2.0, 2.0], [4.0, 4.0, 4.0
// Batch triangular matrix multiply
A = 0], [1.0, 1.0]], 0], [1.0, 1.0]
B = 1.0, 1.0], [1.0, 1.0, 1.0]], 0.5, 0.5], [0.5, 0.5, 0.5]
trmm(A, B, alpha=2.0) = 2.0, 2.0], [4.0, 4.0, 4.0]],
1.0, 1.0], [2.0, 2.0, 2.0]
Defined in src/operator/tensor/la_op.cc:L316
Tensor of lower triangular matrices
Tensor of matrices
Use transposed of the triangular matrix
Multiply triangular matrix from the right to nontriangular one.
Scalar factor to be applied to the result.
org.apache.mxnet.Symbol
Solves matrix equation involving a lower triangular matrix.
Input are tensors *A*, *B*, each of dimension *n >= 2* and having the same shape
on the leading *n2* dimensions.
If *n=2*, *A* must be lower triangular.
Solves matrix equation involving a lower triangular matrix.
Input are tensors *A*, *B*, each of dimension *n >= 2* and having the same shape
on the leading *n2* dimensions.
If *n=2*, *A* must be lower triangular. The operator performs the BLAS3 function
*trsm*, solving for *out* in:
*op*\ (*A*) \* *out* = *alpha* \* *B*
if *rightside=False*, or
*out* \* *op*\ (*A*) = *alpha* \* *B*
if *rightside=True*. Here, *alpha* is a scalar parameter, and *op()* is either the
identity or the matrix transposition (depending on *transpose*).
If *n>2*, *trsm* is performed separately on the trailing two dimensions for all inputs
(batch mode).
.. note:: The operator supports float32 and float64 data types only.
Examples::
// Single matrix solve
A = 0], [1.0, 1.0
B = 2.0, 2.0], [4.0, 4.0, 4.0
trsm(A, B, alpha=0.5) = 1.0, 1.0], [1.0, 1.0, 1.0
// Batch matrix solve
A = 0], [1.0, 1.0]], 0], [1.0, 1.0]
B = 2.0, 2.0], [4.0, 4.0, 4.0]],
4.0, 4.0], [8.0, 8.0, 8.0]
trsm(A, B, alpha=0.5) = 1.0, 1.0], [1.0, 1.0, 1.0]],
2.0, 2.0], [2.0, 2.0, 2.0]
Defined in src/operator/tensor/la_op.cc:L379
Tensor of lower triangular matrices
Tensor of matrices
Use transposed of the triangular matrix
Multiply triangular matrix from the right to nontriangular one.
Scalar factor to be applied to the result.
org.apache.mxnet.Symbol
Returns elementwise Natural logarithmic value of the input.
The natural logarithm is logarithm in base *e*, so that log(exp(x)) = x
The storage type of
output is always denselog
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L851
Returns elementwise Natural logarithmic value of the input.
The natural logarithm is logarithm in base *e*, so that log(exp(x)) = x
The storage type of
output is always denselog
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L851
The input array.
org.apache.mxnet.Symbol
Returns elementwise Base10 logarithmic value of the input.
10**log10(x) = x
The storage type of
output is always denselog10
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L863
Returns elementwise Base10 logarithmic value of the input.
10**log10(x) = x
The storage type of
output is always denselog10
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L863
The input array.
org.apache.mxnet.Symbol
Returns elementwise
value of the input.log(1 + x)
This function is more accurate than
for small log(1 + x)
so thatx
:math:1+x\approx 1
The storage type of
output depends upon the input storage type:log1p
Returns elementwise
value of the input.log(1 + x)
This function is more accurate than
for small log(1 + x)
so thatx
:math:1+x\approx 1
The storage type of
output depends upon the input storage type:log1p
The input array.
org.apache.mxnet.Symbol
Returns elementwise Base2 logarithmic value of the input.
2**log2(x) = x
The storage type of
output is always denselog2
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L875
Returns elementwise Base2 logarithmic value of the input.
2**log2(x) = x
The storage type of
output is always denselog2
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L875
The input array.
org.apache.mxnet.Symbol
Computes the log softmax of the input.
This is equivalent to computing softmax followed by log.
Examples::
>>> x = mx.nd.array([1, 2, .1])
>>> mx.nd.log_softmax(x).asnumpy()
array([1.41702998, 0.41702995, 2.31702995], dtype=float32)
>>> x = mx.nd.array( 2, .1],[.1, 2, 1 )
>>> mx.nd.log_softmax(x, axis=0).asnumpy()
array(0.69314718, 1.24115396],
[1.24115396, 0.69314718, 0.34115392, dtype=float32)
Computes the log softmax of the input.
This is equivalent to computing softmax followed by log.
Examples::
>>> x = mx.nd.array([1, 2, .1])
>>> mx.nd.log_softmax(x).asnumpy()
array([1.41702998, 0.41702995, 2.31702995], dtype=float32)
>>> x = mx.nd.array( 2, .1],[.1, 2, 1 )
>>> mx.nd.log_softmax(x, axis=0).asnumpy()
array(0.69314718, 1.24115396],
[1.24115396, 0.69314718, 0.34115392, dtype=float32)
The input array.
The axis along which to compute softmax.
Temperature parameter in softmax
org.apache.mxnet.Symbol
Returns the result of logical NOT (!) function
Example:
logical_not([2., 0., 1.]) = [0., 1., 0.]
Returns the result of logical NOT (!) function
Example:
logical_not([2., 0., 1.]) = [0., 1., 0.]
The input array.
org.apache.mxnet.Symbol
Make your own loss function in network construction.
This operator accepts a customized loss function symbol as a terminal loss and
the symbol should be an operator with no backward dependency.
The output of this function is the gradient of loss with respect to the input data.
For example, if you are a making a cross entropy loss function.
Make your own loss function in network construction.
This operator accepts a customized loss function symbol as a terminal loss and
the symbol should be an operator with no backward dependency.
The output of this function is the gradient of loss with respect to the input data.
For example, if you are a making a cross entropy loss function. Assume
is theout
predicted output and
is the true label, then the cross entropy can be defined as::label
cross_entropy = label * log(out) + (1  label) * log(1  out)
loss = make_loss(cross_entropy)
We will need to use
when we are creating our own loss function or we want tomake_loss
combine multiple loss functions. Also we may want to stop some variables' gradients
from backpropagation. See more detail in
or BlockGrad
.stop_gradient
The storage type of
output depends upon the input storage type:make_loss
The input array.
org.apache.mxnet.Symbol
Computes the max of array elements over given axes.
Defined in src/operator/tensor/broadcast_reduce_op_value.cc:L190
Computes the max of array elements over given axes.
Defined in src/operator/tensor/broadcast_reduce_op_value.cc:L190
The input
The axis or axes along which to perform the reduction.
The default, axis=()
, will compute over all elements into a
scalar array with shape (1,)
.
If axis
is int, a reduction is performed on a particular axis.
If axis
is a tuple of ints, a reduction is performed on all the axes
specified in the tuple.
If exclude
is true, reduction will be performed on the axes that are
NOT in axis instead.
Negative values means indexing from right to left.
If this is set to True
, the reduced axes are left in the result as dimension with size one.
Whether to perform reduction on axis that are NOT in axis instead.
org.apache.mxnet.Symbol
Computes the max of array elements over given axes.
Defined in src/operator/tensor/broadcast_reduce_op_value.cc:L190
Computes the max of array elements over given axes.
Defined in src/operator/tensor/broadcast_reduce_op_value.cc:L190
The input
The axis or axes along which to perform the reduction.
The default, axis=()
, will compute over all elements into a
scalar array with shape (1,)
.
If axis
is int, a reduction is performed on a particular axis.
If axis
is a tuple of ints, a reduction is performed on all the axes
specified in the tuple.
If exclude
is true, reduction will be performed on the axes that are
NOT in axis instead.
Negative values means indexing from right to left.
If this is set to True
, the reduced axes are left in the result as dimension with size one.
Whether to perform reduction on axis that are NOT in axis instead.
org.apache.mxnet.Symbol
Computes the mean of array elements over given axes.
Defined in src/operator/tensor/broadcast_reduce_op_value.cc:L131
Computes the mean of array elements over given axes.
Defined in src/operator/tensor/broadcast_reduce_op_value.cc:L131
The input
The axis or axes along which to perform the reduction.
The default, axis=()
, will compute over all elements into a
scalar array with shape (1,)
.
If axis
is int, a reduction is performed on a particular axis.
If axis
is a tuple of ints, a reduction is performed on all the axes
specified in the tuple.
If exclude
is true, reduction will be performed on the axes that are
NOT in axis instead.
Negative values means indexing from right to left.
If this is set to True
, the reduced axes are left in the result as dimension with size one.
Whether to perform reduction on axis that are NOT in axis instead.
org.apache.mxnet.Symbol
Computes the min of array elements over given axes.
Defined in src/operator/tensor/broadcast_reduce_op_value.cc:L204
Computes the min of array elements over given axes.
Defined in src/operator/tensor/broadcast_reduce_op_value.cc:L204
The input
The axis or axes along which to perform the reduction.
The default, axis=()
, will compute over all elements into a
scalar array with shape (1,)
.
If axis
is int, a reduction is performed on a particular axis.
If axis
is a tuple of ints, a reduction is performed on all the axes
specified in the tuple.
If exclude
is true, reduction will be performed on the axes that are
NOT in axis instead.
Negative values means indexing from right to left.
If this is set to True
, the reduced axes are left in the result as dimension with size one.
Whether to perform reduction on axis that are NOT in axis instead.
org.apache.mxnet.Symbol
Computes the min of array elements over given axes.
Defined in src/operator/tensor/broadcast_reduce_op_value.cc:L204
Computes the min of array elements over given axes.
Defined in src/operator/tensor/broadcast_reduce_op_value.cc:L204
The input
The axis or axes along which to perform the reduction.
The default, axis=()
, will compute over all elements into a
scalar array with shape (1,)
.
If axis
is int, a reduction is performed on a particular axis.
If axis
is a tuple of ints, a reduction is performed on all the axes
specified in the tuple.
If exclude
is true, reduction will be performed on the axes that are
NOT in axis instead.
Negative values means indexing from right to left.
If this is set to True
, the reduced axes are left in the result as dimension with size one.
Whether to perform reduction on axis that are NOT in axis instead.
org.apache.mxnet.Symbol
Updater function for multiprecision sgd optimizer
Updater function for multiprecision sgd optimizer
Weight
Gradient
Momentum
Weight32
Learning rate
The decay rate of momentum estimates at each epoch.
Weight decay augments the objective function with a regularization term that penalizes large weights. The penalty scales with the square of the magnitude of each weight.
Rescale gradient to grad = rescale_grad*grad.
Clip gradient to the range of [clip_gradient, clip_gradient] If clip_gradient <= 0, gradient clipping is turned off. grad = max(min(grad, clip_gradient), clip_gradient).
If true, lazy updates are applied if gradient's stype is row_sparse and both weight and momentum have the same stype
org.apache.mxnet.Symbol
Updater function for multiprecision sgd optimizer
Updater function for multiprecision sgd optimizer
Weight
gradient
Weight32
Learning rate
Weight decay augments the objective function with a regularization term that penalizes large weights. The penalty scales with the square of the magnitude of each weight.
Rescale gradient to grad = rescale_grad*grad.
Clip gradient to the range of [clip_gradient, clip_gradient] If clip_gradient <= 0, gradient clipping is turned off. grad = max(min(grad, clip_gradient), clip_gradient).
If true, lazy updates are applied if gradient's stype is row_sparse.
org.apache.mxnet.Symbol
Computes the product of array elements over given axes treating Not a Numbers (
) as one.NaN
Defined in src/operator/tensor/broadcast_reduce_op_value.cc:L176
Computes the product of array elements over given axes treating Not a Numbers (
) as one.NaN
Defined in src/operator/tensor/broadcast_reduce_op_value.cc:L176
The input
The axis or axes along which to perform the reduction.
The default, axis=()
, will compute over all elements into a
scalar array with shape (1,)
.
If axis
is int, a reduction is performed on a particular axis.
If axis
is a tuple of ints, a reduction is performed on all the axes
specified in the tuple.
If exclude
is true, reduction will be performed on the axes that are
NOT in axis instead.
Negative values means indexing from right to left.
If this is set to True
, the reduced axes are left in the result as dimension with size one.
Whether to perform reduction on axis that are NOT in axis instead.
org.apache.mxnet.Symbol
Computes the sum of array elements over given axes treating Not a Numbers (
) as zero.NaN
Defined in src/operator/tensor/broadcast_reduce_op_value.cc:L161
Computes the sum of array elements over given axes treating Not a Numbers (
) as zero.NaN
Defined in src/operator/tensor/broadcast_reduce_op_value.cc:L161
The input
The axis or axes along which to perform the reduction.
The default, axis=()
, will compute over all elements into a
scalar array with shape (1,)
.
If axis
is int, a reduction is performed on a particular axis.
If axis
is a tuple of ints, a reduction is performed on all the axes
specified in the tuple.
If exclude
is true, reduction will be performed on the axes that are
NOT in axis instead.
Negative values means indexing from right to left.
If this is set to True
, the reduced axes are left in the result as dimension with size one.
Whether to perform reduction on axis that are NOT in axis instead.
org.apache.mxnet.Symbol
Numerical negative of the argument, elementwise.
The storage type of
output depends upon the input storage type:negative
Numerical negative of the argument, elementwise.
The storage type of
output depends upon the input storage type:negative
The input array.
org.apache.mxnet.Symbol
Computes the norm on an NDArray.
This operator computes the norm on an NDArray with the specified axis, depending
on the value of the ord parameter.
Computes the norm on an NDArray.
This operator computes the norm on an NDArray with the specified axis, depending
on the value of the ord parameter. By default, it computes the L2 norm on the entire
array. Currently only ord=2 supports sparse ndarrays.
Examples::
x = 2],
[3, 4]],
2],
[5, 6]
norm(x, ord=2, axis=1) = 4.472136 ]
[5.3851647 6.3245554
norm(x, ord=1, axis=1) = 6.],
[7., 8.
rsp = x.cast_storage('row_sparse')
norm(rsp) = [5.47722578]
csr = x.cast_storage('csr')
norm(csr) = [5.47722578]
Defined in src/operator/tensor/broadcast_reduce_op_value.cc:L345
The input
Order of the norm. Currently ord=1 and ord=2 is supported.
The axis or axes along which to perform the reduction.
The default, axis=()
, will compute over all elements into a
scalar array with shape (1,)
.
If axis
is int, a reduction is performed on a particular axis.
If axis
is a 2tuple, it specifies the axes that hold 2D matrices,
and the matrix norms of these matrices are computed.
If this is set to True
, the reduced axis is left in the result as dimension with size one.
org.apache.mxnet.Symbol
Draw random samples from a normal (Gaussian) distribution.
..
Draw random samples from a normal (Gaussian) distribution.
.. note:: The existing alias
is deprecated.normal
Samples are distributed according to a normal distribution parametrized by *loc* (mean) and *scale* (standard deviation).
Example::
normal(loc=0, scale=1, shape=(2,2)) = 1.89171135, 1.16881478],
[1.23474145, 1.55807114
Defined in src/operator/random/sample_op.cc:L85
Mean of the distribution.
Standard deviation of the distribution.
Shape of the output.
Context of output, in format [cpugpucpu_pinned](n). Only used for imperative calls.
DType of the output in case this can't be inferred. Defaults to float32 if not defined (dtype=None).
org.apache.mxnet.Symbol
Returns a onehot array.
The locations represented by indices
take value on_value
, while all
other locations take value off_value
.
one_hot
operation with indices
of shape
and (i0, i1)
depth
of
would resultd
in an output array of shape
with::(i0, i1, d)
output[i,j,:] = off_value
output[i,j,indices[i,j]] = on_value
Examples::
one_hot([1,0,2,0], 3) = 0. 1. 0.]
[ 1. 0. 0.]
[ 0. 0. 1.]
[ 1. 0. 0.
one_hot([1,0,2,0], 3, on_value=8, off_value=1,
dtype='int32') = 8 1]
[8 1 1]
[1 1 8]
[8 1 1
one_hot(1,0],[1,0],[2,0, 3) = 0. 1. 0.]
[ 1. 0. 0.]]
0. 1. 0.]
[ 1. 0. 0.
0. 0. 1.]
[ 1. 0. 0.]
Defined in src/operator/tensor/indexing_op.cc:L508
Returns a onehot array.
The locations represented by indices
take value on_value
, while all
other locations take value off_value
.
one_hot
operation with indices
of shape
and (i0, i1)
depth
of
would resultd
in an output array of shape
with::(i0, i1, d)
output[i,j,:] = off_value
output[i,j,indices[i,j]] = on_value
Examples::
one_hot([1,0,2,0], 3) = 0. 1. 0.]
[ 1. 0. 0.]
[ 0. 0. 1.]
[ 1. 0. 0.
one_hot([1,0,2,0], 3, on_value=8, off_value=1,
dtype='int32') = 8 1]
[8 1 1]
[1 1 8]
[8 1 1
one_hot(1,0],[1,0],[2,0, 3) = 0. 1. 0.]
[ 1. 0. 0.]]
0. 1. 0.]
[ 1. 0. 0.
0. 0. 1.]
[ 1. 0. 0.]
Defined in src/operator/tensor/indexing_op.cc:L508
array of locations where to set on_value
Depth of the one hot dimension.
The value assigned to the locations represented by indices.
The value assigned to the locations not represented by indices.
DType of the output
org.apache.mxnet.Symbol
Return an array of ones with the same shape and type
as the input array.
Examples::
x = 0., 0., 0.],
[ 0., 0., 0.
ones_like(x) = 1., 1., 1.],
[ 1., 1., 1.
Return an array of ones with the same shape and type
as the input array.
Examples::
x = 0., 0., 0.],
[ 0., 0., 0.
ones_like(x) = 1., 1., 1.],
[ 1., 1., 1.
The input
org.apache.mxnet.Symbol
Pads an input array with a constant or edge values of the array.
..
Pads an input array with a constant or edge values of the array.
.. note:: Pad
is deprecated. Use pad
instead.
.. note:: Current implementation only supports 4D and 5D input arrays with padding applied
only on axes 1, 2 and 3. Expects axes 4 and 5 in pad_width
to be zero.
This operation pads an input array with either a constant_value
or edge values
along each axis of the input array. The amount of padding is specified by pad_width
.
pad_width
is a tuple of integer padding widths for each axis of the format
. The (before_1, after_1, ... , before_N, after_N)
pad_width
should be of length 2*N
where
is the number of dimensions of the array.N
For dimension
of the input array, N
and before_N
indicates how many valuesafter_N
to add before and after the elements of the array along dimension
.N
The widths of the higher two dimensions
, before_1
, after_1
,before_2
must be 0.after_2
Example::
x = 1. 2. 3.]
[ 4. 5. 6.]]
7. 8. 9.]
[ 10. 11. 12.]
11. 12. 13.]
[ 14. 15. 16.]]
17. 18. 19.]
[ 20. 21. 22.]]
pad(x,mode="edge", pad_width=(0,0,0,0,1,1,1,1)) =
1. 1. 2. 3. 3.]
[ 1. 1. 2. 3. 3.]
[ 4. 4. 5. 6. 6.]
[ 4. 4. 5. 6. 6.]]
7. 7. 8. 9. 9.]
[ 7. 7. 8. 9. 9.]
[ 10. 10. 11. 12. 12.]
[ 10. 10. 11. 12. 12.]
11. 11. 12. 13. 13.]
[ 11. 11. 12. 13. 13.]
[ 14. 14. 15. 16. 16.]
[ 14. 14. 15. 16. 16.]]
17. 17. 18. 19. 19.]
[ 17. 17. 18. 19. 19.]
[ 20. 20. 21. 22. 22.]
[ 20. 20. 21. 22. 22.]]
pad(x, mode="constant", constant_value=0, pad_width=(0,0,0,0,1,1,1,1)) =
0. 0. 0. 0. 0.]
[ 0. 1. 2. 3. 0.]
[ 0. 4. 5. 6. 0.]
[ 0. 0. 0. 0. 0.]]
0. 0. 0. 0. 0.]
[ 0. 7. 8. 9. 0.]
[ 0. 10. 11. 12. 0.]
[ 0. 0. 0. 0. 0.]
0. 0. 0. 0. 0.]
[ 0. 11. 12. 13. 0.]
[ 0. 14. 15. 16. 0.]
[ 0. 0. 0. 0. 0.]]
0. 0. 0. 0. 0.]
[ 0. 17. 18. 19. 0.]
[ 0. 20. 21. 22. 0.]
[ 0. 0. 0. 0. 0.]]
Defined in src/operator/pad.cc:L766
An ndimensional input array.
Padding type to use. "constant" pads with constant_value
"edge" pads using the edge values of the input array "reflect" pads by reflecting values with respect to the edges.
Widths of the padding regions applied to the edges of each axis. It is a tuple of integer padding widths for each axis of the format
. It should be of length (before_1, after_1, ... , before_N, after_N)
where 2*N
is the number of dimensions of the array.This is equivalent to pad_width in numpy.pad, but flattened.N
The value used for padding when mode
is "constant".
org.apache.mxnet.Symbol
Picks elements from an input array according to the input indices along the given axis.
Given an input array of shape
and indices of shape (d0, d1)
, the result will be(i0,)
an output array of shape
with::(i0,)
output[i] = input[i, indices[i]]
By default, if any index mentioned is too large, it is replaced by the index that addresses
the last element along an axis (the clip
mode).
This function supports ndimensional input and (n1)dimensional indices arrays.
Examples::
x = 1., 2.],
[ 3., 4.],
[ 5., 6.
// picks elements with specified indices along axis 0
pick(x, y=[0,1], 0) = [ 1., 4.]
// picks elements with specified indices along axis 1
pick(x, y=[0,1,0], 1) = [ 1., 4., 5.]
y = 1.],
[ 0.],
[ 2.
// picks elements with specified indices along axis 1 and dims are maintained
pick(x,y, 1, keepdims=True) = 2.],
[ 3.],
[ 6.
Defined in src/operator/tensor/broadcast_reduce_op_index.cc:L145
Picks elements from an input array according to the input indices along the given axis.
Given an input array of shape
and indices of shape (d0, d1)
, the result will be(i0,)
an output array of shape
with::(i0,)
output[i] = input[i, indices[i]]
By default, if any index mentioned is too large, it is replaced by the index that addresses
the last element along an axis (the clip
mode).
This function supports ndimensional input and (n1)dimensional indices arrays.
Examples::
x = 1., 2.],
[ 3., 4.],
[ 5., 6.
// picks elements with specified indices along axis 0
pick(x, y=[0,1], 0) = [ 1., 4.]
// picks elements with specified indices along axis 1
pick(x, y=[0,1,0], 1) = [ 1., 4., 5.]
y = 1.],
[ 0.],
[ 2.
// picks elements with specified indices along axis 1 and dims are maintained
pick(x,y, 1, keepdims=True) = 2.],
[ 3.],
[ 6.
Defined in src/operator/tensor/broadcast_reduce_op_index.cc:L145
The input array
The index array
The axis along which to perform the reduction. Negative values means indexing from right to left. Requires axis to be set as int, because global reduction is not supported yet.
If this is set to True
, the reduced axis is left in the result as dimension with size one.
org.apache.mxnet.Symbol
Computes the product of array elements over given axes.
Defined in src/operator/tensor/broadcast_reduce_op_value.cc:L146
Computes the product of array elements over given axes.
Defined in src/operator/tensor/broadcast_reduce_op_value.cc:L146
The input
The axis or axes along which to perform the reduction.
The default, axis=()
, will compute over all elements into a
scalar array with shape (1,)
.
If axis
is int, a reduction is performed on a particular axis.
If axis
is a tuple of ints, a reduction is performed on all the axes
specified in the tuple.
If exclude
is true, reduction will be performed on the axes that are
NOT in axis instead.
Negative values means indexing from right to left.
If this is set to True
, the reduced axes are left in the result as dimension with size one.
Whether to perform reduction on axis that are NOT in axis instead.
org.apache.mxnet.Symbol
Converts each element of the input array from degrees to radians.
..
Converts each element of the input array from degrees to radians.
.. math::
radians([0, 90, 180, 270, 360]) = [0, \pi/2, \pi, 3\pi/2, 2\pi]
The storage type of
output depends upon the input storage type:radians
The input array.
org.apache.mxnet.Symbol
Draw random samples from an exponential distribution.
Samples are distributed according to an exponential distribution parametrized by *lambda* (rate).
Example::
exponential(lam=4, shape=(2,2)) = 0.0097189 , 0.08999364],
[ 0.04146638, 0.31715935
Defined in src/operator/random/sample_op.cc:L115
Draw random samples from an exponential distribution.
Samples are distributed according to an exponential distribution parametrized by *lambda* (rate).
Example::
exponential(lam=4, shape=(2,2)) = 0.0097189 , 0.08999364],
[ 0.04146638, 0.31715935
Defined in src/operator/random/sample_op.cc:L115
Lambda parameter (rate) of the exponential distribution.
Shape of the output.
Context of output, in format [cpugpucpu_pinned](n). Only used for imperative calls.
DType of the output in case this can't be inferred. Defaults to float32 if not defined (dtype=None).
org.apache.mxnet.Symbol
Draw random samples from a gamma distribution.
Samples are distributed according to a gamma distribution parametrized by *alpha* (shape) and *beta* (scale).
Example::
gamma(alpha=9, beta=0.5, shape=(2,2)) = 7.10486984, 3.37695289],
[ 3.91697288, 3.65933681
Defined in src/operator/random/sample_op.cc:L100
Draw random samples from a gamma distribution.
Samples are distributed according to a gamma distribution parametrized by *alpha* (shape) and *beta* (scale).
Example::
gamma(alpha=9, beta=0.5, shape=(2,2)) = 7.10486984, 3.37695289],
[ 3.91697288, 3.65933681
Defined in src/operator/random/sample_op.cc:L100
Alpha parameter (shape) of the gamma distribution.
Beta parameter (scale) of the gamma distribution.
Shape of the output.
Context of output, in format [cpugpucpu_pinned](n). Only used for imperative calls.
DType of the output in case this can't be inferred. Defaults to float32 if not defined (dtype=None).
org.apache.mxnet.Symbol
Draw random samples from a generalized negative binomial distribution.
Samples are distributed according to a generalized negative binomial distribution parametrized by
*mu* (mean) and *alpha* (dispersion).
Draw random samples from a generalized negative binomial distribution.
Samples are distributed according to a generalized negative binomial distribution parametrized by
*mu* (mean) and *alpha* (dispersion). *alpha* is defined as *1/k* where *k* is the failure limit of the
number of unsuccessful experiments (generalized to real numbers).
Samples will always be returned as a floating point data type.
Example::
generalized_negative_binomial(mu=2.0, alpha=0.3, shape=(2,2)) = 2., 1.],
[ 6., 4.
Defined in src/operator/random/sample_op.cc:L168
Mean of the negative binomial distribution.
Alpha (dispersion) parameter of the negative binomial distribution.
Shape of the output.
Context of output, in format [cpugpucpu_pinned](n). Only used for imperative calls.
DType of the output in case this can't be inferred. Defaults to float32 if not defined (dtype=None).
org.apache.mxnet.Symbol
Draw random samples from a negative binomial distribution.
Samples are distributed according to a negative binomial distribution parametrized by
*k* (limit of unsuccessful experiments) and *p* (failure probability in each experiment).
Samples will always be returned as a floating point data type.
Example::
negative_binomial(k=3, p=0.4, shape=(2,2)) = 4., 7.],
[ 2., 5.
Defined in src/operator/random/sample_op.cc:L149
Draw random samples from a negative binomial distribution.
Samples are distributed according to a negative binomial distribution parametrized by
*k* (limit of unsuccessful experiments) and *p* (failure probability in each experiment).
Samples will always be returned as a floating point data type.
Example::
negative_binomial(k=3, p=0.4, shape=(2,2)) = 4., 7.],
[ 2., 5.
Defined in src/operator/random/sample_op.cc:L149
Limit of unsuccessful experiments.
Failure probability in each experiment.
Shape of the output.
Context of output, in format [cpugpucpu_pinned](n). Only used for imperative calls.
DType of the output in case this can't be inferred. Defaults to float32 if not defined (dtype=None).
org.apache.mxnet.Symbol
Draw random samples from a normal (Gaussian) distribution.
..
Draw random samples from a normal (Gaussian) distribution.
.. note:: The existing alias
is deprecated.normal
Samples are distributed according to a normal distribution parametrized by *loc* (mean) and *scale* (standard deviation).
Example::
normal(loc=0, scale=1, shape=(2,2)) = 1.89171135, 1.16881478],
[1.23474145, 1.55807114
Defined in src/operator/random/sample_op.cc:L85
Mean of the distribution.
Standard deviation of the distribution.
Shape of the output.
Context of output, in format [cpugpucpu_pinned](n). Only used for imperative calls.
DType of the output in case this can't be inferred. Defaults to float32 if not defined (dtype=None).
org.apache.mxnet.Symbol
Draw random samples from a Poisson distribution.
Samples are distributed according to a Poisson distribution parametrized by *lambda* (rate).
Samples will always be returned as a floating point data type.
Example::
poisson(lam=4, shape=(2,2)) = 5., 2.],
[ 4., 6.
Defined in src/operator/random/sample_op.cc:L132
Draw random samples from a Poisson distribution.
Samples are distributed according to a Poisson distribution parametrized by *lambda* (rate).
Samples will always be returned as a floating point data type.
Example::
poisson(lam=4, shape=(2,2)) = 5., 2.],
[ 4., 6.
Defined in src/operator/random/sample_op.cc:L132
Lambda parameter (rate) of the Poisson distribution.
Shape of the output.
Context of output, in format [cpugpucpu_pinned](n). Only used for imperative calls.
DType of the output in case this can't be inferred. Defaults to float32 if not defined (dtype=None).
org.apache.mxnet.Symbol
Draw random samples from a uniform distribution.
..
Draw random samples from a uniform distribution.
.. note:: The existing alias
is deprecated.uniform
Samples are uniformly distributed over the halfopen interval *[low, high)*
(includes *low*, but excludes *high*).
Example::
uniform(low=0, high=1, shape=(2,2)) = 0.60276335, 0.85794562],
[ 0.54488319, 0.84725171
Defined in src/operator/random/sample_op.cc:L66
Lower bound of the distribution.
Upper bound of the distribution.
Shape of the output.
Context of output, in format [cpugpucpu_pinned](n). Only used for imperative calls.
DType of the output in case this can't be inferred. Defaults to float32 if not defined (dtype=None).
org.apache.mxnet.Symbol
Converts a batch of index arrays into an array of flat indices.
Converts a batch of index arrays into an array of flat indices. The operator follows numpy conventions so a single multi index is given by a column of the input matrix.
Examples::
A = 3,6,6],[4,5,1
ravel(A, shape=(7,6)) = [22,41,37]
Defined in src/operator/tensor/ravel.cc:L41
Batch of multiindices
Shape of the array into which the multiindices apply.
org.apache.mxnet.Symbol
Returns elementwise inverse cuberoot value of the input.
..
Returns elementwise inverse cuberoot value of the input.
.. math::
rcbrt(x) = 1/\sqrt[3]{x}
Example::
rcbrt([1,8,125]) = [1.0, 0.5, 0.2]
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L816
The input array.
org.apache.mxnet.Symbol
Returns the reciprocal of the argument, elementwise.
Calculates 1/x.
Example::
reciprocal([2, 1, 3, 1.6, 0.2]) = [0.5, 1.0, 0.33333334, 0.625, 5.0]
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L556
Returns the reciprocal of the argument, elementwise.
Calculates 1/x.
Example::
reciprocal([2, 1, 3, 1.6, 0.2]) = [0.5, 1.0, 0.33333334, 0.625, 5.0]
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L556
The input array.
org.apache.mxnet.Symbol
Computes rectified linear.
..
Computes rectified linear.
.. math::
max(features, 0)
The storage type of
output depends upon the input storage type:relu
The input array.
org.apache.mxnet.Symbol
Repeats elements of an array.
By default,
flattens the input array into 1D and then repeats therepeat
elements::
x = 1, 2],
[ 3, 4
repeat(x, repeats=2) = [ 1., 1., 2., 2., 3., 3., 4., 4.]
The parameter
specifies the axis along which to perform repeat::axis
repeat(x, repeats=2, axis=1) = 1., 1., 2., 2.],
[ 3., 3., 4., 4.
repeat(x, repeats=2, axis=0) = 1., 2.],
[ 1., 2.],
[ 3., 4.],
[ 3., 4.
repeat(x, repeats=2, axis=1) = 1., 1., 2., 2.],
[ 3., 3., 4., 4.
Defined in src/operator/tensor/matrix_op.cc:L690
Repeats elements of an array.
By default,
flattens the input array into 1D and then repeats therepeat
elements::
x = 1, 2],
[ 3, 4
repeat(x, repeats=2) = [ 1., 1., 2., 2., 3., 3., 4., 4.]
The parameter
specifies the axis along which to perform repeat::axis
repeat(x, repeats=2, axis=1) = 1., 1., 2., 2.],
[ 3., 3., 4., 4.
repeat(x, repeats=2, axis=0) = 1., 2.],
[ 1., 2.],
[ 3., 4.],
[ 3., 4.
repeat(x, repeats=2, axis=1) = 1., 1., 2., 2.],
[ 3., 3., 4., 4.
Defined in src/operator/tensor/matrix_op.cc:L690
Input data array
The number of repetitions for each element.
The axis along which to repeat values. The negative numbers are interpreted counting from the backward. By default, use the flattened input array, and return a flat output array.
org.apache.mxnet.Symbol
Reshapes the input array.
..
Reshapes the input array.
.. note::
is deprecated, use Reshape
reshape
Given an array and a shape, this function returns a copy of the array in the new shape.
The shape is a tuple of integers such as (2,3,4). The size of the new shape should be same as the size of the input array.
Example::
reshape([1,2,3,4], shape=(2,2)) = [3,4
Some dimensions of the shape can take special values from the set {0, 1, 2, 3, 4}. The significance of each is explained below:

copy this dimension from the input to the output shape.0
Example::
1
infers the dimension of the output shape by using the remainder of the input dimensions2
copy all/remainder of the input dimensions to the output shape.3
use the product of two consecutive dimensions of the input shape as the output dimension.4
split one dimension of the input into two dimensions passed subsequent to 4 in shape (can contain 1).reverse
is set to 1, then the special values are inferred from right to left.Input data to reshape.
The target shape
If true then the special values are inferred from right to left
(Deprecated! Use
instead.) Target new shape. One and only one dim can be 0, in which case it will be inferred from the rest of dimsshape
(Deprecated! Use
instead.) Whether keep the highest dim unchanged.If set to true, then the first dim in target_shape is ignored,and always fixed as inputshape
org.apache.mxnet.Symbol
Reshape lhs to have the same shape as rhs.
Reshape lhs to have the same shape as rhs.
First input.
Second input.
org.apache.mxnet.Symbol
Reverses the order of elements along given axis while preserving array shape.
Note: reverse and flip are equivalent.
Reverses the order of elements along given axis while preserving array shape.
Note: reverse and flip are equivalent. We use reverse in the following examples.
Examples::
x = 0., 1., 2., 3., 4.],
[ 5., 6., 7., 8., 9.
reverse(x, axis=0) = 5., 6., 7., 8., 9.],
[ 0., 1., 2., 3., 4.
reverse(x, axis=1) = 4., 3., 2., 1., 0.],
[ 9., 8., 7., 6., 5.
Defined in src/operator/tensor/matrix_op.cc:L792
Input data array
The axis which to reverse elements.
org.apache.mxnet.Symbol
Returns elementwise rounded value to the nearest integer of the input.
..
Returns elementwise rounded value to the nearest integer of the input.
.. note::
n.5
rint
returns n
while round
returns n+1
.n.5
both rint
and round
returns n1
.rint
output depends upon the input storage type:The input array.
org.apache.mxnet.Symbol
Update function for RMSProp
optimizer.
RMSprop
is a variant of stochastic gradient descent where the gradients are
divided by a cache which grows with the sum of squares of recent gradients?
RMSProp
is similar to AdaGrad
, a popular variant of SGD
which adaptively
tunes the learning rate of each parameter.
Update function for RMSProp
optimizer.
RMSprop
is a variant of stochastic gradient descent where the gradients are
divided by a cache which grows with the sum of squares of recent gradients?
RMSProp
is similar to AdaGrad
, a popular variant of SGD
which adaptively
tunes the learning rate of each parameter. AdaGrad
lowers the learning rate for
each parameter monotonically over the course of training.
While this is analytically motivated for convex optimizations, it may not be ideal
for nonconvex problems. RMSProp
deals with this heuristically by allowing the
learning rates to rebound as the denominator decays over time.
Define the Root Mean Square (RMS) error criterion of the gradient as
:math:RMS[g]_t = \sqrt{E[g^{2]_t + \epsilon}, where :math:g represents
gradient and :math:E[g}2]_t
is the decaying average over past squared gradient.
The :math:E[g^2]_t
is given by:
.. math::
E[g^{2]_t = \gamma * E[g}2]_{t1} + (1\gamma) * g_t^2
The update step is
.. math::
\theta_{t+1} = \theta_t  \frac{\eta}{RMS[g]_t} g_t
The RMSProp code follows the version in
http://www.cs.toronto.edu/~tijmen/csc321/slides/lecture_slides_lec6.pdf
Tieleman & Hinton, 2012.
Hinton suggests the momentum term :math:\gamma
to be 0.9 and the learning rate
:math:\eta
to be 0.001.
Defined in src/operator/optimizer_op.cc:L553
Weight
Gradient
n
Learning rate
The decay rate of momentum estimates.
A small constant for numerical stability.
Weight decay augments the objective function with a regularization term that penalizes large weights. The penalty scales with the square of the magnitude of each weight.
Rescale gradient to grad = rescale_grad*grad.
Clip gradient to the range of [clip_gradient, clip_gradient] If clip_gradient <= 0, gradient clipping is turned off. grad = max(min(grad, clip_gradient), clip_gradient).
Clip weights to the range of [clip_weights, clip_weights] If clip_weights <= 0, weight clipping is turned off. weights = max(min(weights, clip_weights), clip_weights).
org.apache.mxnet.Symbol
Update function for RMSPropAlex optimizer.
RMSPropAlex
is noncentered version of RMSProp
.
Define :math:E[g^2]_t
is the decaying average over past squared gradient and
E[g]_t
:math: is the decaying average over past gradient.
.. math::
E[g^{2]_t = \gamma_1 * E[g}2]_{t1} + (1  \gamma_1) * g_t^{2\\
E[g]_t = \gamma_1 * E[g]_{t1} + (1  \gamma_1) * g_t\\
\Delta_t = \gamma_2 * \Delta_{t1}  \frac{\eta}{\sqrt{E[g}2]_t  E[g]_t^2 + \epsilon}} g_t\\
The update step is
.. math::
\theta_{t+1} = \theta_t + \Delta_t
The RMSPropAlex code follows the version in
http://arxiv.org/pdf/1308.0850v5.pdf Eq(38)  Eq(45) by Alex Graves, 2013.
Graves suggests the momentum term :math:\gamma_1
to be 0.95, :math:\gamma_2
to be 0.9 and the learning rate :math:\eta
to be 0.0001.
Defined in src/operator/optimizer_op.cc:L592
Update function for RMSPropAlex optimizer.
RMSPropAlex
is noncentered version of RMSProp
.
Define :math:E[g^2]_t
is the decaying average over past squared gradient and
E[g]_t
:math: is the decaying average over past gradient.
.. math::
E[g^{2]_t = \gamma_1 * E[g}2]_{t1} + (1  \gamma_1) * g_t^{2\\
E[g]_t = \gamma_1 * E[g]_{t1} + (1  \gamma_1) * g_t\\
\Delta_t = \gamma_2 * \Delta_{t1}  \frac{\eta}{\sqrt{E[g}2]_t  E[g]_t^2 + \epsilon}} g_t\\
The update step is
.. math::
\theta_{t+1} = \theta_t + \Delta_t
The RMSPropAlex code follows the version in
http://arxiv.org/pdf/1308.0850v5.pdf Eq(38)  Eq(45) by Alex Graves, 2013.
Graves suggests the momentum term :math:\gamma_1
to be 0.95, :math:\gamma_2
to be 0.9 and the learning rate :math:\eta
to be 0.0001.
Defined in src/operator/optimizer_op.cc:L592
Weight
Gradient
n
g
delta
Learning rate
Decay rate.
Decay rate.
A small constant for numerical stability.
Weight decay augments the objective function with a regularization term that penalizes large weights. The penalty scales with the square of the magnitude of each weight.
Rescale gradient to grad = rescale_grad*grad.
Clip gradient to the range of [clip_gradient, clip_gradient] If clip_gradient <= 0, gradient clipping is turned off. grad = max(min(grad, clip_gradient), clip_gradient).
Clip weights to the range of [clip_weights, clip_weights] If clip_weights <= 0, weight clipping is turned off. weights = max(min(weights, clip_weights), clip_weights).
org.apache.mxnet.Symbol
Returns elementwise rounded value to the nearest integer of the input.
Example::
round([1.5, 1.5, 1.9, 1.9, 2.1]) = [2., 2., 2., 2., 2.]
The storage type of
output depends upon the input storage type:round
Returns elementwise rounded value to the nearest integer of the input.
Example::
round([1.5, 1.5, 1.9, 1.9, 2.1]) = [2., 2., 2., 2., 2.]
The storage type of
output depends upon the input storage type:round
The input array.
org.apache.mxnet.Symbol
Returns elementwise inverse squareroot value of the input.
..
Returns elementwise inverse squareroot value of the input.
.. math::
rsqrt(x) = 1/\sqrt{x}
Example::
rsqrt([4,9,16]) = [0.5, 0.33333334, 0.25]
The storage type of
output is always densersqrt
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L776
The input array.
org.apache.mxnet.Symbol
Concurrent sampling from multiple
exponential distributions with parameters lambda (rate).
The parameters of the distributions are provided as an input array.
Let *[s]* be the shape of the input array, *n* be the dimension of *[s]*, *[t]*
be the shape specified as the parameter of the operator, and *m* be the dimension
of *[t]*.
Concurrent sampling from multiple
exponential distributions with parameters lambda (rate).
The parameters of the distributions are provided as an input array.
Let *[s]* be the shape of the input array, *n* be the dimension of *[s]*, *[t]*
be the shape specified as the parameter of the operator, and *m* be the dimension
of *[t]*. Then the output will be a *(n+m)*dimensional array with shape *[s]x[t]*.
For any valid *n*dimensional index *i* with respect to the input array, *output[i]*
will be an *m*dimensional array that holds randomly drawn samples from the distribution
which is parameterized by the input value at index *i*. If the shape parameter of the
operator is not set, then one sample will be drawn per distribution and the output array
has the same shape as the input array.
Examples::
lam = [ 1.0, 8.5 ]
// Draw a single sample for each distribution
sample_exponential(lam) = [ 0.51837951, 0.09994757]
// Draw a vector containing two samples for each distribution
sample_exponential(lam, shape=(2)) = 0.51837951, 0.19866663],
[ 0.09994757, 0.50447971
Defined in src/operator/random/multisample_op.cc:L284
Lambda (rate) parameters of the distributions.
Shape to be sampled from each random distribution.
DType of the output in case this can't be inferred. Defaults to float32 if not defined (dtype=None).
org.apache.mxnet.Symbol
Concurrent sampling from multiple
gamma distributions with parameters *alpha* (shape) and *beta* (scale).
The parameters of the distributions are provided as input arrays.
Let *[s]* be the shape of the input arrays, *n* be the dimension of *[s]*, *[t]*
be the shape specified as the parameter of the operator, and *m* be the dimension
of *[t]*.
Concurrent sampling from multiple
gamma distributions with parameters *alpha* (shape) and *beta* (scale).
The parameters of the distributions are provided as input arrays.
Let *[s]* be the shape of the input arrays, *n* be the dimension of *[s]*, *[t]*
be the shape specified as the parameter of the operator, and *m* be the dimension
of *[t]*. Then the output will be a *(n+m)*dimensional array with shape *[s]x[t]*.
For any valid *n*dimensional index *i* with respect to the input arrays, *output[i]*
will be an *m*dimensional array that holds randomly drawn samples from the distribution
which is parameterized by the input values at index *i*. If the shape parameter of the
operator is not set, then one sample will be drawn per distribution and the output array
has the same shape as the input arrays.
Examples::
alpha = [ 0.0, 2.5 ]
beta = [ 1.0, 0.7 ]
// Draw a single sample for each distribution
sample_gamma(alpha, beta) = [ 0. , 2.25797319]
// Draw a vector containing two samples for each distribution
sample_gamma(alpha, beta, shape=(2)) = 0. , 0. ],
[ 2.25797319, 1.70734084
Defined in src/operator/random/multisample_op.cc:L282
Alpha (shape) parameters of the distributions.
Shape to be sampled from each random distribution.
DType of the output in case this can't be inferred. Defaults to float32 if not defined (dtype=None).
Beta (scale) parameters of the distributions.
org.apache.mxnet.Symbol
Concurrent sampling from multiple
generalized negative binomial distributions with parameters *mu* (mean) and *alpha* (dispersion).
The parameters of the distributions are provided as input arrays.
Let *[s]* be the shape of the input arrays, *n* be the dimension of *[s]*, *[t]*
be the shape specified as the parameter of the operator, and *m* be the dimension
of *[t]*.
Concurrent sampling from multiple
generalized negative binomial distributions with parameters *mu* (mean) and *alpha* (dispersion).
The parameters of the distributions are provided as input arrays.
Let *[s]* be the shape of the input arrays, *n* be the dimension of *[s]*, *[t]*
be the shape specified as the parameter of the operator, and *m* be the dimension
of *[t]*. Then the output will be a *(n+m)*dimensional array with shape *[s]x[t]*.
For any valid *n*dimensional index *i* with respect to the input arrays, *output[i]*
will be an *m*dimensional array that holds randomly drawn samples from the distribution
which is parameterized by the input values at index *i*. If the shape parameter of the
operator is not set, then one sample will be drawn per distribution and the output array
has the same shape as the input arrays.
Samples will always be returned as a floating point data type.
Examples::
mu = [ 2.0, 2.5 ]
alpha = [ 1.0, 0.1 ]
// Draw a single sample for each distribution
sample_generalized_negative_binomial(mu, alpha) = [ 0., 3.]
// Draw a vector containing two samples for each distribution
sample_generalized_negative_binomial(mu, alpha, shape=(2)) = 0., 3.],
[ 3., 1.
Defined in src/operator/random/multisample_op.cc:L293
Means of the distributions.
Shape to be sampled from each random distribution.
DType of the output in case this can't be inferred. Defaults to float32 if not defined (dtype=None).
Alpha (dispersion) parameters of the distributions.
org.apache.mxnet.Symbol
Concurrent sampling from multiple multinomial distributions.
*data* is an *n* dimensional array whose last dimension has length *k*, where
*k* is the number of possible outcomes of each multinomial distribution.
Concurrent sampling from multiple multinomial distributions.
*data* is an *n* dimensional array whose last dimension has length *k*, where
*k* is the number of possible outcomes of each multinomial distribution. This
operator will draw *shape* samples from each distribution. If shape is empty
one sample will be drawn from each distribution.
If *get_prob* is true, a second array containing log likelihood of the drawn
samples will also be returned. This is usually used for reinforcement learning
where you can provide reward as head gradient for this array to estimate
gradient.
Note that the input distribution must be normalized, i.e. *data* must sum to
1 along its last axis.
Examples::
probs = 0.1, 0.2, 0.3, 0.4], [0.4, 0.3, 0.2, 0.1, 0
// Draw a single sample for each distribution
sample_multinomial(probs) = [3, 0]
// Draw a vector containing two samples for each distribution
sample_multinomial(probs, shape=(2)) = 2],
[0, 0
// requests log likelihood
sample_multinomial(probs, get_prob=True) = [2, 1], [0.2, 0.3]
Distribution probabilities. Must sum to one on the last axis.
Shape to be sampled from each random distribution.
Whether to also return the log probability of sampled result. This is usually used for differentiating through stochastic variables, e.g. in reinforcement learning.
DType of the output in case this can't be inferred.
org.apache.mxnet.Symbol
Concurrent sampling from multiple
negative binomial distributions with parameters *k* (failure limit) and *p* (failure probability).
The parameters of the distributions are provided as input arrays.
Let *[s]* be the shape of the input arrays, *n* be the dimension of *[s]*, *[t]*
be the shape specified as the parameter of the operator, and *m* be the dimension
of *[t]*.
Concurrent sampling from multiple
negative binomial distributions with parameters *k* (failure limit) and *p* (failure probability).
The parameters of the distributions are provided as input arrays.
Let *[s]* be the shape of the input arrays, *n* be the dimension of *[s]*, *[t]*
be the shape specified as the parameter of the operator, and *m* be the dimension
of *[t]*. Then the output will be a *(n+m)*dimensional array with shape *[s]x[t]*.
For any valid *n*dimensional index *i* with respect to the input arrays, *output[i]*
will be an *m*dimensional array that holds randomly drawn samples from the distribution
which is parameterized by the input values at index *i*. If the shape parameter of the
operator is not set, then one sample will be drawn per distribution and the output array
has the same shape as the input arrays.
Samples will always be returned as a floating point data type.
Examples::
k = [ 20, 49 ]
p = [ 0.4 , 0.77 ]
// Draw a single sample for each distribution
sample_negative_binomial(k, p) = [ 15., 16.]
// Draw a vector containing two samples for each distribution
sample_negative_binomial(k, p, shape=(2)) = 15., 50.],
[ 16., 12.
Defined in src/operator/random/multisample_op.cc:L289
Limits of unsuccessful experiments.
Shape to be sampled from each random distribution.
DType of the output in case this can't be inferred. Defaults to float32 if not defined (dtype=None).
Failure probabilities in each experiment.
org.apache.mxnet.Symbol
Concurrent sampling from multiple
normal distributions with parameters *mu* (mean) and *sigma* (standard deviation).
The parameters of the distributions are provided as input arrays.
Let *[s]* be the shape of the input arrays, *n* be the dimension of *[s]*, *[t]*
be the shape specified as the parameter of the operator, and *m* be the dimension
of *[t]*.
Concurrent sampling from multiple
normal distributions with parameters *mu* (mean) and *sigma* (standard deviation).
The parameters of the distributions are provided as input arrays.
Let *[s]* be the shape of the input arrays, *n* be the dimension of *[s]*, *[t]*
be the shape specified as the parameter of the operator, and *m* be the dimension
of *[t]*. Then the output will be a *(n+m)*dimensional array with shape *[s]x[t]*.
For any valid *n*dimensional index *i* with respect to the input arrays, *output[i]*
will be an *m*dimensional array that holds randomly drawn samples from the distribution
which is parameterized by the input values at index *i*. If the shape parameter of the
operator is not set, then one sample will be drawn per distribution and the output array
has the same shape as the input arrays.
Examples::
mu = [ 0.0, 2.5 ]
sigma = [ 1.0, 3.7 ]
// Draw a single sample for each distribution
sample_normal(mu, sigma) = [0.56410581, 0.95934606]
// Draw a vector containing two samples for each distribution
sample_normal(mu, sigma, shape=(2)) = 0.2928229 ],
[ 0.95934606, 4.48287058
Defined in src/operator/random/multisample_op.cc:L279
Means of the distributions.
Shape to be sampled from each random distribution.
DType of the output in case this can't be inferred. Defaults to float32 if not defined (dtype=None).
Standard deviations of the distributions.
org.apache.mxnet.Symbol
Concurrent sampling from multiple
Poisson distributions with parameters lambda (rate).
The parameters of the distributions are provided as an input array.
Let *[s]* be the shape of the input array, *n* be the dimension of *[s]*, *[t]*
be the shape specified as the parameter of the operator, and *m* be the dimension
of *[t]*.
Concurrent sampling from multiple
Poisson distributions with parameters lambda (rate).
The parameters of the distributions are provided as an input array.
Let *[s]* be the shape of the input array, *n* be the dimension of *[s]*, *[t]*
be the shape specified as the parameter of the operator, and *m* be the dimension
of *[t]*. Then the output will be a *(n+m)*dimensional array with shape *[s]x[t]*.
For any valid *n*dimensional index *i* with respect to the input array, *output[i]*
will be an *m*dimensional array that holds randomly drawn samples from the distribution
which is parameterized by the input value at index *i*. If the shape parameter of the
operator is not set, then one sample will be drawn per distribution and the output array
has the same shape as the input array.
Samples will always be returned as a floating point data type.
Examples::
lam = [ 1.0, 8.5 ]
// Draw a single sample for each distribution
sample_poisson(lam) = [ 0., 13.]
// Draw a vector containing two samples for each distribution
sample_poisson(lam, shape=(2)) = 0., 4.],
[ 13., 8.
Defined in src/operator/random/multisample_op.cc:L286
Lambda (rate) parameters of the distributions.
Shape to be sampled from each random distribution.
DType of the output in case this can't be inferred. Defaults to float32 if not defined (dtype=None).
org.apache.mxnet.Symbol
Concurrent sampling from multiple
uniform distributions on the intervals given by *[low,high)*.
The parameters of the distributions are provided as input arrays.
Let *[s]* be the shape of the input arrays, *n* be the dimension of *[s]*, *[t]*
be the shape specified as the parameter of the operator, and *m* be the dimension
of *[t]*.
Concurrent sampling from multiple
uniform distributions on the intervals given by *[low,high)*.
The parameters of the distributions are provided as input arrays.
Let *[s]* be the shape of the input arrays, *n* be the dimension of *[s]*, *[t]*
be the shape specified as the parameter of the operator, and *m* be the dimension
of *[t]*. Then the output will be a *(n+m)*dimensional array with shape *[s]x[t]*.
For any valid *n*dimensional index *i* with respect to the input arrays, *output[i]*
will be an *m*dimensional array that holds randomly drawn samples from the distribution
which is parameterized by the input values at index *i*. If the shape parameter of the
operator is not set, then one sample will be drawn per distribution and the output array
has the same shape as the input arrays.
Examples::
low = [ 0.0, 2.5 ]
high = [ 1.0, 3.7 ]
// Draw a single sample for each distribution
sample_uniform(low, high) = [ 0.40451524, 3.18687344]
// Draw a vector containing two samples for each distribution
sample_uniform(low, high, shape=(2)) = 0.40451524, 0.18017688],
[ 3.18687344, 3.68352246
Defined in src/operator/random/multisample_op.cc:L277
Lower bounds of the distributions.
Shape to be sampled from each random distribution.
DType of the output in case this can't be inferred. Defaults to float32 if not defined (dtype=None).
Upper bounds of the distributions.
org.apache.mxnet.Symbol
Scatters data into a new tensor according to indices.
Given data
with shape (Y_0, ..., Y_{K1}, X_M, ..., X_{N1})
and indices with shape
(M, Y_0, ..., Y_{K1})
, the output will have shape (X_0, X_1, ..., X_{N1})
,
where M <= N
.
Scatters data into a new tensor according to indices.
Given data
with shape (Y_0, ..., Y_{K1}, X_M, ..., X_{N1})
and indices with shape
(M, Y_0, ..., Y_{K1})
, the output will have shape (X_0, X_1, ..., X_{N1})
,
where M <= N
. If M == N
, data shape should simply be (Y_0, ..., Y_{K1})
.
The elements in output is defined as follows::
output[indices[0, y_0, ..., y_{K1}],
...,
indices[M1, y_0, ..., y_{K1}],
x_M, ..., x_{N1}] = data[y_0, ..., y_{K1}, x_M, ..., x_{N1}]
all other entries in output are 0.
.. warning::
If the indices have duplicates, the result will be nondeterministic and
the gradient of scatter_nd
will not be correct!!
Examples::
data = [2, 3, 0]
indices = 1, 0], [0, 1, 0
shape = (2, 2)
scatter_nd(data, indices, shape) = 0], [2, 3
data
indices
Shape of output.
org.apache.mxnet.Symbol
Momentum update function for Stochastic Gradient Descent (SGD) optimizer.
Momentum update has better convergence rates on neural networks.
Momentum update function for Stochastic Gradient Descent (SGD) optimizer.
Momentum update has better convergence rates on neural networks. Mathematically it looks
like below:
.. math::
v_1 = \alpha * \nabla J(W_0)\\
v_t = \gamma v_{t1}  \alpha * \nabla J(W_{t1})\\
W_t = W_{t1} + v_t
It updates the weights using::
v = momentum * v  learning_rate * gradient
weight += v
Where the parameter
is the decay rate of momentum estimates at each epoch.momentum
However, if grad's storage type is
, row_sparse
is True and weight's storagelazy_update
type is the same as momentum's storage type,
only the row slices whose indices appear in grad.indices are updated (for both weight and momentum)::
for row in gradient.indices:
v[row] = momentum[row] * v[row]  learning_rate * gradient[row]
weight[row] += v[row]
Defined in src/operator/optimizer_op.cc:L372
Weight
Gradient
Momentum
Learning rate
The decay rate of momentum estimates at each epoch.
Weight decay augments the objective function with a regularization term that penalizes large weights. The penalty scales with the square of the magnitude of each weight.
Rescale gradient to grad = rescale_grad*grad.
Clip gradient to the range of [clip_gradient, clip_gradient] If clip_gradient <= 0, gradient clipping is turned off. grad = max(min(grad, clip_gradient), clip_gradient).
If true, lazy updates are applied if gradient's stype is row_sparse and both weight and momentum have the same stype
org.apache.mxnet.Symbol
Update function for Stochastic Gradient Descent (SDG) optimizer.
It updates the weights using::
weight = weight  learning_rate * (gradient + wd * weight)
However, if gradient is of
storage type and row_sparse
is True,lazy_update
only the row slices whose indices appear in grad.indices are updated::
for row in gradient.indices:
weight[row] = weight[row]  learning_rate * (gradient[row] + wd * weight[row])
Defined in src/operator/optimizer_op.cc:L331
Update function for Stochastic Gradient Descent (SDG) optimizer.
It updates the weights using::
weight = weight  learning_rate * (gradient + wd * weight)
However, if gradient is of
storage type and row_sparse
is True,lazy_update
only the row slices whose indices appear in grad.indices are updated::
for row in gradient.indices:
weight[row] = weight[row]  learning_rate * (gradient[row] + wd * weight[row])
Defined in src/operator/optimizer_op.cc:L331
Weight
Gradient
Learning rate
Weight decay augments the objective function with a regularization term that penalizes large weights. The penalty scales with the square of the magnitude of each weight.
Rescale gradient to grad = rescale_grad*grad.
Clip gradient to the range of [clip_gradient, clip_gradient] If clip_gradient <= 0, gradient clipping is turned off. grad = max(min(grad, clip_gradient), clip_gradient).
If true, lazy updates are applied if gradient's stype is row_sparse.
org.apache.mxnet.Symbol
Returns a 1D int64 array containing the shape of data.
Example::
shape_array([5,6,7,8) = [2,4]
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L416
Returns a 1D int64 array containing the shape of data.
Example::
shape_array([5,6,7,8) = [2,4]
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L416
Input Array.
org.apache.mxnet.Symbol
Randomly shuffle the elements.
This shuffles the array along the first axis.
The order of the elements in each subarray does not change.
For example, if a 2D array is given, the order of the rows randomly changes,
but the order of the elements in each row does not change.
Randomly shuffle the elements.
This shuffles the array along the first axis.
The order of the elements in each subarray does not change.
For example, if a 2D array is given, the order of the rows randomly changes,
but the order of the elements in each row does not change.
Data to be shuffled.
org.apache.mxnet.Symbol
Computes sigmoid of x elementwise.
..
Computes sigmoid of x elementwise.
.. math::
y = 1 / (1 + exp(x))
The storage type of
output is always densesigmoid
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L101
The input array.
org.apache.mxnet.Symbol
Returns elementwise sign of the input.
Example::
sign([2, 0, 3]) = [1, 0, 1]
The storage type of
output depends upon the input storage type:sign
Returns elementwise sign of the input.
Example::
sign([2, 0, 3]) = [1, 0, 1]
The storage type of
output depends upon the input storage type:sign
The input array.
org.apache.mxnet.Symbol
Update function for SignSGD optimizer.
..
Update function for SignSGD optimizer.
.. math::
g_t = \nabla J(W_{t1})\\
W_t = W_{t1}  \eta_t \text{sign}(g_t)
It updates the weights using::
weight = weight  learning_rate * sign(gradient)
.. note::
Weight
Gradient
Learning rate
Weight decay augments the objective function with a regularization term that penalizes large weights. The penalty scales with the square of the magnitude of each weight.
Rescale gradient to grad = rescale_grad*grad.
Clip gradient to the range of [clip_gradient, clip_gradient] If clip_gradient <= 0, gradient clipping is turned off. grad = max(min(grad, clip_gradient), clip_gradient).
org.apache.mxnet.Symbol
SIGN momentUM (Signum) optimizer.
..
SIGN momentUM (Signum) optimizer.
.. math::
g_t = \nabla J(W_{t1})\\
m_t = \beta m_{t1} + (1  \beta) g_t\\
W_t = W_{t1}  \eta_t \text{sign}(m_t)
It updates the weights using::
state = momentum * state + (1momentum) * gradient
weight = weight  learning_rate * sign(state)
Where the parameter
is the decay rate of momentum estimates at each epoch.momentum
.. note::
Weight
Gradient
Momentum
Learning rate
The decay rate of momentum estimates at each epoch.
Weight decay augments the objective function with a regularization term that penalizes large weights. The penalty scales with the square of the magnitude of each weight.
Rescale gradient to grad = rescale_grad*grad.
Clip gradient to the range of [clip_gradient, clip_gradient] If clip_gradient <= 0, gradient clipping is turned off. grad = max(min(grad, clip_gradient), clip_gradient).
The amount of weight decay that does not go into gradient/momentum calculationsotherwise do weight decay algorithmically only.
org.apache.mxnet.Symbol
Computes the elementwise sine of the input array.
The input should be in radians (:math:2\pi
rad equals 360 degrees).
..
Computes the elementwise sine of the input array.
The input should be in radians (:math:2\pi
rad equals 360 degrees).
.. math::
sin([0, \pi/4, \pi/2]) = [0, 0.707, 1]
The storage type of
output depends upon the input storage type:sin
The input array.
org.apache.mxnet.Symbol
Returns the hyperbolic sine of the input array, computed elementwise.
..
Returns the hyperbolic sine of the input array, computed elementwise.
.. math::
sinh(x) = 0.5\times(exp(x)  exp(x))
The storage type of
output depends upon the input storage type:sinh
The input array.
org.apache.mxnet.Symbol
Returns a 1D int64 array containing the size of data.
Example::
size_array([5,6,7,8) = [8]
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L466
Returns a 1D int64 array containing the size of data.
Example::
size_array([5,6,7,8) = [8]
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L466
Input Array.
org.apache.mxnet.Symbol
Slices a region of the array.
..
Slices a region of the array.
.. note::
is deprecated. Use crop
instead.slice
This function returns a sliced array between the indices given
by begin
and end
with the corresponding step
.
For an input array of
,shape=(d_0, d_1, ..., d_n1)
slice operation with
,begin=(b_0, b_1...b_m1)
, and end=(e_0, e_1, ..., e_m1)
,step=(s_0, s_1, ..., s_m1)
where m <= n, results in an array with the shape
.(e_0b_0/s_0, ..., e_m1b_m1/s_m1, d_m, ..., d_n1)
The resulting array's *k*th dimension contains elements
from the *k*th dimension of the input array starting
from index
(inclusive) with step b_k
s_k
until reaching
(exclusive).e_k
If the *k*th elements are None
in the sequence of begin
, end
,
and step
, the following rule will be used to set default values.
If s_k
is None
, set s_k=1
. If s_k > 0
, set b_k=0
, e_k=d_k
;
else, set b_k=d_k1
, e_k=1
.
The storage type of
output depends on storage types of inputsslice
 slice(csr) = csr
 otherwise,
generates output with default storageslice
.. note:: When input data storage type is csr, it only supports
step=(), or step=(None,), or step=(1,) to generate a csr output.
For other step parameter values, it falls back to slicing
a dense tensor.
Example::
x = 1., 2., 3., 4.],
[ 5., 6., 7., 8.],
[ 9., 10., 11., 12.
slice(x, begin=(0,1), end=(2,4)) = 2., 3., 4.],
[ 6., 7., 8.
slice(x, begin=(None, 0), end=(None, 3), step=(1, 2)) = 11.],
[5., 7.],
[1., 3.
Defined in src/operator/tensor/matrix_op.cc:L412
Source input
starting indices for the slice operation, supports negative indices.
ending indices for the slice operation, supports negative indices.
step for the slice operation, supports negative values.
org.apache.mxnet.Symbol
Slices along a given axis.
Returns an array slice along a given axis
starting from the begin
index
to the end
index.
Examples::
x = 1., 2., 3., 4.],
[ 5., 6., 7., 8.],
[ 9., 10., 11., 12.
slice_axis(x, axis=0, begin=1, end=3) = 5., 6., 7., 8.],
[ 9., 10., 11., 12.
slice_axis(x, axis=1, begin=0, end=2) = 1., 2.],
[ 5., 6.],
[ 9., 10.
slice_axis(x, axis=1, begin=3, end=1) = 2., 3.],
[ 6., 7.],
[ 10., 11.
Defined in src/operator/tensor/matrix_op.cc:L499
Slices along a given axis.
Returns an array slice along a given axis
starting from the begin
index
to the end
index.
Examples::
x = 1., 2., 3., 4.],
[ 5., 6., 7., 8.],
[ 9., 10., 11., 12.
slice_axis(x, axis=0, begin=1, end=3) = 5., 6., 7., 8.],
[ 9., 10., 11., 12.
slice_axis(x, axis=1, begin=0, end=2) = 1., 2.],
[ 5., 6.],
[ 9., 10.
slice_axis(x, axis=1, begin=3, end=1) = 2., 3.],
[ 6., 7.],
[ 10., 11.
Defined in src/operator/tensor/matrix_op.cc:L499
Source input
Axis along which to be sliced, supports negative indexes.
The beginning index along the axis to be sliced, supports negative indexes.
The ending index along the axis to be sliced, supports negative indexes.
org.apache.mxnet.Symbol
Slices a region of the array like the shape of another array.
This function is similar to
, however, the slice
begin
are always 0
s
and end
of specific axes are inferred from the second input shape_like
.
Given the second shape_like
input of
,shape=(d_0, d_1, ..., d_n1)
a
operator with default empty slice_like
axes
, it performs the
following operation:
. out = slice(input, begin=(0, 0, ..., 0), end=(d_0, d_1, ..., d_n1))
When axes
is not empty, it is used to speficy which axes are being sliced.
Given a 4d input data,
operator with slice_like
axes=(0, 2, 1)
will perform the following operation:
. out = slice(input, begin=(0, 0, 0, 0), end=(d_0, None, d_2, d_3))
Note that it is allowed to have first and second input with different dimensions,
however, you have to make sure the axes
are specified and not exceeding the
dimension limits.
For example, given input_1
with
and shape=(2,3,4,5)
input_2
with
, it is not allowed to use:shape=(1,2,3)
because ndim of out = slice_like(a, b)
input_1
is 4, and ndim of input_2
is 3.
The following is allowed in this situation:
out = slice_like(a, b, axes=(0, 2))
Example::
x = 1., 2., 3., 4.],
[ 5., 6., 7., 8.],
[ 9., 10., 11., 12.
y = 0., 0., 0.],
[ 0., 0., 0.
slice_like(x, y) = 1., 2., 3.]
[ 5., 6., 7.
slice_like(x, y, axes=(0, 1)) = 1., 2., 3.]
[ 5., 6., 7.
slice_like(x, y, axes=(0)) = 1., 2., 3., 4.]
[ 5., 6., 7., 8.
slice_like(x, y, axes=(1)) = 1., 2., 3.]
[ 5., 6., 7.]
[ 9., 10., 11.
Defined in src/operator/tensor/matrix_op.cc:L568
Slices a region of the array like the shape of another array.
This function is similar to
, however, the slice
begin
are always 0
s
and end
of specific axes are inferred from the second input shape_like
.
Given the second shape_like
input of
,shape=(d_0, d_1, ..., d_n1)
a
operator with default empty slice_like
axes
, it performs the
following operation:
. out = slice(input, begin=(0, 0, ..., 0), end=(d_0, d_1, ..., d_n1))
When axes
is not empty, it is used to speficy which axes are being sliced.
Given a 4d input data,
operator with slice_like
axes=(0, 2, 1)
will perform the following operation:
. out = slice(input, begin=(0, 0, 0, 0), end=(d_0, None, d_2, d_3))
Note that it is allowed to have first and second input with different dimensions,
however, you have to make sure the axes
are specified and not exceeding the
dimension limits.
For example, given input_1
with
and shape=(2,3,4,5)
input_2
with
, it is not allowed to use:shape=(1,2,3)
because ndim of out = slice_like(a, b)
input_1
is 4, and ndim of input_2
is 3.
The following is allowed in this situation:
out = slice_like(a, b, axes=(0, 2))
Example::
x = 1., 2., 3., 4.],
[ 5., 6., 7., 8.],
[ 9., 10., 11., 12.
y = 0., 0., 0.],
[ 0., 0., 0.
slice_like(x, y) = 1., 2., 3.]
[ 5., 6., 7.
slice_like(x, y, axes=(0, 1)) = 1., 2., 3.]
[ 5., 6., 7.
slice_like(x, y, axes=(0)) = 1., 2., 3., 4.]
[ 5., 6., 7., 8.
slice_like(x, y, axes=(1)) = 1., 2., 3.]
[ 5., 6., 7.]
[ 9., 10., 11.
Defined in src/operator/tensor/matrix_op.cc:L568
Source input
Shape like input
List of axes on which input data will be sliced according to the corresponding size of the second input. By default will slice on all axes. Negative axes are supported.
org.apache.mxnet.Symbol
Calculate Smooth L1 Loss(lhs, scalar) by summing
..
Calculate Smooth L1 Loss(lhs, scalar) by summing
.. math::
f(x) =
\begin{cases}
(\sigma x)^{2/2,& \text{if }x < 1/\sigma}2\\
x0.5/\sigma^2,& \text{otherwise}
\end{cases}
where :math:x
is an element of the tensor *lhs* and :math:\sigma
is the scalar.
Example::
smooth_l1([1, 2, 3, 4], scalar=1) = [0.5, 1.5, 2.5, 3.5]
Defined in src/operator/tensor/elemwise_binary_scalar_op_extended.cc:L103
source input
scalar input
org.apache.mxnet.Symbol
Applies the softmax function.
The resulting array contains elements in the range (0,1) and the elements along the given axis sum up to 1.
..
Applies the softmax function.
The resulting array contains elements in the range (0,1) and the elements along the given axis sum up to 1.
.. math::
softmax(\mathbf{z/t})_j = \frac{e^{{z_j/t}}{\sum_{k=1}}K e^{z_k/t}}
for :math:j = 1, ..., K
t is the temperature parameter in softmax function. By default, t equals 1.0
Example::
x = 1. 1. 1.]
[ 1. 1. 1.
softmax(x,axis=0) = 0.5 0.5 0.5]
[ 0.5 0.5 0.5
softmax(x,axis=1) = 0.33333334, 0.33333334, 0.33333334],
[ 0.33333334, 0.33333334, 0.33333334
Defined in src/operator/nn/softmax.cc:L98
The input array.
The axis along which to compute softmax.
Temperature parameter in softmax
org.apache.mxnet.Symbol
Calculate cross entropy of softmax output and onehot label.
 This operator computes the cross entropy in two steps:
Calculate cross entropy of softmax output and onehot label.
 This operator computes the cross entropy in two steps:
Input data
Input label
org.apache.mxnet.Symbol
Computes softsign of x elementwise.
..
Computes softsign of x elementwise.
.. math::
y = x / (1 + abs(x))
The storage type of
output is always densesoftsign
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L145
The input array.
org.apache.mxnet.Symbol
Returns a sorted copy of an input array along the given axis.
Examples::
x = 1, 4],
[ 3, 1
// sorts along the last axis
sort(x) = 1., 4.],
[ 1., 3.
// flattens and then sorts
sort(x) = [ 1., 1., 3., 4.]
// sorts along the first axis
sort(x, axis=0) = 1., 1.],
[ 3., 4.
// in a descend order
sort(x, is_ascend=0) = 4., 1.],
[ 3., 1.
Defined in src/operator/tensor/ordering_op.cc:L126
Returns a sorted copy of an input array along the given axis.
Examples::
x = 1, 4],
[ 3, 1
// sorts along the last axis
sort(x) = 1., 4.],
[ 1., 3.
// flattens and then sorts
sort(x) = [ 1., 1., 3., 4.]
// sorts along the first axis
sort(x, axis=0) = 1., 1.],
[ 3., 4.
// in a descend order
sort(x, is_ascend=0) = 4., 1.],
[ 3., 1.
Defined in src/operator/tensor/ordering_op.cc:L126
The input array
Axis along which to choose sort the input tensor. If not given, the flattened array is used. Default is 1.
Whether to sort in ascending or descending order.
org.apache.mxnet.Symbol
Splits an array along a particular axis into multiple subarrays.
..
Splits an array along a particular axis into multiple subarrays.
.. note::
is deprecated. Use SliceChannel
instead.split
**Note** that num_outputs
should evenly divide the length of the axis
along which to split the array.
Example::
x = 1.]
[ 2.]]
3.]
[ 4.
5.]
[ 6.]
x.shape = (3, 2, 1)
y = split(x, axis=1, num_outputs=2) // a list of 2 arrays with shape (3, 1, 1)
y = 1.]]
3.
5.]
2.]]
4.
6.]
y[0].shape = (3, 1, 1)
z = split(x, axis=0, num_outputs=3) // a list of 3 arrays with shape (1, 2, 1)
z = 1.]
[ 2.
3.]
[ 4.
5.]
[ 6.
z[0].shape = (1, 2, 1)
squeeze_axis=1
removes the axis with length 1 from the shapes of the output arrays.
**Note** that setting squeeze_axis
to
removes axis with length 1 only1
along the axis
which it is split.
Also squeeze_axis
can be set to true only if
.input.shape[axis] == num_outputs
Example::
z = split(x, axis=0, num_outputs=3, squeeze_axis=1) // a list of 3 arrays with shape (2, 1)
z = 1.]
[ 2.
3.]
[ 4.
5.]
[ 6.
z[0].shape = (2 ,1 )
Defined in src/operator/slice_channel.cc:L107
The input
Number of splits. Note that this should evenly divide the length of the axis
.
Axis along which to split.
If true, Removes the axis with length 1 from the shapes of the output arrays. **Note** that setting squeeze_axis
to
removes axis with length 1 only along the true
axis
which it is split. Also squeeze_axis
can be set to
only if true
.input.shape[axis] == num_outputs
org.apache.mxnet.Symbol
Returns elementwise squareroot value of the input.
..
Returns elementwise squareroot value of the input.
.. math::
\textrm{sqrt}(x) = \sqrt{x}
Example::
sqrt([4, 9, 16]) = [2, 3, 4]
The storage type of
output depends upon the input storage type:sqrt
The input array.
org.apache.mxnet.Symbol
Returns elementwise squared value of the input.
..
Returns elementwise squared value of the input.
.. math::
square(x) = x^2
Example::
square([2, 3, 4]) = [4, 9, 16]
The storage type of
output depends upon the input storage type:square
The input array.
org.apache.mxnet.Symbol
Remove singledimensional entries from the shape of an array.
Same behavior of defining the output tensor shape as numpy.squeeze for the most of cases.
See the following note for exception.
Examples::
data = [1], [2
squeeze(data) = [0, 1, 2]
squeeze(data, axis=0) = [1], [2
squeeze(data, axis=2) = 1, 2
squeeze(data, axis=(0, 2)) = [0, 1, 2]
..
Remove singledimensional entries from the shape of an array.
Same behavior of defining the output tensor shape as numpy.squeeze for the most of cases.
See the following note for exception.
Examples::
data = [1], [2
squeeze(data) = [0, 1, 2]
squeeze(data, axis=0) = [1], [2
squeeze(data, axis=2) = 1, 2
squeeze(data, axis=(0, 2)) = [0, 1, 2]
.. Note::
The output of this operator will keep at least one dimension not removed. For example,
squeeze(4) = [4], while in numpy.squeeze, the output will become a scalar.
data to squeeze
Selects a subset of the singledimensional entries in the shape. If an axis is selected with shape entry greater than one, an error is raised.
org.apache.mxnet.Symbol
Join a sequence of arrays along a new axis.
The axis parameter specifies the index of the new axis in the dimensions of the
result.
Join a sequence of arrays along a new axis.
The axis parameter specifies the index of the new axis in the dimensions of the
result. For example, if axis=0 it will be the first dimension and if axis=1 it
will be the last dimension.
Examples::
x = [1, 2]
y = [3, 4]
stack(x, y) = 2],
[3, 4
stack(x, y, axis=1) = 3],
[2, 4
List of arrays to stack
The axis in the result array along which the input arrays are stacked.
Number of inputs to be stacked.
org.apache.mxnet.Symbol
Stops gradient computation.
Stops the accumulated gradient of the inputs from flowing through this operator
in the backward direction.
Stops gradient computation.
Stops the accumulated gradient of the inputs from flowing through this operator
in the backward direction. In other words, this operator prevents the contribution
of its inputs to be taken into account for computing gradients.
Example::
v1 = [1, 2]
v2 = [0, 1]
a = Variable('a')
b = Variable('b')
b_stop_grad = stop_gradient(3 * b)
loss = MakeLoss(b_stop_grad + a)
executor = loss.simple_bind(ctx=cpu(), a=(1,2), b=(1,2))
executor.forward(is_train=True, a=v1, b=v2)
executor.outputs
[ 1. 5.]
executor.backward()
executor.grad_arrays
[ 0. 0.]
[ 1. 1.]
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L265
The input array.
org.apache.mxnet.Symbol
Computes the sum of array elements over given axes.
..
Computes the sum of array elements over given axes.
.. Note::
sum
and sum_axis
are equivalent.
For ndarray of csr storage type summation along axis 0 and axis 1 is supported.
Setting keepdims or exclude to True will cause a fallback to dense operator.
Example::
data = 2], [2, 3], [1, 3]],
4], [4, 3], [5, 2,
1], [7, 2], [7, 3]
sum(data, axis=1)
4. 8.]
[ 10. 9.]
[ 21. 6.
sum(data, axis=[1,2])
[ 12. 19. 27.]
data = 2, 0],
[3, 0, 1],
[4, 1, 0
csr = cast_storage(data, 'csr')
sum(csr, axis=0)
[ 8. 3. 1.]
sum(csr, axis=1)
[ 3. 4. 5.]
Defined in src/operator/tensor/broadcast_reduce_op_value.cc:L115
The input
The axis or axes along which to perform the reduction.
The default, axis=()
, will compute over all elements into a
scalar array with shape (1,)
.
If axis
is int, a reduction is performed on a particular axis.
If axis
is a tuple of ints, a reduction is performed on all the axes
specified in the tuple.
If exclude
is true, reduction will be performed on the axes that are
NOT in axis instead.
Negative values means indexing from right to left.
If this is set to True
, the reduced axes are left in the result as dimension with size one.
Whether to perform reduction on axis that are NOT in axis instead.
org.apache.mxnet.Symbol
Computes the sum of array elements over given axes.
..
Computes the sum of array elements over given axes.
.. Note::
sum
and sum_axis
are equivalent.
For ndarray of csr storage type summation along axis 0 and axis 1 is supported.
Setting keepdims or exclude to True will cause a fallback to dense operator.
Example::
data = 2], [2, 3], [1, 3]],
4], [4, 3], [5, 2,
1], [7, 2], [7, 3]
sum(data, axis=1)
4. 8.]
[ 10. 9.]
[ 21. 6.
sum(data, axis=[1,2])
[ 12. 19. 27.]
data = 2, 0],
[3, 0, 1],
[4, 1, 0
csr = cast_storage(data, 'csr')
sum(csr, axis=0)
[ 8. 3. 1.]
sum(csr, axis=1)
[ 3. 4. 5.]
Defined in src/operator/tensor/broadcast_reduce_op_value.cc:L115
The input
The axis or axes along which to perform the reduction.
The default, axis=()
, will compute over all elements into a
scalar array with shape (1,)
.
If axis
is int, a reduction is performed on a particular axis.
If axis
is a tuple of ints, a reduction is performed on all the axes
specified in the tuple.
If exclude
is true, reduction will be performed on the axes that are
NOT in axis instead.
Negative values means indexing from right to left.
If this is set to True
, the reduced axes are left in the result as dimension with size one.
Whether to perform reduction on axis that are NOT in axis instead.
org.apache.mxnet.Symbol
Interchanges two axes of an array.
Examples::
x = 2, 3)
swapaxes(x, 0, 1) = 1],
[ 2],
[ 3
x = 0, 1],
[ 2, 3]],
4, 5],
[ 6, 7] // (2,2,2) array
swapaxes(x, 0, 2) = 0, 4],
[ 2, 6]],
1, 5],
[ 3, 7]
Defined in src/operator/swapaxis.cc:L70
Interchanges two axes of an array.
Examples::
x = 2, 3)
swapaxes(x, 0, 1) = 1],
[ 2],
[ 3
x = 0, 1],
[ 2, 3]],
4, 5],
[ 6, 7] // (2,2,2) array
swapaxes(x, 0, 2) = 0, 4],
[ 2, 6]],
1, 5],
[ 3, 7]
Defined in src/operator/swapaxis.cc:L70
Input array.
the first axis to be swapped.
the second axis to be swapped.
org.apache.mxnet.Symbol
Takes elements from an input array along the given axis.
This function slices the input array along a particular axis with the provided indices.
Given data tensor of rank r >= 1, and indices tensor of rank q, gather entries of the axis
dimension of data (by default outermost one as axis=0) indexed by indices, and concatenates them
in an output tensor of rank q + (r  1).
Examples::
x = [4.
Takes elements from an input array along the given axis.
This function slices the input array along a particular axis with the provided indices.
Given data tensor of rank r >= 1, and indices tensor of rank q, gather entries of the axis
dimension of data (by default outermost one as axis=0) indexed by indices, and concatenates them
in an output tensor of rank q + (r  1).
Examples::
x = [4. 5. 6.]
// Trivial case, take the second element along the first axis.
take(x, [1]) = [ 5. ]
// The other trivial case, axis=1, take the third element along the first axis
take(x, [3], axis=1, mode='clip') = [ 6. ]
x = 1., 2.],
[ 3., 4.],
[ 5., 6.
// In this case we will get rows 0 and 1, then 1 and 2. Along axis 0
take(x, 0,1],[1,2) = 1., 2.],
[ 3., 4.]],
3., 4.],
[ 5., 6.]
// In this case we will get rows 0 and 1, then 1 and 2 (calculated by wrapping around).
// Along axis 1
take(x, 3], [1, 2, axis=1, mode='wrap') = 1., 2.],
[ 3., 4.]],
3., 4.],
[ 5., 6.]
Defined in src/operator/tensor/indexing_op.cc:L406
The input array.
The indices of the values to be extracted.
The axis of input array to be taken.For input tensor of rank r, it could be in the range of [r, r1]
Specify how outofbound indices bahave. Default is "clip". "clip" means clip to the range. So, if all indices mentioned are too large, they are replaced by the index that addresses the last element along an axis. "wrap" means to wrap around. "raise" means to raise an error, not supported yet.
org.apache.mxnet.Symbol
Computes the elementwise tangent of the input array.
The input should be in radians (:math:2\pi
rad equals 360 degrees).
..
Computes the elementwise tangent of the input array.
The input should be in radians (:math:2\pi
rad equals 360 degrees).
.. math::
tan([0, \pi/4, \pi/2]) = [0, 1, inf]
The storage type of
output depends upon the input storage type:tan
The input array.
org.apache.mxnet.Symbol
Returns the hyperbolic tangent of the input array, computed elementwise.
..
Returns the hyperbolic tangent of the input array, computed elementwise.
.. math::
tanh(x) = sinh(x) / cosh(x)
The storage type of
output depends upon the input storage type:tanh
The input array.
org.apache.mxnet.Symbol
Repeats the whole array multiple times.
If
has length *d*, and input array has dimension of *n*.reps
Repeats the whole array multiple times.
If
has length *d*, and input array has dimension of *n*. There arereps
three cases:
 **n=d**. Repeat *i*th dimension of the input by
times::reps[i]
x = 2],
[3, 4
tile(x, reps=(2,3)) = 1., 2., 1., 2., 1., 2.],
[ 3., 4., 3., 4., 3., 4.],
[ 1., 2., 1., 2., 1., 2.],
[ 3., 4., 3., 4., 3., 4.
 **n>d**.
is promoted to length *n* by prepending 1's to it. Thus forreps
an input shape
, (2,3)
is treated as repos=(2,)
::(1,2)
tile(x, reps=(2,)) = 1., 2., 1., 2.],
[ 3., 4., 3., 4.
 **n<d**. The input is promoted to be ddimensional by prepending new axes. So a
shape
array is promoted to (2,2)
for 3D replication::(1,2,2)
tile(x, reps=(2,2,3)) = 1., 2., 1., 2., 1., 2.],
[ 3., 4., 3., 4., 3., 4.],
[ 1., 2., 1., 2., 1., 2.],
[ 3., 4., 3., 4., 3., 4.]],
1., 2., 1., 2., 1., 2.],
[ 3., 4., 3., 4., 3., 4.],
[ 1., 2., 1., 2., 1., 2.],
[ 3., 4., 3., 4., 3., 4.]
Defined in src/operator/tensor/matrix_op.cc:L751
Input data array
The number of times for repeating the tensor a. Each dim size of reps must be a positive integer. If reps has length d, the result will have dimension of max(d, a.ndim); If a.ndim < d, a is promoted to be ddimensional by prepending new axes. If a.ndim > d, reps is promoted to a.ndim by prepending 1's to it.
org.apache.mxnet.Symbol
Returns the top *k* elements in an input array along the given axis.
Examples::
x = 0.3, 0.2, 0.4],
[ 0.1, 0.3, 0.2
// returns an index of the largest element on last axis
topk(x) = 2.],
[ 1.
// returns the value of top2 largest elements on last axis
topk(x, ret_typ='value', k=2) = 0.4, 0.3],
[ 0.3, 0.2
// returns the value of top2 smallest elements on last axis
topk(x, ret_typ='value', k=2, is_ascend=1) = 0.2 , 0.3],
[ 0.1 , 0.2
// returns the value of top2 largest elements on axis 0
topk(x, axis=0, ret_typ='value', k=2) = 0.3, 0.3, 0.4],
[ 0.1, 0.2, 0.2
// flattens and then returns list of both values and indices
topk(x, ret_typ='both', k=2) = 0.4, 0.3], [ 0.3, 0.2]] , 2., 0.], [ 1., 2.]
Defined in src/operator/tensor/ordering_op.cc:L63
Returns the top *k* elements in an input array along the given axis.
Examples::
x = 0.3, 0.2, 0.4],
[ 0.1, 0.3, 0.2
// returns an index of the largest element on last axis
topk(x) = 2.],
[ 1.
// returns the value of top2 largest elements on last axis
topk(x, ret_typ='value', k=2) = 0.4, 0.3],
[ 0.3, 0.2
// returns the value of top2 smallest elements on last axis
topk(x, ret_typ='value', k=2, is_ascend=1) = 0.2 , 0.3],
[ 0.1 , 0.2
// returns the value of top2 largest elements on axis 0
topk(x, axis=0, ret_typ='value', k=2) = 0.3, 0.3, 0.4],
[ 0.1, 0.2, 0.2
// flattens and then returns list of both values and indices
topk(x, ret_typ='both', k=2) = 0.4, 0.3], [ 0.3, 0.2]] , 2., 0.], [ 1., 2.]
Defined in src/operator/tensor/ordering_op.cc:L63
The input array
Axis along which to choose the top k indices. If not given, the flattened array is used. Default is 1.
Number of top elements to select, should be always smaller than or equal to the element number in the given axis. A global sort is performed if set k < 1.
The return type. "value" means to return the top k values, "indices" means to return the indices of the top k values, "mask" means to return a mask array containing 0 and 1. 1 means the top k values. "both" means to return a list of both values and indices of top k elements.
Whether to choose k largest or k smallest elements. Top K largest elements will be chosen if set to false.
org.apache.mxnet.Symbol
Permutes the dimensions of an array.
Examples::
x = 1, 2],
[ 3, 4
transpose(x) = 1., 3.],
[ 2., 4.
x = 1., 2.],
[ 3., 4.]],
5., 6.],
[ 7., 8.]
transpose(x) = 1., 5.],
[ 3., 7.]],
2., 6.],
[ 4., 8.]
transpose(x, axes=(1,0,2)) = 1., 2.],
[ 5., 6.]],
3., 4.],
[ 7., 8.]
Defined in src/operator/tensor/matrix_op.cc:L310
Permutes the dimensions of an array.
Examples::
x = 1, 2],
[ 3, 4
transpose(x) = 1., 3.],
[ 2., 4.
x = 1., 2.],
[ 3., 4.]],
5., 6.],
[ 7., 8.]
transpose(x) = 1., 5.],
[ 3., 7.]],
2., 6.],
[ 4., 8.]
transpose(x, axes=(1,0,2)) = 1., 2.],
[ 5., 6.]],
3., 4.],
[ 7., 8.]
Defined in src/operator/tensor/matrix_op.cc:L310
Source input
Target axis order. By default the axes will be inverted.
org.apache.mxnet.Symbol
Return the elementwise truncated value of the input.
The truncated value of the scalar x is the nearest integer i which is closer to
zero than x is.
Return the elementwise truncated value of the input.
The truncated value of the scalar x is the nearest integer i which is closer to
zero than x is. In short, the fractional part of the signed number x is discarded.
Example::
trunc([2.1, 1.9, 1.5, 1.9, 2.1]) = [2., 1., 1., 1., 2.]
The storage type of
output depends upon the input storage type:trunc
The input array.
org.apache.mxnet.Symbol
Draw random samples from a uniform distribution.
..
Draw random samples from a uniform distribution.
.. note:: The existing alias
is deprecated.uniform
Samples are uniformly distributed over the halfopen interval *[low, high)*
(includes *low*, but excludes *high*).
Example::
uniform(low=0, high=1, shape=(2,2)) = 0.60276335, 0.85794562],
[ 0.54488319, 0.84725171
Defined in src/operator/random/sample_op.cc:L66
Lower bound of the distribution.
Upper bound of the distribution.
Shape of the output.
Context of output, in format [cpugpucpu_pinned](n). Only used for imperative calls.
DType of the output in case this can't be inferred. Defaults to float32 if not defined (dtype=None).
org.apache.mxnet.Symbol
Converts an array of flat indices into a batch of index arrays.
Converts an array of flat indices into a batch of index arrays. The operator follows numpy conventions so a single multi index is given by a column of the output matrix.
Examples::
A = [22,41,37]
unravel(A, shape=(7,6)) = 3,6,6],[4,5,1
Defined in src/operator/tensor/ravel.cc:L65
Array of flat indices
Shape of the array into which the multiindices apply.
org.apache.mxnet.Symbol
Return the elements, either from x or y, depending on the condition.
Given three ndarrays, condition, x, and y, return an ndarray with the elements from x or y,
depending on the elements from condition are true or false.
Return the elements, either from x or y, depending on the condition.
Given three ndarrays, condition, x, and y, return an ndarray with the elements from x or y,
depending on the elements from condition are true or false. x and y must have the same shape.
If condition has the same shape as x, each element in the output array is from x if the
corresponding element in the condition is true, and from y if false.
If condition does not have the same shape as x, it must be a 1D array whose size is
the same as x's first dimension size. Each row of the output array is from x's row
if the corresponding element from condition is true, and from y's row if false.
Note that all nonzero values are interpreted as
in condition.True
Examples::
x = 2], [3, 4
y = 6], [7, 8
cond = 1], [1, 0
where(cond, x, y) = 2], [3, 8
csr_cond = cast_storage(cond, 'csr')
where(csr_cond, x, y) = 2], [3, 8
Defined in src/operator/tensor/control_flow_op.cc:L57
condition array
org.apache.mxnet.Symbol
Return an array of zeros with the same shape, type and storage type
as the input array.
The storage type of
output depends on the storage type of the inputzeros_like
 zeros_like(row_sparse) = row_sparse
 zeros_like(csr) = csr
 zeros_like(default) = default
Examples::
x = 1., 1., 1.],
[ 1., 1., 1.
zeros_like(x) = 0., 0., 0.],
[ 0., 0., 0.
Return an array of zeros with the same shape, type and storage type
as the input array.
The storage type of
output depends on the storage type of the inputzeros_like
 zeros_like(row_sparse) = row_sparse
 zeros_like(csr) = csr
 zeros_like(default) = default
Examples::
x = 1., 1., 1.],
[ 1., 1., 1.
zeros_like(x) = 0., 0., 0.],
[ 0., 0., 0.
The input
org.apache.mxnet.Symbol