# Parametric Function Classes¶

class nnabla.experimental.parametric_function_class.affine.Affine(n_inmaps, n_outmaps, base_axis=1, w_init=None, b_init=None, fix_parameters=False, rng=None, with_bias=True)[source]

The affine layer, also known as the fully connected layer. Computes

${\mathbf y} = {\mathbf A} {\mathbf x} + {\mathbf b}.$

where $${\mathbf x}, {\mathbf y}$$ are the inputs and outputs respectively, and $${\mathbf A}, {\mathbf b}$$ are constants.

Parameters
Returns

$$(B + 1)$$-D array. ($$M_0 \times \ldots \times M_{B-1} \times L$$)f

Return type

Variable

nnabla.experimental.parametric_function_class.affine.Linear
class nnabla.experimental.parametric_function_class.convolution.Convolution(inmaps, outmaps, kernel, pad=None, stride=None, dilation=None, group=1, w_init=None, b_init=None, base_axis=1, fix_parameters=False, rng=None, with_bias=True)[source]

N-D Convolution with a bias term.

For Dilated Convolution (a.k.a. Atrous Convolution), refer to:

Note

Convolution is a computationally intensive operation that should preferably be run with the cudnn backend. NNabla then uses CuDNN library functions to determine and cache the fastest algorithm for the given set of convolution parameters, which results in additional memory consumption which may pose a problem for GPUs with insufficient memory size. In that case, the NNABLA_CUDNN_WORKSPACE_LIMIT environment variable can be used to restrict the choice of algorithms to those that fit the given workspace memory limit, expressed in bytes. In some cases it may also be desired to restrict the automatic search to algorithms that produce deterministic (reproducable) results. This can be requested by setting the the environment variable NNABLA_CUDNN_DETERMINISTIC to a non-zero value.

Parameters
Returns

N-D array. See convolution for the output shape.

Return type

Variable

nnabla.experimental.parametric_function_class.convolution.Conv1d
nnabla.experimental.parametric_function_class.convolution.Conv2d
nnabla.experimental.parametric_function_class.convolution.Conv3d
nnabla.experimental.parametric_function_class.convolution.ConvNd
class nnabla.experimental.parametric_function_class.deconvolution.Deconvolution(inmaps, outmaps, kernel, pad=None, stride=None, dilation=None, group=1, w_init=None, b_init=None, base_axis=1, fix_parameters=False, rng=None, with_bias=True)[source]

Deconvolution layer.

Parameters
Returns

N-D array. See deconvolution for the output shape.

Return type

Variable

nnabla.experimental.parametric_function_class.deconvolution.Deconv1d
nnabla.experimental.parametric_function_class.deconvolution.Deconv2d
nnabla.experimental.parametric_function_class.deconvolution.Deconv3d
nnabla.experimental.parametric_function_class.deconvolution.DeconvNd
class nnabla.experimental.parametric_function_class.batch_normalization.BatchNormalization(n_features, n_dims, axes=[1], decay_rate=0.9, eps=1e-05, batch_stat=True, output_stat=False, fix_parameters=False, param_init=None)[source]

Batch normalization layer.

$\begin{split}\begin{array}{lcl} \mu &=& \frac{1}{M} \sum x_i\\ \sigma^2 &=& \frac{1}{M} \left(\sum x_i - \mu\right)^2\\ \hat{x}_i &=& \frac{x_i - \mu}{\sqrt{\sigma^2 + \epsilon }}\\ y_i &= & \hat{x}_i \gamma + \beta. \end{array}\end{split}$

where $$x_i, y_i$$ are the inputs. In testing, the mean and variance computed by moving average calculated during training are used.

Parameters
• inp (Variable) – N-D array of input.

• axes (tuple of int) – Mean and variance for each element in axes are calculated using elements on the rest axes. For example, if an input is 4 dimensions, and axes is [1], batch mean is calculated as np.mean(inp.d, axis=(0, 2, 3), keepdims=True) (using numpy expression as an example).

• decay_rate (float) – Decay rate of running mean and variance.

• eps (float) – Tiny value to avoid zero division by std.

• batch_stat (bool) – Use mini-batch statistics rather than running ones.

• output_stat (bool) – Output batch mean and variance.

• fix_parameters (bool) – When set to True, the beta and gamma will not be updated.

• param_init (dict) – Parameter initializers can be set with a dict. A key of the dict must be 'beta', 'gamma', 'mean' or 'var'. A value of the dict must be an Initializer or a numpy.ndarray. E.g. {'beta': ConstantInitializer(0), 'gamma': np.ones(gamma_shape) * 2}.

Returns

N-D array.

Return type

Variable

References

The shape of parameters has the same number of dimensions with the input data, and the shapes in axes has the same dimensions with the input, while the rest has 1. If an input is 4-dim and axes=[1], the parameter shape will be param_shape  = np.mean(inp.d, axis=(0, 2, 3), keepdims=True).shape (using numpy expression as an example).

class nnabla.experimental.parametric_function_class.batch_normalization.BatchNorm1d(n_features, axes=[1], decay_rate=0.9, eps=1e-05, batch_stat=True, output_stat=False, fix_parameters=False, param_init=None)[source]

Batch normalization layer for 3d-Array or 3d-Variable. This is typically used together with Conv1d.

$\begin{split}\begin{array}{lcl} \mu &=& \frac{1}{M} \sum x_i\\ \sigma^2 &=& \frac{1}{M} \left(\sum x_i - \mu\right)^2\\ \hat{x}_i &=& \frac{x_i - \mu}{\sqrt{\sigma^2 + \epsilon }}\\ y_i &= & \hat{x}_i \gamma + \beta. \end{array}\end{split}$

where $$x_i, y_i$$ are the inputs. In testing, the mean and variance computed by moving average calculated during training are used.

Parameters
• inp (Variable) – N-D array of input.

• axes (tuple of int) – Mean and variance for each element in axes are calculated using elements on the rest axes. For example, if an input is 4 dimensions, and axes is [1], batch mean is calculated as np.mean(inp.d, axis=(0, 2, 3), keepdims=True) (using numpy expression as an example).

• decay_rate (float) – Decay rate of running mean and variance.

• eps (float) – Tiny value to avoid zero division by std.

• batch_stat (bool) – Use mini-batch statistics rather than running ones.

• output_stat (bool) – Output batch mean and variance.

• fix_parameters (bool) – When set to True, the beta and gamma will not be updated.

• param_init (dict) – Parameter initializers can be set with a dict. A key of the dict must be 'beta', 'gamma', 'mean' or 'var'. A value of the dict must be an Initializer or a numpy.ndarray. E.g. {'beta': ConstantInitializer(0), 'gamma': np.ones(gamma_shape) * 2}.

Returns

N-D array.

Return type

Variable

References

The shape of parameters has the same number of dimensions with the input data, and the shapes in axes has the same dimensions with the input, while the rest has 1. If an input is 4-dim and axes=[1], the parameter shape will be param_shape  = np.mean(inp.d, axis=(0, 2, 3), keepdims=True).shape (using numpy expression as an example).

class nnabla.experimental.parametric_function_class.batch_normalization.BatchNorm2d(n_features, axes=[1], decay_rate=0.9, eps=1e-05, batch_stat=True, output_stat=False, fix_parameters=False, param_init=None)[source]

Batch normalization layer for 4d-Array or 4d-Variable. This is typically used together with Conv2d.

$\begin{split}\begin{array}{lcl} \mu &=& \frac{1}{M} \sum x_i\\ \sigma^2 &=& \frac{1}{M} \left(\sum x_i - \mu\right)^2\\ \hat{x}_i &=& \frac{x_i - \mu}{\sqrt{\sigma^2 + \epsilon }}\\ y_i &= & \hat{x}_i \gamma + \beta. \end{array}\end{split}$

where $$x_i, y_i$$ are the inputs. In testing, the mean and variance computed by moving average calculated during training are used.

Parameters
• inp (Variable) – N-D array of input.

• axes (tuple of int) – Mean and variance for each element in axes are calculated using elements on the rest axes. For example, if an input is 4 dimensions, and axes is [1], batch mean is calculated as np.mean(inp.d, axis=(0, 2, 3), keepdims=True) (using numpy expression as an example).

• decay_rate (float) – Decay rate of running mean and variance.

• eps (float) – Tiny value to avoid zero division by std.

• batch_stat (bool) – Use mini-batch statistics rather than running ones.

• output_stat (bool) – Output batch mean and variance.

• fix_parameters (bool) – When set to True, the beta and gamma will not be updated.

• param_init (dict) – Parameter initializers can be set with a dict. A key of the dict must be 'beta', 'gamma', 'mean' or 'var'. A value of the dict must be an Initializer or a numpy.ndarray. E.g. {'beta': ConstantInitializer(0), 'gamma': np.ones(gamma_shape) * 2}.

Returns

N-D array.

Return type

Variable

References

The shape of parameters has the same number of dimensions with the input data, and the shapes in axes has the same dimensions with the input, while the rest has 1. If an input is 4-dim and axes=[1], the parameter shape will be param_shape  = np.mean(inp.d, axis=(0, 2, 3), keepdims=True).shape (using numpy expression as an example).

class nnabla.experimental.parametric_function_class.batch_normalization.BatchNorm3d(n_features, axes=[1], decay_rate=0.9, eps=1e-05, batch_stat=True, output_stat=False, fix_parameters=False, param_init=None)[source]

Batch normalization layer for 5d-Array or 5d-Variable. This is typically used together with Conv3d.

$\begin{split}\begin{array}{lcl} \mu &=& \frac{1}{M} \sum x_i\\ \sigma^2 &=& \frac{1}{M} \left(\sum x_i - \mu\right)^2\\ \hat{x}_i &=& \frac{x_i - \mu}{\sqrt{\sigma^2 + \epsilon }}\\ y_i &= & \hat{x}_i \gamma + \beta. \end{array}\end{split}$

where $$x_i, y_i$$ are the inputs. In testing, the mean and variance computed by moving average calculated during training are used.

Parameters
• inp (Variable) – N-D array of input.

• axes (tuple of int) – Mean and variance for each element in axes are calculated using elements on the rest axes. For example, if an input is 4 dimensions, and axes is [1], batch mean is calculated as np.mean(inp.d, axis=(0, 2, 3), keepdims=True) (using numpy expression as an example).

• decay_rate (float) – Decay rate of running mean and variance.

• eps (float) – Tiny value to avoid zero division by std.

• batch_stat (bool) – Use mini-batch statistics rather than running ones.

• output_stat (bool) – Output batch mean and variance.

• fix_parameters (bool) – When set to True, the beta and gamma will not be updated.

• param_init (dict) – Parameter initializers can be set with a dict. A key of the dict must be 'beta', 'gamma', 'mean' or 'var'. A value of the dict must be an Initializer or a numpy.ndarray. E.g. {'beta': ConstantInitializer(0), 'gamma': np.ones(gamma_shape) * 2}.

Returns

N-D array.

Return type

Variable

References

The shape of parameters has the same number of dimensions with the input data, and the shapes in axes has the same dimensions with the input, while the rest has 1. If an input is 4-dim and axes=[1], the parameter shape will be param_shape  = np.mean(inp.d, axis=(0, 2, 3), keepdims=True).shape (using numpy expression as an example).

class nnabla.experimental.parametric_function_class.embed.Embed(n_inputs, n_features, w_init=None, fix_parameters=False)[source]

Embed.

Embed slices a matrix/tensor with indexing array/tensor. Weights are initialized with nnabla.initializer.UniformInitializer within the range of $$-\sqrt{3}$$ and $$\sqrt{3}$$.

Parameters
• x (Variable) – [Integer] Indices with shape $$(I_0, ..., I_N)$$

• n_inputs – number of possible inputs, words or vocabraries

• n_features – number of embedding features

• fix_parameters (bool) – When set to True, the embedding weight matrix will not be updated.

Returns

Output with shape $$(I_0, ..., I_N, W_1, ..., W_M)$$

Return type

Variable