The nnabla.core.module.Module class represents a construction block of neural network.


class nnabla.core.module.Module[source]

Module is a construction block of a computation model. Modules normally are constructed by lower level operators or other Modules, thus, nesting them in a tree-like structure may construct a more complex computation model.


User may construct his model by derived from this class. Like:

import nnabla as nn
import nnabla.parametric_functions as PF
import nnabla.functions as F

class ConvBn(nn.Module):
    def __init__(self, outmaps, kernel=1, stride=1, act=None):
        self.outmaps = outmaps
        self.kernel = kernel
        self.stride = stride
        self.act = act

    def call(self, x, training=True):
        kernel = (self.kernel, self.kernel)
        pad = (self.kernel // 2, self.kernel // 2)
        stride = (self.stride, self.stride)
        h = PF.convolution(x, self.outmaps, kernel,
                           pad, stride, with_bias=False)
        h = PF.batch_normalization(h, batch_stat=training)
        if self.act is None:
            return h
        return self.act(h)

class ResUnit(nn.Module):
    def __init__(self, channels, stride=1, skip_by_conv=True):
        self.conv1 = ConvBn(channels // 4, 1, 1,
                            act=lambda x: F.relu(x, inplace=True))
        self.conv2 = ConvBn(channels // 4, 3, stride,
                            act=lambda x: F.relu(x, inplace=True))
        self.conv3 = ConvBn(channels, 1)
        self.skip_by_conv = skip_by_conv
        self.skip = ConvBn(channels, 1, stride)

    def call(self, x, training=True):
        h = self.conv1(x)
        h = self.conv2(h)
        h = self.conv3(h)

        s = x
        if self.skip_by_conv:
            s = self.skip(s)
        h = F.relu(F.add2(h, s, inplace=True), inplace=True)
        return h

To use this model, user may do like the following code:

res_unit = ResUnit(1024)
x = nn.Variable((64, 3, 32, 32))
x.d = np.random.random(x.shape)
y = res_unit(x)

For working with dynamic network, user may do like the following:

res_unit = ResUnit(1024)
with nn.auto_forward():
    x = nn.Variable.from_numpy_array(np.random.random((1, 3, 32, 32)))
    y = res_unit(x)

For training, please set the parameters in module scope to optimizer. For example,

import nnabla.solvers as S

resnet = ResNet(18)
loss = resnet(x, y_)

solver = S.Sgd(lr=1e-3)

for _ in range(max_iter):
    x.d, y_.d =

In this example, we supposed ResNet is a derived class of Module, x, y_ is Variable, data is an instance of DataIterator, supposed it has already been attached to a DataSet.


From this example, we knew that model parameters are owned by model. Here it is variable resnet. These parameters will be referred when network is forward or backward or solve.update(). Hence, it is necessary to keep this module instance from being unexpectedly released, to ensure forward() or backward() can refer to these variables.

call(*args, **kwargs)[source]

User needs implement this function to construct their neural network. In the implementation, user may instantiate existing predefined Modules as its members, then use it. For example:

class AModule(nn.Module):
   def __init__(...):
      self.cnb = ConvBN(128) # A submodule is instantiated here.

   def call(...):
      h = self.cnb(x) # Using beforehand instantiated submodule.

or directly use parametric functions or functions:

class AModule(nn.Module):
    def call(...):
        h = PF.convolution(x, self.outmaps, ...)
        return h


The following usage is currently not supported, it might be supported in future version:

class AModule(nn.Module):
   def __init__(...):
      self.cnb = [ConvBN(k) for k in [8, 16, 32]] # using an array to hold module instances.
      self.cnb = {f'name_{k}': ConvBN(k) for k in [8, 16, 32]} # using a dict to hold module instances.


The following method to temporarily instantiate a module is also not allowed:

class AModule(nn.Module):
   def call(...):
      cnb = ConvBN(k) # Instantiate a temporary instance of Module is not allowed
      y = cnb(x)
      return y

Because when leave this scope, the parameters registered to cnb module will be released, which cause unexpected result.

get_parameters(recursive=True, grad_only=False, memo=None)[source]

Obtain an OrderedDict object of all parameters in current Module.

For example,

x = nn.Variable.from_numpy_array((np.random.random((8, 32, 256, 256))))
conv_bn = ConvBn(2)
y = conv_bn(x)

params = conv_bn.get_parameters()
for parameter_name, parameter_value in params.items():
    print("{}:{}".format(parameter_name, parameter_value.shape))

The output looks like:

conv/W:(2, 32, 1, 1)
bn/beta:(1, 2, 1, 1)
bn/gamma:(1, 2, 1, 1)
bn/mean:(1, 2, 1, 1)
bn/var:(1, 2, 1, 1)

Notice that the parameter name looks like a filepath, with splash separated nested scope name. In addition, module name default is used with a prefix @.

  • recursive (bool, optional, default=True) – Whether obtain the parameters of current module’s submodules. Default is True.

  • grad_only (bool, optional, default=False) – Whether only obtain the grad. Default is False.


Flattened parameter’s name-value pairs of current Module.

Return type:


load_parameters(path, extension='.h5', raise_if_missing=True)[source]

Load parameters from a file into this module.


path – str or file-like object

property parameter_scope

A module has its owned parameter_scope, which can avoid to pollute global parameter name space. User may obtain the parameter_scope of a module by this property.


The parameter scope of current Module.

Return type:


save_parameters(path, extension='.h5')[source]

Save parameters of this module to a file.


path – str or file-like object

property training

Return a bool value which indicates whether current Module is in training state or not. A module may be set to training state or not, so that the computation graph created from this module can be changed according to this state. For example,

class ConvBN(Module):
    def call(self, x):
        h = self.conv1(x)
            h = self.drop_out(h)
        h = F.relu(h, inplace=True)
        return h

conv_bn = ConvBN() = True
train_y = conv_bn(x) = False
eval_y = conv_bn(x)

which indicates whether current Module is in training state.

Return type:



Clear the gradient of the parameters in this module to 0.