Modules¶
The nnabla.core.module.Module
class represents a construction block of neural network.
Module¶
- class nnabla.core.module.Module[source]¶
Module is a construction block of a computation model. Modules normally are constructed by lower level operators or other Modules, thus, nesting them in a tree-like structure may construct a more complex computation model.
Example
User may construct his model by derived from this class. Like:
import nnabla as nn import nnabla.parametric_functions as PF import nnabla.functions as F class ConvBn(nn.Module): def __init__(self, outmaps, kernel=1, stride=1, act=None): self.outmaps = outmaps self.kernel = kernel self.stride = stride self.act = act def call(self, x, training=True): kernel = (self.kernel, self.kernel) pad = (self.kernel // 2, self.kernel // 2) stride = (self.stride, self.stride) h = PF.convolution(x, self.outmaps, kernel, pad, stride, with_bias=False) h = PF.batch_normalization(h, batch_stat=training) if self.act is None: return h return self.act(h) class ResUnit(nn.Module): def __init__(self, channels, stride=1, skip_by_conv=True): self.conv1 = ConvBn(channels // 4, 1, 1, act=lambda x: F.relu(x, inplace=True)) self.conv2 = ConvBn(channels // 4, 3, stride, act=lambda x: F.relu(x, inplace=True)) self.conv3 = ConvBn(channels, 1) self.skip_by_conv = skip_by_conv self.skip = ConvBn(channels, 1, stride) def call(self, x, training=True): h = self.conv1(x) h = self.conv2(h) h = self.conv3(h) s = x if self.skip_by_conv: s = self.skip(s) h = F.relu(F.add2(h, s, inplace=True), inplace=True) return h
To use this model, user may do like the following code:
res_unit = ResUnit(1024) x = nn.Variable((64, 3, 32, 32)) x.d = np.random.random(x.shape) y = res_unit(x) y.forward(clear_buffer=True)
For working with dynamic network, user may do like the following:
res_unit = ResUnit(1024) with nn.auto_forward(): x = nn.Variable.from_numpy_array(np.random.random((1, 3, 32, 32))) y = res_unit(x) print(y.d)
For training, please set the parameters in module scope to optimizer. For example,
import nnabla.solvers as S resnet = ResNet(18) loss = resnet(x, y_) solver = S.Sgd(lr=1e-3) solver.set_parameters(resnet.get_parameters()) for _ in range(max_iter): x.d, y_.d = data.next() loss.forward() solver.zero_grad() loss.backward() solver.weight_decay(1e-5) solver.update()
In this example, we supposed ResNet is a derived class of Module, x, y_ is
Variable
,data
is an instance ofDataIterator
, supposed it has already been attached to a DataSet.- Note:
From this example, we knew that model parameters are owned by model. Here it is variable
resnet
. These parameters will be referred when network is forward or backward or solve.update(). Hence, it is necessary to keep this module instance from being unexpectedly released, to ensure forward() or backward() can refer to these variables.
- call(*args, **kwargs)[source]¶
User needs implement this function to construct their neural network. In the implementation, user may instantiate existing predefined Modules as its members, then use it. For example:
class AModule(nn.Module): def __init__(...): ... self.cnb = ConvBN(128) # A submodule is instantiated here. def call(...): h = self.cnb(x) # Using beforehand instantiated submodule.
or directly use parametric functions or functions:
class AModule(nn.Module): ... def call(...): ... h = PF.convolution(x, self.outmaps, ...) return h
Note
The following usage is currently not supported, it might be supported in future version:
class AModule(nn.Module): def __init__(...): ... self.cnb = [ConvBN(k) for k in [8, 16, 32]] # using an array to hold module instances. self.cnb = {f'name_{k}': ConvBN(k) for k in [8, 16, 32]} # using a dict to hold module instances.
Note
The following method to temporarily instantiate a module is also not allowed:
class AModule(nn.Module): def call(...): ... cnb = ConvBN(k) # Instantiate a temporary instance of Module is not allowed y = cnb(x) return y
Because when leave this scope, the parameters registered to
cnb
module will be released, which cause unexpected result.
- get_parameters(recursive=True, grad_only=False, memo=None)[source]¶
Obtain an OrderedDict object of all parameters in current Module.
For example,
x = nn.Variable.from_numpy_array((np.random.random((8, 32, 256, 256)))) conv_bn = ConvBn(2) y = conv_bn(x) params = conv_bn.get_parameters() for parameter_name, parameter_value in params.items(): print("{}:{}".format(parameter_name, parameter_value.shape))
The output looks like:
conv/W:(2, 32, 1, 1) bn/beta:(1, 2, 1, 1) bn/gamma:(1, 2, 1, 1) bn/mean:(1, 2, 1, 1) bn/var:(1, 2, 1, 1)
Notice that the parameter name looks like a filepath, with splash separated nested scope name. In addition, module name default is used with a prefix
@
.- Parameters
- Returns
Flattened parameter’s name-value pairs of current Module.
- Return type
OrderedDict
- load_parameters(path, extension='.h5', raise_if_missing=True)[source]¶
Load parameters from a file into this module.
- Parameters
path – str or file-like object
- property parameter_scope¶
A module has its owned parameter_scope, which can avoid to pollute global parameter name space. User may obtain the parameter_scope of a module by this property.
- Returns
The parameter scope of current Module.
- Return type
OrderedDict
- save_parameters(path, extension='.h5')[source]¶
Save parameters of this module to a file.
- Parameters
path – str or file-like object
- property training¶
Return a bool value which indicates whether current Module is in training state or not. A module may be set to training state or not, so that the computation graph created from this module can be changed according to this state. For example,
class ConvBN(Module): ... def call(self, x): h = self.conv1(x) if self.training: h = self.drop_out(h) h = F.relu(h, inplace=True) return h conv_bn = ConvBN() conv_bn.training = True train_y = conv_bn(x) conv_bn.training = False eval_y = conv_bn(x)
- Returns
which indicates whether current Module is in training state.
- Return type