Modules
The nnabla.core.module.Module class represents a construction block of neural network.
Module
- class nnabla.core.module.Module[ソース]
Module is a construction block of a computation model. Modules normally are constructed by lower level operators or other Modules, thus, nesting them in a tree-like structure may construct a more complex computation model.
サンプル
User may construct his model by derived from this class. Like:
import nnabla as nn import nnabla.parametric_functions as PF import nnabla.functions as F class ConvBn(nn.Module): def __init__(self, outmaps, kernel=1, stride=1, act=None): self.outmaps = outmaps self.kernel = kernel self.stride = stride self.act = act def call(self, x, training=True): kernel = (self.kernel, self.kernel) pad = (self.kernel // 2, self.kernel // 2) stride = (self.stride, self.stride) h = PF.convolution(x, self.outmaps, kernel, pad, stride, with_bias=False) h = PF.batch_normalization(h, batch_stat=training) if self.act is None: return h return self.act(h) class ResUnit(nn.Module): def __init__(self, channels, stride=1, skip_by_conv=True): self.conv1 = ConvBn(channels // 4, 1, 1, act=lambda x: F.relu(x, inplace=True)) self.conv2 = ConvBn(channels // 4, 3, stride, act=lambda x: F.relu(x, inplace=True)) self.conv3 = ConvBn(channels, 1) self.skip_by_conv = skip_by_conv self.skip = ConvBn(channels, 1, stride) def call(self, x, training=True): h = self.conv1(x) h = self.conv2(h) h = self.conv3(h) s = x if self.skip_by_conv: s = self.skip(s) h = F.relu(F.add2(h, s, inplace=True), inplace=True) return h
To use this model, user may do like the following code:
res_unit = ResUnit(1024) x = nn.Variable((64, 3, 32, 32)) x.d = np.random.random(x.shape) y = res_unit(x) y.forward(clear_buffer=True)
For working with dynamic network, user may do like the following:
res_unit = ResUnit(1024) with nn.auto_forward(): x = nn.Variable.from_numpy_array(np.random.random((1, 3, 32, 32))) y = res_unit(x) print(y.d)
For training, please set the parameters in module scope to optimizer. For example,
import nnabla.solvers as S resnet = ResNet(18) loss = resnet(x, y_) solver = S.Sgd(lr=1e-3) solver.set_parameters(resnet.get_parameters()) for _ in range(max_iter): x.d, y_.d = data.next() loss.forward() solver.zero_grad() loss.backward() solver.weight_decay(1e-5) solver.update()
In this example, we supposed ResNet is a derived class of Module, x, y_ is
Variable,datais an instance ofDataIterator, supposed it has already been attached to a DataSet.- Note:
From this example, we knew that model parameters are owned by model. Here it is variable
resnet. These parameters will be referred when network is forward or backward or solve.update(). Hence, it is necessary to keep this module instance from being unexpectedly released, to ensure forward() or backward() can refer to these variables.
- call(*args, **kwargs)[ソース]
User needs implement this function to construct their neural network. In the implementation, user may instantiate existing predefined Modules as its members, then use it. For example:
class AModule(nn.Module): def __init__(...): ... self.cnb = ConvBN(128) # A submodule is instantiated here. def call(...): h = self.cnb(x) # Using beforehand instantiated submodule.
or directly use parametric functions or functions:
class AModule(nn.Module): ... def call(...): ... h = PF.convolution(x, self.outmaps, ...) return h
注釈
The following usage is currently not supported, it might be supported in future version:
class AModule(nn.Module): def __init__(...): ... self.cnb = [ConvBN(k) for k in [8, 16, 32]] # using an array to hold module instances. self.cnb = {f'name_{k}': ConvBN(k) for k in [8, 16, 32]} # using a dict to hold module instances.
注釈
The following method to temporarily instantiate a module is also not allowed:
class AModule(nn.Module): def call(...): ... cnb = ConvBN(k) # Instantiate a temporary instance of Module is not allowed y = cnb(x) return y
Because when leave this scope, the parameters registered to
cnbmodule will be released, which cause unexpected result.
- get_parameters(recursive=True, grad_only=False, memo=None)[ソース]
Obtain an OrderedDict object of all parameters in current Module.
For example,
x = nn.Variable.from_numpy_array((np.random.random((8, 32, 256, 256)))) conv_bn = ConvBn(2) y = conv_bn(x) params = conv_bn.get_parameters() for parameter_name, parameter_value in params.items(): print("{}:{}".format(parameter_name, parameter_value.shape))
The output looks like:
conv/W:(2, 32, 1, 1) bn/beta:(1, 2, 1, 1) bn/gamma:(1, 2, 1, 1) bn/mean:(1, 2, 1, 1) bn/var:(1, 2, 1, 1)
Notice that the parameter name looks like a filepath, with splash separated nested scope name. In addition, module name default is used with a prefix
@.- パラメータ:
- 戻り値:
Flattened parameter's name-value pairs of current Module.
- 戻り値の型:
OrderedDict
- load_parameters(path, extension='.h5', raise_if_missing=True)[ソース]
Load parameters from a file into this module.
- パラメータ:
path -- str or file-like object
- property parameter_scope
A module has its owned parameter_scope, which can avoid to pollute global parameter name space. User may obtain the parameter_scope of a module by this property.
- 戻り値:
The parameter scope of current Module.
- 戻り値の型:
OrderedDict
- save_parameters(path, extension='.h5')[ソース]
Save parameters of this module to a file.
- パラメータ:
path -- str or file-like object
- property training
Return a bool value which indicates whether current Module is in training state or not. A module may be set to training state or not, so that the computation graph created from this module can be changed according to this state. For example,
class ConvBN(Module): ... def call(self, x): h = self.conv1(x) if self.training: h = self.drop_out(h) h = F.relu(h, inplace=True) return h conv_bn = ConvBN() conv_bn.training = True train_y = conv_bn(x) conv_bn.training = False eval_y = conv_bn(x)
- 戻り値:
which indicates whether current Module is in training state.
- 戻り値の型: