Graph Converters¶
- class nnabla.experimental.graph_converters.GraphConverter(modifiers=[])[ソース]¶
Convert a graph with the modifiers by traversing from output variables.
- convert(o)[ソース]¶
- パラメータ
o (list of
nnabla.Variable
) -- Output variables.
- class nnabla.experimental.graph_converters.FunctionModifier[ソース]¶
Base class of modifiers.
The
modify
method is called for a function with inputs in a graph topological order when you call the GraphConverter(<modifiers>).convert(<root variable>) method.- finish_up()[ソース]¶
Finish the very time function modification.
Clean up the internal modifier states.
- パラメータ
None --
- 戻り値
None
- get_parameter_scope(v)[ソース]¶
Get the parameter name corresponding to v
- パラメータ
v (
nnabla.Variable
) -- NNabla Variable Object.- 戻り値
Scope name
- 戻り値の型
- modify(f, inputs)[ソース]¶
Modify the function.
Implement this method in a sub class to modify a function.
Examples:
class ReLUToLeakyReLUModifier(FunctionModifier): def __init__(self): super(ReLUToLeakyReLUModifier, self).__init__() def modify(self, f, inputs): if f.info.type_name == 'ReLU': x = inputs[0] return F.leaky_relu(x)
This examples is a simple case since the network topological order does not change. In GraphConverter, we expect the modify method is called along the original network tolopogical order not the modified order. In such a complex case, see themodify method of
BatchNormalizationFoldingModifierInner
as a reference.- パラメータ
f (
nnabla.function.Function
) -- NNabla function object.inputs (list of
Variable
) -- New inputs tof
. This may be modified one or the same as f.inputs.
- 戻り値
Variable
or list ofVariable
.
Function Modifiers¶
- class nnabla.experimental.graph_converters.BatchNormalizationFoldingModifier(opposite=False, channel_last=False)[ソース]¶
Single
Convolution -> BatchNormalization
pass is folded into oneConvolution
.If there is a
Convolution -> BatchNormalization
pass, fold the batch normalization parameters to the kernel and bias (if it exists) of the preceding convolution, then skip the batch normalization following the convolution.Supported folding functions:
Convolution
,Deconvolution
,Affine
.Examples:
pred = Model(...) import nnabla.experimental.graph_converters as GC modifiers = [GC.BatchNormalizationFoldingModifier()] gc = GC.GraphConverter(modifiers) pred = gc.convert(pred)
- class nnabla.experimental.graph_converters.AddBiasModifier[ソース]¶
Add bias to
Convolution
in BatchNormalization folding case if it doesn't have bias.Supported folding functions:
Convolution
,Deconvolution
,Affine
.Examples:
pred = Model(...) import nnabla.experimental.graph_converters as GC modifiers = [GC.AddBiasModifier()] gc = GC.GraphConverter(modifiers) pred = gc.convert(pred)
- class nnabla.experimental.graph_converters.BatchNormalizationFoldingModifierInner(channel_last=False)[ソース]¶
Single
Convolution -> BatchNormalization
pass is folded into oneConvolution
.If there is a
Convolution -> BatchNormalization
pass, fold the batch normalization parameters to the kernel and bias (if it exists) of the preceding convolution, then skip the batch normalization following the convolution.Supported folding functions:
Convolution
,Deconvolution
,Affine
.
- class nnabla.experimental.graph_converters.BatchNormalizationFoldingOppositeModifierInner(channel_last=False)[ソース]¶
Single
BatchNormalization -> Convolution
pass is folded into oneConvolution
.If there is a
BatchNormalization -> Convolution
pass, fold the batch normalization parameters to the kernel and bias (if it exists) of the preceding convolution, then skip the batch normalization following the convolution.Supported folding functions:
Convolution
,Deconvolution
,Affine
.
- class nnabla.experimental.graph_converters.BatchNormalizationSelfFoldingModifier(name='bn-self-folding')[ソース]¶
The parameters of the batch normalization replaced simple scale and bias.
- パラメータ
name (
str
) -- Prefix of the parameter scope.
Examples:
pred = Model(...) import nnabla.experimental.graph_converters as GC modifiers = [GC.BatchNormalizationSelfFoldingModifier()] gc = GC.GraphConverter(modifiers) pred = gc.convert(pred)
- class nnabla.experimental.graph_converters.FusedBatchNormalizationModifier[ソース]¶
Block
BatchNormalization -> Add2 -> Non-Linear
pass is fused into oneFusedBatchNormalization
.If there is a block
BatchNormalization -> Add2 -> Non-Linear
pass, remove all the block functions and replace the whole block toFusedBatchNormalization
.Examples:
pred = Model(...) import nnabla.experimental.graph_converters as GC modifiers = [GC.FusedBatchNormalizationModifier()] gc = GC.GraphConverter(modifiers) pred = gc.convert(pred)
- class nnabla.experimental.graph_converters.UnfusedBatchNormalizationModifier[ソース]¶
Unfuse
FusedBatchNormalization
toBatchNormalization -> Add2 -> Non-Linear
block.If there is a
FusedBatchNormalization
pass, remove the fused batch normalization and replace it with the blockBatchNormalization -> Add2 -> Non-Linear
.Supported Non-Linear functions:
relu
Examples:
pred = Model(...) import nnabla.experimental.graph_converters as GC modifiers = [GC.UnfusedBatchNormalizationModifier()] gc = GC.GraphConverter(modifiers) pred = gc.convert(pred)
- class nnabla.experimental.graph_converters.ChannelLastModifier(inputs, inputs_cl=None)[ソース]¶
Convert graph shape from Channel first (NCHW) to Channel last (NHWC) format.
Supported functions:
Convolution
,Deconvolution
,BatchNormalization
,MaxPooling
,AveragePooling
,SumPooling
,Unpooling
,Concatenate
- パラメータ
inputs (list of nn.Variable) -- Original very begining inputs (NCHW) of a network.
inputs_cl (list of nn.Variable) -- Channel last version of very begining inputs (NHWC) of a network. If this is not given,
inputs_cl
are generated internally and holded.
Examples:
pred = Model(...) import nnabla.experimental.graph_converters as GC modifiers = [GC.ChannelLastModifier(<inputs of pred>)] gc = GC.GraphConverter(modifiers) pred = gc.convert(pred)
- class nnabla.experimental.graph_converters.ChannelFirstModifier(inputs, inputs_cf=None)[ソース]¶
Convert graph shape from Channel last (NHWC) to Channel first (NCHW) format.
Supported functions:
Convolution
,Deconvolution
,BatchNormalization
,MaxPooling
,AveragePooling
,SumPooling
,Unpooling
,Concatenate
- パラメータ
inputs (list of nn.Variable) -- Original channel last version of very begining inputs (NHWC) of a network.
inputs_cf (list of nn.Variable) -- Channel first version of very begining inputs (NCHW) of a network. If this is not given,
inputs_cf
are generated internally and holded.
Examples:
pred = Model(...) import nnabla.experimental.graph_converters as GC modifiers = [GC.ChannelFirstModifier(<inputs of pred>)] gc = GC.GraphConverter(modifiers) pred = gc.convert(pred)
- class nnabla.experimental.graph_converters.RemoveFunctionModifier(rm_funcs=[])[ソース]¶
Remove specified function layer(s) from a graph.
A convenient converter when one or more functions in an existing graph needs to be removed. This converter remove specified function(s) without recreating a new graph from scratch.
- パラメータ
rm_funcs (list of
str
) -- list of function name
Examples:
pred = Model(...) import nnabla.experimental.graph_converters as GC modifiers = [GC.RemoveFunctionModifier(rm_funcs=['BatchNormalization', 'MulScalar'])] gc = GC.GraphConverter(modifiers) pred = gc.convert(pred)
- class nnabla.experimental.graph_converters.BatchNormBatchStatModifier[ソース]¶
Change
batch_stat
toFalse
. Supported functions:BatchNormalization
,FusedBatchNormalization
,SyncBatchNormalization
.Examples:
pred = Model(...) import nnabla.experimental.graph_converters as GC modifiers = [GC.BatchNormBatchStatModifier()] gc = GC.GraphConverter(modifiers) pred = gc.convert(pred)
- class nnabla.experimental.graph_converters.TestModeModifier(rm_funcs=[])[ソース]¶
This converter combines BatNormBatchStateModifier and RemoveFunctionModifer. It changes
batch_stat
toFalse
. Supported functions:BatchNormalization
,FusedBatchNormalization
,SyncBatchNormalization
.Functions that specified
rm_funcs
will be removed from a graph.- パラメータ
rm_funcs (list of
str
) -- list of function name
Examples:
pred = Model(...) import nnabla.experimental.graph_converters as GC modifiers = [GC.TestModeModifier(rm_funcs=['MulScalar'])] gc = GC.GraphConverter(modifiers) pred = gc.convert(pred)
- class nnabla.experimental.graph_converters.IdentityModifier(inputs={}, copy_value=False)[ソース]¶
All functions are replaced to the same
new
function.- パラメータ
inputs (
dict
) -- Input variable mapping from the original input to another input. Default is the empty dictionary, so the new graph shares the original inputs.
Examples:
pred = Model(...) x = nn.Variable(...) import nnabla.experimental.graph_converters as GC modifiers = [GC.IdentityModifier({x0: x1})] gc = GC.GraphConverter(modifiers) pred = gc.convert(pred)
- class nnabla.experimental.graph_converters.NoGradModifier[ソース]¶
All functions are replaced to the same
new
function.- パラメータ
inputs (
dict
) -- Input variable mapping from the original input to another input. Default is the empty dictionary, so the new graph shares the original inputs.
Examples:
pred = Model(...) x = nn.Variable(...) import nnabla.experimental.graph_converters as GC modifiers = [GC.NoGradModifier()] gc = GC.GraphConverter(modifiers) pred = gc.convert(pred)
- class nnabla.experimental.graph_converters.PruningModifier(pruning_threshold, functions_to_prune=('Convolution', 'Deconvolution', 'DepthwiseConvolution', 'DepthwiseDeconvolution', 'Affine'), channel_last=False)[ソース]¶
Use
PruningModifier
to prune the small weight value to 0. The pruning is channel-wise. Using the channel-wise L2 norm to represent the degree of sparsity. If the L2 norm less than the threshold provided, all the value of this channel will be set to 0.Supported pruning functions:
Convolution
,Deconvolution
,DepthwiseConvolution
, 'DepthwiseDeconvolution', 'Affine'Examples:
pred = Model(...) import nnabla.experimental.graph_converters as GC modifiers = [GC.PruningModifier(pruning_threshold=0.1)] gc = GC.GraphConverter(modifiers) pred = gc.convert(pred)