Variable¶

class nnabla.Variable¶

Bases: object

nnabla.Variable is used to construct computation graphs (neural networks) together with functions in Functions and List of Parametric Functions . It also provides a method to execute forward and backward propagation of the network. The nnabla.Variable class holds:

Reference to the parent function in a computation graph. This provides traceability of all connections in the computation graph.
Both data and error signal (gradient) containers as nnabla._nd_array.NdArray s.
Some additional information of the computation graph.

Variable overrides some arithmetic operators (+, -, *, /, **). Operands can be either a scalar number, NdArray or Variable. If NdArray is given as either of left or right operand, the arithmetic operation returns an NdArray which stores the output of the computation immediately invoked. Otherwise, it returns Variable holds the graph connection. The computation is invoked immediately when :function:`nnabla.auto_forward` or :function:`nnabla.set_auto_forward(True)` is used.

Note

Relational operators == and != of two Variable s are defined as an address comparison of underlying C++ instances (nbla::Variable). Also, hash() function, which is often used in a key for set and dict, is based on the address.

See also

Python API Tutorial.

Parameters:	shape (Iterable of int) – Shape of variable. need_grad (bool) – Flag for backprop or not.

apply(self, **kwargs)¶: Helper for setting property, then return self.

backward(self, grad=1, bool clear_buffer=False, communicator_callbacks=None, function_pre_hook=None, function_post_hook=None)¶

Performs a backward propagation starting from this variable until the root variable(s) is/are reached in the function graph. The propagation will stop at a variable with need_grad=False.

Parameters:

grad (scalar, numpy.ndarray, or nnabla._nd_array.NdArray) – The gradient signal value(s) of this variable. The default value 1 is used in an usual neural network training. This option is useful if you have a gradient computation module outside NNabla, and want to use it as a gradient signal of the neural network built in NNabla. Note that this doesn’t modifies the grad values of this variable.
clear_buffer (bool) – Clears the no longer referenced variables during backpropagation to save memory.
communicator_callbacks (nnabla.CommunicatorBackwardCallback or list of nnabla.CommunicatorBackwardCallback) – The callback functions invoked when 1) backward computation of each function is finished and 2) all backward computation is finished.
function_pre_hook (callable) – This callable object is called immediately before each function is executed. It must take Function as an input. The default is None.
function_post_hook (callable) – This callable object is called immediately after each function is executed. It must take Function as an input. The default is None.

clear_all_graph_links(self)¶

Clear all intermediate functions and variables.

This method clear all intermediate functions and variables up to this variable in forward pass and is useful for the truncated backpropagation through time (truncated BPTT) in dynamic graph.

d¶

Returns the values held by this variable, as a numpy.ndarray. Note that the values are referenced (not copied). Therefore, the modification of the returned ndarray will affect the data of the NNabla array. This method can be called as a setter to set the value held by this variable.

Parameters:	value (`numpy.ndarray`) (optional) –
Returns:	`numpy.ndarray`

data¶

Returns the data held by this variable, as a NdArray. This can also be used as a setter.

Parameters:	ndarray (NdArray) – NdArray object. Size must be the same as this Variable.
Returns:	`NdArray`

forward(self, bool clear_buffer=False, bool clear_no_need_grad=False, function_pre_hook=None, function_post_hook=None)¶

Performs a forward propagation from the root node to this variable. The forward propagation is performed on a subset of variables determined by the dependency of this variable. The subset is recursively constructed by tracking variables that the variables in the subset depend on, starting from this variable, until it reaches the root variable(s) in the function graph.

Parameters:

clear_buffer (bool) – Clear the no longer referenced variables during forward propagation to save memory. This is usually set as True in an inference or a validation phase. Default is False.
clear_no_need_grad (bool) – Clear the unreferenced variables with need_grad=False during forward propagation. True is usually used when calling this during training. This is ignored when clear_buffer=True.
function_pre_hook (callable) – This callable object is called immediately before each function is executed. It must take Function as an input. The default is None.
function_post_hook (callable) – This callable object is called immediately after each function is executed. It must take Function as an input. The default is None.

static from_numpy_array(data, grad=None, need_grad=None)¶

Create a Variable object from Numpy array(s).

The data is initialized with the given Numpy array, as well as grad if given.

The shape is also determined by the given array.

Parameters:	data (ndarray) – Values copied to the `data` of the created Variable. grad (ndarray) – Values copied to the `grad` of the created Variable. need_grad (bool) – Flag for backprop or not.

Returns: ~nnabla.Variable

function_references¶

Returns a list of functions which take this variable as an input. This method can be called only as a getter.

Returns:	list of `nnabla.function.Function`

g¶

Returns the gradient values held by this variable, as a numpy.ndarray. Note that the values are referenced (not copied). Therefore, the modification of the returned ndarray will affect the data of the NNabla array. This method can be called as a setter to set the gradient held by this variable.

Parameters:	value (`numpy.ndarray`) –
Returns:	`numpy.ndarray`

get_unlinked_variable(self, need_grad=None)¶

Gets an unlinked (forgetting parent) variable that shares a Variable buffer instance.

Parameters:	need_grad (bool, optional) – By default, the unlinked variable will have the same need_grad flag with this variable instance. By specifying a boolean value, the new need_grad flags will be set to the unlinked variable. It is recommended to explicitly specify this option to avoid an unintended behavior.

Returns: nnabla._variable.Variable

Note

The unlinked Variable behaves equivalent to the original variable in a comparison operator and hash function regardless whether or not the need_grad attribute is changed. See a note in the Variable class documentation.

Example

import numpy as np
import nnabla as nn
import nnabla.parametric_functions as PF

x = nn.Variable.from_numpy_array(np.array([[1, 2], [3, 4]]))
y = PF.affine(x, 4, name="y")

# Create a new variable of which graph connection is unlinked.
# Recommend to specify need_grad option explicitly .
z = y.get_unlinked_variable(need_grad=False)

print(y.parent)
# Affine
print(z.parent)  # z is unlinked from the parent x but shares the buffers of y.
# None

grad¶

Returns the gradient held by this variable, as a NdArray. This can also be used as a setter.

Parameters:	ndarray (NdArray) – NdArray object. Size must be the same as this Variable.
Returns:	`NdArray`

info¶

object

Information of the variable.

Type:	info

ndim¶

Gets the number of dimensions of this variable.

Returns: int

need_grad¶

Gets or sets a boolean indicating whether backpropagation is performed at this variable.

Parameters:	b (bool) – Whether backpropagation is performed at this variable.
Returns:	Whether this variable requires gradient or not.
Return type:	bool

parent¶

Returns the parent function of this variable. This method can also be called as a setter.

Parameters:	func (`nnabla.function.Function`) –
Returns:	`nnabla.function.Function`

persistent¶

Returns the persistent flag of this variable. If True, the variable is not cleared even if clear options in nnabla._variable.Variable.forward() and nnabla._variable.Variable.backward() are enabled. This is useful when you debug the variable values, or log them. This method can also be called as a setter.

Parameters:	b (bool) –

Returns: bool

reset_shape(self, shape, force=False)¶

Resizes the shape of the variable to a specified shape.

Parameters:	shape (Iterable of int) – Target shape. force (bool) – Flag to force reshape.

Note

This method destructively changes the shape of the target variable. For safety, reshape() should be used instead.

Returns:	None

reshape(self, shape, unlink=False)¶

Returns a new variable, where this variable is reshaped to a specified shape.

Parameters:	shape (Iterable of int) – Target shape. unlink (bool) – Unlink graph connection. Or, keep graph connection, i.e. the gradient will be backprop-ed to the original variable.
Returns:	`Variable`

rewire_on(self, var)¶

Rewire a successor graph of this variable on top of var.

Parameters:	var (`nnabla.Variable`) – The array elements and the parent function of `var` is copied to `self as references. Note that the parent function of `var` is removed.

Example

# A. Create a graph A.
xa = nn.Variable((2, 8), need_grad=True)
ya = F.tanh(PF.affine(xa, 10, name='a'))

# B. Create a graph B.
xb = nn.Variable((2, 16), need_grad=True)
yb = F.tanh(PF.affine(
    F.tanh(PF.affine(xb, 8, name='b1')),
    8, name='b2'))

# C. Rewire the graph A on top of B such that
#    `xb->B->(yb->)xa->A->ya`. Note `yb` is gone.
xa.rewire_on(yb)

# D. Execute the rewired graph.
xb.d = 1
ya.forward()
ya.backward()

shape¶

Gets the shape of the variable.

Returns: tuple of int

size¶

Gets the size of the variable.

Returns: int

size_from_axis(self, axis=-1)¶

Gets the size followed by the provided axis.

Example

a = nnabla.Variable([10,9])
a.size_from_axis()
# ==> 90
a.size_from_axis(0)
# ==> 90
a.size_from_axis(1)
# ==> 9
a.size_from_axis(2)
# ==> 1

Parameters:	axis (`int`, optional) – -1 as default
Returns:	`int`

unlinked(self, need_grad=None)¶: This function is deprecated, use get_unlinked_variable instead.

visit(self, f)¶

Visit functions recursively in forward order.

Parameters:	f (function) – Function object which takes `nnabla._function.Function` object as an argument.

Returns: None

Example

import nnabla as nn
import nnabla.functions as F
import nnabla.parametric_functions as PF

# Define a simple network-graph
def network_graph(x, maps=16, test=False):
    h = x
    h = PF.convolution(h, maps, kernel=(3, 3), pad=(1, 1), name="first-conv", with_bias=False)
    h = F.average_pooling(h, h.shape[2:])
    pred = PF.affine(h, 10, name="pred")
    return pred

# You can modify this PrintFunc to get the other informations like inputs(nnabla_func.inputs), outputs and arguments(nnabla_func.info.args) of nnabla functions.
class PrintFunc(object):
    def __call__(self, nnabla_func):
        print(nnabla_func.info.type_name)

x = nn.Variable([1, 3, 16, 16])
output = network_graph(x)
output.visit(PrintFunc())

Output :

Convolution
AveragePooling
Affine

visit_check(self, f)¶

Visit functions recursively in forward order.

Note

If any of evaluation of the function object returns True, the visit propagation will stop immediately, and will return True.

Parameters:	f (function) – Function object which takes `nnabla._function.Function` object as an argument.

Returns: bool: Returns True if any of the function object call returns True.

Example

Define a simple network-graph where AveragePooling function can be added explicitly as below:

def network_graph(x, add_avg_pool=False, maps=16, test=False):
    h = x
    h = PF.convolution(h, maps, kernel=(3, 3), pad=(1, 1), name="first-conv", with_bias=False)
    if add_avg_pool :
        h = F.average_pooling(h, h.shape[2:])
    else :
        h = F.relu(h)
    pred = PF.affine(h, 10, name="pred")
    return pred

# Define 'PrintFunc()' to check whether "AveragePooling" function exists in the network-graph
class PrintFunc(object):
    def __call__(self, nnabla_func):
        if nnabla_func.info.type_name =="AveragePooling" :
            print("{} exists in the graph".format(nnabla_func.info.type_name))
            return True
        else :
            return False

Create a network-graph which has AveragePooling function and call visit_check() method :

x = nn.Variable([1, 3, 16, 16])
output = network_graph(x, add_avg_pool=True)  #Adding AveragePooling function to the graph
print("The return value of visit_check() method is : {}".format(output.visit_check(PrintFunc())))

Output :

AveragePooling exists in the graph
The return value of visit_check() method is : True

Create a network-graph which doesn’t have AveragePooling function and call visit_check() method :

nn.clear_parameters()                         # call this in case you want to run the following code agian
output = network_graph(x, add_avg_pool=False) # Exclusion of AveragePooling function in the graph
print("The return value of visit_check() method is : {}".format(output.visit_check(PrintFunc())))

Output :

The return value of visit_check() method is : False