RNN Utils

class nnabla.utils.rnn.PackedSequence[source]
Parameters

  • data (nnabla.Variable) – Packed sequence.

  • batch_sizes (nnabla.Variable) – Batch size for each time step and always resides in CPU.

  • sorted_indices (nnabla.Variable) – Sorted indices to reconstruct the original sequences.

  • unsorted_indices (nnabla.Variable) – Unsorted indices to reconstruct the original sequences.

nnabla.utils.rnn.pad_sequence(sequences, batch_first=False, padding_value=0.0)[source]

Pad a list of variable-length Variables.

This method stacks a list of variable-length nnabla.Variable s with the padding_value.

\(T_i\) is the length of the \(i\)-th Variable in the sequences. \(B\) is the batch size equal to the length of the sequences. \(T\) is the max of \(T_i\) for all \(i\). \(*\) is the remaining dimensions including none.

Note

This function must be used the dynamic computation mode.

Example:

import numpy as np
import nnabla as nn
import nnabla.functions as F
import nnabla.utils.rnn as rnn_utils

nn.set_auto_forward(True)

l2v = lambda ldata: nn.Variable.from_numpy_array(np.asarray(ldata))
a = l2v([1, 1, 1, 1])
b = l2v([2, 2, 2])
c = l2v([2, 2, 2])
d = l2v([3, 3])
e = l2v([3, 3])
sequences = [a, b, c, d, e]

padded_sequence = rnn_utils.pad_sequence(sequences)
print(padded_sequence.d)
Parameters

  • sequences (list of nnabla.Variable) – Sequence of the variable of (\(T_i\), \(*\)) shape.

  • batch_first (bool) – If False, output is of (\(T\), \(B\), \(*\)) shape, otherwise (\(B\), \(T\), \(*\)).

  • padding_value (float) – Padding value.

Returns

nnabla.Variable of (\(T\), \(B\), \(*\)) or (\(B\), \(T\), \(*\)) shape

nnabla.utils.rnn.pack_padded_sequence(padded_sequence, lengths, batch_first=False, enforce_sorted=True)[source]

Pack a padded variable-length sequences.

This method packs a padded variable-length sequences.

\(T\) is the max length over the lengths of sequences. \(B\) is the batch size equal to the length of the sequences. \(*\) is the remaining dimensions including none.

Note

This function must be used the dynamic computation mode.

Example:

import numpy as np
import nnabla as nn
import nnabla.functions as F
import nnabla.utils.rnn as rnn_utils

nn.set_auto_forward(True)

l2v = lambda ldata: nn.Variable.from_numpy_array(np.asarray(ldata))
a = l2v([1, 1, 1, 1])
b = l2v([2, 2, 2])
c = l2v([2, 2, 2])
d = l2v([3, 3])
e = l2v([3, 3])
sequences = [a, b, c, d, e]
lengths = l2v([seq.shape[0] for seq in sequences])

padded_sequence = rnn_utils.pad_sequence(sequences)
print(padded_sequence.d)

packed_sequence = rnn_utils.pack_padded_sequence(padded_sequence, lengths)
print(packed_sequence.data.d)
print(packed_sequence.batch_sizes.d)
Parameters

  • padded_sequence (nnabla.Variable) – Padded sequence of (\(T \times B \times *\)) or (\(B \times T \times *\)) shape.

  • lengths (nnabla.Variable) – Sequence length for each batch and always resides in CPU.

  • batch_first (bool) – padded_sequence is of (\(T\), \(B\), \(*\)) shape if False, otherwise (\(B\), \(T\), \(*\)).

  • enforce_sorted (bool) – Sequences are sorted by the length in a decreasing order if True. Default is True.

Returns

PackedSequence

nnabla.utils.rnn.pack_sequence(sequences, batch_first=False, enforce_sorted=True)[source]

Pack a list of variable-length Variables.

This method packs a list of variable-length Variables.

\(T_i\) is the length of the \(i\)-th Variable in the sequences. \(T\) is the max of \(T_i\) for all \(i\). \(B\) is the batch size equal to the length of the sequences. \(*\) is the remaining dimensions including none.

Note

This function must be used the dynamic computation mode.

Example:

import numpy as np
import nnabla as nn
import nnabla.functions as F
import nnabla.utils.rnn as rnn_utils

nn.set_auto_forward(True)

l2v = lambda ldata: nn.Variable.from_numpy_array(np.asarray(ldata))
a = l2v([3, 3])
b = l2v([2, 2, 2])
c = l2v([2, 2, 2])
d = l2v([1, 1, 1, 1])
e = l2v([3, 3])
sequences = [a, b, c, d, e]

packed_sequence = rnn_utils.pack_sequence(sequences, enforce_sorted=False)
print(packed_sequence.data.d)
print(packed_sequence.batch_sizes.d)
Parameters

  • sequences (list of nnabla.Variable) – List of nnabla.Variable of (\(T_i\), \(*\)) shape.

  • enforce_sorted (bool) – Sequences are sorted by the length in a decreasing order if True. Default is True.

Returns

packed_sequence

Return type

PackedSequence

nnabla.utils.rnn.pad_packed_sequence(sequence, batch_first=False, padding_value=0.0, total_length=None)[source]

Pad packed sequence.

This method unpacks the packed sequqnce and pad it, the inverse operation of pack_padded_sequence().

\(T_i\) is the length of the \(i\)-th Variable in the sequences. \(B\) is the batch size equal to the length of the sequences. \(T\) is the max of \(T_i\) for all \(i\). \(*\) is the remaining dimensions including none.

Note

This function must be used the dynamic computation mode.

Example:

import numpy as np
import nnabla as nn
import nnabla.functions as F
import nnabla.utils.rnn as rnn_utils

nn.set_auto_forward(True)

l2v = lambda ldata: nn.Variable.from_numpy_array(np.asarray(ldata))
a = l2v([3, 3])
b = l2v([2, 2, 2])
c = l2v([2, 2, 2])
d = l2v([1, 1, 1, 1])
e = l2v([3, 3])
sequences = [a, b, c, d, e]

packed_sequence = rnn_utils.pack_sequence(sequences, enforce_sorted=False)
print(packed_sequence.data.d)
print(packed_sequence.batch_sizes.d)

padded_sequence, lengths = rnn_utils.pad_packed_sequence(packed_sequence)
print(padded_sequence.d)
print(lengths.d)
Parameters

  • sequence (PackedSequence) – PackedSequence.

  • batch_first (bool) – If False, output is of (\(T\), \(B\), \(*\)) shape, otherwise (\(B\), \(T\), \(*\)).

  • padding_value (float) – Padding value.

  • total_length (int) – If not None, the outputs are padded up to the total_length. If the total_length is less than the max length in the sequences, the error is thrown. This is normally used in the distributed training to align with the longest sequence in a distributed system.

Returns

nnabla.Variable of (\(T\), \(B\), \(*\)) or (\(B\), \(T\), \(*\)) shape