class nbla::SpectralNorm

template<typename T>
class SpectralNorm : public nbla::BaseFunction<int, int, float, bool, bool>

Spectral Normalization.

\[ W_{sn} = \frac{W}{\sigma(W)}. \]

Inputs:

• N-D array of learnable weights. This is normally network parameter.

• 1-D array of singular vector. When `test == false`, the data region will be updated during forward calculation.

Outputs:

• Spectrally normalized \(W_{sn}\) with the same shape as \(W\).

See also

Takeru Miyato, Toshiki Kataoka, Masanori Koyama, Yuichi Yoshida, “Spectral

Normalization for Generative Adversarial Networks”, International Conference on Learning Representations. 2018.

Template Parameters:

T – Data type for computation.

Param dim:

Output dimension. If the dimension is not 0, then the specified dimension becomes the most-left dimension by transposing.

Param itr:

Number of power iterations.

Param eps:

Epsilon for the normalization. This `eps` is added before taking the sqrt in the norm computation.

Param test:

When in `true`, `u` will not be updated.

Public Functions

inline virtual shared_ptr<Function> copy() const

Copy another instance of Function with the same context.

inline virtual int min_inputs()

Get minimum number of inputs.

This is meant to be used in setup function with in_types which is used to get maximum number of inputs.

inline virtual int min_outputs()

Get minimum number of outputs.

This is meant to be used in setup function with out_types which is used to get max number of outputs.

inline virtual vector<dtypes> in_types()

Get input dtypes.

Last in_type will be used repeatedly if size of in_types is smaller than size of inputs

inline virtual vector<dtypes> out_types()

Get output dtypes.

Last out_type will be used repeatedly if size of out_types is smaller than size of outputs

inline virtual vector<string> allowed_array_classes()

Get array classes that are allowed to be specified by Context.

inline virtual string name()

Get function name in string.

inline virtual bool grad_depends_output_data(int i, int o) const

Dependency flag for checking if in-grad depends on out-data.

Checking if i-th input’ gradient computation requires o-th output’s data or not.

Note

If any of inputs requires an output variable data when computing its gradient, this function must be overridden to return appropriate boolean value. Otherwise, backward computation will be incorrect.

Parameters:

• i – [in] Input variable index.

• o – [in] Output variable index.

inline virtual bool need_setup_recompute(int o) const

A flag for checking if setup_recompute() is needed.

Checking if o-th output’ data requires setup_recompute().

Note

setup_recompute() will skipped during forward execution if none of outputs requires setup_recompute().

Parameters:

o – [in] Output variable index.