class nbla::DataParallelCommunicatorNccl

template<typename T>
class DataParallelCommunicatorNccl : public DataParallelCommunicator<T>

Communicator interface which is extended to implement a new Communicator class.

Communicator exchanges gradients parameters or parameters itself.

Public Functions

void add_context_and_parameters(const pair<Context, vector<pair<string, VariablePtr>>> &ctx_params)

Adding context and parameters communicated via this class.

Parameters:

cparams – pair<Context, vector of pair<name, VariablePtr>

void remove_context_parameters(const pair<Context, vector<string>> &ctx_keys)

Remove previously registered parameters by keys.

void clear_context_parameters()

Clear all parameters.

virtual void init()

Initall or initrank, depending multi-threads or multi-processes.

This function MUST be called after all parameters communicated are added by `add_context_and_parameters` method.

void sync_all_params()

Sync all parameters added in this communicator based on `Context`.

Coerce to copy all parameters to the device specified by `Context`.

vector<string> allowed_array_classes()

Get array classes that are allowed to be specified by Context.