class nbla::DataParallelCommunicatorNccl
-
template<typename T>
class DataParallelCommunicatorNccl : public DataParallelCommunicator<T> Communicator interface which is extended to implement a new Communicator class.
Communicator exchanges gradients parameters or parameters itself.
Public Functions
-
void add_context_and_parameters(const pair<Context, vector<pair<string, VariablePtr>>> &ctx_params)
Adding context and parameters communicated via this class.
- Parameters:
cparams – pair<Context, vector of pair<name, VariablePtr>
-
void remove_context_parameters(const pair<Context, vector<string>> &ctx_keys)
Remove previously registered parameters by keys.
-
void clear_context_parameters()
Clear all parameters.
-
virtual void init()
Initall or initrank, depending multi-threads or multi-processes.
This function MUST be called after all parameters communicated are added by `add_context_and_parameters` method.
-
void sync_all_params()
Sync all parameters added in this communicator based on `Context`.
Coerce to copy all parameters to the device specified by `Context`.
-
vector<string> allowed_array_classes()
Get array classes that are allowed to be specified by Context.
-
void add_context_and_parameters(const pair<Context, vector<pair<string, VariablePtr>>> &ctx_params)