# Operator support status¶

## Support status importing from ONNX¶

This is a status list of [ONNX operators](https://github.com/onnx/onnx/blob/master/docs/Operators.md) that indicates if each operator can be converted to NNP.

- OK The ONNX operator can map to a NNabla operator.
- Not test means the ONNX operator has not been verified. It might be one of the following cases:
- The ONNX operator hasn’t been checked if it can be converted to NNabla.
- The ONNX operator has been checked if it can be converted to NNabla, but the implementation has not started.
- The solution is not perfect/finished, for example, the operator can map to a combination of NNabla operators.
- Hard to find a solution with existing NNabla operators.

Total 44/108

As the following table, Opset column means the maximal opset version are supported to convert to NNP. In user’s model, if there is any function opset version exceed the maximal opset(as the following table), the importer might fail to convert NNP model due to this function.

Operator | Opset | Status | Description |
---|---|---|---|

Abs | 1,6 | OK | |

Add | 1,6 | OK | broadcast will be converted to a BroadcastTo |

And | 1 | OK | broadcast will be converted to a BroadcastTo |

ArgMax | Unimplemented | Operator does not exist in NNabla | |

ArgMin | Unimplemented | Operator does not exist in NNabla | |

AveragePool | 1 | OK | autopad not supported. pads must have same value for begin and end. |

BatchNormalization | 1,6 | OK | is_test=false not supported (only inference) |

Cast | Unimplemented | Operator does not exist in NNabla(No type information is exposed in NNP) | |

Ceil | Unimplemented | Should map to Ceil | |

Clip | 6 | OK | Converted to Identity, MaximumScalar, MinimumScalar, or both depending on the attribute |

Concat | 1,4 | OK | |

Constant | 1,9 | OK | Converted to an input parameter |

Conv | 1 | OK | auto_pad not supported. pads must have same value for begin and end. |

ConvTranspose | Unimplemented | Should map to Deconvolution? | |

DepthToSpace | Unimplemented | Operator does not exist in NNabla | |

Div | 1,6 | OK | broadcast will be converted to a BroadcastTo |

Dropout | 1,6,7 | OK | mask output will be removed since NNabla does not produce mask output. |

Elu | 1,6 | OK | |

Equal | 1 | OK | broadcast will be converted to a BroadcastTo. Input data type will all be converted to int64 since NNP does not have type information |

Exp | 1,6 | OK | |

Flatten | Unimplemented | Operator does not exist in NNabla | |

Floor | Unimplemented | Should map to Floor | |

GRU | Unimplemented | Operator does not exist in NNabla | |

Gather | Unimplemented | Operator does not exist in NNabla | |

Gemm | 1,6,7,9 | OK | alpha and beta is not supported. Input A and B must be two dimensional, and input C must be one dimensional. transA, transB will be converted to a separate transpose operator |

GlobalAveragePool | 1 | OK | |

GlobalLpPool | Unimplemented | Operator does not exist in NNabla | |

GlobalMaxPool | Unimplemented | Operator does not exist in NNabla | |

Greater | 1 | OK | broadcast will be converted to a BroadcastTo |

HardSigmoid | Unimplemented | Should be able to map to MulScalar+AddScalar+MinimumScalar+ReLU | |

Hardmax | Unimplemented | Operator does not exist in NNabla | |

Identity | 1 | OK | |

InstanceNormalization | Unimplemented | Operator does not exist in NNabla | |

LRN | 1 | OK | Converted to PowScalar+Tranpose+SumPooling+Transpose+MulScalar+AddScalar+PowScalar. Currently only odd size is allowed. |

LSTM | Unimplemented | ||

LeakyRelu | 1,6 | OK | |

Less | 1 | OK | broadcast will be converted to a BroadcastTo |

Log | 1,6 | OK | |

LogSoftmax | 1 | Not test | Converted to Exp+Sum+Log+Sub2. Only works on input shape like N*C*1*1 |

LpNormalization | Unimplemented | Operator does not exist in NNabla | |

LpPool | Unimplemented | Operator does not exist in NNabla | |

MatMul | 1,9 | OK | |

Max | 1,6,8 | OK | Only input of two tensors is currently supported |

MaxPool | 1,8 | OK | auto_pad is not supported. pads must have same value for begin and end. |

MaxRoiPool | Unimplemented | Operator does not exist in NNabla | |

Mean | 1,6,8 | Not test | Operator does not exist in NNabla |

Min | 1,6,8 | OK | Only input of two tensors is currently supported |

Mul | OK | broadcast will be converted to a BroadcastTo | |

Neg | 1,6 | Not test | Converted to MulScalar |

Not | OK | ||

Or | 1,7 | OK | broadcast will be converted to a BroadcastTo |

PRelu | 1,6 | OK | |

Pad | 1,2 | Not test | For NNP to ONNX conversion, input buffer’s dimension is assumed to be 4D if the shape cannot be determined. |

Pow | 1 | OK | broadcast will be converted to a BroadcastTo |

RNN | Unimplemented | Operator does not exist in NNabla | |

RandomNormal | Unimplemented | Should be able to map to Randn | |

RandomNormalLike | Unimplemented | Operator does not exist in NNabla | |

RandomUniform | Unimplemented | Should be able to map to Rand | |

RandomUniformLike | Unimplemented | Operator does not exist in NNabla | |

Reciprocal | 1,6 | Not test | Converted to RDivScalar |

ReduceL1 | Unimplemented | Operator does not exist in NNabla | |

ReduceL2 | Unimplemented | Operator does not exist in NNabla | |

ReduceLogSum | Unimplemented | Operator does not exist in NNabla | |

ReduceLogSumExp | Unimplemented | Operator does not exist in NNabla | |

ReduceMax | 1 | Not test | |

ReduceMean | 1 | OK | |

ReduceMin | 1 | Not test | |

ReduceProd | 1 | Not test | |

ReduceSum | 1 | OK | |

ReduceSumSquare | Unimplemented | Operator does not exist in NNabla | |

Relu | 1,6 | OK | |

Reshape | 1,5 | Not test | Not completedly supported. |

Selu | 1,6 | OK | |

Sigmoid | 1,6 | OK | |

Size | Unimplemented | Operator does not exist in NNabla | |

Slice | Unimplemented | Operator does not exist in NNabla | |

Softmax | 1 | OK | Only works on input shape like N*C*1*1 |

Softplus | 1 | OK | Converted to Exp + AddScalar + Log |

Softsign | 1 | OK | Converted to Abs + AddScalar + Div2 |

SpaceToDepth | Unimplemented | Operator does not exist in NNabla | |

Split | Unimplemented | Operator does not exist in NNabla | |

Sqrt | Unimplemented | Operator does not exist in NNabla | |

Squeeze | Unimplemented | Operator does not exist in NNabla | |

Sub | 1,6 | OK | broadcast will be converted to a BroadcastTo |

Sum | 1,6,8 | OK | Supporting two inputs only |

Tanh | 1,6 | OK | |

Tile | Unimplemented | Operator does not exist in NNabla | |

TopK | Unimplemented | Operator does not exist in NNabla | |

Transpose | 1 | OK | |

Unsqueeze | Unimplemented | Operator does not exist in NNabla | |

Xor | 1 | OK | broadcast will be converted to a BroadcastTo |

experimental ATen | Unimplemented | ||

experimental Affine | Unimplemented | ||

experimental ConstantFill | Unimplemented | ||

experimental Crop | Unimplemented | ||

experimental FC | Unimplemented | ||

experimental GRUUnit | Unimplemented | ||

experimental GivenTensorFill | Unimplemented | ||

experimental If | Unimplemented | ||

experimental ImageScaler | Unimplemented | ||

experimental Loop | Unimplemented | ||

experimental LoopIndexTensor | Unimplemented | ||

experimental MeanVarianceNormalization | Unimplemented | ||

experimental ParametricSoftplus | Unimplemented | ||

experimental Scale | Unimplemented | ||

experimental ScaledTanh | Unimplemented | ||

experimental ThresholdedRelu | Unimplemented | ||

experimental Upsample | Unimplemented |

## Support status exporting to ONNX¶

The column of opset means current operator is mapped with which the opset specifications. For example, Affine() is converted to 3 onnx functions, Reshape@5, Flatten@1, Gemm@6 and so on. Hence, the exporter will generate the onnx model which require the executor at least support opset 6, when the model contains Affine() function.

Total 45/136

### Neural Network Layer¶

Count 4/11

Operator | Opset | Status | Description |
---|---|---|---|

Affine | 1,5,6 | Not test | Implemented by Reshape,Flatten,Gemm |

Convolution | 1 | OK | Implemented by Conv |

DepthwiseConvolution | 1 | Not test | Implemented by Conv |

Deconvolution | 1,6 | Not test | Implemented by ConvTranspose,Add |

DepthwiseDeconvolution | Not test | Not implemented | |

MaxPooling | 1 | OK | Implemented by MaxPool |

AveragePooling | 1 | OK | Implemented by AveragePool |

GlobalAveragePooling | 1 | OK | Implemented by GlobalAveragePool |

SumPooling | 1 | Not test | Implemented by Mul |

Unpooling | Not test | Not implemented | |

Embed | Not test | Not implemented |

### Neural Network Activation Functions¶

Count 8/11

### Normalization¶

Count 1/4

Operator | Opset | Status | Description |
---|---|---|---|

BatchNormalization | 6 | OK | Implemented by InstanceNormalization,BatchNormalization |

MeanSubtraction | Not test | Not implemented | |

ClipGradByValue | Not test | Not implemented | |

ClipGradByNorm | Not test | Not implemented |

### Reduction¶

Count 5/7

Operator | Opset | Status | Description |
---|---|---|---|

Sum | 1 | OK | Implemented by ReduceSum |

Mean | 1 | OK | Implemented by ReduceMean |

Max | 1 | OK | Implemented by ReduceMax |

Min | 1 | OK | Implemented by ReduceMin |

Prod | 1 | OK | Implemented by ReduceProd |

ReduceSum | Not test | Not implemented | |

ReduceMean | Not test | Not implemented |

### Arithmetic¶

Count 8/12

Operator | Opset | Status | Description |
---|---|---|---|

Add2 | 6 | OK | Implemented by Add |

BcAdd2 | Not test | Not implemented | |

Sub2 | 1 | OK | Implemented by Sub |

Mul2 | 1 | OK | Implemented by Mul |

Div2 | 6 | OK | Implemented by Div |

Pow2 | 1 | OK | Implemented by Pow |

AddScalar | 6 | Not test | Implemented by Add |

MulScalar | 1 | OK | Implemented by Mul |

PowScalar | 1 | OK | Implemented by Pow |

RSubScalar | 6 | Not test | Implemented by Sub |

RDivScalar | 6 | OK | Implemented by Div |

RPowScalar | 1 | Not test | Implemented by Pow |

### Logical¶

Count 11/24

Operator | Opset | Status | Description |
---|---|---|---|

Sign | Not test | Not implemented | |

Minimum2 | 6 | OK | Implemented by Min |

Maximum2 | 6 | OK | Implemented by Max |

MinimumScalar | 6 | OK | Implemented by Clip |

MaximumScalar | 6 | OK | Implemented by Clip |

LogicalAnd | 1 | OK | Implemented by And |

LogicalOr | 1 | OK | Implemented by Or |

LogicalXor | 1 | OK | Implemented by Xor |

Equal | 1 | OK | Implemented by Equal |

NotEqual | Not test | Not implemented | |

GreaterEqual | Not test | Not implemented | |

Greater | 1 | OK | Implemented by Greater |

LessEqual | Not test | Not implemented | |

Less | 1 | OK | Implemented by Less |

LogicalAndScalar | Not test | Not implemented | |

LogicalOrScalar | Not test | Not implemented | |

LogicalXorScalar | Not test | Not implemented | |

EqualScalar | Not test | Not implemented | |

NotEqualScalar | Not test | Not implemented | |

GreaterEqualScalar | Not test | Not implemented | |

GreaterScalar | Not test | Not implemented | |

LessEqualScalar | Not test | Not implemented | |

LessScalar | Not test | Not implemented | |

LogicalNot | 1 | OK | Implemented by Not |

### Math¶

Count 5/18

Operator | Opset | Status | Description |
---|---|---|---|

Constant | Not test | Not implemented | |

Abs | 6 | OK | Implemented by Abs |

Exp | 6 | OK | Implemented by Exp |

Log | 6 | OK | Implemented by Log |

Identity | 1 | OK | Implemented by Identity |

BatchMatmul | 1 | OK | Implemented by Matmul |

Round | Not test | Not implemented | |

Sin | Not test | Not implemented | |

Cos | Not test | Not implemented | |

Tan | Not test | Not implemented | |

Sinh | Not test | Not implemented | |

Cosh | Not test | Not implemented | |

ASin | Not test | Not implemented | |

ACos | Not test | Not implemented | |

ATan | Not test | Not implemented | |

ASinh | Not test | Not implemented | |

ACosh | Not test | Not implemented | |

ATanh | Not test | Not implemented |

### Array Manipulation¶

Count 3/13

Operator | Opset | Status | Description |
---|---|---|---|

Concatenate | 4 | OK | Implemented by Concat |

Split | 1,2 | Not test | Implemented by Split,Squeeze |

Stack | 1,4 | Not test | Implemented by Unsqueeze,Concat |

Slice | 1 | Not test | Implemented by Slice |

Pad | 2 | OK | Implemented by Pad |

Transpose | 1 | OK | Implemented by Transpose |

Broadcast | Not test | Not implemented | |

OneHot | 1,5 | Not test | Implemented by Flatten,Gather,Reshape |

Flip | 1 | Not test | Implemented by Gather,Transpose,Identity |

Shift | Not test | Not implemented | |

Reshape | 5 | Not test | Implemented by Reshape |

MatrixDiag | Not test | Not implemented | |

MatrixDiagPart | Not test | Not implemented |

### Stochasticity¶

Count 0/10

Operator | Opset | Status | Description |
---|---|---|---|

Dropout | 6 | NG | Implemented by Dropout |

TopKData | Not test | Not implemented | |

TopKGrad | Not test | Not implemented | |

Rand | Not test | Not implemented | |

Randint | Not test | Not implemented | |

Randn | Not test | Not implemented | |

RandomCrop | Not test | Not implemented | |

RandomFlip | Not test | Not implemented | |

RandomShift | Not test | Not implemented | |

ImageAugmentation | Not test | Not implemented |

### Loss Functions¶

Count 0/9

Operator | Opset | Status | Description |
---|---|---|---|

SigmoidCrossEntropy | Not test | Not implemented | |

BinaryCrossEntropy | Not test | Not implemented | |

SoftmaxCrossEntropy | Not test | Not implemented | |

CategoricalCrossEntropy | Not test | Not implemented | |

SquaredError | Not test | Not implemented | |

AbsoluteError | Not test | Not implemented | |

HuberLoss | Not test | Not implemented | |

EpsilonInsensitiveLoss | Not test | Not implemented | |

KLMultinomial | Not test | Not implemented |

### Quantization Neural Network Layers¶

Count 0/10

Operator | Opset | Status | Description |
---|---|---|---|

BinarySigmoid | 6 | Not test | Implemented by HardSigmoid |

BinaryTanh | Not test | Not implemented | |

BinaryConnectAffine | Not test | Not implemented | |

BinaryConnectConvolution | 1,5 | Not test | Implemented by Conv,Reshape |

BinaryWeightAffine | Not test | Not implemented | |

BinaryWeightConvolution | Not test | Not implemented | |

INQAffine | Not test | Not implemented | |

INQConvolution | Not test | Not implemented | |

FixedPointQuantize | Not test | Not implemented | |

Pow2Quantize | Not test | Not implemented |

### Validation¶

Count 0/3

Operator | Opset | Status | Description |
---|---|---|---|

TopNError | Not test | Not implemented | |

BinaryError | Not test | Not implemented | |

ConfusionMatrix | Not test | Not implemented |

### Unsupported,SpecialUse¶

Count 0/4

Operator | Opset | Status | Description |
---|---|---|---|

VATNoise | Not test | Not implemented | |

Unlink | Not test | Not implemented | |

Sink | Not test | Not implemented | |

NmsDetection2d | Not test | Not implemented |