# Function-Level Support Status¶

## ONNX Support Status¶

Note: | In this document, the numbers in the header of all tables represent the version of onnx opset. |
---|---|

ONNX Version: | 1.6.0 |

### Import¶

- ✓: onnx specification defined, and supported.
- X: onnx specification defined, but not support yet.
- Empty: Not defined (Support status follows latest).

Total: 93/155

ONNX Operator | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | NNabla Func | Description |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|

Abs | ✓ | ✓ | Abs | ||||||||||

Acos | ✓ | ACos | |||||||||||

Acosh | ✓ | ACosh | |||||||||||

Add | ✓ | ✓ | ✓ | Reshape, Add2 | |||||||||

And | ✓ | ✓ | ✓ | LogicalAnd, Reshape | |||||||||

ArgMax | ✓ | ✓ | X | ✓ | Max | ||||||||

ArgMin | ✓ | ✓ | X | ✓ | Min | ||||||||

Asin | ✓ | ASin | |||||||||||

Asinh | ✓ | ASinh | |||||||||||

Atan | ✓ | ATan | |||||||||||

Atanh | ✓ | ATanh | |||||||||||

AveragePool | ✓ | ✓ | ✓ | X | X | Pad, AveragePooling | Not all features are verified, since some features are not supported by caffe2. Those features can be verified by ONNXRuntime when opset > 6. Some feature is not supported by Nnabla such as Pad’s edge mode. if opset >= 10, the ceil_mode is not supported. | ||||||

BatchNormalization | X | X | X | ✓ | BatchNormalization | ||||||||

BitShift | X | Not yet implemented. | |||||||||||

Cast | ✓ | ✓ | X | Abs, Log | |||||||||

Ceil | ✓ | ✓ | Ceil | ||||||||||

Clip | ✓ | ✓ | ✓ | MinimumScalar, MaximumScalar, Identity | |||||||||

Compress | X | X | Not yet implemented. | ||||||||||

Concat | ✓ | ✓ | ✓ | X | Concatenate | ||||||||

ConcatFromSequence | X | Not yet implemented. | |||||||||||

Constant | ✓ | ✓ | X | X | Identity | ||||||||

ConstantOfShape | ✓ | Constant | |||||||||||

Conv | ✓ | ✓ | X | Convolution | |||||||||

ConvInteger | X | Not yet implemented. | |||||||||||

ConvTranspose | ✓ | ✓ | X | Deconvolution, Pad | |||||||||

Cos | ✓ | Cos | |||||||||||

Cosh | ✓ | Cosh | |||||||||||

CumSum | X | Not yet implemented. | |||||||||||

DepthToSpace | ✓ | ✓ | ✓ | Reshape, Transpose | |||||||||

DequantizeLinear | X | Not yet implemented. | |||||||||||

Det | X | Not yet implemented. | |||||||||||

Div | ✓ | ✓ | ✓ | Div2, Reshape | |||||||||

Dropout | X | X | ✓ | X | Identity | ||||||||

DynamicQuantizeLinear | X | Not yet implemented. | |||||||||||

Elu | ✓ | ✓ | ELU | ||||||||||

Equal | ✓ | ✓ | ✓ | X | Reshape, Equal | ||||||||

Erf | X | Not yet implemented. | |||||||||||

Exp | ✓ | ✓ | Exp | ||||||||||

Expand | ✓ | ✓ | Broadcast, Reshape | ||||||||||

EyeLike | X | Not yet implemented. | |||||||||||

Flatten | ✓ | ✓ | ✓ | ✓ | Reshape | ||||||||

Floor | ✓ | ✓ | Floor | ||||||||||

GRU | X | X | X | Not yet implemented. | |||||||||

Gather | ✓ | ✓ | ✓ | Slice, Concatenate | |||||||||

GatherElements | X | Not yet implemented. | |||||||||||

GatherND | X | Not yet implemented. | |||||||||||

Gemm | ✓ | ✓ | ✓ | ✓ | ✓ | BatchMatmul, Reshape, Add2, MulScalar | |||||||

GlobalAveragePool | ✓ | ✓ | GlobalAveragePooling | ||||||||||

GlobalLpPool | X | X | Not yet implemented. | ||||||||||

GlobalMaxPool | X | Not yet implemented. | |||||||||||

Greater | ✓ | ✓ | ✓ | ✓ | Reshape, Greater | ||||||||

HardSigmoid | ✓ | ✓ | MaximumScalar, MulScalar, AddScalar, HardSigmoid, MinimumScalar | ||||||||||

Hardmax | ✓ | ✓ | ✓ | Reshape, OneHot, Max | |||||||||

Identity | ✓ | ✓ | Identity | ||||||||||

If | X | Not yet implemented. | |||||||||||

InstanceNormalization | ✓ | ✓ | Split, Reshape, BatchNormalization, Concatenate | ||||||||||

IsInf | ✓ | IsInf | |||||||||||

IsNaN | ✓ | IsNaN | |||||||||||

LRN | ✓ | ✓ | Transpose, MulScalar, PowScalar, AddScalar, Div2, SumPooling | ||||||||||

LSTM | X | X | Not yet implemented. | ||||||||||

LeakyRelu | ✓ | ✓ | LeakyReLU | ||||||||||

Less | ✓ | ✓ | ✓ | ✓ | Reshape, Less | ||||||||

Log | ✓ | ✓ | Log | ||||||||||

LogSoftmax | ✓ | ✓ | ✓ | Reshape, Max, Exp, Log, Sub2, Sum, Add2 | |||||||||

Loop | X | X | Not yet implemented. | ||||||||||

LpNormalization | X | Not yet implemented. | |||||||||||

LpPool | X | X | X | Not yet implemented. | |||||||||

MatMul | ✓ | ✓ | ✓ | BatchMatmul | |||||||||

MatMulInteger | X | Not yet implemented. | |||||||||||

Max | ✓ | ✓ | ✓ | ✓ | Maximum2 | ||||||||

MaxPool | ✓ | ✓ | X | X | X | Pad, MaxPooling | Not all features are verified, since some features are not supported by caffe2. Those features can be verified by ONNXRuntime. if opset >= 10, the ceil_mode is not supported, dilations is not equal to 1 is not supported. | ||||||

MaxRoiPool | X | Not yet implemented. | |||||||||||

MaxUnpool | X | X | Not yet implemented. | ||||||||||

Mean | ✓ | ✓ | ✓ | ✓ | Broadcast, Stack, Mean | ||||||||

MeanVarianceNormalization | X | Not yet implemented. | |||||||||||

Min | ✓ | ✓ | ✓ | ✓ | Minimum2 | ||||||||

Mod | X | Not yet implemented. | |||||||||||

Mul | ✓ | ✓ | ✓ | Reshape, Mul2 | |||||||||

Multinomial | X | Not yet implemented. | |||||||||||

Neg | ✓ | ✓ | MulScalar | ||||||||||

NonMaxSuppression | X | X | Not yet implemented. | ||||||||||

NonZero | X | Not yet implemented. | |||||||||||

Not | ✓ | ✓ | LogicalNot | ||||||||||

OneHot | X | X | Not yet implemented. | ||||||||||

Or | ✓ | ✓ | ✓ | Reshape, LogicalOr | |||||||||

PRelu | ✓ | ✓ | X | X | PReLU | ||||||||

Pad | ✓ | ✓ | ✓ | ✓ | Pad | Onnx required to support “edge” mode, while nnabla does not support it. | |||||||

Pow | ✓ | ✓ | ✓ | Reshape, Pow2 | |||||||||

QLinearConv | X | Not yet implemented. | |||||||||||

QLinearMatMul | X | Not yet implemented. | |||||||||||

QuantizeLinear | X | Not yet implemented. | |||||||||||

RNN | X | X | Not yet implemented. | ||||||||||

RandomNormal | X | Not yet implemented. | |||||||||||

RandomNormalLike | X | Not yet implemented. | |||||||||||

RandomUniform | X | Not yet implemented. | |||||||||||

RandomUniformLike | X | Not yet implemented. | |||||||||||

Range | X | Not yet implemented. | |||||||||||

Reciprocal | ✓ | ✓ | RDivScalar | ||||||||||

ReduceL1 | X | X | Not yet implemented. | ||||||||||

ReduceL2 | X | X | Not yet implemented. | ||||||||||

ReduceLogSum | X | X | Not yet implemented. | ||||||||||

ReduceLogSumExp | X | X | Not yet implemented. | ||||||||||

ReduceMax | ✓ | ✓ | ✓ | Max | |||||||||

ReduceMean | ✓ | ✓ | ✓ | Mean | |||||||||

ReduceMin | ✓ | ✓ | ✓ | Min | |||||||||

ReduceProd | ✓ | ✓ | ✓ | Prod | |||||||||

ReduceSum | ✓ | ✓ | ✓ | Sum | |||||||||

ReduceSumSquare | ✓ | ✓ | ✓ | PowScalar, Sum | |||||||||

Relu | ✓ | ✓ | ReLU | ||||||||||

Reshape | ✓ | ✓ | ✓ | Reshape | |||||||||

Resize | X | X | Not yet implemented. | ||||||||||

ReverseSequence | X | Not yet implemented. | |||||||||||

RoiAlign | X | Not yet implemented. | |||||||||||

Round | ✓ | Round | |||||||||||

Scan | X | X | X | Not yet implemented. | |||||||||

Scatter | X | X | Not yet implemented. | ||||||||||

ScatterElements | X | Not yet implemented. | |||||||||||

ScatterND | X | Not yet implemented. | |||||||||||

Selu | ✓ | ✓ | SELU | ||||||||||

SequenceAt | X | Not yet implemented. | |||||||||||

SequenceConstruct | X | Not yet implemented. | |||||||||||

SequenceErase | X | Not yet implemented. | |||||||||||

SequenceInsert | X | Not yet implemented. | |||||||||||

SequenceLength | X | Not yet implemented. | |||||||||||

Shape | X | Not yet implemented. | |||||||||||

Shrink | X | Not yet implemented. | |||||||||||

Sigmoid | ✓ | ✓ | Sigmoid | ||||||||||

Sign | ✓ | Sign | |||||||||||

Sin | ✓ | Sin | |||||||||||

Sinh | ✓ | Sinh | |||||||||||

Size | X | Not yet implemented. | |||||||||||

Slice | ✓ | ✓ | ✓ | X | Slice | ||||||||

Softmax | ✓ | ✓ | ✓ | Reshape, Exp, Div2, Sub2, Sum, Max | |||||||||

Softplus | ✓ | ✓ | SoftPlus | ||||||||||

Softsign | ✓ | ✓ | SoftSign | ||||||||||

SpaceToDepth | ✓ | ✓ | Reshape, Transpose | ||||||||||

Split | ✓ | ✓ | ✓ | ✓ | Stack, Split | ||||||||

SplitToSequence | X | Not yet implemented. | |||||||||||

Sqrt | ✓ | ✓ | PowScalar | ||||||||||

Squeeze | ✓ | ✓ | ✓ | Reshape | |||||||||

StringNormalizer | X | Not yet implemented. | |||||||||||

Sub | ✓ | ✓ | ✓ | Sub2, Reshape | |||||||||

Sum | ✓ | ✓ | ✓ | ✓ | Add2 | ||||||||

Tan | ✓ | Tan | |||||||||||

Tanh | ✓ | ✓ | Tanh | ||||||||||

TfIdfVectorizer | X | Not yet implemented. | |||||||||||

ThresholdedRelu | ✓ | GreaterScalar, Where, Constant | |||||||||||

Tile | ✓ | ✓ | Tile | ||||||||||

TopK | X | X | X | Not yet implemented. | |||||||||

Transpose | ✓ | ✓ | Transpose | ||||||||||

Unique | X | Not yet implemented. | |||||||||||

Unsqueeze | ✓ | ✓ | ✓ | Reshape | |||||||||

Upsample | ✓ | ✓ | ✓ | ✓ | X | Unpooling | |||||||

Where | ✓ | Where | |||||||||||

Xor | ✓ | ✓ | ✓ | Reshape, LogicalXor |

### Export¶

- ✓: Support to export this opset.
- △: Partially support to export this opset (e.g. some cases cannot be supported, or not completely tested).
- X: Supported, but test failed.
- Empty: Not support corresponding opset version.

Total: 119/173

#### Neural Network Layer¶

Count 11/14

NNabla Function 6 7 9 10 11 ONNX Op Description Affine ✓ ✓ ✓ ✓ ✓ Gemm, Reshape RNN Not yet implemented. LSTM Not yet implemented. GRU Not yet implemented. Convolution ✓ ✓ ✓ ✓ ✓ Reshape, Conv DepthwiseConvolution ✓ ✓ ✓ ✓ ✓ Reshape, Conv Deconvolution △ △ △ △ △ ConvTranspose, Reshape Caffe2 and onnxruntime do not support dilations != 1. DepthwiseDeconvolution △ △ △ △ △ ConvTranspose, Reshape Caffe2 and onnxruntime do not support dilations != 1. MaxPooling ✓ ✓ ✓ ✓ X Reshape, Pad, MaxPool AveragePooling △ △ △ △ X Reshape, Pad, AveragePool Currently only supports the cases where both ignore_border and including_pad are True. GlobalAveragePooling ✓ ✓ ✓ ✓ ✓ GlobalAveragePool SumPooling X ✓ ✓ ✓ X Reshape, Pad, Constant, Mul, AveragePool Unpooling △ ✓ ✓ ✓ ✓ Resize The kernel only supports 2d on opset 6. Embed ✓ ✓ ✓ ✓ ✓ Gather

#### Neural Network Activation Functions¶

Count 21/21

NNabla Function 6 7 9 10 11 ONNX Op Description Sigmoid ✓ ✓ ✓ ✓ ✓ Sigmoid Swish ✓ ✓ ✓ ✓ ✓ Mul, Sigmoid Tanh ✓ ✓ ✓ ✓ ✓ Tanh ReLU ✓ ✓ ✓ ✓ ✓ Relu LeakyReLU ✓ ✓ ✓ ✓ ✓ LeakyRelu Softmax △ ✓ ✓ ✓ ✓ Exp, ReduceMax, Sub, ReduceSum, Div ONNX Add, Sub operator does not support multidirectional broadcasting on opset 6. LogSoftmax △ ✓ ✓ ✓ ✓ Exp, ReduceMax, Sub, Log, ReduceSum ELU ✓ ✓ ✓ ✓ ✓ Elu SELU ✓ ✓ ✓ ✓ ✓ Selu CReLU ✓ ✓ ✓ ✓ ✓ Concat, Neg, Relu CELU ✓ ✓ ✓ ✓ ✓ Elu, Concat, Neg PReLU ✓ ✓ ✓ ✓ ✓ Reshape, PRelu GELU ✓ ✓ ✓ ✓ ✓ Sqrt, Constant, Pow, Tanh, Mul, Add, Div ReLU6 ✓ ✓ ✓ ✓ ✓ Min, Relu, Constant HardSigmoid ✓ ✓ ✓ ✓ ✓ HardSigmoid HardTanh ✓ ✓ ✓ ✓ ✓ Neg, Min, Max, Constant LogSigmoid ✓ ✓ ✓ ✓ ✓ Sigmoid, Log SoftPlus ✓ ✓ ✓ ✓ ✓ Softplus SoftSign ✓ ✓ ✓ ✓ ✓ Softsign TanhShrink ✓ ✓ ✓ ✓ ✓ Sub, Tanh Sinc X X ✓ ✓ ✓ Constant, Where, Sin, Equal, Div

#### Normalization¶

Count 1/6

NNabla Function 6 7 9 10 11 ONNX Op Description FusedBatchNormalization X X X X X Relu, Constant, ReduceMean, BatchNormalization, Mul, Sub, ReduceSum, Div Not yet implemented. BatchNormalization ✓ ✓ ✓ ✓ ✓ Reshape, Constant, ReduceMean, BatchNormalization, Mul, Sub, ReduceSum, Div In inferring stage, caffe2 mistmatch onnx 1.4.x’s implementation, “in-place” feature cannot be applied. SyncBatchNormalization Not yet implemented. MeanSubtraction Not yet implemented. ClipGradByValue Not yet implemented. ClipGradByNorm Not yet implemented.

#### Reduction¶

Count 5/7

NNabla Function 6 7 9 10 11 ONNX Op Description Sum ✓ ✓ ✓ ✓ ✓ ReduceSum Mean ✓ ✓ ✓ ✓ ✓ ReduceMean Max ✓ ✓ ✓ ✓ ✓ ReduceMax Min ✓ ✓ ✓ ✓ ✓ ReduceMin Prod ✓ ✓ ✓ ✓ ✓ ReduceProd ReduceSum Not yet implemented. ReduceMean Not yet implemented.

#### Arithmetic¶

Count 11/12

NNabla Function 6 7 9 10 11 ONNX Op Description Add2 △ ✓ ✓ ✓ ✓ Add ONNX Add operator does not support multidirectional broadcasting on opset 6. BcAdd2 Not yet implemented. Sub2 △ ✓ ✓ ✓ ✓ Sub ONNX Sub operator does not support multidirectional broadcasting on opset 6. Mul2 △ ✓ ✓ ✓ ✓ Mul ONNX Mul operator does not support multidirectional broadcasting on opset 6. Div2 △ ✓ ✓ ✓ ✓ Div ONNX Div operator does not support multidirectional broadcasting on opset 6. Pow2 △ ✓ ✓ ✓ ✓ Pow ONNX Pow operator does not support multidirectional broadcasting on opset 6. AddScalar ✓ ✓ ✓ ✓ ✓ Add, Constant MulScalar ✓ ✓ ✓ ✓ ✓ Mul, Constant PowScalar ✓ ✓ ✓ ✓ ✓ Pow, Constant RSubScalar ✓ ✓ ✓ ✓ ✓ Sub, Constant RDivScalar ✓ ✓ ✓ ✓ ✓ Constant, Div RPowScalar ✓ ✓ ✓ ✓ ✓ Pow, Constant

#### Logical¶

Count 29/29

NNabla Function 6 7 9 10 11 ONNX Op Description Sign X X ✓ ✓ ✓ Sign Minimum2 △ ✓ ✓ ✓ ✓ Min, Add, Constant ONNX Add operator does not support multidirectional broadcasting on opset 6. Maximum2 △ ✓ ✓ ✓ ✓ Add, Max, Constant ONNX Add operator does not support multidirectional broadcasting on opset 6. MinimumScalar ✓ ✓ ✓ ✓ ✓ Min, Add, Constant MaximumScalar ✓ ✓ ✓ ✓ ✓ Add, Max, Constant LogicalAnd ✓ ✓ ✓ ✓ ✓ And LogicalOr ✓ ✓ ✓ ✓ ✓ Or LogicalXor ✓ ✓ ✓ ✓ ✓ Xor Equal ✓ ✓ ✓ ✓ ✓ Equal NotEqual ✓ ✓ ✓ ✓ ✓ Equal, Not GreaterEqual ✓ ✓ ✓ ✓ ✓ Less, Not Greater ✓ ✓ ✓ ✓ ✓ Greater LessEqual ✓ ✓ ✓ ✓ ✓ Greater, Not Less ✓ ✓ ✓ ✓ ✓ Less LogicalAndScalar ✓ ✓ ✓ ✓ ✓ And, Constant LogicalOrScalar ✓ ✓ ✓ ✓ ✓ Or, Constant LogicalXorScalar ✓ ✓ ✓ ✓ ✓ Xor, Constant EqualScalar ✓ ✓ ✓ ✓ ✓ Equal, Constant NotEqualScalar ✓ ✓ ✓ ✓ ✓ Equal, Not, Constant GreaterEqualScalar ✓ ✓ ✓ ✓ ✓ Less, Not, Constant GreaterScalar ✓ ✓ ✓ ✓ ✓ Greater, Constant LessEqualScalar ✓ ✓ ✓ ✓ ✓ Greater, Not, Constant LessScalar ✓ ✓ ✓ ✓ ✓ Less, Constant LogicalNot ✓ ✓ ✓ ✓ ✓ Not IsNaN X X ✓ ✓ ✓ IsNaN IsInf X X X ✓ ✓ IsInf ResetNaN X X ✓ ✓ ✓ Where, IsNaN, Constant ResetInf X X X ✓ ✓ Where, IsInf, Constant Where X X ✓ ✓ ✓ Where

#### Math¶

Count 22/22

NNabla Function 6 7 9 10 11 ONNX Op Description Constant ✓ ✓ ✓ ✓ ✓ Identity, Constant Arange ✓ ✓ ✓ ✓ ✓ Identity, Constant Abs ✓ ✓ ✓ ✓ ✓ Abs Exp ✓ ✓ ✓ ✓ ✓ Exp Log ✓ ✓ ✓ ✓ ✓ Log Identity ✓ ✓ ✓ ✓ ✓ Identity BatchMatmul ✓ ✓ ✓ ✓ ✓ Reshape, MatMul, Transpose Round X X X X ✓ Round Ceil ✓ ✓ ✓ ✓ ✓ Ceil Floor ✓ ✓ ✓ ✓ ✓ Floor Sin X ✓ ✓ ✓ ✓ Sin Cos X ✓ ✓ ✓ ✓ Cos Tan X ✓ ✓ ✓ ✓ Tan Sinh X X ✓ ✓ ✓ Sinh Cosh X X ✓ ✓ ✓ Cosh ASin X ✓ ✓ ✓ ✓ Asin ACos X ✓ ✓ ✓ ✓ Acos ATan X ✓ ✓ ✓ ✓ Atan ATan2 X ✓ ✓ ✓ ✓ Div, Atan ASinh X X ✓ ✓ ✓ Asinh ACosh X X ✓ ✓ ✓ Acosh ATanh X X ✓ ✓ ✓ Atanh

#### Array Manipulation¶

Count 12/19

NNabla Function 6 7 9 10 11 ONNX Op Description Concatenate ✓ ✓ ✓ ✓ ✓ Concat Split ✓ ✓ ✓ ✓ ✓ Split, Squeeze Stack ✓ ✓ ✓ ✓ ✓ Concat, Unsqueeze Slice △ △ △ △ △ Slice, Constant ONNX slice cannot support step != 1 on opset < 10. Pad △ △ △ △ △ Pad, Constant When the mode of the pad is reflect, if the size of the pad exceeds the input size, caffe2 and onnxruntime cannot handle it. Transpose ✓ ✓ ✓ ✓ ✓ Transpose Broadcast X X ✓ ✓ ✓ BroadcastTo ✓ ✓ ✓ ✓ ✓ Tile ✓ ✓ ✓ ✓ △ Reshape, Tile, Constant OneHot ✓ ✓ ✓ ✓ ✓ Gather, Reshape, Flatten Flip ✓ ✓ ✓ ✓ ✓ Gather, Transpose, Identity Shift Not yet implemented. Sort Not yet implemented. Reshape ✓ ✓ ✓ ✓ ✓ Reshape, Constant MatrixDiag Not yet implemented. MatrixDiagPart Not yet implemented. Assign Not yet implemented. GatherNd Not yet implemented. ScatterNd Not yet implemented.

#### Signal Processing¶

Count 1/3

NNabla Function 6 7 9 10 11 ONNX Op Description Interpolate X X X X △ Reshape, Resize FFT Not yet implemented. IFFT Not yet implemented.

#### Stochasticity¶

Count 0/11

NNabla Function 6 7 9 10 11 ONNX Op Description Dropout X X X X X Dropout The Dropout in nnabla has no test mode and contains random parameters, so the test result is not the same as onnx. TopKData Not yet implemented. TopKGrad Not yet implemented. Rand Not yet implemented. Randint Not yet implemented. Randn Not yet implemented. RandomChoice Not yet implemented. RandomCrop Not yet implemented. RandomFlip Not yet implemented. RandomShift Not yet implemented. ImageAugmentation Not yet implemented.

#### Loss Functions¶

Count 0/9

NNabla Function 6 7 9 10 11 ONNX Op Description SigmoidCrossEntropy Not yet implemented. BinaryCrossEntropy Not yet implemented. SoftmaxCrossEntropy Not yet implemented. CategoricalCrossEntropy Not yet implemented. SquaredError Not yet implemented. AbsoluteError Not yet implemented. HuberLoss Not yet implemented. EpsilonInsensitiveLoss Not yet implemented. KLMultinomial Not yet implemented.

#### Quantization Neural Network Layers¶

Count 6/12

NNabla Function 6 7 9 10 11 ONNX Op Description BinarySigmoid X X ✓ ✓ ✓ Where, Greater, Constant BinaryTanh X X ✓ ✓ ✓ Where, Greater, Constant BinaryConnectAffine ✓ ✓ ✓ ✓ ✓ Gemm, Reshape BinaryConnectConvolution ✓ ✓ ✓ ✓ ✓ Reshape, Conv BinaryWeightAffine ✓ ✓ ✓ ✓ ✓ Mul, Reshape, Add, MatMul BinaryWeightConvolution ✓ ✓ ✓ ✓ ✓ Mul, Reshape, Conv, Add INQAffine Not yet implemented. INQConvolution Not yet implemented. FixedPointQuantize Not yet implemented. MinMaxQuantize Not yet implemented. Pow2Quantize Not yet implemented. Prune Not yet implemented.

#### Validation¶

Count 0/3

NNabla Function 6 7 9 10 11 ONNX Op Description TopNError Not yet implemented. BinaryError Not yet implemented. ConfusionMatrix Not yet implemented.

#### Unsupported, Special Use¶

Count 0/5

NNabla Function 6 7 9 10 11 ONNX Op Description VATNoise Not yet implemented. Unlink Not yet implemented. Sink Not yet implemented. NmsDetection2d Not yet implemented. MaxPoolingBackward Not yet implemented.

## Tensorflow Support Status¶

Note: | In this document, the numbers in the header of all tables represent the version of onnx opset. |
---|

### Import¶

- ✓: Supported
- △: Partially supported
- X: Supported, but test failed.
- Empty: Not support yet.

Total: 86/120

Tensorflow Function | Status | NNabla Func | Description |
---|---|---|---|

Abs | ✓ | Abs | |

Acos | ✓ | ACos | |

Acosh | ✓ | ACosh | |

Add | ✓ | Add2 | |

AddN | ✓ | Add2 | |

All | Not yet implemented. | ||

Any | Not yet implemented. | ||

ArgMax | ✓ | Max | |

ArgMin | ✓ | Min | |

Asin | ✓ | ASin | |

Asinh | ✓ | ASinh | |

Atan | ✓ | ATan | |

Atan2 | Not yet implemented. | ||

Atanh | ✓ | ATanh | |

AvgPool | △ | Pad, Transpose, AveragePooling | Some feature is not supported by Nnabla such as Pad’s edge mode. |

AvgPool3D | Not yet implemented. | ||

BatchMatMul | ✓ | BatchMatmul, Transpose | |

BiasAdd | ✓ | Reshape, Add2 | |

Cast | Not yet implemented. | ||

Ceil | ✓ | Ceil | |

ConcatV2 | ✓ | Concatenate | |

Const | ✓ | Add2 | |

Conv2D | △ | Pad, Convolution, Transpose | Tensorflow require GPU to perform related test cases. This issue is recorded only for memo. |

Conv2DBackpropFilter | Not yet implemented. | ||

Conv2DBackpropInput | △ | Deconvolution, Transpose | Tensorflow require GPU to perform related test cases. This issue is recorded only for memo. |

Conv3D | Not yet implemented. | ||

Conv3DBackpropFilterV2 | Not yet implemented. | ||

Conv3DBackpropInputV2 | Not yet implemented. | ||

Cos | ✓ | Cos | |

Cosh | ✓ | Cosh | |

DepthToSpace | △ | Reshape, Transpose | Tensorflow require GPU to perform related test cases. This issue is recorded only for memo. |

DepthwiseConv2dNative | Not yet implemented. | ||

DepthwiseConv2dNativeBackpropFilter | Not yet implemented. | ||

DepthwiseConv2dNativeBackpropInput | Not yet implemented. | ||

Div | ✓ | Div2 | |

Elu | ✓ | ELU | |

Equal | ✓ | Equal | |

Erf | Not yet implemented. | ||

Erfc | Not yet implemented. | ||

Exp | ✓ | Exp | |

ExpandDims | ✓ | Reshape | |

Fill | Not yet implemented. | ||

Flatten | ✓ | Reshape | |

Floor | ✓ | Floor | |

FloorDiv | ✓ | Div2, Floor | |

FloorMod | ✓ | Div2, Sub2, Mul2, Floor | |

FusedBatchNorm | △ | BatchNormalization, Transpose | It did not pass testing for training mode. |

GatherNd | Not yet implemented. | ||

GatherV2 | Not yet implemented. | ||

Greater | ✓ | Greater | |

GreaterEqual | ✓ | Less, LogicalNot | |

Identity | ✓ | Identity | |

IsInf | Not yet implemented. | ||

IsNan | ✓ | IsNaN | |

LeakyRelu | ✓ | LeakyReLU | |

Less | ✓ | Less | |

LessEqual | ✓ | Greater, LogicalNot | |

Log | ✓ | Log | |

LogSoftmax | Not yet implemented. | ||

LogicalAnd | ✓ | LogicalAnd | |

LogicalNot | ✓ | LogicalNot | |

LogicalOr | ✓ | LogicalOr | |

LogicalXor | ✓ | LogicalAnd, LogicalOr, LogicalNot | |

MatrixBandPart | Not yet implemented. | ||

Max | ✓ | Max | |

MaxPool | ✓ | Transpose, Pad, MaxPooling | |

MaxPool3D | Not yet implemented. | ||

MaxPoolWithArgmax | Not yet implemented. | ||

Maximum | ✓ | Maximum2 | |

Mean | ✓ | Mean | |

Min | ✓ | Min | |

Minimum | ✓ | Minimum2 | |

Mul | ✓ | Mul2 | |

Neg | ✓ | MulScalar | |

NotEqual | ✓ | Equal, LogicalNot | |

OneHot | Not yet implemented. | ||

Pack | ✓ | Reshape, Concatenate | |

Pad | ✓ | Pad | |

Pow | ✓ | Pow2 | |

Prod | ✓ | Prod | |

RandomShuffle | Not yet implemented. | ||

RandomStandardNormal | Not yet implemented. | ||

RandomUniform | Not yet implemented. | ||

RealDiv | ✓ | Div2 | |

Reciprocal | ✓ | RDivScalar | |

Relu | ✓ | ReLU | |

Relu6 | ✓ | MinimumScalar, MaximumScalar | |

Reshape | △ | Reshape | Some test cases failed for some nnabla’s implementation limitation (e.g. -1 is regarded as batch_size). |

ReverseSequence | Not yet implemented. | ||

Rsqrt | ✓ | PowScalar, RDivScalar | |

Select | Not yet implemented. | ||

Selu | ✓ | SELU | |

Shape | Not yet implemented. | ||

Sigmoid | ✓ | Sigmoid | |

Sign | ✓ | Sign | |

Sin | ✓ | Sin | |

Sinh | ✓ | Sinh | |

Size | Not yet implemented. | ||

Slice | ✓ | Slice | |

Softmax | Not yet implemented. | ||

Softplus | ✓ | SoftPlus | |

Softsign | ✓ | SoftSign | |

SpaceToDepth | △ | Reshape, Transpose | Tensorflow require GPU to perform related test cases. This issue is recorded only for memo. |

SplitV | ✓ | Stack, Split | |

Sqrt | ✓ | PowScalar | |

Square | ✓ | Mul2 | |

SquaredDifference | ✓ | Sub2, Mul2 | |

Squeeze | ✓ | Reshape | |

StopGradient | ✓ | Identity | |

StridedSlice | ✓ | Slice | |

Sub | ✓ | Sub2 | |

Sum | ✓ | Sum | |

Tan | ✓ | Tan | |

Tanh | ✓ | Tanh | |

Tile | ✓ | Tile | |

TopKV2 | Not yet implemented. | ||

Transpose | ✓ | Transpose | |

TruncateDiv | Not yet implemented. | ||

TruncateMod | Not yet implemented. | ||

Unpack | ✓ | Reshape, Stack, Split, Concatenate |

### Export¶

- ✓: Supported
- △: Partially supported
- X: Supported, but test failed.
- Empty: Not support yet.

Total: 115/173

#### Neural Network Layer¶

Count 11/14

NNabla Function Status TF Op Description Affine ✓ Reshape, Placeholder, MatMul, Const, Mul, Add RNN Not yet implemented. LSTM Not yet implemented. GRU Not yet implemented. Convolution △ Reshape, SpaceToBatchND, Pad, Placeholder, Transpose, Const, Add, Split, Identity, ConcatV2, Conv2D, BatchToSpaceND The cases `dilations`

and`strides`

larger than 1 are not supported by tensorflow.DepthwiseConvolution △ Conv2D, Reshape, SpaceToBatchND, Pad, Placeholder, Transpose, Const, Split, ConcatV2, Add, BatchToSpaceND The cases `dilations`

and`strides`

larger than 1 are not supported by tensorflow.Deconvolution △ Reshape, Slice, Placeholder, Transpose, Const, Split, Identity, ConcatV2, Conv2DBackpropInput, Add The cases `dilations`

larger than 1 are not supported by tensorflow.DepthwiseDeconvolution △ Reshape, Slice, Placeholder, Transpose, Const, Split, ConcatV2, Conv2DBackpropInput, Add The cases `dilations`

larger than 1 are not supported by tensorflow.MaxPooling ✓ Reshape, PadV2, Placeholder, Transpose, Const, MaxPool, MaxPool3D AveragePooling △ AvgPool3D, Reshape, Pad, Placeholder, Transpose, Const, AvgPool Currently only supports the cases both ignore_border and including_pad are True. GlobalAveragePooling ✓ Const, Pack, Sub, SplitV, Range, Mean SumPooling ✓ AvgPool3D, Reshape, Pad, Placeholder, Transpose, Const, Mul, AvgPool Unpooling △ Reshape, Placeholder, Transpose, ResizeNearestNeighbor, Const, LogicalAnd, Merge, Switch, Identity, Assert, Mul, Equal, StridedSlice, NoOp, Cast The kernel only supports 2d. Embed ✓ GatherV2, Placeholder, Const

#### Neural Network Activation Functions¶

Count 21/21

NNabla Function Status TF Op Description Sigmoid ✓ Sigmoid, Placeholder Swish ✓ Mul, Sigmoid, Placeholder Tanh ✓ Placeholder, Tanh ReLU ✓ Relu, Placeholder LeakyReLU ✓ LeakyRelu, Placeholder Softmax ✓ Placeholder, Const, Exp, Sub, RealDiv, Sum, Max LogSoftmax ✓ Placeholder, Const, Exp, Sub, Log, Sum, Max ELU ✓ Less, Placeholder, Const, Elu, Exp, Mul, GreaterEqual, Sub, Cast, Add SELU ✓ Minimum, Placeholder, Const, Exp, Mul, Sub, Maximum, Add CReLU ✓ Relu, Placeholder, Const, ConcatV2, Neg CELU ✓ Less, Placeholder, Const, Elu, Exp, ConcatV2, Mul, GreaterEqual, Sub, Neg, Cast, Add PReLU ✓ Reshape, Relu, Placeholder, Const, Mul, Sub, Abs, Add GELU ✓ Sqrt, Placeholder, Const, Pow, Tanh, Mul, RealDiv, Add ReLU6 ✓ Relu, Placeholder, Const, Pack, Min HardSigmoid ✓ Minimum, Placeholder, Const, Mul, Maximum, Add HardTanh ✓ Placeholder, Const, Pack, Neg, Min, Max LogSigmoid ✓ Sigmoid, Log, Placeholder SoftPlus ✓ Softplus, Placeholder SoftSign ✓ Placeholder, Softsign TanhShrink ✓ Sub, Placeholder, Tanh Sinc ✓ Placeholder, Select, Const, Sin, Equal, RealDiv

#### Normalization¶

Count 1/6

NNabla Function Status TF Op Description FusedBatchNormalization X Reshape, Relu, Placeholder, Const, Mul, Rsqrt, Sub, Mean, RealDiv, Sum, Add Not yet implemented. BatchNormalization ✓ Reshape, Placeholder, Const, Mul, Rsqrt, Sub, Mean, RealDiv, Sum, Add In inferring stage, caffe2 mistmatch onnx 1.4.x’s implementation, “in-place” feature cannot be applied. SyncBatchNormalization Not yet implemented. MeanSubtraction Not yet implemented. ClipGradByValue Not yet implemented. ClipGradByNorm Not yet implemented.

#### Reduction¶

Count 5/7

NNabla Function Status TF Op Description Sum ✓ Sum, Placeholder, Const Mean ✓ Mean, Placeholder, Const Max ✓ Placeholder, Max, Const Min ✓ Min, Placeholder, Const Prod ✓ Prod, Placeholder, Const ReduceSum Not yet implemented. ReduceMean Not yet implemented.

#### Arithmetic¶

Count 11/12

NNabla Function Status TF Op Description Add2 ✓ Placeholder, Add BcAdd2 Not yet implemented. Sub2 ✓ Sub, Placeholder Mul2 ✓ Mul, Placeholder Div2 ✓ RealDiv, Placeholder Pow2 ✓ Pow, Placeholder AddScalar ✓ Placeholder, Add, Const MulScalar ✓ Mul, Placeholder, Const PowScalar ✓ Pow, Placeholder, Const RSubScalar ✓ Sub, Placeholder, Const RDivScalar ✓ RealDiv, Placeholder, Const RPowScalar ✓ Pow, Placeholder, Const

#### Logical¶

Count 27/29

NNabla Function Status TF Op Description Sign ✓ Placeholder, Sign Minimum2 ✓ Placeholder, Const, Pack, Min, Add Maximum2 ✓ Placeholder, Add, Const, Pack, Max MinimumScalar ✓ Placeholder, Const, Pack, Min, Add MaximumScalar ✓ Placeholder, Add, Const, Pack, Max LogicalAnd ✓ LogicalAnd, Placeholder LogicalOr ✓ LogicalOr, Placeholder LogicalXor ✓ LogicalAnd, LogicalOr, Placeholder, LogicalNot Equal ✓ Equal, Placeholder NotEqual ✓ LogicalNot, Equal, Placeholder GreaterEqual ✓ Less, Placeholder, LogicalNot Greater ✓ Greater, Placeholder LessEqual ✓ Greater, Placeholder, LogicalNot Less ✓ Less, Placeholder LogicalAndScalar ✓ LogicalAnd, Placeholder, Const LogicalOrScalar ✓ LogicalOr, Placeholder, Const LogicalXorScalar ✓ Placeholder, Const, LogicalAnd, LogicalOr, LogicalNot EqualScalar ✓ Equal, Placeholder, Const NotEqualScalar ✓ LogicalNot, Equal, Placeholder, Const GreaterEqualScalar ✓ Less, Placeholder, LogicalNot, Const GreaterScalar ✓ Greater, Placeholder, Const LessEqualScalar ✓ Greater, Placeholder, LogicalNot, Const LessScalar ✓ Less, Placeholder, Const LogicalNot ✓ Placeholder, LogicalNot IsNaN ✓ IsNan, Placeholder IsInf X Not yet implemented. ResetNaN ✓ IsNan, Placeholder, Select, Const ResetInf X Not yet implemented. Where ✓ Placeholder, Select

#### Math¶

Count 21/22

NNabla Function Status TF Op Description Constant ✓ Identity, Const Arange ✓ Identity, Const Abs ✓ Abs, Placeholder Exp ✓ Exp, Placeholder Log ✓ Log, Placeholder Identity ✓ Placeholder, Identity BatchMatmul ✓ Reshape, Placeholder, Transpose, Const, BatchMatMulV2 Round X Not yet implemented. Ceil ✓ Ceil, Placeholder Floor ✓ Floor, Placeholder Sin ✓ Sin, Placeholder Cos ✓ Cos, Placeholder Tan ✓ Tan, Placeholder Sinh ✓ Sinh, Placeholder Cosh ✓ Cosh, Placeholder ASin ✓ Asin, Placeholder ACos ✓ Acos, Placeholder ATan ✓ Placeholder, Atan ATan2 ✓ RealDiv, Placeholder, Atan ASinh ✓ Asinh, Placeholder ACosh ✓ Acosh, Placeholder ATanh ✓ Atanh, Placeholder

#### Array Manipulation¶

Count 12/19

NNabla Function Status TF Op Description Concatenate ✓ ConcatV2, Placeholder, Const Split ✓ Squeeze, SplitV, Placeholder, Const Stack ✓ ConcatV2, ExpandDims, Placeholder, Const Slice △ Slice, Placeholder, Const step != 1” exceed the scope of onnx opset 9, not supported. Pad △ MirrorPad, PadV2, Placeholder, Const When the mode of the pad is reflect, if the size of the pad exceeds the input size, tensorflow cannot handle it. Transpose ✓ Placeholder, Transpose, Const Broadcast ✓ BroadcastTo ✓ Tile ✓ Reshape, Tile, Placeholder, Const OneHot ✓ Reshape, GatherV2, Placeholder, Const Flip ✓ Placeholder, Transpose, Const, Identity, GatherV2 Shift Not yet implemented. Sort Not yet implemented. Reshape ✓ Reshape, Placeholder, Const MatrixDiag Not yet implemented. MatrixDiagPart Not yet implemented. Assign Not yet implemented. GatherNd Not yet implemented. ScatterNd Not yet implemented.

#### Signal Processing¶

Count 0/3

NNabla Function Status TF Op Description Interpolate X Not yet implemented. FFT Not yet implemented. IFFT Not yet implemented.

#### Stochasticity¶

Count 0/11

NNabla Function Status TF Op Description Dropout X Placeholder The Dropout in nnabla has no test mode and contains random parameters, so the test result is not the same as tensorflow. TopKData Not yet implemented. TopKGrad Not yet implemented. Rand Not yet implemented. Randint Not yet implemented. Randn Not yet implemented. RandomChoice Not yet implemented. RandomCrop Not yet implemented. RandomFlip Not yet implemented. RandomShift Not yet implemented. ImageAugmentation Not yet implemented.

#### Loss Functions¶

Count 0/9

NNabla Function Status TF Op Description SigmoidCrossEntropy Not yet implemented. BinaryCrossEntropy Not yet implemented. SoftmaxCrossEntropy Not yet implemented. CategoricalCrossEntropy Not yet implemented. SquaredError Not yet implemented. AbsoluteError Not yet implemented. HuberLoss Not yet implemented. EpsilonInsensitiveLoss Not yet implemented. KLMultinomial Not yet implemented.

#### Quantization Neural Network Layers¶

Count 6/12

NNabla Function Status TF Op Description BinarySigmoid ✓ Greater, Placeholder, Select, Const BinaryTanh ✓ Greater, Placeholder, Select, Const BinaryConnectAffine ✓ Reshape, Placeholder, MatMul, Const, Mul, Add BinaryConnectConvolution △ Conv2D, Reshape, Pad, Placeholder, Transpose, Const, Split, Identity, ConcatV2, Add The cases `dilations`

and`strides`

larger than 1 are not supported by tensorflow.BinaryWeightAffine ✓ Reshape, Placeholder, MatMul, Const, Mul, Add BinaryWeightConvolution △ Conv2D, Reshape, Pad, Placeholder, Transpose, Const, Split, Identity, Mul, ConcatV2, Add The cases `dilations`

and`strides`

larger than 1 are not supported by tensorflow.INQAffine Not yet implemented. INQConvolution Not yet implemented. FixedPointQuantize Not yet implemented. MinMaxQuantize Not yet implemented. Pow2Quantize Not yet implemented. Prune Not yet implemented.

#### Validation¶

Count 0/3

NNabla Function Status TF Op Description TopNError Not yet implemented. BinaryError Not yet implemented. ConfusionMatrix Not yet implemented.

#### Unsupported, Special Use¶

Count 0/5

NNabla Function Status TF Op Description VATNoise Not yet implemented. Unlink Not yet implemented. Sink Not yet implemented. NmsDetection2d Not yet implemented. MaxPoolingBackward Not yet implemented.

## NNabla C Runtime Support Status¶

nnabla version: 1.0.21

- ✓: Supported
- △: Partially supported
- X: Supported, but test failed or no test data.
- Empty: Not support yet.

### Export¶

Total: 56/173

#### Neural Network Layer¶

Count 8/14

NNabla Function Status Description Affine ✓ RNN LSTM GRU Convolution ✓ DepthwiseConvolution ✓ Deconvolution ✓ DepthwiseDeconvolution MaxPooling ✓ AveragePooling △ GlobalAveragePooling SumPooling ✓ Unpooling ✓ Embed

#### Neural Network Activation Functions¶

Count 11/21

NNabla Function Status Description Sigmoid ✓ Swish ✓ Tanh ✓ ReLU ✓ LeakyReLU ✓ Softmax ✓ LogSoftmax ELU ✓ SELU ✓ CReLU ✓ CELU ✓ PReLU ✓ GELU ReLU6 HardSigmoid HardTanh LogSigmoid SoftPlus SoftSign TanhShrink Sinc

#### Normalization¶

Count 1/6

NNabla Function Status Description FusedBatchNormalization BatchNormalization ✓ SyncBatchNormalization MeanSubtraction X ClipGradByValue ClipGradByNorm

#### Reduction¶

Count 1/7

NNabla Function Status Description Sum ✓ Mean Max Min Prod ReduceSum ReduceMean

#### Arithmetic¶

Count 11/12

NNabla Function Status Description Add2 ✓ BcAdd2 Sub2 ✓ Mul2 ✓ Div2 ✓ Pow2 ✓ AddScalar ✓ MulScalar ✓ PowScalar ✓ RSubScalar ✓ RDivScalar ✓ RPowScalar ✓

#### Logical¶

Count 5/29

NNabla Function Status Description Sign ✓ Minimum2 ✓ Maximum2 ✓ MinimumScalar ✓ MaximumScalar ✓ LogicalAnd LogicalOr LogicalXor Equal NotEqual GreaterEqual Greater LessEqual Less LogicalAndScalar LogicalOrScalar LogicalXorScalar EqualScalar NotEqualScalar GreaterEqualScalar GreaterScalar LessEqualScalar LessScalar LogicalNot IsNaN IsInf ResetNaN ResetInf Where

#### Math¶

Count 6/22

NNabla Function Status Description Constant Arange Abs ✓ Exp ✓ Log ✓ Identity ✓ BatchMatmul ✓ Round ✓ Ceil Floor Sin Cos Tan Sinh Cosh ASin ACos ATan ATan2 ASinh ACosh ATanh

#### Array Manipulation¶

Count 7/19

NNabla Function Status Description Concatenate ✓ Split ✓ Stack ✓ Slice ✓ Pad Transpose ✓ Broadcast BroadcastTo Tile OneHot Flip ✓ Shift X Sort Reshape ✓ MatrixDiag X MatrixDiagPart X Assign GatherNd ScatterNd

#### Stochasticity¶

Count 0/11

NNabla Function Status Description Dropout X TopKData TopKGrad Rand Randint Randn RandomChoice RandomCrop RandomFlip RandomShift ImageAugmentation

#### Loss Functions¶

Count 0/9

NNabla Function Status Description SigmoidCrossEntropy BinaryCrossEntropy SoftmaxCrossEntropy CategoricalCrossEntropy SquaredError AbsoluteError HuberLoss EpsilonInsensitiveLoss KLMultinomial

#### Quantization Neural Network Layers¶

Count 6/12

NNabla Function Status Description BinarySigmoid ✓ BinaryTanh ✓ BinaryConnectAffine ✓ BinaryConnectConvolution ✓ BinaryWeightAffine ✓ BinaryWeightConvolution ✓ INQAffine INQConvolution FixedPointQuantize MinMaxQuantize Pow2Quantize Prune

#### Unsupported, Special Use¶

Count 0/5

NNabla Function Status Description VATNoise Unlink Sink NmsDetection2d MaxPoolingBackward