FunctionLevel Support Status¶
ONNX Support Status¶
 Note
In this document, the numbers in the header of all tables represent the version of onnx opset.
 ONNX Version
1.6.0
Import¶
✓: onnx specification defined, and supported.
X: onnx specification defined, but not support yet.
Empty: Not defined (Support status follows latest).
Total: 93/155
ONNX Operator 
1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
NNabla Func 
Description 

Abs 
✓ 
✓ 
Abs 

Acos 
✓ 
ACos 

Acosh 
✓ 
ACosh 

Add 
✓ 
✓ 
✓ 
Reshape, Add2 

And 
✓ 
✓ 
✓ 
LogicalAnd, Reshape 

ArgMax 
✓ 
✓ 
X 
✓ 
Max 

ArgMin 
✓ 
✓ 
X 
✓ 
Min 

Asin 
✓ 
ASin 

Asinh 
✓ 
ASinh 

Atan 
✓ 
ATan 

Atanh 
✓ 
ATanh 

AveragePool 
✓ 
✓ 
✓ 
X 
X 
Pad, AveragePooling 
Not all features are verified, since some features are not supported by caffe2. Those features can be verified by ONNXRuntime when opset > 6. Some feature is not supported by Nnabla such as Pad’s edge mode. if opset >= 10, the ceil_mode is not supported. 

BatchNormalization 
X 
X 
X 
✓ 
BatchNormalization 

BitShift 
X 
Not yet implemented. 

Cast 
✓ 
✓ 
X 
Abs, Log 

Ceil 
✓ 
✓ 
Ceil 

Clip 
✓ 
✓ 
✓ 
MinimumScalar, MaximumScalar, Identity 

Compress 
X 
X 
Not yet implemented. 

Concat 
✓ 
✓ 
✓ 
X 
Concatenate 

ConcatFromSequence 
X 
Not yet implemented. 

Constant 
✓ 
✓ 
X 
X 
Identity 

ConstantOfShape 
✓ 
Constant 

Conv 
✓ 
✓ 
X 
Convolution 

ConvInteger 
X 
Not yet implemented. 

ConvTranspose 
✓ 
✓ 
X 
Deconvolution, Pad 

Cos 
✓ 
Cos 

Cosh 
✓ 
Cosh 

CumSum 
X 
Not yet implemented. 

DepthToSpace 
✓ 
✓ 
✓ 
Reshape, Transpose 

DequantizeLinear 
X 
Not yet implemented. 

Det 
X 
Not yet implemented. 

Div 
✓ 
✓ 
✓ 
Div2, Reshape 

Dropout 
X 
X 
✓ 
X 
Identity 

DynamicQuantizeLinear 
X 
Not yet implemented. 

Elu 
✓ 
✓ 
ELU 

Equal 
✓ 
✓ 
✓ 
X 
Reshape, Equal 

Erf 
X 
Not yet implemented. 

Exp 
✓ 
✓ 
Exp 

Expand 
✓ 
✓ 
Broadcast, Reshape 

EyeLike 
X 
Not yet implemented. 

Flatten 
✓ 
✓ 
✓ 
✓ 
Reshape 

Floor 
✓ 
✓ 
Floor 

GRU 
X 
X 
X 
Not yet implemented. 

Gather 
✓ 
✓ 
✓ 
Slice, Concatenate 

GatherElements 
X 
Not yet implemented. 

GatherND 
X 
Not yet implemented. 

Gemm 
✓ 
✓ 
✓ 
✓ 
✓ 
BatchMatmul, Reshape, Add2, MulScalar 

GlobalAveragePool 
✓ 
✓ 
GlobalAveragePooling 

GlobalLpPool 
X 
X 
Not yet implemented. 

GlobalMaxPool 
X 
Not yet implemented. 

Greater 
✓ 
✓ 
✓ 
✓ 
Reshape, Greater 

HardSigmoid 
✓ 
✓ 
MaximumScalar, MulScalar, AddScalar, HardSigmoid, MinimumScalar 

Hardmax 
✓ 
✓ 
✓ 
Reshape, OneHot, Max 

Identity 
✓ 
✓ 
Identity 

If 
X 
Not yet implemented. 

InstanceNormalization 
✓ 
✓ 
Split, Reshape, BatchNormalization, Concatenate 

IsInf 
✓ 
IsInf 

IsNaN 
✓ 
IsNaN 

LRN 
✓ 
✓ 
Transpose, MulScalar, PowScalar, AddScalar, Div2, SumPooling 

LSTM 
X 
X 
Not yet implemented. 

LeakyRelu 
✓ 
✓ 
LeakyReLU 

Less 
✓ 
✓ 
✓ 
✓ 
Reshape, Less 

Log 
✓ 
✓ 
Log 

LogSoftmax 
✓ 
✓ 
✓ 
Reshape, Max, Exp, Log, Sub2, Sum, Add2 

Loop 
X 
X 
Not yet implemented. 

LpNormalization 
X 
Not yet implemented. 

LpPool 
X 
X 
X 
Not yet implemented. 

MatMul 
✓ 
✓ 
✓ 
BatchMatmul 

MatMulInteger 
X 
Not yet implemented. 

Max 
✓ 
✓ 
✓ 
✓ 
Maximum2 

MaxPool 
✓ 
✓ 
X 
X 
X 
Pad, MaxPooling 
Not all features are verified, since some features are not supported by caffe2. Those features can be verified by ONNXRuntime. if opset >= 10, the ceil_mode is not supported, dilations is not equal to 1 is not supported. 

MaxRoiPool 
X 
Not yet implemented. 

MaxUnpool 
X 
X 
Not yet implemented. 

Mean 
✓ 
✓ 
✓ 
✓ 
Broadcast, Stack, Mean 

MeanVarianceNormalization 
X 
Not yet implemented. 

Min 
✓ 
✓ 
✓ 
✓ 
Minimum2 

Mod 
X 
Not yet implemented. 

Mul 
✓ 
✓ 
✓ 
Reshape, Mul2 

Multinomial 
X 
Not yet implemented. 

Neg 
✓ 
✓ 
MulScalar 

NonMaxSuppression 
X 
X 
Not yet implemented. 

NonZero 
X 
Not yet implemented. 

Not 
✓ 
✓ 
LogicalNot 

OneHot 
X 
X 
Not yet implemented. 

Or 
✓ 
✓ 
✓ 
Reshape, LogicalOr 

PRelu 
✓ 
✓ 
X 
X 
PReLU 

Pad 
✓ 
✓ 
✓ 
✓ 
Pad 
Onnx required to support “edge” mode, while nnabla does not support it. 

Pow 
✓ 
✓ 
✓ 
Reshape, Pow2 

QLinearConv 
X 
Not yet implemented. 

QLinearMatMul 
X 
Not yet implemented. 

QuantizeLinear 
X 
Not yet implemented. 

RNN 
X 
X 
Not yet implemented. 

RandomNormal 
X 
Not yet implemented. 

RandomNormalLike 
X 
Not yet implemented. 

RandomUniform 
X 
Not yet implemented. 

RandomUniformLike 
X 
Not yet implemented. 

Range 
X 
Not yet implemented. 

Reciprocal 
✓ 
✓ 
RDivScalar 

ReduceL1 
X 
X 
Not yet implemented. 

ReduceL2 
X 
X 
Not yet implemented. 

ReduceLogSum 
X 
X 
Not yet implemented. 

ReduceLogSumExp 
X 
X 
Not yet implemented. 

ReduceMax 
✓ 
✓ 
✓ 
Max 

ReduceMean 
✓ 
✓ 
✓ 
Mean 

ReduceMin 
✓ 
✓ 
✓ 
Min 

ReduceProd 
✓ 
✓ 
✓ 
Prod 

ReduceSum 
✓ 
✓ 
✓ 
Sum 

ReduceSumSquare 
✓ 
✓ 
✓ 
PowScalar, Sum 

Relu 
✓ 
✓ 
ReLU 

Reshape 
✓ 
✓ 
✓ 
Reshape 

Resize 
X 
X 
Not yet implemented. 

ReverseSequence 
X 
Not yet implemented. 

RoiAlign 
X 
Not yet implemented. 

Round 
✓ 
Round 

Scan 
X 
X 
X 
Not yet implemented. 

Scatter 
X 
X 
Not yet implemented. 

ScatterElements 
X 
Not yet implemented. 

ScatterND 
X 
Not yet implemented. 

Selu 
✓ 
✓ 
SELU 

SequenceAt 
X 
Not yet implemented. 

SequenceConstruct 
X 
Not yet implemented. 

SequenceErase 
X 
Not yet implemented. 

SequenceInsert 
X 
Not yet implemented. 

SequenceLength 
X 
Not yet implemented. 

Shape 
X 
Not yet implemented. 

Shrink 
X 
Not yet implemented. 

Sigmoid 
✓ 
✓ 
Sigmoid 

Sign 
✓ 
Sign 

Sin 
✓ 
Sin 

Sinh 
✓ 
Sinh 

Size 
X 
Not yet implemented. 

Slice 
✓ 
✓ 
✓ 
X 
Slice 

Softmax 
✓ 
✓ 
✓ 
Reshape, Exp, Div2, Sub2, Sum, Max 

Softplus 
✓ 
✓ 
SoftPlus 

Softsign 
✓ 
✓ 
SoftSign 

SpaceToDepth 
✓ 
✓ 
Reshape, Transpose 

Split 
✓ 
✓ 
✓ 
✓ 
Stack, Split 

SplitToSequence 
X 
Not yet implemented. 

Sqrt 
✓ 
✓ 
PowScalar 

Squeeze 
✓ 
✓ 
✓ 
Reshape 

StringNormalizer 
X 
Not yet implemented. 

Sub 
✓ 
✓ 
✓ 
Sub2, Reshape 

Sum 
✓ 
✓ 
✓ 
✓ 
Add2 

Tan 
✓ 
Tan 

Tanh 
✓ 
✓ 
Tanh 

TfIdfVectorizer 
X 
Not yet implemented. 

ThresholdedRelu 
✓ 
GreaterScalar, Where, Constant 

Tile 
✓ 
✓ 
Tile 

TopK 
X 
X 
X 
Not yet implemented. 

Transpose 
✓ 
✓ 
Transpose 

Unique 
X 
Not yet implemented. 

Unsqueeze 
✓ 
✓ 
✓ 
Reshape 

Upsample 
✓ 
✓ 
✓ 
✓ 
X 
Unpooling 

Where 
✓ 
Where 

Xor 
✓ 
✓ 
✓ 
Reshape, LogicalXor 
Export¶
✓: Support to export this opset.
△: Partially support to export this opset (e.g. some cases cannot be supported, or not completely tested).
X: Supported, but test failed.
Empty: Not support corresponding opset version.
Total: 119/173
Neural Network Layer¶
Count 11/14
NNabla Function
6
7
9
10
11
ONNX Op
Description
Affine
✓
✓
✓
✓
✓
Gemm, Reshape
RNN
Not yet implemented.
LSTM
Not yet implemented.
GRU
Not yet implemented.
Convolution
✓
✓
✓
✓
✓
Reshape, Conv
DepthwiseConvolution
✓
✓
✓
✓
✓
Reshape, Conv
Deconvolution
△
△
△
△
△
ConvTranspose, Reshape
Caffe2 and onnxruntime do not support dilations != 1.
DepthwiseDeconvolution
△
△
△
△
△
ConvTranspose, Reshape
Caffe2 and onnxruntime do not support dilations != 1.
MaxPooling
✓
✓
✓
✓
X
Reshape, Pad, MaxPool
AveragePooling
△
△
△
△
X
Reshape, Pad, AveragePool
Currently only supports the cases where both ignore_border and including_pad are True.
GlobalAveragePooling
✓
✓
✓
✓
✓
GlobalAveragePool
SumPooling
X
✓
✓
✓
X
Reshape, Pad, Constant, Mul, AveragePool
Unpooling
△
✓
✓
✓
✓
Resize
The kernel only supports 2d on opset 6.
Embed
✓
✓
✓
✓
✓
Gather
Neural Network Activation Functions¶
Count 21/21
NNabla Function
6
7
9
10
11
ONNX Op
Description
Sigmoid
✓
✓
✓
✓
✓
Sigmoid
Swish
✓
✓
✓
✓
✓
Mul, Sigmoid
Tanh
✓
✓
✓
✓
✓
Tanh
ReLU
✓
✓
✓
✓
✓
Relu
LeakyReLU
✓
✓
✓
✓
✓
LeakyRelu
Softmax
△
✓
✓
✓
✓
Exp, ReduceMax, Sub, ReduceSum, Div
ONNX Add, Sub operator does not support multidirectional broadcasting on opset 6.
LogSoftmax
△
✓
✓
✓
✓
Exp, ReduceMax, Sub, Log, ReduceSum
ELU
✓
✓
✓
✓
✓
Elu
SELU
✓
✓
✓
✓
✓
Selu
CReLU
✓
✓
✓
✓
✓
Concat, Neg, Relu
CELU
✓
✓
✓
✓
✓
Elu, Concat, Neg
PReLU
✓
✓
✓
✓
✓
Reshape, PRelu
GELU
✓
✓
✓
✓
✓
Sqrt, Constant, Pow, Tanh, Mul, Add, Div
ReLU6
✓
✓
✓
✓
✓
Min, Relu, Constant
HardSigmoid
✓
✓
✓
✓
✓
HardSigmoid
HardTanh
✓
✓
✓
✓
✓
Neg, Min, Max, Constant
LogSigmoid
✓
✓
✓
✓
✓
Sigmoid, Log
SoftPlus
✓
✓
✓
✓
✓
Softplus
SoftSign
✓
✓
✓
✓
✓
Softsign
TanhShrink
✓
✓
✓
✓
✓
Sub, Tanh
Sinc
X
X
✓
✓
✓
Constant, Where, Sin, Equal, Div
Normalization¶
Count 1/6
NNabla Function
6
7
9
10
11
ONNX Op
Description
FusedBatchNormalization
X
X
X
X
X
Relu, Constant, ReduceMean, BatchNormalization, Mul, Sub, ReduceSum, Div
Not yet implemented.
BatchNormalization
✓
✓
✓
✓
✓
Reshape, Constant, ReduceMean, BatchNormalization, Mul, Sub, ReduceSum, Div
In inferring stage, caffe2 mistmatch onnx 1.4.x’s implementation, “inplace” feature cannot be applied.
SyncBatchNormalization
Not yet implemented.
MeanSubtraction
Not yet implemented.
ClipGradByValue
Not yet implemented.
ClipGradByNorm
Not yet implemented.
Reduction¶
Count 5/7
NNabla Function
6
7
9
10
11
ONNX Op
Description
Sum
✓
✓
✓
✓
✓
ReduceSum
Mean
✓
✓
✓
✓
✓
ReduceMean
Max
✓
✓
✓
✓
✓
ReduceMax
Min
✓
✓
✓
✓
✓
ReduceMin
Prod
✓
✓
✓
✓
✓
ReduceProd
ReduceSum
Not yet implemented.
ReduceMean
Not yet implemented.
Arithmetic¶
Count 11/12
NNabla Function
6
7
9
10
11
ONNX Op
Description
Add2
△
✓
✓
✓
✓
Add
ONNX Add operator does not support multidirectional broadcasting on opset 6.
BcAdd2
Not yet implemented.
Sub2
△
✓
✓
✓
✓
Sub
ONNX Sub operator does not support multidirectional broadcasting on opset 6.
Mul2
△
✓
✓
✓
✓
Mul
ONNX Mul operator does not support multidirectional broadcasting on opset 6.
Div2
△
✓
✓
✓
✓
Div
ONNX Div operator does not support multidirectional broadcasting on opset 6.
Pow2
△
✓
✓
✓
✓
Pow
ONNX Pow operator does not support multidirectional broadcasting on opset 6.
AddScalar
✓
✓
✓
✓
✓
Add, Constant
MulScalar
✓
✓
✓
✓
✓
Mul, Constant
PowScalar
✓
✓
✓
✓
✓
Pow, Constant
RSubScalar
✓
✓
✓
✓
✓
Sub, Constant
RDivScalar
✓
✓
✓
✓
✓
Constant, Div
RPowScalar
✓
✓
✓
✓
✓
Pow, Constant
Logical¶
Count 29/29
NNabla Function
6
7
9
10
11
ONNX Op
Description
Sign
X
X
✓
✓
✓
Sign
Minimum2
△
✓
✓
✓
✓
Min, Add, Constant
ONNX Add operator does not support multidirectional broadcasting on opset 6.
Maximum2
△
✓
✓
✓
✓
Add, Max, Constant
ONNX Add operator does not support multidirectional broadcasting on opset 6.
MinimumScalar
✓
✓
✓
✓
✓
Min, Add, Constant
MaximumScalar
✓
✓
✓
✓
✓
Add, Max, Constant
LogicalAnd
✓
✓
✓
✓
✓
And
LogicalOr
✓
✓
✓
✓
✓
Or
LogicalXor
✓
✓
✓
✓
✓
Xor
Equal
✓
✓
✓
✓
✓
Equal
NotEqual
✓
✓
✓
✓
✓
Equal, Not
GreaterEqual
✓
✓
✓
✓
✓
Less, Not
Greater
✓
✓
✓
✓
✓
Greater
LessEqual
✓
✓
✓
✓
✓
Greater, Not
Less
✓
✓
✓
✓
✓
Less
LogicalAndScalar
✓
✓
✓
✓
✓
And, Constant
LogicalOrScalar
✓
✓
✓
✓
✓
Or, Constant
LogicalXorScalar
✓
✓
✓
✓
✓
Xor, Constant
EqualScalar
✓
✓
✓
✓
✓
Equal, Constant
NotEqualScalar
✓
✓
✓
✓
✓
Equal, Not, Constant
GreaterEqualScalar
✓
✓
✓
✓
✓
Less, Not, Constant
GreaterScalar
✓
✓
✓
✓
✓
Greater, Constant
LessEqualScalar
✓
✓
✓
✓
✓
Greater, Not, Constant
LessScalar
✓
✓
✓
✓
✓
Less, Constant
LogicalNot
✓
✓
✓
✓
✓
Not
IsNaN
X
X
✓
✓
✓
IsNaN
IsInf
X
X
X
✓
✓
IsInf
ResetNaN
X
X
✓
✓
✓
Where, IsNaN, Constant
ResetInf
X
X
X
✓
✓
Where, IsInf, Constant
Where
X
X
✓
✓
✓
Where
Math¶
Count 22/22
NNabla Function
6
7
9
10
11
ONNX Op
Description
Constant
✓
✓
✓
✓
✓
Identity, Constant
Arange
✓
✓
✓
✓
✓
Identity, Constant
Abs
✓
✓
✓
✓
✓
Abs
Exp
✓
✓
✓
✓
✓
Exp
Log
✓
✓
✓
✓
✓
Log
Identity
✓
✓
✓
✓
✓
Identity
BatchMatmul
✓
✓
✓
✓
✓
Reshape, MatMul, Transpose
Round
X
X
X
X
✓
Round
Ceil
✓
✓
✓
✓
✓
Ceil
Floor
✓
✓
✓
✓
✓
Floor
Sin
X
✓
✓
✓
✓
Sin
Cos
X
✓
✓
✓
✓
Cos
Tan
X
✓
✓
✓
✓
Tan
Sinh
X
X
✓
✓
✓
Sinh
Cosh
X
X
✓
✓
✓
Cosh
ASin
X
✓
✓
✓
✓
Asin
ACos
X
✓
✓
✓
✓
Acos
ATan
X
✓
✓
✓
✓
Atan
ATan2
X
✓
✓
✓
✓
Div, Atan
ASinh
X
X
✓
✓
✓
Asinh
ACosh
X
X
✓
✓
✓
Acosh
ATanh
X
X
✓
✓
✓
Atanh
Array Manipulation¶
Count 12/19
NNabla Function
6
7
9
10
11
ONNX Op
Description
Concatenate
✓
✓
✓
✓
✓
Concat
Split
✓
✓
✓
✓
✓
Split, Squeeze
Stack
✓
✓
✓
✓
✓
Concat, Unsqueeze
Slice
△
△
△
△
△
Slice, Constant
ONNX slice cannot support step != 1 on opset < 10.
Pad
△
△
△
△
△
Pad, Constant
When the mode of the pad is reflect, if the size of the pad exceeds the input size, caffe2 and onnxruntime cannot handle it.
Transpose
✓
✓
✓
✓
✓
Transpose
Broadcast
X
X
✓
✓
✓
BroadcastTo
✓
✓
✓
✓
✓
Tile
✓
✓
✓
✓
△
Reshape, Tile, Constant
OneHot
✓
✓
✓
✓
✓
Gather, Reshape, Flatten
Flip
✓
✓
✓
✓
✓
Gather, Transpose, Identity
Shift
Not yet implemented.
Sort
Not yet implemented.
Reshape
✓
✓
✓
✓
✓
Reshape, Constant
MatrixDiag
Not yet implemented.
MatrixDiagPart
Not yet implemented.
Assign
Not yet implemented.
GatherNd
Not yet implemented.
ScatterNd
Not yet implemented.
Signal Processing¶
Count 1/3
NNabla Function
6
7
9
10
11
ONNX Op
Description
Interpolate
X
X
X
X
△
Reshape, Resize
FFT
Not yet implemented.
IFFT
Not yet implemented.
Stochasticity¶
Count 0/11
NNabla Function
6
7
9
10
11
ONNX Op
Description
Dropout
X
X
X
X
X
Dropout
The Dropout in nnabla has no test mode and contains random parameters, so the test result is not the same as onnx.
TopKData
Not yet implemented.
TopKGrad
Not yet implemented.
Rand
Not yet implemented.
Randint
Not yet implemented.
Randn
Not yet implemented.
RandomChoice
Not yet implemented.
RandomCrop
Not yet implemented.
RandomFlip
Not yet implemented.
RandomShift
Not yet implemented.
ImageAugmentation
Not yet implemented.
Loss Functions¶
Count 0/9
NNabla Function
6
7
9
10
11
ONNX Op
Description
SigmoidCrossEntropy
Not yet implemented.
BinaryCrossEntropy
Not yet implemented.
SoftmaxCrossEntropy
Not yet implemented.
CategoricalCrossEntropy
Not yet implemented.
SquaredError
Not yet implemented.
AbsoluteError
Not yet implemented.
HuberLoss
Not yet implemented.
EpsilonInsensitiveLoss
Not yet implemented.
KLMultinomial
Not yet implemented.
Quantization Neural Network Layers¶
Count 6/12
NNabla Function
6
7
9
10
11
ONNX Op
Description
BinarySigmoid
X
X
✓
✓
✓
Where, Greater, Constant
BinaryTanh
X
X
✓
✓
✓
Where, Greater, Constant
BinaryConnectAffine
✓
✓
✓
✓
✓
Gemm, Reshape
BinaryConnectConvolution
✓
✓
✓
✓
✓
Reshape, Conv
BinaryWeightAffine
✓
✓
✓
✓
✓
Mul, Reshape, Add, MatMul
BinaryWeightConvolution
✓
✓
✓
✓
✓
Mul, Reshape, Conv, Add
INQAffine
Not yet implemented.
INQConvolution
Not yet implemented.
FixedPointQuantize
Not yet implemented.
MinMaxQuantize
Not yet implemented.
Pow2Quantize
Not yet implemented.
Prune
Not yet implemented.
Validation¶
Count 0/3
NNabla Function
6
7
9
10
11
ONNX Op
Description
TopNError
Not yet implemented.
BinaryError
Not yet implemented.
ConfusionMatrix
Not yet implemented.
Unsupported, Special Use¶
Count 0/5
NNabla Function
6
7
9
10
11
ONNX Op
Description
VATNoise
Not yet implemented.
Unlink
Not yet implemented.
Sink
Not yet implemented.
NmsDetection2d
Not yet implemented.
MaxPoolingBackward
Not yet implemented.
Tensorflow Support Status¶
 Note
In this document, the numbers in the header of all tables represent the version of onnx opset.
Import¶
✓: Supported
△: Partially supported
X: Supported, but test failed.
Empty: Not support yet.
Total: 86/120
Tensorflow Function 
Status 
NNabla Func 
Description 

Abs 
✓ 
Abs 

Acos 
✓ 
ACos 

Acosh 
✓ 
ACosh 

Add 
✓ 
Add2 

AddN 
✓ 
Add2 

All 
Not yet implemented. 

Any 
Not yet implemented. 

ArgMax 
✓ 
Max 

ArgMin 
✓ 
Min 

Asin 
✓ 
ASin 

Asinh 
✓ 
ASinh 

Atan 
✓ 
ATan 

Atan2 
Not yet implemented. 

Atanh 
✓ 
ATanh 

AvgPool 
△ 
Pad, Transpose, AveragePooling 
Some feature is not supported by Nnabla such as Pad’s edge mode. 
AvgPool3D 
Not yet implemented. 

BatchMatMul 
✓ 
BatchMatmul, Transpose 

BiasAdd 
✓ 
Reshape, Add2 

Cast 
Not yet implemented. 

Ceil 
✓ 
Ceil 

ConcatV2 
✓ 
Concatenate 

Const 
✓ 
Add2 

Conv2D 
△ 
Pad, Convolution, Transpose 
Tensorflow require GPU to perform related test cases. This issue is recorded only for memo. 
Conv2DBackpropFilter 
Not yet implemented. 

Conv2DBackpropInput 
△ 
Deconvolution, Transpose 
Tensorflow require GPU to perform related test cases. This issue is recorded only for memo. 
Conv3D 
Not yet implemented. 

Conv3DBackpropFilterV2 
Not yet implemented. 

Conv3DBackpropInputV2 
Not yet implemented. 

Cos 
✓ 
Cos 

Cosh 
✓ 
Cosh 

DepthToSpace 
△ 
Reshape, Transpose 
Tensorflow require GPU to perform related test cases. This issue is recorded only for memo. 
DepthwiseConv2dNative 
Not yet implemented. 

DepthwiseConv2dNativeBackpropFilter 
Not yet implemented. 

DepthwiseConv2dNativeBackpropInput 
Not yet implemented. 

Div 
✓ 
Div2 

Elu 
✓ 
ELU 

Equal 
✓ 
Equal 

Erf 
Not yet implemented. 

Erfc 
Not yet implemented. 

Exp 
✓ 
Exp 

ExpandDims 
✓ 
Reshape 

Fill 
Not yet implemented. 

Flatten 
✓ 
Reshape 

Floor 
✓ 
Floor 

FloorDiv 
✓ 
Div2, Floor 

FloorMod 
✓ 
Div2, Sub2, Mul2, Floor 

FusedBatchNorm 
△ 
BatchNormalization, Transpose 
It did not pass testing for training mode. 
GatherNd 
Not yet implemented. 

GatherV2 
Not yet implemented. 

Greater 
✓ 
Greater 

GreaterEqual 
✓ 
Less, LogicalNot 

Identity 
✓ 
Identity 

IsInf 
Not yet implemented. 

IsNan 
✓ 
IsNaN 

LeakyRelu 
✓ 
LeakyReLU 

Less 
✓ 
Less 

LessEqual 
✓ 
Greater, LogicalNot 

Log 
✓ 
Log 

LogSoftmax 
Not yet implemented. 

LogicalAnd 
✓ 
LogicalAnd 

LogicalNot 
✓ 
LogicalNot 

LogicalOr 
✓ 
LogicalOr 

LogicalXor 
✓ 
LogicalAnd, LogicalOr, LogicalNot 

MatrixBandPart 
Not yet implemented. 

Max 
✓ 
Max 

MaxPool 
✓ 
Transpose, Pad, MaxPooling 

MaxPool3D 
Not yet implemented. 

MaxPoolWithArgmax 
Not yet implemented. 

Maximum 
✓ 
Maximum2 

Mean 
✓ 
Mean 

Min 
✓ 
Min 

Minimum 
✓ 
Minimum2 

Mul 
✓ 
Mul2 

Neg 
✓ 
MulScalar 

NotEqual 
✓ 
Equal, LogicalNot 

OneHot 
Not yet implemented. 

Pack 
✓ 
Reshape, Concatenate 

Pad 
✓ 
Pad 

Pow 
✓ 
Pow2 

Prod 
✓ 
Prod 

RandomShuffle 
Not yet implemented. 

RandomStandardNormal 
Not yet implemented. 

RandomUniform 
Not yet implemented. 

RealDiv 
✓ 
Div2 

Reciprocal 
✓ 
RDivScalar 

Relu 
✓ 
ReLU 

Relu6 
✓ 
MinimumScalar, MaximumScalar 

Reshape 
△ 
Reshape 
Some test cases failed for some nnabla’s implementation limitation (e.g. 1 is regarded as batch_size). 
ReverseSequence 
Not yet implemented. 

Rsqrt 
✓ 
PowScalar, RDivScalar 

Select 
Not yet implemented. 

Selu 
✓ 
SELU 

Shape 
Not yet implemented. 

Sigmoid 
✓ 
Sigmoid 

Sign 
✓ 
Sign 

Sin 
✓ 
Sin 

Sinh 
✓ 
Sinh 

Size 
Not yet implemented. 

Slice 
✓ 
Slice 

Softmax 
Not yet implemented. 

Softplus 
✓ 
SoftPlus 

Softsign 
✓ 
SoftSign 

SpaceToDepth 
△ 
Reshape, Transpose 
Tensorflow require GPU to perform related test cases. This issue is recorded only for memo. 
SplitV 
✓ 
Stack, Split 

Sqrt 
✓ 
PowScalar 

Square 
✓ 
Mul2 

SquaredDifference 
✓ 
Sub2, Mul2 

Squeeze 
✓ 
Reshape 

StopGradient 
✓ 
Identity 

StridedSlice 
✓ 
Slice 

Sub 
✓ 
Sub2 

Sum 
✓ 
Sum 

Tan 
✓ 
Tan 

Tanh 
✓ 
Tanh 

Tile 
✓ 
Tile 

TopKV2 
Not yet implemented. 

Transpose 
✓ 
Transpose 

TruncateDiv 
Not yet implemented. 

TruncateMod 
Not yet implemented. 

Unpack 
✓ 
Reshape, Stack, Split, Concatenate 
Export¶
✓: Supported
△: Partially supported
X: Supported, but test failed.
Empty: Not support yet.
Total: 115/173
Neural Network Layer¶
Count 11/14
NNabla Function
Status
TF Op
Description
Affine
✓
Reshape, Placeholder, MatMul, Const, Mul, Add
RNN
Not yet implemented.
LSTM
Not yet implemented.
GRU
Not yet implemented.
Convolution
△
Reshape, SpaceToBatchND, Pad, Placeholder, Transpose, Const, Add, Split, Identity, ConcatV2, Conv2D, BatchToSpaceND
The cases
dilations
andstrides
larger than 1 are not supported by tensorflow.DepthwiseConvolution
△
Conv2D, Reshape, SpaceToBatchND, Pad, Placeholder, Transpose, Const, Split, ConcatV2, Add, BatchToSpaceND
The cases
dilations
andstrides
larger than 1 are not supported by tensorflow.Deconvolution
△
Reshape, Slice, Placeholder, Transpose, Const, Split, Identity, ConcatV2, Conv2DBackpropInput, Add
The cases
dilations
larger than 1 are not supported by tensorflow.DepthwiseDeconvolution
△
Reshape, Slice, Placeholder, Transpose, Const, Split, ConcatV2, Conv2DBackpropInput, Add
The cases
dilations
larger than 1 are not supported by tensorflow.MaxPooling
✓
Reshape, PadV2, Placeholder, Transpose, Const, MaxPool, MaxPool3D
AveragePooling
△
AvgPool3D, Reshape, Pad, Placeholder, Transpose, Const, AvgPool
Currently only supports the cases both ignore_border and including_pad are True.
GlobalAveragePooling
✓
Const, Pack, Sub, SplitV, Range, Mean
SumPooling
✓
AvgPool3D, Reshape, Pad, Placeholder, Transpose, Const, Mul, AvgPool
Unpooling
△
Reshape, Placeholder, Transpose, ResizeNearestNeighbor, Const, LogicalAnd, Merge, Switch, Identity, Assert, Mul, Equal, StridedSlice, NoOp, Cast
The kernel only supports 2d.
Embed
✓
GatherV2, Placeholder, Const
Neural Network Activation Functions¶
Count 21/21
NNabla Function
Status
TF Op
Description
Sigmoid
✓
Sigmoid, Placeholder
Swish
✓
Mul, Sigmoid, Placeholder
Tanh
✓
Placeholder, Tanh
ReLU
✓
Relu, Placeholder
LeakyReLU
✓
LeakyRelu, Placeholder
Softmax
✓
Placeholder, Const, Exp, Sub, RealDiv, Sum, Max
LogSoftmax
✓
Placeholder, Const, Exp, Sub, Log, Sum, Max
ELU
✓
Less, Placeholder, Const, Elu, Exp, Mul, GreaterEqual, Sub, Cast, Add
SELU
✓
Minimum, Placeholder, Const, Exp, Mul, Sub, Maximum, Add
CReLU
✓
Relu, Placeholder, Const, ConcatV2, Neg
CELU
✓
Less, Placeholder, Const, Elu, Exp, ConcatV2, Mul, GreaterEqual, Sub, Neg, Cast, Add
PReLU
✓
Reshape, Relu, Placeholder, Const, Mul, Sub, Abs, Add
GELU
✓
Sqrt, Placeholder, Const, Pow, Tanh, Mul, RealDiv, Add
ReLU6
✓
Relu, Placeholder, Const, Pack, Min
HardSigmoid
✓
Minimum, Placeholder, Const, Mul, Maximum, Add
HardTanh
✓
Placeholder, Const, Pack, Neg, Min, Max
LogSigmoid
✓
Sigmoid, Log, Placeholder
SoftPlus
✓
Softplus, Placeholder
SoftSign
✓
Placeholder, Softsign
TanhShrink
✓
Sub, Placeholder, Tanh
Sinc
✓
Placeholder, Select, Const, Sin, Equal, RealDiv
Normalization¶
Count 1/6
NNabla Function
Status
TF Op
Description
FusedBatchNormalization
X
Reshape, Relu, Placeholder, Const, Mul, Rsqrt, Sub, Mean, RealDiv, Sum, Add
Not yet implemented.
BatchNormalization
✓
Reshape, Placeholder, Const, Mul, Rsqrt, Sub, Mean, RealDiv, Sum, Add
In inferring stage, caffe2 mistmatch onnx 1.4.x’s implementation, “inplace” feature cannot be applied.
SyncBatchNormalization
Not yet implemented.
MeanSubtraction
Not yet implemented.
ClipGradByValue
Not yet implemented.
ClipGradByNorm
Not yet implemented.
Reduction¶
Count 5/7
NNabla Function
Status
TF Op
Description
Sum
✓
Sum, Placeholder, Const
Mean
✓
Mean, Placeholder, Const
Max
✓
Placeholder, Max, Const
Min
✓
Min, Placeholder, Const
Prod
✓
Prod, Placeholder, Const
ReduceSum
Not yet implemented.
ReduceMean
Not yet implemented.
Arithmetic¶
Count 11/12
NNabla Function
Status
TF Op
Description
Add2
✓
Placeholder, Add
BcAdd2
Not yet implemented.
Sub2
✓
Sub, Placeholder
Mul2
✓
Mul, Placeholder
Div2
✓
RealDiv, Placeholder
Pow2
✓
Pow, Placeholder
AddScalar
✓
Placeholder, Add, Const
MulScalar
✓
Mul, Placeholder, Const
PowScalar
✓
Pow, Placeholder, Const
RSubScalar
✓
Sub, Placeholder, Const
RDivScalar
✓
RealDiv, Placeholder, Const
RPowScalar
✓
Pow, Placeholder, Const
Logical¶
Count 27/29
NNabla Function
Status
TF Op
Description
Sign
✓
Placeholder, Sign
Minimum2
✓
Placeholder, Const, Pack, Min, Add
Maximum2
✓
Placeholder, Add, Const, Pack, Max
MinimumScalar
✓
Placeholder, Const, Pack, Min, Add
MaximumScalar
✓
Placeholder, Add, Const, Pack, Max
LogicalAnd
✓
LogicalAnd, Placeholder
LogicalOr
✓
LogicalOr, Placeholder
LogicalXor
✓
LogicalAnd, LogicalOr, Placeholder, LogicalNot
Equal
✓
Equal, Placeholder
NotEqual
✓
LogicalNot, Equal, Placeholder
GreaterEqual
✓
Less, Placeholder, LogicalNot
Greater
✓
Greater, Placeholder
LessEqual
✓
Greater, Placeholder, LogicalNot
Less
✓
Less, Placeholder
LogicalAndScalar
✓
LogicalAnd, Placeholder, Const
LogicalOrScalar
✓
LogicalOr, Placeholder, Const
LogicalXorScalar
✓
Placeholder, Const, LogicalAnd, LogicalOr, LogicalNot
EqualScalar
✓
Equal, Placeholder, Const
NotEqualScalar
✓
LogicalNot, Equal, Placeholder, Const
GreaterEqualScalar
✓
Less, Placeholder, LogicalNot, Const
GreaterScalar
✓
Greater, Placeholder, Const
LessEqualScalar
✓
Greater, Placeholder, LogicalNot, Const
LessScalar
✓
Less, Placeholder, Const
LogicalNot
✓
Placeholder, LogicalNot
IsNaN
✓
IsNan, Placeholder
IsInf
X
Not yet implemented.
ResetNaN
✓
IsNan, Placeholder, Select, Const
ResetInf
X
Not yet implemented.
Where
✓
Placeholder, Select
Math¶
Count 21/22
NNabla Function
Status
TF Op
Description
Constant
✓
Identity, Const
Arange
✓
Identity, Const
Abs
✓
Abs, Placeholder
Exp
✓
Exp, Placeholder
Log
✓
Log, Placeholder
Identity
✓
Placeholder, Identity
BatchMatmul
✓
Reshape, Placeholder, Transpose, Const, BatchMatMulV2
Round
X
Not yet implemented.
Ceil
✓
Ceil, Placeholder
Floor
✓
Floor, Placeholder
Sin
✓
Sin, Placeholder
Cos
✓
Cos, Placeholder
Tan
✓
Tan, Placeholder
Sinh
✓
Sinh, Placeholder
Cosh
✓
Cosh, Placeholder
ASin
✓
Asin, Placeholder
ACos
✓
Acos, Placeholder
ATan
✓
Placeholder, Atan
ATan2
✓
RealDiv, Placeholder, Atan
ASinh
✓
Asinh, Placeholder
ACosh
✓
Acosh, Placeholder
ATanh
✓
Atanh, Placeholder
Array Manipulation¶
Count 12/19
NNabla Function
Status
TF Op
Description
Concatenate
✓
ConcatV2, Placeholder, Const
Split
✓
Squeeze, SplitV, Placeholder, Const
Stack
✓
ConcatV2, ExpandDims, Placeholder, Const
Slice
△
Slice, Placeholder, Const
step != 1” exceed the scope of onnx opset 9, not supported.
Pad
△
MirrorPad, PadV2, Placeholder, Const
When the mode of the pad is reflect, if the size of the pad exceeds the input size, tensorflow cannot handle it.
Transpose
✓
Placeholder, Transpose, Const
Broadcast
✓
BroadcastTo
✓
Tile
✓
Reshape, Tile, Placeholder, Const
OneHot
✓
Reshape, GatherV2, Placeholder, Const
Flip
✓
Placeholder, Transpose, Const, Identity, GatherV2
Shift
Not yet implemented.
Sort
Not yet implemented.
Reshape
✓
Reshape, Placeholder, Const
MatrixDiag
Not yet implemented.
MatrixDiagPart
Not yet implemented.
Assign
Not yet implemented.
GatherNd
Not yet implemented.
ScatterNd
Not yet implemented.
Signal Processing¶
Count 0/3
NNabla Function
Status
TF Op
Description
Interpolate
X
Not yet implemented.
FFT
Not yet implemented.
IFFT
Not yet implemented.
Stochasticity¶
Count 0/11
NNabla Function
Status
TF Op
Description
Dropout
X
Placeholder
The Dropout in nnabla has no test mode and contains random parameters, so the test result is not the same as tensorflow.
TopKData
Not yet implemented.
TopKGrad
Not yet implemented.
Rand
Not yet implemented.
Randint
Not yet implemented.
Randn
Not yet implemented.
RandomChoice
Not yet implemented.
RandomCrop
Not yet implemented.
RandomFlip
Not yet implemented.
RandomShift
Not yet implemented.
ImageAugmentation
Not yet implemented.
Loss Functions¶
Count 0/9
NNabla Function
Status
TF Op
Description
SigmoidCrossEntropy
Not yet implemented.
BinaryCrossEntropy
Not yet implemented.
SoftmaxCrossEntropy
Not yet implemented.
CategoricalCrossEntropy
Not yet implemented.
SquaredError
Not yet implemented.
AbsoluteError
Not yet implemented.
HuberLoss
Not yet implemented.
EpsilonInsensitiveLoss
Not yet implemented.
KLMultinomial
Not yet implemented.
Quantization Neural Network Layers¶
Count 6/12
NNabla Function
Status
TF Op
Description
BinarySigmoid
✓
Greater, Placeholder, Select, Const
BinaryTanh
✓
Greater, Placeholder, Select, Const
BinaryConnectAffine
✓
Reshape, Placeholder, MatMul, Const, Mul, Add
BinaryConnectConvolution
△
Conv2D, Reshape, Pad, Placeholder, Transpose, Const, Split, Identity, ConcatV2, Add
The cases
dilations
andstrides
larger than 1 are not supported by tensorflow.BinaryWeightAffine
✓
Reshape, Placeholder, MatMul, Const, Mul, Add
BinaryWeightConvolution
△
Conv2D, Reshape, Pad, Placeholder, Transpose, Const, Split, Identity, Mul, ConcatV2, Add
The cases
dilations
andstrides
larger than 1 are not supported by tensorflow.INQAffine
Not yet implemented.
INQConvolution
Not yet implemented.
FixedPointQuantize
Not yet implemented.
MinMaxQuantize
Not yet implemented.
Pow2Quantize
Not yet implemented.
Prune
Not yet implemented.
Validation¶
Count 0/3
NNabla Function
Status
TF Op
Description
TopNError
Not yet implemented.
BinaryError
Not yet implemented.
ConfusionMatrix
Not yet implemented.
Unsupported, Special Use¶
Count 0/5
NNabla Function
Status
TF Op
Description
VATNoise
Not yet implemented.
Unlink
Not yet implemented.
Sink
Not yet implemented.
NmsDetection2d
Not yet implemented.
MaxPoolingBackward
Not yet implemented.
NNabla C Runtime Support Status¶
nnabla version: 1.0.21
✓: Supported
△: Partially supported
X: Supported, but test failed or no test data.
Empty: Not support yet.
Export¶
Total: 56/173
Neural Network Layer¶
Count 8/14
NNabla Function
Status
Description
Affine
✓
RNN
LSTM
GRU
Convolution
✓
DepthwiseConvolution
✓
Deconvolution
✓
DepthwiseDeconvolution
MaxPooling
✓
AveragePooling
△
GlobalAveragePooling
SumPooling
✓
Unpooling
✓
Embed
Neural Network Activation Functions¶
Count 11/21
NNabla Function
Status
Description
Sigmoid
✓
Swish
✓
Tanh
✓
ReLU
✓
LeakyReLU
✓
Softmax
✓
LogSoftmax
ELU
✓
SELU
✓
CReLU
✓
CELU
✓
PReLU
✓
GELU
ReLU6
HardSigmoid
HardTanh
LogSigmoid
SoftPlus
SoftSign
TanhShrink
Sinc
Normalization¶
Count 1/6
NNabla Function
Status
Description
FusedBatchNormalization
BatchNormalization
✓
SyncBatchNormalization
MeanSubtraction
X
ClipGradByValue
ClipGradByNorm
Reduction¶
Count 1/7
NNabla Function
Status
Description
Sum
✓
Mean
Max
Min
Prod
ReduceSum
ReduceMean
Arithmetic¶
Count 11/12
NNabla Function
Status
Description
Add2
✓
BcAdd2
Sub2
✓
Mul2
✓
Div2
✓
Pow2
✓
AddScalar
✓
MulScalar
✓
PowScalar
✓
RSubScalar
✓
RDivScalar
✓
RPowScalar
✓
Logical¶
Count 5/29
NNabla Function
Status
Description
Sign
✓
Minimum2
✓
Maximum2
✓
MinimumScalar
✓
MaximumScalar
✓
LogicalAnd
LogicalOr
LogicalXor
Equal
NotEqual
GreaterEqual
Greater
LessEqual
Less
LogicalAndScalar
LogicalOrScalar
LogicalXorScalar
EqualScalar
NotEqualScalar
GreaterEqualScalar
GreaterScalar
LessEqualScalar
LessScalar
LogicalNot
IsNaN
IsInf
ResetNaN
ResetInf
Where
Math¶
Count 6/22
NNabla Function
Status
Description
Constant
Arange
Abs
✓
Exp
✓
Log
✓
Identity
✓
BatchMatmul
✓
Round
✓
Ceil
Floor
Sin
Cos
Tan
Sinh
Cosh
ASin
ACos
ATan
ATan2
ASinh
ACosh
ATanh
Array Manipulation¶
Count 7/19
NNabla Function
Status
Description
Concatenate
✓
Split
✓
Stack
✓
Slice
✓
Pad
Transpose
✓
Broadcast
BroadcastTo
Tile
OneHot
Flip
✓
Shift
X
Sort
Reshape
✓
MatrixDiag
X
MatrixDiagPart
X
Assign
GatherNd
ScatterNd
Stochasticity¶
Count 0/11
NNabla Function
Status
Description
Dropout
X
TopKData
TopKGrad
Rand
Randint
Randn
RandomChoice
RandomCrop
RandomFlip
RandomShift
ImageAugmentation
Loss Functions¶
Count 0/9
NNabla Function
Status
Description
SigmoidCrossEntropy
BinaryCrossEntropy
SoftmaxCrossEntropy
CategoricalCrossEntropy
SquaredError
AbsoluteError
HuberLoss
EpsilonInsensitiveLoss
KLMultinomial
Quantization Neural Network Layers¶
Count 6/12
NNabla Function
Status
Description
BinarySigmoid
✓
BinaryTanh
✓
BinaryConnectAffine
✓
BinaryConnectConvolution
✓
BinaryWeightAffine
✓
BinaryWeightConvolution
✓
INQAffine
INQConvolution
FixedPointQuantize
MinMaxQuantize
Pow2Quantize
Prune
Unsupported, Special Use¶
Count 0/5
NNabla Function
Status
Description
VATNoise
Unlink
Sink
NmsDetection2d
MaxPoolingBackward