Function-Level Support Status

ONNX Support Status

Note

In this document, the numbers in the header of all tables represent the version of onnx opset.

ONNX Version

1.6.0

Import

  • ✓: onnx specification defined, and supported.

  • X: onnx specification defined, but not support yet.

  • Empty: Not defined (Support status follows latest).

Total: 93/155

ONNX Operator

1

2

3

4

5

6

7

8

9

10

11

NNabla Func

Description

Abs

Abs

Acos

ACos

Acosh

ACosh

Add

Reshape, Add2

And

LogicalAnd, Reshape

ArgMax

X

Max

ArgMin

X

Min

Asin

ASin

Asinh

ASinh

Atan

ATan

Atanh

ATanh

AveragePool

X

X

Pad, AveragePooling

Not all features are verified, since some features are not supported by caffe2. Those features can be verified by ONNXRuntime when opset > 6. Some feature is not supported by Nnabla such as Pad’s edge mode. if opset >= 10, the ceil_mode is not supported.

BatchNormalization

X

X

X

BatchNormalization

BitShift

X

Not yet implemented.

Cast

X

Abs, Log

Ceil

Ceil

Clip

MinimumScalar, MaximumScalar, Identity

Compress

X

X

Not yet implemented.

Concat

X

Concatenate

ConcatFromSequence

X

Not yet implemented.

Constant

X

X

Identity

ConstantOfShape

Constant

Conv

X

Convolution

ConvInteger

X

Not yet implemented.

ConvTranspose

X

Deconvolution, Pad

Cos

Cos

Cosh

Cosh

CumSum

X

Not yet implemented.

DepthToSpace

Reshape, Transpose

DequantizeLinear

X

Not yet implemented.

Det

X

Not yet implemented.

Div

Div2, Reshape

Dropout

X

X

X

Identity

DynamicQuantizeLinear

X

Not yet implemented.

Elu

ELU

Equal

X

Reshape, Equal

Erf

X

Not yet implemented.

Exp

Exp

Expand

Broadcast, Reshape

EyeLike

X

Not yet implemented.

Flatten

Reshape

Floor

Floor

GRU

X

X

X

Not yet implemented.

Gather

Slice, Concatenate

GatherElements

X

Not yet implemented.

GatherND

X

Not yet implemented.

Gemm

BatchMatmul, Reshape, Add2, MulScalar

GlobalAveragePool

GlobalAveragePooling

GlobalLpPool

X

X

Not yet implemented.

GlobalMaxPool

X

Not yet implemented.

Greater

Reshape, Greater

HardSigmoid

MaximumScalar, MulScalar, AddScalar, HardSigmoid, MinimumScalar

Hardmax

Reshape, OneHot, Max

Identity

Identity

If

X

Not yet implemented.

InstanceNormalization

Split, Reshape, BatchNormalization, Concatenate

IsInf

IsInf

IsNaN

IsNaN

LRN

Transpose, MulScalar, PowScalar, AddScalar, Div2, SumPooling

LSTM

X

X

Not yet implemented.

LeakyRelu

LeakyReLU

Less

Reshape, Less

Log

Log

LogSoftmax

Reshape, Max, Exp, Log, Sub2, Sum, Add2

Loop

X

X

Not yet implemented.

LpNormalization

X

Not yet implemented.

LpPool

X

X

X

Not yet implemented.

MatMul

BatchMatmul

MatMulInteger

X

Not yet implemented.

Max

Maximum2

MaxPool

X

X

X

Pad, MaxPooling

Not all features are verified, since some features are not supported by caffe2. Those features can be verified by ONNXRuntime. if opset >= 10, the ceil_mode is not supported, dilations is not equal to 1 is not supported.

MaxRoiPool

X

Not yet implemented.

MaxUnpool

X

X

Not yet implemented.

Mean

Broadcast, Stack, Mean

MeanVarianceNormalization

X

Not yet implemented.

Min

Minimum2

Mod

X

Not yet implemented.

Mul

Reshape, Mul2

Multinomial

X

Not yet implemented.

Neg

MulScalar

NonMaxSuppression

X

X

Not yet implemented.

NonZero

X

Not yet implemented.

Not

LogicalNot

OneHot

X

X

Not yet implemented.

Or

Reshape, LogicalOr

PRelu

X

X

PReLU

Pad

Pad

Onnx required to support “edge” mode, while nnabla does not support it.

Pow

Reshape, Pow2

QLinearConv

X

Not yet implemented.

QLinearMatMul

X

Not yet implemented.

QuantizeLinear

X

Not yet implemented.

RNN

X

X

Not yet implemented.

RandomNormal

X

Not yet implemented.

RandomNormalLike

X

Not yet implemented.

RandomUniform

X

Not yet implemented.

RandomUniformLike

X

Not yet implemented.

Range

X

Not yet implemented.

Reciprocal

RDivScalar

ReduceL1

X

X

Not yet implemented.

ReduceL2

X

X

Not yet implemented.

ReduceLogSum

X

X

Not yet implemented.

ReduceLogSumExp

X

X

Not yet implemented.

ReduceMax

Max

ReduceMean

Mean

ReduceMin

Min

ReduceProd

Prod

ReduceSum

Sum

ReduceSumSquare

PowScalar, Sum

Relu

ReLU

Reshape

Reshape

Resize

X

X

Not yet implemented.

ReverseSequence

X

Not yet implemented.

RoiAlign

X

Not yet implemented.

Round

Round

Scan

X

X

X

Not yet implemented.

Scatter

X

X

Not yet implemented.

ScatterElements

X

Not yet implemented.

ScatterND

X

Not yet implemented.

Selu

SELU

SequenceAt

X

Not yet implemented.

SequenceConstruct

X

Not yet implemented.

SequenceErase

X

Not yet implemented.

SequenceInsert

X

Not yet implemented.

SequenceLength

X

Not yet implemented.

Shape

X

Not yet implemented.

Shrink

X

Not yet implemented.

Sigmoid

Sigmoid

Sign

Sign

Sin

Sin

Sinh

Sinh

Size

X

Not yet implemented.

Slice

X

Slice

Softmax

Reshape, Exp, Div2, Sub2, Sum, Max

Softplus

SoftPlus

Softsign

SoftSign

SpaceToDepth

Reshape, Transpose

Split

Stack, Split

SplitToSequence

X

Not yet implemented.

Sqrt

PowScalar

Squeeze

Reshape

StringNormalizer

X

Not yet implemented.

Sub

Sub2, Reshape

Sum

Add2

Tan

Tan

Tanh

Tanh

TfIdfVectorizer

X

Not yet implemented.

ThresholdedRelu

GreaterScalar, Where, Constant

Tile

Tile

TopK

X

X

X

Not yet implemented.

Transpose

Transpose

Unique

X

Not yet implemented.

Unsqueeze

Reshape

Upsample

X

Unpooling

Where

Where

Xor

Reshape, LogicalXor

Export

  • ✓: Support to export this opset.

  • △: Partially support to export this opset (e.g. some cases cannot be supported, or not completely tested).

  • X: Supported, but test failed.

  • Empty: Not support corresponding opset version.

Total: 119/173

Neural Network Layer

Count 11/14

NNabla Function

6

7

9

10

11

ONNX Op

Description

Affine

Gemm, Reshape

RNN

Not yet implemented.

LSTM

Not yet implemented.

GRU

Not yet implemented.

Convolution

Reshape, Conv

DepthwiseConvolution

Reshape, Conv

Deconvolution

ConvTranspose, Reshape

Caffe2 and onnxruntime do not support dilations != 1.

DepthwiseDeconvolution

ConvTranspose, Reshape

Caffe2 and onnxruntime do not support dilations != 1.

MaxPooling

X

Reshape, Pad, MaxPool

AveragePooling

X

Reshape, Pad, AveragePool

Currently only supports the cases where both ignore_border and including_pad are True.

GlobalAveragePooling

GlobalAveragePool

SumPooling

X

X

Reshape, Pad, Constant, Mul, AveragePool

Unpooling

Resize

The kernel only supports 2d on opset 6.

Embed

Gather

Neural Network Activation Functions

Count 21/21

NNabla Function

6

7

9

10

11

ONNX Op

Description

Sigmoid

Sigmoid

Swish

Mul, Sigmoid

Tanh

Tanh

ReLU

Relu

LeakyReLU

LeakyRelu

Softmax

Exp, ReduceMax, Sub, ReduceSum, Div

ONNX Add, Sub operator does not support multidirectional broadcasting on opset 6.

LogSoftmax

Exp, ReduceMax, Sub, Log, ReduceSum

ELU

Elu

SELU

Selu

CReLU

Concat, Neg, Relu

CELU

Elu, Concat, Neg

PReLU

Reshape, PRelu

GELU

Sqrt, Constant, Pow, Tanh, Mul, Add, Div

ReLU6

Min, Relu, Constant

HardSigmoid

HardSigmoid

HardTanh

Neg, Min, Max, Constant

LogSigmoid

Sigmoid, Log

SoftPlus

Softplus

SoftSign

Softsign

TanhShrink

Sub, Tanh

Sinc

X

X

Constant, Where, Sin, Equal, Div

Normalization

Count 1/6

NNabla Function

6

7

9

10

11

ONNX Op

Description

FusedBatchNormalization

X

X

X

X

X

Relu, Constant, ReduceMean, BatchNormalization, Mul, Sub, ReduceSum, Div

Not yet implemented.

BatchNormalization

Reshape, Constant, ReduceMean, BatchNormalization, Mul, Sub, ReduceSum, Div

In inferring stage, caffe2 mistmatch onnx 1.4.x’s implementation, “in-place” feature cannot be applied.

SyncBatchNormalization

Not yet implemented.

MeanSubtraction

Not yet implemented.

ClipGradByValue

Not yet implemented.

ClipGradByNorm

Not yet implemented.

Reduction

Count 5/7

NNabla Function

6

7

9

10

11

ONNX Op

Description

Sum

ReduceSum

Mean

ReduceMean

Max

ReduceMax

Min

ReduceMin

Prod

ReduceProd

ReduceSum

Not yet implemented.

ReduceMean

Not yet implemented.

Arithmetic

Count 11/12

NNabla Function

6

7

9

10

11

ONNX Op

Description

Add2

Add

ONNX Add operator does not support multidirectional broadcasting on opset 6.

BcAdd2

Not yet implemented.

Sub2

Sub

ONNX Sub operator does not support multidirectional broadcasting on opset 6.

Mul2

Mul

ONNX Mul operator does not support multidirectional broadcasting on opset 6.

Div2

Div

ONNX Div operator does not support multidirectional broadcasting on opset 6.

Pow2

Pow

ONNX Pow operator does not support multidirectional broadcasting on opset 6.

AddScalar

Add, Constant

MulScalar

Mul, Constant

PowScalar

Pow, Constant

RSubScalar

Sub, Constant

RDivScalar

Constant, Div

RPowScalar

Pow, Constant

Logical

Count 29/29

NNabla Function

6

7

9

10

11

ONNX Op

Description

Sign

X

X

Sign

Minimum2

Min, Add, Constant

ONNX Add operator does not support multidirectional broadcasting on opset 6.

Maximum2

Add, Max, Constant

ONNX Add operator does not support multidirectional broadcasting on opset 6.

MinimumScalar

Min, Add, Constant

MaximumScalar

Add, Max, Constant

LogicalAnd

And

LogicalOr

Or

LogicalXor

Xor

Equal

Equal

NotEqual

Equal, Not

GreaterEqual

Less, Not

Greater

Greater

LessEqual

Greater, Not

Less

Less

LogicalAndScalar

And, Constant

LogicalOrScalar

Or, Constant

LogicalXorScalar

Xor, Constant

EqualScalar

Equal, Constant

NotEqualScalar

Equal, Not, Constant

GreaterEqualScalar

Less, Not, Constant

GreaterScalar

Greater, Constant

LessEqualScalar

Greater, Not, Constant

LessScalar

Less, Constant

LogicalNot

Not

IsNaN

X

X

IsNaN

IsInf

X

X

X

IsInf

ResetNaN

X

X

Where, IsNaN, Constant

ResetInf

X

X

X

Where, IsInf, Constant

Where

X

X

Where

Math

Count 22/22

NNabla Function

6

7

9

10

11

ONNX Op

Description

Constant

Identity, Constant

Arange

Identity, Constant

Abs

Abs

Exp

Exp

Log

Log

Identity

Identity

BatchMatmul

Reshape, MatMul, Transpose

Round

X

X

X

X

Round

Ceil

Ceil

Floor

Floor

Sin

X

Sin

Cos

X

Cos

Tan

X

Tan

Sinh

X

X

Sinh

Cosh

X

X

Cosh

ASin

X

Asin

ACos

X

Acos

ATan

X

Atan

ATan2

X

Div, Atan

ASinh

X

X

Asinh

ACosh

X

X

Acosh

ATanh

X

X

Atanh

Array Manipulation

Count 12/19

NNabla Function

6

7

9

10

11

ONNX Op

Description

Concatenate

Concat

Split

Split, Squeeze

Stack

Concat, Unsqueeze

Slice

Slice, Constant

ONNX slice cannot support step != 1 on opset < 10.

Pad

Pad, Constant

When the mode of the pad is reflect, if the size of the pad exceeds the input size, caffe2 and onnxruntime cannot handle it.

Transpose

Transpose

Broadcast

X

X

BroadcastTo

Tile

Reshape, Tile, Constant

OneHot

Gather, Reshape, Flatten

Flip

Gather, Transpose, Identity

Shift

Not yet implemented.

Sort

Not yet implemented.

Reshape

Reshape, Constant

MatrixDiag

Not yet implemented.

MatrixDiagPart

Not yet implemented.

Assign

Not yet implemented.

GatherNd

Not yet implemented.

ScatterNd

Not yet implemented.

Signal Processing

Count 1/3

NNabla Function

6

7

9

10

11

ONNX Op

Description

Interpolate

X

X

X

X

Reshape, Resize

FFT

Not yet implemented.

IFFT

Not yet implemented.

Stochasticity

Count 0/11

NNabla Function

6

7

9

10

11

ONNX Op

Description

Dropout

X

X

X

X

X

Dropout

The Dropout in nnabla has no test mode and contains random parameters, so the test result is not the same as onnx.

TopKData

Not yet implemented.

TopKGrad

Not yet implemented.

Rand

Not yet implemented.

Randint

Not yet implemented.

Randn

Not yet implemented.

RandomChoice

Not yet implemented.

RandomCrop

Not yet implemented.

RandomFlip

Not yet implemented.

RandomShift

Not yet implemented.

ImageAugmentation

Not yet implemented.

Loss Functions

Count 0/9

NNabla Function

6

7

9

10

11

ONNX Op

Description

SigmoidCrossEntropy

Not yet implemented.

BinaryCrossEntropy

Not yet implemented.

SoftmaxCrossEntropy

Not yet implemented.

CategoricalCrossEntropy

Not yet implemented.

SquaredError

Not yet implemented.

AbsoluteError

Not yet implemented.

HuberLoss

Not yet implemented.

EpsilonInsensitiveLoss

Not yet implemented.

KLMultinomial

Not yet implemented.

Quantization Neural Network Layers

Count 6/12

NNabla Function

6

7

9

10

11

ONNX Op

Description

BinarySigmoid

X

X

Where, Greater, Constant

BinaryTanh

X

X

Where, Greater, Constant

BinaryConnectAffine

Gemm, Reshape

BinaryConnectConvolution

Reshape, Conv

BinaryWeightAffine

Mul, Reshape, Add, MatMul

BinaryWeightConvolution

Mul, Reshape, Conv, Add

INQAffine

Not yet implemented.

INQConvolution

Not yet implemented.

FixedPointQuantize

Not yet implemented.

MinMaxQuantize

Not yet implemented.

Pow2Quantize

Not yet implemented.

Prune

Not yet implemented.

Validation

Count 0/3

NNabla Function

6

7

9

10

11

ONNX Op

Description

TopNError

Not yet implemented.

BinaryError

Not yet implemented.

ConfusionMatrix

Not yet implemented.

Unsupported, Special Use

Count 0/5

NNabla Function

6

7

9

10

11

ONNX Op

Description

VATNoise

Not yet implemented.

Unlink

Not yet implemented.

Sink

Not yet implemented.

NmsDetection2d

Not yet implemented.

MaxPoolingBackward

Not yet implemented.

Tensorflow Support Status

Note

In this document, the numbers in the header of all tables represent the version of onnx opset.

Import

  • ✓: Supported

  • △: Partially supported

  • X: Supported, but test failed.

  • Empty: Not support yet.

Total: 86/120

Tensorflow support status

Tensorflow Function

Status

NNabla Func

Description

Abs

Abs

Acos

ACos

Acosh

ACosh

Add

Add2

AddN

Add2

All

Not yet implemented.

Any

Not yet implemented.

ArgMax

Max

ArgMin

Min

Asin

ASin

Asinh

ASinh

Atan

ATan

Atan2

Not yet implemented.

Atanh

ATanh

AvgPool

Pad, Transpose, AveragePooling

Some feature is not supported by Nnabla such as Pad’s edge mode.

AvgPool3D

Not yet implemented.

BatchMatMul

BatchMatmul, Transpose

BiasAdd

Reshape, Add2

Cast

Not yet implemented.

Ceil

Ceil

ConcatV2

Concatenate

Const

Add2

Conv2D

Pad, Convolution, Transpose

Tensorflow require GPU to perform related test cases. This issue is recorded only for memo.

Conv2DBackpropFilter

Not yet implemented.

Conv2DBackpropInput

Deconvolution, Transpose

Tensorflow require GPU to perform related test cases. This issue is recorded only for memo.

Conv3D

Not yet implemented.

Conv3DBackpropFilterV2

Not yet implemented.

Conv3DBackpropInputV2

Not yet implemented.

Cos

Cos

Cosh

Cosh

DepthToSpace

Reshape, Transpose

Tensorflow require GPU to perform related test cases. This issue is recorded only for memo.

DepthwiseConv2dNative

Not yet implemented.

DepthwiseConv2dNativeBackpropFilter

Not yet implemented.

DepthwiseConv2dNativeBackpropInput

Not yet implemented.

Div

Div2

Elu

ELU

Equal

Equal

Erf

Not yet implemented.

Erfc

Not yet implemented.

Exp

Exp

ExpandDims

Reshape

Fill

Not yet implemented.

Flatten

Reshape

Floor

Floor

FloorDiv

Div2, Floor

FloorMod

Div2, Sub2, Mul2, Floor

FusedBatchNorm

BatchNormalization, Transpose

It did not pass testing for training mode.

GatherNd

Not yet implemented.

GatherV2

Not yet implemented.

Greater

Greater

GreaterEqual

Less, LogicalNot

Identity

Identity

IsInf

Not yet implemented.

IsNan

IsNaN

LeakyRelu

LeakyReLU

Less

Less

LessEqual

Greater, LogicalNot

Log

Log

LogSoftmax

Not yet implemented.

LogicalAnd

LogicalAnd

LogicalNot

LogicalNot

LogicalOr

LogicalOr

LogicalXor

LogicalAnd, LogicalOr, LogicalNot

MatrixBandPart

Not yet implemented.

Max

Max

MaxPool

Transpose, Pad, MaxPooling

MaxPool3D

Not yet implemented.

MaxPoolWithArgmax

Not yet implemented.

Maximum

Maximum2

Mean

Mean

Min

Min

Minimum

Minimum2

Mul

Mul2

Neg

MulScalar

NotEqual

Equal, LogicalNot

OneHot

Not yet implemented.

Pack

Reshape, Concatenate

Pad

Pad

Pow

Pow2

Prod

Prod

RandomShuffle

Not yet implemented.

RandomStandardNormal

Not yet implemented.

RandomUniform

Not yet implemented.

RealDiv

Div2

Reciprocal

RDivScalar

Relu

ReLU

Relu6

MinimumScalar, MaximumScalar

Reshape

Reshape

Some test cases failed for some nnabla’s implementation limitation (e.g. -1 is regarded as batch_size).

ReverseSequence

Not yet implemented.

Rsqrt

PowScalar, RDivScalar

Select

Not yet implemented.

Selu

SELU

Shape

Not yet implemented.

Sigmoid

Sigmoid

Sign

Sign

Sin

Sin

Sinh

Sinh

Size

Not yet implemented.

Slice

Slice

Softmax

Not yet implemented.

Softplus

SoftPlus

Softsign

SoftSign

SpaceToDepth

Reshape, Transpose

Tensorflow require GPU to perform related test cases. This issue is recorded only for memo.

SplitV

Stack, Split

Sqrt

PowScalar

Square

Mul2

SquaredDifference

Sub2, Mul2

Squeeze

Reshape

StopGradient

Identity

StridedSlice

Slice

Sub

Sub2

Sum

Sum

Tan

Tan

Tanh

Tanh

Tile

Tile

TopKV2

Not yet implemented.

Transpose

Transpose

TruncateDiv

Not yet implemented.

TruncateMod

Not yet implemented.

Unpack

Reshape, Stack, Split, Concatenate

Export

  • ✓: Supported

  • △: Partially supported

  • X: Supported, but test failed.

  • Empty: Not support yet.

Total: 115/173

Neural Network Layer

Count 11/14

NNabla Function

Status

TF Op

Description

Affine

Reshape, Placeholder, MatMul, Const, Mul, Add

RNN

Not yet implemented.

LSTM

Not yet implemented.

GRU

Not yet implemented.

Convolution

Reshape, SpaceToBatchND, Pad, Placeholder, Transpose, Const, Add, Split, Identity, ConcatV2, Conv2D, BatchToSpaceND

The cases dilations and strides larger than 1 are not supported by tensorflow.

DepthwiseConvolution

Conv2D, Reshape, SpaceToBatchND, Pad, Placeholder, Transpose, Const, Split, ConcatV2, Add, BatchToSpaceND

The cases dilations and strides larger than 1 are not supported by tensorflow.

Deconvolution

Reshape, Slice, Placeholder, Transpose, Const, Split, Identity, ConcatV2, Conv2DBackpropInput, Add

The cases dilations larger than 1 are not supported by tensorflow.

DepthwiseDeconvolution

Reshape, Slice, Placeholder, Transpose, Const, Split, ConcatV2, Conv2DBackpropInput, Add

The cases dilations larger than 1 are not supported by tensorflow.

MaxPooling

Reshape, PadV2, Placeholder, Transpose, Const, MaxPool, MaxPool3D

AveragePooling

AvgPool3D, Reshape, Pad, Placeholder, Transpose, Const, AvgPool

Currently only supports the cases both ignore_border and including_pad are True.

GlobalAveragePooling

Const, Pack, Sub, SplitV, Range, Mean

SumPooling

AvgPool3D, Reshape, Pad, Placeholder, Transpose, Const, Mul, AvgPool

Unpooling

Reshape, Placeholder, Transpose, ResizeNearestNeighbor, Const, LogicalAnd, Merge, Switch, Identity, Assert, Mul, Equal, StridedSlice, NoOp, Cast

The kernel only supports 2d.

Embed

GatherV2, Placeholder, Const

Neural Network Activation Functions

Count 21/21

NNabla Function

Status

TF Op

Description

Sigmoid

Sigmoid, Placeholder

Swish

Mul, Sigmoid, Placeholder

Tanh

Placeholder, Tanh

ReLU

Relu, Placeholder

LeakyReLU

LeakyRelu, Placeholder

Softmax

Placeholder, Const, Exp, Sub, RealDiv, Sum, Max

LogSoftmax

Placeholder, Const, Exp, Sub, Log, Sum, Max

ELU

Less, Placeholder, Const, Elu, Exp, Mul, GreaterEqual, Sub, Cast, Add

SELU

Minimum, Placeholder, Const, Exp, Mul, Sub, Maximum, Add

CReLU

Relu, Placeholder, Const, ConcatV2, Neg

CELU

Less, Placeholder, Const, Elu, Exp, ConcatV2, Mul, GreaterEqual, Sub, Neg, Cast, Add

PReLU

Reshape, Relu, Placeholder, Const, Mul, Sub, Abs, Add

GELU

Sqrt, Placeholder, Const, Pow, Tanh, Mul, RealDiv, Add

ReLU6

Relu, Placeholder, Const, Pack, Min

HardSigmoid

Minimum, Placeholder, Const, Mul, Maximum, Add

HardTanh

Placeholder, Const, Pack, Neg, Min, Max

LogSigmoid

Sigmoid, Log, Placeholder

SoftPlus

Softplus, Placeholder

SoftSign

Placeholder, Softsign

TanhShrink

Sub, Placeholder, Tanh

Sinc

Placeholder, Select, Const, Sin, Equal, RealDiv

Normalization

Count 1/6

NNabla Function

Status

TF Op

Description

FusedBatchNormalization

X

Reshape, Relu, Placeholder, Const, Mul, Rsqrt, Sub, Mean, RealDiv, Sum, Add

Not yet implemented.

BatchNormalization

Reshape, Placeholder, Const, Mul, Rsqrt, Sub, Mean, RealDiv, Sum, Add

In inferring stage, caffe2 mistmatch onnx 1.4.x’s implementation, “in-place” feature cannot be applied.

SyncBatchNormalization

Not yet implemented.

MeanSubtraction

Not yet implemented.

ClipGradByValue

Not yet implemented.

ClipGradByNorm

Not yet implemented.

Reduction

Count 5/7

NNabla Function

Status

TF Op

Description

Sum

Sum, Placeholder, Const

Mean

Mean, Placeholder, Const

Max

Placeholder, Max, Const

Min

Min, Placeholder, Const

Prod

Prod, Placeholder, Const

ReduceSum

Not yet implemented.

ReduceMean

Not yet implemented.

Arithmetic

Count 11/12

NNabla Function

Status

TF Op

Description

Add2

Placeholder, Add

BcAdd2

Not yet implemented.

Sub2

Sub, Placeholder

Mul2

Mul, Placeholder

Div2

RealDiv, Placeholder

Pow2

Pow, Placeholder

AddScalar

Placeholder, Add, Const

MulScalar

Mul, Placeholder, Const

PowScalar

Pow, Placeholder, Const

RSubScalar

Sub, Placeholder, Const

RDivScalar

RealDiv, Placeholder, Const

RPowScalar

Pow, Placeholder, Const

Logical

Count 27/29

NNabla Function

Status

TF Op

Description

Sign

Placeholder, Sign

Minimum2

Placeholder, Const, Pack, Min, Add

Maximum2

Placeholder, Add, Const, Pack, Max

MinimumScalar

Placeholder, Const, Pack, Min, Add

MaximumScalar

Placeholder, Add, Const, Pack, Max

LogicalAnd

LogicalAnd, Placeholder

LogicalOr

LogicalOr, Placeholder

LogicalXor

LogicalAnd, LogicalOr, Placeholder, LogicalNot

Equal

Equal, Placeholder

NotEqual

LogicalNot, Equal, Placeholder

GreaterEqual

Less, Placeholder, LogicalNot

Greater

Greater, Placeholder

LessEqual

Greater, Placeholder, LogicalNot

Less

Less, Placeholder

LogicalAndScalar

LogicalAnd, Placeholder, Const

LogicalOrScalar

LogicalOr, Placeholder, Const

LogicalXorScalar

Placeholder, Const, LogicalAnd, LogicalOr, LogicalNot

EqualScalar

Equal, Placeholder, Const

NotEqualScalar

LogicalNot, Equal, Placeholder, Const

GreaterEqualScalar

Less, Placeholder, LogicalNot, Const

GreaterScalar

Greater, Placeholder, Const

LessEqualScalar

Greater, Placeholder, LogicalNot, Const

LessScalar

Less, Placeholder, Const

LogicalNot

Placeholder, LogicalNot

IsNaN

IsNan, Placeholder

IsInf

X

Not yet implemented.

ResetNaN

IsNan, Placeholder, Select, Const

ResetInf

X

Not yet implemented.

Where

Placeholder, Select

Math

Count 21/22

NNabla Function

Status

TF Op

Description

Constant

Identity, Const

Arange

Identity, Const

Abs

Abs, Placeholder

Exp

Exp, Placeholder

Log

Log, Placeholder

Identity

Placeholder, Identity

BatchMatmul

Reshape, Placeholder, Transpose, Const, BatchMatMulV2

Round

X

Not yet implemented.

Ceil

Ceil, Placeholder

Floor

Floor, Placeholder

Sin

Sin, Placeholder

Cos

Cos, Placeholder

Tan

Tan, Placeholder

Sinh

Sinh, Placeholder

Cosh

Cosh, Placeholder

ASin

Asin, Placeholder

ACos

Acos, Placeholder

ATan

Placeholder, Atan

ATan2

RealDiv, Placeholder, Atan

ASinh

Asinh, Placeholder

ACosh

Acosh, Placeholder

ATanh

Atanh, Placeholder

Array Manipulation

Count 12/19

NNabla Function

Status

TF Op

Description

Concatenate

ConcatV2, Placeholder, Const

Split

Squeeze, SplitV, Placeholder, Const

Stack

ConcatV2, ExpandDims, Placeholder, Const

Slice

Slice, Placeholder, Const

step != 1” exceed the scope of onnx opset 9, not supported.

Pad

MirrorPad, PadV2, Placeholder, Const

When the mode of the pad is reflect, if the size of the pad exceeds the input size, tensorflow cannot handle it.

Transpose

Placeholder, Transpose, Const

Broadcast

BroadcastTo

Tile

Reshape, Tile, Placeholder, Const

OneHot

Reshape, GatherV2, Placeholder, Const

Flip

Placeholder, Transpose, Const, Identity, GatherV2

Shift

Not yet implemented.

Sort

Not yet implemented.

Reshape

Reshape, Placeholder, Const

MatrixDiag

Not yet implemented.

MatrixDiagPart

Not yet implemented.

Assign

Not yet implemented.

GatherNd

Not yet implemented.

ScatterNd

Not yet implemented.

Signal Processing

Count 0/3

NNabla Function

Status

TF Op

Description

Interpolate

X

Not yet implemented.

FFT

Not yet implemented.

IFFT

Not yet implemented.

Stochasticity

Count 0/11

NNabla Function

Status

TF Op

Description

Dropout

X

Placeholder

The Dropout in nnabla has no test mode and contains random parameters, so the test result is not the same as tensorflow.

TopKData

Not yet implemented.

TopKGrad

Not yet implemented.

Rand

Not yet implemented.

Randint

Not yet implemented.

Randn

Not yet implemented.

RandomChoice

Not yet implemented.

RandomCrop

Not yet implemented.

RandomFlip

Not yet implemented.

RandomShift

Not yet implemented.

ImageAugmentation

Not yet implemented.

Loss Functions

Count 0/9

NNabla Function

Status

TF Op

Description

SigmoidCrossEntropy

Not yet implemented.

BinaryCrossEntropy

Not yet implemented.

SoftmaxCrossEntropy

Not yet implemented.

CategoricalCrossEntropy

Not yet implemented.

SquaredError

Not yet implemented.

AbsoluteError

Not yet implemented.

HuberLoss

Not yet implemented.

EpsilonInsensitiveLoss

Not yet implemented.

KLMultinomial

Not yet implemented.

Quantization Neural Network Layers

Count 6/12

NNabla Function

Status

TF Op

Description

BinarySigmoid

Greater, Placeholder, Select, Const

BinaryTanh

Greater, Placeholder, Select, Const

BinaryConnectAffine

Reshape, Placeholder, MatMul, Const, Mul, Add

BinaryConnectConvolution

Conv2D, Reshape, Pad, Placeholder, Transpose, Const, Split, Identity, ConcatV2, Add

The cases dilations and strides larger than 1 are not supported by tensorflow.

BinaryWeightAffine

Reshape, Placeholder, MatMul, Const, Mul, Add

BinaryWeightConvolution

Conv2D, Reshape, Pad, Placeholder, Transpose, Const, Split, Identity, Mul, ConcatV2, Add

The cases dilations and strides larger than 1 are not supported by tensorflow.

INQAffine

Not yet implemented.

INQConvolution

Not yet implemented.

FixedPointQuantize

Not yet implemented.

MinMaxQuantize

Not yet implemented.

Pow2Quantize

Not yet implemented.

Prune

Not yet implemented.

Validation

Count 0/3

NNabla Function

Status

TF Op

Description

TopNError

Not yet implemented.

BinaryError

Not yet implemented.

ConfusionMatrix

Not yet implemented.

Unsupported, Special Use

Count 0/5

NNabla Function

Status

TF Op

Description

VATNoise

Not yet implemented.

Unlink

Not yet implemented.

Sink

Not yet implemented.

NmsDetection2d

Not yet implemented.

MaxPoolingBackward

Not yet implemented.

NNabla C Runtime Support Status

nnabla version: 1.0.21

  • ✓: Supported

  • △: Partially supported

  • X: Supported, but test failed or no test data.

  • Empty: Not support yet.

Export

Total: 56/173

Neural Network Layer

Count 8/14

NNabla Function

Status

Description

Affine

RNN

LSTM

GRU

Convolution

DepthwiseConvolution

Deconvolution

DepthwiseDeconvolution

MaxPooling

AveragePooling

GlobalAveragePooling

SumPooling

Unpooling

Embed

Neural Network Activation Functions

Count 11/21

NNabla Function

Status

Description

Sigmoid

Swish

Tanh

ReLU

LeakyReLU

Softmax

LogSoftmax

ELU

SELU

CReLU

CELU

PReLU

GELU

ReLU6

HardSigmoid

HardTanh

LogSigmoid

SoftPlus

SoftSign

TanhShrink

Sinc

Normalization

Count 1/6

NNabla Function

Status

Description

FusedBatchNormalization

BatchNormalization

SyncBatchNormalization

MeanSubtraction

X

ClipGradByValue

ClipGradByNorm

Reduction

Count 1/7

NNabla Function

Status

Description

Sum

Mean

Max

Min

Prod

ReduceSum

ReduceMean

Arithmetic

Count 11/12

NNabla Function

Status

Description

Add2

BcAdd2

Sub2

Mul2

Div2

Pow2

AddScalar

MulScalar

PowScalar

RSubScalar

RDivScalar

RPowScalar

Logical

Count 5/29

NNabla Function

Status

Description

Sign

Minimum2

Maximum2

MinimumScalar

MaximumScalar

LogicalAnd

LogicalOr

LogicalXor

Equal

NotEqual

GreaterEqual

Greater

LessEqual

Less

LogicalAndScalar

LogicalOrScalar

LogicalXorScalar

EqualScalar

NotEqualScalar

GreaterEqualScalar

GreaterScalar

LessEqualScalar

LessScalar

LogicalNot

IsNaN

IsInf

ResetNaN

ResetInf

Where

Math

Count 6/22

NNabla Function

Status

Description

Constant

Arange

Abs

Exp

Log

Identity

BatchMatmul

Round

Ceil

Floor

Sin

Cos

Tan

Sinh

Cosh

ASin

ACos

ATan

ATan2

ASinh

ACosh

ATanh

Array Manipulation

Count 7/19

NNabla Function

Status

Description

Concatenate

Split

Stack

Slice

Pad

Transpose

Broadcast

BroadcastTo

Tile

OneHot

Flip

Shift

X

Sort

Reshape

MatrixDiag

X

MatrixDiagPart

X

Assign

GatherNd

ScatterNd

Signal Processing

Count 0/3

NNabla Function

Status

Description

Interpolate

FFT

IFFT

Stochasticity

Count 0/11

NNabla Function

Status

Description

Dropout

X

TopKData

TopKGrad

Rand

Randint

Randn

RandomChoice

RandomCrop

RandomFlip

RandomShift

ImageAugmentation

Loss Functions

Count 0/9

NNabla Function

Status

Description

SigmoidCrossEntropy

BinaryCrossEntropy

SoftmaxCrossEntropy

CategoricalCrossEntropy

SquaredError

AbsoluteError

HuberLoss

EpsilonInsensitiveLoss

KLMultinomial

Quantization Neural Network Layers

Count 6/12

NNabla Function

Status

Description

BinarySigmoid

BinaryTanh

BinaryConnectAffine

BinaryConnectConvolution

BinaryWeightAffine

BinaryWeightConvolution

INQAffine

INQConvolution

FixedPointQuantize

MinMaxQuantize

Pow2Quantize

Prune

Validation

Count 0/3

NNabla Function

Status

Description

TopNError

BinaryError

ConfusionMatrix

Unsupported, Special Use

Count 0/5

NNabla Function

Status

Description

VATNoise

Unlink

Sink

NmsDetection2d

MaxPoolingBackward