Debug Utils

Graph Profiler

class nnabla.utils.profiler.GraphProfiler(graph, device_id, ext_name, solver=None, n_run=100, max_measure_execution_time=1, time_scale='m', backward_accum=False)

ネットワークに含まれるそれぞれの関数の計算時間を計測するためのクラスです。

以下の項目について、実行時間を計測することができます。

• 関数ごとのforward演算実行速度

• 関数ごとのbackward演算実行速度

• グラフ全体のforward演算実行速度

• グラフ全体のbackward演算実行速度

• 学習実行速度 ( forward + backward + update演算実行にかかる合計時間) ( solver が None でない場合)

例:

import nnabla as nn
import nnabla.functions as F
import nnabla.solvers as S
from nnabla.utils.profiler import GraphProfiler

# Set up nnabla context
device = "cpu"  # you can also use GPU ("cudnn")
ctx = get_extension_context(device)
nn.set_default_context(ctx)

# Network building
x = nn.Variable(shape=...)
t = nn.Variable(shape=...)
y = CNN(x) # you can build not only CNN but any networks
loss = F.mean(F.softmax_cross_entropy(y, t)) # any loss functions or variables can be used

# solver setting
solver = S.Sgd()
solver.set_parameters(nn.get_parameters())

# SOME CODE (data loading or so on)

B = GraphProfiler(loss, solver=solver, device_id=0, ext_name=device, n_run=1000)
B.run()

パラメータ:

• graph (nnabla.Variable) -- nnabla.Variable インスタンス。 GraphProfiler は、ルートとなる nnabla.Variable からこの 引数として与えられた nnabla.Variable までに含まれるすべての関数の実行時間を計測します。

• device_id (str) -- GPU デバイスID。

• ext_name (str) -- nnablaのctxとして指定できるコンテキスト名。指定されたctx下での実行速度が計測されます。例: ‘cpu’, ‘cuda’, ‘cudnn’ など。

• solver (nnabla.solvers.Solver) -- nnabla.solvers.Solver のインスタンス。None が与えられた場合には、学習プロセスは計測されません。デフォルトは None です。

• n_run (int) -- それぞれの関数に対して実行時間を計測する回数。デフォルトは 100 です。

• max_measure_execution_time (float) -- それぞれの関数の実行時間を計測する最大の時間。この引数は n_run よりも優先されます。すなわち、それぞれの関数での計測時間がこの引数より大きくなる場合、計測した回数が n_run 未満であっても、現在の関数に対する計測を中止して次の関数に進みます。デフォルトは 1 [sec] です。

• time_scale (str) -- 表示する時間スケール。[‘m’, ‘u’, ‘n’] (それぞれ ‘ミリ’, ‘マイクロ’, ‘ナノ’ の略記号 )

• backward_accum (bool) -- Accumulation flag passed to the each backward function. The flag will fulfill the all accumulation flags with the same value of backward_accum. This flag is only valid for the time measurement of each function. For whole graph computation, the NNabla graph engine set the appropriate accumulation flags to functions. Pay attention to inplace flag for your graph because accumulation and inplace flags cannot be set at the same time. If even one inplace flag is true in your graph, this backward_accum must be false. Default value is False.

run()

プロファイリングを実行します。

以下の5つの項目についての実行時間を計測します。

• 関数ごとのforward演算実行速度

• 関数ごとのbackward演算実行速度

• グラフ全体のforward演算実行速度

• グラフ全体のbackward演算実行速度

• 学習実行速度 ( forward + backward + update演算実行にかかる合計時間) ( solver が None でない場合)

class nnabla.utils.profiler.GraphProfilerCsvWriter(gb, file=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>)

GraphProfiler によって計測した時間をcsvファイルとして書き出すためのクラスです。

例:

from nnabla.utils.profiler import GraphProfiler, GraphProfilerCsvWriter

# Network building comes above

B = GraphProfiler(variable, solver=solver, device_id=0, ext_name=device, n_run=1000)
B.run()

with open("./profile.csv", "w") as f:
    writer = GraphProfilerCsvWriter(B, file=f)
    writer.write()

パラメータ:

• gb (GraphProfiler) -- GraphProfiler クラスのインスタンス。

• file (Python file object) -- 出力ファイルオブジェクト。この引数で指定されたファイルにプロファイリングの結果を書き込みます。

write()

指定された file に計測結果を出力します。

Time Profiler

class nnabla.utils.inspection.profile.TimeProfiler(ext_name, device_id)

An utility API to create function_hook callbacks to profile the execution time of each function. Passing ext_name and device_id, you can define which device time you want to profile. If ext_name = "cuda" or "cudnn", then cudaEvent will be used to measure the execution time. For more information about cudaEvent, see the CUDA document. If `ext_name`="cpu" , then wall-clock-time on host will be used.

例:

ext_name = "cpu"
device_id = "0"

from nnabla.ext_utils import get_extension_context
ctx = get_extension_context(ext_name, device_id=device_id)
nn.set_default_context(ctx)

y = model(...)

from nnabla.utils.inspection import TimeProfiler
tp = TimeProfiler(ext_name=ext_name, device_id=device_id)

for i in range(max_iter):
    # All results of executions under "forward" scope are registered as "forward" execution.
    with tp.scope("forward"):
        y.forward(function_pre_hook=tp.pre_hook, function_post_hook=tp.post_hook)

    # All results of executions under "backward" scope are registered as "backward" execution.
    with tp.scope("backward") as tp:
        y.backward(function_pre_hook=tp.pre_hook, function_post_hook=tp.post_hook)

    # All results are evaluated by passing scopes to .calc_elapsed_time().
    # Be sure to call calc_elapsed_time at each iteration, otherwise nothing is measured.
    tp.calc_elapsed_time(["forward", "backward", "summary"])

# To output results on stdout, call instance as a function.
tp()

# To write out as csv file, call .to_csv().
tp.to_csv(output_file_name)

calc_elapsed_time(names=None)

Evaluate all elapsed times. Note that elapsed time is not recorded until calc_elapsed_time is called.

パラメータ:

names (str or list of str) -- Scope name(s) to evaluate elapsed time.

property post_hook

Get a callback for function_post_hook. This function can be used like the example below:

tp = TimeProfiler(..)
with tp.scope("forward"):
    v.forward(function_post_hook=tp.post_hook())

with tp.scope("backward"):
    v.backward(function_post_hook=tp.post_hook())

property pre_hook

Get a callback for function_pre_hook. This function can be used like the example below:

tp = TimeProfiler(..)
with tp.scope("forward"):
    v.forward(function_pre_hook=tp.pre_hook())

with tp.scope("backward"):
    v.backward(function_pre_hook=tp.pre_hook())

scope(scope_name)

Change a scope to aggregate results. This function is used as context (The with statement statement),

and all results under the context are labeled by scope_name.

In addition to the execution time of each function, the elapsed times between entering and exiting the each context are also recorded

and they are aggregated as "summary" scope.

パラメータ:

scope_name (str) -- Scope name.

to_csv(out_dir='./', ignore_init=True)

Writes out to csv file. Output directory can be specified by out_dir. As default, the elapsed times of first iteration will be omitted. If you evaluate the first iteration as well, pass True to ignore_init.

パラメータ:

• out_dir (str) -- Output directory.

• ignore_init (bool) -- Ignore the result of the first iteration or not.

Nan/Inf Tracer

class nnabla.utils.inspection.value_trace.NanInfTracer(trace_nan=True, trace_inf=True, need_details=True)

An utility API to create function_hook callbacks to check whether the outputs of all layers have NaN or inf as their values. During forward and backward execution, passed as function_hook, this API reports ValueError if at least one of all layer outputs has Nan or inf as its values. Otherwise, all tensors passed to next layer or function as is.

例:

pred = model(...)

from nnabla.utils.inspection import NanInfTracer
nit = NanInfTracer(trace_inf=True, trace_nan=True, need_details=True)

with nit.trace():
    pred.forward(function_post_hook=nit.forward_post_hook)
    pred.backward(function_post_hook=nit.backward_post_hook)

property backward_post_hook

Create callback function object which can be used as a function_post_hook argument of backward().

check()

Checks nan/inf existence at all outputs of all layers and raises ValueError only if exist.

property forward_post_hook

Create callback function object which can be used as a function_post_hook argument of forward().

trace()

Create context manager to check nan/inf existence by using with statement. Using this context manager, checking nan/inf is performed automatically just before exiting with scope. Unless you use this context manager, be sure to call .check() explicitly to check nan/inf.

例:

nit = NanInfTracer()
with nit.trace():
    pred.forward(function_post_hook=nit.forward_post_hook)
    pred.backward(function_post_hook=nit.backward_post_hook)

Pretty Printer

class nnabla.utils.inspection.pretty_print.PrettyPrinter(summary=False, hidden=False)

Pretty printer to print the graph structure used with the visit method of a Variable.

functions

List of functions of which element is the dictionary. The (key, value) pair is the (name, function name), (inputs, list of input variables), and (outputs, list of output variables) of a function.

Type:

list of dict

nnabla.utils.inspection.pretty_print.pprint(v, forward=False, backward=False, summary=False, hidden=False, printer=False)

Pretty print information of a graph from a root variable v.

Note that in order to print the summary statistics, this function stores, i.e., does not reuse the intermediate buffers of a computation graph, increasing the memory usage if either the forward or backward is True.

パラメータ:

• v (nnabla.Variable) -- Root variable.

• forward (bool) -- Call the forward method of a variable v.

• backward (bool) -- Call the backward method of a variable v.

• summary (bool) -- Print statictis of a intermediate variable.

• hidden (bool) -- Store the intermediate input and output variables if True.

• printer (bool) -- Return the printer object if True.

例:

pred = Model(...)

from nnabla.utils.inspection import pprint

pprint(pred, summary=True, forward=True, backward=True)