1. Getting started#
scope-profiler measures how long named regions of your code take, stores the raw per-call timings in HDF5, and gives you a Python API and CLI to analyse them.
This notebook covers the recording side:
configuring the profiler with
ProfileManager.setup()marking regions with a context manager and with a decorator
writing the results with
ProfileManager.finalize()a first look at the output file
Everything here runs in a single process — see 4. Profiling modes for MPI, LIKWID and the CLI.
[1]:
import tempfile
from pathlib import Path
from scope_profiler import ProfileManager
# Keep the tutorial's output out of your working directory.
WORKDIR = Path(tempfile.mkdtemp(prefix="scope-profiler-tutorial-"))
DATA_FILE = WORKDIR / "profiling_data.h5"
print(WORKDIR)
/tmp/scope-profiler-tutorial-j3uwu4sg
Configuring the profiler#
ProfileManager is a singleton: you never instantiate it, you just call class methods on it. setup() picks the recording strategy and the output path.
The defaults record wall-clock timings for every call and flush them to disk, so for most runs ProfileManager.setup(file_path=...) is all you need.
[2]:
ProfileManager.setup(file_path=str(DATA_FILE))
Marking a region with a context manager#
ProfileManager.profile_region(name) returns a region object you can use as a context manager. Entering it records a start timestamp, leaving it records the end — the region is created on first use and reused afterwards.
[3]:
import time
def load_data(n):
time.sleep(0.01)
return list(range(n))
def transform(values):
time.sleep(0.005)
return [value * 2 for value in values]
with ProfileManager.profile_region("load"):
values = load_data(1000)
for _ in range(3):
with ProfileManager.profile_region("transform"):
values = transform(values)
Each with block is one call. transform above was entered three times, so that region ends up with three recorded durations — the profiler keeps every call, not just an aggregate.
Marking a function with a decorator#
@ProfileManager.profile wraps a whole function. It works with or without parentheses, and takes an optional region name (the function’s __name__ is used otherwise).
[4]:
@ProfileManager.profile
def solve():
time.sleep(0.02)
@ProfileManager.profile("assemble_matrix")
def assemble():
time.sleep(0.008)
for _ in range(2):
assemble()
solve()
Decorators may be applied before setup() is called — at import time, for instance. setup() re-binds every registered decorator to the newly configured region class, so the decorated function always records with the current configuration and pays no per-call check for it.
Nesting regions#
Regions nest freely; the profiler records each one independently. Nesting is what the flame chart in 3. Visualizing results reconstructs.
[5]:
with ProfileManager.profile_region("timestep"):
for _ in range(2):
with ProfileManager.profile_region("timestep.residual"):
time.sleep(0.004)
with ProfileManager.profile_region("timestep.update"):
time.sleep(0.002)
Finalizing#
finalize() flushes the buffers, merges the per-rank files into the output file and (by default) prints a per-region summary. Call it once, at the end of the run.
[6]:
ProfileManager.finalize()
╭──────────────────────┬─────┬─────────────┬───────────╮
│ region │ n │ total [s] │ avg [s] │
├──────────────────────┼─────┼─────────────┼───────────┤
│ load │ 1 │ 1.0e-02 │ 1.0e-02 │
│ transform │ 3 │ 1.5e-02 │ 5.1e-03 │
│ assemble_matrix │ 2 │ 1.6e-02 │ 8.1e-03 │
│ solve │ 2 │ 4.0e-02 │ 2.0e-02 │
│ timestep │ 1 │ 1.2e-02 │ 1.2e-02 │
│ └─ timestep.residual │ 2 │ 8.1e-03 │ 4.1e-03 │
│ └─ timestep.update │ 2 │ 4.1e-03 │ 2.1e-03 │
│ TOTAL │ 13 │ 1.1e-01 │ │
╰──────────────────────┴─────┴─────────────┴───────────╯
╭─ Info ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ Summary: ../../../../../../../../tmp/scope-profiler-tutorial-j3uwu4sg/profiling_data.h5 (1 rank) │
│ │
│ Explore: │
│ Inspect: scope-profiler inspect ../../../../../../../../tmp/scope-profiler-tutorial-j3uwu4sg/profiling_data.h5 │
│ TUI: scope-profiler tui ../../../../../../../../tmp/scope-profiler-tutorial-j3uwu4sg/profiling_data.h5 │
│ │
│ Visualize and export: │
│ Plot: scope-profiler plot default ../../../../../../../../tmp/scope-profiler-tutorial-j3uwu4sg/profiling_data.h5 -o plots --show │
│ Report: scope-profiler report ../../../../../../../../tmp/scope-profiler-tutorial-j3uwu4sg/profiling_data.h5 -o report.html │
│ Export: scope-profiler export plot-data ../../../../../../../../tmp/scope-profiler-tutorial-j3uwu4sg/profiling_data.h5 -o data │
│ Lines: scope-profiler line-profile ../../../../../../../../tmp/scope-profiler-tutorial-j3uwu4sg/profiling_data.h5 │
│ │
│ Compare runs: │
│ Diff: scope-profiler diff BASE.h5 CANDIDATE.h5 │
│ Check: scope-profiler check BASE.h5 CANDIDATE.h5 │
│ │
│ Durations are in seconds. │
│ Regions may nest, so the summed total can exceed the wall-clock time. │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
Reading the results back#
read_h5() loads the merged file into a ProfilingResults. print_summary() is the quickest way to see what was recorded — all durations are in seconds.
[7]:
from scope_profiler import read_h5
results = read_h5(DATA_FILE)
results.print_summary()
╭──────────────────────┬─────┬─────────────┬───────────╮
│ region │ n │ total [s] │ avg [s] │
├──────────────────────┼─────┼─────────────┼───────────┤
│ load │ 1 │ 1.0e-02 │ 1.0e-02 │
│ transform │ 3 │ 1.5e-02 │ 5.1e-03 │
│ assemble_matrix │ 2 │ 1.6e-02 │ 8.1e-03 │
│ solve │ 2 │ 4.0e-02 │ 2.0e-02 │
│ timestep │ 1 │ 1.2e-02 │ 1.2e-02 │
│ └─ timestep.residual │ 2 │ 8.1e-03 │ 4.1e-03 │
│ └─ timestep.update │ 2 │ 4.1e-03 │ 2.1e-03 │
│ TOTAL │ 13 │ 1.1e-01 │ │
╰──────────────────────┴─────┴─────────────┴───────────╯
╭─ Info ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ Summary: ../../../../../../../../tmp/scope-profiler-tutorial-j3uwu4sg/profiling_data.h5 (1 rank) │
│ │
│ Explore: │
│ Inspect: scope-profiler inspect ../../../../../../../../tmp/scope-profiler-tutorial-j3uwu4sg/profiling_data.h5 │
│ TUI: scope-profiler tui ../../../../../../../../tmp/scope-profiler-tutorial-j3uwu4sg/profiling_data.h5 │
│ │
│ Visualize and export: │
│ Plot: scope-profiler plot default ../../../../../../../../tmp/scope-profiler-tutorial-j3uwu4sg/profiling_data.h5 -o plots --show │
│ Report: scope-profiler report ../../../../../../../../tmp/scope-profiler-tutorial-j3uwu4sg/profiling_data.h5 -o report.html │
│ Export: scope-profiler export plot-data ../../../../../../../../tmp/scope-profiler-tutorial-j3uwu4sg/profiling_data.h5 -o data │
│ Lines: scope-profiler line-profile ../../../../../../../../tmp/scope-profiler-tutorial-j3uwu4sg/profiling_data.h5 │
│ │
│ Compare runs: │
│ Diff: scope-profiler diff BASE.h5 CANDIDATE.h5 │
│ Check: scope-profiler check BASE.h5 CANDIDATE.h5 │
│ │
│ Durations are in seconds. │
│ Regions may nest, so the summed total can exceed the wall-clock time. │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
[8]:
solve_region = results["solve"]
print(solve_region)
print("calls:", solve_region.num_calls)
print("average duration [s]:", solve_region.average_duration)
print("per-call durations [s]:", solve_region[0].durations)
<MPIRegion 'solve': 2 calls on 1 rank(s), total 0.040185 s, avg 0.0200925 s>
calls: 2
average duration [s]: 0.0200925035
per-call durations [s]: [0.0200782 0.02010681]
What is in the file#
The output is a plain HDF5 file, so it is readable with any HDF5 tool. Regions live under rank<N>/regions/<name>/ as start_times and end_times datasets of nanosecond timestamps, and run metadata lives under metadata.
For a quick look at a file without writing any code, scope-profiler inspect profiling_data.h5 prints its metadata and per-region statistics.
[9]:
import h5py
with h5py.File(DATA_FILE, "r") as handle:
handle.visit(print)
events
events/call_ids
events/end_times
events/parent_ids
events/start_times
metadata
rank_region_index
rank_region_index/event_counts
rank_region_index/event_offsets
rank_region_index/exclusive_totals
rank_region_index/ranks
rank_region_index/region_ids
rank_region_index/source_files
rank_region_index/source_lines
rank_region_index/source_texts
rank_region_index/summary_statistics
rank_region_index/tags
region_table
region_table/names
[10]:
for key, value in results.metadata.items():
print(f"{key:>24}: {value}")
LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.10.21/x64/lib
PATH: /opt/hostedtoolcache/Python/3.10.21/x64/bin:/opt/hostedtoolcache/Python/3.10.21/x64:/snap/bin:/home/runner/.local/bin:/opt/pipx_bin:/home/runner/.cargo/bin:/home/runner/.config/composer/vendor/bin:/usr/local/.ghcup/bin:/home/runner/.dotnet/tools:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
chip_information: AMD EPYC 9V74 80-Core Processor
finalize_time_ns: 155322330989
hostname: runnervmgx7h7
modules: []
mpi_size: 1
omp_num_threads: 4
platform: Linux-6.17.0-1022-azure-x86_64-with-glibc2.39
python_version: 3.10.21
scope_profiler_version: 0.5.0
start_time_ns: 155202294605
timestamp: 2026-09-01T07:52:02.724891+00:00
total_cores: 4
uname: Linux runnervmgx7h7 6.17.0-1022-azure #22-Ubuntu SMP Mon Jul 27 17:24:03 UTC 2026 x86_64 x86_64
user: runner
working_directory: /home/runner/work/scope-profiler/scope-profiler/docs/source/tutorials
Next steps#
2. Post-processing — the analysis API in depth
3. Visualizing results — Gantt, flame and duration charts
4. Profiling modes — configuration, recursive profiling, line profiling, MPI, LIKWID and the CLI