1. Getting started#
Install for this tutorial:
pip install scope-profiler(base install only).
scope-profiler measures how long named regions of your code take,
stores the raw per-call timings in HDF5, and gives you a Python API and
CLI to analyse them.
This notebook covers the recording side:
configuring the profiler with
ProfileManager.setup()marking regions with a context manager and with a decorator
writing the results with
ProfileManager.finalize()a first look at the output file
Everything here runs in a single process — see 4. Profiling modes for MPI, LIKWID and the CLI.
import tempfile
from pathlib import Path
from scope_profiler import ProfileManager
# Keep the tutorial's output out of your working directory.
WORKDIR = Path(tempfile.mkdtemp(prefix="scope-profiler-tutorial-"))
DATA_FILE = WORKDIR / "profiling_data.h5"
print(WORKDIR)
/tmp/scope-profiler-tutorial-39m3mq9e
Configuring the profiler#
The ProfileManager class methods use a process-wide default manager.
setup() picks its recording strategy and output path; instantiate
ProfileManager() only when you need independent sessions.
The defaults record wall-clock timings for every call and flush them to
disk, so for most runs ProfileManager.setup(file_path=...) is all you
need.
ProfileManager.setup(file_path=str(DATA_FILE))
<scope_profiler.profile_manager.ProfilingRun at 0x7f19a030ebf0>
Marking a region with a context manager#
ProfileManager.region(name) returns a region object you can use as a
context manager. Entering it records a start timestamp, leaving it
records the end — the region is created on first use and reused
afterwards.
import time
def load_data(n):
time.sleep(0.01)
return list(range(n))
def transform(values):
time.sleep(0.005)
return [value * 2 for value in values]
with ProfileManager.region("load"):
values = load_data(1000)
for _ in range(3):
with ProfileManager.region("transform"):
values = transform(values)
Each with block is one call. transform above was entered three
times, so that region ends up with three recorded durations — the
profiler keeps every call, not just an aggregate.
Marking a function with a decorator#
@ProfileManager.profile wraps a whole function. It works with or
without parentheses, and takes an optional region name (the function’s
__name__ is used otherwise).
@ProfileManager.profile
def solve():
time.sleep(0.02)
@ProfileManager.profile("assemble_matrix")
def assemble():
time.sleep(0.008)
for _ in range(2):
assemble()
solve()
Decorators may be applied before setup() is called — at import time,
for instance. setup() re-binds every registered decorator to the newly
configured region class, so the decorated function always records with
the current configuration and pays no per-call check for it.
Nesting regions#
Regions nest freely; the profiler records each one independently. Nesting is what the flame chart in 3. Visualizing results reconstructs.
with ProfileManager.region("timestep"):
for _ in range(2):
with ProfileManager.region("timestep.residual"):
time.sleep(0.004)
with ProfileManager.region("timestep.update"):
time.sleep(0.002)
Finalizing#
finalize() flushes the buffers, merges the per-rank files into the
output file and (by default) prints a per-region summary. Call it once,
at the end of the run.
ProfileManager.finalize()
╭──────────────────────────────────────────╮
│ region total [s] │
├──────────────────────────────────────────┤
│ load 0.010120 │
│ transform (3x) 0.015403 │
│ assemble_matrix (2x) 0.016163 │
│ solve (2x) 0.040154 │
│ timestep 0.012544 │
│ └─ (own) 0.000227 │
│ └─ timestep.residual (2x) 0.008163 │
│ └─ timestep.update (2x) 0.004153 │
╰──────────────────────────────────────────╯
╭─ Info ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ Summary: ../../../../../../tmp/scope-profiler-tutorial-39m3mq9e/profiling_data.h5 (1 rank) │
│ │
│ Explore: │
│ Inspect: scope-profiler inspect ../../../../../../tmp/scope-profiler-tutorial-39m3mq9e/profiling_data.h5 │
│ TUI: scope-profiler tui ../../../../../../tmp/scope-profiler-tutorial-39m3mq9e/profiling_data.h5 │
│ │
│ Visualize and export: │
│ Plot: scope-profiler plot default ../../../../../../tmp/scope-profiler-tutorial-39m3mq9e/profiling_data.h5 -o plots --show │
│ Report: scope-profiler report ../../../../../../tmp/scope-profiler-tutorial-39m3mq9e/profiling_data.h5 -o report.html │
│ Export: scope-profiler export plot-data ../../../../../../tmp/scope-profiler-tutorial-39m3mq9e/profiling_data.h5 -o data │
│ Lines: scope-profiler line-profile ../../../../../../tmp/scope-profiler-tutorial-39m3mq9e/profiling_data.h5 │
│ │
│ Compare runs: │
│ Diff: scope-profiler diff BASE.h5 CANDIDATE.h5 │
│ Check: scope-profiler check BASE.h5 CANDIDATE.h5 │
│ │
│ Durations are in seconds. │
│ Regions may nest, so the summed total can exceed the wall-clock time. │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
Reading the results back#
load() reads a profile back into a ProfilingResults, in whichever
format the file is in. print_summary() is the quickest way to see what
was recorded — all durations are in seconds.
from scope_profiler import load
results = load(DATA_FILE)
results.print_summary()
╭──────────────────────────────────────────╮
│ region total [s] │
├──────────────────────────────────────────┤
│ load 0.010120 │
│ transform (3x) 0.015403 │
│ assemble_matrix (2x) 0.016163 │
│ solve (2x) 0.040154 │
│ timestep 0.012544 │
│ └─ (own) 0.000227 │
│ └─ timestep.residual (2x) 0.008163 │
│ └─ timestep.update (2x) 0.004153 │
╰──────────────────────────────────────────╯
╭─ Info ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ Summary: ../../../../../../tmp/scope-profiler-tutorial-39m3mq9e/profiling_data.h5 (1 rank) │
│ │
│ Explore: │
│ Inspect: scope-profiler inspect ../../../../../../tmp/scope-profiler-tutorial-39m3mq9e/profiling_data.h5 │
│ TUI: scope-profiler tui ../../../../../../tmp/scope-profiler-tutorial-39m3mq9e/profiling_data.h5 │
│ │
│ Visualize and export: │
│ Plot: scope-profiler plot default ../../../../../../tmp/scope-profiler-tutorial-39m3mq9e/profiling_data.h5 -o plots --show │
│ Report: scope-profiler report ../../../../../../tmp/scope-profiler-tutorial-39m3mq9e/profiling_data.h5 -o report.html │
│ Export: scope-profiler export plot-data ../../../../../../tmp/scope-profiler-tutorial-39m3mq9e/profiling_data.h5 -o data │
│ Lines: scope-profiler line-profile ../../../../../../tmp/scope-profiler-tutorial-39m3mq9e/profiling_data.h5 │
│ │
│ Compare runs: │
│ Diff: scope-profiler diff BASE.h5 CANDIDATE.h5 │
│ Check: scope-profiler check BASE.h5 CANDIDATE.h5 │
│ │
│ Durations are in seconds. │
│ Regions may nest, so the summed total can exceed the wall-clock time. │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
solve_region = results["solve"]
print(solve_region)
print("calls:", solve_region.num_calls)
print("average duration [s]:", solve_region.average_duration)
print("per-call durations [s]:", solve_region[0].durations)
<MPIRegion 'solve': 2 calls on 1 rank(s), total 0.040154 s, avg 0.020077 s>
calls: 2
average duration [s]: 0.020077021
per-call durations [s]: [0.02007668 0.02007736]
What is in the file#
The output is a plain HDF5 file, so it is readable with any HDF5 tool.
Regions live under rank<N>/regions/<name>/ as start_times and
end_times datasets of nanosecond timestamps, and run metadata lives
under metadata.
For a quick look at a file without writing any code,
scope-profiler inspect profiling_data.h5 prints its metadata and
per-region statistics.
import h5py
with h5py.File(DATA_FILE, "r") as handle:
handle.visit(print)
events
events/call_ids
events/durations
events/parent_ids
events/start_deltas
metadata
rank_region_index
rank_region_index/event_counts
rank_region_index/event_metadata
rank_region_index/event_offsets
rank_region_index/exclusive_totals
rank_region_index/ranks
rank_region_index/region_ids
rank_region_index/source_files
rank_region_index/source_lines
rank_region_index/source_texts
rank_region_index/summary_statistics
rank_region_index/tags
region_table
region_table/names
for key, value in results.metadata.items():
print(f"{key:>24}: {value}")
chip_information: AMD EPYC 7763 64-Core Processor
finalize_time_ns: 144321968160
mpi_size: 1
num_nodes: 1
omp_num_threads: 4
platform: Linux-6.17.0-1022-azure-x86_64-with-glibc2.39
python_version: 3.10.22
scope_profiler_version: 0.7.4
start_time_ns: 144196800323
timestamp: 2026-10-16T14:13:07.515145+00:00
total_cores: 4
Next steps#
2. Post-processing — the analysis API in depth
3. Visualizing results — Gantt, flame and duration charts
4. Profiling modes — configuration, recursive profiling, line profiling, MPI, LIKWID and the CLI