1. Getting started

Download notebook View on GitHubRun when the site was built.

Install for this tutorial: pip install scope-profiler (base install only).

scope-profiler measures how long named regions of your code take, stores the raw per-call timings in HDF5, and gives you a Python API and CLI to analyse them.

This notebook covers the recording side:

configuring the profiler with ProfileManager.setup()
marking regions with a context manager and with a decorator
writing the results with ProfileManager.finalize()
a first look at the output file

Everything here runs in a single process — see 4. Profiling modes for MPI, LIKWID and the CLI.

import time

from scope_profiler import ProfileManager

Configuring the profiler

The ProfileManager class methods use a process-wide default manager. setup() picks its recording strategy and output path; instantiate ProfileManager() only when you need independent sessions.

The defaults record wall-clock timings for every call and flush them to disk, so for most runs ProfileManager.setup(file_path=...) is all you need. The optional label names the run wherever several runs have to be told apart: summaries, chart legends and scope-profiler inspect.

Regions entered before setup() are inert: nothing is recorded until the profiler is configured, so instrumented code can be imported and run without profiling it.

run = ProfileManager.setup(file_path="profiling_data.h5", label="tutorial 1")

setup() returns a ProfilingRun handle. You rarely need it — ProfileManager itself remembers the configuration — but run.file_path is the output path and run.finalize() is the same as ProfileManager.finalize().

Marking a region with a context manager

ProfileManager.region(name) returns a region object you can use as a context manager. Entering it records a start timestamp, leaving it records the end — the region is created on first use and reused afterwards.

def load_data(n):
    time.sleep(0.01)
    return list(range(n))


def transform(values):
    time.sleep(0.005)
    return [value * 2 for value in values]


with ProfileManager.region("load"):
    values = load_data(1000)

for _ in range(3):
    with ProfileManager.region("transform"):
        values = transform(values)

Each with block is one call. transform above was entered three times, so that region ends up with three recorded durations — the profiler keeps every call, not just an aggregate.

Marking a function with a decorator

@ProfileManager.profile wraps a whole function. It works with or without parentheses, and takes an optional region name (the function’s __name__ is used otherwise).

@ProfileManager.profile
def solve():
    time.sleep(0.02)


@ProfileManager.profile("assemble_matrix")
def assemble():
    time.sleep(0.008)


for _ in range(2):
    assemble()
    solve()

Decorators may be applied before setup() is called — at import time, for instance. setup() re-binds every registered decorator to the newly configured region class, so the decorated function always records with the current configuration and pays no per-call check for it.

Nesting regions

Regions nest freely; the profiler records each one independently. Nesting is what the flame chart in 3. Visualizing results reconstructs.

with ProfileManager.region("timestep"):
    for _ in range(2):
        with ProfileManager.region("timestep.residual"):
            time.sleep(0.004)
        with ProfileManager.region("timestep.update"):
            time.sleep(0.002)

Finalizing

finalize() collects the buffers (under MPI, from every rank onto rank 0), writes them into one HDF5 file and, by default, prints a per-region summary. Call it once, at the end of the run. The cell below passes verbose=False because we print the summary ourselves in the next step.

ProfileManager.finalize(verbose=False)

Reading the results back

load() reads a profile back into a ProfilingResults, in whichever format the file is in (HDF5 or JSON). print_summary() is the quickest way to see what was recorded — all durations are in seconds, and nested regions are indented under their parent. suppress_notes=True leaves out the footer of follow-up commands that finalize() prints.

from scope_profiler import load

results = load("profiling_data.h5")
results.print_summary(suppress_notes=True)

  ╭──────────────────────────────────────────╮
  │ region                       total [s]   │
  ├──────────────────────────────────────────┤
  │ load                         0.010107    │
  │ transform (3x)               0.015250    │
  │ assemble_matrix (2x)         0.016142    │
  │ solve (2x)                   0.040184    │
  │ timestep                     0.012330    │
  │ └─ (own)                        0.000079 │
  │ └─ timestep.residual (2x)       0.008129 │
  │ └─ timestep.update (2x)         0.004122 │
  ╰──────────────────────────────────────────╯

solve_region = results["solve"]
print(solve_region)
print("calls:", solve_region.num_calls)
print("average duration [s]:", solve_region.average_duration)
print("per-call durations [s]:", solve_region[0].durations)

<MPIRegion 'solve': 2 calls on 1 rank(s), total 0.0401841 s, avg 0.0200921 s>
calls: 2
average duration [s]: 0.0200920745
per-call durations [s]: [0.02006598 0.02011817]

What is in the file

The output is a plain HDF5 file, so it is readable with any HDF5 tool. Timestamps are int64 nanoseconds, stored as columns shared by all regions under events/. region_table/names holds each region name once, rank_region_index/ says which slice of the event columns belongs to each (rank, region) pair, and the run metadata are attributes of the metadata group. The HDF5 and Python API guide documents the full layout.

For a quick look at a file without writing any code, scope-profiler inspect profiling_data.h5 prints its metadata and per-region statistics.

import h5py

with h5py.File("profiling_data.h5", "r") as handle:
    handle.visit(print)

HDF5 groups and datasets

events
events/call_ids
events/durations
events/parent_ids
events/start_deltas
metadata
rank_region_index
rank_region_index/event_counts
rank_region_index/event_metadata
rank_region_index/event_offsets
rank_region_index/exclusive_totals
rank_region_index/ranks
rank_region_index/region_ids
rank_region_index/source_files
rank_region_index/source_lines
rank_region_index/source_texts
rank_region_index/summary_statistics
rank_region_index/tags
region_table
region_table/names

By default the file records a minimal description of the environment — versions, platform, CPU model, rank and thread counts, and the label. User and host names, the working directory, environment variables and the like are only stored with metadata_detail="full" (see 2. Post-processing).

for key, value in results.metadata.items():
    print(f"{key:>24}: {value}")

        chip_information: AMD EPYC 9V45 96-Core Processor
        finalize_time_ns: 182227170982
                   label: tutorial 1
                mpi_size: 1
               num_nodes: 1
         omp_num_threads: 4
                platform: Linux-7.0.0-1017-azure-x86_64-with-glibc2.39
          python_version: 3.12.3
  scope_profiler_version: 0.7.5
           start_time_ns: 182116509753
               timestamp: 2026-10-22T06:38:31.036018+00:00
             total_cores: 4

Sessions and the module-level API

ProfileManager.setup() plus finalize() is the explicit form. A session does both around a block, and finalizes even if the block raises. The same calls are also available as module-level functions, which saves writing the class name:

import scope_profiler as sp

with sp.session(file_path="session.h5", label="session demo", verbose=False):
    with sp.region("work"):
        time.sleep(0.01)

load("session.h5").print_summary(suppress_notes=True)

  ╭──────────────────────────────────────────────────────╮
  │ region                    % session      total [s]   │
  ├──────────────────────────────────────────────────────┤
  │ scope_profiler.session    100.00%        0.010109    │
  │ └─ (own)                     0.41%          0.000042 │
  │ └─ work                      99.59%         0.010067 │
  ╰──────────────────────────────────────────────────────╯

Next steps

2. Post-processing — the analysis API in depth
3. Visualizing results — Gantt, flame and duration charts
4. Profiling modes — configuration, recursive profiling, line profiling, MPI, LIKWID and the CLI
8. Reports and comparison — HTML reports, diff and check