scope_profiler.profile_manager

Managers for creating, configuring, and finalizing profiling regions.

Classes

Name	Description
ProfileManager	Manage and track a set of profiling regions.
ProfilingRun	Handle returned by :meth:ProfileManager.setup.
RankPayload	Everything one rank has to hand to rank 0 at finalize().
RegionFactory	Callable region definition for repeated scopes with metadata.
RegionHandle	Persistent region definition that follows manager reconfiguration.

ProfileManagerclass#

View source

class ProfileManager

Manage and track a set of profiling regions.

The class methods remain the process-wide default API. Instantiating the class creates an independent manager with its own configuration, regions, decorators, and call-id space, allowing multiple profiling sessions to be active at the same time::

cpu = ProfileManager()
io = ProfileManager()

with cpu.session(file_path="cpu.h5"):
    with io.session(file_path="io.h5"):
        with cpu.profile_region("compute"), io.profile_region("write"):
            work()

profile_regionattributeclass attributeinstance attribute#

View source

profile_region = region

define_regionmethodclassmethod#

View source

def define_region(region_name, functions=None, tags=None) -> RegionHandle

Define a reusable region handle that survives later setup calls.

finalizemethodclassmethod#

View source

def finalize(verbose: bool = True, return_results: bool = False, native_traces=None, verbose_line_profiler: bool = False)

Finalize profiling and write the run’s data to a single output file.

Copies each region’s buffered timestamps out, moves every rank’s copy to rank 0, and has rank 0 write them into one HDF5 file. Nothing is staged on the filesystem, so no shared $TMPDIR is needed. Optionally prints profiling statistics for each region.

Under MPI this is collective: every rank must call it, with the same arguments, or the job hangs - rank 0 waits for a payload from every other rank. A rank that dies before reaching it therefore leaves the job waiting rather than silently dropping that rank’s data.

With use_likwid=True this is also where the LIKWID markers are closed and every marker region of the run is read back and stored in the output file under rank<r>/likwid/regions/<tag>; see :meth:~scope_profiler.results.ProfilingResults.get_likwid_regions for reading it back.

Parameters

verbose : bool, optional If True, prints the concise profiling summary (default: True). verbose_line_profiler : bool, optional If True, prints detailed line-profiler tables when line profiling is enabled (default: False). return_results : bool, optional If True, return the run’s data as a :class:~scope_profiler.results.ProfilingResults - the same post-processing API :func:~scope_profiler.h5reader.read_h5 gives back, built straight from the in-memory buffers instead of by reading the output file back::

    results = ProfileManager.finalize(return_results=True)
    results.print_summary()
    df = results.to_dataframe()

This works with ``deactivate_file_output=True``, where no file is
written at all. Under MPI the per-rank data is gathered on rank 0,
which is collective: every rank must pass the same value.

path or sequence of paths, optional

Files (or directories of them) written by the C or Fortran region API in this same process, to fold into this run’s output - either native format, a .spt trace or the .h5 an SP_USE_HDF5 C build writes. Each rank picks up the file matching its own rank, so a mixed-language MPI run still produces one file::

kernels.stop_profiling()            # native sp_finalize()
ProfileManager.finalize(native_traces=".")

Call the native side’s sp_finalize() first: its output has to exist by the time this reads it. A region name recorded on both sides raises, rather than silently double-counting.

Returns

ProfilingResults or None The run’s profiling data when return_results=True, and None otherwise. Under MPI rank 0 gets the whole run, like the merged output file; the other ranks get an empty result set for which print_summary(), the plot_* functions and the exporters do nothing, so the script above needs no rank guard. See :attr:~scope_profiler.results.ProfilingResults.is_root.

get_all_regionsmethodclassmethod#

View source

def get_all_regions() -> dict[str, BaseProfileRegion]

Get all registered ProfileRegion instances.

Returns

dict: Dictionary of all registered ProfileRegion instances.

get_configmethodclassmethod#

View source

def get_config() -> ProfilingConfig

Get the current profiling configuration, creating a default one if setup() has not been called.

This is the only place a configuration comes into being outside setup(), and it is deliberately lazy: constructing one resolves the MPI communicator, which imports mpi4py and therefore calls MPI_Init in any process the launcher marked as a rank. Doing that at import time would mean import scope_profiler silently joins the MPI job - fatal in a process forked from a rank, which is exactly what the LIKWID counter read-back does.

Resolving defaults is not itself setup. Regions remain disabled until :meth:setup (directly or through :meth:session) installs a config with :meth:set_config. This lets applications declare decorators at import time without accidentally starting a profiling run.

Returns

ProfilingConfig The current profiling configuration.

get_regionmethodclassmethod#

View source

def get_region(region_name) -> BaseProfileRegion

Get a registered ProfileRegion by name.

Parameters

region_name: str The name of the profiling region.

Returns

ProfileRegion or None: The registered ProfileRegion instance or None if not found.

instrumentmethodclassmethod#

View source

def instrument(module, *, include=('*'), exclude=(), prefix=None, tags=None)

Decorate matching module-level functions without editing source.

tags is passed to every :meth:profile decorator, so a library can put a whole module behind one opt-in tag (see :meth:register_opt_in_tag).

is_activemethodclassmethod#

View source

def is_active() -> bool

Whether instrumentation currently records calls.

is_configuredmethodclassmethod#

View source

def is_configured() -> bool

Whether explicit setup has configured this manager.

last_resultsmethodclassmethod#

View source

def last_results()

Most recent in-memory results requested from this manager, if any.

metadatamethodclassmethod#

View source

def metadata(**values)

Attach key/value metadata to calls completed inside this scope.

pausemethodclassmethod#

View source

def pause() -> None

Temporarily stop recording new profiling scopes.

Pause/resume is intended for boundaries between simulation phases. An open scope cannot be split safely without changing its call identity, so pausing while a region is active raises a clear error.

profilemethodclassmethod#

View source

def profile(region_name=None, recursive=None, metadata=None, record_outcome=False, tags=None)

Decorator factory for profiling a function.

Parameters

region_name : str, optional Name for the profiling region. If not provided, uses the decorated function’s name. Supports being used with or without parentheses. recursive : bool, optional If True, also profiles Python function calls made by the decorated function (excluding scope-profiler internals). If None, falls back to ProfileManager.setup(recursive_profile=...). metadata : callable or dict, optional Metadata to attach to each call. A callable receives the decorated function’s (*args, **kwargs) and must return a mapping. record_outcome : bool, optional Add status and, for failures, exception metadata to each decorated call (default: False). tags : iterable of str, optional Labels persisted with the region, as for :meth:region. An opt-in tag (see :meth:register_opt_in_tag) leaves the function unprofiled unless the run enables it.

Returns

Callable Decorated function wrapped with profiling instrumentation.

Notes

The decorated function is registered so that calling ProfileManager.setup() after decoration re-binds the wrapper to the new region class at zero per-call cost. This means @ProfileManager.profile can be applied at class-definition time even when setup() is called later. While its region is disabled (before setup(), or behind an opt-in tag that is off) the wrapper calls the function directly, skipping metadata and recursive tracing.

profile_commandmethodclassmethod#

View source

def profile_command(command, *, output=None, recursive=False, all=False, mpi_calls=False)

Run a command under scope-profiler run and return its result.

profile_scriptmethodclassmethod#

View source

def profile_script(script_path, *, script_args=None, output=None, **kwargs)

Profile a script from Python using the CLI runner semantics.

read_resultsmethodclassmethod#

View source

def read_results() -> ProfilingResults

Open the merged profiling file this run wrote, for post-processing.

Convenience for analysing results in the same script that produced them::

ProfileManager.finalize()
results = ProfileManager.read_results()
results.print_summary()

Returns

ProfilingResults The data in the file at config.file_path.

Raises

FileNotFoundError If the merged file does not exist yet. It is written by :meth:finalize, and only on rank 0 - guard the call with if ProfileManager.get_config()._rank == 0 under MPI.

recorded_regionsmethodclassmethod#

View source

def recorded_regions() -> tuple[str, ...]

Names of regions that have recorded at least one call.

regionmethodclassmethod#

View source

def region(region_name, functions=None, tags=None) -> BaseProfileRegion

Get the profiling region named region_name, creating it if needed.

The returned region is a context manager, which is how a block of code is timed::

with ProfileManager.region("solve"):
    solve()

Parameters

region_name: str The name of the profiling region. functions : list of callable, optional Functions to register for line-by-line profiling. Only has an effect when use_line_profiler=True. Useful when using the context manager form, since the decorator form (wrap) registers functions automatically::

    with ProfileManager.region("my_region", functions=[my_func]):
        my_func()

iterable of str, optional

User-defined labels persisted with the region. Reusing a region name with a different non-None tag set raises ValueError. A region carrying a tag registered with :meth:register_opt_in_tag is disabled unless the run enables that tag.

Returns

ProfileRegion : The ProfileRegion instance.

Notes

profile_region is the original name for this method and remains available as an alias; the two are the same object.

region_factorymethodclassmethod#

View source

def region_factory(region_name, *, tags=None) -> RegionFactory

Create reusable regions whose calls may carry dynamic metadata.

register_opt_in_tagmethodclassmethod#

View source

def register_opt_in_tag(*tags: str) -> None

Mark tags as opt-in: their regions are off unless enabled.

Meant for libraries that ship their own instrumentation. Register the tag once, at import time, and tag the library’s regions with it::

mylib/__init__.py

import scope_profiler as sp

sp.register_opt_in_tag("mylib")

@sp.profile("mylib.solve", tags=("mylib",))
def solve(): ...

Applications that profile themselves then don’t record mylib’s regions until they ask for them, without editing mylib: with setup(enable_tags=["mylib"]) (also a ProfilingOptions field and a TOML setting), the SCOPE_PROFILER_ENABLE_TAGS=mylib environment variable, or scope-profiler run --enable-tag mylib. "*" enables every opt-in tag. Listing the tag in region_tags enables it too.

Regions and decorated functions that already exist pick the change up, so the call may come after the library’s own imports.

Instrumentation that is off costs one extra Python call (roughly 0.1-0.3 µs per call), so keep it to operations that take microseconds or more.

registered_regionsmethodclassmethod#

View source

def registered_regions() -> tuple[str, ...]

Names of all instrumentation points known to this manager.

resetmethodclassmethod#

View source

def reset() -> None

Forget the configuration and everything recorded; keep the instrumentation.

Leaves the manager as importing the instrumented code does: not configured, every region disabled until the next :meth:setup. Decorated functions and :meth:define_region definitions stay registered, and decorated functions call straight through until then. Meant for test suites that profile in more than one test::

@pytest.fixture(autouse=True)
def _fresh_profiler():
    sp.reset()
    yield
    sp.reset()

resumemethodclassmethod#

View source

def resume() -> None

Resume recording after :meth:pause.

Calling resume before pause is harmless, which makes it safe to place in conditional simulation control paths.

runmethodclassmethod#

View source

def run(func, *args, output=None, **kwargs)

Run a callable inside a managed profiling session.

run_scriptmethodclassmethod#

View source

def run_script(script_path: str, script_args: list | None = None, region_name: str | None = None, only_user_code: bool = True, recursive: bool = True, include_patterns=(), exclude_patterns=(), entrypoint: str | None = None) -> None

Run a script under recursive profiling, similar to python -m cProfile.

Instruments every Python function call made while the script runs and records each as its own region, without requiring any decorators or context managers in the script itself. Intended to be called after ProfileManager.setup() and followed by ProfileManager.finalize(); see python -m scope_profiler for the CLI wrapper around this.

Parameters

script_path : str Path to the script to execute. script_args : list of str, optional Arguments exposed to the script as sys.argv[1:]. region_name : str, optional Name for the region wrapping the whole script’s execution (default: the script’s basename). only_user_code : bool, optional If True (default), skip instrumenting standard-library and installed-package frames, tracing only the script’s own code. This keeps overhead low and the output focused. Set to False to trace everything, including third-party and stdlib calls. recursive : bool, optional Trace Python function calls and add a region around the script (default: True). When False, run only explicitly instrumented profile decorators and region blocks. include_patterns, exclude_patterns : iterable of str, optional Glob filters for recursively traced function names. entrypoint : str, optional Function name to use as the root region when recursively tracing.

sample_everymethodclassmethod#

View source

def sample_every(every: int, start: int = 0)

Profile selected timesteps in a loop.

The yielded callable is a context manager. Pass it the timestep number at the boundary of each iteration::

with ProfileManager.sample_every(10) as profile_step:
    for timestep in range(num_steps):
        with profile_step(timestep):
            advance_simulation()

Timestep start and then every every-th timestep are profiled; all other iterations run while profiling is paused. The timestep context must surround the profiled work and must not be entered while another profiled scope is active. On exit, the previous pause state is restored.

sessionmethodclassmethod#

View source

def session(options: ProfilingOptions | None = None, *, verbose: bool = True, verbose_line_profiler: bool = False, return_results: bool = False, native_traces=None, output=_UNSET, replace: bool = False, config_path: str | os.PathLike[str] | None = None, **overrides: Unpack[SetupOptions])

Return a context manager that sets up and finalizes profiling.

Every argument other than verbose, verbose_line_profiler, return_results and native_traces is passed to :meth:setup, with the same meaning and precedence there - options (a :class:~scope_profiler.profile_config.ProfilingOptions), config_path, and any setting as a keyword::

with ProfileManager.session(options=options) as run:
    ...

with ProfileManager.session(file_path="run.h5") as run:
    ...

Finalization runs even when the profiled block raises; the original exception is preserved.

native_traces is handed to :meth:finalize, which folds the output of a C or Fortran library profiled in this same process into this run’s file - in either native format. See its documentation for the two rules that mixing languages imposes.

When return_results=True, the context object exposes the finalized :class:~scope_profiler.results.ProfilingResults as results::

with ProfileManager.session(return_results=True, verbose=False) as run:
    with ProfileManager.profile_region("solve"):
        solve()
results = run.results

set_configmethodclassmethod#

View source

def set_config(config: ProfilingConfig) -> None

Set a new profiling configuration and update the region class.

Parameters

config : ProfilingConfig The new profiling configuration to apply.

setupmethodclassmethod#

View source

def setup(options: ProfilingOptions | None = None, *, auto_finalize: bool = False, output=_UNSET, replace: bool = False, region_tags=None, config_path: str | os.PathLike[str] | None = None, **overrides: Unpack[SetupOptions])

Initialize and configure the profiling system.

Parameters

options : ProfilingOptions, optional A :class:~scope_profiler.profile_config.ProfilingOptions bag holding any of the settings below, for reuse across calls or construction away from the call site::

    options = ProfilingOptions(use_likwid=True, file_path="run.h5")
    ProfileManager.setup(options=options)

An explicit keyword argument passed alongside ``options`` wins
over the same field on ``options``, which in turn wins over
``config_path`` and the defaults below.

config_path : str or os.PathLike, optional TOML file containing a [profiling] table with these settings. Values passed directly to setup() take precedence. See :func:~scope_profiler.profile_config.load_profiling_config. auto_finalize : bool, optional Finalize once at normal interpreter exit, unless finalize() is called explicitly first (default: False). This is convenient for scripts that want setup without a surrounding session. Avoid it when MPI ranks may not exit together. output : str, os.PathLike or None, optional Direct spelling of the output destination. None keeps results in memory and writes no file. This is an alias for file_path and deactivate_file_output; do not combine those spellings. replace : bool, optional Replace an active configuration (default: False). Without this, accidental setup of an already active manager raises. **overrides Any of the settings below, passed as keyword arguments::

    ProfileManager.setup(file_path="run.h5", use_likwid=True)

They are the fields of
:class:`~scope_profiler.profile_config.ProfilingOptions`, which
is where they are declared once and typed; an unrecognised name
raises ``TypeError`` naming the closest match. Prefixed settings
can also be given as groups on ``options`` - see
:class:`~scope_profiler.profile_config.MemrayOptions`,
:class:`~scope_profiler.profile_config.GPUOptions` and
:class:`~scope_profiler.profile_config.HDF5Options`.

file_path : str, optional Path to the output profiling data file (default: “profiling_data.h5”). label : str or None, optional Short name for this run (default: None, i.e. the output file’s stem). Post-processing uses it wherever a run has to be named — chart legends, the summary heading, scope-profiler inspect, the JSON statistics - which is what makes several runs distinguishable when they are compared::

    ProfileManager.setup(file_path="run_a.h5", label="128 ranks")

It is stored in the output file as the ``label`` metadata field,
so it survives into every later post-processing step.

use_likwid : bool, optional Enable LIKWID hardware counter collection (default: False). use_line_profiler : bool, optional Enable line-by-line profiling via line_profiler (default: False). use_memray : bool, optional Record process-wide allocations with Memray (default: False). The capture is a separate Memray .bin file and requires the separately installed memray package. memory_profile_path : str, optional Memray capture path (default: <file-stem>.memray.bin). memray_native_traces, memray_trace_python_allocators, memray_follow_fork : bool, optional Memray capture options. Python allocator tracing can create much larger traces and has materially higher overhead. deactivate_profiling : bool, optional Turn profiling off entirely (default: False). Every region becomes a no-op, so the instrumentation can stay in the code at near-zero cost instead of being removed. use_nvtx : bool, optional Add NVTX ranges to profiled regions for NVIDIA Nsight tools (default: False). Requires scope-profiler[nvtx]. use_gpu_timing : bool, optional Record CUDA-event elapsed device time for each profiled region (default: False). CPU timestamps are still recorded, so the normal timeline remains enqueue-side timing. gpu_sync_on_exit : bool, optional Wait for the current GPU device before recording CPU end time. Requires use_gpu_timing=True; defaults to False. Applies to contexts and decorators and removes CPU/GPU overlap. Custom backends must provide synchronize() when enabled. gpu_timing_backend : str or object, optional CUDA-event backend for use_gpu_timing: "auto", "torch", "cupy", or a custom object implementing record_event() and elapsed_time_ns(start_event, end_event). deactivate_file_output : bool, optional Write no HDF5 file at all (default: False), not even the run metadata. Use it with finalize(return_results=True) to analyse a run entirely in memory::

    ProfileManager.setup(deactivate_file_output=True)
    ...
    results = ProfileManager.finalize(return_results=True)

bool, optional

Enable recursive profiling for all decorated functions by default (default: False). This can be overridden per decorator with @ProfileManager.profile(..., recursive=...).

aggregation_mode : bool, optional Record only count, inclusive total, minimum, maximum, and exclusive total per region. Timeline events are unavailable in this mode; it cannot be combined with line, GPU, NVTX, or LIKWID profiling. profile_mpi_calls : bool, optional Profile mpi4py operations made through predefined and derived communicators (default: False). mpi4py is imported only if the application imports it. With :meth:session, obtain MPI.COMM_WORLD inside the session so it receives the proxy. track_threads : bool, optional Profile every thread (default: False). Each thread gets its own buffers and scope stack, so regions entered concurrently no longer overwrite one another, and every recorded call carries the thread it ran on. The run also reports each thread’s name, OS ids, lifetime and CPU time - see :attr:~scope_profiler.results.ProfilingResults.threads::

    with ProfileManager.session(track_threads=True):
        ...

Cannot be combined with line, GPU, NVTX, LIKWID or aggregation
profiling.

track_async : bool, optional Follow asyncio tasks and greenlets as well (default: False, and implies track_threads). Each call additionally carries the task it ran in and the time that task spent suspended inside the call, so a with block held across an await reports its await time rather than charging it to the region. Per-task running and awaiting totals are available from :attr:~scope_profiler.results.ProfilingResults.tasks. metadata_detail : {“minimal”, “full”}, optional How much of the run’s environment is recorded (default: "minimal"): versions, platform, CPU model and thread/rank counts, with source files relative to the working directory. "full" adds the user name, host name, working directory, loaded modules, environment variables, SLURM_* variables and absolute source paths. Region source text is controlled separately by capture_region_source. capture_region_source : bool, optional Record where each region is defined - the with block or the decorated function - once per distinct source file, the first time any of its regions is created (default: False). See :attr:~scope_profiler.region.Region.source_text. Off by default because the cost, while cheap for a typical file, is not always: it is one ast.parse + tree walk of that file, so it tracks the file’s total size, not the size or number of the regions in it - under a millisecond for a typical few-hundred- line file, but tenths of a second per rank for one containing thousands of lines across many regions. Every rank pays that independently, so it can compound to whole seconds under contention on a job with more ranks than idle cores (measured: ~0.3s/rank at 8 ranks, ~2.9s/rank at 64, for a single ~10,000-line file, on a shared/oversubscribed node)::

    ProfileManager.setup(capture_region_source=True)

int, optional

Initial number of profiling events preallocated per region (default: 1024). Buffers grow on demand, so this is a starting size rather than a limit; raise it for very hot regions to avoid repeated reallocation.

output_mode : {“auto”, “direct”, “parallel”}, optional MPI file writer. auto prefers MPI-enabled h5py when compatible with the active instrumentation and otherwise lets ranks append directly to one serial-HDF5 file in token order. hdf5_compression : {“gzip”, “lzf”, “zstd”} or None, optional Compression filter for timestamp and GPU-duration datasets. hdf5_compression_level : int or None, optional GZIP level 0—9 or Zstandard level 1—22. hdf5_chunk_size : int or None, optional Maximum events per dataset chunk. Enables chunked partial reads even without compression. enable_tags : iterable of str, optional Opt-in tags (see :meth:register_opt_in_tag) whose regions this run records; "*" enables all of them. The comma-separated SCOPE_PROFILER_ENABLE_TAGS environment variable adds to this list, as do the tags in region_tags.

Notes

The run’s start time is the moment setup() is called; it is stored as the start_time_ns metadata field and is the origin of the relative timeline in post-processing. MPI rank detection is separate from profile_mpi_calls: collectives are used exactly when the process was started by an MPI launcher. See :mod:scope_profiler.mpi_launch for detection and its SCOPE_PROFILER_MPI override.

setup_from_envmethodclassmethod#

View source

def setup_from_env(**overrides)

Configure profiling from SCOPE_PROFILER_* environment variables.

ProfilingRunclass#

View source

class ProfilingRun(manager, config, output=None)

Handle returned by :meth:ProfileManager.setup.

resultsattributeinstance attribute#

View source

results = None

file_pathproperty#

View source

file_path

Configured output path for this run.

is_activeproperty#

View source

is_active: bool

Whether this run’s manager currently records calls.

finalizemethod#

View source

def finalize(**kwargs)

Finalize through the owning manager and retain the results.

RankPayloadclass#

View source

class RankPayload(NamedTuple)

Bases: NamedTuple

Everything one rank has to hand to rank 0 at finalize().

This is what crosses the wire under MPI, and it is also what rank 0 writes into the output file and folds into the returned results - one transport feeding both, so they cannot disagree. A NamedTuple because it pickles as a plain tuple, which is what mpi4py’s send/recv use.

aggregate_statsattributeclass attributeinstance attribute#

View source

aggregate_stats: dict | None = None

event_metadataattributeclass attributeinstance attribute#

View source

event_metadata: dict | None = None

exclusive_totalsattributeclass attributeinstance attribute#

View source

exclusive_totals: dict | None = None

lanesattributeclass attributeinstance attribute#

View source

lanes: dict | None = None

Region name -> timing arrays, in nanoseconds.

Values are (start_times, end_times) for CPU timing only, or (start_times, end_times, gpu_durations) when CUDA-event timing is enabled. CPU timestamps still measure enqueue-side region duration.

likwidattributeinstance attribute#

View source

likwid: dict

likwid_environmentattributeinstance attribute#

View source

likwid_environment: dict

line_profileattributeclass attributeinstance attribute#

View source

line_profile: list | None = None

perf_eventsattributeclass attributeinstance attribute#

View source

perf_events: dict | None = None

regionsattributeinstance attribute#

View source

regions: dict

sourcesattributeclass attributeinstance attribute#

View source

sources: dict | None = None

tagsattributeclass attributeinstance attribute#

View source

tags: dict | None = None

RegionFactoryclass#

View source

class RegionFactory(manager, name, tags=None)

Callable region definition for repeated scopes with metadata.

nameattributeinstance attribute#

View source

name = name

RegionHandleclass#

View source

class RegionHandle(manager, name, functions=None, tags=None)

Persistent region definition that follows manager reconfiguration.

nameattributeinstance attribute#

View source

name = name