scope_profiler

Search symbols

Type part of a name or a dotted path

scope-profiler: lightweight region-based profiling for Python and HPC applications.

Attributes

Name	Description
define_region	No description.
finalize	No description.
instrument	No description.
is_active	No description.
is_configured	No description.
last_results	No description.
plot_flame	No description.
profile	No description.
profile_command	No description.
profile_script	No description.
recorded_regions	No description.
region_factory	No description.
register_opt_in_tag	No description.
registered_regions	No description.
reset	No description.
run	No description.
session	No description.
setup	No description.
setup_from_env	No description.

Classes

Name	Description
CallArrays	One rank's reconstructed nesting, in start-sorted (call_id) order.
CorruptProfileError	Raised when a profiling file cannot be opened as HDF5 at all.
EventDataUnavailableError	Raised when a summary-only result is asked for per-call event data.
GPUOptions	GPU timing and NVTX settings, as a group.
HDF5Options	HDF5 storage settings, as a group.
JSONProfileError	Raised when a file is not a scope-profiler JSON profile.
LikwidRegionResult	Counter results for a single LIKWID marker region.
MPIRegion	One named region across all ranks that recorded it.
MPIRegionMetadata	Structured metadata decoded from a Python or native MPI region name.
MemrayOptions	Memray capture settings, as a group.
NestingError	Raised when recorded intervals are not properly nested.
PerfEventError	A requested kernel performance event could not be opened or read.
PerfEventTotals	Aggregated counts for one region.
ProfileManager	Manage and track a set of profiling regions.
ProfiledMPIComm	A transparent communicator proxy for common mpi4py operations.
ProfiledMPIRequest	A request proxy that attributes blocking completion time to MPI wait.
ProfilingOptions	A bag of :meth:ProfileManager.setup settings, to reuse or pass around.
ProfilingResults	The profiling data of one run, as an ordered mapping of region name to
ProfilingRun	Handle returned by :meth:ProfileManager.setup.
Region	Timing data for one region on one rank.
RegionFactory	Callable region definition for repeated scopes with metadata.
RegionHandle	Persistent region definition that follows manager reconfiguration.

Functions

Name	Description
build_call_arrays	Reconstruct one rank's nesting as numpy arrays.
build_call_stack	Reconstruct per-call nesting for one rank, one dict per call.
call_stack_children	Child indices per call, for walking a :func:build_call_stack result.
call_stack_roots	Indices of the top-level calls in a :func:build_call_stack result.
collect_file_metadata	Collect the run metadata of one or more profiling files.
collect_region_statistics	Collect aggregate region-duration statistics for one or more profiling files.
collect_roofline_points	Derive one roofline point per selected LIKWID region and rank.
create_html_report	Write a standalone HTML report for one or more profiling results.
export_chrome_trace	Write Chrome Trace JSON files, one per input profiling run.
export_flamegraph_svg	Render per-rank flame graphs to standalone SVG files with flameprof.
export_json	Write one JSON document per run, exactly as export_speedscope does.
export_prof	Write per-rank .prof files readable by pstats/snakeviz.
export_speedscope	Write speedscope JSON files for the selected ranks.
format_mpi_region	Return the canonical region name shared by Python and C++ wrappers.
inspect_file	Print a summary of one profiling HDF5 file.
load	Load a profile for post-processing, whatever format it is in.
load_prof	Read a .prof file written by :func:export_prof back as a Stats.
merge_results	Combine several result sets into one.
message_nbytes	Estimate payload bytes for an mpi4py object or buffer specification.
parse_mpi_region	Decode a canonical MPI region name, or return None for another region.
plot_duration_timeseries	Plot each region's call duration over wall-clock time, with a min-max band.
plot_durations	Plot duration bar charts for one or more profiling files using maxplotlib.
plot_flame_chart	Plot a flame chart reconstructing the call stack from region timings.
plot_flame_graph	Plot an aggregated flame graph whose x-axis represents total time.
plot_gantt	Plot a Gantt chart of all (or selected) regions with per-rank lanes using maxplotlib.
plot_perf_events	Plot one raw perf event or derived ratio for every region and rank.
plot_rank_heatmap	Plot total or exclusive region time as a rank-by-region heatmap.
plot_roofline	Plot attained performance against arithmetic intensity for LIKWID regions.
plot_scaling_efficiency	Plot parallel scaling efficiency (measured speedup / ideal speedup).
plot_speedup	Plot scope speedup versus a chosen parallelism/metadata field using maxplotlib.
plot_weak_scaling	Plot weak-scaling runtime versus a chosen parallelism/metadata field.
plot_weak_scaling_efficiency	Plot weak-scaling efficiency: baseline runtime over runtime at each scale.
profile_mpi4py	Temporarily profile mpi4py's predefined communicators.
profile_mpi_comm	Wrap a communicator without importing mpi4py.
read_h5	Load a merged profiling file for post-processing.
read_h5_summary	Load fixed-size statistics, optionally falling back for older files.
read_json	Read a JSON profile back into :class:ProfilingResults.
read_profile	Read a profile from HDF5 or JSON, whichever the name says it is.
sniff_profile_format	Which format the contents of path are in.
to_pstats	Build a :class:pstats.Stats per (run label, rank), without disk I/O.
write_json	Write one run to filepath as JSON, and return the path written.
write_profile	Write results to file_path in the format its name asks for.
write_region_statistics_json	Write aggregate region-duration statistics to a JSON file.

Modules

• astro_siteAstro/Starlight content for a catalogue of profiling runs.
• benchmarkRepeatable benchmark workflows for coding agents and CI.
• call_stackReconstruct a call stack from the recorded region intervals.
• catalogueCatalogue page linking several generated HTML reports.
• chrome_trace_exportExport profiling runs to the Chrome Trace Event format.
• concurrencyThread, asyncio-task and greenlet identity for the region profiler.
• diffCLI entry point for scope-profiler diff: compare two profiling files.
• gpu_timingOptional CUDA-event timing backends for asynchronous GPU work.
• h5readerParsing of merged HDF5 profiling output files.
• h5schemaVersioning and compatibility helpers for scope-profiler HDF5 files.
• h5writerWriting a merged HDF5 profiling file.
• html_reportSelf-contained HTML reports for profiling results.
• inspectionCLI entry point for scope-profiler inspect: summarize a profiling file.
• ipython_magicsJupyter/IPython magics for scope-profiler.
• json_exportExport and re-read profiling runs as JSON.
• likwid_dataCollection and storage of LIKWID marker results.
• line_profile_cliCLI for displaying line-profiler records persisted in HDF5 files.
• mcp_serverMCP server exposing scope-profiler to AI coding agents.
• memrayOptional Memray allocation-trace integration.
• metadataCollects descriptive metadata about the environment a profiling run executed in.
• mpi_launchDetection of whether the process was launched by an MPI launcher.
• mpi_regionContainer for per-rank Region data within an MPI-parallel profiling run.
• mpi_wrappersOpt-in profiling wrappers for mpi4py communicators and requests.
• native_samplesLine-level profiles of native (C) regions, from the sampler's .sps files.
• native_traceReading the per-rank output written by the native (C and Fortran) region APIs.
• perf_eventsSmall Linux perf_event_open wrapper used for region counter totals.
• plotting_scriptsPlotting utilities for visualizing profiling data using maxplotlib.
• post_processingCLI entry point for post-processing HDF5 profiling data.
• prof_exportExport merged HDF5 profiling data to the cProfile .prof (pstats) format.
• profile_configConfiguration for profiling managers.
• profile_ioPick the right reader or writer for a profile path.
• profile_managerManagers for creating, configuring, and finalizing profiling regions.
• pytest_pluginOpt-in pytest integration for recording test-phase durations.
• regionData container for a single profiling region loaded from HDF5.
• region_profilerProfile region classes implementing the strategy pattern for different profiling modes.
• report_dataData behind the report charts and its scaling axes, without any HTML.
• report_modelWhat a profiling report says, as plain data.
• resultsPost-processing API over a set of profiling regions.
• speedscope_exportExport merged HDF5 profiling data to the speedscope JSON format.
• summaryShared per-region summary table.
• tuiInteractive Textual browser for scope-profiler HDF5 files.

define_regionattributemodule attribute#

View source

define_region = ProfileManager.define_region

finalizeattributemodule attribute#

View source

finalize = ProfileManager.finalize

instrumentattributemodule attribute#

View source

instrument = ProfileManager.instrument

is_activeattributemodule attribute#

View source

is_active = ProfileManager.is_active

is_configuredattributemodule attribute#

View source

is_configured = ProfileManager.is_configured

last_resultsattributemodule attribute#

View source

last_results = ProfileManager.last_results

plot_flameattributemodule attribute#

Re-exported from scope_profiler.plotting_scripts.flameView source

plot_flame = plot_flame_chart

profileattributemodule attribute#

View source

profile = ProfileManager.profile

profile_commandattributemodule attribute#

View source

profile_command = ProfileManager.profile_command

profile_scriptattributemodule attribute#

View source

profile_script = ProfileManager.profile_script

recorded_regionsattributemodule attribute#

View source

recorded_regions = ProfileManager.recorded_regions

region_factoryattributemodule attribute#

View source

region_factory = ProfileManager.region_factory

register_opt_in_tagattributemodule attribute#

View source

register_opt_in_tag = ProfileManager.register_opt_in_tag

registered_regionsattributemodule attribute#

View source

registered_regions = ProfileManager.registered_regions

resetattributemodule attribute#

View source

reset = ProfileManager.reset

runattributemodule attribute#

View source

run = ProfileManager.run

sessionattributemodule attribute#

View source

session = ProfileManager.session

setupattributemodule attribute#

View source

setup = ProfileManager.setup

setup_from_envattributemodule attribute#

View source

setup_from_env = ProfileManager.setup_from_env

CallArraysclass#

Re-exported from scope_profiler.call_stackView source

class CallArrays(NamedTuple)

Bases: NamedTuple

One rank’s reconstructed nesting, in start-sorted (call_id) order.

Every array is indexed by call_id, so parent indexes straight back into the same arrays. region_index and call_index point at where a call came from: region names[region_index[i]], call number call_index[i] within that region’s buffers.

call_indexattributeinstance attribute#

View source

call_index: np.ndarray

depthattributeinstance attribute#

View source

depth: np.ndarray

end_nsattributeinstance attribute#

View source

end_ns: np.ndarray

exclusive_nsattributeinstance attribute#

View source

exclusive_ns: np.ndarray

laneattributeclass attributeinstance attribute#

View source

lane: np.ndarray = _NO_LANE

Which stack each call belongs to; see :func:scope_profiler.concurrency.lane_ids.

Empty for a run that did not track threads, meaning the whole rank is one stack - rather than a full-length array of -1, which would cost three more passes over the events of every single-threaded run to say nothing. Consumers must therefore check lane.size before indexing it. Defaulted so the field can be omitted by older callers constructing this tuple themselves.

namesattributeinstance attribute#

View source

names: list[str]

parentattributeinstance attribute#

View source

parent: np.ndarray

region_indexattributeinstance attribute#

View source

region_index: np.ndarray

source_filesattributeinstance attribute#

View source

source_files: list[str | None]

source_linesattributeinstance attribute#

View source

source_lines: list[int | None]

start_nsattributeinstance attribute#

View source

start_ns: np.ndarray

CorruptProfileErrorclass#

Re-exported from scope_profiler.h5readerView source

class CorruptProfileError(OSError)

Bases: OSError

Raised when a profiling file cannot be opened as HDF5 at all.

The common cause is a run that was killed while writing, leaving a truncated file behind. h5py reports that as a bare OSError naming neither the file nor what the caller was trying to do, so it is re-raised here with both. It stays an OSError so existing handlers keep working.

EventDataUnavailableErrorclass#

Re-exported from scope_profiler.regionView source

class EventDataUnavailableError(RuntimeError)

Bases: RuntimeError

Raised when a summary-only result is asked for per-call event data.

GPUOptionsclassdataclass#

Re-exported from scope_profiler.profile_configView source

class GPUOptions(timing: bool | None = None, backend: Any = None, nvtx: bool | None = None, sync_on_exit: bool | None = None)

Bases: _OptionGroup

GPU timing and NVTX settings, as a group.

In a TOML config file this is the [profiling.gpu] table.

Attributes

timing : bool or None Record CUDA-event elapsed device time per region (use_gpu_timing). backend : str, object, or None CUDA-event backend (gpu_timing_backend): "auto", "torch", "cupy", or an object implementing record_event() and elapsed_time_ns(start_event, end_event). sync_on_exit : bool or None Synchronize the device at region exit (gpu_sync_on_exit). Requires timing and removes CPU/GPU overlap; disabled by default. nvtx : bool or None Emit NVTX ranges for NVIDIA Nsight tools (use_nvtx).

backendattributeclass attributeinstance attribute#

View source

backend: Any = None

nvtxattributeclass attributeinstance attribute#

View source

nvtx: bool | None = None

sync_on_exitattributeclass attributeinstance attribute#

View source

sync_on_exit: bool | None = None

timingattributeclass attributeinstance attribute#

View source

timing: bool | None = None

Inherited from scope_profiler.profile_config._OptionGroup

to_kwargsmethod#

Inherited from scope_profiler.profile_config._OptionGroupView source

def to_kwargs() -> dict

This group’s explicitly-set fields, under their flat names.

HDF5Optionsclassdataclass#

Re-exported from scope_profiler.profile_configView source

class HDF5Options(compression: str | None = None, compression_level: int | None = None, chunk_size: int | None = None)

Bases: _OptionGroup

HDF5 storage settings, as a group.

In a TOML config file this is the [profiling.hdf5] table.

Attributes

compression : str or None Compression filter for timestamp and GPU-duration datasets (hdf5_compression): "gzip", "lzf", "zstd", or "auto" to compress only runs large enough to repay the write CPU. compression_level : int or None GZIP level 0—9 or Zstandard level 1—22 (hdf5_compression_level). chunk_size : int or None Maximum events per dataset chunk (hdf5_chunk_size). Enables chunked partial reads even without compression.

chunk_sizeattributeclass attributeinstance attribute#

View source

chunk_size: int | None = None

compressionattributeclass attributeinstance attribute#

View source

compression: str | None = None

compression_levelattributeclass attributeinstance attribute#

View source

compression_level: int | None = None

Inherited from scope_profiler.profile_config._OptionGroup

to_kwargsmethod#

Inherited from scope_profiler.profile_config._OptionGroupView source

def to_kwargs() -> dict

This group’s explicitly-set fields, under their flat names.

JSONProfileErrorclass#

Re-exported from scope_profiler.json_exportView source

class JSONProfileError(ValueError)

Bases: ValueError

Raised when a file is not a scope-profiler JSON profile.

LikwidRegionResultclassdataclass#

Re-exported from scope_profiler.likwid_dataView source

class LikwidRegionResult(tag: str, group_id: int = -1, group_name: str = '', cpus: list[int] = list(), times: np.ndarray = lambda : np.zeros(0)(), call_counts: np.ndarray = lambda : np.zeros(0, dtype=np.int64)(), event_names: list[str] = list(), counter_names: list[str] = list(), events: np.ndarray = lambda : np.zeros((0, 0))(), metric_names: list[str] = list(), metrics: np.ndarray = lambda : np.zeros((0, 0))(), source: str = 'full_api')

Counter results for a single LIKWID marker region.

The per-thread arrays are all indexed by the same thread axis, so events[e, t] and metrics[m, t] refer to the thread whose runtime is times[t] and whose CPU is cpus[t].

Attributes

tag : str Region name as passed to markerstartregion. group_id : int Index of the LIKWID event group this region was measured with. group_name : str Name of that group (e.g. "CLOCK"), empty if unknown. cpus : list of int Hardware threads that took part in the region. times : numpy.ndarray Per-thread accumulated runtime in seconds, shape (nthreads,). call_counts : numpy.ndarray Per-thread number of times the region was entered, shape (nthreads,). event_names : list of str Names of the raw hardware events, length nevents. Not unique: a group may program one event on several counters (see :attr:event_labels). counter_names : list of str Hardware counter register each event was programmed on (FIXC0, PMC1, MBOX3C0, …), length nevents. Empty for files written before counter names were recorded. events : numpy.ndarray Raw counter values, shape (nevents, nthreads). metric_names : list of str Names of LIKWID’s derived metrics, length nmetrics. metrics : numpy.ndarray Derived metric values, shape (nmetrics, nthreads). source : str Which collection path produced this result: "full_api" (perfmon read-back, the only one with real event names and metrics), "marker_file" (LIKWID’s marker file parsed directly --- real values, placeholder event names, no metrics) or "marker_api" (a markergetregion snapshot of the calling thread only).

call_countsattributeclass attributeinstance attribute#

View source

call_counts: np.ndarray = field(default_factory=lambda : np.zeros(0, dtype=np.int64))

counter_namesattributeclass attributeinstance attribute#

View source

counter_names: list[str] = field(default_factory=list)

cpusattributeclass attributeinstance attribute#

View source

cpus: list[int] = field(default_factory=list)

event_namesattributeclass attributeinstance attribute#

View source

event_names: list[str] = field(default_factory=list)

eventsattributeclass attributeinstance attribute#

View source

events: np.ndarray = field(default_factory=lambda : np.zeros((0, 0)))

group_idattributeclass attributeinstance attribute#

View source

group_id: int = -1

group_nameattributeclass attributeinstance attribute#

View source

group_name: str = ''

metric_namesattributeclass attributeinstance attribute#

View source

metric_names: list[str] = field(default_factory=list)

metricsattributeclass attributeinstance attribute#

View source

metrics: np.ndarray = field(default_factory=lambda : np.zeros((0, 0)))

sourceattributeclass attributeinstance attribute#

View source

source: str = 'full_api'

tagattributeinstance attribute#

View source

tag: str

timesattributeclass attributeinstance attribute#

View source

times: np.ndarray = field(default_factory=lambda : np.zeros(0))

event_labelsproperty#

View source

event_labels: list[str]

Unique per-event labels, safe to use as dict or column keys.

event_names alone is not unique. A group such as MEM_DP programs the same event on one counter per memory channel, so CAS_COUNT_RD legitimately appears eight times on a socket with eight channels (the ones reading zero are unpopulated). Keying anything by the bare name silently keeps only the last channel.

Names that occur once are returned unchanged; repeated ones get the hardware counter appended --- CAS_COUNT_RD:MBOX0C0, CAS_COUNT_RD:MBOX1C0, … --- or a positional suffix if the file predates counter names being recorded.

as_dictmethod#

View source

def as_dict() -> dict

Return the region’s results as a plain dictionary.

Convenience for callers that want to build a DataFrame or dump the numbers without touching the array layout.

MPIRegionclass#

Re-exported from scope_profiler.mpi_regionView source

class MPIRegion(name: str, regions: dict[int, Region])

One named region across all ranks that recorded it.

Indexing gives the per-rank :class:~scope_profiler.region.Region (region[0]), while the properties on this class aggregate over every rank. All durations are in seconds.

colorattributeinstance attribute#

View source

color = None

average_durationproperty#

View source

average_duration: float

Get the mean duration over every call on every rank.

Note this pools all calls, so ranks with more calls weigh more heavily; use average_durations() for the per-rank breakdown.

Returns

float Average duration in seconds, or 0.0 if no timing was recorded.

durationsproperty#

View source

durations: np.ndarray

Get every recorded call duration on every rank, in seconds.

Returns

np.ndarray Pooled durations. Empty if no rank recorded timing.

exclusive_durationproperty#

View source

exclusive_duration: float

Alias for :attr:total_exclusive_duration.

exclusive_durationsproperty#

View source

exclusive_durations: np.ndarray

Exclusive durations pooled across ranks, in seconds.

first_durationproperty#

View source

first_duration: float

Get the duration of the call that started earliest across all ranks.

Returns

float Duration in seconds of the chronologically first call, or 0.0 if no rank recorded timing.

first_start_timeproperty#

View source

first_start_time: float

Get the earliest start time across all ranks.

Returns

float The earliest start time among all ranks, in seconds, or 0.0 if no rank recorded timing.

gpu_average_durationproperty#

View source

gpu_average_duration: float | None

Average CUDA-event elapsed device time across ranks, or None if absent.

gpu_durationsproperty#

View source

gpu_durations: np.ndarray | None

CUDA-event elapsed device times pooled across ranks, or None if absent.

gpu_total_durationproperty#

View source

gpu_total_duration: float | None

Total CUDA-event elapsed device time across ranks, or None if absent.

has_event_dataproperty#

View source

has_event_data: bool

Whether every represented rank has per-call timestamps loaded.

has_gpu_timingproperty#

View source

has_gpu_timing: bool

Whether any rank recorded CUDA-event timings for this region.

has_sourceproperty#

View source

has_source: bool

Whether any rank captured this region’s call-site source.

has_timingproperty#

View source

has_timing: bool

Whether any rank recorded timestamps for this region.

inclusive_durationproperty#

View source

inclusive_duration: float

Total inclusive time across ranks, in seconds.

inclusive_durationsproperty#

View source

inclusive_durations: np.ndarray

Inclusive durations pooled across ranks, in seconds.

last_durationproperty#

View source

last_duration: float

Get the duration of the call that ended latest across all ranks.

Returns

float Duration in seconds of the chronologically last call, or 0.0 if no rank recorded timing.

last_end_timeproperty#

View source

last_end_time: float

Get the latest end time across all ranks.

Returns

float The latest end time among all ranks, in seconds, or 0.0 if no rank recorded timing.

max_durationproperty#

View source

max_duration: float

Get the maximum duration across all ranks.

Returns

float The maximum duration among all ranks, in seconds.

min_durationproperty#

View source

min_duration: float

Get the minimum duration across all ranks.

Returns

float The minimum duration among all ranks, in seconds.

nameproperty#

View source

name: str

Name of the region.

num_callsproperty#

View source

num_calls: int

Total number of calls summed over all ranks.

Returns

int Summed call count.

p50_durationproperty#

View source

p50_duration: float | None

Median duration across all ranks, in seconds.

p95_durationproperty#

View source

p95_duration: float | None

95th-percentile duration across all ranks, in seconds.

p99_durationproperty#

View source

p99_duration: float | None

99th-percentile duration across all ranks, in seconds.

rank_imbalanceproperty#

View source

rank_imbalance: float

Maximum per-rank total divided by the mean per-rank total.

A value of 1.0 means perfectly balanced ranks. Values are 0.0 when no calls were recorded or only one rank has timing data.

rank_imbalance_pctproperty#

View source

rank_imbalance_pct: float

Excess of the slowest rank over the mean, as a percentage.

ranksproperty#

View source

ranks: list[int]

Sorted list of ranks that recorded this region.

regionsproperty#

View source

regions: dict[int, Region]

Dictionary of rank IDs to their corresponding Region objects.

source_fileproperty#

View source

source_file: str | None

Path of the file this region is defined in, or None if not captured.

source_linenoproperty#

View source

source_lineno: int | None

Line the region’s call site starts at, or None if not captured.

source_textproperty#

View source

source_text: str | None

Source text of the region’s with block or decorated function.

std_durationproperty#

View source

std_duration: float

Get the standard deviation over every call on every rank.

Returns

float Standard deviation in seconds, or 0.0 if no timing was recorded.

tagsproperty#

View source

tags: tuple[str, ...]

User-defined tags attached to this region.

total_durationproperty#

View source

total_duration: float

Get the total duration summed over all ranks and calls.

Returns

float Total duration in seconds.

total_exclusive_durationproperty#

View source

total_exclusive_duration: float

Total time excluding nested regions, across ranks, in seconds.

average_durationsmethod#

View source

def average_durations() -> dict[int, float]

Get the average duration for each rank.

Returns

Dict[int, float] Dictionary mapping rank IDs to their average durations.

eventsmethod#

View source

def events(ranks: list[int] | int | None = None, origin: float = 0.0) -> list[dict[str, Any]]

Return one dict per recorded call, on every rank.

This is the long-form view custom plots and dataframes want: each entry is a single call rather than a per-region aggregate.

Parameters

ranks : list of int or int, optional Restrict to these ranks (default: all ranks that recorded the region). Ranks without data for this region are skipped. origin : float, optional Seconds subtracted from every timestamp, so passing results.minimum_start_time yields a timeline starting at zero (default: 0.0, i.e. raw timestamps).

Returns

list of dict Entries with keys name, rank, call_index, start, end and duration, in seconds, ordered by rank and then by call order.

get_summarymethod#

View source

def get_summary() -> dict[str, Any]

Return statistics aggregated over every rank.

Returns

Dict[str, Any] Dictionary with name, num_ranks, num_calls (summed over ranks) and duration statistics in seconds, pooled over all calls on all ranks.

max_durationsmethod#

View source

def max_durations() -> dict[int, float]

Get the maximum duration for each rank.

Returns

Dict[int, float] Dictionary mapping rank IDs to their maximum durations.

min_durationsmethod#

View source

def min_durations() -> dict[int, float]

Get the minimum duration for each rank.

Returns

Dict[int, float] Dictionary mapping rank IDs to their minimum durations.

num_calls_per_rankmethod#

View source

def num_calls_per_rank() -> dict[int, int]

Get the call count for each rank.

Returns

Dict[int, int] Dictionary mapping rank IDs to their call counts.

percentile_durationmethod#

View source

def percentile_duration(percentile: float) -> float | None

Return a pooled duration percentile across all ranks.

total_durationsmethod#

View source

def total_durations() -> dict[int, float]

Get the total duration for each rank.

Returns

Dict[int, float] Dictionary mapping rank IDs to their total durations.

MPIRegionMetadataclassdataclass#

Re-exported from scope_profiler.mpi_wrappersView source

class MPIRegionMetadata(operation: str, kind: str, bytes: int, peer: int, root: int, tag: int, communicator: int, request: str | None = None)

Structured metadata decoded from a Python or native MPI region name.

bytesattributeinstance attribute#

View source

bytes: int

communicatorattributeinstance attribute#

View source

communicator: int

kindattributeinstance attribute#

View source

kind: str

operationattributeinstance attribute#

View source

operation: str

peerattributeinstance attribute#

View source

peer: int

requestattributeclass attributeinstance attribute#

View source

request: str | None = None

rootattributeinstance attribute#

View source

root: int

tagattributeinstance attribute#

View source

tag: int

MemrayOptionsclassdataclass#

Re-exported from scope_profiler.profile_configView source

class MemrayOptions(enabled: bool | None = None, path: str | None = None, native_traces: bool | None = None, trace_python_allocators: bool | None = None, follow_fork: bool | None = None)

Bases: _OptionGroup

Memray capture settings, as a group.

ProfilingOptions(memray=MemrayOptions(enabled=True, native_traces=True)) configures the same run as ProfilingOptions(use_memray=True, memray_native_traces=True); use whichever reads better at the call site. In a TOML config file the group is the [profiling.memray] table.

Attributes

enabled : bool or None Record process-wide allocations with Memray (use_memray). path : str or None Capture path (memory_profile_path); defaults to <file-stem>.memray.bin. native_traces : bool or None Capture native stack frames (memray_native_traces). trace_python_allocators : bool or None Trace Python’s own allocators (memray_trace_python_allocators). Creates much larger traces and has materially higher overhead. follow_fork : bool or None Keep tracking across fork() (memray_follow_fork).

enabledattributeclass attributeinstance attribute#

View source

enabled: bool | None = None

follow_forkattributeclass attributeinstance attribute#

View source

follow_fork: bool | None = None

native_tracesattributeclass attributeinstance attribute#

View source

native_traces: bool | None = None

pathattributeclass attributeinstance attribute#

View source

path: str | None = None

trace_python_allocatorsattributeclass attributeinstance attribute#

View source

trace_python_allocators: bool | None = None

Inherited from scope_profiler.profile_config._OptionGroup

to_kwargsmethod#

Inherited from scope_profiler.profile_config._OptionGroupView source

def to_kwargs() -> dict

This group’s explicitly-set fields, under their flat names.

NestingErrorclass#

Re-exported from scope_profiler.call_stackView source

class NestingError(ValueError)

Bases: ValueError

Raised when recorded intervals are not properly nested.

A lane’s calls must form a forest: any two intervals on one lane are either disjoint or one contains the other. Partial overlap has no call stack to reconstruct, and every consumer of this module (flame chart, .prof, speedscope, exclusive time) would have to invent one.

In practice this means a mismatched sp_begin/sp_end pair in C or Fortran, or regions driven from several threads in a run that did not profile with track_threads=True - without that, every thread’s calls land in one lane and interleave.

PerfEventErrorclass#

Re-exported from scope_profiler.perf_eventsView source

class PerfEventError(RuntimeError)

Bases: RuntimeError

A requested kernel performance event could not be opened or read.

PerfEventTotalsclassdataclass#

Re-exported from scope_profiler.perf_eventsView source

class PerfEventTotals(calls: int, values: dict[str, int])

Aggregated counts for one region.

callsattributeinstance attribute#

View source

calls: int

valuesattributeinstance attribute#

View source

values: dict[str, int]

ProfileManagerclass#

Re-exported from scope_profiler.profile_managerView source

class ProfileManager

Manage and track a set of profiling regions.

The class methods remain the process-wide default API. Instantiating the class creates an independent manager with its own configuration, regions, decorators, and call-id space, allowing multiple profiling sessions to be active at the same time::

cpu = ProfileManager()
io = ProfileManager()

with cpu.session(file_path="cpu.h5"):
    with io.session(file_path="io.h5"):
        with cpu.profile_region("compute"), io.profile_region("write"):
            work()

profile_regionattributeclass attributeinstance attribute#

View source

profile_region = region

define_regionmethodclassmethod#

View source

def define_region(region_name, functions=None, tags=None) -> RegionHandle

Define a reusable region handle that survives later setup calls.

finalizemethodclassmethod#

View source

def finalize(verbose: bool = True, return_results: bool = False, native_traces=None, verbose_line_profiler: bool = False)

Finalize profiling and write the run’s data to a single output file.

Copies each region’s buffered timestamps out, moves every rank’s copy to rank 0, and has rank 0 write them into one HDF5 file. Nothing is staged on the filesystem, so no shared $TMPDIR is needed. Optionally prints profiling statistics for each region.

Under MPI this is collective: every rank must call it, with the same arguments, or the job hangs - rank 0 waits for a payload from every other rank. A rank that dies before reaching it therefore leaves the job waiting rather than silently dropping that rank’s data.

With use_likwid=True this is also where the LIKWID markers are closed and every marker region of the run is read back and stored in the output file under rank<r>/likwid/regions/<tag>; see :meth:~scope_profiler.results.ProfilingResults.get_likwid_regions for reading it back.

Parameters

verbose : bool, optional If True, prints the concise profiling summary (default: True). verbose_line_profiler : bool, optional If True, prints detailed line-profiler tables when line profiling is enabled (default: False). return_results : bool, optional If True, return the run’s data as a :class:~scope_profiler.results.ProfilingResults - the same post-processing API :func:~scope_profiler.h5reader.read_h5 gives back, built straight from the in-memory buffers instead of by reading the output file back::

    results = ProfileManager.finalize(return_results=True)
    results.print_summary()
    df = results.to_dataframe()

This works with ``deactivate_file_output=True``, where no file is
written at all. Under MPI the per-rank data is gathered on rank 0,
which is collective: every rank must pass the same value.

path or sequence of paths, optional

Files (or directories of them) written by the C or Fortran region API in this same process, to fold into this run’s output - either native format, a .spt trace or the .h5 an SP_USE_HDF5 C build writes. Each rank picks up the file matching its own rank, so a mixed-language MPI run still produces one file::

kernels.stop_profiling()            # native sp_finalize()
ProfileManager.finalize(native_traces=".")

Call the native side’s sp_finalize() first: its output has to exist by the time this reads it. A region name recorded on both sides raises, rather than silently double-counting.

Returns

ProfilingResults or None The run’s profiling data when return_results=True, and None otherwise. Under MPI rank 0 gets the whole run, like the merged output file; the other ranks get an empty result set for which print_summary(), the plot_* functions and the exporters do nothing, so the script above needs no rank guard. See :attr:~scope_profiler.results.ProfilingResults.is_root.

get_all_regionsmethodclassmethod#

View source

def get_all_regions() -> dict[str, BaseProfileRegion]

Get all registered ProfileRegion instances.

Returns

dict: Dictionary of all registered ProfileRegion instances.

get_configmethodclassmethod#

View source

def get_config() -> ProfilingConfig

Get the current profiling configuration, creating a default one if setup() has not been called.

This is the only place a configuration comes into being outside setup(), and it is deliberately lazy: constructing one resolves the MPI communicator, which imports mpi4py and therefore calls MPI_Init in any process the launcher marked as a rank. Doing that at import time would mean import scope_profiler silently joins the MPI job - fatal in a process forked from a rank, which is exactly what the LIKWID counter read-back does.

Resolving defaults is not itself setup. Regions remain disabled until :meth:setup (directly or through :meth:session) installs a config with :meth:set_config. This lets applications declare decorators at import time without accidentally starting a profiling run.

Returns

ProfilingConfig The current profiling configuration.

get_regionmethodclassmethod#

View source

def get_region(region_name) -> BaseProfileRegion

Get a registered ProfileRegion by name.

Parameters

region_name: str The name of the profiling region.

Returns

ProfileRegion or None: The registered ProfileRegion instance or None if not found.

instrumentmethodclassmethod#

View source

def instrument(module, *, include=('*'), exclude=(), prefix=None, tags=None)

Decorate matching module-level functions without editing source.

tags is passed to every :meth:profile decorator, so a library can put a whole module behind one opt-in tag (see :meth:register_opt_in_tag).

is_activemethodclassmethod#

View source

def is_active() -> bool

Whether instrumentation currently records calls.

is_configuredmethodclassmethod#

View source

def is_configured() -> bool

Whether explicit setup has configured this manager.

last_resultsmethodclassmethod#

View source

def last_results()

Most recent in-memory results requested from this manager, if any.

metadatamethodclassmethod#

View source

def metadata(**values)

Attach key/value metadata to calls completed inside this scope.

pausemethodclassmethod#

View source

def pause() -> None

Temporarily stop recording new profiling scopes.

Pause/resume is intended for boundaries between simulation phases. An open scope cannot be split safely without changing its call identity, so pausing while a region is active raises a clear error.

profilemethodclassmethod#

View source

def profile(region_name=None, recursive=None, metadata=None, record_outcome=False, tags=None)

Decorator factory for profiling a function.

Parameters

region_name : str, optional Name for the profiling region. If not provided, uses the decorated function’s name. Supports being used with or without parentheses. recursive : bool, optional If True, also profiles Python function calls made by the decorated function (excluding scope-profiler internals). If None, falls back to ProfileManager.setup(recursive_profile=...). metadata : callable or dict, optional Metadata to attach to each call. A callable receives the decorated function’s (*args, **kwargs) and must return a mapping. record_outcome : bool, optional Add status and, for failures, exception metadata to each decorated call (default: False). tags : iterable of str, optional Labels persisted with the region, as for :meth:region. An opt-in tag (see :meth:register_opt_in_tag) leaves the function unprofiled unless the run enables it.

Returns

Callable Decorated function wrapped with profiling instrumentation.

Notes

The decorated function is registered so that calling ProfileManager.setup() after decoration re-binds the wrapper to the new region class at zero per-call cost. This means @ProfileManager.profile can be applied at class-definition time even when setup() is called later. While its region is disabled (before setup(), or behind an opt-in tag that is off) the wrapper calls the function directly, skipping metadata and recursive tracing.

profile_commandmethodclassmethod#

View source

def profile_command(command, *, output=None, recursive=False, all=False, mpi_calls=False)

Run a command under scope-profiler run and return its result.

profile_scriptmethodclassmethod#

View source

def profile_script(script_path, *, script_args=None, output=None, **kwargs)

Profile a script from Python using the CLI runner semantics.

read_resultsmethodclassmethod#

View source

def read_results() -> ProfilingResults

Open the merged profiling file this run wrote, for post-processing.

Convenience for analysing results in the same script that produced them::

ProfileManager.finalize()
results = ProfileManager.read_results()
results.print_summary()

Returns

ProfilingResults The data in the file at config.file_path.

Raises

FileNotFoundError If the merged file does not exist yet. It is written by :meth:finalize, and only on rank 0 - guard the call with if ProfileManager.get_config()._rank == 0 under MPI.

recorded_regionsmethodclassmethod#

View source

def recorded_regions() -> tuple[str, ...]

Names of regions that have recorded at least one call.

regionmethodclassmethod#

View source

def region(region_name, functions=None, tags=None) -> BaseProfileRegion

Get the profiling region named region_name, creating it if needed.

The returned region is a context manager, which is how a block of code is timed::

with ProfileManager.region("solve"):
    solve()

Parameters

region_name: str The name of the profiling region. functions : list of callable, optional Functions to register for line-by-line profiling. Only has an effect when use_line_profiler=True. Useful when using the context manager form, since the decorator form (wrap) registers functions automatically::

    with ProfileManager.region("my_region", functions=[my_func]):
        my_func()

iterable of str, optional

User-defined labels persisted with the region. Reusing a region name with a different non-None tag set raises ValueError. A region carrying a tag registered with :meth:register_opt_in_tag is disabled unless the run enables that tag.

Returns

ProfileRegion : The ProfileRegion instance.

Notes

profile_region is the original name for this method and remains available as an alias; the two are the same object.

region_factorymethodclassmethod#

View source

def region_factory(region_name, *, tags=None) -> RegionFactory

Create reusable regions whose calls may carry dynamic metadata.

register_opt_in_tagmethodclassmethod#

View source

def register_opt_in_tag(*tags: str) -> None

Mark tags as opt-in: their regions are off unless enabled.

Meant for libraries that ship their own instrumentation. Register the tag once, at import time, and tag the library’s regions with it::

mylib/__init__.py

import scope_profiler as sp

sp.register_opt_in_tag("mylib")

@sp.profile("mylib.solve", tags=("mylib",))
def solve(): ...

Applications that profile themselves then don’t record mylib’s regions until they ask for them, without editing mylib: with setup(enable_tags=["mylib"]) (also a ProfilingOptions field and a TOML setting), the SCOPE_PROFILER_ENABLE_TAGS=mylib environment variable, or scope-profiler run --enable-tag mylib. "*" enables every opt-in tag. Listing the tag in region_tags enables it too.

Regions and decorated functions that already exist pick the change up, so the call may come after the library’s own imports.

Instrumentation that is off costs one extra Python call (roughly 0.1-0.3 µs per call), so keep it to operations that take microseconds or more.

registered_regionsmethodclassmethod#

View source

def registered_regions() -> tuple[str, ...]

Names of all instrumentation points known to this manager.

resetmethodclassmethod#

View source

def reset() -> None

Forget the configuration and everything recorded; keep the instrumentation.

Leaves the manager as importing the instrumented code does: not configured, every region disabled until the next :meth:setup. Decorated functions and :meth:define_region definitions stay registered, and decorated functions call straight through until then. Meant for test suites that profile in more than one test::

@pytest.fixture(autouse=True)
def _fresh_profiler():
    sp.reset()
    yield
    sp.reset()

resumemethodclassmethod#

View source

def resume() -> None

Resume recording after :meth:pause.

Calling resume before pause is harmless, which makes it safe to place in conditional simulation control paths.

runmethodclassmethod#

View source

def run(func, *args, output=None, **kwargs)

Run a callable inside a managed profiling session.

run_scriptmethodclassmethod#

View source

def run_script(script_path: str, script_args: list | None = None, region_name: str | None = None, only_user_code: bool = True, recursive: bool = True, include_patterns=(), exclude_patterns=(), entrypoint: str | None = None) -> None

Run a script under recursive profiling, similar to python -m cProfile.

Instruments every Python function call made while the script runs and records each as its own region, without requiring any decorators or context managers in the script itself. Intended to be called after ProfileManager.setup() and followed by ProfileManager.finalize(); see python -m scope_profiler for the CLI wrapper around this.

Parameters

script_path : str Path to the script to execute. script_args : list of str, optional Arguments exposed to the script as sys.argv[1:]. region_name : str, optional Name for the region wrapping the whole script’s execution (default: the script’s basename). only_user_code : bool, optional If True (default), skip instrumenting standard-library and installed-package frames, tracing only the script’s own code. This keeps overhead low and the output focused. Set to False to trace everything, including third-party and stdlib calls. recursive : bool, optional Trace Python function calls and add a region around the script (default: True). When False, run only explicitly instrumented profile decorators and region blocks. include_patterns, exclude_patterns : iterable of str, optional Glob filters for recursively traced function names. entrypoint : str, optional Function name to use as the root region when recursively tracing.

sample_everymethodclassmethod#

View source

def sample_every(every: int, start: int = 0)

Profile selected timesteps in a loop.

The yielded callable is a context manager. Pass it the timestep number at the boundary of each iteration::

with ProfileManager.sample_every(10) as profile_step:
    for timestep in range(num_steps):
        with profile_step(timestep):
            advance_simulation()

Timestep start and then every every-th timestep are profiled; all other iterations run while profiling is paused. The timestep context must surround the profiled work and must not be entered while another profiled scope is active. On exit, the previous pause state is restored.

sessionmethodclassmethod#

View source

def session(options: ProfilingOptions | None = None, *, verbose: bool = True, verbose_line_profiler: bool = False, return_results: bool = False, native_traces=None, output=_UNSET, replace: bool = False, config_path: str | os.PathLike[str] | None = None, **overrides: Unpack[SetupOptions])

Return a context manager that sets up and finalizes profiling.

Every argument other than verbose, verbose_line_profiler, return_results and native_traces is passed to :meth:setup, with the same meaning and precedence there - options (a :class:~scope_profiler.profile_config.ProfilingOptions), config_path, and any setting as a keyword::

with ProfileManager.session(options=options) as run:
    ...

with ProfileManager.session(file_path="run.h5") as run:
    ...

Finalization runs even when the profiled block raises; the original exception is preserved.

native_traces is handed to :meth:finalize, which folds the output of a C or Fortran library profiled in this same process into this run’s file - in either native format. See its documentation for the two rules that mixing languages imposes.

When return_results=True, the context object exposes the finalized :class:~scope_profiler.results.ProfilingResults as results::

with ProfileManager.session(return_results=True, verbose=False) as run:
    with ProfileManager.profile_region("solve"):
        solve()
results = run.results

set_configmethodclassmethod#

View source

def set_config(config: ProfilingConfig) -> None

Set a new profiling configuration and update the region class.

Parameters

config : ProfilingConfig The new profiling configuration to apply.

setupmethodclassmethod#

View source

def setup(options: ProfilingOptions | None = None, *, auto_finalize: bool = False, output=_UNSET, replace: bool = False, region_tags=None, config_path: str | os.PathLike[str] | None = None, **overrides: Unpack[SetupOptions])

Initialize and configure the profiling system.

Parameters

options : ProfilingOptions, optional A :class:~scope_profiler.profile_config.ProfilingOptions bag holding any of the settings below, for reuse across calls or construction away from the call site::

    options = ProfilingOptions(use_likwid=True, file_path="run.h5")
    ProfileManager.setup(options=options)

An explicit keyword argument passed alongside ``options`` wins
over the same field on ``options``, which in turn wins over
``config_path`` and the defaults below.

config_path : str or os.PathLike, optional TOML file containing a [profiling] table with these settings. Values passed directly to setup() take precedence. See :func:~scope_profiler.profile_config.load_profiling_config. auto_finalize : bool, optional Finalize once at normal interpreter exit, unless finalize() is called explicitly first (default: False). This is convenient for scripts that want setup without a surrounding session. Avoid it when MPI ranks may not exit together. output : str, os.PathLike or None, optional Direct spelling of the output destination. None keeps results in memory and writes no file. This is an alias for file_path and deactivate_file_output; do not combine those spellings. replace : bool, optional Replace an active configuration (default: False). Without this, accidental setup of an already active manager raises. **overrides Any of the settings below, passed as keyword arguments::

    ProfileManager.setup(file_path="run.h5", use_likwid=True)

They are the fields of
:class:`~scope_profiler.profile_config.ProfilingOptions`, which
is where they are declared once and typed; an unrecognised name
raises ``TypeError`` naming the closest match. Prefixed settings
can also be given as groups on ``options`` - see
:class:`~scope_profiler.profile_config.MemrayOptions`,
:class:`~scope_profiler.profile_config.GPUOptions` and
:class:`~scope_profiler.profile_config.HDF5Options`.

file_path : str, optional Path to the output profiling data file (default: “profiling_data.h5”). label : str or None, optional Short name for this run (default: None, i.e. the output file’s stem). Post-processing uses it wherever a run has to be named — chart legends, the summary heading, scope-profiler inspect, the JSON statistics - which is what makes several runs distinguishable when they are compared::

    ProfileManager.setup(file_path="run_a.h5", label="128 ranks")

It is stored in the output file as the ``label`` metadata field,
so it survives into every later post-processing step.

use_likwid : bool, optional Enable LIKWID hardware counter collection (default: False). use_line_profiler : bool, optional Enable line-by-line profiling via line_profiler (default: False). use_memray : bool, optional Record process-wide allocations with Memray (default: False). The capture is a separate Memray .bin file and requires the separately installed memray package. memory_profile_path : str, optional Memray capture path (default: <file-stem>.memray.bin). memray_native_traces, memray_trace_python_allocators, memray_follow_fork : bool, optional Memray capture options. Python allocator tracing can create much larger traces and has materially higher overhead. deactivate_profiling : bool, optional Turn profiling off entirely (default: False). Every region becomes a no-op, so the instrumentation can stay in the code at near-zero cost instead of being removed. use_nvtx : bool, optional Add NVTX ranges to profiled regions for NVIDIA Nsight tools (default: False). Requires scope-profiler[nvtx]. use_gpu_timing : bool, optional Record CUDA-event elapsed device time for each profiled region (default: False). CPU timestamps are still recorded, so the normal timeline remains enqueue-side timing. gpu_sync_on_exit : bool, optional Wait for the current GPU device before recording CPU end time. Requires use_gpu_timing=True; defaults to False. Applies to contexts and decorators and removes CPU/GPU overlap. Custom backends must provide synchronize() when enabled. gpu_timing_backend : str or object, optional CUDA-event backend for use_gpu_timing: "auto", "torch", "cupy", or a custom object implementing record_event() and elapsed_time_ns(start_event, end_event). deactivate_file_output : bool, optional Write no HDF5 file at all (default: False), not even the run metadata. Use it with finalize(return_results=True) to analyse a run entirely in memory::

    ProfileManager.setup(deactivate_file_output=True)
    ...
    results = ProfileManager.finalize(return_results=True)

bool, optional

Enable recursive profiling for all decorated functions by default (default: False). This can be overridden per decorator with @ProfileManager.profile(..., recursive=...).

aggregation_mode : bool, optional Record only count, inclusive total, minimum, maximum, and exclusive total per region. Timeline events are unavailable in this mode; it cannot be combined with line, GPU, NVTX, or LIKWID profiling. profile_mpi_calls : bool, optional Profile mpi4py operations made through predefined and derived communicators (default: False). mpi4py is imported only if the application imports it. With :meth:session, obtain MPI.COMM_WORLD inside the session so it receives the proxy. track_threads : bool, optional Profile every thread (default: False). Each thread gets its own buffers and scope stack, so regions entered concurrently no longer overwrite one another, and every recorded call carries the thread it ran on. The run also reports each thread’s name, OS ids, lifetime and CPU time - see :attr:~scope_profiler.results.ProfilingResults.threads::

    with ProfileManager.session(track_threads=True):
        ...

Cannot be combined with line, GPU, NVTX, LIKWID or aggregation
profiling.

track_async : bool, optional Follow asyncio tasks and greenlets as well (default: False, and implies track_threads). Each call additionally carries the task it ran in and the time that task spent suspended inside the call, so a with block held across an await reports its await time rather than charging it to the region. Per-task running and awaiting totals are available from :attr:~scope_profiler.results.ProfilingResults.tasks. metadata_detail : {“minimal”, “full”}, optional How much of the run’s environment is recorded (default: "minimal"): versions, platform, CPU model and thread/rank counts, with source files relative to the working directory. "full" adds the user name, host name, working directory, loaded modules, environment variables, SLURM_* variables and absolute source paths. Region source text is controlled separately by capture_region_source. capture_region_source : bool, optional Record where each region is defined - the with block or the decorated function - once per distinct source file, the first time any of its regions is created (default: False). See :attr:~scope_profiler.region.Region.source_text. Off by default because the cost, while cheap for a typical file, is not always: it is one ast.parse + tree walk of that file, so it tracks the file’s total size, not the size or number of the regions in it - under a millisecond for a typical few-hundred- line file, but tenths of a second per rank for one containing thousands of lines across many regions. Every rank pays that independently, so it can compound to whole seconds under contention on a job with more ranks than idle cores (measured: ~0.3s/rank at 8 ranks, ~2.9s/rank at 64, for a single ~10,000-line file, on a shared/oversubscribed node)::

    ProfileManager.setup(capture_region_source=True)

int, optional

Initial number of profiling events preallocated per region (default: 1024). Buffers grow on demand, so this is a starting size rather than a limit; raise it for very hot regions to avoid repeated reallocation.

output_mode : {“auto”, “direct”, “parallel”}, optional MPI file writer. auto prefers MPI-enabled h5py when compatible with the active instrumentation and otherwise lets ranks append directly to one serial-HDF5 file in token order. hdf5_compression : {“gzip”, “lzf”, “zstd”} or None, optional Compression filter for timestamp and GPU-duration datasets. hdf5_compression_level : int or None, optional GZIP level 0—9 or Zstandard level 1—22. hdf5_chunk_size : int or None, optional Maximum events per dataset chunk. Enables chunked partial reads even without compression. enable_tags : iterable of str, optional Opt-in tags (see :meth:register_opt_in_tag) whose regions this run records; "*" enables all of them. The comma-separated SCOPE_PROFILER_ENABLE_TAGS environment variable adds to this list, as do the tags in region_tags.

Notes

The run’s start time is the moment setup() is called; it is stored as the start_time_ns metadata field and is the origin of the relative timeline in post-processing. MPI rank detection is separate from profile_mpi_calls: collectives are used exactly when the process was started by an MPI launcher. See :mod:scope_profiler.mpi_launch for detection and its SCOPE_PROFILER_MPI override.

setup_from_envmethodclassmethod#

View source

def setup_from_env(**overrides)

Configure profiling from SCOPE_PROFILER_* environment variables.

ProfiledMPICommclass#

Re-exported from scope_profiler.mpi_wrappersView source

class ProfiledMPIComm(communicator: Any, *, pickle_object_sizes: bool = False)

A transparent communicator proxy for common mpi4py operations.

communicatorproperty#

View source

communicator: Any

The unwrapped mpi4py communicator.

Allreducemethod#

View source

def Allreduce(sendbuf, recvbuf, op=None)

Barriermethod#

View source

def Barrier()

Bcastmethod#

View source

def Bcast(buf, root: int = 0)

Irecvmethod#

View source

def Irecv(buf, source: int = -1, tag: int = -1) -> ProfiledMPIRequest

Isendmethod#

View source

def Isend(buf, dest: int, tag: int = 0) -> ProfiledMPIRequest

Recvmethod#

View source

def Recv(buf, source: int = -1, tag: int = -1, status=None)

Reducemethod#

View source

def Reduce(sendbuf, recvbuf, op=None, root: int = 0)

Sendmethod#

View source

def Send(buf, dest: int, tag: int = 0)

allreducemethod#

View source

def allreduce(sendobj, op=None)

barriermethod#

View source

def barrier()

bcastmethod#

View source

def bcast(obj, root: int = 0)

irecvmethod#

View source

def irecv(buf=None, source: int = -1, tag: int = -1) -> ProfiledMPIRequest

isendmethod#

View source

def isend(obj, dest: int, tag: int = 0) -> ProfiledMPIRequest

recvmethod#

View source

def recv(buf=None, source: int = -1, tag: int = -1, status=None)

reducemethod#

View source

def reduce(sendobj, op=None, root: int = 0)

sendmethod#

View source

def send(obj, dest: int, tag: int = 0)

ProfiledMPIRequestclass#

Re-exported from scope_profiler.mpi_wrappersView source

class ProfiledMPIRequest(request: Any, *, operation: str, bytes: int, peer: int, tag: int, communicator: int)

A request proxy that attributes blocking completion time to MPI wait.

Waitmethod#

View source

def Wait(status=None)

waitmethod#

View source

def wait(status=None)

ProfilingOptionsclassdataclass#

Re-exported from scope_profiler.profile_configView source

class ProfilingOptions(file_path: str | None = None, label: str | None = None, use_likwid: bool | None = None, perf_events: list[str] | tuple[str, ...] | str | None = None, use_line_profiler: bool | None = None, use_memray: bool | None = None, memory_profile_path: str | None = None, memray_native_traces: bool | None = None, memray_trace_python_allocators: bool | None = None, memray_follow_fork: bool | None = None, deactivate_profiling: bool | None = None, use_nvtx: bool | None = None, use_gpu_timing: bool | None = None, gpu_timing_backend: Any = None, deactivate_file_output: bool | None = None, recursive_profile: bool | None = None, aggregation_mode: bool | None = None, profile_mpi_calls: bool | None = None, track_threads: bool | None = None, track_async: bool | None = None, capture_region_source: bool | None = None, metadata_detail: str | None = None, buffer_limit: int | None = None, output_mode: str | None = None, hdf5_compression: str | None = None, hdf5_compression_level: int | None = None, hdf5_chunk_size: int | None = None, enable_tags: list[str] | tuple[str, ...] | None = None, memray: MemrayOptions | None = None, gpu: GPUOptions | None = None, hdf5: HDF5Options | None = None, gpu_sync_on_exit: bool | None = None)

A bag of :meth:ProfileManager.setup settings, to reuse or pass around.

Every field mirrors a setup()/session() keyword argument and defaults to None, meaning “unset - let setup()’s own default, or its config_path file, decide”. Pass one to setup() or session() instead of repeating the same handful of keyword arguments at every call site::

options = ProfilingOptions(use_likwid=True, file_path="run.h5")
ProfileManager.setup(options=options)
# ... or, equivalently:
with ProfileManager.session(options=options):
    ...

An explicit keyword argument passed alongside options still wins over the same field on options, so a shared ProfilingOptions can be reused across runs with one-off overrides::

ProfileManager.setup(options=options, file_path="run_b.h5")

Settings that share a prefix can also be given as groups — :class:MemrayOptions, :class:GPUOptions and :class:HDF5Options — which spell the same settings without the prefix::

options = ProfilingOptions(
    file_path="run.h5",
    hdf5=HDF5Options(compression="gzip", compression_level=4),
)

Setting one thing both ways raises ValueError rather than picking a winner.

Attributes

file_path : str or None Path to the output profiling data file (default: "profiling_data.h5"). label : str or None Short name for this run (default: None, i.e. the output file’s stem). Post-processing uses it wherever a run has to be named — chart legends, the summary heading, scope-profiler inspect, the JSON statistics - and it is stored in the output file as the label metadata field, so it survives into every later post-processing step. use_likwid : bool or None Enable LIKWID hardware counter collection (default: False). perf_events : sequence of str or None Linux perf_event_open events to collect per region (default: None). use_line_profiler : bool or None Enable line-by-line profiling via line_profiler (default: False). use_memray : bool or None Record process-wide memory allocations with Memray (default: False). deactivate_profiling : bool or None Turn profiling off entirely (setup() default: False). Every region becomes a no-op, so the instrumentation can stay in the code at near-zero cost instead of being removed. use_nvtx : bool or None Add NVTX ranges to profiled regions for NVIDIA Nsight tools (default: False). Requires scope-profiler[nvtx]. use_gpu_timing : bool or None Record CUDA-event elapsed device time for each profiled region (default: False). CPU timestamps are still recorded, so the normal timeline remains enqueue-side timing. gpu_sync_on_exit : bool or None Wait for the GPU device at region exit. Requires use_gpu_timing and removes CPU/GPU overlap; disabled by default. gpu_timing_backend : str, object, or None CUDA-event backend for use_gpu_timing: "auto", "torch", "cupy", or a custom object implementing record_event() and elapsed_time_ns(start_event, end_event) (default: "auto"). deactivate_file_output : bool or None Write no HDF5 file at all (default: False), not even the run metadata. Pair it with finalize(return_results=True) to analyse a run entirely in memory. recursive_profile : bool or None Enable recursive profiling for all decorated functions by default (default: False). Overridable per decorator with @ProfileManager.profile(..., recursive=...). aggregation_mode : bool or None Record only count, inclusive total, minimum, maximum, and exclusive total per region (default: False). Timeline events are unavailable in this mode; it cannot be combined with line, GPU, NVTX, or LIKWID profiling. profile_mpi_calls : bool or None Profile mpi4py operations made through predefined or derived communicators (default: False). mpi4py remains lazily imported. track_threads : bool or None Record which thread each call ran on, and describe every thread the run touched (default: False). Required for correct results from concurrently used regions; see :class:~scope_profiler.region_profiler.ThreadedProfileRegion. track_async : bool or None Also record which asyncio task or greenlet each call ran in, and how much of the call was spent awaiting (default: False). Implies track_threads. capture_region_source : bool or None Record where each region is defined - the with block or the decorated function - once per distinct source file, the first time any of its regions is created (default: False). See :attr:~scope_profiler.region.Region.source_text. Off by default: the cost tracks that file’s total size (one ast.parse + tree walk), so it stays under a millisecond for a typical file but can reach tenths of a second per rank for a single file with thousands of lines, paid independently by every rank. metadata_detail : {“minimal”, “full”} or None How much of the run’s environment is recorded (default: "minimal"). "minimal" stores versions, platform, CPU model and thread/rank counts, and each region’s and line profile’s source file relative to the working directory (or by name alone, outside it). "full" adds the user name, host name, working directory, loaded modules, environment variables such as PATH and VIRTUAL_ENV, SLURM_* variables and absolute source paths; opt into it for runs that stay on the machine they were recorded on. buffer_limit : int or None Initial number of profiling events preallocated per region (default: 1024). Buffers grow on demand, so this is a starting size rather than a limit; raise it for very hot regions to avoid repeated reallocation. output_mode : str or None MPI file writer: "auto", "direct", or "parallel" (default: "auto"). auto prefers MPI-enabled h5py when compatible with the active instrumentation and otherwise lets ranks append directly to one serial-HDF5 file in token order. hdf5_compression : str or None Compression filter for timestamp and GPU-duration datasets: None, "auto", "gzip", "lzf", or "zstd" (default: None). "auto" compresses at publication time, and only when the run is large enough for the saving to repay the write CPU. hdf5_compression_level : int or None GZIP level 0—9 or Zstandard level 1—22 (default: None). hdf5_chunk_size : int or None Maximum events per dataset chunk (default: None). Enables chunked partial reads even without compression. enable_tags : sequence of str or None Opt-in region tags to record (default: None). Regions carrying a tag registered with :meth:ProfileManager.register_opt_in_tag - typically a library’s own instrumentation - stay disabled unless one of their opt-in tags is listed here, in region_tags, or in the comma-separated SCOPE_PROFILER_ENABLE_TAGS environment variable. "*" enables every opt-in tag. memray : MemrayOptions or None The use_memray/memory_profile_path/memray_* settings as one group. gpu : GPUOptions or None The use_gpu_timing/gpu_timing_backend/use_nvtx settings as one group. hdf5 : HDF5Options or None The hdf5_* settings as one group.

aggregation_modeattributeclass attributeinstance attribute#

View source

aggregation_mode: bool | None = None

buffer_limitattributeclass attributeinstance attribute#

View source

buffer_limit: int | None = None

capture_region_sourceattributeclass attributeinstance attribute#

View source

capture_region_source: bool | None = None

deactivate_file_outputattributeclass attributeinstance attribute#

View source

deactivate_file_output: bool | None = None

deactivate_profilingattributeclass attributeinstance attribute#

View source

deactivate_profiling: bool | None = None

enable_tagsattributeclass attributeinstance attribute#

View source

enable_tags: list[str] | tuple[str, ...] | None = None

file_pathattributeclass attributeinstance attribute#

View source

file_path: str | None = None

gpuattributeclass attributeinstance attribute#

View source

gpu: GPUOptions | None = None

gpu_sync_on_exitattributeclass attributeinstance attribute#

View source

gpu_sync_on_exit: bool | None = None

gpu_timing_backendattributeclass attributeinstance attribute#

View source

gpu_timing_backend: Any = None

hdf5attributeclass attributeinstance attribute#

View source

hdf5: HDF5Options | None = None

hdf5_chunk_sizeattributeclass attributeinstance attribute#

View source

hdf5_chunk_size: int | None = None

hdf5_compressionattributeclass attributeinstance attribute#

View source

hdf5_compression: str | None = None

hdf5_compression_levelattributeclass attributeinstance attribute#

View source

hdf5_compression_level: int | None = None

labelattributeclass attributeinstance attribute#

View source

label: str | None = None

memory_profile_pathattributeclass attributeinstance attribute#

View source

memory_profile_path: str | None = None

memrayattributeclass attributeinstance attribute#

View source

memray: MemrayOptions | None = None

memray_follow_forkattributeclass attributeinstance attribute#

View source

memray_follow_fork: bool | None = None

memray_native_tracesattributeclass attributeinstance attribute#

View source

memray_native_traces: bool | None = None

memray_trace_python_allocatorsattributeclass attributeinstance attribute#

View source

memray_trace_python_allocators: bool | None = None

metadata_detailattributeclass attributeinstance attribute#

View source

metadata_detail: str | None = None

output_modeattributeclass attributeinstance attribute#

View source

output_mode: str | None = None

perf_eventsattributeclass attributeinstance attribute#

View source

perf_events: list[str] | tuple[str, ...] | str | None = None

profile_mpi_callsattributeclass attributeinstance attribute#

View source

profile_mpi_calls: bool | None = None

recursive_profileattributeclass attributeinstance attribute#

View source

recursive_profile: bool | None = None

track_asyncattributeclass attributeinstance attribute#

View source

track_async: bool | None = None

track_threadsattributeclass attributeinstance attribute#

View source

track_threads: bool | None = None

use_gpu_timingattributeclass attributeinstance attribute#

View source

use_gpu_timing: bool | None = None

use_likwidattributeclass attributeinstance attribute#

View source

use_likwid: bool | None = None

use_line_profilerattributeclass attributeinstance attribute#

View source

use_line_profiler: bool | None = None

use_memrayattributeclass attributeinstance attribute#

View source

use_memray: bool | None = None

use_nvtxattributeclass attributeinstance attribute#

View source

use_nvtx: bool | None = None

to_kwargsmethod#

View source

def to_kwargs() -> dict

This options’ explicitly-set fields, as setup() keyword arguments.

Group fields are expanded to the flat names they stand for. Setting the same thing both ways raises ValueError rather than picking a winner.

ProfilingResultsclass#

Re-exported from scope_profiler.resultsView source

class ProfilingResults(regions: dict[str, MPIRegion], metadata: dict | None = None, num_ranks: int | None = None, likwid: dict[int, dict[str, LikwidRegionResult]] | None = None, perf_events: dict[int, dict] | None = None, line_profile: dict[int, list] | None = None, file_path: str | Path = '', is_root: bool = True, exclusive_totals: dict[str, dict[int, int]] | None = None, event_data_available: bool = True, threads: dict[int, list] | None = None, tasks: dict[int, list] | None = None)

The profiling data of one run, as an ordered mapping of region name to :class:~scope_profiler.mpi_region.MPIRegion::

results = ProfileManager.finalize(return_results=True)
for region in results:
    print(region.name, region.total_duration)

solve = results["solve"]        # same as results.get_region("solve")
solve[0].average_duration       # rank 0, in seconds

The same object comes back from a merged output file, via :meth:from_h5 (or its module-level twin :func:~scope_profiler.h5reader.read_h5)::

results = ProfilingResults.from_h5("profiling_data.h5")

All durations are reported in seconds.

display_labelproperty#

View source

display_label: str

What to call this run in charts, tables and exports.

The :attr:label when the run has one, and otherwise the stem of its output file --- which is what post-processing named runs by before labels existed, and remains the default.

Returns

str A non-empty name for the run.

file_pathproperty#

View source

file_path: Path

The run’s output file.

Returns

Path The file path as a pathlib.Path object. For results taken straight from memory this is the path setup() was configured with, which need not exist on disk.

finalize_timeproperty#

View source

finalize_time: float | None

When finalize() was called, in seconds on the recording clock.

This is the finalize_time_ns metadata field, read as the first thing ProfileManager.finalize() does - before it spends any time collecting or writing the run’s data, so it marks the moment finalize() was reached rather than the moment it returned.

Returns

float or None The registered finalize time, or None for runs without one (older files, or a run that never reached finalize()).

has_event_dataproperty#

View source

has_event_data: bool

Whether per-call timestamps are available to event-based APIs.

has_likwidproperty#

View source

has_likwid: bool

Whether the run recorded LIKWID hardware counter results.

has_perf_eventsproperty#

View source

has_perf_events: bool

Whether this run recorded built-in Linux perf-event counters.

is_rootproperty#

View source

is_root: bool

Whether this rank holds the run’s data.

Always True for a file that was read back, and for serial runs. Under MPI, finalize(return_results=True) gathers everything on rank 0, so only rank 0’s results are the root ones; the others come back empty and with this False.

Everything that produces output - :meth:print_summary, the plot_* functions, the exporters - does nothing for non-root results. That is what lets a parallel script call them unguarded and still write each figure exactly once, from rank 0. Read it when a script needs to make the same distinction for output of its own.

Returns

bool True unless this is a non-root rank’s share of an MPI run.

labelpropertywritable#

View source

label: str | None

The run’s label, as given to ProfileManager.setup(label=...).

Returns

str or None The label, or None for a run that was not given one. Use :attr:display_label to name the run in output regardless.

likwid_ranksproperty#

View source

likwid_ranks: list[int]

Ranks that recorded LIKWID results, in ascending order.

line_profileproperty#

View source

line_profile: dict[int, list]

Persisted line-profiler records keyed by rank.

Each record contains region, filename, function, first_lineno, line_numbers, hits, times and unit. The elapsed seconds for a line are times * unit.

maximum_end_timeproperty#

View source

maximum_end_time: float

Get the maximum end time across all regions and ranks.

Returns

float Maximum end time in seconds, or 0.0 if no region recorded timing.

metadataproperty#

View source

metadata: dict

Get environment metadata for the run (gathered from rank 0).

Returns

dict Metadata dict (hostname, OpenMP thread count, platform, versions, etc.), or an empty dict if the run recorded none.

minimum_start_timeproperty#

View source

minimum_start_time: float

Get the minimum start time across all regions and ranks.

This is the origin of the timeline: subtract it from any timestamp to get seconds since the first region entry. Regions with no recorded calls are ignored; the result is 0.0 if no region recorded any.

Returns

float Minimum start time in seconds.

num_nodesproperty#

View source

num_nodes: int | None

Number of distinct hosts the run’s ranks ran on, or None if unknown.

Read from the num_nodes metadata field. Files written before it was recorded still answer for a single-rank run, which can only have used one node; a multi-rank run without the field returns None.

num_ranksproperty#

View source

num_ranks: int

Get the number of ranks recorded in the profiling data.

Returns

int Number of ranks.

region_namesproperty#

View source

region_names: list[str]

Names of all regions, in order of appearance.

run_start_timeproperty#

View source

run_start_time: float | None

When the run started, in seconds on the recording clock.

This is the start_time_ns metadata field, written by ProfileManager.setup() — by default the moment setup() was called, or an earlier instant if one was passed to it.

Returns

float or None The registered start time, or None for runs without one (older files, or runs that never called setup()). Use :attr:startup_time for the elapsed time before the first region, which stays defined either way.

startup_timeproperty#

View source

startup_time: float

Seconds between the start of the run and the first profiled region.

Time the instrumentation never saw: imports, reading input, building a mesh. Zero when there is no registered start time (:attr:run_start_time is None), since the run is then only known from its first region onwards.

Returns

float Elapsed time before the first region entry, in seconds.

tasksproperty#

View source

tasks: dict[int, list]

Rank -> the asyncio tasks and greenlets the run followed.

Each entry is a :class:~scope_profiler.concurrency.TaskInfo, whose running_time and awaiting_time split the lane’s life into the part it held a thread and the part it was suspended. Empty unless the run profiled with track_async=True.

threadsproperty#

View source

threads: dict[int, list]

Rank -> the threads that recorded calls, in registration order.

Each entry is a :class:~scope_profiler.concurrency.ThreadInfo: name, OS ids, when the thread started and ended, and the CPU time it burned. Empty unless the run profiled with track_threads=True::

for rank, threads in results.threads.items():
    for thread in threads:
        print(rank, thread.name, thread.cpu_time)

The per-call thread_ids column of every region indexes into this rank’s list, so results["solve"][0].for_thread(2) is the part of solve that ran on results.threads[0][2].

time_originproperty#

View source

time_origin: float

Zero point of the relative timeline, in seconds.

The registered :attr:run_start_time when there is one, and otherwise the first region entry (:attr:minimum_start_time). This is what :meth:events and :meth:call_stack measure from.

Note the plot_* functions instead frame their x axis on the first region entry, so that a long gap between setup() and the first region does not fill a chart with empty space. Pass origin=results.minimum_start_time to :meth:events to reproduce the numbers on a chart’s axis.

Returns

float Origin timestamp on the recording clock.

time_spanproperty#

View source

time_span: float

Wall-clock seconds between the first region entry and the last exit.

Returns

float Duration of the profiled window in seconds, or 0.0 if no region recorded timing.

total_timeproperty#

View source

total_time: float | None

Wall-clock seconds from setup() to finalize().

Unlike :attr:time_span (first region entry to last exit), this covers the whole instrumented program: startup work before the first region, gaps between regions, and any teardown after the last one but before finalize() is called - the number to report as “how long did the run take” alongside the region breakdown.

Returns

float or None :attr:finalize_time minus :attr:run_start_time, or None if either is missing - an older file, or a run that set up profiling (or called finalize) some other way than ProfileManager.setup()/finalize().

call_graphmethod#

View source

def call_graph(rank: int = 0, include=None, exclude=None) -> list[dict]

Return call relationships without timestamps or durations.

New profiles persist call_id and parent_id for every Python event. Legacy profiles fall back to the timestamp-based call stack. The returned nodes contain only call_id, parent_id, name, call_index and depth.

Filtering renests: a call whose parent was excluded is reported under its nearest surviving ancestor, with the depth that implies, exactly as :meth:call_stack does. Leaving the excluded id in parent_id would hand back a graph with edges pointing at nodes that are not in it.

call_id is unique within this rank, not across the file - each rank numbers its own calls.

call_stackmethod#

View source

def call_stack(rank: int = 0, include: list[str] | str | None = None, exclude: list[str] | str | None = None, relative: bool = True, origin: float | None = None) -> list[dict]

Reconstruct the nested call stack for one rank.

Regions record no call graph, so nesting is recovered from timestamp containment - the same reconstruction the flame chart, the .prof export and the speedscope export use.

Parameters

rank : int, optional Rank whose calls to reconstruct (default: 0). include, exclude : list of str or str, optional Regex patterns selecting which regions to include, matched as in :meth:get_regions. relative : bool, optional If True (default), timestamps start at zero; see :meth:events. origin : float, optional Measure from this timestamp instead; see :meth:events.

Returns

List[dict] One entry per call, parents before children, with keys call_id, name, start, end, duration, depth and parent (the enclosing call’s id, or None). See :func:~scope_profiler.call_stack.build_call_stack.

default_titlemethod#

View source

def default_title() -> str

Heading naming this run, for a summary table.

The label leads when the run has one, since that is the name the user chose; the file path follows either way, because when several runs are being compared it is what tells them apart on disk.

Returns

str e.g. "results/run_a.h5 (128 ranks)".

eventsmethod#

View source

def events(include: list[str] | str | None = None, exclude: list[str] | str | None = None, ranks: list[int] | int | None = None, relative: bool = True, origin: float | None = None) -> list[dict]

Return one dict per recorded call, across all regions and ranks.

This is the long-form (“tidy”) view to build custom plots from: one row per call rather than one per region.

Parameters

include, exclude : list of str or str, optional Regex patterns selecting which regions to include, matched as in :meth:get_regions. ranks : list of int or int, optional Restrict to these ranks (default: all). relative : bool, optional If True (default), timestamps are measured from :attr:time_origin — the start time registered by ProfileManager.setup(), or the first region entry for runs without one — so the timeline starts at zero. If False, the raw monotonic-clock timestamps are returned; those are only comparable within a single run. origin : float, optional Measure from this timestamp instead, in seconds on the recording clock. Overrides relative; pass origin=results.minimum_start_time to zero the timeline on the first region entry regardless of what the run registered.

Returns

List[dict] Entries with keys name, rank, call_index, start, end and duration, in seconds, ordered by region (as in :meth:get_regions) then rank then call order. Entries also carry gpu_duration when CUDA-event timing was enabled for that region.

Examples

for event in results.events(include=“solve”): # doctest: +SKIP … print(event[“rank”], event[“start”], event[“duration”])

from_h5methodclassmethod#

View source

def from_h5(file_path: str | Path, verbose: bool = False) -> ProfilingResults

Load a merged profiling file written by :meth:ProfileManager.finalize <scope_profiler.profile_manager.ProfileManager.finalize>::

results = ProfilingResults.from_h5("profiling_data.h5")
results.print_summary()

:func:scope_profiler.read_h5 is the same thing under a shorter name.

Parameters

file_path : str | Path Path to the merged HDF5 file containing profiling data. verbose : bool, optional Print each rank group as it is read (default: False).

Returns

ProfilingResults The run’s profiling data, of whichever class this was called on.

Raises

FileNotFoundError If the specified HDF5 file does not exist.

get_likwid_regionmethod#

View source

def get_likwid_region(tag: str, rank: int = 0) -> LikwidRegionResult

Get one region’s LIKWID results for a single rank.

Parameters

tag : str LIKWID marker region tag (the profiled region’s name). rank : int, optional Rank to read from (default: 0).

Returns

LikwidRegionResult The region’s counters, event names and derived metrics.

Raises

KeyError If the rank recorded no LIKWID data or has no such region.

get_likwid_regionsmethod#

View source

def get_likwid_regions(rank: int | None = None) -> dict

Get the LIKWID marker results of the run.

Parameters

rank : int, optional Return only this rank’s regions. By default every rank is included, keyed by rank.

Returns

dict With rank given, a mapping of region tag to :class:~scope_profiler.likwid_data.LikwidRegionResult; otherwise a mapping of rank to such a dict. Empty when the run did not use LIKWID.

Examples

::

results = read_h5("profiling_data.h5")
for rank, regions in results.get_likwid_regions().items():
    for tag, result in regions.items():
        for name, values in zip(result.metric_names, result.metrics):
            print(rank, tag, name, values)

get_perf_eventsmethod#

View source

def get_perf_events(rank: int | None = None) -> dict

Return aggregated Linux perf_event_open counts by region.

get_regionmethod#

View source

def get_region(region_name: str) -> MPIRegion

Retrieve profiling data for a specific region.

Parameters

region_name : str Name of the region to retrieve.

Returns

Region Region object containing profiling data for all ranks.

Raises

KeyError If the specified region name does not exist.

get_regionsmethod#

View source

def get_regions(include: list[str] | str | None = None, exclude: list[str] | str | None = None) -> list[MPIRegion]

Get a list of all regions in order of appearance.

Returns

List[Region] List of Region objects.

lane_labelmethod#

View source

def lane_label(lane: int, rank: int = 0) -> str

A human-readable name for one lane of rank.

Lanes are the stacks a rank’s calls were reconstructed into (see :func:~scope_profiler.concurrency.lane_ids). This turns one back into something worth putting on a chart or a speedscope profile selector: a task’s own name where the lane is a task, the thread’s name where it is a bare thread.

likwid_to_dataframemethod#

View source

def likwid_to_dataframe()

Return every LIKWID event and metric as a tidy pandas DataFrame.

One row per (rank, region, hardware thread), with a column per event and per derived metric, plus the region’s LIKWID runtime and call count. Regions measured with different event groups simply leave the other group’s columns empty.

Returns

pandas.DataFrame Empty if there is no LIKWID data.

Raises

ImportError If pandas is not installed.

print_likwid_summarymethod#

View source

def print_likwid_summary(stream=None) -> None

Print every LIKWID region’s events and derived metrics.

Parameters

stream : file-like, optional Destination (default: stdout).

print_summarymethod#

View source

def print_summary(include: list[str] | str | None = None, exclude: list[str] | str | None = None, ranks: list[int] | None = None, sort: str = 'start', title: str | None = None, stream=None, suppress_notes: bool = False, columns: list[str] | str | None = None, percentage_mode: str = 'coverage') -> None

Print a region summary table, aggregated over ranks.

Renders the same table as scope-profiler inspect and the summary printed by ProfileManager.finalize().

Parameters

include, exclude : list of str or str, optional Regex patterns selecting which regions to print, matched as in :meth:get_regions. ranks : list of int, optional Restrict the statistics to these ranks (default: all). sort : str, optional Column to order by: start (default), total, calls, avg, min, max, std or name. title : str, optional Heading above the table (default: the file path and rank count). stream : file-like, optional Where to write (default: stdout). columns : list of str or str, optional Region summary columns to print. Defaults to region, calls, percent, parent_percent, total and avg. percent is relative to scope_profiler.session; parent_percent is relative to the enclosing call path. Use region for the region-name column. percentage_mode : {“coverage”, “exclusive”}, optional Quantity used for % session. Defaults to wall-clock coverage; use exclusive to attribute time after nested regions.

Notes

Does nothing on a non-root rank (see :attr:is_root), so a parallel script can call it unguarded and print the table once.

summarymethod#

View source

def summary(include: list[str] | str | None = None, exclude: list[str] | str | None = None) -> list[dict]

Summarize every region, aggregated over ranks.

Parameters

include, exclude : list of str or str, optional Regex patterns selecting which regions to summarize, matched as in :meth:get_regions.

Returns

List[dict] One dict per region (see :meth:~scope_profiler.mpi_region.MPIRegion.get_summary), ordered by first start time. inclusive_duration includes nested regions; exclusive_duration excludes them. Durations are in seconds.

thread_summarymethod#

View source

def thread_summary(rank: int = 0) -> list[dict]

Per-thread wall time, CPU time and call count for one rank.

Wall and CPU time come from the thread table; the call count and the time attributed to regions come from the events, so a thread that burned CPU outside every profiled region shows the gap directly.

Returns

list of dict One entry per thread with keys index, name, alive, wall_time, cpu_time, num_calls and region_time (the summed inclusive duration of that thread’s top-level calls, so nested regions are not counted twice), in thread-table order. Empty when the run did not track threads.

to_dataframemethod#

View source

def to_dataframe(include: list[str] | str | None = None, exclude: list[str] | str | None = None, per_rank: bool = False)

Return the region summaries as a pandas DataFrame.

Parameters

include, exclude : list of str or str, optional Regex patterns selecting which regions to include, matched as in :meth:get_regions. per_rank : bool, optional If True, emit one row per (region, rank) with a rank column instead of one aggregated row per region (default: False).

Returns

pandas.DataFrame Region statistics, with durations in seconds.

Raises

ImportError If pandas is not installed.

to_events_dataframemethod#

View source

def to_events_dataframe(include: list[str] | str | None = None, exclude: list[str] | str | None = None, ranks: list[int] | int | None = None, relative: bool = True, origin: float | None = None)

Return every recorded call as a pandas DataFrame (one row per call).

Parameters

include, exclude, ranks, relative, origin As in :meth:events.

Returns

pandas.DataFrame Columns name, rank, call_index, start, end and duration, with times in seconds. A gpu_duration column is included when any selected event has CUDA-event timing.

Raises

ImportError If pandas is not installed.

ProfilingRunclass#

Re-exported from scope_profiler.profile_managerView source

class ProfilingRun(manager, config, output=None)

Handle returned by :meth:ProfileManager.setup.

resultsattributeinstance attribute#

View source

results = None

file_pathproperty#

View source

file_path

Configured output path for this run.

is_activeproperty#

View source

is_active: bool

Whether this run’s manager currently records calls.

finalizemethod#

View source

def finalize(**kwargs)

Finalize through the owning manager and retain the results.

Regionclass#

Re-exported from scope_profiler.regionView source

class Region(start_times: np.ndarray, end_times: np.ndarray, gpu_durations: np.ndarray | None = None, call_ids: np.ndarray | None = None, parent_ids: np.ndarray | None = None, thread_ids: np.ndarray | None = None, task_ids: np.ndarray | None = None, await_times: np.ndarray | None = None, event_metadata: list[dict] | tuple[dict, ...] | None = None, source_file: str | None = None, source_lineno: int | None = None, source_text: str | None = None, tags=(), aggregate: dict | None = None, event_data_available: bool = True)

Timing data for one region on one rank.

All duration and timestamp properties are reported in seconds; the underlying HDF5 data is stored in nanoseconds.

average_durationproperty#

View source

average_duration: float

Average duration per call in seconds.

await_timesproperty#

View source

await_times: np.ndarray | None

Seconds each call spent with its task suspended, or None.

The wall time inside the with block during which the task was not running: awaiting, or switched away from. Subtracting it from the call’s duration leaves the time the lane actually held its thread.

await_times_nsproperty#

View source

await_times_ns: np.ndarray | None

Nanoseconds each call spent with its task suspended, or None.

call_idsproperty#

View source

call_ids

Explicit call ids, or None for legacy profiles.

durationsproperty#

View source

durations: np.ndarray

Duration of all calls in seconds.

durations_nsproperty#

View source

durations_ns: np.ndarray

Duration of all calls in nanoseconds, as integers.

end_timesproperty#

View source

end_times: np.ndarray

End times of all calls in seconds.

end_times_nsproperty#

View source

end_times_ns: np.ndarray

End times of all calls in nanoseconds, exactly as stored.

event_metadataproperty#

View source

event_metadata: tuple[dict, ...]

Per-call scoped metadata, aligned with the event arrays.

exclusive_durationproperty#

View source

exclusive_duration: float

Alias for :attr:total_exclusive_duration.

exclusive_durationsproperty#

View source

exclusive_durations: np.ndarray

Exclusive duration of every call in seconds.

exclusive_durations_nsproperty#

View source

exclusive_durations_ns: np.ndarray

Exclusive duration of every call in nanoseconds.

first_durationproperty#

View source

first_duration: float

Duration of the first recorded call, in seconds.

first_start_timeproperty#

View source

first_start_time: float

First start time in seconds.

gpu_average_durationproperty#

View source

gpu_average_duration: float | None

Average CUDA-event elapsed device time in seconds, or None if absent.

gpu_durationsproperty#

View source

gpu_durations: np.ndarray | None

CUDA-event elapsed device times in seconds, or None if absent.

gpu_durations_nsproperty#

View source

gpu_durations_ns: np.ndarray | None

CUDA-event elapsed device times in nanoseconds, or None if absent.

gpu_total_durationproperty#

View source

gpu_total_duration: float | None

Total CUDA-event elapsed device time in seconds, or None if absent.

has_event_dataproperty#

View source

has_event_data: bool

Whether per-call timestamps were loaded for this region.

has_gpu_timingproperty#

View source

has_gpu_timing: bool

Whether this region has CUDA-event elapsed timings.

has_sourceproperty#

View source

has_source: bool

Whether this region’s call-site source was captured.

has_thread_dataproperty#

View source

has_thread_data: bool

Whether this region recorded the thread each call ran on.

has_timingproperty#

View source

has_timing: bool

Whether this region recorded any calls at all.

inclusive_durationproperty#

View source

inclusive_duration: float

Total inclusive time, including nested regions, in seconds.

inclusive_durationsproperty#

View source

inclusive_durations: np.ndarray

Inclusive duration of every call in seconds.

inclusive_durations_nsproperty#

View source

inclusive_durations_ns: np.ndarray

Inclusive duration of every call in nanoseconds.

last_durationproperty#

View source

last_duration: float

Duration of the last recorded call, in seconds.

last_end_timeproperty#

View source

last_end_time: float

Last end time in seconds.

max_durationproperty#

View source

max_duration: float

Maximum duration among all calls in seconds.

min_durationproperty#

View source

min_duration: float

Minimum duration among all calls in seconds.

num_callsproperty#

View source

num_calls: int

Number of recorded calls.

p50_durationproperty#

View source

p50_duration: float | None

Median duration in seconds.

p95_durationproperty#

View source

p95_duration: float | None

95th-percentile duration in seconds.

p99_durationproperty#

View source

p99_duration: float | None

99th-percentile duration in seconds.

parent_idsproperty#

View source

parent_ids

Explicit parent ids, or None for legacy profiles.

source_fileproperty#

View source

source_file: str | None

Path of the file this region is defined in, or None if not captured.

source_linenoproperty#

View source

source_lineno: int | None

Line the region’s call site starts at, or None if not captured.

source_textproperty#

View source

source_text: str | None

Source text of the region’s with block or decorated function.

None if it was not captured - either the file it came from is no longer readable, or the file predates this being recorded.

start_timesproperty#

View source

start_times: np.ndarray

Start times of all calls in seconds.

start_times_nsproperty#

View source

start_times_ns: np.ndarray

Start times of all calls in nanoseconds, exactly as stored.

std_durationproperty#

View source

std_duration: float

Standard deviation of durations in seconds.

stored_summaryproperty#

View source

stored_summary: dict | None

Fixed-size statistics used by aggregate and summary-only results.

tagsproperty#

View source

tags: tuple[str, ...]

User-defined tags attached to this region.

task_idsproperty#

View source

task_ids: np.ndarray | None

Task-table index of every call, -1 outside a task.

None unless the run used track_async.

thread_idsproperty#

View source

thread_ids: np.ndarray | None

Thread-table index of every call, or None if threads were not tracked.

threadsproperty#

View source

threads: list[int]

Sorted thread-table indices this region recorded calls on.

total_await_durationproperty#

View source

total_await_duration: float | None

Total awaited seconds over every call, or None if not tracked.

total_durationproperty#

View source

total_duration: float

Total time spent in this region in seconds (sum of all durations).

total_exclusive_durationproperty#

View source

total_exclusive_duration: float

Total time excluding nested regions, in seconds.

eventsmethod#

View source

def events(origin: float = 0.0) -> list[dict[str, Any]]

Return one dict per recorded call.

Parameters

origin : float, optional Seconds subtracted from every timestamp, so passing results.minimum_start_time yields a timeline starting at zero (default: 0.0, i.e. raw timestamps).

Returns

list of dict One entry per call with keys call_index, start, end and duration, in seconds and in recorded order.

for_threadmethod#

View source

def for_thread(thread_index: int) -> Region

A view of this region holding only the calls of one thread.

Every column is sliced with the same mask, so the returned region is an ordinary :class:Region - summaries, durations and percentiles all describe that thread alone.

Raises

ValueError If this region has no thread column (the run did not use track_threads).

get_summarymethod#

View source

def get_summary() -> dict[str, Any]

Return a summary of the region’s statistics as a dictionary.

Returns

Dict[str, Any] Dictionary containing statistics: num_calls, total_duration, average_duration, min_duration, max_duration, first_duration, last_duration, and std_duration. Durations are in seconds.

percentile_durationmethod#

View source

def percentile_duration(percentile: float) -> float | None

Return a duration percentile in seconds.

percentile follows :func:numpy.percentile and must be between 0 and 100. Empty regions return 0.0 for consistency with the other duration statistics.

RegionFactoryclass#

Re-exported from scope_profiler.profile_managerView source

class RegionFactory(manager, name, tags=None)

Callable region definition for repeated scopes with metadata.

nameattributeinstance attribute#

View source

name = name

RegionHandleclass#

Re-exported from scope_profiler.profile_managerView source

class RegionHandle(manager, name, functions=None, tags=None)

Persistent region definition that follows manager reconfiguration.

nameattributeinstance attribute#

View source

name = name

build_call_arraysfunction#

Re-exported from scope_profiler.call_stackView source

def build_call_arrays(regions: Iterable, rank: int) -> CallArrays

Reconstruct one rank’s nesting as numpy arrays.

The workhorse behind :func:build_call_stack, and what anything processing a whole run should call: it never builds a dict per call, so it stays usable on the tens of millions of events a long simulation produces.

Parameters

regions : iterable of MPIRegion Regions to include, e.g. results.get_regions(). Regions with no data for rank are skipped. rank : int Rank whose calls to reconstruct.

Returns

CallArrays

Raises

NestingError If any interval ends before it starts, or two intervals overlap without one containing the other.

build_call_stackfunction#

Re-exported from scope_profiler.call_stackView source

def build_call_stack(regions: Iterable, rank: int, origin: float = 0.0) -> list[dict]

Reconstruct per-call nesting for one rank, one dict per call.

A convenience wrapper over :func:build_call_arrays for callers that want to iterate calls rather than columns. It allocates a dict per call (~700 bytes), so prefer the arrays for anything that has to scale with a long run.

Parameters

regions : iterable of MPIRegion Regions to include, e.g. results.get_regions(). rank : int Rank whose calls to reconstruct. Regions with no data for this rank are skipped. origin : float, optional Seconds subtracted from every timestamp, so passing results.minimum_start_time yields a timeline starting at zero (default: 0.0, i.e. raw timestamps).

Returns

list of dict One dict per call, ordered by start time (parents before children), with keys call_id, name, call_path, lane, start, end, duration (the inclusive duration), inclusive_duration, exclusive_duration (seconds), depth (0 for a top-level call) and parent (the call_id of the enclosing call in this same list, or None). call_path joins a call and each of its ancestors with " > ", keeping same-named regions at different call sites distinct. A color key carries whatever the plotting code assigned to the region. call_id is stable for the returned list and is unique within this rank/stack reconstruction.

Raises

NestingError If the intervals are not properly nested.

Notes

Calls are indexed by position in the returned list rather than by name, because a region called more than once - or recursively - contributes several calls under one name.

call_stack_childrenfunction#

Re-exported from scope_profiler.call_stackView source

def call_stack_children(calls: list[dict]) -> list[list[int]]

Child indices per call, for walking a :func:build_call_stack result.

Returns a list parallel to calls: entry i holds the indices of the calls directly nested inside call i, in start-time order.

call_stack_rootsfunction#

Re-exported from scope_profiler.call_stackView source

def call_stack_roots(calls: list[dict]) -> list[int]

Indices of the top-level calls in a :func:build_call_stack result.

collect_file_metadatafunction#

Re-exported from scope_profiler.inspectionView source

def collect_file_metadata(profiling_data: ProfilingResults | str | Path | Sequence) -> dict

Collect the run metadata of one or more profiling files.

Parameters

profiling_data : ProfilingResults, path, or sequence of either Files to read the metadata from.

Returns

dict {"files": [{"file_path": ..., "num_ranks": ..., "metadata": {...}}]}, matching the envelope used by :func:~scope_profiler.plotting_scripts.collect_region_statistics, so a single document can describe several runs.

collect_region_statisticsfunction#

Re-exported from scope_profiler.plotting_scripts.statisticsView source

def collect_region_statistics(profiling_data: ProfilingResults | Sequence[ProfilingResults], ranks: list[int] | int | None = None, include: list[str] | str | None = None, exclude: list[str] | str | None = None, labels: Sequence[str] | None = None) -> dict

Collect aggregate region-duration statistics for one or more profiling files.

collect_roofline_pointsfunction#

Re-exported from scope_profiler.plotting_scripts.rooflineView source

def collect_roofline_points(profiling_data: ProfilingResults | Sequence[ProfilingResults], *, flops_metric: str | None = None, bandwidth_metric: str | None = None, ranks: list[int] | int | None = None, include: list[str] | str | None = None, exclude: list[str] | str | None = None) -> list[dict]

Derive one roofline point per selected LIKWID region and rank.

flops_metric and bandwidth_metric select exact LIKWID derived metric names. When omitted, the first names containing FLOP/s and bandwidth respectively are selected for each region. Supplying exact names is recommended when a group offers both single- and double-precision FLOP rates.

create_html_reportfunction#

Re-exported from scope_profiler.html_reportView source

def create_html_report(profiling_data: ProfilingResults | str | Path | Sequence[ProfilingResults | str | Path], filepath: str | Path, *, include=None, exclude=None, ranks: list[int] | None = None, sort: str = 'start', columns=None, charts_cdn: bool = False, include_charts: bool = True, individual_reports: bool = True, scaling: str = 'both', speedup_x: str = 'auto') -> Path

Write a standalone HTML report for one or more profiling results.

One run gets a full report. Several runs get a comparison report, the first run being the baseline; with individual_reports (the default) a full report for each run is also written next to it, as <stem>-<index>-<label>.html, and linked from the comparison.

When the compared runs differ in ranks or threads, scaling says what kind of study they are, which the profiles cannot tell: "strong" (the same total problem size on every run) charts each region’s speedup, "weak" (the same problem size per rank or core) its weak-scaling efficiency, and "both" (the default) charts both.

speedup_x sets the x-axis of a comparison’s scaling charts: "ranks" (MPI ranks), "nodes", "threads" (OpenMP threads) or "cores" (ranks times threads). "auto", the default, uses MPI ranks when they change, threads when only they change, and draws no scaling chart when the runs differ in neither. A named axis must be recorded by every run, or this raises ValueError. Either way, each chart has a button for every other axis the runs differ in.

export_chrome_tracefunction#

Re-exported from scope_profiler.chrome_trace_exportView source

def export_chrome_trace(profiling_data: ProfilingResults | Sequence[ProfilingResults], filepath: str | Path, ranks: list[int] | int | None = None, include: list[str] | str | None = None, exclude: list[str] | str | None = None, verbose: bool = True) -> list[Path]

Write Chrome Trace JSON files, one per input profiling run.

export_flamegraph_svgfunction#

Re-exported from scope_profiler.prof_exportView source

def export_flamegraph_svg(profiling_data: ProfilingResults | Sequence[ProfilingResults], filepath: str | Path, ranks: list[int] | int | None = None, include: list[str] | str | None = None, exclude: list[str] | str | None = None, call_paths: bool = True, width: int = 1200, threshold: float = 0.001, scalable: bool = True, verbose: bool = True) -> list[Path]

Render per-rank flame graphs to standalone SVG files with flameprof.

The aggregated, call-stack view of the same reconstruction :func:export_prof writes: every call of a region below one parent is one frame, sized by cumulative time. :func:~scope_profiler.plot_flame draws the same thing through the plotting stack; this goes through pstats instead, so the result is a self-contained SVG with no JavaScript, which is what a static site or an emailed report wants.

Requires flameprof (pip install "scope-profiler[pproc]").

Parameters

filepath : str | Path Base output path, e.g. figures/flamegraph.svg; a _rank<N> suffix is appended per rank, as for :func:export_prof. width : int Width of the rendered document in pixels. With scalable this only sets the aspect ratio and the point at which frame labels are elided. threshold : float Smallest fraction of total time a frame must take to be drawn (flameprof’s own default, 0.001, is a tenth of a percent). scalable : bool Replace the fixed pixel size with a viewBox so the graph scales to whatever element it is placed in, and drop the XML prolog so it can be inlined in an HTML page. Set False for a standalone SVG file.

Returns

list[Path] The files written, in the order they were written.

export_jsonfunction#

Re-exported from scope_profiler.json_exportView source

def export_json(profiling_data: ProfilingResults | Sequence[ProfilingResults], filepath: str | Path, ranks: list[int] | int | None = None, include: list[str] | str | None = None, exclude: list[str] | str | None = None, verbose: bool = True, indent: int | None = None) -> list[Path]

Write one JSON document per run, exactly as export_speedscope does.

Parameters

profiling_data : ProfilingResults | Sequence[ProfilingResults] The run(s) to export: file runs, in-memory results, or a mix. filepath : str | Path Base output path, e.g. figures/profile.json. Ending it in .gz compresses the output. The run’s label is appended to the stem when more than one run is exported. ranks : list[int] | int, optional Ranks to export (default: every rank). include, exclude : list[str] | str, optional Region name filters, as for the plotting functions. indent : int, optional Passed to json.dumps; the default writes the document on one line.

Returns

list[Path] The files written, in the order they were written.

export_proffunction#

Re-exported from scope_profiler.prof_exportView source

def export_prof(profiling_data: ProfilingResults | Sequence[ProfilingResults], filepath: str | Path, ranks: list[int] | int | None = None, include: list[str] | str | None = None, exclude: list[str] | str | None = None, call_paths: bool = True, verbose: bool = True) -> list[Path]

Write per-rank .prof files readable by pstats/snakeviz.

Parameters

profiling_data : ProfilingResults | Sequence[ProfilingResults] The run(s) to export: file runs, in-memory results from ProfileManager.finalize(return_results=True), or a mix. filepath : str | Path Base output path, e.g. figures/profile.prof. A _rank<N> suffix is appended per rank (and the input file’s stem too, when more than one file is exported), since .prof has no notion of ranks or runs. ranks : list[int] | int, optional Ranks to export (default: rank 0 only). include, exclude : list[str] | str, optional Region name filters, as for the plotting functions. call_paths : bool, optional Preserve each reconstructed parent/child path as a separate node in the exported pstats tree. This makes SnakeViz distinguish same-named regions called below different parents. Set to False for a compact, name-aggregated export. verbose : bool, optional Print each file written, with the snakeviz command that opens it.

Returns

list[Path] The files written, in the order they were written.

export_speedscopefunction#

Re-exported from scope_profiler.speedscope_exportView source

def export_speedscope(profiling_data: ProfilingResults | Sequence[ProfilingResults], filepath: str | Path, ranks: list[int] | int | None = None, include: list[str] | str | None = None, exclude: list[str] | str | None = None, verbose: bool = True) -> list[Path]

Write speedscope JSON files for the selected ranks.

One file is written per input HDF5 file, holding one profile per rank --- or, for a run profiled with track_threads, one per thread and task. Unlike .prof, the format carries several profiles per file, and speedscope switches between them from its profile selector.

Parameters

profiling_data : ProfilingResults | Sequence[ProfilingResults] The run(s) to export: file runs, in-memory results from ProfileManager.finalize(return_results=True), or a mix. filepath : str | Path Base output path, e.g. figures/profile.speedscope.json. The input file’s stem is appended when more than one file is exported. ranks : list[int] | int, optional Ranks to export (default: rank 0 only). include, exclude : list[str] | str, optional Region name filters, as for the plotting functions.

Returns

list[Path] The files written, in the order they were written.

format_mpi_regionfunction#

Re-exported from scope_profiler.mpi_wrappersView source

def format_mpi_region(operation: str, *, bytes: int = -1, peer: int = -1, root: int = -1, tag: int = -1, communicator: int = -1, request: str | None = None) -> str

Return the canonical region name shared by Python and C++ wrappers.

inspect_filefunction#

Re-exported from scope_profiler.inspectionView source

def inspect_file(file_path, include=None, exclude=None, ranks=None, sort: str = 'start', show_metadata: bool = True, show_regions: bool = True, full: bool = False, source: list | None = None, columns=None, percentage_mode: str = 'coverage', stream=None) -> None

Print a summary of one profiling HDF5 file.

Parameters

file_path : str or Path Merged profiling file to inspect. include, exclude : list of str or str, optional Regex patterns selecting which regions to report. ranks : list of int, optional Restrict the region statistics to these ranks (default: all). sort : str, optional Region ordering: one of total, calls, avg, max (all descending) or name (alphabetical). Default: total. show_metadata, show_regions : bool, optional Sections to print (both default to True). full : bool, optional Print long metadata values in full instead of clipping them. source : list of str, optional Region names to print the captured call-site source of (the with block or decorated function that defines each region - see ProfileManager.profile_region and issue #161). Printed after the region table, regardless of include/exclude. columns : list of str or str, optional Region summary columns to print. Defaults to region, calls, percent, total and avg. percentage_mode : {“coverage”, “exclusive”}, optional Quantity used for % session. Defaults to wall-clock coverage. stream : file-like, optional Where to write (default: stdout).

loadfunction#

Re-exported from scope_profiler.profile_ioView source

def load(path, verbose: bool = False) -> ProfilingResults

Load a profile for post-processing, whatever format it is in.

The one entry point for reading a run back::

import scope_profiler

results = scope_profiler.load("profiling_data.h5")
results.print_summary()

HDF5 and JSON profiles are both accepted, and the format is taken from the file’s contents rather than its name (see :func:sniff_profile_format), so the caller does not have to know which of :func:~scope_profiler.h5reader.read_h5 and :func:~scope_profiler.json_export.read_json it wanted. Those remain available for reading a file whose format is already known.

Parameters

path : str | Path The profile to read. verbose : bool, optional Print progress while reading (default: False).

Returns

ProfilingResults The run’s profiling data, with all durations in seconds.

Raises

FileNotFoundError If path does not exist. ValueError If path is an HTML report. A report is a rendering of a run, not a copy of it, so there is nothing to read back.

load_proffunction#

Re-exported from scope_profiler.prof_exportView source

def load_prof(filepath: str | Path) -> pstats.Stats

Read a .prof file written by :func:export_prof back as a Stats.

The .prof twin of :func:~scope_profiler.json_export.read_json.

merge_resultsfunction#

Re-exported from scope_profiler.resultsView source

def merge_results(*result_sets, label: str | None = None, file_path=None)

Combine several result sets into one.

The case this exists for is a mixed-language run: a Python driver records its own regions while the Fortran (or other) code it calls records theirs, and what the user wants at the end is one profile covering both. Ranks line up by number, so rank 3’s Python regions and rank 3’s Fortran regions end up side by side.

Parameters

*result_sets : ProfilingResults The sets to combine. Non-root sets (the empty ones every rank but 0 gets back under MPI) are ignored, so this can be called unguarded in a parallel script. label : str, optional Name for the combined run. Defaults to the first set’s label. file_path : str or Path, optional Output path to attribute the combined set to. Defaults to the first set’s.

Returns

ProfilingResults One result set holding every region of every input.

Raises

ValueError If no result sets were given, or if a region name appears in more than one of them. Merging same-named regions would silently double-count a Python wrapper and the native region inside it, so the collision has to be resolved by the caller - name the regions apart, for instance with a "fortran:" prefix.

message_nbytesfunction#

Re-exported from scope_profiler.mpi_wrappersView source

def message_nbytes(value: Any, *, pickle_objects: bool = True) -> int

Estimate payload bytes for an mpi4py object or buffer specification.

Buffer-mode calls are exact when their datatype exposes Get_size. Object-mode values use their buffer size when possible. With pickle_objects=True (the default for this explicit helper), other objects use the size of their protocol-5 pickle. -1 means unknown.

parse_mpi_regionfunction#

Re-exported from scope_profiler.mpi_wrappersView source

def parse_mpi_region(name: str) -> MPIRegionMetadata | None

Decode a canonical MPI region name, or return None for another region.

plot_duration_timeseriesfunction#

Re-exported from scope_profiler.plotting_scripts.duration_timeseriesView source

def plot_duration_timeseries(profiling_data: ProfilingResults | Sequence[ProfilingResults], ranks: list[int] | int | None = None, include: list[str] | str | None = None, exclude: list[str] | str | None = None, filepath: str | None = None, show: bool = False, verbose: bool = True, cmap: str = DEFAULT_CMAP, log_scale: bool = False, data_filepath: str | Path | None = None, data_format: str = 'csv', backend: str = 'matplotlib', return_fig: bool = False) -> None

Plot each region’s call duration over wall-clock time, with a min-max band.

One line per region tracks the mean duration over the ranks that recorded each call, shaded between the minimum and maximum duration seen across those ranks, so rank imbalance shows up as a widening band.

Parameters

backend : str Backend to use for rendering: “matplotlib” (default) or “plotly”. return_fig : bool Return the rendered figure instead of the default None. Matplotlib returns (fig, axes); Plotly returns its figure object.

plot_durationsfunction#

Re-exported from scope_profiler.plotting_scripts.durationsView source

def plot_durations(profiling_data: ProfilingResults | Sequence[ProfilingResults], ranks: list[int] | int | None = None, include: list[str] | str | None = None, exclude: list[str] | str | None = None, labels: Sequence[str] | None = None, metric: str = 'total', metrics: Sequence[str] | None = None, sort_by: str | None = None, top_n: int | None = None, combine_regions: dict[str, list[str] | str] | None = None, stack_children: bool = False, filepath: str | None = None, show: bool = False, verbose: bool = True, cmap: str = DEFAULT_CMAP, log_scale: bool = False, data_filepath: str | Path | None = None, data_format: str = 'csv', backend: str = 'matplotlib', return_fig: bool = False) -> list[str] | list[object] | object

Plot duration bar charts for one or more profiling files using maxplotlib.

Parameters

combine_regions : dict[str, list[str] | str], optional Merge several regions into a single bar, e.g. {"setup": ["setup: .*"]} combines every setup: ... region into one bar named “setup”, pooling their calls the same way sort_by and the other duration statistics pool a single region’s calls. Each value is one or more regex patterns (matched like include); a region matching several groups is claimed by whichever group is listed first. metric : str Duration metric to render (avg, min, max, total, first or last). first/last are the durations of the chronologically first and last call, which show how much of a region’s cost is one-off warmup rather than steady state. metrics : Sequence[str], optional Render several metrics instead of one. Each gets its own figure — filepath is suffixed with the metric name, as it always was for multiple metrics - but a single data_filepath holds them all, with the metric named per row, so one export can back a chart whose metric the viewer switches. Overrides metric when given. stack_children : bool Split each bar into the region’s own (exclusive) time plus one segment per region called directly from it, stacked on top of each other, so a bar shows where its time went instead of only how much there was. The nesting comes from timestamp containment, the same reconstruction the flame chart uses, over every region in the run — a child that is filtered out of the bars still gets its own segment. Only total and avg decompose this way; min/max are rejected. Colors then identify segments rather than runs, so with several runs the bars are still grouped per region in run order but the legend names the segments. backend : str Backend to use for rendering: “matplotlib” (default) or “plotly”. return_fig : bool Return the rendered figure instead of the saved filepath list.

Returns

list[str] List of filepaths that were written (empty if filepath is None). With return_fig, the rendered figure, or the list of figures when several metrics were requested.

plot_flame_chartfunction#

Re-exported from scope_profiler.plotting_scripts.flameView source

def plot_flame_chart(profiling_data: ProfilingResults | Sequence[ProfilingResults], ranks: list[int] | int | None = None, include: list[str] | str | None = None, exclude: list[str] | str | None = None, filepath: str | None = None, show: bool = False, verbose: bool = True, cmap: str = FLAME_CMAP, data_filepath: str | Path | None = None, data_format: str = 'csv', backend: str = 'matplotlib', return_fig: bool = False) -> object | None

Plot a flame chart reconstructing the call stack from region timings.

Parameters

backend : str Backend to use for rendering: “matplotlib” (default) or “plotly”. return_fig : bool Return the rendered figure instead of the default None. Matplotlib returns (fig, axes); Plotly returns its figure object.

plot_flame_graphfunction#

Re-exported from scope_profiler.plotting_scripts.flameView source

def plot_flame_graph(profiling_data: ProfilingResults | Sequence[ProfilingResults], ranks: list[int] | int | None = None, include: list[str] | str | None = None, exclude: list[str] | str | None = None, filepath: str | None = None, show: bool = False, verbose: bool = True, cmap: str = FLAME_CMAP, data_filepath: str | Path | None = None, data_format: str = 'csv', backend: str = 'matplotlib', return_fig: bool = False) -> object | None

Plot an aggregated flame graph whose x-axis represents total time.

plot_ganttfunction#

Re-exported from scope_profiler.plotting_scripts.ganttView source

def plot_gantt(profiling_data: ProfilingResults | Sequence[ProfilingResults], ranks: list[int] | int | None = None, include: list[str] | str | None = None, exclude: list[str] | str | None = None, filepath: str | None = None, show: bool = False, verbose: bool = True, cmap: str = DEFAULT_CMAP, data_filepath: str | Path | None = None, data_format: str = 'csv', backend: str = 'matplotlib', return_fig: bool = False, min_duration: float = 0.0, start_time: float | None = None, end_time: float | None = None, aggregate_calls: int = 1, collapse_depth: int | None = None) -> object | None

Plot a Gantt chart of all (or selected) regions with per-rank lanes using maxplotlib.

Parameters

backend : str Backend to use for rendering: “matplotlib” (default) or “plotly”. return_fig : bool Return the rendered figure instead of the default None. Matplotlib returns (fig, axes); Plotly returns its figure object.

plot_perf_eventsfunction#

Re-exported from scope_profiler.plotting_scripts.perf_eventsView source

def plot_perf_events(profiling_data: ProfilingResults | Sequence[ProfilingResults], metric: str = 'ipc', ranks: list[int] | int | None = None, include: list[str] | str | None = None, exclude: list[str] | str | None = None, labels: Sequence[str] | None = None, filepath: str | None = None, show: bool = False, verbose: bool = True, cmap: str = DEFAULT_CMAP, backend: str = 'matplotlib') -> None

Plot one raw perf event or derived ratio for every region and rank.

metric may be an event name recorded in the run, "ipc" (instructions/cycle), or "cache-misses-per-ki". Derived metrics are calculated from the region’s aggregated counters, not averages of call ratios, which preserves their proper weighting.

plot_rank_heatmapfunction#

Re-exported from scope_profiler.plotting_scripts.heatmapView source

def plot_rank_heatmap(profiling_data: ProfilingResults | Sequence[ProfilingResults], ranks: list[int] | int | None = None, include: list[str] | str | None = None, exclude: list[str] | str | None = None, filepath: str | None = None, show: bool = False, verbose: bool = True, cmap: str = 'viridis', data_filepath: str | Path | None = None, data_format: str = 'csv', backend: str = 'matplotlib', return_fig: bool = False, exclusive: bool = False) -> object | None

Plot total or exclusive region time as a rank-by-region heatmap.

plot_rooflinefunction#

Re-exported from scope_profiler.plotting_scripts.rooflineView source

def plot_roofline(profiling_data: ProfilingResults | Sequence[ProfilingResults], *, flops_metric: str | None = None, bandwidth_metric: str | None = None, peak_flops: float | None = None, peak_bandwidth: float | None = None, ranks: list[int] | int | None = None, include: list[str] | str | None = None, exclude: list[str] | str | None = None, filepath: str | None = None, show: bool = False, verbose: bool = True, cmap: str = DEFAULT_CMAP, data_filepath: str | Path | None = None, data_format: str = 'csv', backend: str = 'matplotlib') -> dict

Plot attained performance against arithmetic intensity for LIKWID regions.

peak_flops (GFLOP/s) and peak_bandwidth (GB/s) should be measured hardware ceilings for a true roofline. If either is absent, the largest selected observation supplies that ceiling and the plot labels it empirical; it is useful for comparing regions but must not be read as a machine limit.

plot_scaling_efficiencyfunction#

Re-exported from scope_profiler.plotting_scripts.scalingView source

def plot_scaling_efficiency(profiling_data: ProfilingResults | Sequence[ProfilingResults], x_field: str = 'num_ranks', ranks: list[int] | int | None = None, include: list[str] | str | None = None, exclude: list[str] | str | None = None, filepath: str | None = None, show: bool = False, verbose: bool = True, cmap: str = DEFAULT_CMAP, data_filepath: str | Path | None = None, data_format: str = 'csv', backend: str = 'matplotlib', return_fig: bool = False) -> object | None

Plot parallel scaling efficiency (measured speedup / ideal speedup).

plot_speedupfunction#

Re-exported from scope_profiler.plotting_scripts.scalingView source

def plot_speedup(profiling_data: ProfilingResults | Sequence[ProfilingResults], x_field: str = 'num_ranks', ranks: list[int] | int | None = None, include: list[str] | str | None = None, exclude: list[str] | str | None = None, filepath: str | None = None, show: bool = False, verbose: bool = True, cmap: str = DEFAULT_CMAP, data_filepath: str | Path | None = None, data_format: str = 'csv', backend: str = 'matplotlib', return_fig: bool = False) -> object | None

Plot scope speedup versus a chosen parallelism/metadata field using maxplotlib.

Parameters

backend : str Backend to use for rendering: “matplotlib” (default) or “plotly”.

plot_weak_scalingfunction#

Re-exported from scope_profiler.plotting_scripts.scalingView source

def plot_weak_scaling(profiling_data: ProfilingResults | Sequence[ProfilingResults], x_field: str = 'num_ranks', ranks: list[int] | int | None = None, include: list[str] | str | None = None, exclude: list[str] | str | None = None, filepath: str | None = None, show: bool = False, verbose: bool = True, cmap: str = DEFAULT_CMAP, data_filepath: str | Path | None = None, data_format: str = 'csv', backend: str = 'matplotlib', return_fig: bool = False) -> object | None

Plot weak-scaling runtime versus a chosen parallelism/metadata field.

Runtime is normalized to the smallest scale, so ideal weak scaling is a horizontal line at 1.0. Lower values are not inherently better here: the useful signal is how closely each region stays near that line.

See :func:plot_weak_scaling_efficiency for the same data read as a fraction of ideal instead.

plot_weak_scaling_efficiencyfunction#

Re-exported from scope_profiler.plotting_scripts.scalingView source

def plot_weak_scaling_efficiency(profiling_data: ProfilingResults | Sequence[ProfilingResults], x_field: str = 'num_ranks', ranks: list[int] | int | None = None, include: list[str] | str | None = None, exclude: list[str] | str | None = None, work_per_rank: Sequence[float] | None = None, filepath: str | None = None, show: bool = False, verbose: bool = True, cmap: str = DEFAULT_CMAP, data_filepath: str | Path | None = None, data_format: str = 'csv', backend: str = 'matplotlib', return_fig: bool = False) -> object | None

Plot weak-scaling efficiency: baseline runtime over runtime at each scale.

A weak-scaling study grows the problem with the machine, so every run does the same work per rank and ideal efficiency is a flat 1.0. A region at 0.6 on 64 ranks is spending 40% of its time on costs that only appear at that scale - communication, load imbalance, or contention.

This differs from :func:plot_scaling_efficiency, which is for a strong-scaling study: there the problem size is fixed, so the ideal is a speedup proportional to the rank count and efficiency divides the measured speedup by that. Here the ideal is constant runtime, so no such division applies - using the strong-scaling plot on weak-scaling runs reports a near-zero efficiency that means nothing.

Parameters

work_per_rank : Sequence[float], optional Work per rank for each profiling run, in whatever unit suits the problem (grid cells, particles, unknowns). Only the caller can know this - it is a property of the problem, not of the profile - so it is optional, but when given it is checked: the runs must agree, and a mismatch raises rather than plotting an efficiency that silently compares runs doing different amounts of work. The value is recorded in the exported plot data.

profile_mpi4pyfunction#

Re-exported from scope_profiler.mpi_wrappersView source

def profile_mpi4py(mpi_module: Any = None, *, pickle_object_sizes: bool = False, lazy: bool = False) -> Iterator[None]

Temporarily profile mpi4py’s predefined communicators.

When lazy is true, mpi4py is wrapped only if the target imports it. This lets scope-profiler run --mpi-calls enable MPI-call profiling without importing or initializing MPI for an ordinary serial program. Both globals and Python’s import function are restored even when the program raises.

profile_mpi_commfunction#

Re-exported from scope_profiler.mpi_wrappersView source

def profile_mpi_comm(communicator: Any, *, pickle_object_sizes: bool = False) -> ProfiledMPIComm

Wrap a communicator without importing mpi4py.

Set pickle_object_sizes=True to measure serialized object-mode payloads at the cost of one extra serialization per call. It is off by default to avoid overhead and repeated __reduce__ side effects.

read_h5function#

Re-exported from scope_profiler.h5readerView source

def read_h5(file_path: str | Path, verbose: bool = False) -> ProfilingResults

Load a merged profiling file for post-processing.

The discoverable spelling of :meth:ProfilingResults.from_h5 <scope_profiler.results.ProfilingResults.from_h5>; the two are interchangeable::

from scope_profiler import read_h5

results = read_h5("profiling_data.h5")
results.print_summary()

solve = results["solve"]        # same as results.get_region("solve")
solve[0].average_duration       # rank 0, in seconds

Parameters

file_path : str | Path Path to the merged HDF5 file containing profiling data. verbose : bool, optional Print each rank group as it is read (default: False).

Returns

ProfilingResults The run’s profiling data. All durations are reported in seconds.

Raises

FileNotFoundError If the specified HDF5 file does not exist.

read_h5_summaryfunction#

Re-exported from scope_profiler.h5readerView source

def read_h5_summary(file_path: str | Path, verbose: bool = False, *, fallback: bool = True, include_likwid: bool = True, include_line_profile: bool = True, regions: str | list[str] | tuple[str, ...] | None = None, ranks: int | list[int] | tuple[int, ...] | None = None) -> ProfilingResults

Load fixed-size statistics, optionally falling back for older files.

The returned object supports metadata and scalar region statistics. Event, timeline, call-stack and percentile APIs intentionally have no per-call data; use :func:read_h5 when those are required.

read_jsonfunction#

Re-exported from scope_profiler.json_exportView source

def read_json(file_path: str | Path, verbose: bool = False) -> ProfilingResults

Read a JSON profile back into :class:ProfilingResults.

The JSON twin of :func:~scope_profiler.h5reader.read_h5::

results = read_json("profiling_data.json")
results.print_summary()

read_profilefunction#

Re-exported from scope_profiler.profile_ioView source

def read_profile(file_path, verbose: bool = False) -> ProfilingResults

Read a profile from HDF5 or JSON, whichever the name says it is.

Dispatches on the file name; :func:load dispatches on the contents and is the better default for a file that already exists.

sniff_profile_formatfunction#

Re-exported from scope_profiler.profile_ioView source

def sniff_profile_format(path) -> str

Which format the contents of path are in.

:func:profile_format goes by the file name, which is all that can be known before a file exists. For a file that does exist the contents are the better authority: a run written as profile.dat, or a JSON profile someone renamed to .h5, still reads correctly.

Falls back to :func:profile_format when the leading bytes match nothing known, so an unrecognisable file is still treated as HDF5 and produces h5py’s own error rather than one invented here.

Raises

FileNotFoundError If path does not exist.

to_pstatsfunction#

Re-exported from scope_profiler.prof_exportView source

def to_pstats(profiling_data: ProfilingResults | Sequence[ProfilingResults], ranks: list[int] | int | None = None, include: list[str] | str | None = None, exclude: list[str] | str | None = None, call_paths: bool = True) -> dict[tuple[str, int], pstats.Stats]

Build a :class:pstats.Stats per (run label, rank), without disk I/O.

The in-memory twin of :func:export_prof: same call reconstruction and filtering, but returns real pstats.Stats objects keyed by (run_label, rank) instead of writing .prof files.

Parameters

profiling_data : ProfilingResults | Sequence[ProfilingResults] The run(s) to export: file runs, in-memory results from ProfileManager.finalize(return_results=True), or a mix. ranks : list[int] | int, optional Ranks to export (default: rank 0 only). include, exclude : list[str] | str, optional Region name filters, as for the plotting functions. call_paths : bool, optional Preserve each reconstructed parent/child path as a separate node in the resulting tree; see :func:build_pstats_dict.

Returns

dict[tuple[str, int], pstats.Stats] Maps (run_label, rank) to its pstats.Stats, empty on ranks that own no files.

write_jsonfunction#

Re-exported from scope_profiler.json_exportView source

def write_json(results: ProfilingResults, filepath: str | Path, *, include=None, exclude=None, ranks: list[int] | int | None = None, indent: int | None = None) -> Path

Write one run to filepath as JSON, and return the path written.

write_profilefunction#

Re-exported from scope_profiler.profile_ioView source

def write_profile(results, file_path, **kwargs) -> Path

Write results to file_path in the format its name asks for.

Parameters

results : ProfilingResults The run to write. file_path : str | Path Destination. .json/.json.gz writes a JSON profile, .html an HTML report, and anything else an HDF5 file. **kwargs Passed to the format’s own writer.

Returns

Path The file written.

write_region_statistics_jsonfunction#

Re-exported from scope_profiler.plotting_scripts.statisticsView source

def write_region_statistics_json(profiling_data: ProfilingResults | Sequence[ProfilingResults], filepath: str | Path, ranks: list[int] | int | None = None, include: list[str] | str | None = None, exclude: list[str] | str | None = None, labels: Sequence[str] | None = None) -> dict

Write aggregate region-duration statistics to a JSON file.