scope_profiler.results

Post-processing API over a set of profiling regions.

This is the analysis layer, deliberately independent of where the data came from: :meth:ProfilingResults.from_h5 builds one of these from a merged HDF5 file, while :meth:ProfileManager.finalize(return_results=True) <scope_profiler.profile_manager.ProfileManager.finalize> builds the same thing straight out of the in-memory buffers, without a round trip through disk. Both give back the same type, and everything downstream (summaries, dataframes, the plotting functions, the exporters) cannot tell them apart.

Classes

Name	Description
`ProfilingResults`	The profiling data of one run, as an ordered mapping of region name to

Functions

Name	Description
`merge_results`	Combine several result sets into one.

`ProfilingResults`class#

View source

class ProfilingResults(regions: dict[str, MPIRegion], metadata: dict | None = None, num_ranks: int | None = None, likwid: dict[int, dict[str, LikwidRegionResult]] | None = None, perf_events: dict[int, dict] | None = None, line_profile: dict[int, list] | None = None, file_path: str | Path = '', is_root: bool = True, exclusive_totals: dict[str, dict[int, int]] | None = None, event_data_available: bool = True, threads: dict[int, list] | None = None, tasks: dict[int, list] | None = None)

The profiling data of one run, as an ordered mapping of region name to :class:~scope_profiler.mpi_region.MPIRegion::

results = ProfileManager.finalize(return_results=True)
for region in results:
    print(region.name, region.total_duration)

solve = results["solve"]        # same as results.get_region("solve")
solve[0].average_duration       # rank 0, in seconds

The same object comes back from a merged output file, via :meth:from_h5 (or its module-level twin :func:~scope_profiler.h5reader.read_h5)::

results = ProfilingResults.from_h5("profiling_data.h5")

All durations are reported in seconds.

`display_label`property#

View source

display_label: str

What to call this run in charts, tables and exports.

The :attr:label when the run has one, and otherwise the stem of its output file --- which is what post-processing named runs by before labels existed, and remains the default.

Returns

str A non-empty name for the run.

`file_path`property#

View source

file_path: Path

The run’s output file.

Returns

Path The file path as a pathlib.Path object. For results taken straight from memory this is the path setup() was configured with, which need not exist on disk.

`finalize_time`property#

View source

finalize_time: float | None

When finalize() was called, in seconds on the recording clock.

This is the finalize_time_ns metadata field, read as the first thing ProfileManager.finalize() does - before it spends any time collecting or writing the run’s data, so it marks the moment finalize() was reached rather than the moment it returned.

Returns

float or None The registered finalize time, or None for runs without one (older files, or a run that never reached finalize()).

`has_event_data`property#

View source

has_event_data: bool

Whether per-call timestamps are available to event-based APIs.

`has_likwid`property#

View source

has_likwid: bool

Whether the run recorded LIKWID hardware counter results.

`has_perf_events`property#

View source

has_perf_events: bool

Whether this run recorded built-in Linux perf-event counters.

`is_root`property#

View source

is_root: bool

Whether this rank holds the run’s data.

Always True for a file that was read back, and for serial runs. Under MPI, finalize(return_results=True) gathers everything on rank 0, so only rank 0’s results are the root ones; the others come back empty and with this False.

Everything that produces output - :meth:print_summary, the plot_* functions, the exporters - does nothing for non-root results. That is what lets a parallel script call them unguarded and still write each figure exactly once, from rank 0. Read it when a script needs to make the same distinction for output of its own.

Returns

bool True unless this is a non-root rank’s share of an MPI run.

`label`propertywritable#

View source

label: str | None

The run’s label, as given to ProfileManager.setup(label=...).

Returns

str or None The label, or None for a run that was not given one. Use :attr:display_label to name the run in output regardless.

`likwid_ranks`property#

View source

likwid_ranks: list[int]

Ranks that recorded LIKWID results, in ascending order.

`line_profile`property#

View source

line_profile: dict[int, list]

Persisted line-profiler records keyed by rank.

Each record contains region, filename, function, first_lineno, line_numbers, hits, times and unit. The elapsed seconds for a line are times * unit.

`maximum_end_time`property#

View source

maximum_end_time: float

Get the maximum end time across all regions and ranks.

Returns

float Maximum end time in seconds, or 0.0 if no region recorded timing.

`metadata`property#

View source

metadata: dict

Get environment metadata for the run (gathered from rank 0).

Returns

dict Metadata dict (hostname, OpenMP thread count, platform, versions, etc.), or an empty dict if the run recorded none.

`minimum_start_time`property#

View source

minimum_start_time: float

Get the minimum start time across all regions and ranks.

This is the origin of the timeline: subtract it from any timestamp to get seconds since the first region entry. Regions with no recorded calls are ignored; the result is 0.0 if no region recorded any.

Returns

float Minimum start time in seconds.

`num_nodes`property#

View source

num_nodes: int | None

Number of distinct hosts the run’s ranks ran on, or None if unknown.

Read from the num_nodes metadata field. Files written before it was recorded still answer for a single-rank run, which can only have used one node; a multi-rank run without the field returns None.

`num_ranks`property#

View source

num_ranks: int

Get the number of ranks recorded in the profiling data.

Returns

int Number of ranks.

`region_names`property#

View source

region_names: list[str]

Names of all regions, in order of appearance.

`run_start_time`property#

View source

run_start_time: float | None

When the run started, in seconds on the recording clock.

This is the start_time_ns metadata field, written by ProfileManager.setup() — by default the moment setup() was called, or an earlier instant if one was passed to it.

Returns

float or None The registered start time, or None for runs without one (older files, or runs that never called setup()). Use :attr:startup_time for the elapsed time before the first region, which stays defined either way.

`startup_time`property#

View source

startup_time: float

Seconds between the start of the run and the first profiled region.

Time the instrumentation never saw: imports, reading input, building a mesh. Zero when there is no registered start time (:attr:run_start_time is None), since the run is then only known from its first region onwards.

Returns

float Elapsed time before the first region entry, in seconds.

`tasks`property#

View source

tasks: dict[int, list]

Rank -> the asyncio tasks and greenlets the run followed.

Each entry is a :class:~scope_profiler.concurrency.TaskInfo, whose running_time and awaiting_time split the lane’s life into the part it held a thread and the part it was suspended. Empty unless the run profiled with track_async=True.

`threads`property#

View source

threads: dict[int, list]

Rank -> the threads that recorded calls, in registration order.

Each entry is a :class:~scope_profiler.concurrency.ThreadInfo: name, OS ids, when the thread started and ended, and the CPU time it burned. Empty unless the run profiled with track_threads=True::

for rank, threads in results.threads.items():
    for thread in threads:
        print(rank, thread.name, thread.cpu_time)

The per-call thread_ids column of every region indexes into this rank’s list, so results["solve"][0].for_thread(2) is the part of solve that ran on results.threads[0][2].

`time_origin`property#

View source

time_origin: float

Zero point of the relative timeline, in seconds.

The registered :attr:run_start_time when there is one, and otherwise the first region entry (:attr:minimum_start_time). This is what :meth:events and :meth:call_stack measure from.

Note the plot_* functions instead frame their x axis on the first region entry, so that a long gap between setup() and the first region does not fill a chart with empty space. Pass origin=results.minimum_start_time to :meth:events to reproduce the numbers on a chart’s axis.

Returns

float Origin timestamp on the recording clock.

`time_span`property#

View source

time_span: float

Wall-clock seconds between the first region entry and the last exit.

Returns

float Duration of the profiled window in seconds, or 0.0 if no region recorded timing.

`total_time`property#

View source

total_time: float | None

Wall-clock seconds from setup() to finalize().

Unlike :attr:time_span (first region entry to last exit), this covers the whole instrumented program: startup work before the first region, gaps between regions, and any teardown after the last one but before finalize() is called - the number to report as “how long did the run take” alongside the region breakdown.

Returns

float or None :attr:finalize_time minus :attr:run_start_time, or None if either is missing - an older file, or a run that set up profiling (or called finalize) some other way than ProfileManager.setup()/finalize().

`call_graph`method#

View source

def call_graph(rank: int = 0, include=None, exclude=None) -> list[dict]

Return call relationships without timestamps or durations.

New profiles persist call_id and parent_id for every Python event. Legacy profiles fall back to the timestamp-based call stack. The returned nodes contain only call_id, parent_id, name, call_index and depth.

Filtering renests: a call whose parent was excluded is reported under its nearest surviving ancestor, with the depth that implies, exactly as :meth:call_stack does. Leaving the excluded id in parent_id would hand back a graph with edges pointing at nodes that are not in it.

call_id is unique within this rank, not across the file - each rank numbers its own calls.

`call_stack`method#

View source

def call_stack(rank: int = 0, include: list[str] | str | None = None, exclude: list[str] | str | None = None, relative: bool = True, origin: float | None = None) -> list[dict]

Reconstruct the nested call stack for one rank.

Regions record no call graph, so nesting is recovered from timestamp containment - the same reconstruction the flame chart, the .prof export and the speedscope export use.

Parameters

rank : int, optional Rank whose calls to reconstruct (default: 0). include, exclude : list of str or str, optional Regex patterns selecting which regions to include, matched as in :meth:get_regions. relative : bool, optional If True (default), timestamps start at zero; see :meth:events. origin : float, optional Measure from this timestamp instead; see :meth:events.

Returns

List[dict] One entry per call, parents before children, with keys call_id, name, start, end, duration, depth and parent (the enclosing call’s id, or None). See :func:~scope_profiler.call_stack.build_call_stack.

`default_title`method#

View source

def default_title() -> str

Heading naming this run, for a summary table.

The label leads when the run has one, since that is the name the user chose; the file path follows either way, because when several runs are being compared it is what tells them apart on disk.

Returns

str e.g. "results/run_a.h5 (128 ranks)".

`events`method#

View source

def events(include: list[str] | str | None = None, exclude: list[str] | str | None = None, ranks: list[int] | int | None = None, relative: bool = True, origin: float | None = None) -> list[dict]

Return one dict per recorded call, across all regions and ranks.

This is the long-form (“tidy”) view to build custom plots from: one row per call rather than one per region.

Parameters

include, exclude : list of str or str, optional Regex patterns selecting which regions to include, matched as in :meth:get_regions. ranks : list of int or int, optional Restrict to these ranks (default: all). relative : bool, optional If True (default), timestamps are measured from :attr:time_origin — the start time registered by ProfileManager.setup(), or the first region entry for runs without one — so the timeline starts at zero. If False, the raw monotonic-clock timestamps are returned; those are only comparable within a single run. origin : float, optional Measure from this timestamp instead, in seconds on the recording clock. Overrides relative; pass origin=results.minimum_start_time to zero the timeline on the first region entry regardless of what the run registered.

Returns

List[dict] Entries with keys name, rank, call_index, start, end and duration, in seconds, ordered by region (as in :meth:get_regions) then rank then call order. Entries also carry gpu_duration when CUDA-event timing was enabled for that region.

Examples

for event in results.events(include=“solve”): # doctest: +SKIP … print(event[“rank”], event[“start”], event[“duration”])

`from_h5`methodclassmethod#

View source

def from_h5(file_path: str | Path, verbose: bool = False) -> ProfilingResults

Load a merged profiling file written by :meth:ProfileManager.finalize <scope_profiler.profile_manager.ProfileManager.finalize>::

results = ProfilingResults.from_h5("profiling_data.h5")
results.print_summary()

:func:scope_profiler.read_h5 is the same thing under a shorter name.

Parameters

file_path : str | Path Path to the merged HDF5 file containing profiling data. verbose : bool, optional Print each rank group as it is read (default: False).

Returns

ProfilingResults The run’s profiling data, of whichever class this was called on.

Raises

FileNotFoundError If the specified HDF5 file does not exist.

`get_likwid_region`method#

View source

def get_likwid_region(tag: str, rank: int = 0) -> LikwidRegionResult

Get one region’s LIKWID results for a single rank.

Parameters

tag : str LIKWID marker region tag (the profiled region’s name). rank : int, optional Rank to read from (default: 0).

Returns

LikwidRegionResult The region’s counters, event names and derived metrics.

Raises

KeyError If the rank recorded no LIKWID data or has no such region.

`get_likwid_regions`method#

View source

def get_likwid_regions(rank: int | None = None) -> dict

Get the LIKWID marker results of the run.

Parameters

rank : int, optional Return only this rank’s regions. By default every rank is included, keyed by rank.

Returns

dict With rank given, a mapping of region tag to :class:~scope_profiler.likwid_data.LikwidRegionResult; otherwise a mapping of rank to such a dict. Empty when the run did not use LIKWID.

Examples

results = read_h5("profiling_data.h5")
for rank, regions in results.get_likwid_regions().items():
    for tag, result in regions.items():
        for name, values in zip(result.metric_names, result.metrics):
            print(rank, tag, name, values)

`get_perf_events`method#

View source

def get_perf_events(rank: int | None = None) -> dict

Return aggregated Linux perf_event_open counts by region.

`get_region`method#

View source

def get_region(region_name: str) -> MPIRegion

Retrieve profiling data for a specific region.

Parameters

region_name : str Name of the region to retrieve.

Returns

Region Region object containing profiling data for all ranks.

Raises

KeyError If the specified region name does not exist.

`get_regions`method#

View source

def get_regions(include: list[str] | str | None = None, exclude: list[str] | str | None = None) -> list[MPIRegion]

Get a list of all regions in order of appearance.

Returns

List[Region] List of Region objects.

`lane_label`method#

View source

def lane_label(lane: int, rank: int = 0) -> str

A human-readable name for one lane of rank.

Lanes are the stacks a rank’s calls were reconstructed into (see :func:~scope_profiler.concurrency.lane_ids). This turns one back into something worth putting on a chart or a speedscope profile selector: a task’s own name where the lane is a task, the thread’s name where it is a bare thread.

`likwid_to_dataframe`method#

View source

def likwid_to_dataframe()

Return every LIKWID event and metric as a tidy pandas DataFrame.

One row per (rank, region, hardware thread), with a column per event and per derived metric, plus the region’s LIKWID runtime and call count. Regions measured with different event groups simply leave the other group’s columns empty.

Returns

pandas.DataFrame Empty if there is no LIKWID data.

Raises

ImportError If pandas is not installed.

`print_likwid_summary`method#

View source

def print_likwid_summary(stream=None) -> None

Print every LIKWID region’s events and derived metrics.

Parameters

stream : file-like, optional Destination (default: stdout).

`print_summary`method#

View source

def print_summary(include: list[str] | str | None = None, exclude: list[str] | str | None = None, ranks: list[int] | None = None, sort: str = 'start', title: str | None = None, stream=None, suppress_notes: bool = False, columns: list[str] | str | None = None, percentage_mode: str = 'coverage') -> None

Print a region summary table, aggregated over ranks.

Renders the same table as scope-profiler inspect and the summary printed by ProfileManager.finalize().

Parameters

include, exclude : list of str or str, optional Regex patterns selecting which regions to print, matched as in :meth:get_regions. ranks : list of int, optional Restrict the statistics to these ranks (default: all). sort : str, optional Column to order by: start (default), total, calls, avg, min, max, std or name. title : str, optional Heading above the table (default: the file path and rank count). stream : file-like, optional Where to write (default: stdout). columns : list of str or str, optional Region summary columns to print. Defaults to region, calls, percent, parent_percent, total and avg. percent is relative to scope_profiler.session; parent_percent is relative to the enclosing call path. Use region for the region-name column. percentage_mode : {“coverage”, “exclusive”}, optional Quantity used for % session. Defaults to wall-clock coverage; use exclusive to attribute time after nested regions.

Notes

Does nothing on a non-root rank (see :attr:is_root), so a parallel script can call it unguarded and print the table once.

`summary`method#

View source

def summary(include: list[str] | str | None = None, exclude: list[str] | str | None = None) -> list[dict]

Summarize every region, aggregated over ranks.

Parameters

include, exclude : list of str or str, optional Regex patterns selecting which regions to summarize, matched as in :meth:get_regions.

Returns

List[dict] One dict per region (see :meth:~scope_profiler.mpi_region.MPIRegion.get_summary), ordered by first start time. inclusive_duration includes nested regions; exclusive_duration excludes them. Durations are in seconds.

`thread_summary`method#

View source

def thread_summary(rank: int = 0) -> list[dict]

Per-thread wall time, CPU time and call count for one rank.

Wall and CPU time come from the thread table; the call count and the time attributed to regions come from the events, so a thread that burned CPU outside every profiled region shows the gap directly.

Returns

list of dict One entry per thread with keys index, name, alive, wall_time, cpu_time, num_calls and region_time (the summed inclusive duration of that thread’s top-level calls, so nested regions are not counted twice), in thread-table order. Empty when the run did not track threads.

`to_dataframe`method#

View source

def to_dataframe(include: list[str] | str | None = None, exclude: list[str] | str | None = None, per_rank: bool = False)

Return the region summaries as a pandas DataFrame.

Parameters

include, exclude : list of str or str, optional Regex patterns selecting which regions to include, matched as in :meth:get_regions. per_rank : bool, optional If True, emit one row per (region, rank) with a rank column instead of one aggregated row per region (default: False).

Returns

pandas.DataFrame Region statistics, with durations in seconds.

Raises

ImportError If pandas is not installed.

`to_events_dataframe`method#

View source

def to_events_dataframe(include: list[str] | str | None = None, exclude: list[str] | str | None = None, ranks: list[int] | int | None = None, relative: bool = True, origin: float | None = None)

Return every recorded call as a pandas DataFrame (one row per call).

Parameters

include, exclude, ranks, relative, origin As in :meth:events.

Returns

pandas.DataFrame Columns name, rank, call_index, start, end and duration, with times in seconds. A gpu_duration column is included when any selected event has CUDA-event timing.

Raises

ImportError If pandas is not installed.

`merge_results`function#

View source

def merge_results(*result_sets, label: str | None = None, file_path=None)

Combine several result sets into one.

The case this exists for is a mixed-language run: a Python driver records its own regions while the Fortran (or other) code it calls records theirs, and what the user wants at the end is one profile covering both. Ranks line up by number, so rank 3’s Python regions and rank 3’s Fortran regions end up side by side.

Parameters

*result_sets : ProfilingResults The sets to combine. Non-root sets (the empty ones every rank but 0 gets back under MPI) are ignored, so this can be called unguarded in a parallel script. label : str, optional Name for the combined run. Defaults to the first set’s label. file_path : str or Path, optional Output path to attribute the combined set to. Defaults to the first set’s.

Returns

ProfilingResults One result set holding every region of every input.

Raises

ValueError If no result sets were given, or if a region name appears in more than one of them. Merging same-named regions would silently double-count a Python wrapper and the native region inside it, so the collision has to be resolved by the caller - name the regions apart, for instance with a "fortran:" prefix.

scope_profiler.results

ProfilingResultsclass#

display_labelproperty#

Returns

file_pathproperty#

Returns

finalize_timeproperty#

Returns

has_event_dataproperty#

has_likwidproperty#

has_perf_eventsproperty#

is_rootproperty#

Returns

labelpropertywritable#

Returns

likwid_ranksproperty#

line_profileproperty#

maximum_end_timeproperty#

Returns

metadataproperty#

Returns

minimum_start_timeproperty#

Returns

num_nodesproperty#

num_ranksproperty#

Returns

region_namesproperty#

run_start_timeproperty#

Returns

startup_timeproperty#

Returns

tasksproperty#

threadsproperty#

time_originproperty#

Returns

time_spanproperty#

Returns

total_timeproperty#

Returns

call_graphmethod#

call_stackmethod#

Parameters

Returns

default_titlemethod#

Returns

eventsmethod#

Parameters

Returns

Examples

from_h5methodclassmethod#

Parameters

Returns

Raises

get_likwid_regionmethod#

Parameters

Returns

Raises

get_likwid_regionsmethod#

Parameters

Returns

Examples

get_perf_eventsmethod#

get_regionmethod#

Parameters

Returns

Raises

get_regionsmethod#

Returns

lane_labelmethod#

likwid_to_dataframemethod#

Returns

Raises

print_likwid_summarymethod#

Parameters

print_summarymethod#

Parameters

Notes

summarymethod#

Parameters

Returns

`ProfilingResults`class#

`display_label`property#

`file_path`property#

`finalize_time`property#

`has_event_data`property#

`has_likwid`property#

`has_perf_events`property#

`is_root`property#

`label`propertywritable#

`likwid_ranks`property#

`line_profile`property#

`maximum_end_time`property#

`metadata`property#

`minimum_start_time`property#

`num_nodes`property#

`num_ranks`property#

`region_names`property#

`run_start_time`property#

`startup_time`property#

`tasks`property#

`threads`property#

`time_origin`property#

`time_span`property#

`total_time`property#

`call_graph`method#

`call_stack`method#

`default_title`method#

`events`method#

`from_h5`methodclassmethod#

`get_likwid_region`method#

`get_likwid_regions`method#

`get_perf_events`method#

`get_region`method#

`get_regions`method#

`lane_label`method#

`likwid_to_dataframe`method#

`print_likwid_summary`method#

`print_summary`method#

`summary`method#

`thread_summary`method#

`to_dataframe`method#

`to_events_dataframe`method#

`merge_results`function#