scope_profiler.call_stack
Reconstruct a call stack from the recorded region intervals.
Regions carry no call graph of their own: each call is just a (start, end)
pair. Nesting is therefore recovered from timestamp containment - a call that
starts while another is still open is its child. The flame chart, the .prof
export and the speedscope export all build on this, and so can user code that
wants to draw its own nested view.
Intervals must be properly nested, within a lane. A lane is one
sequential flow of control: a thread, or an asyncio task or greenlet running
on one. Two calls recorded on the same lane either nest completely or do not
overlap at all; a call that starts inside another and ends after it is
rejected rather than reconstructed. That is what a stack of with blocks
always produces, and assuming it is what makes :func:build_call_arrays a
handful of vectorized passes instead of a per-call Python loop - the
reconstruction is O(events) in numpy rather than O(events) in dicts, which is
the difference between 2 s and 160 s on a run with ten million events.
Calls on different lanes routinely overlap without nesting, and are
reconstructed as separate stacks: a run profiled with track_threads (see
:mod:scope_profiler.concurrency) carries a lane per call, and every lane is
reconstructed on its own. Without that column the whole rank is one lane,
which is correct for the single-threaded runs it describes and is why
profiling concurrent threads without track_threads raises
:class:NestingError instead of inventing a call graph.
Manual sp_begin/sp_end pairs in native code are the other realistic
way to violate the assumption.
Classes
| Name | Description |
|---|---|
CallArrays | One rank's reconstructed nesting, in start-sorted (call_id) order. |
NestingError | Raised when recorded intervals are not properly nested. |
Functions
| Name | Description |
|---|---|
build_call_arrays | Reconstruct one rank's nesting as numpy arrays. |
build_call_stack | Reconstruct per-call nesting for one rank, one dict per call. |
call_stack_children | Child indices per call, for walking a :func:build_call_stack result. |
call_stack_roots | Indices of the top-level calls in a :func:build_call_stack result. |
exclusive_totals_ns | Total exclusive time per region, in nanoseconds, for one rank's regions. |
regions_from_snapshot | Present a ProfileManager._snapshot_regions() dict as MPIRegions. |
split_by_lane | Split a reconstruction into one :class:CallArrays per lane. |
CallArraysclass#
class CallArrays(NamedTuple)Bases: NamedTuple
One rank’s reconstructed nesting, in start-sorted (call_id) order.
Every array is indexed by call_id, so parent indexes straight back
into the same arrays. region_index and call_index point at where a
call came from: region names[region_index[i]], call number
call_index[i] within that region’s buffers.
call_indexattributeinstance attribute#
call_index: np.ndarraydepthattributeinstance attribute#
depth: np.ndarrayend_nsattributeinstance attribute#
end_ns: np.ndarrayexclusive_nsattributeinstance attribute#
exclusive_ns: np.ndarraylaneattributeclass attributeinstance attribute#
lane: np.ndarray = _NO_LANEWhich stack each call belongs to; see :func:scope_profiler.concurrency.lane_ids.
Empty for a run that did not track threads, meaning the whole rank is
one stack - rather than a full-length array of -1, which would cost
three more passes over the events of every single-threaded run to say
nothing. Consumers must therefore check lane.size before indexing it.
Defaulted so the field can be omitted by older callers constructing this
tuple themselves.
namesattributeinstance attribute#
names: list[str]parentattributeinstance attribute#
parent: np.ndarrayregion_indexattributeinstance attribute#
region_index: np.ndarraysource_filesattributeinstance attribute#
source_files: list[str | None]source_linesattributeinstance attribute#
source_lines: list[int | None]start_nsattributeinstance attribute#
start_ns: np.ndarrayNestingErrorclass#
class NestingError(ValueError)Bases: ValueError
Raised when recorded intervals are not properly nested.
A lane’s calls must form a forest: any two intervals on one lane are
either disjoint or one contains the other. Partial overlap has no call
stack to reconstruct, and every consumer of this module (flame chart,
.prof, speedscope, exclusive time) would have to invent one.
In practice this means a mismatched sp_begin/sp_end pair in C or
Fortran, or regions driven from several threads in a run that did not
profile with track_threads=True - without that, every thread’s calls
land in one lane and interleave.
build_call_arraysfunction#
def build_call_arrays(regions: Iterable, rank: int) -> CallArraysReconstruct one rank’s nesting as numpy arrays.
The workhorse behind :func:build_call_stack, and what anything
processing a whole run should call: it never builds a dict per call, so
it stays usable on the tens of millions of events a long simulation
produces.
Parameters
regions : iterable of MPIRegion
Regions to include, e.g. results.get_regions(). Regions with no
data for rank are skipped.
rank : int
Rank whose calls to reconstruct.
Returns
CallArrays
Raises
NestingError If any interval ends before it starts, or two intervals overlap without one containing the other.
build_call_stackfunction#
def build_call_stack(regions: Iterable, rank: int, origin: float = 0.0) -> list[dict]Reconstruct per-call nesting for one rank, one dict per call.
A convenience wrapper over :func:build_call_arrays for callers that
want to iterate calls rather than columns. It allocates a dict per call
(~700 bytes), so prefer the arrays for anything that has to scale with a
long run.
Parameters
regions : iterable of MPIRegion
Regions to include, e.g. results.get_regions().
rank : int
Rank whose calls to reconstruct. Regions with no data for this rank
are skipped.
origin : float, optional
Seconds subtracted from every timestamp, so passing
results.minimum_start_time yields a timeline starting at zero
(default: 0.0, i.e. raw timestamps).
Returns
list of dict
One dict per call, ordered by start time (parents before children),
with keys call_id, name, call_path, lane, start, end,
duration (the inclusive duration), inclusive_duration,
exclusive_duration (seconds), depth (0 for a top-level call)
and parent (the call_id of the enclosing call in this same
list, or None). call_path joins a call and each of its ancestors
with " > ", keeping same-named regions at different call sites
distinct. A color key carries whatever the plotting code assigned
to the region. call_id is stable for the returned list and is
unique within this rank/stack reconstruction.
Raises
NestingError If the intervals are not properly nested.
Notes
Calls are indexed by position in the returned list rather than by name, because a region called more than once - or recursively - contributes several calls under one name.
call_stack_childrenfunction#
def call_stack_children(calls: list[dict]) -> list[list[int]]Child indices per call, for walking a :func:build_call_stack result.
Returns a list parallel to calls: entry i holds the indices of the
calls directly nested inside call i, in start-time order.
call_stack_rootsfunction#
def call_stack_roots(calls: list[dict]) -> list[int]Indices of the top-level calls in a :func:build_call_stack result.
exclusive_totals_nsfunction#
def exclusive_totals_ns(regions: dict, rank: int = 0) -> dictTotal exclusive time per region, in nanoseconds, for one rank’s regions.
The write-side counterpart of
:meth:ProfilingResults._populate_exclusive_durations <scope_profiler.results.ProfilingResults._populate_exclusive_durations>.
Both sum the same int64 nanosecond values produced by
:func:build_call_arrays, so a total stored in the output file is
bit-identical to the one a reader reconstructs from the events.
finalize() calls this because a rank already holds its whole region
set in memory, which is exactly the set exclusive time is defined against
- and reconstructing the nesting is by far the most expensive part of reading a run back.
Parameters
regions : dict
Region name -> (start_times, end_times, ...) int64 arrays in
nanoseconds, as :meth:ProfileManager._snapshot_regions returns them.
rank : int, optional
Rank to label the temporary regions with; irrelevant to the result.
Returns
dict Region name -> total exclusive nanoseconds, for every named region.
regions_from_snapshotfunction#
def regions_from_snapshot(regions: dict, rank: int) -> listPresent a ProfileManager._snapshot_regions() dict as MPIRegions.
The finalize path holds raw (starts, ends, ...) arrays rather than
result objects; this is the adapter both it and
:func:exclusive_totals_ns use to reach :func:build_call_arrays.
split_by_lanefunction#
def split_by_lane(calls: CallArrays) -> list[tuple[int, CallArrays]]Split a reconstruction into one :class:CallArrays per lane.
Anything that has to emit calls in time order - the speedscope exporter, above all - cannot walk a multi-lane reconstruction as one tree: two lanes interleave, so a depth-first walk of the roots emits an event at a time earlier than the one before it. Per lane the intervals nest, so each lane walks exactly the way a single-threaded run always did.
Parents are remapped to positions within each returned lane, and the
lane column is carried through so callers can name the result.
Returns
list of (int, CallArrays)
One entry per lane, in lane order. A reconstruction with no lane
column comes back as a single entry keyed -1, i.e. unchanged.