scope_profiler.likwid_data

Collection and storage of LIKWID marker results.

Counter collection happens at finalize(), on top of a run that has already completed. It is a bonus, never the point of the job, so no failure here may cost the user their timing data. That constraint shapes the whole module: there are three sources of counter data, tried richest first, and the risky one is fenced off in a child process.

• Full API, out of process (:func:collect_marker_results_isolated) --- the richest source. After markerclose() has written the marker file, re-initializing the perfmon module and calling markerreadfile() exposes every region, for every thread, with event names, counter registers and LIKWID’s derived metrics (Clock [MHz], CPI, Energy [J], …).

  Re-initializing perfmon is also the one step that can take the interpreter down rather than raise: on hosts where LIKWID cannot really count (a virtualized CI runner with an unreadable TSC and counters disabled by HyperThreading, say) it has been observed to segfault. It therefore runs in a subprocess, where a crash costs nothing but the enrichment.

• Marker file (:func:parse_marker_file) --- the file LIKWID writes is plain text, so it can be read with no LIKWID calls at all and cannot fail catastrophically. It yields every region, thread, call count, runtime and raw counter value; only the names and the derived metrics are missing. This is the fallback whenever the subprocess above does not come back.

• Marker API (:func:collect_region_snapshots) --- markergetregion(tag) read while the markers are still open. Last resort, for when the marker file is absent entirely; reports only the calling thread.

All three require the process to have been started under likwid-perfctr -m (or likwid-mpirun ... -marker). Without that, LIKWID sets no environment and there is nothing to collect.

Attributes

Name	Description
LIKWID_GROUP	No description.

Classes

Name	Description
LikwidRegionResult	Counter results for a single LIKWID marker region.

Functions

Name	Description
collect_marker_results	Read every region of the run back from LIKWID's marker file.
collect_marker_results_isolated	Run the perfmon read-back in a child process and return its results.
collect_region_snapshots	Read the current counter values for each named region.
likwid_environment	Return the LIKWID environment of the current process.
markers_available	Whether this process runs under the LIKWID marker API.
parse_marker_file	Read LIKWID's marker file directly, without calling into LIKWID.
snapshots_to_results	Convert marker-API snapshots into the common result structure.
write_likwid_results	Write collected LIKWID results into an open HDF5 file.

LIKWID_GROUPattributemodule attribute#

View source

LIKWID_GROUP = 'likwid'

LikwidRegionResultclassdataclass#

View source

class LikwidRegionResult(tag: str, group_id: int = -1, group_name: str = '', cpus: list[int] = list(), times: np.ndarray = lambda : np.zeros(0)(), call_counts: np.ndarray = lambda : np.zeros(0, dtype=np.int64)(), event_names: list[str] = list(), counter_names: list[str] = list(), events: np.ndarray = lambda : np.zeros((0, 0))(), metric_names: list[str] = list(), metrics: np.ndarray = lambda : np.zeros((0, 0))(), source: str = 'full_api')

Counter results for a single LIKWID marker region.

The per-thread arrays are all indexed by the same thread axis, so events[e, t] and metrics[m, t] refer to the thread whose runtime is times[t] and whose CPU is cpus[t].

Attributes

tag : str Region name as passed to markerstartregion. group_id : int Index of the LIKWID event group this region was measured with. group_name : str Name of that group (e.g. "CLOCK"), empty if unknown. cpus : list of int Hardware threads that took part in the region. times : numpy.ndarray Per-thread accumulated runtime in seconds, shape (nthreads,). call_counts : numpy.ndarray Per-thread number of times the region was entered, shape (nthreads,). event_names : list of str Names of the raw hardware events, length nevents. Not unique: a group may program one event on several counters (see :attr:event_labels). counter_names : list of str Hardware counter register each event was programmed on (FIXC0, PMC1, MBOX3C0, …), length nevents. Empty for files written before counter names were recorded. events : numpy.ndarray Raw counter values, shape (nevents, nthreads). metric_names : list of str Names of LIKWID’s derived metrics, length nmetrics. metrics : numpy.ndarray Derived metric values, shape (nmetrics, nthreads). source : str Which collection path produced this result: "full_api" (perfmon read-back, the only one with real event names and metrics), "marker_file" (LIKWID’s marker file parsed directly --- real values, placeholder event names, no metrics) or "marker_api" (a markergetregion snapshot of the calling thread only).

call_countsattributeclass attributeinstance attribute#

View source

call_counts: np.ndarray = field(default_factory=lambda : np.zeros(0, dtype=np.int64))

counter_namesattributeclass attributeinstance attribute#

View source

counter_names: list[str] = field(default_factory=list)

cpusattributeclass attributeinstance attribute#

View source

cpus: list[int] = field(default_factory=list)

event_namesattributeclass attributeinstance attribute#

View source

event_names: list[str] = field(default_factory=list)

eventsattributeclass attributeinstance attribute#

View source

events: np.ndarray = field(default_factory=lambda : np.zeros((0, 0)))

group_idattributeclass attributeinstance attribute#

View source

group_id: int = -1

group_nameattributeclass attributeinstance attribute#

View source

group_name: str = ''

metric_namesattributeclass attributeinstance attribute#

View source

metric_names: list[str] = field(default_factory=list)

metricsattributeclass attributeinstance attribute#

View source

metrics: np.ndarray = field(default_factory=lambda : np.zeros((0, 0)))

sourceattributeclass attributeinstance attribute#

View source

source: str = 'full_api'

tagattributeinstance attribute#

View source

tag: str

timesattributeclass attributeinstance attribute#

View source

times: np.ndarray = field(default_factory=lambda : np.zeros(0))

event_labelsproperty#

View source

event_labels: list[str]

Unique per-event labels, safe to use as dict or column keys.

event_names alone is not unique. A group such as MEM_DP programs the same event on one counter per memory channel, so CAS_COUNT_RD legitimately appears eight times on a socket with eight channels (the ones reading zero are unpopulated). Keying anything by the bare name silently keeps only the last channel.

Names that occur once are returned unchanged; repeated ones get the hardware counter appended --- CAS_COUNT_RD:MBOX0C0, CAS_COUNT_RD:MBOX1C0, … --- or a positional suffix if the file predates counter names being recorded.

as_dictmethod#

View source

def as_dict() -> dict

Return the region’s results as a plain dictionary.

Convenience for callers that want to build a DataFrame or dump the numbers without touching the array layout.

collect_marker_resultsfunction#

View source

def collect_marker_results(pylikwid) -> list[LikwidRegionResult]

Read every region of the run back from LIKWID’s marker file.

Must be called after markerclose(), which is what writes the file. Because markerclose() also tears the perfmon module down, the event sets named in LIKWID_EVENTS are re-registered first; that is what makes event and metric names available for the values in the file.

Parameters

pylikwid : module The imported pylikwid module.

Returns

list of LikwidRegionResult One entry per region recorded by LIKWID, in file order. Empty when the process is not running under the marker API, or when the performance counters cannot be re-opened.

collect_marker_results_isolatedfunction#

View source

def collect_marker_results_isolated(timeout: float = 120.0)

Run the perfmon read-back in a child process and return its results.

:func:collect_marker_results has to re-initialize LIKWID’s perfmon module, and on hosts where the counters are not really usable that can abort the process outright instead of raising --- which at finalize() time would destroy a completed run’s output. Running it behind a process boundary turns that worst case into a missing enrichment.

Parameters

timeout : float, optional Seconds to wait for the child before giving up (default: 120).

Returns

list of LikwidRegionResult or None None when the child could not deliver results (crashed, timed out, or LIKWID refused to re-open the counters), which tells the caller to fall back to :func:parse_marker_file.

collect_region_snapshotsfunction#

View source

def collect_region_snapshots(pylikwid, region_names: Iterable[str]) -> list[dict]

Read the current counter values for each named region.

Uses the marker API’s markergetregion, so it must be called before markerclose() and reports only the calling thread. Regions that LIKWID does not know about (never entered, or measured under a different name) are skipped.

Parameters

pylikwid : module The imported pylikwid module. region_names : iterable of str Region tags to query.

Returns

list of dict One entry per region that returned data, with keys tag, nevents, events, time and count.

likwid_environmentfunction#

View source

def likwid_environment() -> dict

Return the LIKWID environment of the current process.

Returns

dict The LIKWID_* variables set by the launcher. Empty when the process was not started under likwid-perfctr -m.

markers_availablefunction#

View source

def markers_available() -> bool

Whether this process runs under the LIKWID marker API.

markerinit() silently degrades to a no-op when LIKWID’s environment is absent, so this is what distinguishes “there will be counter data” from “the script was started as a plain python script.py”.

parse_marker_filefunction#

View source

def parse_marker_file(path=None) -> list[LikwidRegionResult]

Read LIKWID’s marker file directly, without calling into LIKWID.

The file markerclose() writes is plain text::

<nthreads> <nregions> <ngroups>
<region_id>:<tag>-<group_id>          (one line per region)
<region_id> <group_id> <cpu> <call_count> <time> <nevents> <values...>

Parsing it here is the crash-proof path: it touches no counters and calls no LIKWID function, so it cannot take the interpreter down the way re-initializing perfmon can. The cost is that event names, counter registers and derived metrics are not in the file --- those exist only inside LIKWID’s group definitions --- so events come back positionally named and the metric list is empty.

Parameters

path : str, optional Marker file to read (default: $LIKWID_FILEPATH).

Returns

list of LikwidRegionResult One entry per region, ordered by region id. Empty if the file is missing or malformed.

snapshots_to_resultsfunction#

View source

def snapshots_to_results(snapshots: Iterable[dict]) -> list[LikwidRegionResult]

Convert marker-API snapshots into the common result structure.

Used as the fallback when the full API cannot re-open the counters, so the snapshots taken before markerclose() can still be written to HDF5.

write_likwid_resultsfunction#

View source

def write_likwid_results(h5file, results: Iterable[LikwidRegionResult], environment: dict | None = None) -> None

Write collected LIKWID results into an open HDF5 file.

Creates (replacing any previous copy) a likwid group holding one subgroup per region under likwid/regions/<tag>.

Parameters

h5file : h5py.File or h5py.Group Destination, typically the per-rank profiling file. results : iterable of LikwidRegionResult Regions to store. environment : dict, optional LIKWID environment variables to record as attributes on the group, so the event set a file was measured with stays with the data.