# scope_profiler.native_trace

*module*

Reading the per-rank output written by the native (C and Fortran) region APIs.

The C and Fortran modules shipped in ``scope_profiler/c/`` and
``scope_profiler/fortran/`` record regions with no Python involved, and write
one small file per rank at ``sp_finalize()``.

A C build compiled with ``SP_USE_HDF5`` writes ``<prefix>_rank<NNNNN>.h5``
directly, in the same schema-2 layout :mod:`scope_profiler.h5writer`
produces: one rank's profile, already readable by
:func:`~scope_profiler.read_h5` with no import step. Everything this module
then does for those files is *merging* the ranks of one run into a single
profile.

Every other native build - Fortran, and C without HDF5 - writes the compact
binary trace documented below instead, one ``.spt`` per rank. Both write a
*compatible* trace format, so a program built from either (or both) lands in
one profile. This module turns those files into the same
:class:`~scope_profiler.results.ProfilingResults` - and the same HDF5 layout
-- a Python run produces, so a Fortran run gets the whole post-processing
stack (summaries, plots, exporters, ``plot``) for free. The two kinds of
input mix freely in one import.

Trace layout, little- or big-endian, as written by ``sp_finalize``::

    char[8]   "SCOPEPRF"
    int32     format version (1 or 2)
    int32     rank
    int64     number of regions
    per region:
        int32     length of the name in bytes
        char[]    name
        - version 2 only --
        int32     length of the source file path in bytes (0 if unknown)
        char[]    source file path
        int32     source line (-1 if unknown)
        - end version 2 only --
        int64     number of calls
        int64[]   start timestamps, nanoseconds
        int64[]   end timestamps, nanoseconds

Version 1 is what the Fortran API writes, and what older C releases wrote: no
per-region source location. Version 2 is the current C API, which can attach
one via ``sp_region_at()``. Both are readable, per file, so a mixed C/Fortran
run merges normally.

The timestamps come from the same clock as :func:`time.perf_counter_ns`, so
Fortran and Python regions from one process tree share a timeline.

[View source](https://github.com/max-models/scope-profiler/blob/devel/src/scope_profiler/native_trace.py#L1-L1)

## scope_profiler.native_trace.C_DIR

*attribute* · *module attribute*

```python
C_DIR = Path(__file__).resolve().parent / 'c'
```

[View source](https://github.com/max-models/scope-profiler/blob/devel/src/scope_profiler/native_trace.py#L58-L58)

## scope_profiler.native_trace.FORMAT_VERSION

*attribute* · *module attribute*

```python
FORMAT_VERSION = 2
```

Newest layout this module writes metadata for; ``sp_finalize`` writes the
same number. Version 1 (written by the Fortran API, and by older C releases)
has no per-region source location; version 2 (the C API) adds one. Both are
readable - see :func:`read_trace`.

[View source](https://github.com/max-models/scope-profiler/blob/devel/src/scope_profiler/native_trace.py#L110-L110)

## scope_profiler.native_trace.FORTRAN_DIR

*attribute* · *module attribute*

```python
FORTRAN_DIR = Path(__file__).resolve().parent / 'fortran'
```

[View source](https://github.com/max-models/scope-profiler/blob/devel/src/scope_profiler/native_trace.py#L55-L55)

## scope_profiler.native_trace.HDF5_SUFFIX

*attribute* · *module attribute*

```python
HDF5_SUFFIX = '.h5'
```

Extension ``sp_finalize`` gives a directly-written HDF5 profile.

[View source](https://github.com/max-models/scope-profiler/blob/devel/src/scope_profiler/native_trace.py#L122-L122)

## scope_profiler.native_trace.KNOWN_FORMAT_VERSIONS

*attribute* · *module attribute*

```python
KNOWN_FORMAT_VERSIONS = (1, 2)
```

Trace format versions this reader accepts.

[View source](https://github.com/max-models/scope-profiler/blob/devel/src/scope_profiler/native_trace.py#L116-L116)

## scope_profiler.native_trace.MAGIC

*attribute* · *module attribute*

```python
MAGIC = b'SCOPEPRF'
```

First eight bytes of every trace file.

[View source](https://github.com/max-models/scope-profiler/blob/devel/src/scope_profiler/native_trace.py#L107-L107)

## scope_profiler.native_trace.RANK_HDF5_GLOB

*attribute* · *module attribute*

```python
RANK_HDF5_GLOB = *_rank[0-9]*{HDF5_SUFFIX}
```

How :func:`find_traces` recognizes per-rank HDF5 output in a directory.

Matching the ``_rank<NNNNN>`` that ``sp_finalize`` writes, rather than every
``.h5``, keeps a merged profile sitting in the same directory (the output of
a previous import, say) from being swallowed as an input to the next one.
Naming such a file explicitly on the command line still reads it.

[View source](https://github.com/max-models/scope-profiler/blob/devel/src/scope_profiler/native_trace.py#L125-L125)

## scope_profiler.native_trace.TRACE_SUFFIX

*attribute* · *module attribute*

```python
TRACE_SUFFIX = '.spt'
```

Extension ``sp_finalize`` gives a binary trace.

[View source](https://github.com/max-models/scope-profiler/blob/devel/src/scope_profiler/native_trace.py#L119-L119)

## scope_profiler.native_trace.TraceFormatError

*class*

```python
class TraceFormatError(ValueError)
```

Bases: `ValueError`

A file is not a scope-profiler Fortran trace, or is truncated.

[View source](https://github.com/max-models/scope-profiler/blob/devel/src/scope_profiler/native_trace.py#L137-L138)

## scope_profiler.native_trace.c_include_dir

*function*

```python
def c_include_dir() -> Path
```

Directory holding ``scope_profiler.h``, for the compiler's ``-I``.

Returns
-------
Path
    The include directory.

[View source](https://github.com/max-models/scope-profiler/blob/devel/src/scope_profiler/native_trace.py#L96-L104)

## scope_profiler.native_trace.c_source_path

*function*

```python
def c_source_path() -> Path
```

Path to ``scope_profiler.c``, the implementation entry point to compile.

Its public and private implementation headers sit next to it;
:func:`c_include_dir` is what to put on the compiler's include path::

    cc -c $(python -c             "import scope_profiler.native_trace as t; print(t.c_source_path())")            -I$(python -c             "import scope_profiler.native_trace as t; print(t.c_include_dir())")

Returns
-------
Path
    The C source file.

[View source](https://github.com/max-models/scope-profiler/blob/devel/src/scope_profiler/native_trace.py#L77-L93)

## scope_profiler.native_trace.convert_traces

*function*

```python
def convert_traces(inputs, output_path, label: str | None = None)
```

Convert Fortran traces into a standard scope-profiler HDF5 file.

The result is indistinguishable from one a Python run wrote, so
``scope-profiler plot`` / ``inspect`` and :func:`~scope_profiler.read_h5`
work on it directly.

Parameters
----------
inputs : path or sequence of paths
    Trace files and/or directories containing them.
output_path : str or Path
    HDF5 file to write.
label : str, optional
    Name for the run; defaults to the output file's stem.

Returns
-------
Path
    The file that was written.

[View source](https://github.com/max-models/scope-profiler/blob/devel/src/scope_profiler/native_trace.py#L532-L557)

## scope_profiler.native_trace.find_traces

*function*

```python
def find_traces(inputs) -> list
```

Collect native output files from paths, directories, or a mix of both.

Parameters
----------
inputs : path or sequence of paths
    Files to read, and/or directories to search (non-recursively) for
    ``*.spt`` traces and ``*_rank<NNNNN>.h5`` per-rank profiles.

Returns
-------
list of Path
    The files, sorted, with duplicates removed.

Raises
------
FileNotFoundError
    If an input does not exist, or a directory holds neither kind of file.

[View source](https://github.com/max-models/scope-profiler/blob/devel/src/scope_profiler/native_trace.py#L283-L323)

## scope_profiler.native_trace.fortran_source_path

*function*

```python
def fortran_source_path() -> Path
```

Path to ``scope_profiler.f90``, the module to compile into your program.

It ships with the package, so this works from an installed wheel::

    gfortran -c $(python -c             "import scope_profiler.native_trace as t; print(t.fortran_source_path())")

Returns
-------
Path
    The Fortran module source.

[View source](https://github.com/max-models/scope-profiler/blob/devel/src/scope_profiler/native_trace.py#L61-L74)

## scope_profiler.native_trace.load_traces

*function*

```python
def load_traces(inputs, label: str | None = None)
```

Read native output into the standard post-processing API.

Both formats are accepted, and mix: ``.spt`` traces from a Fortran or
plain C build, and the per-rank ``.h5`` profiles an ``SP_USE_HDF5`` C
build writes. Every rank of a run lands in one result either way.

Parameters
----------
inputs : path or sequence of paths
    Files and/or directories containing them (see :func:`find_traces`).
label : str, optional
    Name for the run in summaries, charts and exports.

Returns
-------
ProfilingResults
    The same object a Python run produces, so every summary, plot and
    exporter works on it unchanged.

Raises
------
TraceFormatError
    If two files claim the same rank.

[View source](https://github.com/max-models/scope-profiler/blob/devel/src/scope_profiler/native_trace.py#L397-L470)

## scope_profiler.native_trace.read_native_h5

*function*

```python
def read_native_h5(path) -> tuple
```

Read the per-rank HDF5 profile an ``SP_USE_HDF5`` C build writes.

The file is an ordinary schema-2 profile holding one rank, so this is
:func:`~scope_profiler.read_h5` plus a regrouping into the ``(rank,
regions)`` shape :func:`read_trace` returns - which is what lets one
import mix directly-written HDF5 with ``.spt`` traces from other ranks.

Parameters
----------
path : str or Path
    An ``.h5`` file written by ``sp_finalize()``. A file holding several
    ranks (a merged profile) is accepted too, and contributes all of them.

Returns
-------
tuple
    ``(ranks, metadata)``, where ``ranks`` maps a rank to its
    ``{region name: Region}`` and ``metadata`` is the run metadata stored
    in the file.

[View source](https://github.com/max-models/scope-profiler/blob/devel/src/scope_profiler/native_trace.py#L326-L354)

## scope_profiler.native_trace.read_native_ranks

*function*

```python
def read_native_ranks(path) -> tuple
```

Read one native output file, whichever of the two formats it is in.

The format-agnostic entry point: callers that must accept whatever a
native build happened to write - :func:`load_traces` and
``ProfileManager.finalize(native_traces=...)`` - go through this rather
than choosing :func:`read_trace` or :func:`read_native_h5` by suffix
themselves.

Parameters
----------
path : str or Path
    A ``.spt`` trace or an ``.h5`` profile written by ``sp_finalize()``.

Returns
-------
tuple
    ``(ranks, metadata)``, where ``ranks`` maps a rank to its
    ``{region name: Region}``. A ``.spt`` trace contributes exactly one
    rank and no metadata; an ``.h5`` file contributes every rank it holds
    (one, as ``sp_finalize()`` writes it) and the metadata it stores.

[View source](https://github.com/max-models/scope-profiler/blob/devel/src/scope_profiler/native_trace.py#L357-L394)

## scope_profiler.native_trace.read_trace

*function*

```python
def read_trace(path) -> tuple
```

Read one rank's trace file.

Parameters
----------
path : str or Path
    A ``.spt`` file written by ``sp_finalize()``.

Returns
-------
tuple
    ``(rank, regions)``, where ``regions`` maps a region name to
    ``(start_times, end_times)`` int64 arrays in nanoseconds - exactly the
    shape :class:`~scope_profiler.profile_manager.RankPayload` carries.
    Each value also carries ``.source_file`` / ``.source_lineno``
    attributes (both None on a version-1 trace, or a region registered
    without ``sp_region_at()``), without changing how it unpacks.

Raises
------
TraceFormatError
    If the file is not a trace, is truncated, or was written by a newer
    format version.

[View source](https://github.com/max-models/scope-profiler/blob/devel/src/scope_profiler/native_trace.py#L190-L280)

## scope_profiler.native_trace.write_results

*function*

```python
def write_results(results, output_path)
```

Write any :class:`ProfilingResults` out as a standard HDF5 file.

Goes through :class:`~scope_profiler.h5writer.ProfilingWriter`, so the
result has exactly the layout a Python run produces - which is what lets
an imported (or merged) run be read back by
:func:`~scope_profiler.read_h5` and fed to every plot and exporter.

Parameters
----------
results : ProfilingResults
    The run to write.
output_path : str or Path
    HDF5 file to create.

Returns
-------
Path
    The file that was written.

[View source](https://github.com/max-models/scope-profiler/blob/devel/src/scope_profiler/native_trace.py#L473-L529)