Skip to content
scope-profiler
Projects

Deploying a profiling dashboard

Download notebookView on GitHubRun when the site was built.

Install for this page: pip install scope-profiler to write the data; npm install @scope-profiler/plotly plus a Plotly bundle to render it.

Plotly figures for the web covers the figure builders one at a time. This page is the whole loop: one command that writes everything a page needs, and the browser-side glue that keeps a page of figures in step with a theme toggle and a filter box.

Nothing here is a framework. The output is static JSON and the input is a <div>, so the same recipe works under Astro, Eleventy, Next, or a hand-written index.html served from a bucket. See Try it: drag and drop a profile for the smallest possible version: a drag-and-drop page with no build step at all.

One command for a run’s data

Section titled “One command for a run’s data”

export plot-data writes the plot JSON. --with adds the three files a page usually embeds next to the charts, so a site generator makes one pass over the profiles instead of three. The cell below runs it on three runs of a small mock solver (at 1, 2 and 4 MPI ranks, run_1.h5 to run_4.h5):

sp(
    "export plot-data run*.h5 "
    "-o site/public/figures "
    "--format json "
    "--plots gantt durations timeseries flame_graph speedup "
    "--with"
)
$ scope-profiler export plot-data run*.h5 -o site/public/figures --format json --plots gantt durations timeseries flame_graph speedup --with
Plotting combined Gantt chart for files: run_1, run_2, run_4
Plotting flame graph for: run_1, run_2 (rank 0), run_4 (rank 0)
For interactive flame-chart hover details, use --backend plotly.
Plotting duration comparison (total) for files: run_1, run_2, run_4
Plotting duration over time for files: run_1, run_2, run_4
Plotting speedup comparison using x_field='num_ranks', values: 1, 2, 4
Outputs saved to:
  site/public/figures/gantt_data.json
  site/public/figures/flame_graph_data.json
  site/public/figures/durations_data.json
  site/public/figures/duration_timeseries_data.json
  site/public/figures/speedup_data.json
  site/public/figures/region_statistics.json
  site/public/figures/profile_run_1_rank0.prof
  site/public/figures/profile_run_2_rank0.prof
  site/public/figures/profile_run_4_rank0.prof
  site/public/figures/profile_run_1.speedscope.json
  site/public/figures/profile_run_2.speedscope.json
  site/public/figures/profile_run_4.speedscope.json
  site/public/figures/flamegraph_run_1_rank0.svg
  site/public/figures/flamegraph_run_2_rank0.svg
  site/public/figures/flamegraph_run_4_rank0.svg

That writes, into site/public/figures/:

listing("site/public/figures")
Show the written files
site/public/figures/duration_timeseries_data.json  (12164 bytes)
site/public/figures/durations_data.json  (2508 bytes)
site/public/figures/flame_graph_data.json  (6577 bytes)
site/public/figures/flamegraph_run_1_rank0.svg  (8937 bytes)
site/public/figures/flamegraph_run_2_rank0.svg  (8280 bytes)
site/public/figures/flamegraph_run_4_rank0.svg  (8209 bytes)
site/public/figures/gantt_data.json  (18298 bytes)
site/public/figures/profile_run_1.speedscope.json  (1917 bytes)
site/public/figures/profile_run_1_rank0.prof  (667 bytes)
site/public/figures/profile_run_2.speedscope.json  (1922 bytes)
site/public/figures/profile_run_2_rank0.prof  (667 bytes)
site/public/figures/profile_run_4.speedscope.json  (1925 bytes)
site/public/figures/profile_run_4_rank0.prof  (667 bytes)
site/public/figures/region_statistics.json  (27202 bytes)
site/public/figures/speedup_data.json  (2038 bytes)
FileWhat renders it
<kind>_data.json@scope-profiler/plotly, one file per --plots entry
region_statistics.jsonthe summary and comparison builders, and any HTML table
profile_<run>_rank0.profany pstats viewer (snakeviz), offered as a download
profile_<run>.speedscope.jsonspeedscope, embedded in an iframe
flamegraph_<run>_rank0.svginlined into the page, so its per-frame tooltips work

With one input file the companions are named profile_rank0.prof, profile.speedscope.json and flamegraph_rank0.svg; with several, the run name is part of each filename, as above.

The flame graph SVG is inlined as it is, so its tooltips work when you hover a frame:

from IPython.display import SVG


SVG(filename=WORKDIR / "site/public/figures/flamegraph_run_2_rank0.svg")
~:0:<run_2 rank 0> 100.00% (1 1 0.0 0.218033338) <run_2 rank 0> solver.py:18:setup 15.86% (1 1 0.034574757 0.034574757) setup solver.py:22:timestep 84.14% (3 3 0.000131249 0.183458581) timestep solver.py:23:timestep > assemble 19.88% (3 3 0.043335566 0.043335566) timestep > assemble solver.py:25:timestep > solve 47.56% (3 3 0.10370690400000002 0.10370690400000002) timestep > solve solver.py:27:timestep > halo_exchange 11.10% (3 3 0.024210522999999998 0.024210522999999998) timestep > halo_exchange solver.py:30:timestep > io 5.54% (1 1 0.012074339 0.012074339) timestep > io solver.py:18:setup 15.86% (1 1 0.034574757 0.034574757) setup ~:0:<run_2 rank 0> 15.86% (1 1 0.034574757 0.034574757) <run_2 rank 0> solver.py:23:timestep > assemble 19.88% (3 3 0.043335566 0.043335566) timestep > assemble solver.py:22:timestep 19.88% (3 3 0.043335566 0.043335566) timestep ~:0:<run_2 rank 0> 0.06% (3 3 0.000131249 0.183458581) <run_2 rank 0> solver.py:25:timestep > solve 47.56% (3 3 0.10370690400000002 0.10370690400000002) timestep > solve solver.py:22:timestep 47.56% (3 3 0.10370690400000002 0.10370690400000002) timestep ~:0:<run_2 rank 0> 0.06% (3 3 0.000131249 0.183458581) <run_2 rank 0> solver.py:27:timestep > halo_exchange 11.10% (3 3 0.024210522999999998 0.024210522999999998) timestep > halo_exchange solver.py:22:timestep 11.10% (3 3 0.024210522999999998 0.024210522999999998) timestep ~:0:<run_2 rank 0> 0.06% (3 3 0.000131249 0.183458581) <run_2 rank 0> solver.py:30:timestep > io 5.54% (1 1 0.012074339 0.012074339) timestep > io solver.py:22:timestep 5.54% (1 1 0.012074339 0.012074339) timestep ~:0:<run_2 rank 0> 0.06% (3 3 0.000131249 0.183458581) <run_2 rank 0>

Name the companions to pick a subset - --with prof speedscope. A bare --with means “everything this install can produce”: the flame graph needs the pproc extra, and without it that one export is reported and skipped rather than failing the run. Naming --with flamegraph outright still fails, because then you asked for that file specifically.

A page of figures

Section titled “A page of figures”

Every figure on a real page shares two problems. Plotly bakes colours into the layout, so a theme toggle means rebuilding every figure - which means knowing which containers hold one and what each was built from. And a filter box needs a syntax, which the builders deliberately do not have: filterRegion takes a predicate, because a library should not decide how a user spells a filter.

@scope-profiler/plotly/dashboard is a second entry point holding exactly that glue. It is separate because it reaches for document and window; importing @scope-profiler/plotly itself never does, and stays usable server-side.

import Plotly from "plotly.js-dist-min";
import { createFigureRegistry } from "@scope-profiler/plotly/dashboard";


const figures = createFigureRegistry(Plotly);


// Rebuild everything when the page's theme toggle fires. The registry reads
// `data-theme` off the document element by default.
figures.watch();


const payload = await fetch("/figures/durations_data.json").then((r) => r.json());
await figures.render(document.querySelector("#durations"), payload, {
  metric: "total",
  regionFilter: filterBox.value,
});

render dispatches on the document’s own plot field, so one call handles any payload. Call it again on the same container - a new metric, an edited filter — and the figure is redrawn through Plotly’s react, so the reader keeps the zoom and pan they had.

For a figure no plot kind names, pass the builder:

import { buildComparisonFigure } from "@scope-profiler/plotly";


await figures.render(element, statistics, {
  build: buildComparisonFigure,
  files: ["baseline", "candidate"],
});

figures.forget(element) stops tracking one container and purges its plot; figures.clear() does the same for all of them. Both matter in a single-page app, where a route change leaves Plotly’s handlers attached to nodes the page has finished with.

The filter syntax

Section titled “The filter syntax”

A filter is a comma-separated list of terms, each matched case-insensitively as a substring of the region name. A leading ^ anchors a term to the start, which is how a dashboard separates a group of regions from the regions that merely mention it:

FilterKeeps
solveevery region whose name contains “solve”
prop:, setup: totalboth groups
^prop:prop: push, but not setup prop: push
(empty)every region - no predicate is built at all

The same three functions drive a chart and an HTML table, so a page can filter both from one box:

import {
  matchesRegionFilter,
  parseRegionFilter,
  regionFilter,
} from "@scope-profiler/plotly/dashboard";


// For a figure, via the registry's `regionFilter` option, or by hand:
buildDurationsFigure(payload, { filterRegion: regionFilter(box.value) });


// For a table of the same regions:
const terms = parseRegionFilter(box.value);
for (const row of tbody.querySelectorAll("tr[data-region]"))
  row.hidden = terms.length && !matchesRegionFilter(row.dataset.region, terms);

regionFilter returns undefined for an empty box rather than a predicate that accepts everything, which is both faster and what the builders document.

What stays in your site

Section titled “What stays in your site”

The registry and the filter are the parts every deployment needs. Everything that makes a dashboard yours stays in it: how runs are grouped into cases, what a run’s metadata means, which regions the filter boxes start out prefilled with, and the pages themselves. scope-profiler has no opinion about any of that, and a page is a better place to decide it than a profiler is.