Plotly figures for the web

Download notebook View on GitHubRun when the site was built.

Install for this page: export data with pip install scope-profiler; render it in a web project with npm install @scope-profiler/plotly plotly.js-dist-min.

@scope-profiler/plotly is the browser-facing companion to scope-profiler export plot-data --format json. It builds plain Plotly figure specifications from the exported JSON, without importing Plotly itself. That makes it suitable for static sites and any JavaScript framework.

Install

Install the figure builders plus the Plotly bundle your application wants to use:

npm install @scope-profiler/plotly plotly.js-dist-min

Then fetch a JSON payload and render it with your own Plotly instance:

import Plotly from "plotly.js-dist-min";
import { buildGanttFigure, renderFigure } from "@scope-profiler/plotly";

const payload = await fetch("/figures/gantt_data.json").then((response) => response.json());
const figure = buildGanttFigure(payload);
await renderFigure(Plotly, document.querySelector("#gantt"), figure);

The gantt_data.json fetched there is written by the exporter. The cell below produces it from a small mock solver run (at 1, 2 and 4 MPI ranks) and shows the start of the payload:

sp("export plot-data run_2.h5 -o figures --format json --plots gantt durations")

import json

payload = json.loads((WORKDIR / "figures" / "gantt_data.json").read_text())
print(json.dumps({**payload, "intervals": payload["intervals"][:2]}, indent=2))

Show gantt_data.json (first two intervals)

$ scope-profiler export plot-data run_2.h5 -o figures --format json --plots gantt durations
Plotting Gantt chart for ranks: [0, 1]
Plotting duration comparison (total) for files: run_2
Outputs saved to:
  figures/gantt_data.json
  figures/durations_data.json
  figures/region_statistics.json
{
  "format": "scope-profiler-plot-data",
  "format_version": 1,
  "plot": "gantt",
  "intervals": [
    {
      "file": "run_2",
      "rank": 0,
      "region": "setup",
      "start_seconds": 2.749999907791789e-07,
      "end_seconds": 0.03457793000001175,
      "depth": 0
    },
    {
      "file": "run_2",
      "rank": 1,
      "region": "setup",
      "start_seconds": 0.0,
      "end_seconds": 0.03457517000001076,
      "depth": 0
    }
  ],
  "colors": {
    "setup": "#e377c2",
    "timestep": "#9edae5",
    "assemble": "#1f77b4",
    "solve": "#bcbd22",
    "halo_exchange": "#2ca02c",
    "io": "#9467bd"
  },
  "file_ranks": {
    "run_2": 2
  }
}

Live examples

The figures below are built by the published npm package in this page. Each uses the same shape of payload that scope-profiler export plot-data --format json writes.

Gantt — a lane per region and rank

Durations — compare regions between runs

Flame — nested calls

Speedup — region scaling

Duration time series — performance drift

Histogram — call-duration distribution

Rank heatmap — duration by region and rank

Imbalance — per-rank duration

Density — timeline occupancy per region

Call graph — who calls whom

Region summary — the slowest regions in a run

One call for any payload

buildFigure reads the plot kind stamped on every plot-data document and picks the builder itself, so a page that renders whatever the profiler wrote does not need to know which chart it is getting:

import Plotly from "plotly.js-dist-min";
import { buildFigure, renderFigure } from "@scope-profiler/plotly";

for (const name of ["gantt", "durations", "rank_heatmap"]) {
  const payload = await fetch(`/figures/${name}_data.json`).then((response) => response.json());
  await renderFigure(Plotly, document.querySelector(`#${name}`), buildFigure(payload));
}

It throws on a document that is not scope-profiler-plot-data, and on one whose format_version is newer than the installed package understands. For JSON written by a scope-profiler old enough to predate the envelope, it falls back to inferPlotKind(payload), which reads the shape of the payload instead; pass { plot: "gantt" } to settle the case by hand.

Builders

Builder	Input payload	Use case
`buildGanttFigure(payload, { laneBy })`	`gantt_data.json`	Timeline of every call, a lane per region and rank (`laneBy: "rank"` for one row per rank)
`buildFlameFigure(payload)`	`flame_chart_data.json` (or `flame_graph_data.json`)	Nested call hierarchy
`buildDurationsFigure(payload, { metric })`	`durations_data.json`	Per-region duration bars
`buildSpeedupFigure(payload)`	`speedup_data.json`	Speedup curve
`buildDurationTimeseriesFigure(payload)`	`duration_timeseries_data.json`	Duration drift over time
`buildHistogramFigure(payload)`	`histogram_data.json`	Call-duration distribution
`buildRankHeatmapFigure(payload)`	`rank_heatmap_data.json`	Rank × region duration matrix
`buildImbalanceFigure(payload)`	`imbalance_data.json`	Per-rank duration imbalance
`buildDensityFigure(payload)`	`timeline_density_data.json`	Binned timeline occupancy, for traces too large to draw call by call
`buildCallgraphFigure(payload)`	`callgraph_data.json`	Sankey call graph, from either callgraph shape
`buildRegionSummaryFigure(payload, { metric, topN, files, orientation })`	`region_statistics.json`	The slowest regions of a run, ranked
`buildComparisonFigure(payload, { files, metric })`	`region_statistics.json`	Every run side by side, over the regions they all recorded; `{ comparison: "percent" \| "absolute" }` draws a single delta bar instead, for exactly two runs
`buildLikwidFigure(payload)`	`likwid_data.json`	One LIKWID hardware-counter metric
`buildRooflineFigure(payload)`	`roofline_data.json`	Per-region LIKWID roofline placement
`buildWeakScalingFigure(payload)`	`weak_scaling_data.json`	Weak-scaling curve
`buildScalingEfficiencyFigure(payload)`	`scaling_efficiency_data.json`	Strong-scaling parallel-efficiency curve
`buildWeakScalingEfficiencyFigure(payload)`	`weak_scaling_efficiency_data.json`	Weak-scaling efficiency curve
`buildFigure(payload)`	any of the above	Dispatch on the document’s own `plot` kind

The exporter writes these files. Here every kind that needs no LIKWID data is exported from the three mock runs, with the plot stamp and the row list of each payload:

sp(
    "export plot-data run_1.h5 run_2.h5 run_4.h5 -o all_kinds --format json "
    "--plots gantt flame_chart durations speedup timeseries histogram rank_heatmap "
    "imbalance density callgraph weak_scaling scaling_efficiency weak_scaling_efficiency"
)

for path in sorted((WORKDIR / "all_kinds").glob("*_data.json")):
    document = json.loads(path.read_text())
    rows = next(key for key, value in document.items() if isinstance(value, list))
    print(f"{path.name:36} plot={document['plot']!r:28} rows under {rows!r}")

Show the exported payload files

$ scope-profiler export plot-data run_1.h5 run_2.h5 run_4.h5 -o all_kinds --format json --plots gantt flame_chart durations speedup timeseries histogram rank_heatmap imbalance density callgraph weak_scaling scaling_efficiency weak_scaling_efficiency
Plotting combined Gantt chart for files: run_1, run_2, run_4
Plotting timeline density
Plotting flame chart for: run_1, run_2 (rank 0), run_4 (rank 0)
For interactive flame-chart hover details, use --backend plotly.
Plotting duration comparison (total) for files: run_1, run_2, run_4
Plotting duration over time for files: run_1, run_2, run_4
Plotting duration histograms for files: run_1, run_2, run_4
Plotting per-rank imbalance (total) for files: run_1, run_2, run_4
Plotting speedup comparison using x_field='num_ranks', values: 1, 2, 4
Plotting weak scaling comparison using x_field='num_ranks', values: 1, 2, 4
Plotting weak scaling efficiency using x_field='num_ranks', values: 1, 2, 4
Plotting rank × region duration heatmap
Outputs saved to:
  all_kinds/gantt_data.json
  all_kinds/timeline_density_data.json
  all_kinds/flame_chart_data.json
  all_kinds/callgraph_data.json
  all_kinds/durations_data.json
  all_kinds/duration_timeseries_data.json
  all_kinds/histogram_data.json
  all_kinds/imbalance_data.json
  all_kinds/speedup_data.json
  all_kinds/weak_scaling_data.json
  all_kinds/weak_scaling_efficiency_data.json
  all_kinds/rank_heatmap_data.json
  all_kinds/scaling_efficiency_data.json
  all_kinds/region_statistics.json
callgraph_data.json                  plot='callgraph'                  rows under 'calls'
duration_timeseries_data.json        plot='timeseries'                 rows under 'points'
durations_data.json                  plot='durations'                  rows under 'bars'
flame_chart_data.json                plot='flame'                      rows under 'calls'
gantt_data.json                      plot='gantt'                      rows under 'intervals'
histogram_data.json                  plot='histogram'                  rows under 'bins'
imbalance_data.json                  plot='imbalance'                  rows under 'points'
rank_heatmap_data.json               plot='rank_heatmap'               rows under 'points'
scaling_efficiency_data.json         plot='scaling_efficiency'         rows under 'points'
speedup_data.json                    plot='speedup'                    rows under 'points'
timeline_density_data.json           plot='density'                    rows under 'points'
weak_scaling_data.json               plot='weak_scaling'               rows under 'points'
weak_scaling_efficiency_data.json    plot='weak_scaling_efficiency'    rows under 'points'

Use renderFigure(Plotly, element, figure) as a convenience, or pass figure.data and figure.layout to Plotly.newPlot directly.

Both efficiency curves store their y column as efficiency, so which reading a payload gets is decided by the document’s own plot field, not by its rows: buildFigure gets it right on its own, and a direct call needs the right builder. They measure against different ideals. scaling_efficiency is for a strong-scaling study, where the problem size is fixed and the ideal is a speedup proportional to the rank count. weak_scaling_efficiency is for a study that grows the problem with the machine, where every run does the same work per rank and the ideal is constant runtime.

Themes

Every builder paints its chrome - text, gridlines, hover surface, the dashed ideal lines - from a theme. The default, auto, leaves the text colour unset and draws gridlines in a half-transparent grey that reads on any background, so a figure inherits whatever the host page does.

A page that knows which theme it is in gets better contrast by saying so:

import { setTheme, buildDurationsFigure } from "@scope-profiler/plotly";

// Once per theme change, then rebuild the figures on the page.
setTheme(document.documentElement.dataset.theme === "dark" ? "dark" : "light");

// Or per call, which wins over the default.
buildDurationsFigure(payload, { theme: "dark" });

// Or with your own tokens, over any of the three.
buildDurationsFigure(payload, { theme: { text: "#f5f5f5", grid: "#333" } });

Plotly bakes colours into the layout, so a theme change means rebuilding the figures, not restyling them in place. The categorical series palette is not themed: those eight hues are chosen to read on both backgrounds.

Rebuilding means knowing which containers hold a figure and what each was built from. @scope-profiler/plotly/dashboard keeps that bookkeeping, along with a filter syntax for the filterRegion predicate - see Deploying a profiling dashboard.

Payloads with more than one run

export plot-data accepts several profiles at once, and every row of the resulting payload names the run it came from. The builders keep those runs apart rather than pooling them: the gantt, density and rank-heatmap charts give each run its own lanes, while the histogram, imbalance and duration time series give each run its own trace, named run / region. A region keeps its colour across runs - what changes between runs is the marker symbol on a line, or the bar pattern on a histogram - so the same region stays recognisable while the runs stay distinguishable. Single-run payloads are unaffected: series are named by region alone, as before.

The exported payloads from the three runs above show the run name on every row:

for name, rows in [
    ("gantt", "intervals"),
    ("histogram", "bins"),
    ("imbalance", "points"),
]:
    document = json.loads((WORKDIR / "all_kinds" / f"{name}_data.json").read_text())
    print(f"{name}_data.json: runs =", sorted({row["file"] for row in document[rows]}))

gantt_data.json: runs = ['run_1', 'run_2', 'run_4']
histogram_data.json: runs = ['run_1', 'run_2', 'run_4']
imbalance_data.json: runs = ['run_1', 'run_2', 'run_4']