Skip to content
Projects

Vlasov-Ampere, CPU node vs GPU node: GPU node

all 6all levels

gpu-node.h52026-10-17 12:00 UTCsynthetic 4-GPU node4 ranks6 regions0.0895414 s profiled

← Vlasov-Ampere, CPU node vs GPU node · ‹ CPU node

Wall time89.5 mssetup to finalize
Ranks4
Regions6
In regions95%of the session
Top hotspotsolve80.3% of the session
Load imbalance+4%solve, slowest on rank 2
  • solve (in simulation) is the largest hotspot: 278 ms over 40 calls, 80.3% of the session.
  • The three largest hotspots account for 92% of the session.

The leaves of the call tree with the most time: regions without nested regions, and the own time parents spend outside theirs. Shares are of the session. Click one to find it in the table.

region% sessiontotal [s]gpu total [s]gpu avg [s]trend
└─ (own) 4.63% 0.016000
│ └─ (own) 0.00% 0.000000

Durations are in seconds. (own) rows show a region's time excluding its children. % session uses wall-clock coverage; overlapping recursive calls count once. Click a row for its call site and per-rank breakdown.

Each region's own time per rank, excluding nested regions, so a parent does not repeat its children's imbalance. Excess is how much longer the slowest rank spends than the average rank: the most balancing that region could save. Regions are ordered by it.

regionmeanminmaxslowest rankimbalanceexcessmin · mean · max
solve69.4 ms67.2 ms72.5 ms2+4%3.12 ms
setup4 ms4 ms4 ms0+0%0 s
copy_to_device3 ms3 ms3 ms0+0%0 s
exchange6 ms6 ms6 ms0+0%0 s

Each cell compares one rank's time in a region with the region's mean over the ranks. Rank 2, in red in the header, spends the most time in regions overall. Hover a cell for its time.

faster than the meanslower

regionmean0123
solve69.4 ms−2.6%+1.2%+4.5%−3.2%
exchange6 ms0%0%0%0%
setup4 ms0%0%0%0%
copy_to_device3 ms0%0%0%0%
all regions82.4 ms−2.2%+1.0%+3.8%−2.7%
Region durations

Each bar shows a region's total recorded duration, summed over the selected ranks, longest first.

Timeline: GPU node (rank 0)

Each bar is one recorded region call on rank 0. Its position and width show when the call started and how long it ran; colors identify regions.

Rank imbalance

Each line compares a region's total duration by rank; its dashed line marks the mean across ranks. Points far from the mean identify stragglers.

Rank heatmap

This heatmap uses exclusive timings. Exclusive duration is the time spent in a region itself, excluding time spent in nested child regions; this prevents the enclosing session region from dominating the heatmap.

Download run.py (715 B)

"""Landau damping of a Langmuir wave with a Vlasov-Ampere solver: the script both runs use."""
import sys
import yaml
from scope_profiler import ProfileManager
def main(parameter_file):
with open(parameter_file) as handle:
parameters = yaml.safe_load(handle)
with ProfileManager.session(label=parameters["label"]):
with ProfileManager.region("setup"):
state = initialise(parameters)
for _ in range(parameters["steps"]):
with ProfileManager.region("solve"):
state = advance(state, parameters["dt"])
with ProfileManager.region("exchange"):
exchange_halos(state)
if __name__ == "__main__":
main(sys.argv[1])
commit0000000
systemsynthetic
chip_informationsynthetic 4-GPU node
mpi_size4
num_nodes1
omp_num_threads1
platformsynthetic
scope_profiler_versionsample
timestamp2026-10-17T12:00:00Z
total_cores4