HPC Performance Measurement
Measure high-performance computing workloads with controlled MPI and OpenMP placement, strong and weak scaling, and separate compute, communication, and I/O evidence. Use when comparing scientific builds, parallel settings, or cluster runs.
Produce a repeatable scaling report that compares equivalent work and ties slowdowns to observed phases. Choose the smallest benchmark that includes the performance question and validate its numerical result before timing it. ## Key Points - [MPI Forum MPI 4.1 standard and timing semantics](https://www.mpi-forum.org/docs/mpi-4.1/mpi41-report.pdf) - [Open MPI MPI_Wtime documentation](https://docs.open-mpi.org/en/main/man-openmpi/man3/MPI_Wtime.3.html) - [OpenMP thread affinity](https://www.openmp.org/spec-html/5.1/openmpse61.html) - [Slurm multicore and multithread support](https://slurm.schedmd.com/mc_support.html) - [Slurm MPI launch guide](https://slurm.schedmd.com/mpi_guide.html)
skilldb get performance-optimization-skills/hpc-performance-measurementFull skill: 39 linesHPC Performance Measurement
Produce a repeatable scaling report that compares equivalent work and ties slowdowns to observed phases. Choose the smallest benchmark that includes the performance question and validate its numerical result before timing it.
Define the experiment
State the kernel or end-to-end job, dataset shape, precision, solver tolerances, output mode and success checks. Use a fixed total problem for strong scaling. Use a fixed per-worker problem for weak scaling and state what grows, including boundary or exchange work. Pick the baseline and comparison counts before collecting times. If memory prevents the requested one-worker run, use a documented smaller parallel baseline and report the actual baseline size.
Keep software and input revisions, node type, compiler and MPI/OpenMP/runtime options constant for comparisons. Change one scheduling or tuning decision at a time. Record MPI ranks per node, threads per rank, hardware threads, socket/NUMA placement, accelerator ownership and requested versus observed affinity. Check for hidden thread pools in BLAS or other libraries. Confirm the intended process placement through supported runtime reports instead of assuming a scheduler option was honored. Use the site-approved launcher, scheduler and allocation.
Time the correct quantities
Separate queue wait, application setup, warm-up, compute, data exchange, checkpoint and final output as needed. Capture both end-to-end application time and the phase that motivated the change. Decide whether filesystem or application caches should be cold or warm and repeat that condition consistently.
For an MPI phase, calculate elapsed time on each rank with local start and stop readings from MPI_Wtime. Use the maximum rank elapsed time for a coordinated phase whose completion depends on the slowest rank, and keep the distribution to diagnose imbalance. Cross-rank timestamp subtraction is invalid without a verified synchronized-clock contract. If a barrier is needed to define a benchmark start, state where it is and whether its cost is included. Do not add barriers to the measured algorithm merely to make timing easier.
Repeat each point and report the median and spread alongside all recorded samples. Track failed or numerically invalid trials explicitly. Strong-scaling speedup relative to baseline rank count p0 is T(p0)/T(p); efficiency is that ratio divided by p/p0 when ranks are the scaling unit. If both thread and rank counts change, use the declared resource count consistently. For example, a 4-rank baseline at 12 seconds and a 16-rank point at 5 seconds yield 2.4× speedup and 60% efficiency, assuming equal threads per rank. For weak scaling with equal work per worker, compare elapsed time relative to baseline and state any changing communication workload.
Isolate a bottleneck
Compare elapsed phase times and rank variance before explaining a plateau. A split between intra-node and inter-node runs can separate topology effects. Compare compute-only versus full I/O as an explicit alternate measurement. Use only supported counters or profilers and identify instrumentation overhead; elapsed time, CPU cycles, bandwidth and allocated memory answer different questions. If shared-resource contention changes, mark the run and repeat under comparable conditions when possible.
Publish the raw samples, exact launch and affinity evidence, phase definition, correctness checks, comparison table and limits. Include a control with an intentionally different workload or invalid result to verify the report rejects unlike runs. If no cluster is available, a local fixture can test calculation and reporting; it cannot establish multi-node scaling or cluster placement.
References and rights
- MPI Forum MPI 4.1 standard and timing semantics
- Open MPI MPI_Wtime documentation
- OpenMP thread affinity
- Slurm multicore and multithread support
- Slurm MPI launch guide
Original SkillDB instructions under LicenseRef-SkillDB-Content. References were checked 2026-11-13 and used for technical verification.
Install this skill directly: skilldb add performance-optimization-skills
Related Skills
Image Optimization
Optimize image delivery with modern formats, responsive sizing, lazy loading, and CDN-based transformations.
Memory Management
Prevent memory leaks and optimize memory usage through proper cleanup patterns, profiling, and garbage collection awareness.
Server Side Rendering
Improve time-to-first-byte and perceived performance with server-side rendering, static generation, and streaming patterns.
Application Binary and Asset Size Audit
Compare application download and installed sizes across target variants, identify large binaries or duplicated assets, and validate runtime behavior after optimization.
Bundle Optimization
Reduce JavaScript bundle size through tree shaking, dependency analysis, and build configuration tuning.
Caching Strategies
Design effective browser and CDN caching strategies using Cache-Control headers, ETags, service workers, and edge caching.