Performance

The 0.3.0 release makes the severe-weather diagnostic pipeline about 3.3× faster on large grids. The numbers below come from a measured audit on one real high-resolution WRF file — methodology first, then the baseline that was measured, then what changed. This is a single-file pipeline benchmark on stated hardware, not a general benchmark suite.

1 Methodology

2 Measured baseline (pre-0.3.0)

The audited baseline for the full 79-product pass at 24 threads: 185.3 s wall, 2740 s user (1484% CPU), peak RSS 7.47 GB. Two structural costs dominated:

• Effective-inflow-layer (EIL) products: 89.2 s (48% of wall). The EIL scan runs up to nz full moist ascents per column and was recomputed — twice inside some products — for every consumer (stp_effective, scp, vtp_mod, effective_srh, ebwd, effective_inflow, effective_cape): about ten full-grid scans at roughly 8.8 s each.
• Depth-limited CAPE products: 39.1 s (21%). The top_m CAPE path ran single-threaded, so each of the six 0–3 km / 0–6 km CAPE variants cost 6–7 s while the unlimited path was grid-parallel.

Thread scaling (baseline, same 79-product set)

4× the threads bought 2.46× at baseline because roughly 45 s of the pass was single-threaded product code (the six depth-limited CAPEs plus el); the parallel remainder scaled at about 2.9× for 4× threads, limited by memory bandwidth and serial read sections. Peak memory is thread-independent (it is the per-timestep intermediate cache).

Cold vs warm (baseline, fresh file handle per product)

With a fresh WrfFile per product, first-call cost is dominated by dependency loading — reading and deriving the 3-D thermodynamic stack costs 6–8 s once, after which most products are sub-second. Selected pairs (cold / warm, seconds): slp 7.10 / 0.016, sbcape 7.79 / 0.001, srh1 7.83 / 0.76, maxdbz 7.42 / 2.56, stp_effective 27.25 / 17.83, ncape 19.20 / 0.002.

3 What changed in 0.3.0

• EIL caching. The effective-inflow-layer scan result is cached per time step and reused by every consumer, collapsing about ten full-grid scans into one and removing the double scan inside individual products.
• Parallelized depth-limited CAPE and EL. The top_m CAPE path and the equilibrium-level grid now use the same column-parallel scheme as the unlimited CAPE path.
• Composite reuse. Severe composites route through the cached CAPE stacks instead of private recomputation, and DCP reuses the cached DCAPE plane.
• Parallel HDF5 chunk decompression. Independent deflate+shuffle chunks decompress in parallel, accelerating cold 3-D reads.
• Reduced per-column allocations in the CAPE/parcel core.

4 Result and honest scope

On the pipeline class above, the audit's identified savings and the release verification put the same 79-product pass in the ≈57 s class at 24 threads, against the measured 185.3 s baseline — the ≈3.3× figure in the 0.3.0 release notes. No science changes were mixed into the optimization work; results remain gated by the parity harness.

Scope statements, so the numbers are not over-read:

• One real file, one grid size, one hardware configuration, warm OS page cache. Different grids, storage, and thread counts will scale differently.
• The workload is the 79-product non-ECAPE severe set; the ECAPE-heavy group is a separate, intrinsically more expensive pass.
• Per-product timings in section 2 are baseline (pre-fix) measurements, published to document where the time went and to make the optimization targets falsifiable.
• Timings cover getvar computation from an open file handle; they exclude Python interpreter startup, plotting, and any downstream storage.