Skip to main content
Reading viewAll insights →
BLOG5 min read

The HPC-Broker Pattern: Call the Expensive Simulator Only When You Must

The HPC-Broker Pattern: Call the Expensive Simulator Only When You Must
Tannistha Maitiby Tannistha MaitiSenior AI Researcher · 30 Jul 2026
Share

A single high-fidelity waveform simulation can burn core-days of compute, and a model search calls for thousands of them. The fix is not a faster simulator but a broker that mostly avoids calling it. Put a neural-operator surrogate in front, let it report its own uncertainty, and escalate to the full finite-difference or ABAQUS solve only where that uncertainty is high. The saving is large and it is not linear in the threshold, which is the part teams get wrong.

A single high-fidelity waveform simulation, a finite-difference run that propagates seismic waves through a crustal model or an ABAQUS solve of how that same rock deforms, can burn core-days of compute. Do that for every candidate model in a search and both your cloud bill and your turnaround explode. The fix is not a faster simulator. It is a smarter dispatcher that mostly avoids calling the simulator at all.

Precision is expensive, and usually unnecessary

Picture sweeping thousands of candidate Earth structures to find the one whose predicted seismograms match what stations actually recorded. In that search you rarely need full-physics precision for every candidate. Most are routine, and a much cheaper estimate would predict the receiver function, the processed seismogram that reveals boundaries like the Moho, well inside tolerance. Paying full price for a precise waveform you did not need is the most common way compute budgets evaporate.

A surrogate in front, a broker in the middle

The pattern has three parts. A surrogate, here a neural operator trained on past finite-difference and ABAQUS runs to map an Earth structure straight to its predicted receiver function, answers instantly. A broker inspects each candidate and decides where it goes. The expensive simulator sits behind the broker, reserved for cases that genuinely need it.

The ingredient that makes it work is that the surrogate does not only return a predicted waveform. It returns a confidence. When the surrogate is confident the broker ships its receiver function; when it is not, the broker escalates.

ES-3504 · CHEAP FIRST, ESCALATE ON UNCERTAINTY62%compute savedCandidates80 structures to testFD / ABAQUSexact · 50 unitsNeural operatorapproximate · 1 unitSURROGATE UNCERTAINTY, PER CANDIDATEthreshold 3551 of 80 answered by the surrogate · 29 escalated · average cost 18.8 units against 50Drag to set the escalation threshold. Most of the saving arrives early, because most candidates are ones the surrogate has seen before.
Routing candidate structures by the surrogate’s own uncertainty. Cost ratio 1:50 is the article’s worked example; the uncertainty population is seeded and fixed so the threshold moves against unchanging data.

The uncertainty estimate is the whole game

Without a reliable confidence signal the broker is guessing, and a guessing broker is worse than no broker: it will return wrong cheap receiver functions with conviction, and you will invert them into a wrong crustal structure. So the engineering effort concentrates on calibrated uncertainty. The surrogate must recognise when a candidate structure is unlike anything it was trained on and flag its prediction as unreliable. Ensembles, where disagreement between networks marks the hard cases, Bayesian estimates, and conformal prediction [1] all do this.

The threshold you set on that uncertainty is the one knob trading cost against risk. Set it high and almost everything routes to the surrogate: cheap, but more borderline structures pass on an approximate waveform. Set it low and you escalate liberally: accurate, and you are paying HPC prices again.

The economics, and the knee in the curve

The arithmetic is what makes the pattern worth building. If the surrogate costs one unit and a full solve costs fifty, and the surrogate confidently handles 80 percent of candidates, average cost per candidate falls from 50 units to about 11, a saving near 78 percent, while the remaining fifth still receive the true simulation.

The saving is not linear in the threshold, and that is the part worth internalising. Real searches are skewed toward structures the surrogate has already seen, so the population of uncertainties piles up low. The first few points of threshold therefore buy most of the saving, and the curve then flattens. Past that knee you are accepting materially more approximate waveforms in exchange for very little money, which is a bad trade made by teams who assume the relationship is straight.

50 : 1

Cost ratio, exact to surrogate

~78%

Saving at 80% surrogate share

~11 units

Average cost per candidate

1

Knob that trades cost for risk

Where teams get it wrong

Two failure modes recur. The first is skipping the confidence estimate and routing by crude heuristics, which leaks errors on exactly the structures the surrogate has never seen, the ones most likely to matter. The second is setting one static threshold and forgetting it. The right threshold depends on the cost of a mistake, and that varies by stage: a broad first-pass sweep tolerates a looser threshold than a final published inversion. The broker should let you tune it per stage, and it should log every escalation so you can audit whether the surrogate is drifting as the search wanders into unfamiliar structures.

Zooming out

The pattern is not specific to geophysics. It is how to think about any workflow mixing a cheap approximate tool with an expensive exact one, which now includes most systems built around large models. Call the surrogate first, measure its confidence, escalate only on the hard cases. Cheap-first, escalate-on-uncertainty turns an unaffordable full-physics search into an affordable mostly-surrogate one, and it puts a dial in your hand for exactly how much accuracy you are willing to pay HPC for.

Key takeaways

  1. Full-physics precision is expensive and usually unnecessary: running the exact solver on every candidate spends most of the budget on answers a surrogate would have got right.
  2. Put a neural-operator surrogate in front and a broker in the middle, predicting cheaply by default and escalating only the hard structures.
  3. Calibrated uncertainty is the linchpin. A broker that cannot trust its confidence signal will invert wrong cheap waveforms into wrong crust.
  4. One threshold trades cost against risk, and it should be tuned per stage: looser for a broad sweep, tighter for a final inversion.
  5. The saving is not linear in the threshold. Most of it arrives early, so pushing past the knee buys little money and costs real accuracy.

References

[1] Angelopoulos, A. N. and Bates, S. A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification. arXiv:2107.07511.

Tannistha Maiti
Tannistha Maiti

Senior AI Researcher

More from EarthScan

Related research

All insights →
A Neural Operator for the Mantle-Flow Stokes Boundary-Value Problem
Research

A Neural Operator for the Mantle-Flow Stokes Boundary-Value Problem

The Dipping-Interface Trap: When a Flat H-κ Reads the Wrong Crust
Insight

The Dipping-Interface Trap: When a Flat H-κ Reads the Wrong Crust

From one craton to a production Moho-picker: the funnel nobody warns you about
Insight

From one craton to a production Moho-picker: the funnel nobody warns you about

Stay ahead

EarthScan insights, in your inbox.

Field-tested research on subsurface and energy-transition AI. About twice a month. No noise.

We use your email only for this newsletter. Unsubscribe anytime Privacy.