False Expertise Has a Geometry: Why the Output That Breaks Is the One Nobody Can Check

On 27 August 2026, at the SPE Artificial Lift Conference and Exhibition, Kreg Flowers, a production engineer at Occidental, told a panel that the foundational skills have not changed and warned that AI can create false expertise. Maggy Burns, an operations supervisor for produced water at ExxonMobil, stressed input data quality and domain knowledge and argued for retaining human oversight on safety-critical decisions. Araceli Rivera Mandujano of SLB warned about information overload and the difficulty of spotting software errors [1].

Two weeks later, on 10 September 2026, the Journal of Petroleum Technology reported a second panel, this one at the SPE Subsea Well Intervention Symposium in Galveston, Texas. Sean McCall, Chief Data Officer at Oceaneering, and Brian Saucier, President of DeepMar Consulting, argued alongside Dana Otillio of SPE that access to modern AI does not guarantee successful deployment, that data quality has to be assessed and improved before AI is deployed against it, that humans must remain in command rather than serve as observers on safety-critical decisions, and that pilots against historical low-risk projects are the right way to validate a tool before operational deployment [2].

Two things about those two paragraphs need saying before anything is built on them. Both accounts are pointer sources. They are the Society of Petroleum Engineers' own journal reporting what people said on a stage, which is better sourced than trade press and is still not an operator-published account of anything. The first panel's quotations are operator employees speaking in their own voice, so Occidental and ExxonMobil are on the record through their people. The second panel's quoted speakers are a service company's Chief Data Officer and a consultant. No operator engineer is quoted there at all, and Aramco Americas appears in that piece by reference only.

The second thing is the one this article is about. Neither account carries a single quantity. No well counts, no percentages, no uplift, no missed detections, no review budgets. Two SPE venues, 2 weeks apart, six named companies, and not one number. That is not a sourcing weakness to apologise for. It is the finding. The industry's own people are saying, in public, that the constraint is trust rather than access, and the thing they are least able to put a number on is the one they keep asking for: human oversight.

We can put numbers on it, because we have measured what happens when a human is asked to check an output that the machine got wrong.

What the sources say, and what they do not

The SLB newsroom published a question and answer with Paul Sims, president of Production Systems, on 8 September 2026. It carries three figures that set the denominator for every mature-asset claim in this market: approximately 70% of global production now comes from assets past peak, that share is expected to approach 80% by 2030, and the conventional reserve replenishment rate is put close to 30% [4]. Those are SLB's own figures, sourced on the page to an analysis the page does not name, and the page makes no AI claim at all. The question this piece puts to them is ours and not SLB's. We use them as a denominator only, never as a finding.

One operator-authored number does exist nearby, and it is worth one sentence purely as contrast: SPE 219553, written by ExxonMobil authors and presented in August 2024, records a closed-loop well by well gas lift workflow deployed to more than 1,300 ExxonMobil Permian wells for a production uplift of about 2.2% with no change to surface or downhole equipment, and the paper surfaced again through JPT on 4 September 2026 [5][6]. The article is recent; the work is one to two years old, and nobody automated artificial lift this week.

Set the pool side by side and the shape is plain. The single operator-authored figure in it is small, checkable, hard-won across more than 1,300 wells, and two years old. The claims that are loud and current carry no denominator at all. And the two panels that are actually about oversight, the ones an asset team would forward to a colleague, carry nothing a reader can test.

What our own record measures, and what it does not

Our measured material here is one thing: borehole-image fracture interpretation on a confidential carbonate engagement in Oman, 14 vertical wells and 5 horizontal wells. It is not artificial lift and it is not subsea intervention. What transfers to those workflows is an argument about the structure of oversight, not a measurement taken inside them, and we will say where the argument stops.

On a blind July 2023 run at a 0.55 probability threshold and the 6.0 cm depth operating point, our vertical-trained fracture detector scored F1 60.87%, with depth MAE 2.11 cm and dip MAE 11.12 degrees holding at workable levels, while azimuth MAE reached 65.4 degrees against roughly 10 degrees on vertical wells [7][8]. That azimuth figure is already in print. We published it in 2025 and the case study that followed it restates it. Take the four numbers as given here; this piece does not re-derive the split between them and does not re-run the argument about what a single blended score hides.

Two further figures from our own files do the load-bearing work below. The interpreter on that engagement agreed permissible-error tolerances with us output by output, and they were 2, 4 and 6 cm on depth, 1, 3 and 5 degrees on dip, and 5, 10 and 15 degrees on azimuth [7]. And at the native digital log resolution one pixel is about 3 cm, so a plus or minus 3 cm depth uncertainty is baked into the image before the model makes a single decision [9].

Why one output breaks and three do not

A planar fracture cutting a borehole traces a sinusoid across the unrolled pad image. Write it as a curve of depth against pad azimuth:

The three parameters are the three outputs. The offset $z_0$ is the fracture's depth. The amplitude $A$ is the dip, through $A = \tfrac{d}{2}\tan\beta$ with $d$ the hole diameter and $\beta$ the apparent dip of the fracture against the plane normal to the borehole, so the curve's peak to trough extent is $d\tan\beta$. The phase $\varphi_0$ is the azimuth: it is where on the circumference of the hole the fracture's low point sits, and that is the same thing as which way the fracture faces.

Now take the Fisher information of a sinusoid sampled at $N$ pad azimuths in noise of standard deviation $s$. For the offset it is $N/s^2$. For the amplitude it is $N/(2s^2)$. For the phase it is

That is the whole mechanism in one line. Offset and amplitude are estimated equally well whether the curve is steep or flat; phase is not. Phase error scales as one over the amplitude. And the amplitude is exactly what a deviated borehole destroys: as the borehole inclination approaches the dip of the fracture set, the apparent dip goes to zero and the sinusoid flattens into a straight line. A flat line has a depth and it has an amplitude of zero. It has no phase at all.

So azimuth is not the output that happened to be unlucky. It is the only one of the three whose precision is coupled to the exact geometric quantity the well trajectory changed. Depth held, dip held, detection held, and azimuth collapsed, for a reason that is written into the estimator before any training run begins.

The arithmetic of a reviewer who reads the same picture

Here is the step the panels are reaching for and cannot price. Suppose an interpreter reviews the model's picks. What does that review buy?

Write the model's azimuth error on one pick as a sum of two parts:

where $\varepsilon_I$ is the image-borne term, the part that the phase information in that picture forces on anyone who reads it, and $\varepsilon_M$ is the model's own idiosyncratic error. Now write the reviewer's error, with $u$ standing for how independent his evidence is of the model's:

At $u=0$ he is reading the same unrolled pad image the model read, which is exactly what a spot-check is, and he inherits $\varepsilon_I$ in full. At $u=1$ he is reading something the model did not use, a pre-registered blind interval scored against the interpreter's own picks or a separate azimuth measurement, and the image term is gone from his error. $\varepsilon_H$ is the interpreter's own hand to hand variability, which we set at 5 degrees, the tightest of the three azimuth bands he agreed with us.

A review catches a pick when the model is outside the agreed band and the reviewer is inside it, because only then does the reviewer have a standard to judge the model against. So the per-check catch probability is a rectangle probability on a bivariate normal:

Look at the covariance term and nothing else. At $u=0$ the reviewer's error and the model's error share $\varepsilon_I$ entirely, so as the image-borne uncertainty grows the two become the same variable and the catch probability goes to zero. Model error and reviewer detectability are driven by the same quantity, the phase information in the picture, so they are correlated rather than independent, and spot-checking therefore fails exactly where the model fails and fails silently.

One calibration makes this ours rather than generic. With the well deviating in the dip direction of the fracture set, the amplitude ratio against a vertical well is $\rho(\theta)=|\tan(\delta-\theta)|/\tan\delta$, so the ratio of the two azimuth errors we measured is $\tan^2\delta$ when the model's own term is set to zero, and the squared ratio the model actually inverts is $\tan^4\delta$. Inverting 65.4 against roughly 10 gives a fracture set dip of 68.6 degrees. That dip is not an observation. It is what our own two measured numbers imply under this model, and the instrument has a control that moves it.

The instrument

The floor is the plane of one pick's two errors: the model's azimuth error from left to right, the reviewer's from front to back, both in units of the model's own spread. The ground rises into the joint density of the pair, so the hill is where a pick is likely to land. The two amber arms are the only places an error is caught, the strips where the reviewer is inside the interpreter's band and the model is outside it, and the share of wrong picks a check catches is the volume of the hill over them. The white ridge is the hill's crest. The amber square on the floor is the 15 degree band over the model's spread, and the four steel squares are the same window for each of our four measured outputs. The column at the back is the catch: filled to the share of wrong azimuths this review catches, with a white ring at half and an amber stretch up to what a separate channel would catch with the same picks checked. The sample is 300 picks a well, a design choice for this exhibit so that the effort control runs from checking none of the well to checking all of it, and it has nothing whatever to do with the 300 interpolated depth points of our raster-log validation framework, which grades a different product on a different dataset.

Two controls reshape the hill and one only moves the column. Evidence independence is the one the argument is about: it swings the ridge from the diagonal, where the reviewer's error is the model's, round onto the arms. The inclination control does not move the ridge at all. It shrinks the amber square, because the model's spread grows as the borehole turns and the band does not. The picks control moves only the column. The fourth, the model's own share of its azimuth error, is there to be argued with: the more of that error belongs to the model rather than to the image, the more a reviewer reading the same image can see, and the weaker our case gets.

The provenance rule is printed on the plate. The four operating points, the tolerance bands and the pixel floor are measured and file-cited. The hill and the column are our model of reviewer behaviour and are labelled as ours.

What the instrument shows

It opens on our own horizontal wells: inclination 90 degrees, all 300 picks checked, evidence independence 0, and the hero reads 2.2%. A reviewer who re-reads every pick in the well, on the same unrolled pad image the model read, catches 2.2% of the azimuths that are outside the band the interpreter signed. Drag the inclination to 0 and the same campaign reads 19.7%. The whole difference is geometric: nothing about the reviewer changed.

Read the correlation readout while you do it. On a vertical well the correlation between the model's azimuth error and the reviewer's is 0.929. On a horizontal well it is 0.998. That is the number the panels are circling without a way to name it. At 0.998 the reviewer is not a second opinion; he is a copy.

Now drag the picks control and watch the column. At the 68.6 degree floor, where the modelled sinusoid crosses zero amplitude, going from checking none of the well to checking all 300 picks moves the catch probability from 0.0% to 0.9%. Across the same axis at vertical it moves from 0.0% to 19.7%. The return is not literally zero, but a 0.9 point return on 300 careful human checks is what "more eyes on the same picture" buys once the phase is gone. Put the same result the other way round: 33.7 picks on a vertical well buy what all 300 buy on a horizontal one.

The four steel squares say it in one frame. Each is the interpreter's band over that output's own measured spread. Depth, at MAE 2.11 cm against a 6 cm band, is a window that stands around the whole hill. Detection, at F1 60.87%, a little less than that. Dip, at MAE 11.12 degrees against a 5 degree band, is a small square at the foot of the hill. Azimuth, at MAE 65.4 degrees against a 15 degree band, is a pinhole in the middle of a hill that stretches to the edge of the plate, and it is the same square as the amber one the instrument opens on, because the opening settings are our horizontal wells. Three outputs whose window holds most of their error, one whose window holds almost none. That is false expertise with a shape.

The white ring on the column is the blunt part. At evidence independence 0 the fill never reaches it: there is no inclination and no review budget at which re-reading the same picture catches half the out-of-band azimuths. Push the independence control and the ridge swings onto the arms and the fill climbs. At 90 degrees and full coverage it takes 78% independence to catch half of them, and 54.8% merely to match what a vertical well gives you for free. On a vertical well the same target needs 28%.

Then push the fourth control against us. Give the model an idiosyncratic error worth half its vertical-well azimuth error in quadrature, which is the most generous reading of a model that is not purely phase-limited, and the horizontal-well campaign improves from 2.2% to 3.6%. The vertical-well campaign improves from 19.7% to 46.9%, because there the reviewer had evidence to spare. The gap the argument rests on survives the control designed to close it.

Two readouts are worth naming because they connect the model to a judgement a human already made. The instrument puts the sinusoid at 9.14 pixels peak to trough on a vertical well and 1.40 pixels on a horizontal one, and it marks the band from 53.0 to 84.3 degrees where the curve is thinner than a single pixel of the log. And it prints the azimuth tolerance the horizontal model would need in order to sit outside its band no more often than it does on a vertical well: 98.1 degrees, which is 6.54 times the 15 degrees the interpreter would sign. Our own case study, written before any of this arithmetic existed, recorded that the horizontal model reached usable accuracy only when the azimuth tolerance was relaxed toward 90 degrees, roughly 6 times looser than the band the interpreter would accept [7]. The interpreter's judgement and the model's phase arithmetic land in the same place.

The symmetric failure, so this is not a one-sided story

A reviewer auditing the wrong artefact can be falsely alarmed as easily as falsely reassured, and we have that measured too. Our raster-log digitiser VeerNet, graded against ground truth at 300 interpolated depth points on a common resampled axis, reaches a peak coefficient of determination of 0.9891, a lowest mean absolute error of 0.0132 and a lowest mean squared error of 0.0004, while the same checkpoints score a peak mask IoU of only 0.51 [10]. Read as a mask the model looks unfinished. Read as the curve a petrophysicist actually consumes, it clears the bar. That file also names a second quantisation floor worth putting beside the 3 cm pixel: an integer per-column argmax is correct only to the nearest pixel, so it caps the recovered curve at half a pixel of error no matter how clean the band is [10].

Both cases are the same defect. The channel the reviewer is looking at is not the channel the decision depends on. In one it hides a catastrophic error behind an agreeable picture. In the other it condemns a good curve on an irrelevant score.

Where this model is weakest

The surface is our model of reviewer behaviour, and it has four soft joints.

The conversion from a mean absolute error to a standard deviation assumes a zero-mean Gaussian error, which is how the 10 degree and 65.4 degree figures become 12.53 and 81.97. Azimuth errors on a flattened sinusoid are not Gaussian; they pile up near the wrap-around, which is why the model caps the image-borne uncertainty at 180 degrees rather than letting it diverge. That cap is a floor under the argument, not an embellishment of it.

The fracture set dip of 68.6 degrees is derived from two of our own numbers, one of which the source file describes as "roughly 10". A vertical-well azimuth error of 8 degrees instead of 10 moves the implied dip and the size of the amber square at every inclination with it. The shape of the hill does not change; the inclination at which the square collapses does.

The amplitude readout in pixels is anchored by treating the pad image's depth noise as one pixel and the phase sensitivity as twice that noise over the amplitude. A real estimator pools many azimuthal samples, so the true amplitudes are larger than the readout says by roughly the square root of the sample count. Read those two figures as a ratio and an ordering rather than as a caliper measurement.

And at full evidence independence the catch is flat in inclination, because the model gives the independent channel a fixed 5 degrees of error regardless of geometry. That is right for a separate azimuth measurement. It is not right for an interpreter re-picking the same image on a blind interval, which shares the image term and therefore sits somewhere in the middle of the control rather than at its top. A blind interval is a real control against overfitting and a partial control against this.

One more boundary, stated plainly because the panels were not about fractures. Our measured record is 19 wells in one carbonate field, on one task, with one interpreter's tolerance bands. What we are exporting to artificial lift and subsea intervention is not the 65.4 degrees. It is the question of whether the reviewer's evidence and the model's evidence are the same evidence, and that question has an answer in every workflow.

Four questions worth asking before a review protocol is signed

Which outputs can a reviewer check with evidence the model did not use? Not which outputs are reviewed. Which ones have an independent channel behind the review. On our engagement depth had one, because the depth track is not the sinusoid's phase. Azimuth on horizontal wells had none, and no review budget was going to produce one.

What is the correlation between the model's error and the reviewer's? It is estimable on any deployment that has ever run a blind interval, and it is the number that decides whether a review campaign is oversight or theatre. Ours moves from 0.929 to 0.998 across a change in well geometry that nobody would list as a model risk.

What does the tool do with the output whose review is uninformative? The two honest answers are to buy an independent channel or to route the output to a person and stop asserting it. On the Oman engagement we and the operator's interpreter took azimuth on horizontal wells out of the automated path rather than granting it a looser bar [7]. A tolerance the expert would not sign off on is not a tolerance you deploy behind.

And what does the surveillance layer declare, per output, about which of its answers a human can independently check? That is the question our ES360 work is built around, because an asset-wide production layer is judged on exactly the two complaints these panels made: alert quality and the cost of a false positive on one side, data-quality readiness and the cost of unsupervised alerting on the other. Neither is answerable with a single confidence number.

What this argues for is not a download. It is a per-output tolerance protocol agreed with your own expert interpreter plus a pre-registered blind interval, which is the thing we built in Oman and which only exists as a conversation. If that is the protocol you need, book a meeting and bring the outputs you are currently spot-checking.

The Occidental engineer was right that AI can create false expertise, and the Galveston panel was right that humans have to be in command rather than observing. Neither statement is a remark about engineers' psychology. On a borehole image it is a measurable claim about correlated evidence channels, and four numbers price it. More eyes on the same picture is not oversight. Oversight needs a different picture.

References

[1] Journal of Petroleum Technology. New Tools, Old Lessons: Experts Stress the Importance of Artificial Lift Basics. Published 8 September 2026, reporting a panel held on 27 August 2026 at the SPE Artificial Lift Conference and Exhibition. A pointer source: SPE's journal reporting operator employees speaking on a panel, not an operator-published account. The article carries no quantities. https://jpt.spe.org/new-tools-old-lessons-experts-stress-the-importance-of-artificial-lift-basics

[2] Journal of Petroleum Technology. Industry Executives Offer Practical Advice on How To Use Modern AI Models Wisely. 10 September 2026, reporting a panel at the SPE Subsea Well Intervention Symposium, Galveston, Texas. A pointer source; the quoted speakers are Oceaneering's Chief Data Officer and the president of DeepMar Consulting, with Aramco Americas named by reference only, and no operator engineer is quoted. The article carries no quantities. https://jpt.spe.org/industry-executives-offer-practical-advice-on-how-to-use-modern-ai-models-wisely

[3] EarthScan. ES360, the asset-wide production surveillance layer. Cited for the per-output declaration this piece argues a surveillance layer owes its users. https://www.earthscan.io/v2/platform/es360

[4] SLB. Q and A with Paul Sims, president of Production Systems. 8 September 2026. The three figures quoted here, approximately 70% now, approaching 80% by 2030 and a replenishment rate close to 30%, are SLB's own and are sourced on the page to an analysis the page does not name. The page makes no AI claim. https://www.slb.com/newsroom/features/2026/2026-0908-paul-sims-qa

[5] SPE 219553. Gas Lift Optimization in the Permian Using Machine Learning and Artificial Intelligence. ExxonMobil authors, SPE Artificial Lift Conference and Exhibition Americas, The Woodlands, Texas, August 2024. Operator-authored, and the work is one to two years older than the coverage that resurfaced it. https://onepetro.org/SPEALCE/proceedings-abstract/24ALCE/24ALCE/550379

[6] Journal of Petroleum Technology. Maximizing Every Barrel: Artificial Lift Embraces Automation. 4 September 2026. Cited only as the pointer that resurfaced SPE 219553; a pointer source, not an operator account. https://jpt.spe.org/maximizing-every-barrel-artificial-lift-embraces-automation

[7] EarthScan. The Generalization Cliff: Turning a Broken Azimuth Metric Into a Deployment Protocol. Source file content/case-studies/the-generalization-cliff-what-horizontal-wells-did-to-our-fracture-model.mdx. The blind July 2023 run, the agreed per-output tolerance bands and the relaxation toward 90 degrees are quoted from this file. https://www.earthscan.io/case-studies/the-generalization-cliff-what-horizontal-wells-did-to-our-fracture-model

[8] EarthScan. From 82% on Paper to 63% Blind: An Honest Accounting of the Vertical-to-Horizontal Generalization Gap. 22 July 2025. The azimuth figure of 65.4 degrees against roughly 10 degrees on vertical wells was published here first and is restated rather than re-measured in this piece. https://www.earthscan.io/insights/from-82-on-paper-to-63-blind-vertical-to-horizontal-generalization-gap

[9] EarthScan. Proving It Generalises: A Blind Zone the Model Had Never Seen. Source file content/case-studies/blind-zone-validation-repeatability-unseen-well-milestone.mdx. The plus or minus 3 cm pixel floor at native digital log resolution is quoted from this file. https://www.earthscan.io/case-studies/blind-zone-validation-repeatability-unseen-well-milestone

[10] EarthScan. Sub-Pixel Centrelines and Depth Registration: A Validation Framework for Digitised Curves. Source file content/whitepapers/sub-pixel-centreline-depth-registration-validation-framework.mdx. The peak coefficient of determination of 0.9891, lowest MAE of 0.0132, lowest MSE of 0.0004, peak mask IoU of 0.51, the 300 interpolated depth points and the half-pixel argmax cap are quoted from this file. https://www.earthscan.io/whitepapers/sub-pixel-centreline-depth-registration-validation-framework