Most write-ups of a subsurface-AI project start at the model. This one starts at a constraint the model never sees: the client's wells arrive slowly, one or two at a time, under an NDA, and there are never as many as you want. In a roughly twenty-month engagement with a major operator in Oman, that constraint set the whole plan. If protected wells are the scarcest input you have, then the question is not "what model do we train" but "what do we build with the wells we do not yet have." The answer we settled on was to treat the pipeline itself as the thing to rehearse, on free public data, so that every scarce log went to the one place a free log cannot substitute.
The reasoning is simple once you separate the pipeline from the model. A pipeline is a sequence of stages: read the raw log format, clean and impute the array, find and fit the structural features, correlate them well to well, pretrain a representation, then train and validate the model that ships. Only the last stage is about this operator's reservoir. Every stage before it is generic engineering that a completely different well can exercise just as well. Public wells are wrong in geology and right in shape, and shape is what a pipeline is made of.
The DLIS reader is generic; the reservoir is not
The clearest example we have of this is the earliest one. A newly received image log came in, and within days the ingestion notebook was pointed straight at it. The reader did a plain, unglamorous job: dlisio opened the file, a channel walk enumerated the 19 channels in the frame (depth, the static image, the four pads and four flaps, calipers, gamma, azimuth), and it pulled 697,502 depth samples out of a single frame. The missing-sample sentinel of -9999 was mapped to NaN, the whitespace-gap columns were dropped, and the image was inverted for thresholding.
Nothing in that paragraph is specific to Oman, or to carbonates, or to this operator. A DLIS reader that walks channels, converts a sentinel to NaN, and hands you a clean array is the same code whether the file came from a client vault or a national archive. Which means it never needed to be written against a protected well at all. If you build it on an open well first, the day a client log finally arrives the reader already works, and the well goes straight to the question only it can answer.
Where the classical methods honestly break
The same run kept going into the part of the pipeline everyone wants to skip past: the classical computer vision that finds sinusoids before any learned model exists. Five cv2 threshold variants were compared side by side at a cutoff of 245. Then medianBlur(5), then Canny(100,255), then a probabilistic Hough transform to pull straight-line evidence out of the edge map. On a clean section this produces candidate lines you can fit into dip and azimuth. On the section in this run it produced nothing, and the notebook ended in an unedited TypeError (a NoneType with no len()) because Hough returned zero lines and the next line of code asked how many there were.
We kept that crash in the record on purpose. It is the cheapest lesson in the whole project: classical line-fitting breaks on real, complex rock, and it breaks silently until something downstream trips over the empty result. Learning that costs you nothing when the well underneath it is a free public log. Learning it on a scarce NDA-bound well is the same lesson at a much worse price, because that well could have been spent on the model instead. The honest failure is exactly the kind of thing you want to happen on the rehearsal track.
The fitting form the classical stage is reaching for is the standard one, a feature described by
where the amplitude carries dip, the phase carries azimuth, and the offset is depth. That geometry is the same in the North Sea as it is in an Omani carbonate. The mechanics of picking it are covered in Path-Opening Across 39 Thresholds; the point here is only that all of it can be built and stress-tested on open wells.
Two open pillars, and what each is good for
The open data came from two places. The first is FORCE 2020, the Norwegian Continental Shelf competition set of 118 public wells, which we used to rehearse well-to-well correlation end to end before touching a client log (Well-to-Well at Field Scale). The second is a national public archive, harvested with a small resumable crawler, which gave us practice at the raw, uneven front of the pipeline that a tidy competition set hides (Crawling a National Well Archive). Rather than re-derive either, it is worth being precise about what an open archive is actually good for, and where it stops.
It is good for three things. It is a place to rehearse the pipeline, so the reader, the imputation, and the classical methods are already exercised before the first protected well. It is a pool of pretraining candidates, real logs with real geological structure a representation can learn from without an NDA. And it is a source of negative controls: a public well from the wrong basin should not correlate with the target field, and a method that finds structure there anyway is a method that is fooling itself.
Where it falls short is precisely the geology. The tools differ, and a Norwegian wireline suite is not the imaging tool a carbonate operator ran. The formations differ; clastic North Sea stratigraphy has nothing to say about a dolomitised carbonate. And the label regimes differ, because every archive was annotated for its own purpose, not for yours. So open data can carry the pipeline and warm up a representation, but it cannot supply the target. The manifold the source data teaches has to be aligned onto the client's wells, and that alignment is the job the scarce logs are for.
Spending the scarce wells last
The map above is the whole argument in one picture. Each pipeline stage sits on one of two tracks, and you get to choose where the frontier between them falls. Slide it all the way left and you have burned protected wells writing a DLIS reader and debugging a Hough crash, work a free well would have absorbed. Slide it right and every rehearsable stage was built on open data, leaving only the orange node on the NDA track: the shipped model, trained and validated on the operator's own reservoir. That node stays orange wherever the frontier sits, because it is the one stage no public well can stand in for.
None of this makes the client's wells optional. The model that mattered was trained and validated on the operator's confidential logs, because that carbonate reservoir is what the operator was paying us to understand, and the accuracy anyone acted on came from those wells and not the public ones. What the sequencing changed was the order of spending. By the time a protected well arrived, the reader worked, the classical methods had already failed in the ways they were going to fail, and the correlation machinery had run on genuine stratigraphy. The scarce, slow, expensive data was spent last, on the question only it could answer.
For a team starting a similar engagement, the practical rule is short. Before you ask for a single NDA-bound log, get the pipeline running end to end on open wells: pick a national archive for the messy front, a competition set like FORCE 2020 for the clean middle, and let the classical methods crash on public rock where a crash is free. Treat the client's wells as what they are, the one input you cannot rehearse, and spend them only on the model.
Limitations
This is a sequencing argument, not a transfer-learning result. Open data rehearses the pipeline and warms a representation; it does not close the domain gap, and every claim about accuracy in the engagement rests on the client's confidential wells, not the public ones. The DLIS-run figures are from one early ingestion notebook on one well and are illustrative of the reader's shape rather than a benchmark. The rehearsal frontier in the instrument is a lever for reasoning about where stages should be built, not a measured quantity. And the "wrong basin, right shape" premise has real limits: a public archive whose tools and formations are too far from the target may not even rehearse the front of the pipeline faithfully, in which case the free-data track buys less than it does here.
References
[1] Bormann, P., Aursand, P., Dilib, F., Manral, S., and Dischington, P. FORCE 2020 Well log and lithofacies dataset for machine learning competition. FORCE / GitHub (2020). https://github.com/bolgebrygg/Force-2020-Machine-Learning-competition
[2] NLOG, the Dutch Oil and Gas Portal (national subsurface data repository), TNO Geological Survey of the Netherlands. Public well and log records via the mapviewer REST API. https://www.nlog.nl/
[3] dlisio: a Python library for reading Digital Log Interchange Standard (DLIS) files. Equinor open-source release. https://github.com/equinor/dlisio
