The Messy Middle of Phase 2: From Data-Quality Chaos to a Working Supervised Model

Most case studies about applied machine learning tell you the start and the end. A problem, then a model, then the accuracy numbers. The middle - the stretch where the data fought back and the first three ideas did not work - gets compressed into a sentence about "iteration." For our carbonate image-log work with a major operator in Oman, the middle was Phase 2, and it was the hardest part of the whole engagement. The running log for that phase has 288 dated blocks between February and December 2022. Read in order, they are not a smooth curve of progress. They are a descent into a data-quality trough, a floor where we stopped and admitted the approach was wrong, then a climb back out to a first supervised model that actually worked. This is that middle, kept honest.

What "clean data" turned out to mean

Phase 2 opened well. Ten wells arrived with their digital log containers, apparent dip and azimuth, well radius, and interpreted PDFs. We had agreed on two clustering techniques and drawn up a plan. Then we started reading the pixels.

One well's static image channel came in with values in a 0 to 15 range where a healthy microresistivity log should span roughly 0 to 255. That is not a scaling quirk you normalise away; it is a corrupted channel that carries almost no dynamic range, and a model trained on it learns noise. On another well the raw dip picks were indexed to depths that did not line up with the depth index inside the digital log container itself - the picks said one thing, the container said another, and across one stretch the image log was simply absent from the container over an interval the picks claimed to cover. And reaching back to the first delivery, Phase 1's opening five wells had already been rejected outright because they arrived with no apparent dip or azimuth at all, the exact fields a geometry-aware detector needs as ground truth.

None of these are exotic. Every one is the sort of defect a human catches by eye and a pipeline swallows silently. The lesson we carried out of it was blunt: on borehole image logs, "clean the data" is not a preprocessing footnote, it is a substantial fraction of the project, and it has to be enforced by automated checks rather than trust.

The floor: three unsupervised ideas, all dropped

With the inputs partly salvaged, we spent the middle of Phase 2 trying to detect sinusoids - the signature a planar fracture or bedding surface leaves on an unwrapped image log - without labels. This is the descent in the arc below, and it kept going down.

We ran a NaN-removal bake-off to make the inter-pad gaps continuous first, comparing 1D interpolation, KNN, and a GAN fill that broke feature continuity and was dropped; that comparison is its own story, covered in our note on why 1D interpolation lost to KNN at well scale. On the clean images we tried classical picking - Hough transform with Canny edges and thresholding - then unsupervised clustering: DBSCAN across more than ten thousand parameter combinations, and hierarchical clustering that ran out of memory even on a 50 GB machine. When those stalled we tried self-supervised segmentation with SCOPS at five, three, and two classes. It was unstable on this data and we dropped it. Why the clustering families could not cleanly separate one sinusoid from the next - and why a supervised transformer later could - is the subject of a separate piece; the short version is that unsupervised methods have no notion of "this is one fracture," so a badly chosen parameter splits a single feature into fragments or fuses two into one.

The honest entry in the log is not a breakthrough. It is a decision. Somewhere in the August blocks the running log stops proposing another unsupervised patch and states the thing plainly: feature and fault detection here needs supervised learning. That sentence is the floor of the project. Everything before it was a descent; it is also the pivot the whole arc turns on.

The climb: two overfit runs, then a model that held

Committing to supervised learning did not hand us a working model. It handed us a new failure mode. Our first two detection-transformer experiments overfit - one trained to 20 epochs, one to 30 - because a handful of wells is very little data for a transformer, and the network memorised them instead of generalising. That is the first up-tick in the arc that is still shaky.

The recovery was incremental and unglamorous, which is the point. We normalised the regression targets - location, dip, azimuth - into a common range so no single output dominated the loss, leaned on heavy augmentation to manufacture variety the eight usable wells could not supply on their own, and iterated. The choice of a detection transformer was itself deliberate: it predicts each feature's depth, dip, and azimuth directly as a set, with no anchor boxes and no post-hoc matching, which for a sparse scatter of sinusoids on an image log is a better fit than a region-proposal detector. The mechanics of how that set prediction works are covered elsewhere; here it is enough that it gave us a single model that reads a patch and emits a fracture's geometry.

The geometry itself is a correction, not a raw read. What the tool reports is an apparent dip relative to the borehole; the true dip and azimuth follow from the well and tool angles:

Two versioned checkpoints came out of that stretch: DETR v1, the first supervised transformer that picked sinusoids end to end, and DETR v1.5, the refinement that we were willing to call a working model on the 8 usable wells. That is the top-right of the arc - not a triumphant plateau, just a curve that has climbed back above where it started, on cleaner data and a fundamentally different method.

Why the middle is worth publishing

The reason to write the middle down is that the shape of Phase 2 is the shape of most honest applied-ML projects, and pretending otherwise sets the wrong expectation for the next one.

Three things transfer. Data quality is not a gate you pass once; two of ten delivered wells were unusable for reasons that had nothing to do with the model, and finding that early - by automated range and depth-index checks rather than by a model that quietly underperforms - saved the phase. Unsupervised is a reasonable thing to try and a reasonable thing to abandon; the value of the DBSCAN and SCOPS detour was not a result, it was the evidence that let us commit to supervised learning with conviction instead of hope. And the first supervised runs overfitting is not a setback, it is the expected first draft on small subsurface datasets, with a fix that is boring and real: normalise, augment, iterate.

Limitations

The dated arc is drawn from a single confidential carbonate engagement; the specific defects (the 0-15 pixel range, the depth-index mismatch, the five rejected wells) are this operator's data, not a general failure rate for image logs. The running log's 288 blocks span February to December 2022, but a running log is a work diary, not an audit trail - the dates order the milestones reliably while the exact day a given experiment "failed" is fuzzier than a single block implies. The vertical "project health" axis in the exhibit is an illustrative ordinal built to make the descent-and-recovery shape readable; it is not a measured metric, and only the dated milestones, methods, epoch counts, and well counts on it are sourced. DETR v1 and v1.5 were working models on 8 wells, not the final production system - later phases added wells, horizontal-well validation, and the ablations that turned them into a publishable result.

References

1. Phase 2 running log and dated milestone timeline (February to December 2022, 288 blocks), the well-status ledger (10 wells delivered, 2 excluded for a corrupt pixel range and a compact-tool image, 8 usable), the data-quality defects (0-15 pixel range vs expected 0-255; raw-dip depth indices mismatched against the digital log container depth index; five Phase 1 wells rejected for missing apparent dip and azimuth), the unsupervised detour (Hough plus Canny sinusoid picking; DBSCAN across >10,000 parameter combinations; hierarchical clustering out of memory on a 50 GB machine; SCOPS self-supervised segmentation at 5/3/2 classes, dropped), the explicit turn to supervised learning, and the two overfit detection-transformer runs (20 and 30 epochs) leading to DETR v1 and v1.5, all derived from internal monthly progress reports, the running-log timeline deck, and the supervised-transition deck for a confidential Oman carbonate engagement; data and code withheld under operator confidentiality.

2. The apparent-to-true dip and azimuth correction (true dip = apparent dip + well angle - 180 degrees; true azimuth = apparent azimuth + tool angle - 180 degrees) from the engagement's geometry-correction notes.