Flange defect inspection with a bounding box on every pinhole in the sealing face

The flange on the bench at station 5 has a sealing face that was turned an hour ago and will be bolted against another one, with a gasket between, holding pressure for years. A pinhole in that face the size of a pencil point is a leak, and the leak will be found by whoever is standing next to the joint when it lets go. The inspector runs a thumbnail across the face and holds it to the light, and on a good day catches most of them.

A camera over the face, under a raking light, catches the pinholes the thumbnail misses, and the question is what to do with the ones it is unsure of.

Four classes, each a bounding box, on a face that is mostly fine

The classes are what the inspector already writes on the reject tag: scratch, crack, dent and pinhole. A scratch is a thin line with contrast under raking light. A crack is an irregular line that often branches, and it is always a reject. A dent is a shallow depression that only shows when the light comes from the side. A pinhole is a small dark dot, the hardest of the four to see and the one that leaks.

Each is a bounding box with a class, on frames from the station camera at its mount, under the station's own light, with the swarf and the coolant film that a face has an hour after turning. You type the four classes once, Lexi proposes a box on every candidate in every frame, and a person checks the proposals before anything trains. The QA pass is mostly the pinholes, since a pinhole and a speck of swarf look alike in one frame. The rule written before labeling is that a dot is a pinhole only if it is still there in the frame taken after the air blast.

Good faces outnumber bad ones by a wide margin, so frames with defects are kept at a far higher share than they occur and the rare classes are sampled up. The surface defect detection use case makes the same point for any reflective part: the defects are small and low-contrast, a lighting artefact looks exactly like a flaw, and the training set is unbalanced by construction.

Two thresholds give the line three answers

The model puts a box on each candidate with a number that ranks it against the others. The lesson on what that number means is worth the read: it is a ranking signal and a poor probability, and a rule set on it is an operating point rather than a measurement. The line does not need a probability. It needs a decision per flange, and the decision comes from two thresholds set by the quality engineer at station 5.

Below the lower threshold a candidate is noise and the flange passes. Above the upper threshold it is a defect and the flange is rejected. Between the two is the band, and a flange with any candidate in the band goes to a person. The two numbers are chosen from what a false reject costs against what a leaking joint costs, and on a flange that will hold pressure the second cost is large enough that the band sits low and wide.

The point of the band is that the model is allowed to say it is not sure. A two-answer gate forces every flange to pass or fail, including the ones the model has weak evidence for, and the weak fails are the ones that go out of the door.

The flange in the band goes to a person with the frame

What the inspector gets is the frame, the candidate boxed, the class the model proposed, and the flange itself, held at the station until the verdict. The inspector looks under the raking light as before, decides, and the verdict becomes a label. On a flange line the doubted frames are almost all pinholes against swarf, so each version is trained on exactly the faces the last one found hardest.

LexData takes the flange model through its whole life. You type what to look for, Lexi puts a bounding box on every frame, and a person checks each label before anything trains on it. The model then watches the station camera, in the cloud, on your servers, or on a runner beside the recorder. Frames it is unsure of come back to a person, the corrections retrain it, and the new version replaces the old one with no downtime.

In our manufacturing work the inspection models hold 99%+ accuracy maintained in production, and on a flange line the review bucket is what holds it, since the faces the model was sure about were never the risk.

The reject is a rule written as a sentence, with a severity and a cooldown, approved before it goes live. A flange rejected for a crack at station 5, routine, to the station log and to the quality engineer in Slack at the end of each shift with the frames attached. A pinhole count that climbs across a shift is the same rule with a count in it, and it is the message that sends the engineer to the lathe before the batch is finished.

An aside from the station. The inspector still runs a thumbnail across the face of every flange that comes back for review, and says it is faster than looking, because a thumbnail finds a pinhole the way a tongue finds a chipped tooth.

A new casting supplier is the day the defect rate moves

The castings come from a new foundry in October and the pinhole rate triples while the scratch rate holds. The faces look the same and the classes mean the same, but how often pinholes occur has moved, and every threshold that was tuned around the old rate is now wrong. The band that sent one flange in fifty to review sends one in five, and the inspector is behind by mid-morning.

The drift catalog calls this the defect rate changed, and the important thing is what has not happened: the model has not degraded, and retraining it on the same classes will not shrink the queue. The signal is in the review queue. A jump in doubted flanges with no jump in corrections means the thresholds are wrong for the new castings and the model is still right. A jump in corrections means the model is seeing something it was not taught, a porosity pattern the old foundry never produced, and that is the case for retraining. Reading the two apart is the quality engineer's weekly job for the life of the station.

Thresholds first, retraining second. A queue that grows because the castings got worse is telling the engineer about the foundry, and moving the thresholds to hide it is throwing the message away.

The thresholds and the queue belong to the quality engineer

My own view, from running QA on stations like this one, is that the two thresholds should be owned by the quality engineer outright and changed by nobody else. A new model version should ship with the thresholds it inherited rather than its own. A version that quietly moves the operating point is a version the engineer finds out about from the queue on Monday.

The queue is sized the same way. A review bucket nobody clears is a reject bucket with extra steps, so the band is set as wide as the inspector can clear in a shift and no wider. It is widened when a new lot of castings lands and narrowed when the next version has learned it. How the labels get checked before training is in the labeling doc.