Visual anomaly detection on the line, catching the defect you never labeled

The fillet line has a defect that has never been photographed. The quality lead knows it exists because a customer found a shred of blue glove in a pack last March, and because the line has had bone fragments, a patch of bruising nobody could describe, and once a piece of the conveyor's own rail. Each was found by a person, downstream, and none of them looks like the others. There is no folder of examples to train a detector on, and there will not be one until the next one turns up.

That is the situation anomaly detection is for. The line has a season of frames of what a good fillet looks like on that blue belt under that light, and the defect is whatever does not.

Object detection needs an example, anomaly detection needs normal

An object detection model learns a class from labeled examples of it. Give it a hundred boxed bone fragments from the belt at 6 am and it will find bone fragments, and it will find nothing it was never shown, however strange. That is the right tool for the defects the line already knows and has photographed, and the surface defect detection use case is built on it.

Anomaly detection turns the problem round. The model learns what a normal fillet on this belt looks like, from clean frames only, and scores every new frame by how far it sits from that. A shred of blue glove has never been seen, and it does not need to have been. It is far from normal, so it scores high.

My own view is that anomaly detection belongs under the detector as a net, never in place of it. The detector catches the known defects at a rate the plant can measure. The net catches the thing nobody has named yet, and hands it to a person who will.

Normal is learned from clean frames under the line's light

The training set is the easy part to collect and the hard part to get right. It is frames of good fillets, from the camera over the belt at its mounted height, across a shift, with the enclosure closed so the light is the line's own. A few hundred frames that a person has looked at and confirmed as clean are enough to start.

What poisons it is a defect in the normal set. A bruised fillet that slipped through the clean frames teaches the model that bruising is normal, and the model will never flag it again. So the clean set is checked by a person, frame by frame, with the same care as any label. The labeling with Lexi guide describes that pass: confirm what is right, reject what is wrong, and treat a frame nobody has looked at as unlabeled.

The gloves are blue because no food is blue. The same logic decides the belt colour, and both make the anomaly model's job easier than it would be on a white line.

The score and heatmap route the fillet to a person

Every fillet gets a score, and above a threshold the frame goes to a person with a heatmap drawn on it: the region of the fillet that pulled the score up. The heatmap is what makes the flag usable, because a person can see in one glance that the strange region is a bit of glove on the left edge rather than a shadow from the rail.

The threshold is set against the plant's own appetite for false flags. Low, and every unusually shaped fillet reaches a person and the queue fills by 10 am. High, and the glove gets through. There is no correct number, only an operating point the quality lead chooses and revisits, and the honest way to choose it is to look at what a week of flags contained.

An alert in LexAlert is a rule written as a sentence, with a severity and a cooldown, approved before it goes live: an anomaly above the line's threshold on the fillet camera, to the QA station's screen and to Slack, with the frame. The person at the QA station pulls the fillet or waves it on.

Flagged parts become classes over time

Each flag a person judges is a label. After a month the flagged frames sort themselves into a handful of things: glove fragments and bone and bruising, and a pile of shadows and belt marks that were nothing. The nothings tune the threshold. The real ones are now a folder of examples, and a folder of examples is what a detector needs.

So the classes are typed, glove fragment and bone fragment, Lexi puts a box on every flagged frame, and a person checks each label before anything trains on it. The detector now catches those two on its own at a rate the plant can measure, and the anomaly net keeps watching for the third thing. On the manufacturing lines we run, 99%+ accuracy maintained in production is measured on the named classes, and the net is how a new class earns its name.

LexData takes the fillet model through its whole life. You type what to look for, Lexi puts a box on every frame, and a person checks each label before anything trains on it. The model then watches the camera over the belt, in the cloud, on your servers, or on a runner beside the recorder. Frames it is unsure of come back to a person, the corrections retrain it, and the new version replaces the old one with no downtime.

A process change or a new defect rate resets normal

The line switches supplier in October and the fillets are paler and thinner. Nothing is wrong with them, and every one of them scores as strange, because normal was learned in June.

The flags flood the QA station on the first shift. The fix is a new clean set from the new fillets, checked by a person, and a retrained normal. A process change is a scheduled event, and the clean set can be collected the day before.

The quieter version is a change in how often the real defects occur. A new deboning machine halves the bone fragments, and the threshold that was tuned to the old rate now flags mostly nothings, and the QA station stops looking. The drift catalog calls this the defect rate changed, and the signal is alert volume against its own history, a number the plant already has. The fix is to re-tune the threshold against the current rate before anybody touches the model.

The flag that the QA station stops looking at is the one that will contain the next shred of glove. The threshold exists to keep that station looking.