Collecting image training data from a production line without keeping every frame

The label model on the carton line has run clean since spring. Every carton passes the station camera, the model boxes the label, reads the lot code and confirms the placement, and the doubted frames arrive at a rate the QA lead clears in ten minutes on a Friday. In the second week of September a new supplier's cartons come onto the line. Same artwork, heavier board, a matte varnish where the old ones were gloss. The model starts missing labels on the matte cartons, and the QA lead's Friday takes an hour.

Every one of those misses is a training frame. The question is which ones to keep, how to keep them, and how they get from the line to the next version without someone building a dataset by hand.

A new carton print arrived and the model started missing it

The matte varnish is the whole problem. The label edge that was a crisp line on gloss board is a soft one on matte, and under the station's lights the reflection the model half-learned as part of the label is gone. The model has weak evidence for matte cartons because it has never seen one, and the cure is a few hundred matte cartons, labeled and checked, folded into the training set.

The line has been producing those cartons every shift since September. Recording every frame the station camera sees would fill a disk with the same carton photographed from the same angle under the same light. The how much footage lesson is blunt about what that is worth: one condition sampled many times, which teaches the model almost nothing it did not know.

Sample the frames the model doubts rather than everything it sees

The model is already telling you which frames matter. On the matte cartons its boxes are low and unstable, the label class flickers between found and not found across consecutive frames, and a growing share of frames come back as doubted. Those frames, and only those, are the ones worth pulling from the stream.

Sampling by doubt inverts the usual pipeline. Instead of capturing everything and filtering down, the consumer that watches the station camera holds only the frames the model was unsure about, plus a small random draw of the ones it was sure about. The random draw is there so the training set does not become a museum of hard cases with no ordinary cartons in it. Across a full shift of matte cartons the doubted frames number in the hundreds, which is a set a person can review on a Friday.

The frames the model got confidently wrong never arrive this way. A label it boxed with high confidence in the wrong place is not a doubted frame. My own view is that those are the most valuable frames on any line, and the only way to get them is to give the operator at the station a way to flag the frame in front of them when they see a miss. On the lines we run, that flag puts the frame in the same queue as the doubted ones.

Adjacent frames are the same frame and one is enough

A carton takes several seconds to cross the station's field of view, and at a sample every two seconds the same matte carton appears twice or three times. Those frames differ by a few pixels of conveyor travel and nothing else. Keeping all of them triples the review work and teaches the model nothing the first one did not.

The filter is cheap. A perceptual hash of each sampled frame, compared against the last few kept, and a frame within a small distance of a kept one is dropped. Adjacent frames of the same carton collapse to one. Frames of different cartons, even the same design, survive, because the board sits differently on the belt each time and the hash moves. The QA lead's hour goes back toward ten minutes without a single hard case lost.

The packaging line keeps a physical golden sample of every carton on a shelf above the station, and the inspector holds it up to the light when a run looks off. The matte supplier's sample went up on the shelf in the second week of September. The model had no shelf.

Preprocess only what the model will see in production

Every step applied to a training frame has to be a step the model will also see on line 3, and nothing else. If the station camera's frames are resized before inference, the training frames are resized the same way, by the same code, before they are labeled. If the label region is cropped out of the wider carton frame for a second model that reads the lot code, the crop is taken at the same margin in training as in production. A crop that is a little generous in training and a little tight on the line is a model that has never seen a label touch the edge of its input.

Resolution consistency is the one most often broken, because the training frames tend to be pulled from the recorder's full-resolution stream and the model is fed from the lighter sub-stream. They are different pictures. Pull the training frames from the stream the model watches.

Nothing is re-encoded on the way. A frame that has been through a second round of compression carries artefacts the line will never produce, and the model will learn them as if they were board texture.

Every sampled frame carries its camera, line, shift and lot

A frame on its own is a picture of a carton. A frame with its camera, line, shift, timestamp and lot code attached is evidence. When the matte cartons are found to cluster on line 3, night shift, lots from the new supplier, the metadata says so without anyone re-watching footage. The per-class evaluation of the next version can then be cut by exactly those fields: matte cartons at night on line 3, before and after.

That metadata is also what makes a training set auditable a year later. A version that can name the lots it trained on can be reverted; one that cannot is a black box that once scored well.

Review is where a doubted frame becomes a label

The sampled, de-duplicated, preprocessed frames land in a review queue, and a person opens it. Lexi has already put a proposed box on each label; the reviewer confirms the ones that are right, moves the ones that are close, and draws the ones the model missed entirely on the matte board. Each verdict is a label checked against the written guideline, and each is a correction counted against the model's prediction.

The counting is what makes review part of the pipeline rather than a chore beside it. The override rate on the label class rises from the second week of September, the rise is on line 3 and on the new lots, and that is the drift signal, before any accuracy number has moved.

The threshold decides when the sampled frames become a version

Corrections accumulate until they cross the project's threshold, and then a new version trains: on the spring set plus the matte corrections, never on the matte corrections alone, because a model that has only seen matte board will swap which carton it fails on. The retraining lesson covers the rest: hold out the future rather than a random slice, compare per class on the condition that triggered the retrain, and promote only when the new version wins on the matte cartons without losing the gloss ones.

LexData takes the label model through its whole life. You type what to look for, Lexi puts a box on every frame, and a person checks each label before anything trains on it. The model then watches the station camera the line already has, in the cloud, on your servers, or on a runner beside the recorder. Frames it is unsure of come back to a person, the corrections retrain it, and the new version replaces the old one with no downtime. The pipeline in this post is the middle of that sentence, and on the platform it is the part nobody has to build.

By October the matte cartons are ordinary. The QA lead's Friday is back to ten minutes, and the frames that took the hour are in the version that runs.