Blur augmentation in computer vision, training for the blur the camera will produce

The camera over the stamping line is bolted to a gantry that shares its footing with the press. Every time the press cycles, about once every four seconds, the gantry rings and the camera with it, and for a moment the frame is soft. The autofocus then hunts, overshoots, settles. On a cold morning the housing fogs on the inside for the first hour of the shift and every frame is soft until the heater catches up.

The model that watches that camera was trained on frames somebody picked because they were sharp.

That is the ordinary way to build a training set, and it is the reason the model on the gantry is worse at 7 am than it was on the validation split. The question of whether to add blur to the training frames comes down to a smaller question: what does the camera actually produce, and can the thing being detected survive it?

Blur on the line has three causes and three shapes

Motion blur from the gantry over press 2 is directional. The camera moves, the whole frame smears a few pixels along the axis the gantry rings in, and edges perpendicular to that axis go soft while edges along it stay crisp. Defocus from the autofocus is uniform, a soft circle around every point, the same in every direction. Condensation is uniform too, and it adds a milky haze that lifts the dark values as well as softening the edges.

They look different and a model that has learned one has not learned the others. A training set augmented with a symmetric soft blur, the default in most pipelines, covers the autofocus case and does a fair job on condensation. It does very little for the gantry, whose blur has a direction, and the direction is the same every cycle. If the augmentation is going to help, it has to produce the blur the line produces, with the gantry's axis and the gantry's amount.

The amount is measurable. Pull a hundred frames from the recorder across a shift, sort them by sharpness with any simple edge measure, and look at the bottom quarter. That is the blur the model has to survive, and it is the blur to train on. A blur much stronger than that teaches the model about a camera that does not exist.

Blur helps on the panel and destroys the scratch

The stamping line camera has two jobs. It counts the steel panels leaving the press, which are large, and it flags surface scratches on them, which are fine. Blur augmentation helps the first and harms the second, and the reason is what each class looks like after the blur.

A panel is a large bright rectangle on a dark conveyor. Soften it and it is still a large bright rectangle. A model trained on blurred panels learns to find the shape whether the edge is crisp or not, and at 7 am when the housing has fogged it still counts.

A scratch is a thin dark line a pixel or two wide. Soften it by a few pixels and it is gone, blended into the panel around it, and there is nothing left in the frame for any model to learn. Training the scratch class on blurred frames trains it on frames that no longer contain a scratch, under a label that says they do. The model learns that a clean panel is sometimes labeled scratched, and its precision on the clean frames falls.

So the augmentation is per class, or per job. The counting model trains with blur. The scratch model trains without it. The honest answer for the scratch job is that a fogged frame cannot be inspected, the runner should say so, and the frames from the first hour of a cold shift should be held for a second look once the housing has cleared.

My own view is that a fine defect class should never be trained on artificially blurred frames, whatever the augmentation library defaults to. If the line really does produce soft frames with visible scratches in them, collect those frames and label them, and the model learns what a scratch looks like through real blur. Synthetic blur applied to a sharp scratch does not produce that frame. It produces a clean panel with a wrong label.

The evaluation set is never blurred

The training set is where augmentation lives. The evaluation set is a sample of what the camera produces, and it has to include the soft frames the camera produces on its own, in the proportion it produces them. Augmenting the evaluation set with synthetic blur measures how well the model handles synthetic blur, which nobody cares about.

The mistake usually happens by accident. An augmentation is set at the dataset level rather than the split level, and every frame that passes through the loader is blurred, including the ones being scored. The score looks fine, because the model was trained on the same blur, and the line at 7 am disagrees. The check is dull: open ten frames from the evaluation loader and look at them.

The footage lesson says the unit of a training set is distinct conditions. The fogged first hour is a condition, and the way to have it in the evaluation set is to pull frames from the recorder at 7 am on a cold day, label them, and keep them out of training. Ten of those are worth more than a thousand sharp ones with a filter applied.

A blur that appears one day and stays is a drift event

Blur that comes and goes with the press cycle and the morning fog is a condition. Blur that starts on a Tuesday and does not leave is something else. A lens that was knocked during a wash-down, a bracket re-tensioned so the camera now vibrates at a different frequency, a housing that has lost its seal and fogs all shift. The frame has changed, and the model has been looking at a soft version of a scene it learned sharp.

The drift catalog files this with a camera that moved, because it has the same signature: one camera's detections diverge from their own history on a maintenance date, while the cameras beside it hold. The first signal on the gantry camera was the count. Panels leaving the press were being counted low against the press's own cycle counter, from one shift onward, and the frames the model doubted were all soft in the same direction.

The fix on a stuck blur is physical first. Re-seat the lens, re-seal the housing, re-tension the bracket, and see whether the frames come back sharp. If they do, there is nothing to retrain. If the camera is going to stay soft, the doubted frames from the soft weeks are the training set for the next version, and they were collected by the model asking for a second look on exactly the frames it could not read.

Somebody on the stamping line keeps a lens cloth in the cabinet beside the recorder, and it has done more for the 7 am count than any augmentation.