Glass defect detection with computer vision, scratches where the background shows through

The camera on the bottle line looks at each container for a second as it passes the inspection station after the annealing lehr, at 6 am and at 4 pm alike. What it sees, through the glass, is the conveyor behind the bottle, the stainless rail, the next bottle along and its own ring light reflected twice. Somewhere in that is a scratch a customer would return the bottle for, a chip on the rim that will fail the seal, and a hairline crack that will open in the filler. A person at the station can find them by tilting the bottle. The camera cannot tilt anything.

Glass is the one material where the defect and the background occupy the same pixels, and most of the work is in arranging the light so they stop doing that.

A scratch and a reflection match until the light moves

Under a ring light on the camera's axis, a scratch on the bottle's surface scatters light back into the lens and shows as a bright line. So does the edge of the reflected ring, the highlight off the next bottle, and the seam of the mould, and at 4 pm so does the skylight. Four bright lines, one defect, and a model trained on these frames learns bright lines.

The fix is in the geometry rather than the model. Light the bottle from the side at a low angle, so the smooth surface sends the light away from the lens and only a scratch scatters it back. Against a dark background the scratch is now the only bright thing on the glass. Chips at the rim want the opposite, a backlight behind the bottle that turns the rim into a crisp silhouette with a bite out of it. Two lights, two frames per bottle, and each defect appears in the frame built for it.

My own view is that on a glass line the lighting engineer should sit in on the labeling review, because half the boxes a labeler will want to draw on a badly lit frame are the light.

The conveyor shows through the glass and must go

An opaque part hides its background. A bottle does not, and on the station at 6 am the conveyor's cleats, the rail and the next bottle all appear inside the bottle's outline, moving, and all of them have edges a model can mistake for a crack. A plain matte backdrop behind the station, in a colour the conveyor is not, removes most of it. A short enclosure removes the hall's skylight, which otherwise moves across the glass all afternoon.

What remains is the next bottle along, which is glass too. Spacing the bottles at the station with a screw feed so only one is in the frame is a mechanical fix to a vision problem, and it is cheaper than any amount of labeling.

Chips and hairlines are different scales and want different cameras

A chip at the rim is a millimetre or two and lives in one known place. A hairline crack can run the height of the bottle and be a fraction of a pixel wide at the station's working distance. A single camera framed for the whole bottle sees the chip as a few pixels and the crack as a faint line it cannot separate from a mould seam. The what vision can see lesson asks the question first: can the camera resolve it at all.

So the rim gets its own camera, close and backlit, and the body gets a wider frame under the low-angle light. Each camera answers one question. On the manufacturing lines we run, 99%+ accuracy maintained in production is a figure that holds per camera and per question, and it would not hold for a single camera asked both.

Only labels drawn under the station's own light transfer

A labeled dataset of glass defects from a bench, a phone or another line is nearly useless on this station, because the defects look like whatever light they were photographed under. The labels that train this model are drawn on frames from these two cameras, under these lights, with the backdrop in place, across a shift. In LexAnnotate you type "scratch and rim chip and crack", Lexi puts a box on every frame, and a person checks each label before anything trains on it.

The reviewer's time goes on the seams, the reflections that survived the low-angle light, and the scratches on the inside of the bottle, which scatter differently. Those are the ones the surface defect detection use case would grade by length and position with a mask rather than a box. A scratch on the body is cosmetic. A crack from the rim is a bottle that fails in the filler.

The glass trade has its own words for what goes wrong in the melt, and the schema should use them. A seed is a small bubble, a stone is an unmelted inclusion, a cord is a streak of glass of a different composition, and a check is a small surface crack. An inspector who has used those words for twenty years will label against them without a glossary.

A new container spec dates every label

In the spring the customer moves to a lighter bottle: thinner walls, a slightly different shoulder, a new mould. The light now passes through less glass, the reflections sit in new places, and the rim silhouette has a different profile. Nothing on the line changed. Every label drawn on the old bottle describes a container that is no longer running.

The drift catalog files this under the spec changed: the mapping from image to correct answer moved, and there is no input signal because the new bottle looks perfectly ordinary. The first evidence is the inspector overriding the model in a consistent direction, calling chips on rims the model passes, and that override rate is the signal worth watching from the first shift of the new bottle. The new mould's date is known in advance, so a window of the new bottles is labeled before the changeover rather than after.

LexData takes the bottle model through its whole life. You type what to look for, Lexi puts a box on every frame, and a person checks each label before anything trains on it. The model then watches the two station cameras, in the cloud, on your servers, or on a runner beside the recorder. Frames it is unsure of come back to a person, the corrections retrain it, and the new version replaces the old one with no downtime.

The person at the station still tilts a bottle now and then. The camera has the light doing the tilting for it.