How to choose a camera for computer vision before you collect a frame

A new inspection station is going in at the end of an assembly line that makes machined brackets. The bracket has a threaded insert pressed into it, and about one in a few hundred leaves the press without one. The station's job is to catch that bracket before it is boxed. The mechanical drawings are done, the conveyor stop is ordered, and somebody has been asked to pick a camera by Friday.

The camera is chosen last on most projects and should be chosen first, because every frame the model ever learns from comes through it. Swap the camera after training and the model is looking at a world it has never seen.

Work back from the smallest thing the model has to see

The insert is a few millimetres across. For a detector to find it reliably, the insert needs to cover a few dozen pixels in the frame. The missing insert has to look different from the present one at that size, which means the empty hole and the threaded ring must both be resolvable. That is the whole specification, and the rest of the camera follows from it.

The field of view is the bracket plus a margin, since the station only needs to see the bracket. Divide the field of view by the pixels the insert needs and the required resolution falls out. A camera with far more resolution than that number costs money, slows the model, and ends up resized down to the training resolution anyway.

The engineer at the bracket station measured the insert with a vernier on Monday and drew the field of view on the conveyor with a marker pen. That drawing was the most useful document in the project.

The shutter decides whether a moving part is a smear

A bracket that stops under the camera at the end of line 4 can be photographed with almost anything. A bracket that keeps moving needs a global shutter, which exposes every pixel at the same instant, rather than a rolling shutter, which reads the sensor line by line and turns a moving edge into a slant. The insert on a rolling-shutter frame at conveyor speed is an oval with a soft edge, and the model will learn ovals.

Exposure time is the other half. A short exposure freezes the motion and starves the sensor of light, which is where the lamp comes in.

The lamp matters more than the sensor

Most inspection failures blamed on the camera are lighting failures. The hall lights above the bracket station come on at 6 am, get brighter when the roller door opens onto a sunny yard, and flicker faintly at mains frequency. A model trained under one of those conditions has weak evidence for the others.

A station light, a ring or a dome over the bracket with the hall light shaded out, makes every frame the same frame. The threaded insert throws a shadow inside the hole that an empty hole does not, and the right lamp angle makes that shadow the most visible thing in the image. Half the labeling effort disappears when the lamp is right, because there is only one look to label.

My own view is that a team choosing between a better sensor and a better lamp should buy the lamp every time. The sensor decides how much detail is available. The lamp decides whether the detail is there to be seen.

The interface has to reach the runner

The frames have to get from the camera to wherever the model runs. A camera that publishes an RTSP stream can be watched like any other feed, and if it records to the plant's recorder the runner beside that recorder can pull from there instead. Machine-vision cameras on other interfaces need a capture step that produces frames the runner can consume, and that step is part of the station, with its own failure modes.

Where the model runs is decided at the same time. A station that has to stop the conveyor on a bad bracket cannot wait for a round trip to somewhere else, so the model lives on a box beside the camera. The deployment guide covers the runner on your own hardware, how the model reaches it, and what leaves the building, which for a station like this is only the frames the model was unsure of.

The bracket station's runner sits in the same cabinet as the conveyor controller, on a shelf the electrician added in Tuesday's shutdown.

Object detection needs frames from the camera as mounted

Once the camera is on its bracket, under its lamp, at its exposure, the frames for training come from there and nowhere else. A set collected with a phone held over the conveyor, or with the camera on a bench before installation, teaches the model a station that does not exist. The insert looks different at a different angle, under a different light, and object detection learns the look, not the part.

The first frames are the boring ones: brackets with the insert present, hundreds of them, plus the rare missing one whenever it turns up. The learn section has a lesson on how much footage a model needs, and the answer for a fixed station under a fixed lamp is less than most teams expect, because the world in front of the camera barely varies.

LexData takes the station model through its whole life. You type what to look for, Lexi puts a box on every insert in every frame, and a person checks each label before anything trains on it. The model then watches the station camera, in the cloud, on your servers, or on a runner beside the recorder. Frames it is unsure of come back to a person, the corrections retrain it, and the new version replaces the old one with no downtime.

A swapped camera is a sensor replacement with a relabel

Cameras fail, and the one in the stores on a Friday is never quite the one that failed. A different sensor has a different colour response and a different noise floor, and to the model a familiar bracket now looks unfamiliar even though the operator sees a picture that is, if anything, better. The drift catalog calls this a camera was replaced, and it is the drift a station meets whenever maintenance opens the cabinet.

The fix is short if it is planned for. A window of frames from the new camera, labeled and folded into the next version, and the model is back on its own numbers. The fix is long if nobody connects the maintenance ticket with the correction rate climbing the following week, which is the usual way it is discovered.

Write the camera's model and serial on the station drawing next to the lamp and the exposure. When the swap happens, the drawing is what tells the next engineer that the training set came from a camera that is no longer there.