Object alignment detection for the label, the connector and the gasket on one station

The station at the end of the controller line does three things in about forty seconds. The operator presses a gasket into the groove on the housing lid, pushes a connector into the socket until it clicks, and applies the product label to the carton. On a Tuesday in May a batch of two hundred units shipped with the connector seated two degrees off, which was enough to stop the mating cable from latching, and the customer sent them back. Every one of those units had passed the presence check, because the connector was present.

Present was never the question. Where it was, and which way it faced, was.

Three questions on one station, all of them about position

Alignment on the station breaks into three checks that look alike and are not. The label check asks whether a rectangle sits inside another rectangle with a margin all round. The gasket check asks whether a ring sits centred in a groove, which is a question of whether the gap is even. The connector check asks about rotation, a few degrees around an axis, on a part whose outline barely changes when it turns.

The assembly verification use case frames the same station as presence, position and orientation, and notes that all three read from a box and its aspect, with no measurement of extent. That is true for the label and mostly true for the gasket. It is not enough for the connector, and the May batch is why.

A box and its aspect answer most of it

The label is the easy case. Box the carton, box the label, and the offset of one box's centre from the other's is the alignment. The class names are typed once, Lexi proposes the two boxes on every frame, and a person checks them. The person's attention goes to the frames where the operator's hand is still on the label, because a box drawn around a hand and a label teaches the model that hands are part of labels.

The gasket is nearly as simple. A gasket that is centred is a ring whose inner and outer edges are the same distance apart all round, which shows in the box as a near-square with the groove's box centred inside it. A gasket pulled to one side pushes its box off the groove's centre by the amount it was pulled.

Somebody on the controller line once asked why the label check passed for a year and then failed every unit for a morning. The label supplier had wound the new roll the other way, so every label came off the applicator upside down, still centred, and the check was only ever looking at the centre.

Pose estimation is for the connector, where two degrees matter

A connector rotated two degrees has a box almost identical to a connector rotated correctly. The outline is the same shape, the centre is in the same place, and the only thing that has changed is the angle of the keying tab, which is a few pixels wide. A box cannot see it, and neither can a person at a glance, which is how two hundred units left the building in May.

This is where pose estimation earns its place. Two points on the connector, the keying tab and the centre of the shell, and the angle between them measured against a reference in the frame. The what a vision model can and cannot see lesson is blunt about the limit: the model finds the points, and the geometry does the rest, and if the points are a few pixels wrong the angle is a few degrees wrong. Tight boxes on the tab, checked by a person on every training frame, are the difference between an angle and a guess.

The reference is the fixture, found on every frame

Every one of the three measurements is made against something that does not move. On the controller station that is the fixture: two bolt heads and a machined edge that hold the housing in the same place for every unit, from the first unit at 6 am to the last. The label's offset, the gasket's centring and the connector's angle are all measured from those bolt heads, and the bolt heads are boxed on every frame by the same model, as their own class.

I would measure everything in pixels against those bolt heads before ever calibrating to millimetres, and I would keep it that way for the first month. A tolerance in pixels on a fixed camera is a tolerance nobody can argue with, and the millimetre conversion can wait until the station has shown what it does across a shift.

Borderline placements go to a person rather than the bin

A label a hair outside its margin, a gasket that might be lifted at one point, a connector angle right at the tolerance edge: these are the frames the model should not decide alone. They come back to the operator with the frame and the measurement drawn on it, and the operator's verdict is a label.

LexData takes the station model through its whole life. You type what to look for, Lexi puts a box on every frame, and a person checks each label before anything trains on it. The model then watches the station camera, in the cloud, on your servers or on a runner beside the recorder. Frames it is unsure of come back to a person, the corrections retrain it, and the new version replaces the old one with no downtime. The manufacturing lines that hold 99%+ accuracy in production are the ones where the tolerance-edge frames were answered rather than guessed.

When the camera moves, the reference and every measurement move

The bolt heads are the fix for one problem and the cause of another. In August the camera bracket is loosened to change a bulb and tightened back a few degrees off. The connector detector still finds connectors and the bolt heads are still found, because a bolt head looks like a bolt head from any angle. But the pixel distance between the two bolt heads has shrunk, the machined edge now runs at a slight slant, and every angle measured against them is off by the same few degrees. A perfectly seated connector now measures as a two-degree rotation.

The drift catalog calls this a camera moved, and the symptom on an alignment station is a step: every measurement shifts on the same day, in the same direction, while the parts have not changed. The check that catches it is cheap, the distance between the two bolt heads compared with its own history every morning. When it moves, the camera has, and the fix is to re-verify the reference from the new framing, re-label a short window of frames and retrain on footage the station already has.