Counting the screws in a kit with object detection over the bench

The kitting bench at the end of line 2 packs a housing, a bracket, a cable and eleven screws into a bag, and the operator counts the screws twice because the customer counts them once. The bag that leaves with ten screws comes back with a complaint and a credit note. The bag that leaves with twelve costs a screw, which sounds like nothing until it is multiplied by a shift.

A camera over the bench sees the tray before the bag is sealed. The question is what the count should be, and that turns out to be two different questions depending on what is being counted.

Object detection gives a count by boxing each item

A count comes from a box per item. A model that says "screws present" is a classifier and cannot count, and a model that colours in every screw-shaped pixel cannot separate two screws lying against each other. Object detection draws one box around each screw, and the count is the number of boxes the tray is left with.

That makes the labeling job simple to describe and easy to get wrong. Every screw in every frame gets its own box, including the one under the edge of the bracket and the two touching head to head. You type "screw" once, Lexi proposes a box on every screw in the frame, and a person checks the proposals before anything trains. The person is there for the touching pair and the one in shadow, because the model counts the way its labels counted.

Frames come from the bench camera at its mounted height, with the tray where the operator actually leaves it. Photographs of screws on a clean white sheet teach the model about a bench that does not exist.

A kit is an exact count and a case is a band

The kit on line 2 has eleven screws or it is wrong. There is no tolerance, because the customer's technician will reach for the eleventh. The rule is an equality: boxes counted equals the number on the kit spec, and anything else stops the bag.

The case packer on line 4 is a different instrument. Bottles go in twelve to a case, and the frame from above catches most of them squarely while one at the back sits half behind the divider. A rule that insists on twelve every time will stop the line for the bottle the camera could not quite see. The count for the case is a band with a floor: eleven or fewer stops the line, twelve passes, and thirteen is impossible by the geometry of the case, so it is a detection error and goes for review rather than out of the door.

Deciding which of the two a station is comes before any labeling. A kit that ships to a customer who counts is exact. A case that the packer physically cannot overfill is a band. Putting an exact rule on a band station is how a plant ends up switching the camera off by Thursday.

The half-hidden item is decided by a written labeling rule

The screw under the bracket edge is the whole problem in one frame. Half of it shows. One labeler boxes it because it is plainly a screw; another leaves it because half a screw could be a reflection off the tray. Both are defensible, and a model trained on both learns to flip a coin on every partial item, which on an exact-count station is a stop every few kits.

The fix is a sentence written down before labeling starts, and applied by every labeler and by the QA pass after them: an item is boxed when its head is visible, whatever the rest is doing. Or the opposite rule. Which one matters less than that there is one, and that it matches how the count is meant to work. On the kit bench the rule is "box it if the head shows", because the head is what the customer's technician will find in the bag.

Consistency here is what the accuracy figure rests on. In our manufacturing work the models hold 99%+ accuracy maintained in production, and the stations where that holds are the ones where the partial-item rule was written before the first frame was labeled rather than argued about after the first false stop.

The stop carries the frame with the boxes drawn on it

When the count is wrong the operator needs to see why, on the spot. A number on a screen saying ten is a puzzle; the frame with ten boxes on it and a gap where the eleventh should be is an instruction. The operator looks at the tray, finds the screw that rolled under the bracket, and moves on.

The stop is a rule written as a sentence, with a severity and a cooldown, approved before it goes live. Count under the kit spec on the line 2 bench, high, to the operator's screen and to the line lead in Slack. A count over the spec is the same rule with a different severity, since an extra screw costs a screw and a missing one costs a customer. The frame travels with every message, so the person reading it in Slack sees the same tray the operator saw.

One aside from the floor. The operators on the kit bench still keep the tally sheet, and nobody has asked them to stop, because the sheet is what the count on the screen gets checked against when the two disagree.

Frames the model doubts come back before they become false stops

LexData takes the counting model through its whole life. You type what to look for, Lexi puts a box on every screw in every frame, and a person checks each label before anything trains on it. The model then watches the bench camera, in the cloud, on your servers, or on a runner beside the recorder. Frames it is unsure of come back to a person, the corrections retrain it, and the new version replaces the old one with no downtime.

On a counting station the doubted frames have a shape. A new screw finish arrives from the supplier and the heads are darker than the ones the model was trained on. The tray gets scratched and the scratches pick up light the way a screw shank does. Each of those produces frames the model is unsure of, and each of those frames, corrected by the person on the bench, is what the next version learns from. The correction rate is the signal to watch: when the operator overrides the count more often than last week, the model is drifting, whatever the accuracy number said at launch.

A new kit spec changes the right answer without changing a pixel

The kit for the spring order on line 2 has nine screws instead of eleven, because the bracket changed and two holes went away. The tray looks the same. The screws look the same. The model finds nine boxes on the first kit, compares it with the eleven it was told to expect, and stops the line for a correct kit.

The drift catalog calls this a spec change: the pixels are identical and the right answer moved. There is no retraining to do, because the model's counting is still right. What has to change is the number the rule compares against. It has to change on the same day the new spec arrives at the bench, which means the kit spec lives where the rule can read it rather than in a binder by the door.

The assembly verification use case is the same check on a bigger part: is every piece present, in the right place, the right way round. Counting is the presence half of it. My own view, from running review queues for these stations, is that a station should launch as a band and be tightened to an exact count only after a week of reviewing its doubted frames. A false stop on the first morning does more damage to a rollout than a week of slightly loose counting. How the labels get checked before training is in the labeling doc.