Vision AI for factory robots, picking from a bin nobody arranged

The bin of sheet metal blanks arrives at the cell as it was tipped out of the press's outfeed. A few hundred flat steel shapes lie overlapping and at every angle, some face down, some standing on edge against the bin wall, all of them wet with drawing oil. The arm's job is to pick one, present it to the CNC machine behind the fence, and come back for the next. Fixed automation cannot do this. A fixture wants parts arranged, and nobody arranged these.

A camera over the bin can. What it has to give the arm is more than a box.

Object detection finds each blank, the arm needs its rotation

The first job is object detection: a box on every blank the camera over cell 2 can see, so the cell knows how many are left and where the top layer is. For a flat blank on a pile that is nearly enough, because a blank's outline is its shape and its shape is known from the drawing. What the arm needs beyond the box is the blank's rotation in the plane, so the gripper closes across the width rather than the diagonal, and its height in the pile, so the arm knows which blank is on top and reachable.

The manipulation and grasping use case says why a plain box falls short: a gripper needs a position and a rotation, and neither a box nor a mask carries orientation. For flat blanks the label is a rotated box or a pair of keypoints at known features, the notch and the far corner, from which the angle follows. The pick point is then a rule on those, written once for the part.

The blank that is standing on edge against the wall is found by the detector and refused by the rule. The arm moves on.

Oily blanks and reflections are the frames a person labels

Wet steel under the cell's lights is a moving reflection. The same blank shows a bright streak from one angle and a dark face from the next, and the streak sits exactly where an edge would be. A model trained on dry blanks on a bench has never seen this, and the first bin teaches it nothing on its own.

So the labels come from the cell camera over real bins, across a whole shift. In LexAnnotate you type "blank, notch corner", Lexi puts a box on every frame, and a person checks each label before anything trains on it. The person's time goes on the blanks half under another blank, the ones whose reflection hides the notch, and the pair that are stuck together with oil and look like one thick blank. My own view is that a cell should never learn a new part on the job. Label it from the bench and from the first real bins before the arm is allowed to trust it.

Suction cups lift a face-down blank as readily as a face-up one, which is why the check after the pick matters as much as the find before it.

The same camera checks the pick after the arm lifts

Once the gripper closes and lifts, the camera looks again. Is one blank in the gripper, or two stuck together. Is it face up. Is the notch where the drawing says it should be relative to the gripper. Each is a box question on a frame with a known background, the empty air above the bin, and each is a class the same model was trained on. A failed check is a blank returned to the bin and a second attempt, and a failed check twice on the same blank is an alert.

An alert in LexAlert is a rule written as a sentence, with a severity and a cooldown, approved before it goes live: two failed picks in a row at the bin cell, to the cell engineer's Slack, with the frame. The frame shows the gripper and the double blank, and the engineer knows before the CNC does.

A mis-pick is a lost cycle. A person inside the fence while the arm is moving is the other kind of event, and the same camera can watch the fence line with a rule about that too. In our robotics work the figure that carries weight is 99%+ safety-critical accuracy, and it belongs to the fence rule rather than the pick.

The new part number is labeled before its changeover

In June the cell switches to a new blank for the next product, a similar shape with the notch on the other side and a slot the old blank did not have. The model has never seen it. On changeover day every blank in the bin is found badly or called the old part with the notch in the wrong place, and every pick is wrong, which reads as a perception failure and is nothing of the kind.

The drift catalog calls this the equipment changed: the model was never wrong about the old part and has no opinion about the new one. The changeover date is on the production plan weeks ahead, so the new blank is labeled from sample bins before the reel of coil is even ordered, and the version running on changeover day has seen both. The cost of missing that is a cell that stops on its first shift and a reputation problem that outlasts the fix.

Corrections from the cell retrain the model

Every refused pick and every returned blank is a judgement the cell has made about the model, and the cell engineer's verdict on the alerts is a label. A double blank the model called single, a notch it placed wrong under a new lamp, a blank it missed against the bin wall: each comes back for review and goes into the next version.

LexData takes the bin model through its whole life. You type what to look for, Lexi puts a box on every frame, and a person checks each label before anything trains on it. The model then watches the cell camera, in the cloud, on your servers, or on a runner beside the recorder. Frames it is unsure of come back to a person, the corrections retrain it, and the new version replaces the old one with no downtime.

The bin still arrives as it was tipped. The arm picks from it anyway, and the engineer finds out about the stuck pair before the CNC does.