Robotic welding defect detection on every weld the robot lays down

The robot in cell 3 lays the same seam on the same bracket every forty seconds, and the quality engineer pulls one part an hour to look at it under the bench light. The other parts go to the next station on the strength of that one. When a batch comes back from the customer with porosity along the seam, the engineer can say which hour it happened in, and nothing more, because the parts that were not pulled were never looked at.

A camera at the cell, looking at the seam as the torch lifts, sees every weld.

The bead has to be found before anything on it can be named

The first job is finding the weld. The frame from the cell camera has the fixture and the part, spatter on the fixture from the last thousand welds, and somewhere across it the bead the torch just laid. A model that looks for defects in the whole frame will find them in the spatter. So the pipeline starts with a box on the bead, and every later question is asked inside that box.

That box is a class of its own, labeled on frames from the cell camera at its mounted position, with the torch fumes still in the air. You type "weld bead" once, Lexi proposes a box on every seam in every frame, and a person checks them. The checking here is for the ends of the seam, where the bead tapers into the start and stop craters, because a model that consistently misses the last centimetre of the seam is a model that never sees the crater cracks that live there.

Object detection assigns the defect class inside the bead

Inside the bead box the classes are what the weld engineer already calls them: porosity and undercut, incomplete fusion and spatter on the bead, a crater crack and a burn-through. Each is a box on the frame with a class. Object detection is the right shape for this, since the engineer's decision is about which defects are present and where along the seam, and a box carries both.

The classes get written down before labeling, with a reference frame for each, in the engineer's words. Porosity is a cluster of pits with a written minimum count; a single pit is not porosity. Undercut is a groove at the toe of the bead; it is distinguished from a normal toe by a written depth the engineer can point at in the frame. A class that two labelers draw differently is a class the model will draw both ways, and on a weld line that means the same seam passing on Monday and failing on Tuesday.

Good welds outnumber bad ones by a wide margin on a working cell, so the frames with defects are kept at a far higher share than they occur. The rare classes, burn-through especially, are sampled up until the model has seen enough of them to draw them. The weld quality inspection use case has the same shape: a class per defect, a box per instance, the seam as the region the classes live in.

In our manufacturing work the inspection models hold 99%+ accuracy maintained in production, and on a weld cell that figure is the detector agreeing with the engineer on the parts the engineer pulled.

The borderline weld goes to the quality engineer with the frame

Some welds are plainly fine and some are plainly not. Between them is a band: a toe that might be undercut, a few pits that might or might not be porosity by the written count, a crater the model boxed at low confidence. Those go to the engineer with the frame, the bead boxed and the candidate defect drawn inside it, and the engineer decides from the bench light as before. The difference is that the engineer is now looking at the parts the model could not decide, rather than at one part an hour chosen by the clock.

The engineer's verdict on each becomes a label. LexData takes the weld model through its whole life. You type what to look for, Lexi puts a box on every frame, and a person checks each label before anything trains on it. The model then watches the cell camera, in the cloud, on your servers, or on a runner beside the recorder. Frames it is unsure of come back to a person, the corrections retrain it, and the new version replaces the old one with no downtime. On a weld cell the doubted frames concentrate in the band, so each version is trained on the welds the last one found hardest.

A defect the rules call reportable is a rule written as a sentence, with a severity and a cooldown, approved before it goes live. Porosity on a seam from cell 3, high, to the cell screen and to the weld engineer in Slack, with the frame attached. The operator at the cell sees the seam with the pits boxed and pulls the part before it leaves the fixture.

Every weld inspected changes what the engineer's day is for

Inspecting every weld instead of one an hour is a bigger change than it sounds. The engineer used to spend the hour walking to the cell, pulling a part, and looking at it under the light. Now the parts that need the light come to the bench on their own, and the hour goes on the review queue and on the question the queue makes visible: which defect classes are rising, on which cell, since when.

That question has a plain answer for the first time. Porosity on cell 3 was flat for a month and has climbed since Tuesday. The engineer walks to cell 3 with a reason rather than a schedule, finds the shielding gas flow has been turned down, and the porosity stops. The camera did not diagnose the gas. It made the pattern visible on the day it started rather than three weeks later when the customer called.

One aside from the cell. The engineer still pulls one part an hour and puts it under the bench light, and the part chosen is now the one the model was most confident about, on the grounds that the confident welds are the ones nobody else is looking at.

A new torch or wire is the change that needs new frames first

The torch on cell 3 is replaced in the spring, and the new one lays a slightly wider bead with a different ripple. Or the wire supplier changes and the bead's surface texture changes with it. The seam is fine. The model, trained on the old bead, sees a texture it has never seen and boxes it as spatter, or as porosity, on every part.

The drift catalog calls this the equipment changed: a new generation of the machine, a new consumable, and the model has never seen the thing it is now looking at. The signal is the correction rate, which steps up on the day of the swap as the engineer overrides the flags on good welds. The fix is a short window of frames from the new torch, labeled and folded in, and it can be done before the swap rather than after it, since a new torch is scheduled. Run the new torch on the cell for a shift and label the seams, then retrain, and the model meets the new bead already knowing it.

My own view is that the consumables log and the review queue should be the same screen. The day a correction rate steps up is almost always a day something on the consumables log changed, and an engineer who can see both at once needs no further explanation. How the labels get checked before training is in the labeling doc.