Computer vision in steel manufacturing, from the ladle shell to the coil

The ladle comes off the crane onto the stand at the caster with a hundred tonnes of liquid steel inside it and a refractory lining between that steel and a shell that is only ever supposed to get warm. The lining wears a little on every heat. The crew knows roughly how many heats a lining lasts, and the ladle is pulled for relining on a schedule with a margin in it. The alternative is a breakout, and a breakout on the stand at 6 am is the worst morning a melt shop can have.

A thermal camera on the stand sees the shell on every heat. Where the lining has thinned, the shell runs hotter, and the hot patch shows up in the thermal frame before anyone on the floor can feel it.

The thermal frame shows the lining from the outside

The camera sees heat, and the shell shows its lining through it. A healthy shell in the thermal view is a fairly even tone with the known warm areas at the slag line and around the nozzle. A worn patch is a region brighter than its neighbours, growing a little each heat, in a place the wear map for that ladle says is plausible.

Labeling here is regions rather than objects. The labeler draws a polygon over each hotspot on the shell, and the class is what the refractory engineer calls it: slag line wear, a nozzle well hotspot, a patch on the barrel. You type the classes once, Lexi proposes the regions on every frame, and the engineer checks them against the ladle's wear map before anything trains. The map is the ground truth, because the engineer has seen the lining that came out of the last ladle that ran hot in the same place.

Two things make thermal harder than it looks in a demo. Emissivity is one: a patch of scale or a splash of slag on the shell reads as a different temperature from bare steel at the same real temperature, and a model trained on clean shells will flag every splash. Reflection is the other: the ladle sits near a furnace and the shell reflects it, so a hot region that moves when the crane moves is the furnace and a hot region that stays put across heats is the lining. The frames for training come from the stand, on real heats, with the reflections and the splashes in them.

Failures are too rare to learn from, so the model learns the wear before them

A plant may go years without a breakout, which is the point of the relining schedule. A model trained to recognise breakouts has nothing to train on. What there is plenty of is wear: every ladle on every heat shows a lining somewhere on its way from new to pulled, and the engineer's post mortem on each pulled lining records which shell regions were thin.

So the target is the proxy. The label on a hotspot region is the wear the engineer confirmed when the lining came out, and Lexi proposes the same region on the heats before the pull. The model then learns what a thinning lining looks like through the shell across the heats before it was pulled. Shell temperature relative to that ladle's own baseline on the same heat count is the quantity to watch, since a ladle fresh from reline runs cool everywhere and one near the end of its campaign runs warm everywhere, and neither of those is a hotspot.

That is a modelling choice a melt shop has to make on purpose. The predictive maintenance on machinery use case is the same problem in a different plant: the failure is the one event nobody has enough of, and the rate of change on the way to it is what there is to learn.

The alert goes to the melt shop before the shell fails

The alert is a rule written as a sentence, with a severity and a cooldown, approved before it goes live. A hotspot on the ladle at station 2 above its baseline by the margin the refractory engineer chose, critical, to the melt shop supervisor and to the caster pulpit in Slack. The frame arrives with the region drawn on it, so the supervisor sees which part of which ladle rather than a temperature with no place attached to it.

The cooldown matters more here than on most alerts. A ladle on the stand is in the thermal view for the length of a cast, and a hotspot does not stop being hot between samples. Without a cooldown the pulpit gets the same message every couple of seconds until the ladle is lifted. With it, the message arrives once, and again only if the region grows.

With a runner beside the recorder on the plant network the alert fires on site first, the thermal footage stays in the plant, and only the doubted frames leave. On a site where the caster pulpit has no outside connection, that is the difference between a monitor that works and one that is switched off.

Scale and scratches on the coil are a different camera and the same loop

The other camera in the plant is at the coil, where the finished strip runs past a line camera under controlled light. Here the objects are small and the classes are what the quality inspector already calls them: scale, scratches, inclusions, edge cracks. The distinction that matters most on this line is scale against scratch, since a scratch is a rework decision and scale is a pickling decision, and the two look alike at the resolution of a wide frame.

Boxes work for presence. Masks earn their cost when the disposition depends on length, because a scratch below a certain length in the coil's cosmetic zone is a different decision from the same scratch across the full width. In our manufacturing work the inspection models hold 99%+ accuracy maintained in production, and the coil line is where that figure is earned, because the strip never stops and the inspector cannot look at all of it.

LexData takes the coil model through its whole life. You type what to look for, Lexi puts a box on every frame, and a person checks each label before anything trains on it. The model then watches the line camera, in the cloud, on your servers, or on a runner beside the recorder. Frames it is unsure of come back to a person, the corrections retrain it, and the new version replaces the old one with no downtime. The doubted frames on a coil line are almost always the scale-or-scratch cases, and the inspector's verdict on each is what the next version learns.

A replaced thermal camera changes every temperature the model learned

The thermal camera on the ladle stand is in the hottest, dirtiest place in the plant, and it gets replaced. The new one has a different sensor, a different lens and a different response curve, and the same shell on the same heat now reads a few tones warmer or cooler across the whole frame. The model, trained on the old camera's tones, sees a plant full of hotspots, or none.

The drift catalog calls this a camera was replaced, and on a thermal camera it is worse than on a visual one, because the number the model was learning from is the camera's own calibration. The signal is a step on the day of the swap: the correction rate from the melt shop jumps, the engineer overrides the alerts in Slack, and the ladles that were fine yesterday are flagged today. The fix is to re-verify the baseline for each ladle from the new camera, re-label a short window of heats, and retrain on those. The linings did not change, which is exactly why the override rate is the thing to watch rather than the model's own accuracy figure.

An aside from the stand: the crew still touches the shell with a temperature gun on the way past, as they did before the camera, and the gun reading is what the new camera's baseline gets checked against after a swap.

My view is that a thermal model with no baseline per ladle should not go live at all, however clean its numbers look on the stand it was trained on. A hotspot is only a hotspot relative to what that ladle did last heat, and a model that does not know which ladle it is looking at is guessing at the most expensive question on the floor. How the alert gets attached to the stand camera and written as a sentence is in the monitoring doc.