Storing a year of computer vision predictions without drowning in them

The camera over the tab welder on line 3 of a battery cell plant looks at every cell that passes. The model draws a box on the weld and a second box on any spatter it finds, and it does that on every frame, all shift, all year. Nobody looks at those boxes while the line is running well. Then in March a customer returns a batch from January, and the question arrives: what did the camera see on that lot.

If the answer is a folder of every frame with every box on it, the answer is no. A year of a line camera is more frames than anyone will open, and the interesting ones are indistinguishable from the rest by filename.

What the plant needed in March was not everything the camera saw. It was the moments the model flagged, tied to the lot, with the frame still there to look at.

Boxes on every frame answer no question anyone asks

A prediction, on its own, is a set of coordinates and a class on one frame. Stored raw, that is what a year produces: coordinates, by the hundreds of millions, with no lot number, no shift and no way to say which welder was running. The question in March is never "what was the box on frame nine hundred thousand". It is "which cells on the January lot were flagged, and were the flags right".

So the unit of storage is the event, and the event is a flagged prediction with its context attached. The context is what the plant already knows at the moment of the frame: the camera, the line, the shift, the lot that was loading, and the model version that produced the box. Attach those at the time the frame is taken, because none of them can be reconstructed from the pixels afterwards.

The line lead on line 3 keeps a printed shift log clipped to the cabinet door, and the lot changes are written on it in marker. That log turns out to be the only record of which lot was running at which minute. It should have been a field on the event.

Each flagged prediction becomes an event with the lot attached

An event on the battery line looks like this: camera 4 on line 3 during shift 2, the lot code and the model version, then the class the model found with its box and the time it was found. That is a row, and a row is queryable in a way a frame is not. A year of flagged events on a line that flags a few cells an hour is a table a laptop can search.

The frame itself is kept beside the event, because a row that says "spatter" with no picture is an assertion nobody can check. The frame is the evidence. When the customer complaint arrives, the person answering it wants to see the weld, and they want the box the model drew on it, so they can judge whether the model was right.

The rule written for the line is a sentence, approved before it goes live, and the alert carries the frame with the detection drawn on it. That same frame, with the same context, is the thing worth keeping. The alert and the stored event are one record seen from two sides.

Unflagged frames are sampled, and most of them are thrown away

A line camera sampled about every two seconds produces a great many frames on which nothing was found, and nothing was found because nothing was there. Keeping them all is storage spent on the least informative thing the camera produces. Keeping none of them means nobody can later ask what a normal weld looked like in January, which matters the first time the process shifts.

The middle path is to sample. One frame in a long stretch of clean ones, kept with its context, is enough to reconstruct what the line looked like on any given shift. The rest go. Sampling also has to be tied to the lot, so that every lot has at least a few clean frames stored even if it ran through a quiet night.

My own view is that plants keep far too many clean frames and far too few of the doubted ones. A clean frame from a good shift teaches nobody anything. The frame the model was unsure about is the one that will matter.

The doubted frames and the corrections are the record that grows

There is a third kind of frame, and it is the one that pays for the whole store. The model on the tab welder returns the frames it is unsure of, and a person looks at each one and either confirms the box or corrects it. That verdict is a label, and the frame plus the verdict is the most useful row in the table.

LexData takes the weld model through its whole life. You type what to look for, Lexi puts a box on every frame, and a person checks each label before anything trains on it. The model then watches the camera on line 3, in the cloud, on your servers, or on a runner beside the recorder. Frames it is unsure of come back to a person, the corrections retrain it, and the new version replaces the old one with no downtime. The store of doubted frames and verdicts is where that next version comes from, which is why those rows are never sampled away.

Corrections are also the signal that the model is drifting. If the person reviewing the queue is overriding the model more often this month than last, that override rate is the number to watch, before any accuracy figure moves. A store that keeps the verdicts can produce that number every week.

A store lets the plant ask what changed on the night shift

Once the events carry their context, the questions the plant could never ask become a filter. Which shifts flag the most spatter. Whether the flags on line 3 rose after the electrode change. Whether a lot that was later returned was flagged more than its neighbours. Whether the model version rolled out in February flags more or fewer welds than the version before it, on the same line.

Asking those questions of LexInsight returns an answer with the frames behind it, so the answer to "what did the camera see on the January lot" is a set of frames with boxes on them and the verdicts a person gave. That is an answer a customer can be shown.

The plant in this story runs its weld cameras at the 99%+ accuracy our manufacturing work maintains in production, and the store is how it proves that on any lot, months later.

Retention follows the question, and the question is a recall

How long to keep each kind of row is decided by the longest question anyone will ask. Flagged events with their frames stay for as long as a lot can come back, which on a battery line is the warranty of the pack it went into. Sampled clean frames can go after the process has been stable for a while. Doubted frames with verdicts stay as long as the model does, because every version that follows was trained on them.

Storage on a runner beside the recorder is finite, and the events leave the site while the footage stays. What leaves is the row and the flagged frame; what stays is the recording. A plant that has to answer to an auditor keeps the rows where the auditor can reach them and the recorder where it always was.

The store is the memory the line keeps, so that a question in March about a lot from January has an answer made of a frame with a box on it and the verdict a person gave, rather than a shrug.