Computer vision event logging that keeps the frame with the prediction

At 6 am the day quality lead on a fish processing line gets a message from the night shift: a customer sample from last night's second lot had a bone fragment in it, and the camera over the trim line was supposed to catch that. What she needs to know before the morning meeting is simple. Did the camera see the fillet, what did the model say about it, and which version of the model was running at the time.

If the answer to any of those is "we do not keep that", the morning meeting is about the camera being unreliable, and it will be that meeting every time.

Event logging for a vision model is the discipline of keeping enough that the question has an answer. Most of it is deciding what belongs in one record.

A miss on the night shift has to be explainable the next morning

The night shift on line 3 runs with fewer people and the same throughput. The model over the trim line draws a box on every fillet and a second box on anything it takes for a bone, and a fillet with a bone box is diverted for a manual check. A fillet with no bone box goes on to packing. The miss in question is a fillet that went to packing.

There are only a few ways that happens. The model saw the fragment and scored it below the line, so no box. The model did not see it, because the fragment was under the fillet or in a shadow. The camera was not looking, because the stream dropped for a minute. Or the model that was running was the new version rolled out on Thursday, and the new version is worse on that fragment.

Each of those has a different fix, and telling them apart needs the record.

The event is the frame, the prediction, the version and the lot together

The record for one fillet is the frame the camera took, the boxes the model drew on it with the class of each, the score behind each box, the version of the model that drew them, the lot that was running, and the time. Stored as one event, the morning question becomes a lookup: find the lot, find the minutes, look at the frames.

Stored as separate things, it becomes archaeology. The frames are on the recorder in a file named by the hour. The predictions are in a log with a timestamp from a clock that drifts. The model version is in a deployment note somebody wrote in a chat channel. The lot is on a printed sheet in the office. Joining those at 6 am is possible and nobody will do it twice.

The rule on line 3 is a sentence approved before it went live, and every alert it sends carries the frame with the detection drawn on it. The event log is the same thing kept for every fillet, including the ones that raised no alert, because the miss is by definition a fillet that raised no alert.

The night shift lead writes the lot changes on a whiteboard by the trim line and photographs it at the end of the shift. It is a better record than most plants have, and it should still be a field on the event.

A prediction without its frame is an assertion nobody can check

A log line that says "no bone detected" on a fillet at 11:42 pm tells the quality lead what the model said. It does not tell her whether the model was right. For that she needs the frame, and she needs the box the model drew, or did not draw, on it. Looking at the frame, she can see the fragment under the edge of the fillet, half in shadow, and she can see that the model drew nothing there.

That is a different finding from "the model missed it". It is "the model missed a fragment in shadow at the fillet edge", which is a labeling instruction. The frame goes back for labeling with the fragment boxed, and the next version has seen it.

Without the frame, the finding is a number in a report and a line about the model being unreliable at night. With the frame, it is a correction.

The doubted frames and the corrections are the record that grows

There is a part of the log that gets more valuable every week, and it is the smallest part. The model returns the frames it is unsure of, the ones where the mark might be a bone or might be a fold in the flesh, and a person on the line looks at each one and confirms or corrects the box. That verdict is a label, and the frame with its verdict is the most useful event in the whole record.

LexData takes the trim line model through its whole life. You type what to look for, Lexi puts a box on every frame, and a person checks each label before anything trains on it. The model then watches the camera on line 3, in the cloud, on your servers, or on a runner beside the recorder. Frames it is unsure of come back to a person, the corrections retrain it, and the new version replaces the old one with no downtime. The doubted frames and the verdicts are where that new version comes from.

My own view is that the corrections are the only part of the log worth keeping forever. The confident detections on a good shift are the report. The corrections are the training set.

Retraining from the record beats collecting fresh footage

When the morning meeting ends with "the model is worse on fragments in shadow", the instinct is to schedule a fresh collection: a week of night footage from line 3, a labeling pass, a new model. The lesson on retraining without starting over argues against it, and the event log is why. The frames the model got wrong are already in the record, boxed by a person, with the conditions that caused the miss. A correction on a frame the model got wrong teaches it what it does not know; a fresh frame from a random night teaches it something it may already know.

So the retrain is built from the record: the doubted frames with their verdicts, added to the original set rather than replacing it, so the model handles the shadowed fragment without forgetting the well lit one. When the corrections cross the threshold the plant set, the new version trains on them and rolls out, and the record shows which version was running on every fillet from then on.

The override rate is the number the record produces every week

One more thing falls out of a log that keeps the verdicts. If the person on the line is correcting the model more often this week than last, that override rate is rising, and a rising override rate is the signal that the model is drifting from the line it was trained on. A new supplier's fillets, a changed light over the trim station, a camera nudged during a wash-down: each shows up as corrections before it shows up anywhere else.

The quality lead now opens one view on Monday mornings: corrections per shift on line 3, plotted against the previous month. Our manufacturing work holds 99%+ accuracy in production, and that plot is how a plant knows it is still holding, rather than finding out from a customer sample.