Build or buy the layer that keeps a vision model accurate

The pilot on the packaging line works. Two engineers spent a month on it, the detector finds the missing label on a carton nine times in ten, and the plant manager has seen the demo twice. The question on the table at the Monday meeting is whether those two engineers now build the rest, and nobody in the room has a clear picture of what the rest is.

The rest is larger than the model, and it is the part that decides whether the detector is still accurate at Christmas.

A production pipeline is seven parts and the model is one of them

The model in the pilot is a trained artifact. Around it, a system that runs for a year needs six more things. Footage coming in from the line camera and stored somewhere the team can find it. A way to label frames and check the labels. A record of which frames trained which version. An evaluation against frames the model has never seen. A way to run the model beside the camera. A way to see what it is doing on the live feed through the Tuesday night shift, and to feed the mistakes back.

Each of the seven can be built. A folder of frames, a labeling tool, a spreadsheet of versions, a training script, a held-out set, a container on a box in the cabinet, a dashboard. The pilot has rough versions of at least four of these already, which is why the build looks close to finished from inside the room.

The distance is in the word "maintain". Every one of the seven has to keep working when the person who wrote it is on holiday, when the line adds a second camera, and when the plant manager asks why last Tuesday's version flagged a hundred good cartons.

Homegrown versioning breaks the first time two people train

The spreadsheet of versions works while one engineer trains. The second engineer trains on a Friday with a slightly different set of frames, the two artifacts have similar names, and by the following Wednesday nobody can say which one is on the box beside the recorder or which frames it saw. When the label reader starts missing the new carton design, the first hour of the investigation goes on establishing what is running.

That is the shape of every homegrown failure in this list. The tool works for the case it was written for, and the case changes.

Dataset versioning that survives needs each model version to keep what it was trained on, and each frame to carry the version of the label that trained it. A retraining pass that does not start over depends on that record existing, because the new version has to be trained on the old frames plus the corrections, and the old frames have to be findable.

The review queue is where the build usually stops

The pilot has no review queue. Nobody built one, because during the pilot the two engineers looked at every frame themselves, on a Friday afternoon with the line running slow. In production the model watches the packaging line through every shift, and the frames it is unsure of have to go somewhere a person on the floor can see them, correct them and send the correction back. Without that, the model's mistakes are found by the customer complaint, weeks later.

A queue is not hard to build once. It is hard to keep. It needs a person assigned, a place in the shift routine, a way to correct a box rather than describe the problem in a ticket, and a path from the correction into the next training run. On the builds I have seen, the queue is the component that goes stale first, because it is the one that needs a non-engineer to use it every day.

My own view is that a team should count the review queue as the product, and the model as the part that comes free with it. That is the opposite of how most plants budget the work.

Rollout with no downtime is a system, not a script

The pilot was deployed by copying a file to the box and restarting the container. The line was stopped for lunch on a Wednesday, so nobody noticed. The third version ships mid-shift, the container is down for four minutes, and the carton counter misses a run. The fourth version is delayed a week while someone writes the rollout script, and the fifth needs a rollback because the evaluation set did not include the night-shift lighting.

Replacing a model on a live camera without a gap, with the previous version kept ready, is a deployment problem the plant's IT team recognises from every other system in the cabinet. The deployment doc covers where the model can run, in the cloud, on the plant's own servers, or on a runner beside the recorder, and the same rollout works in each place.

An aside: in most plants the box beside the recorder is labeled with masking tape and the name of whoever set it up. It is the most important computer on the line and the least documented.

Buying the operations layer replaces the parts that go stale

What a platform replaces is the middle of the list: the labeling with a check on every label, the version record, the review queue, the retraining and the rollout. The camera stays, the line stays, and the two engineers keep the parts they are good at, which are the question the plant is asking and the judgement of whether the answer is right.

LexData takes a vision model through its whole life. You type what to look for, Lexi puts a box on every frame, and a person checks each label before anything trains on it. The model then watches the cameras you already have, in the cloud, on your servers, or on a runner beside the recorder. Frames it is unsure of come back to a person, the corrections retrain it, and the new version replaces the old one with no downtime. The accuracy you launched with is the accuracy you keep.

The platform page is the long version of that paragraph, and pricing is arranged around the work, the number of cameras and the labeling mix, rather than a checkout.

The decision is about what the team wants to be maintaining in a year

Building makes sense when the vision problem is the business, when the team is large enough that the seven parts each have an owner, or when the frames cannot leave a building under any arrangement. A team of forty with a research group will build, and should.

A plant with two engineers and a pilot has a different question. In a year, those two can be maintaining a version spreadsheet and a queue nobody uses, or they can be adding the second and third camera to a loop that already runs. A model that goes live in days is the common outcome once the operations layer is not being written from scratch, and the first thing the plant notices is that the review queue has a person in it on Monday morning.