Pretrained vs fine-tuned detection models, when the stock detector cannot see the thing you need

The camera on the tower crane mast looks down on the scaffold along the north face of the building. A stock detector, downloaded and run on the feed as it is, draws a box around every worker on every deck by 7 am, and it is right about nearly all of them. Somebody then asks it which workers are wearing a hard hat, and it has no answer, because it has no class for a hard hat and never did.

The threshold slider goes down and the model finds more workers. It never finds a hat. That is the whole difference between a pretrained detector and a fine-tuned one, and it is worth being precise about what each of those words means.

The COCO dataset is a fixed list no threshold extends

A pretrained detector is a model somebody else trained on a public dataset, usually the COCO dataset, which stands for Common Objects in Context and is mostly photographs of streets, living rooms and dinner tables. It knows a person, a bicycle, a dog and a cup. It knows nothing about a hard hat, a scaffold deck or a fall arrest lanyard, and a class it was never trained on is a class it cannot output at any threshold.

The threshold decides how sure the model has to be before it reports one of the classes it knows. Lowering it turns weak person detections into reported people. It cannot turn a person into a person wearing a hat, because the model has no such category to be unsure about. Teams lose weeks on this, tuning a slider against a question the model was never asked.

The stock detector is still useful. It tells you in an afternoon that the camera can resolve a worker on the far deck, and that is the first question worth answering before any labeling starts.

Fine-tuning keeps the stock weights and adds the site's classes

A fine-tuned detector starts from those same COCO weights and continues training on frames from the site, with the classes the site cares about. Everything the model learned about edges, textures and the shape of a person carries over. What changes is the last part of the network, which now has a hard hat class beside the person class, and the frames that teach it are the ones from the crane mast camera.

The reason to start from the stock weights rather than from nothing is footage. A model from scratch needs an enormous amount of labeled footage to learn what an edge is. A fine-tuned one already knows, so a few hundred labeled instances of the thing you need are enough to add a class that holds up on that camera. The how much footage lesson gives the unit: instances of the thing, never hours of video.

My own view is that a stock detector should never be the production model even when its classes happen to match the job. The threshold you tune on someone else's photographs is tuned to their light and their distances, and the site's own frames are cheap to label by comparison with a false sense that the model is done.

Frames from the mounted camera are the dataset

The frames that matter come from the crane mast camera at its own height and angle, in the site's own light, across the site's own week. A hard hat seen from forty metres above is a small pale disc, and a model taught on photographs of hats at eye level has weak evidence for it. So the labeling is done on frames from that camera: hats on heads, hats carried, hats on the deck beside a worker at the break, the white hats that vanish against the netting.

In LexAnnotate you type "person, hard hat", Lexi puts a box on every frame, and a person checks each label before anything trains on it. The review is where the site's judgement gets into the model: whether a hood over a hat is a hat, whether a hat in a hand counts, whether the worker half behind the standard is labeled at all. The site safety and PPE compliance use case picks up what happens after the boxes exist, which is attributing the hat to a head rather than to a scene.

A hard hat from above looks a great deal like a bucket, and buckets are common on scaffolds. That pair is worth labeling on purpose.

Live in days, then corrected in days

The path from the stock model to a hat model on this camera is short: a window of frames, the two classes typed, a review pass, a training run, and a model watching the feed. The quickstart covers the first labels in five minutes. Live in days is the honest figure for the whole loop on one camera, and what makes it hold is what happens in the weeks after.

LexData takes the scaffold model through its whole life. You type what to look for, Lexi puts a box on every frame, and a person checks each label before anything trains on it. The model then watches the camera on the crane mast, in the cloud, on your servers, or on a runner beside the recorder. Frames it is unsure of come back to a person, the corrections retrain it, and the new version replaces the old one with no downtime.

The stock detector never gets that second week. It was finished the day somebody else trained it, on their photographs, and it will find every worker on the north face and no hats for as long as it runs. The fine-tuned one was wrong about buckets on Monday and knows them by Friday, because a person said so.