Batch video analysis of archived drone survey footage without a notebook open

A survey contractor has three seasons of drone flights along a pipeline right-of-way in a cloud bucket. Each flight is a folder of video files named by date and pilot, and the folder for the spring flights is the largest because the vegetation was up. Nobody has watched most of it. The client paid for the flights, received a report on the sections a person had time to look at, and the rest went into the bucket in case anyone asked.

Someone has asked. The client wants every concrete cradle along the line found and every rust mark flagged, across all three seasons, by the end of October.

Watching the footage is out of the question. Writing a notebook to loop over the bucket, pull frames and run a detector is the usual answer, and it works until the person who wrote it leaves. The alternative is to treat the archive as a batch job that the platform runs and a person reviews.

01Import the originals, because nothing is re-encoded

The first decision is what to import, and the answer is the files as they were recorded. Footage comes in from S3, from Google Drive, or by upload, and nothing is re-encoded on the way. That matters more on a survey archive than anywhere else: the rust mark on a cradle is a few pixels of orange on grey, and a compressed export of the flight loses it before the model ever sees it.

So the contractor points the import at the bucket's mirrored folder on Google Drive, one project per season, and the originals land in the project at the resolution the drone shot them. The quickstart covers the same step for a single file, and the batch version is the same step repeated.

One project per season rather than one giant project is worth the extra minute. The spring flights look different from the autumn ones, and the review queue for each is easier to reason about on its own.

02Sample frames every few seconds, because adjacent frames are one scene

A drone at survey speed sees the same cradle for several seconds. Every frame of those seconds is the same cradle from a slightly different angle, and labeling or detecting on all of them is the same work done thirty times. Sampling a frame every couple of seconds keeps every cradle and every rust mark and throws away the copies.

The sampling rate is the one knob worth thinking about. Too sparse and a short cradle between two frames is missed; too dense and the review queue fills with duplicates. On a right-of-way flight the ground speed and the altitude are known, so the interval that guarantees every metre of pipe appears in at least one frame is arithmetic, and it comes out near the platform's default of about 2 seconds.

The pilot on the spring flights kept a paper log of where the drone lost signal and circled back. Those sections appear twice in the footage, and the sampling does not know that. The review queue finds out.

03Object detection on the sampled frames is a batch, and a batch finishes

With the frames sampled, the model runs across them as a job rather than a stream. Object detection on a fixed set of frames has a property live monitoring never has: it ends. The three seasons become a count of cradles, a count of rust marks, and a set of frames with boxes drawn on them, and the whole thing is done before anyone has to decide what to do about it.

The model for this job was trained on the contractor's own frames from a previous engagement, two classes, cradle and rust, and it was checked by a person before it trained. If it did not exist, the same archive would be the place to build it: type the two classes, let Lexi propose the boxes on a few hundred sampled frames, and check them. The labeling guide covers how a prompt becomes a class list and what the verification pass does.

04Review what the model doubted, and only that

The batch produces two piles. The frames the model was sure about, cradle found or rust flagged, and the frames it was unsure of, where the mark might be rust or might be a shadow, or the cradle is half under scrub. Only the second pile goes to a person.

That is what makes the month achievable. A reviewer looking at every sampled frame across three seasons would not finish. A reviewer looking at the doubted frames, with the box Lexi drew already on it and the choice reduced to confirm or correct, works through a season in a sitting. Each verdict is a label, and the labels feed the next version.

My own view is that the doubted pile is more valuable than the confident one, and most survey teams archive the wrong pile. The confident detections are the report. The doubted frames are the reason the next report will be better.

05The archive becomes the training set for the next flight

LexData takes the survey model through its whole life. You type what to look for, Lexi puts a box on every frame, and a person checks each label before anything trains on it. The model then watches the footage you bring it, in the cloud, on your servers, or on a runner beside the recorder. Frames it is unsure of come back to a person, the corrections retrain it, and the new version replaces the old one with no downtime. The three seasons in the bucket are three seasons of conditions the next model has already seen.

That is the quiet return on the batch. The lesson on how much footage you actually need argues that the unit that matters is distinct conditions rather than hours, and a survey archive is nothing if not distinct conditions: spring growth, autumn light, a flight in haze, a section flown twice. The next flight along the same line runs against a model that has seen all of it, and its review queue is shorter for it.

The contractor's report to the client is a table of cradles and rust marks by kilometre marker, each row with a frame behind it. The report took a month. The next one will take the length of the flight.