Bike, or Not Bike?

TL;DR by Goose▾

Ingredients

• The street-parking tracker — the camera and pipeline from the first post in this series, already pointed at the block (already set up)
• YOLOv8s + Ultralytics — the off-the-shelf detector doing the seeing, and the Python library that wraps it with a one-call training loop (free)
• A human review queue, backed by SQLite — a dashboard panel that shows me a looping crop with the box drawn on it and asks what it is. Every judgment is a row, which is what turns an afternoon of clicking into a training set (already built)
• A laptop — the fine-tune trained locally on an Apple-silicon MacBook Air; no cloud GPU (already owned)

The Problem: My Camera Kept Seeing Bikes in a Tree

There is a joke in Silicon Valley about an app that was supposed to identify any food you pointed it at, and shipped only ever knowing two answers: hot dog, and not hot dog. My version is bike, or not bike — and when I started, it was getting that answer wrong about a third of the time, because of a tree.

Some background, briefly. A while back I put a camera on my street to track parking occupancy on my block in LA. It watches the bays and logs events, and sometimes when it checks whether a bay is occupied it logs a bicycle in the frame too. The parking part works. The bicycles were the problem.

My street has jacarandas, and a jacaranda branch is thin, articulated, and hangs into the frame. When the wind picks up it swings through — and to a detector working from shape alone, a wind-blown branch and a passing bicycle are close to the same object. Both are transient. Both move. Neither has a rider. I already had geometric rules to kill the obvious phantoms (a two-wheeler that sits perfectly still is scenery, because there is no outdoor bike parking in view), but a swinging branch is not still. What makes a branch obviously not a bike to a human is texture and context; my rules were only looking at motion.

Two numbers run through the rest of this: precision (when the system says “bike,” how often is it right) and recall (of all the real bikes, how many it caught). They trade against each other, and my rules were biased toward never missing a bike — paying for it with a pile of phantoms.

Step One: Trying to Tune My Way Out

I swept 48 threshold combinations against 631 of my own labeled crops and took the best one. Precision went from 66.3% to 70.7% while recall held at 94.4%, and false positives dropped from 163 to 132. A real improvement — and, as it turned out, the last one tuning had to give.

Every threshold in the ghost-suppression rules had been set by eye: I looked at a handful of crops, picked a number that felt right, moved on. But by now the review queue had accumulated 631 human-classified detections, each one a crop I had personally judged, so instead of guessing I could fit. The script rebuilds the suppression registry from cached detections under a given set of parameters, re-decides every labeled detection, and scores the result against my own verdicts. The three settings that moved were fixed_min_occupancy (how much of a clip a spot must fill to count as a fixed object), min_events (how many separate events must confirm a spot as scenery before it is condemned), and tight_min_events (the same idea for the tighter night-time flicker rule).

🔧 Developer section: Fitting the thresholds to human labels

• Ran .venv/bin/python tools/fit_thresholds.py --recall-floor 0.92 — maximize precision, but never below that recall floor
• Scored the 631 twowheeler_reviews rows where source is human; auto-resolved rows would grade the rules against themselves
• Rebuilt the registry from cached data/_reclass_cache.jsonl per candidate — pure geometry, no YOLO re-run, so 48 sweeps stayed cheap
• Winner into config.yaml: fixed_min_occupancy 0.6 → 0.3, min_events 3 → 2, tight_min_events 3 → 2; fixed_min_boxes stayed at 3
• An earlier registry recorded every detection, not just confirmed-fixed spots — it made the bike lane a ghost region and suppressed 89% of events

The Number That Would Not Move

Of all 48 parameter sets, none beat 70.7% precision. Not one. That flat result is the most useful thing this project produced, and it is the reason there is a rest of this post. The ceiling was not a local maximum I could climb out of with a finer sweep — 4.4 percentage points was the entire budget available from tuning, full stop.

The false positives that survived were all the same thing: foliage that is transient, moving, and riderless, and therefore geometrically identical to a passing bike. My rules only reason about where a box is and how long it stays there, and by those measures a swinging branch is a bicycle. There is no threshold that separates two things which, in the only vocabulary your rules speak, are the same thing. That is a detector problem, not a threshold problem — and realizing it was worth more than the tuning was, because it is what stops you burning a month.

Step Two: Learning My Street Is Not Mostly Bicycles

I split the single other label into six classes. That turned a bucket I could not learn from into a training signal — and along the way it surfaced a data bug that would have silently dropped 1,115 rows out of the training set.

To fine-tune, I needed labels, and my review queue only offered three buttons: bicycle, other, none. Going back through the 121 detections I had marked “other,” the composition was the surprise — 35% of everything real on this street is not a bicycle. Motorcycles, kick scooters, strollers, the occasional delivery robot. A third of my real two-wheeler-shaped traffic was flattened into a bucket that told a training run nothing. So I split the taxonomy into seven labels: bicycle, motorcycle, scooter, stroller, delivery_robot, other, and none.

One decision in there is worth explaining, because I got it wrong on the first pass. Only none blacklists a location. When I mark a detection none — nothing is there, that is a tree — the system remembers those pixels and stops trusting them, which is correct for scenery, because scenery never moves. Do the same for a stroller or a delivery robot and you condemn a patch of sidewalk that real things move through, silently suppressing the next actual cyclist to cross those pixels.

The bug that would have quietly poisoned the training set

While wiring the new labels up I found something unpleasant. The review queue has an auto-resolve feature: when I judge one detection, it settles obvious neighbours at the same spot so I do not have to click the same branch forty times. That auto-resolve was setting a verdict but never a label. On the dashboard everything looked fine — rows resolved, queue drained, counts right. But the dataset exporter selects on label IS NOT NULL, so 1,115 rows would have silently dropped out of any training export. Not errored. Not warned. Just quietly not there.

I would have trained on a fraction of my data while believing I had all of it, gotten a mediocre model, and had no idea why. That is my most consistent lesson from building data pipelines without an engineering background: the failures that hurt are the ones where a filter quietly excludes most of your rows and everything downstream keeps running on a fraction of the truth. Count your rows at every stage, and compare the counts.

Step Three: Fine-Tuning (And Why 1280 Instead of 640)

I fine-tuned YOLOv8s on my own labeled crops at an image size of 1280, training locally on a laptop with no cloud GPU — and without drawing a single bounding box by hand.

That last part is what makes this cheaper than it sounds. YOLO had already proposed every box; my verdict was the only missing piece. An approved detection becomes a positive example at coordinates the model itself supplied. A rejected one becomes a hard negative — the same image with that box deliberately left out, which is how you teach a model that a backlit jacaranda branch is not a bicycle.

🔧 Developer section: Building the training set

• Ran .venv/bin/python tools/export_dataset.py --out data/dataset --neg-ratio 2 to turn the verdict table into a YOLO dataset on disk
• none is not a class — it writes an empty label file, which is exactly how a detector learns a hard negative
• Split by event, never by frame (SEED = 1337, --val-frac 0.2). Near-duplicate frames straddling the split make the score fiction
• Capped negatives 2:1. Raw verdicts run about 1809 none against 227 bicycle, and unbalanced the model just learns nothing is ever a bike
• Auto-resolve wrote a verdict but no label, so label IS NOT NULL silently excluded 1,115 rows — the per-class counts printout caught it

The single most consequential setting turned out to be image size. YOLO trains at 640 pixels by default, and my detections are small: a typical two-wheeler box on this camera is around 73 pixels across in a 1920x1080 frame. Resize that frame down to 640 and the box collapses to roughly 24 pixels — right at the floor of what the model can resolve. At that size the wheels, the frame and the rider are all gone, and what is left of a bicycle is a smudge that looks exactly like what is left of a branch. That is a large part of why the two score so similarly, and why the fine-tune trains at 1280.

🔧 Developer section: Running the fine-tune

• Ran .venv/bin/python tools/finetune.py --epochs 60 --imgsz 1280 on a yolov8s.pt base, batch 4, via Ultralytics
• Trained on mps (Apple silicon) with a cpu fallback — no cloud GPU, no rented instance
• Geometric augmentations off: fliplr=0.0, flipud=0.0, degrees=0.0. A fixed camera never sees a mirrored scene
• Kept mosaic=0.3 and the HSV jitter (hsv_s=0.6, hsv_v=0.5), because lighting really does run from bright noon to headlit night
• The two mAP figures are not comparable — stock COCO puts bicycle at index 1, my dataset at index 0. The confusion counts are the verdict

The Results

On the held-out split the fine-tuned model scored 85.7% precision against stock’s 69.1%, cutting false positives from 25 to 9. Recall dipped, 91.8% to 88.5%. Both models were scored on the same events — a model that only looks good against its own training distribution has told you nothing.

I want to be straight about the recall dip, because it would be easy to bury. Recall went down, 91.8% to 88.5% — the fine-tuned model misses slightly more real bikes than the stock one does. That is the trade, and I took it knowingly: at 25 false positives per split, the bicycle count on my dashboard was not a measurement, it was a rumor. Three points of recall for sixteen points of precision turns a number I could not cite into one I can.

The most reassuring result, though, was neither headline number: the improvement held across confidence thresholds. A fine-tune that only wins at one cutoff is a threshold in a trench coat, hiding the errors rather than resolving them. Stable across the range means the model genuinely learned the difference.

Shipping It: The Model That Only Knows Six Things

I deployed the fine-tuned weights as an ensemble rather than a replacement: it supplies bicycle and motorcycle boxes, stock YOLOv8s supplies everything else, and the pipeline falls back to stock entirely if the weights go missing.

Which brings me back to the hot dog app. My fine-tuned model was trained on its six sidewalk classes and nothing else — every other head stock YOLOv8s ships with (person, car, dog) simply is not in there. So it does not replace anything, it joins. Swapping it in wholesale would have silently erased every non-two-wheeler detection in my catalog — frightening precisely because nothing would have crashed.

🔧 Developer section: Wiring the ensemble

• Set twowheeler_model: models/runs/sidewalk/weights/best.pt in config.yaml — relative, so it resolves on either machine
• get_twowheeler_model() returns None when the file is absent, so missing weights degrade to stock instead of throwing
• In _detect() stock runs first, then bicycle and motorcycle boxes are stripped from its results and re-supplied at imgsz=1280
• The fine-tuned model’s scooter, stroller, delivery_robot and other boxes are dropped, not passed through — they are what used to be counted as cyclists
• Both models share a MODEL_LOCK; ultralytics predict is not thread-safe and classify runs beside the live counter

Two things it actually fixed

• Event #6133 — the tail-light ghost. A parked car’s tail light the old rules kept reading as a two-wheeler. They could only kill it after three repeat sightings, so it polluted the count in the meantime, every time. The fine-tuned model rejects it on first sighting.
• Event #5128 — corrected from 2 bicycles to 1. One real cyclist, and a jacaranda branch in the same clip counted as a second bike.

What went fast

• The grid search. With labeled verdicts already in a table, scoring 48 parameter sets was one script and a coffee — cheap enough that there was no excuse not to run it.
• Building the dataset. Because YOLO had already proposed every box, the export was pure bookkeeping: read the verdicts, pull the frame each came from, write a label file. No annotation tool, no drawing rectangles.
• The training run itself. I expected the fine-tune to be the intimidating part; it was the easiest — one library call, then walk away.

What needed patience

• Accepting the ceiling. I did not want the grid search to say what it said. Tuning is comfortable; fine-tuning felt bigger than I had signed up for. Concluding “this is the wrong tool” instead of “I need a finer sweep” took longer than the sweep did.
• The labeling. There is no clever way around it. Somebody has to look at hundreds of crops and say what each one is, one click at a time — the least glamorous part of the project and the only part that moved the number.
• Catching the 1,115 missing rows. No error, no visible symptom. I only found it tracing why a label column existed at all.

The most valuable output of this whole project was a negative result. Forty-eight parameter sets, none better than 70.7%, and that flat line is what told me to stop tuning and go label my own data. If your off-the-shelf model almost works, I would do it in the same order: get real labels, sweep your parameters against them, and look hard at whatever errors survive. If those errors are things your rules cannot even describe, no threshold will save you — and you have just saved yourself a month of turning knobs. Then go do the boring part, because that is the part that works.