Dairy farm: automatic detection of key events, 24/7

TL;DR. Three stages: a dataset pipeline, ongoing support, and automatic detection of key behavioural events. More than 50 cameras, four per pen, bird's-eye view, infrared mode at night. Target accuracy of 83-100% with a warning 1-9 hours ahead. Deployed at the edge on an NVIDIA Jetson Orin NX, so it runs without the cloud.

For how this works under the hood, see the computer vision monitoring guide.

Situation

Catching key behavioural events in cows meant staff on watch around the clock. Night shifts, fatigue, human error — misses were inevitable, and a missed event costs money. The farm needed a system that spots the signs before the event and warns the herd manager in advance.

Task

1. Build a dataset preparation pipeline — a repeatable process for fine-tuning models.
2. Detect key behavioural events automatically through computer vision and give the herd manager an early warning.

What we built

Stage 1. Dataset pipeline. Video → frames → annotation → training set. Quality filtering with CLIP, which discards blurred and irrelevant frames. Annotation managed in Roboflow. The process is reproducible: adding a new scenario doesn't mean rebuilding the pipeline.

Stage 2. Support. Monthly maintenance of the running system: monitoring, fine-tuning for new scenarios, corrections.

Stage 3. Automatic event detection. YOLOv11 for object detection, RTMPose for pose analysis (17 keypoints), and a BiLSTM with attention for temporal behaviour patterns — how often the animal lies down and gets up, tail raising, udder changes. A 4-camera system with a bird's-eye projection, plus a separate branch for infrared at night.

Result

• Automatic event detection gives the herd manager an early warning.
• Target model accuracy is 83-100%, 1-9 hours before the event.
• Deployed at the edge on an NVIDIA Jetson Orin NX: the system runs on the farm without the cloud, so connectivity doesn't matter.
• Three stages delivered by 2026-05.

Key technical decisions

1. A hybrid approach to behaviour. One model can't detect key behavioural events in real time on its own. YOLO gives object detection in frame, RTMPose gives pose, the BiLSTM gives the temporal pattern. The decision comes from the combination, not from any single signal.
2. Night vision as its own branch. Models trained on daytime footage degrade under infrared. We fine-tuned a separate branch on IR frames — without it, accuracy drops at night, which is when most key events happen.
3. Edge deployment. Moving to an NVIDIA Jetson Orin NX means the system runs on the farm without the cloud. Connectivity at remote sites is unreliable, so the model and inference travel to the data rather than the other way round.

FAQ

Why is accuracy given as a range of 83-100% rather than one number?

Accuracy depends on the type of event: some behavioural patterns are pronounced and recognised confidently, others are subtler and sit closer to the signal threshold. The range shows the honest spread between event types, instead of an average that tells you nothing about your specific case.

What if a camera can't see the animal — angle, lighting, crowding?

A system of four cameras per pen with a bird's-eye projection reduces the chance of losing a frame to a single viewpoint. Night infrared runs on a separately fine-tuned branch of the model, not the same weights applied to different lighting.

Can a new behavioural event be added without rebuilding the system?

Yes. The dataset pipeline and the architecture (YOLOv11 + RTMPose + BiLSTM) are reused — a new event means annotating a new dataset and fine-tuning, not building a system from scratch.

What would come next

The logical continuation is extending behavioural analytics to other events: oestrus, illness (mastitis, lameness), feeding. Same camera infrastructure (four per pen), same dataset pipeline.

If you run livestock, greenhouses, a warehouse or a factory and need something tracked automatically on video, let's talk it through in 30 minutes.