Human Action Recognition System

Video surveillance and industrial monitoring generate enormous amounts of footage, most of which is never reviewed. Human Action Recognition changes this: instead of recording and waiting for someone to watch the tape, the system classifies what's happening in real time and flags events that matter.

Golden Sea's HAR pipeline is three-stage. First, YOLOv8 detects and localizes every person in each video frame. Second, ByteTrack assigns persistent identity IDs across frames — even when people are temporarily occluded or move quickly, the tracker maintains consistent identity. Third, skeletal keypoints are extracted per tracked person using MediaPipe, then passed to a Spatial-Temporal Graph Convolutional Network (ST-GCN) that classifies the action based on body joint movement patterns over time.

The ST-GCN approach classifies actions by analyzing the geometric relationships between joints (shoulder-elbow-wrist angles, torso rotation, stride pattern) across a temporal window. This is more robust than frame-by-frame image classification because it captures motion, not just pose snapshots.

The system classifies 20+ action categories relevant to the deployment context: falling, fighting, running, crouching, operating machinery, carrying objects, idle standing. False positive rates are tuned per category — a fall detection model is tuned for high recall (never miss a fall); fighting detection is tuned for high precision (don't alarm unnecessarily).