Multi-Camera Person Re-ID & Counting System
Cross-camera identity matching with no double-count · YOLOv8-Pose + OSNet ReID (512-dim cosine) + BoTSORT + real-time FastAPI/WebSocket dashboard
2024 – 2025
512-dim
ReID Features
17 pts
Pose Keypoints
3000 frames
Identity Memory
Gradio + WebSocket
Interfaces
Problem
Retail and venue operators need accurate headcounts across multi-camera zones — but naive per-camera counting double-counts anyone who moves between camera views, making the numbers meaningless. The core challenge: assign a single global identity to each person across N live camera streams, even when they disappear from all feeds for hundreds of frames and reappear with different lighting, scale, or pose. Simultaneously, the system must count each unique person crossing a virtual entry/exit line — without counting the same person twice across cameras.
Approach
Designed a full end-to-end multi-camera AI pipeline: YOLOv8-Pose detection → BoTSORT multi-object tracking (per camera) → OSNet ReID feature extraction → two-phase cross-camera global ID assignment → geometric line-crossing counter → FastAPI + WebSocket real-time dashboard.
Applied YOLOv8m-Pose for detection, extracting both bounding boxes and 17 pose keypoints per person per frame — enabling downstream pose-quality filtering before any identity is committed.
Enforced strict pose-based quality gates before ID assignment: requires ≥ 12 body keypoints at confidence > 0.8, at least 1 valid head keypoint, and bounding box height > 1/3 of frame height — blocking partial, distant, or occluded detections from corrupting cross-camera identity.
Extracted 512-dimensional identity feature vectors per person using OSNet (pre-trained on DukeMTMC + Market-1501 ReID benchmarks) — L2-normalized and stored as a rolling deque of 10 features per track for robust median-of-top-5 cosine similarity scoring.
Built two-phase cross-camera matching: Phase 1 matches against globally active IDs in the current frame (real-time synchronization, threshold 0.6) — Phase 2 falls back to a 1000-deep historical feature bank with 3000-frame identity memory (threshold 0.55) — handling temporal gaps when a person exits all camera views and reappears later.
Implemented same-camera re-identification (threshold 0.8) to recover track continuity after occlusion within a single view, before cross-camera matching is attempted — producing stable local track IDs fed to global matching.
Built geometric line-crossing counter using cross-product sign change: user interactively defines a 2-point virtual line via mouse GUI (scales between display and original resolution); system records per-person side history and increments IN/OUT counters on crossing — aggregated live across all cameras.
Deployed two interface modes: (1) Gradio web UI for standalone demo — processes 2 video files simultaneously, outputs H.264 annotated video; (2) FastAPI + WebSocket backend streaming annotated frames and live stats (unique count, IN/OUT totals, FPS per camera) to a React/TypeScript dashboard in real time.
Results
Eliminates cross-camera double-counting: each person receives one global ID regardless of how many camera zones they traverse, giving accurate unique-person headcounts.
Pose-quality gates prevent low-confidence detections from corrupting global IDs — improving identity precision on crowded or partially occluded frames.
3000-frame identity memory enables reliable re-identification after extended off-camera gaps, recovering the correct global ID without creating a duplicate.
Two-phase matching (real-time sync + historical fallback) handles both simultaneous multi-camera appearances and asynchronous re-entries — covering the full range of real-world movement patterns.
Fully deployable in two modes: Gradio demo for offline batch processing and real-time FastAPI/WebSocket dashboard for live multi-stream monitoring.
Architecture Overview
Per camera: YOLOv8-Pose
bounding box + 17 keypoints
pose quality gate
BoTSORT local track
OSNet 512-dim feature (rolling deque-10) | Global: Phase 1 active-frame cosine match (threshold 0.6)
Phase 2 historical bank match (threshold 0.55, 3000-frame memory)
new global ID if no match | Counting: cross-product line-side test
IN/OUT delta on sign change | Interfaces: Gradio (offline) · FastAPI + WebSocket
React dashboard (live)
How to Run
Install dependencies: pip install -r requirements.txt (PyTorch + CUDA, ultralytics, boxmot, torchreid, fastapi, gradio)Place model weights: yolov8m.pt and osnet_x1_0_duke_256x128_amsgrad_ep150.pth in project root.CLI mode (batch video): python count_mreid.py --videos cam1.mp4 cam2.mp4 --output-dir ./outputAfter launch, an interactive window opens for each camera — click two points to define the counting line (IN side = left of line direction).Gradio demo: python app.py — opens at http://localhost:7860, upload two video files, system processes and returns annotated videos.Web dashboard: cd UI_Reid_Model && uvicorn main:app --reload (backend on port 8000) + cd frontend && npm run dev (React on port 5173).Dashboard streams live annotated frames via WebSocket and shows real-time IN/OUT counts and unique person totals per camera.