A camera that records everything is a liability. AegisVision turns a CCTV feed, webcam or RTSP stream into something that answers questions about a space, and it does that with a deliberate rule: people it does not recognise are tagged on the frame and never written to disk.
Repo: swadhinbiswas/AegisVision
The frame pipeline
One background capture worker runs per configured camera and shares the latest frame with back-pressure and automatic reconnect. Each frame moves through a fixed pipeline: detect, track, check liveness, embed, match.
Detection uses SCRFD-500M through ONNX, small enough to run in under ten milliseconds on CPU. An IoU tracker keeps an identity across frames and only re-runs the embedding when a track window elapses, which is the difference between a model that keeps up and one that does not. A liveness gate checks face texture so a printed photo or a phone screen scores lower, then ArcFace produces a 512-dimensional aligned embedding, and a top-N cosine lookup happens in Chroma.
The backend is swappable without code changes. FACE_BACKEND=auto prefers insightface and falls back to dlib if the package or model is unavailable, and the vector index repairs itself when the embedding dimension changes from 128 to 512.
Two stores, one truth
Relational rows in SQLite are the source of truth. The Chroma vector index mirrors person_id to embedding so a person can own many photos as they age, change angle or get re-enrolled. Enrollment is a folder drop: create data/known_persons/<person_id>_<Name>/ with an info.json and photos, and the person is live in minutes through a startup sync and a folder watchdog.
What it actually watches
Beyond recognition, the behavioral engines consume identities and liveness to produce gate in and out events with a 30-second cooldown, live occupancy, per-person active, idle-lazy, bunking and phone-distraction minutes, and drowsiness signals. A spatial memory tracks registered assets against their owner and raises a theft suspicion when something moves. Rules escalate on repeated unknowns, repeated phone use or repeated drowsiness, and the alert goes out over WebSocket to the dashboard plus a Telegram or JSON webhook.
There is a Generative-AI copilot that answers plain-English questions over the live database, with a rule-based fallback when no API key is set.
How it deploys
Docker Compose puts NGINX in front of uvicorn with persistent volumes for SQLite, ChromaDB, the roster and logs. One host runs the whole thing, and cameras are added by ID. The test suite has 51 passing tests.
The design decision I would defend hardest is the unknown-face rule. Recognition systems accumulate data by default, and most of it is people who never consented. Making the default "tag and move on" forced the rest of the system to be explicit about what it stores, which made the privacy story clearer and the storage footprint smaller.
The diagrams, the data model and the deployment topology are in the repository above.
Reach me at swadhinbiswas.cse@gmail.com or on GitHub and LinkedIn.
No comments yet.