Smooth before acting
A default 50-frame buffer and strict-majority rule stabilize predictions. Cooldown and no-face handling reduce abrupt changes and define behavior when observations disappear.
Computer vision · IoT / Hardware experiment
Sentient Ambient AI
An experimental desktop-to-ESP32 pipeline that smooths facial-expression model outputs before changing lighting and music.
01 / THE PROBLEM
Frame-by-frame classifier outputs can flicker. A physical lighting system needs a more stable signal and a clear boundary between the computer performing inference and the microcontroller driving LEDs.
02 / FOLLOW THE WORKFLOW
A simplified example of the documented flow. Sample outputs are authored illustrations, not live model predictions.
Conceptual flow, no camera access
Camera → facial-expression classifier
A webcam frame enters the vision pipeline.
Default smoothing configuration
50-frame buffer → strict majority
Require agreement over recent predictions.
Command boundary
Python → Serial or HTTP → ESP32
Send a stable label when it changes.
Output driven by the model label
Ambient light + local audio
Blend LEDs and adjust host-side music.
Step 1 of 4 · Observe
03 / ENGINEERING
A default 50-frame buffer and strict-majority rule stabilize predictions. Cooldown and no-face handling reduce abrupt changes and define behavior when observations disappear.
Python handles camera input and DeepFace. The ESP32 receives a compact command over Serial or optional HTTP and smoothly blends a WS2812B strip using NeoPixel.
Music plays on the laptop, locally or through optional Spotify integration. An optional per-user classifier can be trained on DeepFace embeddings.
04 / EVIDENCE & LIMITS
The README establishes hardware wiring, command paths, configurable smoothing and optional training. The numbers describe configuration; they do not measure emotion-recognition accuracy or a wellbeing outcome.
Facial expressions do not reliably establish someone’s internal emotional state. This is an experimental interaction, not a psychological assessment. Robust evaluation would include false switches, lighting changes, no-face behavior and user control.
05 / READ THE SOURCE
Based on public repository documentation reviewed on 7 November 2026. Results are attributed to the project’s recorded evaluations; they were not rerun for this portfolio.
TOOLS BEHIND THE WORK
Explore the stack. Drag the ribbon or select a tool.