csg
Résumé ↗
Menu
← All projects

Computer vision · IoT / Hardware experiment

Sentient Ambient AI

From a camera signal to ambient light.

An experimental desktop-to-ESP32 pipeline that smooths facial-expression model outputs before changing lighting and music.

Python · DeepFace · ESP32 · NeoPixel

View repository ↗

01 / THE PROBLEM

What needed solving.

Frame-by-frame classifier outputs can flicker. A physical lighting system needs a more stable signal and a clear boundary between the computer performing inference and the microcontroller driving LEDs.

02 / FOLLOW THE WORKFLOW

See how it fits together.

A simplified example of the documented flow. Sample outputs are authored illustrations, not live model predictions.

Sentient Ambient AIIllustrative workflow

Conceptual flow, no camera access

Camera → facial-expression classifier

A webcam frame enters the vision pipeline.

Default smoothing configuration

50-frame buffer → strict majority

Require agreement over recent predictions.

Command boundary

Python → Serial or HTTP → ESP32

Send a stable label when it changes.

Output driven by the model label

Ambient light + local audio

Blend LEDs and adjust host-side music.

03 / ENGINEERING

The decisions behind it.

Smooth before acting

A default 50-frame buffer and strict-majority rule stabilize predictions. Cooldown and no-face handling reduce abrupt changes and define behavior when observations disappear.

Split inference from actuation

Python handles camera input and DeepFace. The ESP32 receives a compact command over Serial or optional HTTP and smoothly blends a WS2812B strip using NeoPixel.

Keep audio on the host

Music plays on the laptop, locally or through optional Spotify integration. An optional per-user classifier can be trained on DeepFace embeddings.

04 / EVIDENCE & LIMITS

Claims with context.

50 framesDefault stabilization window
7 labelsConfigured expression categories

The README establishes hardware wiring, command paths, configurable smoothing and optional training. The numbers describe configuration; they do not measure emotion-recognition accuracy or a wellbeing outcome.

Current limits & next validation

Facial expressions do not reliably establish someone’s internal emotional state. This is an experimental interaction, not a psychological assessment. Robust evaluation would include false switches, lighting changes, no-face behavior and user control.

05 / READ THE SOURCE

Go deeper.

Based on public repository documentation reviewed on 7 November 2026. Results are attributed to the project’s recorded evaluations; they were not rerun for this portfolio.

NEXT PROJECTStock Prediction Model

TOOLS BEHIND THE WORK

Explore the stack. Drag the ribbon or select a tool.