Task
Typical smart lighting systems rely on buttons or a mobile app with fixed commands. The task: build a prototype where lights can be controlled in several natural ways, through a Telegram bot by text and voice, with gestures in front of a camera, by the user's emotion and through a web interface, with commands reaching ESP32 and ESP8266 controllers in real time.
What we did
Results
What we did
We designed the architecture and built a prototype with several input channels. The Telegram bot accepts text or a voice message; voice is converted to text with Speech-to-Text, then OpenAI-based logic determines the lighting command. The command is published to an MQTT topic, an ESP32 or ESP8266 controller receives it, switches the LED lighting and reports its status back to a feedback topic. A separate web server provides HTTP control: turn on, turn off, refresh device state. The computer vision module tracks 32 body keypoints and 21 hand keypoints with BlazePose and MediaPipe models, so a raised hand or a specific finger position becomes a command. There is also an emotional layer: the user picks an emotion with an emoji or sends a photo, and the AI classifies the emotion.
Approach and results
The working prototype shows how AI extends a conventional IoT system: one solution combines a Telegram bot, speech recognition, MQTT and HTTP, embedded controllers and computer vision. Lighting control moves beyond buttons and mobile apps and becomes natural: by voice, by gesture or according to mood. Stack: Python, Telegram Bot API, OpenAI-based AI logic, Speech-to-Text, MQTT and HTTP protocols, ESP32 and ESP8266, pose and hand detection with BlazePose and MediaPipe, a web server for device control.