deerwork-ai
Featured
deer-workflow
An open-source Dynamic Workflow runtime that combines deterministic TypeScript orchestration with replaceable Agent runtimes.
Intelligent-Voice-Driving-System
An in-vehicle driving assistance system that integrates computer vision for driver state monitoring, voice-interactive control via a local Qwen-7B LLM, and safety scoring with automated handover alerts.
Stars
10
7-day growth
No data
Forks
0
Open issues
0
License
MIT
Last updated
2026-07-27
AI repository intelligence
FR-AI / ANALYSISWhy it is worth attention
It combines multiple real-time monitoring modalities—fatigue, distraction, smoking, phone use—with a locally deployed large language model for voice commands and conversational AI, all in a single open-source project.
Who it is for
- Developers building intelligent cockpit or driver assistance systems
- Researchers in driver behavior analysis and human-computer interaction
- Automotive enthusiasts and DIY car hackers
- Fleet safety managers seeking a customizable monitoring prototype
Use cases
- Personal driving assistant that provides fatigue alerts and safety scoring
- Prototype for fleet driver risk analysis and violation logging
- Educational tool to demonstrate vision-LLM integration in automotive contexts
- Testbed for evaluating local LLMs in real-time voice-controlled applications
Strengths
- Comprehensive driver state detection covering fatigue, distraction, smoking, and phone use via MediaPipe and YOLOv8
- Local LLM deployment (Qwen-7B via Ollama) ensures privacy and offline voice interaction
- Face authentication with LBPH recognition and fallback password for access control
- Automatic trip summary and safety score review generated by the AI co-pilot
Considerations
- Requires a webcam, microphone, and local Ollama server with the Qwen-7B model, which may be resource-intensive
- Voice commands and UI are primarily in Chinese, limiting language accessibility
- Dependency on specific model weights (yolov8_smoking.pt, yolov8n.pt) not included in the repo (must be obtained separately)
README quick start
🚗 Intelligent Voice Driving System An intelligent in-vehicle driving assistance and real-time monitoring system that integrates computer vision, multimodal behavior monitoring, voice-interactive control, and a local large language model (Qwen-7B).
✨ Features 🎙️ Pure Voice-Interactive Control
Background microphone listening allows voice commands to control camera states, toggle safety monitoring, and query safety scores.
Powered by a local Qwen-7B large language model (via Ollama), it handles system commands and acts as an intelligent AI co-pilot for casual conversation and relief from driving fatigue.
👀 Comprehensive Driver State Monitoring (MediaPipe + YOLOv8)
Fatigue Monitoring: Calculates Eye Aspect Ratio (EAR) to track drowsiness and Mouth Aspect Ratio (MAR) to capture yawning.
Distraction Detection: Utilizes gaze ratio tracking and head pose yaw to detect looking away or wandering attention.
Dangerous Habit Recognition: Combines YOLOv8 models to precisely detect smoking and mobile phone usage while driving.
Head Posture Detection: Monitors hazardous driving postures such as lowering the head due to fatigue.
🤖 Automated Driving Handover Simulation
Triggers emergency alerts and simulates automated driving intervention when extreme fatigue (such as prolonged eye closure or heavy head dropping) exceeds set thresholds.
👤 Facial Authentication & Access Control
Built-in LBPH face recognition automatically verifies the owner's identity upon starting safety monitoring, with password authorization fallback for unrecognized users.
📊 Safety Scoring & Intelligent Trip Summary
Real-time score deduction system (out of 100 points) with timestamped violation logs. Upon trip completion, the AI co-pilot automatically generates a detailed review and professional evaluation of the driving session.
🛠️ Tech Stack GUI Framework: PySide6 (Qt for Python)
Computer Vision & Tracking: OpenCV, MediaPipe (Holistic face, hand, and pose landmarks)
Object Detection: Ultralytics YOLOv8 (for smoking and phone detection)
Voice Technologies: SpeechRecognition (ASR), pyttsx3 (Offline TTS)
Large Language Model: Ollama + Qwen-7B (Local deployment ensuring privacy and offline capability)
📂 Project Structure Plaintext Intelligent-Voice-Driving-System/ │ ├── a1.py # Main program entry (UI logic, vision threads, ASR and LLM scheduling) ├── testui.py
Description
This is a real-time driver state monitoring system with pure voice control for smart vehicles, equipped with the Qwen-7B large language model, capable of AI conversation and fully controlled by AI throughout the entire process.
Related repositories
Similar projects matched by category, topics, and programming language.
slvDev
Featured
esp32-ai
A 28.9 million parameter language model runs on an $8 ESP32-S3 microcontroller entirely on-device, generating simple stories at about 9.5 tokens per second.
gavamedia
Featured
deltafin
Deltafin is a research project that runs the 2.8-trillion-parameter Mixture-of-Experts model Kimi K3 on a single Apple Silicon Mac (e.g., M1 Max with 64 GB) at about 16 seconds per token, using exact, reproducible inference with local or streaming expert loading.