Motorola-Dream-Machine

<<<<<<< HEAD

Motorola-Dream-Machine

Dream machine project from Motorola grant, using an EEG headset to process brain signals in order to function robotic parts such as an arm. This project is an exploration of real-time brain-computer interfaces.

EEG Pipeline Architecture

The core of this project is a real-time, streaming pipeline that uses Lab Streaming Layer (LSL) and Kafka to process live EEG data.

Components

LSL Data Source (External):
- This is not part of the repository but is a prerequisite. It can be:
  - Real EEG Hardware: An EEG amplifier with acquisition software (e.g., OpenBCI GUI, BrainVision Recorder) that broadcasts its data onto the local network via an LSL stream.
  - Simulated Stream: A tool like OpenViBE can be used to stream an existing .edf file as a live LSL stream for development and testing.
Producer (eeg_pipeline/producer/producer.py):
- Architectural Role: Acts as a bridge between the LSL network and the Kafka messaging system.
- It continuously searches the local network for an LSL stream of type 'EEG'.
- Once a stream is found, it connects, pulls data chunks in real-time, and forwards each individual EEGSample to a Kafka topic.
- It no longer reads from files; it is a pure real-time listener.
Consumer (eeg_pipeline/consumer/consumer.py):
- Architectural Role: The real-time analysis engine.
- It subscribes to the Kafka topic (raw-eeg by default) and consumes the EEGSample stream.
- It dynamically windows the incoming data and calculates the Power Spectral Density (PSD) for each window using Welch's method.
- Upon termination (Ctrl-C), it saves summary analysis artifacts (JSON data and PNG plots) to the root directory.
Kafka:
- The robust, high-throughput message bus that decouples the LSL producer from the analysis consumer.
- A simple docker-compose.yml is provided in eeg_pipeline/config/ to run Kafka.

How to Run the Real-Time Pipeline

The workflow now mirrors a typical real-time BCI experiment.

1. Start an LSL Stream Source

This is the most critical step. The producer is a listener, so something must be broadcasting data. You have two options:

With Real Hardware: Start your EEG amplifier and its acquisition software. Find the setting to enable LSL streaming.
For Development: Use a tool like OpenViBE to create a simple scenario that reads an .edf file and streams it to the network using an LSL output block.

2. Start Kafka Services

This is the same as before. Make sure Docker is running.

cd eeg_pipeline/config
docker-compose up -d

3. Run the Real-Time Producer

Activate your Python virtual environment. The producer no longer takes a file path; it automatically discovers the LSL stream.

# Activate your environment
source eeg_pipeline/venv/bin/activate

# Run the producer
python eeg_pipeline/producer/producer.py --bootstrap-servers localhost:9092

The script will print "Looking for an EEG stream via LSL..." and wait. Once your LSL source is broadcasting, it will connect and begin forwarding data to Kafka.

4. Run the Consumer

In a separate terminal, activate the environment and run the consumer. It will process the live data from Kafka.

# Activate your environment
source eeg_pipeline/venv/bin/activate

# Run the consumer
python eeg_pipeline/consumer/consumer.py --topic raw-eeg --bootstrap-servers localhost:9092 --write-json --write-png

The consumer will now perform continuous analysis on the live data. When you are finished, stop the producer and consumer with `Ctrl-C`. The consumer will save its final analysis files upon termination.

Motorola Dream Machine

Brain-Controlled Robot Arm System
Using EEG signals and AI to control robotic movement

🎯 Project Overview

The Motorola Dream Machine is a complete brain-computer interface (BCI) system that enables direct control of a robot arm using brain signals from an Emotiv EEG headset. An AI model (EEG2Arm) interprets motor imagery and intentions from EEG data to generate robot commands in real-time.

Grant: Motorola Innovation Project
Status: ✅ Fully Integrated - Ready for Hardware Testing
Tech Stack: Python, PyTorch, Kafka, Docker, LSL, Emotiv

⚡ Quick Start

One-Command Test (Recommended)

cd eeg_pipeline
./run_integration_test.sh

This runs the complete pipeline with sample data (no hardware required).

Manual Setup

See QUICK_START_GUIDE.md for detailed instructions.

🏗️ System Architecture

Emotiv EEG Headset → LSL → Kafka → AI Model → Robot Commands → Robot Arm

Key Components:

EEG Acquisition: Emotiv Flex 2.0 (up to 32 channels @ 256 Hz)
Streaming: Apache Kafka for real-time messaging
AI Inference: EEG2Arm model (CNN + GCN + Transformer)
Robot Control: UR/KUKA arms with safety features
Latency: ~100-250ms end-to-end ✅

See ARCHITECTURE_DIAGRAM.md for detailed flow.

📂 Project Structure

Motorola-Dream-Machine/
├── eeg_pipeline/              # Main EEG processing pipeline
│   ├── producer/              # Data acquisition (Emotiv, LSL, files)
│   │   ├── emotiv_producer.py        # ✨ Emotiv-specific integration
│   │   ├── live_producer.py          # Generic LSL producer
│   │   └── producer.py               # File-based producer
│   ├── consumer/              # EEG data analysis
│   │   └── consumer.py               # Band power analysis
│   ├── ai_consumer/           # ✨ AI inference layer
│   │   └── ai_consumer.py            # Real-time EEG→Command prediction
│   ├── analysis/              # Signal processing utilities
│   │   ├── bands.py                  # Frequency band analysis
│   │   └── plotting.py               # Visualization
│   ├── schemas/               # Data models (Pydantic)
│   │   ├── eeg_schemas.py            # EEG sample formats
│   │   └── robot_schemas.py          # ✨ Robot command formats
│   ├── config/                # Kafka/Docker configuration
│   ├── integrated_robot_controller.py # ✨ Universal robot control
│   ├── kuka_eeg_controller.py        # KUKA-specific controller
│   └── hardware_test.py              # Hardware detection tools
├── model/                     # AI model
│   ├── eeg_model.py                  # EEG2Arm architecture
│   └── train_eeg_model.py            # ✨ Training pipeline
├── ursim_test_v1/             # UR robot simulation tests
├── COMPREHENSIVE_REVIEW.md    # ✨ Detailed system analysis (30+ pages)
├── QUICK_START_GUIDE.md       # ✨ Step-by-step setup instructions
├── IMPLEMENTATION_SUMMARY.md  # ✨ What was built and why
└── ARCHITECTURE_DIAGRAM.md    # ✨ Visual system architecture

✨ = New files created during integration

🚀 Features

✅ Implemented

⚠️ Needs Testing/Training

Emotiv hardware testing
AI model training with real data
Physical robot integration
Long-term stability testing

🧠 AI Model: EEG2Arm

Architecture: 3D CNN + Graph Convolution + Transformer

Input:  (Batch, 32 channels, 5 bands, 12 frames)
Output: (Batch, 5 classes) → [REST, LEFT, RIGHT, FORWARD, BACKWARD]

Performance:

Parameters: ~500K trainable
Inference: 10-50ms (CPU), 2-10ms (GPU)
Target Accuracy: >70% (4-class motor imagery)

Training:

python model/train_eeg_model.py --epochs 50 --device cuda

🤖 Supported Robots

Robot Type	Status	Interface	Notes
Mock	✅ Working	N/A	For testing without hardware
UR (Universal Robots)	✅ Ready	ur-rtde	Tested with URSim
KUKA	⚠️ Partial	Custom SDK	Needs KUKA library integration

📊 Performance Benchmarks

Metric	Value	Status
EEG Sampling Rate	256 Hz	✅
AI Prediction Rate	0.5-1 Hz	✅
End-to-End Latency	100-250ms	✅
Inference Time (CPU)	10-50ms	✅
Inference Time (GPU)	2-10ms	✅
Command Throughput	1-5 commands/sec	✅

📖 Documentation

Document	Description
COMPREHENSIVE_REVIEW.md	Complete system analysis, gaps, roadmap (30+ pages)
QUICK_START_GUIDE.md	Setup instructions, troubleshooting, examples
IMPLEMENTATION_SUMMARY.md	What was built, why, and how to use it
ARCHITECTURE_DIAGRAM.md	Visual architecture and data flow
FIRST_TIME_USER_GUIDE.md	Emotiv → KUKA setup for beginners

🛠️ Installation

Prerequisites

Python 3.8+
Docker Desktop
Emotiv headset (optional for testing)

Setup

# 1. Clone repository
git clone <repo-url>
cd Motorola-Dream-Machine/eeg_pipeline

# 2. Create virtual environment
python3 -m venv venv
source venv/bin/activate  # Windows: venv\Scripts\activate

# 3. Install dependencies
pip install -r requirements.txt

# 4. Install LSL (for live EEG)
# macOS: brew install labstreaminglayer/tap/lsl
# Linux: sudo apt-get install liblsl-dev
# Windows: Download from https://github.com/sccn/liblsl/releases

# 5. Start Kafka
cd config
docker compose up -d
cd ..

# 6. Test the system
./run_integration_test.sh

🧪 Testing

Automated Integration Test

cd eeg_pipeline
./run_integration_test.sh

What it tests:

Kafka infrastructure
EEG data streaming
AI model inference
Robot command generation
End-to-end pipeline

Expected output:

Pipeline Status:
  📡 EEG Producer:       Running
  🧠 AI Consumer:        Running
  🤖 Robot Controller:   Running

[Prediction #0001] Command: LEFT     (confidence: 0.654)
⬅️  LEFT (confidence: 0.65)

Manual Testing

See QUICK_START_GUIDE.md for step-by-step manual testing.

🔧 Configuration

Safety Limits

Edit in integrated_robot_controller.py:

SafetyLimits(
    max_velocity=0.2,          # m/s
    max_acceleration=0.5,      # m/s²
    min_confidence=0.6,        # 0-1
    command_timeout_ms=2000,   # ms
    workspace_min=[-0.5, -0.5, 0.0, -3.14, -3.14, -3.14],
    workspace_max=[0.5, 0.5, 0.5, 3.14, 3.14, 3.14]
)

Emotiv Channels

14-channel (EPOC/Insight):

['AF3', 'F7', 'F3', 'FC5', 'T7', 'P7', 'O1',
 'O2', 'P8', 'T8', 'FC6', 'F4', 'F8', 'AF4']

32-channel (Flex):

['AF3', 'AF4', 'F7', 'F3', 'F4', 'F8', 'FC5', 'FC1',
 'FC2', 'FC6', 'T7', 'C3', 'C4', 'T8', 'CP5', 'CP1',
 'CP2', 'CP6', 'P7', 'P3', 'Pz', 'P4', 'P8', 'PO3',
 'PO4', 'O1', 'O2', 'AF7', 'AF8', 'Fp1', 'Fp2', 'Fz']

🐛 Troubleshooting

Emotiv Not Detected

python hardware_test.py --check-streams

Solutions:

Ensure EmotivPRO/BCI is running
Enable LSL in Emotiv software settings
Check headset battery and connection

Poor Signal Quality

Check impedance:

Should be < 20kΩ
Apply saline solution to sensors
Ensure good skin contact

Kafka Connection Failed

docker ps  # Check if Kafka is running
cd config
docker compose restart

📝 Known Issues

Untrained Model: AI model currently has random weights (demo only)
- Solution: Train with real data (see QUICK_START_GUIDE.md)
KUKA Integration Incomplete: Only mock mode works
- Solution: Install KUKA SDK and update KUKARobot class
Import Errors: Missing dependencies
- Solution: pip install -r requirements.txt

🎯 Roadmap

Phase 1: Hardware Testing (Week 1-2) ⏳

Test Emotiv Flex 2.0 connection
Validate signal quality
Verify LSL streaming stability

Phase 2: Data Collection (Week 2-3) ⏳

Design motor imagery experiments
Record labeled EEG data
Create training dataset

Phase 3: Model Training (Week 3-4) ⏳

Train EEG2Arm with real data
Validate accuracy (target: >70%)
Fine-tune hyperparameters

Phase 4: Robot Integration (Week 4-5) ⏳

Connect to UR robot arm
Test safety systems
Calibrate workspace

Phase 5: Optimization (Week 5-6) ⏳

Reduce latency
Improve prediction accuracy
Add user calibration

🤝 Contributing

This is a research project for the Motorola grant. For questions or contributions:

Review the COMPREHENSIVE_REVIEW.md
Check QUICK_START_GUIDE.md for setup
Run tests: ./run_integration_test.sh

📄 License

See LICENSE file for details.

🙏 Acknowledgments

Monash DeepNeuron - Project organization
Motorola - Grant funding
Emotiv - EEG hardware
Universal Robots - Robot arm platform

📞 Support

Documentation: See /docs directory
Issues: Check existing documentation first
Hardware Guide: python hardware_test.py --hardware-guide

Last Updated: October 8, 2025
Version: 1.0
Status: ✅ Ready for Hardware Testing

🧠 Think it. 🤖 Move it.

3df9e08 (updated documentation and added AI consumer module)

Name		Name	Last commit message	Last commit date
Latest commit History 34 Commits
eeg_pipeline		eeg_pipeline
model		model
past streamed files		past streamed files
ursim_test_v1		ursim_test_v1
.DS_Store		.DS_Store
ARCHITECTURE_DIAGRAM.md		ARCHITECTURE_DIAGRAM.md
COMPREHENSIVE_REVIEW.md		COMPREHENSIVE_REVIEW.md
FIRST_TIME_USER_GUIDE.md		FIRST_TIME_USER_GUIDE.md
IMPLEMENTATION_SUMMARY.md		IMPLEMENTATION_SUMMARY.md
LICENSE		LICENSE
PROJECT_CHECKLIST.md		PROJECT_CHECKLIST.md
QUICK_START_GUIDE.md		QUICK_START_GUIDE.md
README.md		README.md
requirements.txt		requirements.txt
ursim_test		ursim_test

License

MonashDeepNeuron/Motorola-Dream-Machine

Folders and files

Latest commit

History

Repository files navigation

Motorola-Dream-Machine

EEG Pipeline Architecture

Components

How to Run the Real-Time Pipeline

1. Start an LSL Stream Source

2. Start Kafka Services

3. Run the Real-Time Producer

4. Run the Consumer

The consumer will now perform continuous analysis on the live data. When you are finished, stop the producer and consumer with Ctrl-C. The consumer will save its final analysis files upon termination.

Motorola Dream Machine

🎯 Project Overview

⚡ Quick Start

One-Command Test (Recommended)

Manual Setup

🏗️ System Architecture

📂 Project Structure

🚀 Features

✅ Implemented

⚠️ Needs Testing/Training

🧠 AI Model: EEG2Arm

🤖 Supported Robots

📊 Performance Benchmarks

📖 Documentation

🛠️ Installation

Prerequisites

Setup

🧪 Testing

Automated Integration Test

Manual Testing

🔧 Configuration

Safety Limits

Emotiv Channels

🐛 Troubleshooting

Emotiv Not Detected

Poor Signal Quality

Kafka Connection Failed

📝 Known Issues

🎯 Roadmap

Phase 1: Hardware Testing (Week 1-2) ⏳

Phase 2: Data Collection (Week 2-3) ⏳

Phase 3: Model Training (Week 3-4) ⏳

Phase 4: Robot Integration (Week 4-5) ⏳

Phase 5: Optimization (Week 5-6) ⏳

🤝 Contributing

📄 License

🙏 Acknowledgments

📞 Support

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Contributors 4

Uh oh!

Languages

The consumer will now perform continuous analysis on the live data. When you are finished, stop the producer and consumer with `Ctrl-C`. The consumer will save its final analysis files upon termination.

Packages