macOS BLE Gamepad Support: Xbox Controller not detected via gilrs

[amiable-dev]

Problem

Xbox Wireless Controller connected via Bluetooth Low Energy (BLE) is not detected by Conductor on macOS, even though the controller is connected and functional with macOS system.

Environment

• Platform: macOS (Apple Silicon)
• Controller: Xbox Wireless Controller (Product ID: 0x0B22, Vendor ID: 0x045E)
• Connection: Bluetooth Low Energy (BLE)
• Current Library: gilrs 0.10

Root Cause Analysis

Investigation Results

✅ Controller is connected to macOS:

Xbox Wireless Controller:
    Address: 3C:FA:06:18:BD:A5
    Vendor ID: 0x045E (Microsoft)
    Product ID: 0x0B22
    Minor Type: Gamepad
    Services: BLE

✅ Conductor code is correct:

• ListDevices IPC command properly enumerates both MIDI and HID devices
• HidDeviceManager::list_gamepads() calls gilrs correctly
• Test confirms: Found 0 game controllers

❌ gilrs doesn't detect BLE controllers on macOS:

• gilrs uses IOKit framework which primarily supports USB HID devices
• BLE controllers are exposed via different macOS APIs
• macOS requires native GameController framework for BLE gamepad support

Test Evidence

cargo test --package conductor-daemon test_list_gamepads -- --nocapture
// Output: Found 0 game controllers

Proposed Solutions

Option 1: Dynamic Library Plugin Architecture ⭐ (Recommended)

Architecture:

conductor-daemon: 3MB (no gamepad libs)
├── plugins/
    ├── libconductor-backend-gilrs.dylib (~500KB)
    ├── libconductor-backend-sdl2.dylib (~2MB)  
    └── libconductor-backend-macos.dylib (~100KB)

Config:
[device]
gamepad_backend = "sdl2"  # Runtime selection

Implementation:

• Define GamepadBackend trait with C ABI
• Use libloading crate for dynamic library loading
• Load backend based on config.toml setting
• Users download only needed plugins

Pros:

• ✅ Smallest core binary (3MB vs 5-8MB)
• ✅ Runtime config selection (no recompile)
• ✅ Download on-demand (install just needed backend)
• ✅ Hot-swap backends (change config, reload)

Cons:

• Distribution complexity (multiple files)
• Version compatibility management
• Security considerations (loading external code)

Effort: ~6-8 hours

---

Option 2: Separate Backend Binaries (Simpler)

Architecture:

conductor-daemon: 3MB (coordinator only)
conductor-backend-gilrs: 1MB (standalone binary)
conductor-backend-sdl2: 3MB (standalone binary)
conductor-backend-macos: 500KB (standalone binary)

Config:
[device]
gamepad_backend = "sdl2"

Implementation:

• Each backend is a separate executable
• Daemon spawns backend as subprocess
• Communicate via stdin/stdout (JSON protocol)
• Reuse existing IPC infrastructure

Pros:

• ✅ Simple to implement (no FFI/ABI concerns)
• ✅ Process isolation (backend crash safe)
• ✅ Easy distribution (separate binaries)
• ✅ Runtime selection via config
• ✅ Can distribute via cargo install conductor-backend-sdl2
• ✅ Fits existing architecture (already have IPC)

Cons:

• Extra process overhead (~1-2ms latency)
• Multiple binaries to distribute

Effort: ~4-6 hours

---

Option 3: Cargo Features (Current Approach)

Architecture:

[features]
default = ["gilrs-backend"]
gilrs-backend = ["gilrs"]
sdl2-backend = ["sdl2"]
macos-native = []  # macOS only

# User builds with:
cargo build --features sdl2-backend

Pros:

• ✅ Smallest binary (only one backend compiled)
• ✅ No runtime overhead
• ✅ Simple implementation

Cons:

• ❌ Requires recompile to switch backends
• ❌ User must know to use correct feature
• ❌ Larger binary if multiple features enabled

Effort: ~2-3 hours

---

Option 4: Immediate SDL2 Integration

Quick Fix: Replace gilrs with sdl2 globally

• Works on macOS BLE controllers
• Proven, battle-tested
• ~2MB binary size increase

Pros:

• ✅ Fastest path to working Xbox support (1 hour)
• ✅ Works on all platforms

Cons:

• ❌ Larger binary for everyone
• ❌ No user choice

Effort: ~1 hour

---

Backend Comparison

Backend	macOS BLE	macOS USB	Linux	Windows	Binary Size
gilrs	❌	✅	✅	✅	+500KB
SDL2	✅	✅	✅	✅	+2MB
macOS GameController	✅	✅	❌	❌	+100KB

Recommendation

Phase 1: Implement Option 2 (Separate Backend Binaries)

• Best balance of flexibility, simplicity, and binary size
• Fits existing IPC architecture
• Process isolation provides safety
• Easy to distribute and update

Phase 2: Add backend implementations

1. conductor-backend-sdl2 (immediate - fixes macOS BLE)
2. conductor-backend-macos (later - optimal macOS performance)
3. Keep conductor-backend-gilrs (default for Linux/Windows)

Phase 3 (Optional): Convert to dynamic library plugins (Option 1) if needed

Implementation Checklist

• Define GamepadBackend trait in conductor-core
• Create conductor-backend-gilrs binary (extract existing code)
• Create conductor-backend-sdl2 binary (new)
• Add subprocess spawning to input_manager.rs
• Implement JSON protocol for backend communication
• Add gamepad_backend config option
• Update documentation
• Add macOS-specific testing guide
• Create releases for all backends

References

• gilrs Issue #139: macOS Bluetooth Support
• SDL2 Rust bindings
• macOS GameController Framework

Labels

enhancement, architecture, macos, gamepad, v3.1

[amiable-dev]

Update: Wired USB Controllers Also Affected

Additional Testing

Tested with wired USB connection (not just Bluetooth):

USB Device Detected by macOS:

Controller:
  Product ID: 0x0b00
  Vendor ID: 0x045e (Microsoft Corporation)
  Version: 5.16
  Serial Number: 3032363330303132333032333435
  Speed: Up to 12 Mb/s
  Manufacturer: Microsoft
  Location ID: 0x00150000 / 17
  Current Required (mA): 500

gilrs Detection Result:

cargo test --package conductor-daemon test_list_gamepads -- --nocapture
// Output: Found 0 game controllers

Root Cause Clarification

The issue is not BLE-specific - it affects both wired and wireless Xbox controllers on macOS.

Why Xbox controllers don't work with gilrs on macOS:

1. Xbox controllers use a proprietary protocol, not standard USB HID
2. macOS doesn't expose Xbox controllers as generic HID devices in IOKit
3. They require the native GameController framework or XInput-like driver
4. gilrs's IOKit implementation doesn't support Xbox-specific protocol

Severity Impact

This makes multi-backend architecture essential rather than optional:

Backend	Xbox Wired	Xbox Wireless	Generic HID	Priority
gilrs	❌	❌	✅	Keep for Linux/Windows
SDL2	✅	✅	✅	Critical for macOS
macOS GameController	✅	✅	✅	Optimal for macOS

Recommendation Updated

Option 2 recommended → Option 2 now required for macOS gamepad support.

Without multi-backend architecture, Conductor cannot support the most common gamepad (Xbox controllers) on macOS at all.

[amiable-dev]

Strategic Update: Build Testing Infrastructure First

After investigation and planning, this issue is now blocked by testing infrastructure work. The Xbox controller detection issue revealed that:

1. gilrs cannot detect Xbox controllers on macOS (both wired and wireless)
2. Multi-backend architecture is essential (not optional)
3. Testing without physical hardware is critical for development

New Blocking Issues

This issue is blocked by:

• MCP Server for Protocol Specifications and AI Test Generation #2: MCP Server for Protocol Specs (AI-powered test generation)
• AI-Powered Multi-Protocol Message Generator #3: AI-Powered Message Generator (virtual device simulation)
• Enhanced Multi-Protocol Diagnostic CLI #4: Enhanced Multi-Protocol Diagnostic CLI (debugging support)

Rationale

Why build testing infrastructure first?

1. Hardware-Free Development: Can't test SDL2/macOS backends without Xbox controller → need virtual devices
2. AI-Generated Test Coverage: LLM can generate edge cases humans miss
3. CI/CD Integration: Automated testing requires virtual device support
4. Better Debugging: Multi-protocol diagnostic tool catches integration issues early
5. Regression Testing: Record real device bugs → replay forever

Implementation Order (Updated)

Phase 1: Testing Infrastructure (Before multi-backend work)

1. ✅ Issue MCP Server for Protocol Specifications and AI Test Generation #2: MCP Server (~6 hours)
2. ✅ Issue AI-Powered Multi-Protocol Message Generator #3: AI Message Generator (~9 hours)
3. ✅ Issue Enhanced Multi-Protocol Diagnostic CLI #4: Enhanced Diagnostics (~3 hours)
4. Total: ~18 hours over 2-3 days

Phase 2: Multi-Backend Architecture (This Issue)

With testing infrastructure ready:

• Test SDL2 backend without physical hardware ✅
• AI-generated edge case coverage ✅
• Diagnostic tools for debugging ✅
• Record real Xbox controller → replay in tests ✅

Benefits of This Approach

Before (without testing infrastructure):

• ❌ Need physical Xbox controller for every test
• ❌ Can't test in CI/CD
• ❌ Manual testing only
• ❌ Hard to reproduce bugs

After (with testing infrastructure):

• ✅ Virtual gamepad simulation
• ✅ AI generates realistic test patterns
• ✅ Automated CI/CD testing
• ✅ Record-replay for bugs
• ✅ Hardware-independent development

Timeline Impact

Original estimate: 4-6 hours (multi-backend only)
Updated estimate: 18 hours (infrastructure) + 4-6 hours (backends) = 22-24 hours total

Investment in testing infrastructure will save time on:

• Debugging without physical hardware
• Automated test coverage
• Future protocol additions (OSC, etc.)
• Regression prevention

Next Steps

1. Implement MCP Server for Protocol Specifications and AI Test Generation #2 (MCP Server)
2. Implement AI-Powered Multi-Protocol Message Generator #3 (Message Generator)
3. Implement Enhanced Multi-Protocol Diagnostic CLI #4 (Diagnostics)
4. Then implement this issue (multi-backend architecture)

---

Status: Blocked by #2, #3, #4
Priority: High (after infrastructure)
Effort: 4-6 hours (backends only, after infrastructure ready)