Voice Command Capabilities on AR device
Edge ML Group Final Project for Harvard CS249R
I collaborated with a team of three to develop a custom keyword spotting model for the Brilliant Labs Monocle, a compact augmented reality (AR) device, enabling intuitive voice-controlled interactions. Our work involved collecting and curating a dataset of relevant voice commands and training a recognition model using TensorFlow. To ensure efficient on-device inference, we optimized the model with TensorFlow Lite Micro (TFLM) and deployed it to the Monocle’s microcontroller unit (MCU), balancing accuracy with minimal computational overhead. The model’s performance was evaluated using key metrics such as accuracy and inference latency.
This project advances AR technology by enhancing user interaction through voice control while demonstrating the feasibility of deploying sophisticated machine learning models on resource-constrained devices. By adapting a keyword spotting model to a microcontroller environment, our work highlights the potential of Tiny Machine Learning (TinyML) in AR applications. This approach paves the way for future innovations in embedded AI, making intelligent, low-power interactions more accessible in wearable and IoT devices.
Final Project Deliverables: