Design and Implementation of a Multimodal Edge ML Detection System Using Audio-Visual Sensor Fusion

, ,

Professor: Beverly Abadines Quon

Description: This project proposes the development of a multi-modal machine learning detection system that uses two Arduino Portenta H7 Lites equipped with an OV2640 camera and an INMP441 microphone to combine audio and visual detection models. The system will utilize late fusion ensembling to function as a detection device for applications such as security monitoring, industrial safety, and smart environment automation. By using multiple sensor modalities rather than relying on a single detection source, the system aims to provide a more accurate and robust edge-computed solution that can operate even when one sensor’s input is obscured or noisy.

Preferred Qualifications:

Contact Us