EarFusion: Quality-Aware Fusion of In-Ear Audio and Photoplethysmography for Heart Rate Monitoring [paper]

EarFusion explores how in-ear audio and PPG, which includes two physiologically complementary modalities and can be dynamically fused for robust heart rate (HR) monitoring in real-world environments.
While PPG is widely used in wearables, it is easily corrupted by motion and skin–sensor variability. In contrast, in-ear audio sensing captures cardio-mechanical vibrations that are resilient to motion but sensitive to acoustic noise. EarFusion bridges these limitations through signal quality–aware fusion that adapts in real time to the reliability of each modality.

Built on the OmniBuds earable platform, EarFusion integrates an in-ear microphone, optical PPG sensor, and reference ECG (Polar H10). The system computes modality-specific fine-grained meterics and Signal Quality Indices (SQIs) which capture SNR, morphology, and rhythm consistency and then are used to fuse HR estimates based on their reliability. Experimental results across multiple participants and conditions show that EarFusion consistently outperforms single-modality baselines, achieving stable HR estimation under both motion and acoustic interference.

System Overview Multimodal data collection setup integrating PPG, in-ear audio, and Polar H10 ECG for synchronized multimodal acquisition.

Signal Quality and Fusion Results Visualization of SQI-driven HR estimation under clean, motion, and audio-noise conditions.