Hu, L;
Zhang, W;
Zhang, W;
He, Y;
Choi, S;
Gao, Y;
Chauhan, J;
(2025)
PPGSpeech: A Wearable Silent Speech Interface Leveraging Neck-worn Photoplethysmography.
IEEE Internet of Things Journal
10.1109/JIOT.2025.3639152.
(In press).
Preview |
Text
PPGSpeech_A_Wearable_Silent_Speech_Interface_Leveraging_Neck-worn_Photoplethysmography.pdf - Accepted Version Download (14MB) | Preview |
Abstract
Silent speech interfaces (SSIs) promise private and noise-immune communication, but current solutions often sacrifice user comfort, mobility, or privacy. This paper introduces PPGSpeech, a novel SSI that overcomes these limitations by pioneering the use of photoplethysmography (PPG) acquired from a comfortable, necklace-style wearable device. Our core discovery is that subtle neck muscle movements during silent articulation induce distinct, measurable modulations in the underlying PPG signal. To harness this phenomenon, we developed a complete end-to-end system featuring (1) a custom neck-worn sensor for multi-wavelength PPG acquisition, (2) a deep learning pipeline that converts 1D PPG signals into 2D time-frequency images via Continuous Wavelet Transform (CWT) and classifies them using a lightweight CNN, and (3) a Pix2Pix GAN model to reconstruct audible speech from the captured signals. In a 16-participant study covering a vocabulary of 15 commands and four confounding actions, our user-dependent model achieved a recognition accuracy of 81.41% ± 9.74. Furthermore, our speech reconstruction achieved a Mean Opinion Score (MOS) of 3.48 and a Word Correct Rate (WCR) of 60.67%, demonstrating that the PPG signal is sufficiently rich to recover intelligible speech. By establishing the viability of neck-based PPG for silent speech, PPGSpeech offers a discreet, privacy-preserving, and continuously wearable paradigm for next-generation human-computer interaction.
| Type: | Article |
|---|---|
| Title: | PPGSpeech: A Wearable Silent Speech Interface Leveraging Neck-worn Photoplethysmography |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1109/JIOT.2025.3639152 |
| Publisher version: | https://doi.org/10.1109/jiot.2025.3639152 |
| Language: | English |
| Additional information: | This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions. |
| Keywords: | PPG, Wearable, Neck-worn Sensor, Silent Speech Recognition |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10219851 |
Archive Staff Only
 |
View Item |
