Wei, Xijia;
Fang, Yuan;
Liu, Zihan;
Zhu, Xiyue;
Wang, Jiawei;
Liu, Yifu;
Petreca, Bruna;
... Bianchi-Berthouze, Nadia; + view all
(2025)
mmWaveTryOn: The first mmWave-RGB Dataset for Clothes Try-On Multimodal Gesture Recognition.
In:
ANAI '25: Proceedings of the 2025 ACM Workshop on Access Networks with Artificial Intelligence.
(pp. pp. 31-35).
Association for Computing Machinery (ACM): New York, NY, USA.
Preview |
Text
Wei_3737904.3768535.pdf Download (3MB) | Preview |
Abstract
In recent years, mmWave sensing has become an emerging research field. mmWave sensor demonstrates its unique advantage of passive, Non-line-of-sight (NLOS) sensing capability with less privacy concerns. It has been widely adapted in various applications such as human activity recognition, healthcare, tracking and navigation, and etc. In fashion industry, RGB-based smart mirror is commonly used in retail for tracking human gestures and offering virtual try-on experience. However, utilizing RGB camera can raises privacy issues and be restricted to lighting conditions. In contrast, mmWave sensor shows unique potentials as an alternative modality. However, there is a lack of exploration on investigating the feasibility of leveraging mmWave for clothes try-on gesture recognition. In this study, we propose the mmWaveTryOn, the first-of-its-kind multimodal dataset including mmWave, RGB-extracted skeleton and action type labels, focusing on try-on gesture recognition. Experiments show that mmWave-based model outperforms RGB-based model for fine-grained try-on gesture classification, and fusion of both modalities further improves the accuracy to 81.51%. The mmWaveTryOn dataset is publicly available to encourage the community to explore the integration of mmWave sensing technology in fashion industry and also serve as strong benchmark baselines.
| Type: | Proceedings paper |
|---|---|
| Title: | mmWaveTryOn: The first mmWave-RGB Dataset for Clothes Try-On Multimodal Gesture Recognition |
| Event: | 2025 ACM Workshop on Access Networks with Artificial Intelligence (ANAI '25) |
| ISBN-13: | 9798400719813 |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1145/3737904.3768535 |
| Publisher version: | https://doi.org/10.1145/3737904.3768535 |
| Language: | English |
| Additional information: | Copyright © 2025 Copyright held by the owner/author(s). This work is licensed under a Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by/4.0/). |
| Keywords: | mmWave Sensing, Human Activity Recognition, Multimodal Machine Learning, Smart Mirror, Virtual Try-on Experience, Human Computer Interaction |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > Div of Psychology and Lang Sciences UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > Div of Psychology and Lang Sciences > UCL Interaction Centre |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10218210 |
Archive Staff Only
 |
View Item |
