Zhong, Shu;
Gatti, Elia;
Cho, Youngjun;
Obrist, Marianna;
(2024)
Feeling Textiles through AI: An Exploration into Multimodal
Language Models and Human Perception Alignment.
In:
Proceedings of the 26th ACM International Conference on Multimodal Interaction.
ACM: San Jose, Costa Rica.
(In press).
Preview |
Text
Zhong_icmi24-59.pdf Download (2MB) | Preview |
Abstract
Human-artificial intelligence (AI) alignment ensures that AI systems align with human goals and behaviors. This paper introduces perceptual alignment as a critical aspect of this alignment, focusing on the concurrence between human judgments and AI evaluations across sensory modalities. We particularly explore how Multimodal Large Language Models (MLLMs), which process both visual and textual data, interpret the tactile qualities of textiles—a significant challenge in online shopping environments. Our research analyzes six vision-based MLLMs to see how they describe the tactile experience of textiles and compares these AI-generated descriptions with human assessments. Through semantic similarity measures and in-person evaluations, we investigate the extent of alignment between human perceptions and AI descriptions. Our findings indicate significant variability in the AI’s ability to interpret different textiles, highlighting both the potential and limitations of current AI models in achieving perceptual alignment. This work contributes to understanding the complexities of aligning AI capabilities with human touch sensory experiences.
| Type: | Proceedings paper |
|---|---|
| Title: | Feeling Textiles through AI: An Exploration into Multimodal Language Models and Human Perception Alignment |
| Event: | 26th ACM International Conference on Multimodal Interaction |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1145/3678957.3685756 |
| Publisher version: | https://icmi.acm.org/2024/ |
| Language: | English |
| Additional information: | This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions. |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10196394 |
Loading...
Loading...Loading...Loading...Archive Staff Only
 |
View Item |
