Azarmi, Mohsen; Rezaei, Mandi; Wang, He; (2025) Pedestrian Intention Prediction via Vision-Language Foundation Models. In: Proceedings of the 2025 IEEE INTELLIGENT VEHICLES SYMPOSIUM, IV. (pp. pp. 1899-1904). IEEE: Cluj-Napoca, Romania.

Preview

PDF 2507.04141v1.pdf - Accepted Version Download (931kB) | Preview

Abstract

Prediction of pedestrian crossing intention is a critical function in autonomous vehicles. Conventional vision-based methods of crossing intention prediction often struggle with generalizability, context understanding, and causal reasoning. This study explores the potential of vision-language foundation models (VLFMs) for predicting pedestrian crossing intentions by integrating multimodal data through hierarchical prompt templates. The methodology incorporates contextual information, including visual frames, physical cues observations, and ego-vehicle dynamics, into systematically refined prompts to guide VLFMs effectively in intention prediction. Experiments were conducted on three common datasets—JAAD, PIE, and FU-PIP. Results demonstrate that incorporating vehicle speed, its variations over time, and time-conscious prompts significantly enhances the prediction accuracy up to 19.8%. Additionally, optimised prompts generated via an automatic prompt engineering framework yielded 12.5% further accuracy gains. These findings highlight the superior performance of VLFMs compared to conventional vision-based models, offering enhanced generalisation and contextual understanding for autonomous driving applications.

Download activity - last month

Export as JSONCSVXML

Download activity - last 12 months

Export as CSVXMLJSON

Downloads by country - last 12 months

Export as XMLCSVJSON

Archive Staff Only

View Item