Diallo, Aissatou; Bikakis, Antonios; Dickens, Luke; Hunter, Anthony; Miller, Rob; (2025) IAO Prompting: Making Knowledge Flow Explicit in LLMs through Structured Reasoning Templates. In: Proceedings of the AAAI Conference on Artificial Intelligence. Association for the Advancement of Artificial Intelligence (AAAI)

Preview

Text 7_IAO_Prompting_Making_Knowled.pdf - Published Version Download (705kB) | Preview

Abstract

While Large Language Models (LLMs) demonstrate impressive reasoning capabilities, understanding and validating their knowledge utilization remains challenging. Chain-of-thought (CoT) prompting partially addresses this by revealing intermediate reasoning steps, but the knowledge flow and application remain implicit. We introduce IAO (Input-ActionOutput) prompting, a structured template-based method that explicitly models how LLMs access and apply their knowledge during complex reasoning tasks. IAO decomposes problems into sequential steps, each clearly identifying the input knowledge being used, the action being performed, and the resulting output. This structured decomposition enables us to trace knowledge flow, verify factual consistency, and identify potential knowledge gaps or misapplications. Through experiments across diverse reasoning tasks, we demonstrate that IAO not only improves zero-shot performance but also provides transparency in how LLMs leverage their stored knowledge. Human evaluation confirms that this structured approach enhances our ability to verify knowledge utilization and detect potential hallucinations or reasoning errors. Our findings provide insights into both knowledge representation within LLMs and methods for more reliable knowledge application.

Download activity - last month

Export as XMLJSONCSV

Download activity - last 12 months

Export as CSVXMLJSON

Downloads by country - last 12 months

Export as XMLJSONCSV

Archive Staff Only

View Item