Diallo, Aissatou; Bikakis, Antonios; Dickens, Luke; Hunter, Anthony; Miller, Rob; (2024) PizzaCommonSense: A Dataset for Commonsense Reasoning about Intermediate Steps in Cooking Recipes. In: Al-Onaizan, Yaser and Bansal, Mohit and Chen, Yun-Nung, (eds.) Findings of the Association for Computational Linguistics: EMNLP 2024. (pp. pp. 12482-12496). Association for Computational Linguistics: Miami, Florida, USA.

Preview

Text Bikakis_2024.findings-emnlp.728.pdf Download (1MB) | Preview

Abstract

Understanding procedural texts, such as cooking recipes, is essential for enabling machines to follow instructions and reason about tasks, a key aspect of intelligent reasoning. In cooking, these instructions can be interpreted as a series of modifications to a food preparation.For a model to effectively reason about cooking recipes, it must accurately discern and understand the inputs and outputs of intermediate steps within the recipe.We present a new corpus of cooking recipes enriched with descriptions of intermediate steps that describe the input and output for each step. PizzaCommonsense serves as a benchmark for the reasoning capabilities of LLMs because it demands rigorous explicit input-output descriptions to demonstrate the acquisition of implicit commonsense knowledge, which is unlikely to beeasily memorized. GPT-4 achieves only 26% human-evaluated preference for generations, leaving room for future improvements.

Download activity - last month

Export as CSVXMLJSON

Download activity - last 12 months

Export as JSONXMLCSV

Downloads by country - last 12 months

Export as CSVXMLJSON

Archive Staff Only

View Item