Teaching LLMs to Infer Real-World Consequences through Embodied Semantic Grounding
Manaswi Kulahara1, Khadija Parwez2, Faisal Alhwikem3,*, Fawwad Hassan Jaskani4
1 Department of Geoinformatics, TERI School of Advanced Studies, Delhi, India
2 Department of Computing and Technology, IQRA University Islamabad Campus, H-9, Islamabad, Pakistan
3 Department of Computer Science, College of Computer, Qassim University, Buraydah, Saudi Arabia
4 Department of Computer Systems Engineering, The Islamia University of Bahawalpur, Pakistan
* Corresponding Author: Faisal Alhwikem. Email: 
Computers, Materials & Continua https://doi.org/10.32604/cmc.2026.084877
Received 07 May 2026; Accepted 01 July 2026; Published online 04 August 2026
Abstract
Large Language Models (LLMs) have recently advanced in real-world commonsense reasoning, including understanding everyday object behaviors and inferring their attributes from text. However, they remain limited in reasoning about the
real-world consequences of events, such as how object failures, obstructions, or structural changes affect the surrounding environment-especially without visual or sensorimotor input. Existing works like PIQA and NEWTON evaluate narrow sub-skills, such as whether an object action makes sense and whether object properties can be inferred, providing valuable benchmarks for commonsense and physical reasoning but offering limited evaluation of how events alter environmental functionality and downstream conditions. To address this gap, we propose Embodied Semantic Grounding (ESG), a framework that equips LLMs with consequence-aware text representations. ESG learns a consequence-grounded space by aligning event descriptions with affordance maps-structured representations of how an environment can be used or traversed after an event-capturing how structural changes modify environmental functionality. A Flan-T5-XL model is trained with a contrastive alignment objective to encode event descriptions into this space, for coherent prediction of consequences such as collapses, blockages, and environmental changes. Rather than introducing a new language-model architecture, ESG extends affordance-grounding with consequence-level representations of post-event environmental functionality. We evaluate ESG on a unified benchmark comprising PIQA, NEWTON, LIBERO-derived affordance text, and 2400 synthetic scenario-based tasks. Results show that ESG improves performance over baseline language models across commonsense reasoning and consequence-prediction benchmarks. Under structured affordance-map supervision, ESG improves zero-shot accuracy on PIQA and NEWTON and demonstrates improved performance on synthetic consequence-prediction scenarios designed to evaluate post-event environmental reasoning.
Keywords
Environment; Flan-T5-XL; text; large language models; real-world
Open Access
Login
Register
Submit a Paper
Propose a Special lssue
Download PDF
Downloads
Citation Tools
