Shi, Z; Fang, M; Chen, L; Du, Y; Wang, J; (2024) Human-Guided Moral Decision Making in Text-Based Games. In: Proceedings of the AAAI Conference on Artificial Intelligence. (pp. pp. 21574-21582). Association for the Advancement of Artificial Intelligence (AAAI)

Preview

Text 30155-Article Text-34209-1-2-20240324.pdf - Published Version Download (241kB) | Preview

Abstract

Training reinforcement learning (RL) agents to achieve desired goals while also acting morally is a challenging problem. Transformer-based language models (LMs) have shown some promise in moral awareness, but their use in different contexts is problematic because of the complexity and implicitness of human morality. In this paper, we build on text-based games, which are challenging environments for current RL agents, and propose the HuMAL (Human-guided Morality Awareness Learning) algorithm, which adaptively learns personal values through human-agent collaboration with minimal manual feedback. We evaluate HuMAL on the Jiminy Cricket benchmark, a set of text-based games with various scenes and dense morality annotations, using both simulated and actual human feedback. The experimental results demonstrate that with a small amount of human feedback, HuMAL can improve task performance and reduce immoral behavior in a variety of games, and is adaptable to different personal values.

Download activity - last month

Export as JSONXMLCSV

Download activity - last 12 months

Export as XMLJSONCSV

Downloads by country - last 12 months

Export as XMLJSONCSV

Archive Staff Only

View Item