eprintid: 10194562 rev_number: 11 eprint_status: archive userid: 699 dir: disk0/10/19/45/62 datestamp: 2024-07-16 13:24:16 lastmod: 2024-11-29 13:33:55 status_changed: 2024-07-16 13:24:16 type: proceedings_section metadata_visibility: show sword_depositor: 699 creators_name: Fung, Ho Long creators_name: Darvariu, Victor creators_name: Hailes, Stephen creators_name: Musolesi, Mirco title: Trust-based Consensus in Multi-Agent Reinforcement Learning Systems ispublished: pub divisions: UCL divisions: B04 divisions: F48 note: This version is the version of record. For information on re-use, please refer to the publisher’s terms and conditions. abstract: An often neglected issue in multi-agent reinforcement learning (MARL) is the potential presence of unreliable agents in the environment whose deviations from expected behavior can prevent a system from accomplishing its intended tasks. In particular, consensus is a fundamental underpinning problem of cooperative distributed multi-agent systems. Consensus requires different agents, situated in a decentralized communication network, to reach an agreement out of a set of initial proposals that they put forward. Learning-based agents should adopt a protocol that allows them to reach consensus despite having one or more unreliable agents in the system. This paper investigates the problem of unreliable agents in MARL, considering consensus as a case study. Echoing established results in the distributed systems literature, our experiments show that even a moderate fraction of such agents can greatly impact the ability of reaching consensus in a networked environment. We propose Reinforcement Learning-based Trusted Consensus (RLTC), a decentralized trust mechanism, in which agents can independently decide which neighbors to communicate with. We empirically demonstrate that our trust mechanism is able to handle unreliable agents effectively, as evidenced by higher consensus success rates. date: 2024-08-15 date_type: published publisher: Reinforcement Learning Conference Organization Committee official_url: https://rlj.cs.umass.edu/2024/papers/Paper103.html oa_status: green full_text_type: pub language: eng primo: open primo_central: open_green verified: verified_manual elements_id: 2296941 lyricists_name: Musolesi, Mirco lyricists_id: MMUSO05 actors_name: Musolesi, Mirco actors_id: MMUSO05 actors_role: owner full_text_status: public pres_type: paper place_of_pub: Amherst, MA, USA pagerange: 714-732 event_title: 1st Reinforcement Learning Conference (RLC) book_title: Proceedings of the 1st Reinforcement Learning Conference (RLC'24) citation: Fung, Ho Long; Darvariu, Victor; Hailes, Stephen; Musolesi, Mirco; (2024) Trust-based Consensus in Multi-Agent Reinforcement Learning Systems. In: Proceedings of the 1st Reinforcement Learning Conference (RLC'24). (pp. pp. 714-732). Reinforcement Learning Conference Organization Committee: Amherst, MA, USA. Green open access document_url: https://discovery.ucl.ac.uk/id/eprint/10194562/1/rlc24_trustbased.pdf