Chen, X;
Du, Y;
Xia, L;
Wang, J;
(2021)
Reinforcement recommendation with user multi-aspect preference.
In:
WWW '21: Proceedings of the Web Conference 2021.
(pp. pp. 425-435).
ACM: Association for Computing Machinery: New York, NY, USA.
Preview |
Text
3442381.3449846.pdf - Published Version Download (1MB) | Preview |
Abstract
Formulating recommender system with reinforcement learning (RL) frameworks has attracted increasing attention from both academic and industry communities. While many promising results have been achieved, existing models mostly simulate the environment reward with a unified value, which may hinder the understanding of users' complex preferences and limit the model performance. In this paper, we consider how to model user multi-aspect preferences in the context of RL-based recommender system. More specifically, we base our model on the framework of deterministic policy gradient (DPG), which is effective in dealing with large action spaces. A major challenge for modeling user multi-aspect preferences lies in the fact that they may contradict with each other. To solve this problem, we introduce Pareto optimization into the DPG framework. We assign each aspect with a tailored critic, and all the critics share the same actor. The Pareto optimization is realized by a gradient-based method, which can be easily integrated into the actor and critic learning process. Based on the designed model, we theoretically analyze its gradient bias in the optimization process, and we design a weight-reuse mechanism to lower the upper bound of this bias, which is shown to be effective for improving the model performance. We conduct extensive experiments based on three real-world datasets to demonstrate our model's superiorities.
| Type: | Proceedings paper |
|---|---|
| Title: | Reinforcement recommendation with user multi-aspect preference |
| Event: | WWW '21: The Web Conference 2021 |
| ISBN-13: | 9781450383127 |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1145/3442381.3449846 |
| Publisher version: | https://doi.org/10.1145/3442381.3449846 |
| Language: | English |
| Additional information: | © 2021 IW3C2 (International World Wide Web Conference Committee), published under Creative Commons CC-BY 4.0 License. This paper is published under the Creative Commons Attribution 4.0 International (CC-BY 4.0) license. Authors reserve their rights to disseminate the work on their personal and corporate Web sites with the appropriate attribution. |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10130485 |
Loading...
Loading...Loading...Loading...Archive Staff Only
 |
View Item |
