eprintid: 10116271 rev_number: 15 eprint_status: archive userid: 608 dir: disk0/10/11/62/71 datestamp: 2020-12-03 16:21:04 lastmod: 2021-10-04 00:28:08 status_changed: 2020-12-03 16:21:04 type: proceedings_section metadata_visibility: show creators_name: Chen, D creators_name: Jin, J creators_name: Zhang, W creators_name: Pan, F creators_name: Niu, L creators_name: Yu, C creators_name: Wang, J creators_name: Li, H creators_name: Xu, J creators_name: Gai, K title: Learning to Advertise for Organic Traffic Maximization in E-Commerce Product Feeds ispublished: pub divisions: UCL divisions: B04 divisions: C05 divisions: F48 note: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions. abstract: Most e-commerce product feeds provide blended results of advertised products and recommended products to consumers. The underlying advertising and recommendation platforms share similar if not exactly the same set of candidate products. Consumers' behaviors on the advertised results constitute part of the recommendation model's training data and therefore can influence the recommended results. We refer to this process as Leverage. Considering this mechanism, we propose a novel perspective that advertisers can strategically bid through the advertising platform to optimize their recommended organic traffic. By analyzing the real-world data, we first explain the principles of Leverage mechanism, i.e., the dynamic models of Leverage. Then we introduce a novel Leverage optimization problem and formulate it with a Markov Decision Process. To deal with the sample complexity challenge in model-free reinforcement learning, we propose a novel Hybrid Training Leverage Bidding (HTLB) algorithm which combines the real-world samples and the emulator-generated samples to boost the learning speed and stability. Our offline experiments as well as the results from the online deployment demonstrate the superior performance of our approach. date: 2019-11-03 date_type: published publisher: Association for Computing Machinery (ACM) official_url: https://doi.org/10.1145/3357384.3357819 oa_status: green full_text_type: other language: eng primo: open primo_central: open_green verified: verified_manual elements_id: 1725292 doi: 10.1145/3357384.3357819 isbn_13: 9781450369763 lyricists_name: Wang, Jun lyricists_id: JWANG00 actors_name: Wang, Jun actors_id: JWANG00 actors_role: owner full_text_status: public series: ACM International Conference on Information and Knowledge Management (CIKM) publication: PROCEEDINGS OF THE 28TH ACM INTERNATIONAL CONFERENCE ON INFORMATION & KNOWLEDGE MANAGEMENT (CIKM '19) volume: 28 place_of_pub: New York, NY, USA pagerange: 2527-2535 pages: 9 event_title: 28th ACM International Conference on Information and Knowledge Management (CIKM '19) event_location: Beijing, PEOPLES R CHINA event_dates: 03 November 2019 - 07 November 2019 institution: 28th ACM International Conference on Information and Knowledge Management (CIKM) book_title: CIKM '19: Proceedings of the 28th ACM International Conference on Information and Knowledge Management editors_name: Zhu, W editors_name: Tao, D editors_name: Cheng, X editors_name: Cui, P editors_name: Rundensteiner, E editors_name: Carmel, D editors_name: He, Q editors_name: Yu, JX citation: Chen, D; Jin, J; Zhang, W; Pan, F; Niu, L; Yu, C; Wang, J; ... Gai, K; + view all <#> Chen, D; Jin, J; Zhang, W; Pan, F; Niu, L; Yu, C; Wang, J; Li, H; Xu, J; Gai, K; - view fewer <#> (2019) Learning to Advertise for Organic Traffic Maximization in E-Commerce Product Feeds. In: Zhu, W and Tao, D and Cheng, X and Cui, P and Rundensteiner, E and Carmel, D and He, Q and Yu, JX, (eds.) CIKM '19: Proceedings of the 28th ACM International Conference on Information and Knowledge Management. (pp. pp. 2527-2535). Association for Computing Machinery (ACM): New York, NY, USA. Green open access document_url: https://discovery.ucl.ac.uk/id/eprint/10116271/1/1908.06698.pdf