清华大学和京东发表于KDD 2019的全新强化学习框架FeedRec

ACM SIGKDD（Conference on Knowledge Discovery and Data Mining, KDD）是世界数据挖掘领域的最高级别的国际会议，由 ACM（Association of Computing Machinery，计算机学会）的数据挖掘及知识发现专委会（SIGKDD）负责协调筹办，被中国计算机协会荐为 A 类会议。

KDD 2019 包括两个 track：Research track 和 Applied Data Science track。

今年的 KDD Research track 共评审约 1200 篇投稿，其中约 110 篇 oral 论文，60 篇 poster 论文，接收率约 14%，比往年的 17%~18% 还要下降了近 4 个百分点。此前 3 年 KDD Research track 的录用情况分别是：投稿 983 篇，收录 178 篇（2018）；投稿748 篇，收录 130 篇（2017）；投稿 784篇，收录142 篇（2016）。

而此次 ADS track 约投稿 700 篇，其中 45 篇 oral 论文，100 篇 poster 论文。

学术君今天为大家推荐的是清华大学和京东发表于KDD 2019的工作。

论文题目
Reinforcement Learning to Optimize Long-term User Engagement in Recommender Systems
作者
Lixin Zou, Long Xia, Zhuoye Ding, Jiaxing Song, Weidong Liu, Dawei Yin
会议/年份
KDD 2019
链接
http://export.arxiv.org/abs/1902.05570
Abstract
Recommender systems play a crucial role in our daily lives. Feed streaming mechanism has been widely used in the recommender system, especially on the mobile Apps. The feed streaming setting provides users the interactive manner of recommendation in never-ending feeds. In such an interactive manner, a good recommender system should pay more attention to user stickiness, which is far beyond classical instant metrics, and typically measured by {\bf long-term user engagement}. Directly optimizing the long-term user engagement is a non-trivial problem, as the learning target is usually not available for conventional supervised learning methods. Though reinforcement learning~(RL) naturally fits the problem of maximizing the long term rewards, applying RL to optimize long-term user engagement is still facing challenges: user behaviors are versatile and difficult to model, which typically consists of both instant feedback~(\eg clicks, ordering) and delayed feedback~(\eg dwell time, revisit); in addition, performing effective off-policy learning is still immature, especially when combining bootstrapping and function approximation.
To address these issues, in this work, we introduce a reinforcement learning framework --- FeedRec to optimize the long-term user engagement. FeedRec includes two components: 1)~a Q-Network which designed in hierarchical LSTM takes charge of modeling complex user behaviors, and 2)~an S-Network, which simulates the environment, assists the Q-Network and voids the instability of convergence in policy learning. Extensive experiments on synthetic data and a real-world large scale data show that FeedRec effectively optimizes the long-term user engagement and outperforms state-of-the-arts.
推荐理由
本文是清华大学和京东发表于 KDD 2019 的工作。论文针对利用强化学习解决推荐系统时存在用户行为难以建模的问题，提出了一种新的强化学习框架 FeedRec，包括两个网络：Q 网络利用层次化 LSTM 对复杂用户行为建模，S 网络用来模拟环境，辅助和稳定 Q 网络的训练。方法在合成数据和真实数据上进行了验证，取得了 SOTA 的结果。