Reinforcement learning for personalization: a systematic literature review

Tracking #: 621-1601


Responsible editor: 

Izabela Moise

Submission Type: 

Survey Paper

Abstract: 

The major application areas of reinforcement learning (RL) have traditionally been game playing and continuous control. In recent years, however, RL has been increasingly applied in systems that interact with humans. RL can personalize digital systems to make them more relevant to individual users. Challenges in personalization settings may be different from challenges found in traditional application areas of RL. An overview of work that uses RL for personalization, however, is lacking. In this work, we introduce a framework of personalization settings and use it in a systematic literature review. Besides setting, we review solutions and evaluation strategies. Results show that RL has been increasingly applied to personalization problems and realistic evaluations have become more prevalent. RL has become sufficiently robust to apply in contexts that involve humans and the field as a whole is growing. However, it seems not to be maturing: the ratios of studies that include a comparison or a realistic evaluation are not showing upward trends and the vast majority of algorithms are used only once. This review can be used to find related work across domains, provides insights into the state of the field and identifies opportunities for future work.

Manuscript: 

Tags: 

  • Reviewed

Data repository URLs: 

Date of Submission: 

Sunday, February 2, 2020

Date of Decision: 

Tuesday, March 3, 2020


Nanopublication URLs:

Decision: 

Accept

Solicited Reviews:


2 Comments

Meta-Review by Editor

The paper provides an overview of reinforcement learning (RL) application for personalization across different application domains. After a brief summary of reinforcement learning in general and its usage in personalization tasks in particular, the literature review is constructed based two main components: a quantitative one and a qualitative one. The quantitative component mainly reports on statistics about the topic in the literature (methods, keywords,  data mentioned in the related literature). The qualitative component is however less complete than the first one.
While the actual counts are a measure of how much attention the topic is receiving in the literature, such an overview would benefit from a more focused analysis of methods: a comparison of methods, summary of their advantages and disadvantag es and guidelines of their suitability for specific domains, a more technical machine learning approach of  reviewing methods.
While this paper represents a nice effort into providing a well-written and comprehensive review in a field that is still new, I encourage the authors to expand the review by adding a more technical analysis of machine learning methods for RL in personalization. Therefore, the authors need to address this concern together with the remarks from the reviewers.

Izabela Moise (https://orcid.org/0000-0003-0370-6749)

Dear editors and reviewers,

Dear editors and reviewers,

Thank you for your useful comments and consideration to accept this manuscript. We have addressed your comments and uploaded a revised manuscript. We have included a response letter detailing the changes.

Kindest regards,

Floris den Hengst