Reviewer has chosen not to be AnonymousOverall Impression:
UndecidedTechnical Quality of the paper:
Incomplete or inappropriateNovelty:
Limited noveltyData availability:
All used and produced data (if any) are FAIR and openly available in established data repositoriesLength of the manuscript:
The length of this manuscript is about right
Summary of paper in a few sentences:
This submission presents a framework to model and to predict user engagement with mobile applications. The framework is evaluated by using a data set of app usage of one particular app focusing on waste recycling. With the successful evaluation, the authors aim to provide evidence that it is possible to predict when users of a mobile application will get disengaged.
Reasons to accept:
The focused topic of modeling and predicting user engagement for mobile applications is timely and relevant for the research communities in Data Science and Human-Computer Interaction. Overall, the presented approach seems to be novel and well-suited. Furthermore, also the results of the evaluation are promising.
Reasons to reject:
I have strong doubts regarding the used data set and features. The used waste recycling app is described only briefly. The authors do not argue, why this is a common mobile application. I would recommend discussing this with consideration of the results presented by Müller et al. . I would question that it is common for mobile applications that gamification aspects (here earning points) are directly connected to providing monetary benefits. Here, it is particularly interesting that the granted points can only be used at local shops. Thereby user’s location becomes an obvious feature for disengagement. Additionally, using the zip code and the geolocation provides only redundant information. In general, the list of features and calculated variables is fuzzy. The authors claim to use 7 features but present only 6 in a list. Also, it reminds unclear how they combined the features to the 122 variables.
The authors do not describe if the application triggered any notifications. However, Sahami Shirazi et al. describe notifications as an essential element for engaging with mobile applications . Hence, I wonder why the authors did not use the number of notifications or the reaction to notifications as a feature. To be able to understand user engagement or disengagement with the waste recycling app, it would be helpful, if the authors would also publish the application or provide at least a reference to the application.
While the authors motivate their work in the introduction very general, also looking on specific application domains such as health (reference  in the submission), the authors discuss the limitation of the used data set only briefly at the end of the paper.
 Hendrik Müller, Jennifer Gove, and John Webb. 2012. Understanding tablet use: a multi-method exploration. In Proceedings of the 14th international conference on Human-computer interaction with mobile devices and services (MobileHCI '12). ACM, New York, NY, USA, 1-10. DOI: https://doi.org/10.1145/2371574.2371576
 Alireza Sahami Shirazi, Niels Henze, Tilman Dingler, Martin Pielot, Dominik Weber, and Albrecht Schmidt. 2014. Large-scale assessment of mobile notifications. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). ACM, New York, NY, USA, 3055-3064. DOI=http://dx.doi.org/10.1145/2556288.2557189
As described the used features and the data set look more specific than general to me. Hence, the submission would be more substantial if the authors would make less general claims and focus particularly on comparable mobile applications. Furthermore, publishing not only the data set but also the application or providing a reference to the application would improve the validity.