I would like to thank the reviewers for their input and feedback. Reviewer 1 1. The part on the pilot is still of limited use to other researchers, maybe it could be shrunk even more We moved further parts (essentially all of the results) from the pilot study descriptions in the paper to the Supplementary Information. The pilot study is now described briefly (though a few details have been added on the recruitment process and the software as requested by reviewer 1, see below) in a rather short section with references to further details in the Supplementary Information. 2. The pilot is run with a software which is different from what is currently being developed, and described in the Supplementary Material. This is a bit confusing, we would like to know more about the potential of the new software! The section on Mobile Phone Application has now been expanded and explanations of the potential of the new software have been added. 3. The procedure for recruitment of subject is not clearly described: how were subjects actually found? A description of the recruitment procedure has been added in the Data Collection subsection, first paragraph. 4. There is no reference to IRB approval (ethics) in the main text (I might have missed in the supplementary material), but the number should be there. A sentence has been added at the end of the first paragraph in the Pilot Study section, stating that ethical approval has been obtained, providing the reference number and the name of the committee. 5. Many readers might not be familiar with the Gaussian Process choice models used to generate the landscapes of interactions in Fig.2. Even if references are correctly provided, a brief description of the method would be very useful. The entire section on choice model results was moved to the Supplementary Information S3.4, where also a brief explanation of the Gaussian Process choice models is given. Reviewer 2 1. The authors claim that their approach allows for experimental setups, in contrast to previous study. If I understand this claim correctly, this is where the novelty lies. (…). I am not at all convinced of this point. Especially in the health domain, but also in other domains. These types of experimental studies with devices / smartphones with bespoke applications in living lab type setups have been done before and I don’t see how the suggestions of the authors push beyond the state of the art. (…). Also, the identification of different types of interventions is not really novel or well worked-out. We made it now clearer and more explicit, that the novelty is not that an experimental intervention has been implemented, but rather that existing studies usually focus on simple behaviours and simple interventions (usually message-based nudging). What we suggest in this paper is to study more complex public goods behaviours using smartphone technology and examining more complex and multiple interventions. Though in the pilot study only two interventions are tested, the paper discusses in more detail what other potential, complex interventions could be tested. We changed explicitly the sentence that reviewer 2 subjected: “Most of these studies however do not implement an experimental design.” Now it says: “Some smartphone-based studies implement also an experimental design.” I then go on to describe more explicitly what has been done and what novel approaches we suggest as described above. Paragraphs 3 and 4 in “State of the Art” where changed. In these paragraph as throughout the paper we tried among others to tone down on the notion of novelty. The study mentioned by reviewer has also been incorporated in the state of the art section, though a different reference was used. 2. The "AI-based emancipatory" intervention sounds interesting but is really under-explained. (…) Many ideas and open questions are posed throughout the paper without details or follow-ups (…) The paragraph, where this approach is explained, has been expanded and slightly changed (in particular with respect to the non-emancipatory intervention), referring among others to other studies. Though, given this is a (shorter) position paper, where ideas are generally explored for further research and not a description of an existing study, it is difficult to go much more into details. 3. I have similar concerns about the questions phrased in p5 "But how could we encourage behavioural change in a more emancipatory, empowering way? (…). Rather than posing open questions, the paper could identify possible answers, pros/cons or identify challenges and solution directions. The questions have been removed, instead answers and solution directions are formulated. 4. We suggest to study multiple experimental interventions" -> I do not understand why this would be stated in a research or position paper. This would indeed be an interesting study, but in my view, a position paper should not just point out interesting research directions. In the pilot study that is discussed/referenced in the paper, two experimental interventions have been implemented, hence, in our paper we do not just point out interesting research directions (multiple interventions) but in fact try with the pilot to show how this could be accomplished. We made this point more explicit now. We also discuss in the paper what other interventions should be added to the ones implemented in the pilot. We also explain why it is necessary to implement multiple experimental interventions and test them against each other. 5. The ideas are still supported by an experimental 'pilot' study. The pilot study has not been updated so my specific remarks regarding this experiment stand. We moved further parts from the pilot study descriptions (essentially all of the results) in the paper to the Supplementary Information S3. The pilot study is now described briefly in a rather short section with references to further details in the Supplementary Information. This revision reduces further the significance of the pilot study in the paper. 6. The paper does not present a clear conclusion to the points raised but rather ends with a discussion section regarding the pilot study. No clear position is identified, lessons learned or a comprehensive methodology presented A conclusion section has now been included, where a summary and critical outlook is given based on the ideas discussed in the paper. Reviewer 3 1. Not good positioning of related work (though as mentioned before a great outlook): I would prefer the authors to discuss the related work in a narrower scope around the main positioning, i.e. experimental design in data collection. I would like to point to some very relevant work here in case authors find it useful… We expanded the discussion of experimental design in data collection and discuss now the study Klinglmayr et al. 2017 that reviewer 3 suggested. The other suggested study (Griego et al. 2017) was included in the section where other mHealth studies are discussed and in the section where experimental studies are discussed. 2. I think the following work captures some relevant social/computational dilemmas in the area of energy and transport sharing: Peter Pilgerstorfer and Evangelos Pournaras, Self-adaptive Learning in Decentralized Combinatorial Optimization-A Design Paradigm for Sharing Economies, … This study is now included in the paragraph, where the emancipatory interventions are discussed. 3. Unclear contributions as well as paper objective: stepping on two boats - position paper vs. research paper without addressing properly each one. Reviewer 1 thought that the paper is now much more clearer a position paper, though he suggested to cut further the pilot study sections in the paper. In response I indeed moved further parts (essentially all of the results) from the pilot study descriptions in the paper to the Supplementary Information. The pilot study is now described briefly in a rather short section with references to further details in the Supplementary Information. 4. Finally, on page 8, 2nd paragraph it is impossible to understand what all the statistical test mean and what purpose they serve, it seems that all numbers are packed in a paragraph. Please explain the rationale of the measurements. All statistical results have been moved to Supplementary Information S3 now, where they are now reported and explained more extensively. 5. Relation between distance and preference to higher CO2 transport means is misleading: it is unlikely in principle that someone will travel abroad by bike, for instance. The entire section on choice model results was moved to the Supplementary Information S3.4, where the models, including the relation between distance and CO2 emissions is explained further. 6. Highly controversial and (self-)conflicting statements: The authors claim that a big data approach without introducing rigorous experimental design may not be relevant to study causality. (…). However, by reading this paper the authors themselves seem to undermine their statement with several remarks they make throughout the paper. For instance, in page 4 & 5 they list several data that "should be collected", "should be obtained", (…), etc. but I am questioning this...Why "should" they be? It sounds like the big data approach, there is not clear link of how a data collection process should be connected with causality studies and what requirements such data collection processes should fulfil to provide causal evidence (lessons learnt from their field tests as well as related work could be integrated here). This would be a paper contribution, and I encourage the authors to move to this direction. We toned down statements with respect to observational data and causality, but generally, we don’t see any conflict (and it has not be flagged up by any other reviewer). We mention several times in the paper, that we want to collect “living laboratory” data and we explain, that this means big data in a field-experimental setting. So yes, it is a big data approach, but combined with an experimental design. We made that now more explicit in the respective section. We also changed all the “should” to “could” and provide justifications why we think it would be beneficial to collect certain data. 7. "The challenge is to find a way [...] users' everyday life". Fair comment, but is not this an utopia in the context of all these "should be collected" data? Given that nowadays inference can be performed even if we share zero data, e.g. social networks, friends of friends, etc. it makes sense here the authors to discuss some way to eliminate inference, e.g. privacy mechanisms (differential privacy), informational self-determination, accountability in data sharing with blockchain, etc. This refers to same paragraph as the statement above. This paragraph has been slightly revised now, providing more justification for why certain data is necessary to answer certain research questions. Additionally we added a brief paragraph discussing the issues of privacy, informational self-determination etc. 8. The same principle holds for nudging: who nudges and for what purpose? How do we make sure that nudges serve sustainability and not only corporate profits for instance? Moreover sustainability is multi-faceted, how do we encounter for rebound effects? I am not expecting the authors to address all these issues, but in the context of this discussion these aspects are relevant. In the paragraph on the nudging intervention a statement was added that acknowledges the issues with nudging. Moreover, in the new conclusion chapter we acknowledge the controversy around nudging and we also briefly discuss sustainability as a multi-faceted problem as well as the rebound effect. 9. Inconclusive results: The experimental methodology of the field test seems impossible to scale it up at large scale: several administrative actions and high level of moderation is required. What would you do and what would you change to scale up the approach to thousands of people? The section on Mobile Phone Application has now been expanded and explanations of the potential of the new software have been added. It is explicitly discussed how the features of the new software that is being developed, allow for scalability of the study. All minor comments were addressed too, e.g. correcting typos, including an introduction header, structuring large sections with subsection headers. Figure 1 (the only remaining figure in the main text) is now in black/white/grey.
5 Comments
Review
Submitted by Colleen Farrelly on
I'm not familiar enough with smartphone softwares and environmental studies to really know if this is significant or feasible. Most studies I've seen in ecology use spatial modeling. Seems like the smartphone things would be hard to use in the real world, especially in remote places.
Meta-Review by Editor
Submitted by Tobias Kuhn on
Overall, the reviewers do not make a unanimous recommendation at this time. However, in my opinion the paper has the potential to be of considerable interest to many readers. As such, I would like to conditionally accept your paper, subject to some changes. I will not send the revised version out again for review, but please submit a short memo outlining how you have addressed the issues outlined above.
1. Reviewer #3 suggests some improvement in how the methodology is presented and structured. I agree and suggest that you pay close attention to how this section is organized.
2. The pilot study is presented as an illustration of the basic methodology, which is fine. But you spent a long time discussing details of the setup, yet at the same time (correctly) dismiss your ability to draw any causal inferences given the limitations of the experimental design. I would therefore suggest that you make better use of the appendix to relegate some of the more procedural aspects, and instead further emphasize the practical setup of the experiment; what it means for students aspiring to use this method; and some of the pitfalls and difficulties that they are likely to encounter.
3. On Fig. 1 it is not clear to me where the first week ends, or whether the results from the first week are included at all.
4. Fig. 1: Are the differences between these values significant (means with overlapping CIs may still be significantly different)?
5. The style and flow could be improved throughout the paper. E.g., pay particular attention to lengthy sentences, run-ons, the use of “etc.”
Thomas Chadefaux (https://orcid.org/0000-0002-8456-8124)
Camera Ready Version
Submitted by Viktoria Spaiser on
Dear Thomas,
thanks for your feedback. I have submitted now the camera-ready version, including a response to comments. There is one issue, the name of the second author is Luzzatti not Luzatti. I had it wrong when submitting the paper for the first time and since then was not able to change it. Is there a way to correct it, please?
Many thanks!
Best wishes,
Viktorai
Name changed
Submitted by Tobias Kuhn on
Thanks! The last name of your second author is corrected.
Tobias
Acceptance
Submitted by Thomas Chadefaux on
The conditions for acceptance are now satisfied. We look forward to seeing it published soon!