By Filip Ilievski https://orcid.org/0000-0002-1735-0686

Review Details

---

Reviewer has chosen not to be Anonymous

Summary of paper in a few sentences:

This paper addresses the gap between the skills of data scientists and the domain experts in its application areas, by giving an example of a cross-disciplinary course at a high-profile University. The paper discusses two relevant theories (constructivism, and transformative learning) that are relevant for applying data science methods in cross-disciplinary context.

Reasons to accept:

1) The paper is well-written, with clear and grammatical sentences.
2) The motivation for cross-disciplinary courses in education from a market perspective is clear from the introduction.
3) The paper deals with a very important gap between what the market needs and the current skills of data scientists and domain experts
4) The paper links to two relevant background theories that gives its work a good grounding.
5) The experience of the course described in this paper would be probably helpful for educators interested in cross-disciplinarity involving data science.

Reasons to reject:

1) My biggest concern with this paper is: I am not sure what exactly is its contribution. I like many pieces of this paper, but I don't know how do they come together to help the research question tackled here, or the new position advocated. Namely, is the position of this paper that there should be (more) interdisciplinary courses that rely on data science (I am not sure how novel would this be)? Or is the contribution in some of the design choices presented e.g. in section 4? Or is the contribution in understanding the relation of some decisions to the theories presented in section 2? I think this is not clear in the paper currently. Some of my points 1.1-1.5 might be resolved once this point is made clearer.
1.1) Related to this, I find the lessons learned in section 7 unclear. Namely, the lessons "software tools" and "data requirements" do not seem to be really learned from the course implementation, but are rather design decisions of the course. Some of the other lessons follow from the course, but then it is important to keep focus and mainly discuss them if they help the position presented in this paper.
1.2) Also, the author might want to link these lessons to the results in section 6.
1.3) I am not sure what is the role of the theories in section 2. I was expecting that these are used in section 7 to explain some of the outcomes from the course.
1.4) I am missing related work that is on the same level as this paper: have other people created and discussed similar cross-disciplinary courses? How does this course differ?
1.5) The tables presented in this paper are clear and informative. But I am unsure what is their relation to the position in this paper (e.g. why does it matter that there are PhDs following the course)?

2) I am wondering whether this course is truly cross-disciplinary. Namely, it seems to me that the course teaches people (from more backgrounds that just CS) to deal with big data. So this is great, and through the applications there is a frame for the course work to be applied to improve existing technology. Still, this link seems one-directional to me, from the data science domain towards application areas where the data science seems logical to apply. But for "true" cross-disciplinarity, I would expect more discussion on the application domain side, namely, looking at real, important problems in various domains that need to be solved. If the application domains are instead "toy" cases, then this course does not differ much from any data science course, because I can imagine one always works on a dataset that represents a certain domain.
I believe that the course designers/students have looked into finding meaningful application problems, but I think this could use some explanation in the paper.

Nanopublication comments:

Further comments:

- Are the answers given by the interviewees 1 to 5 explainable/expected based on their background? I would be interested if this could teach us a lesson and suggest whether some backgrounds fit better the current form of the course than others.
- I find the paragraph on page 4 "ETH Zurich offers.." too abstract and not too informative. Could it be made more specific (this might be just a matter of re-wording some of the sentences)?
- Section 4 needs a bit more focus and consistency - there is a lot of explanation of the first aspect of the course, while less/different detail is given for instance on the Machine learning part. Additionally, is "social media" not falling better under the "applications" aspect?
- Typos and grammar issues: "discipline"->"disciplines" (page 1), "learnign" (page 2), "exlusively" (page 3), "should encounter for this diversity and has the capacity" -> "should encounter for this diversity because it has the capacity"? (page 3), "students attended" -> "students that attended" (page 4), "Table 2 illustrates... " -> ""Table 2 illustrates the semester status of the students who have participated in the course" (page 5), "Step 6" -> "Step 7", "Step 7" -> "Step 8" (page 8), "with 5 students attended" -> "with 5 students that attended" (page 10), "what data mean" (page 13).

Reviewer has chosen not to be Anonymous

Summary of paper in a few sentences:

This position paper describes the experience and lessons learnt from the design and teaching of a post-graduate course to train students from multi-diverse disciplines on data science. This course is held at the ETH Zurich and has been running for three years as a 3-credit course during spring semesters. The authors focus on the novelty and challenges to design a data science course oriented to students from a broad range of different disciplines, a training typically oriented to computer science students. Specifically, the authors discussed 1) the design of the course: cross-disciplinar, practical and research-oriented, and the learning strategies applied, the constructivism (learners' prior knowledge and experience) and the transformative (learners' habits of mind and point of view) learning theories; 2) the educational profile status of the students attending the course every year; 3) experience and lessons learnt from the content design of the course, the research projects and its role as a pedagogical artifacts, and the research project teams; 4) course evaluation and students' feedback; 5) the societal implications; and outline the main conclusions drawn.

Reasons to accept:

I think the paper covers an interesting topic: how to train data science beyond the computer science student. I think this topic is significant to position the important role of data science as a necessary skill for future professionals in an every day more digital world. I really like the point that the authors make about the societal impact on training data science as cross-disciplinary. This knowledge is necessary not only to generate more productive professionals, but to provide proficiency, awarnes and freedom to citizens to better live and make decisions in a digital society. On the other hand, it is easy to read and well-structured.

Reasons to reject:

In my opinion this paper misses some discussions and i have further questions for the authors. In the introduction, i would expect more background, for instance a comparison with other post-grade training courses for data science or disciplines of similar cross-disciplinary characteristics such as Bioinformatics. In page 2, the following claim the authors backed with references: "Given the evident lack of plurality and interest for data scientists in the job market, ..." is very surprising to me, above all with an increasing number of news like [1] and [2] that state the contrary. It would be beneficial for the argumentation to discuss it in the context of these opposite analyses. In section 3, they state the lecturers of the course are not cross-disciplinary, although they all have experience in multi-disciplinary research. May this fact, the lack of presence of cross-disciplinary lecturers, have effect on the learning results of the course? Do they have plans to evaluate that?. I am also wondering if there is a reason why the course schedules during spring semester. In table 1, figures suggest an increasing lose of interest by life sciences students, could you comment on it? Is the presented course following the Bologna process [3]? Regarding some observations made by students about course evaluation in section 6, do the authors think that setting up IT technologies such as online forums, could improve and foster communication among students and tutors, and transmission of background knowledge and skills? Regarding lessons learnt exposed in section 7, in line to cultivate critical thinking and constructive doubt, what the authors think about to include an evaluation made by counterpart students on each research project as a part of the final course evaluation? and finally, do they plans to evaluate the real impact of the course in the job market?

[1] https://www.forbes.com/sites/louiscolumbus/2016/12/03/robert-half-top-te...
[2] http://www.computerweekly.com/news/450302571/UK-data-scientist-roles-gro...
[3] https://en.wikipedia.org/wiki/Bologna_Process

On the other hand, there are some other presentation issues that could be poulish: the paper is too long, repetitive sometimes (e.g. page 2, introduction explains twice what this paper is about). Some parts seem to be written in a hurry with some typos, and lists of references do not have a consistent logical order. Some suggestions:

- p2: typo 'learnign'
- p2: typo 'costructivism'
- p4: 'they concern state-of-the-art' --> 'they concern on state-of-the-art'
- Tables 1 and 2: what does 'n' denotes?
- p5: redundant sentence: 'Sections 4-6 illustrate the content and research projects of the course as well as the course evaluation and students' feedback.'
- p5: 'combining the behavioral and design science research strategies.' - i do not understant the meaning of 'behavioral' in this sentence.
- p7: in 'adjusted in an education context' --> replace by 'educational'
- p7: 'results written and orally' --> 'results in writing and orally'
- p8: 'stands the selection' --> 'stands for the selection'
- p8: 'in their illustrations' --> 'in their presentations'
- p8: 'high-quality illustration' --> 'high-quality presentation'
- p8: guide --> guideline
- p8: in the text: step 5 --> step 6
- p8: in the text: step 6 --> step 7
- p8: in the text: step 7 --> step 8
- p9: revision of verb tenses: 'this project conducted' --> 'this project was conducted'; involves --> involved; the project is --> the project was:
- p9: typo 'ran be' --> 'ran being'
- p10: 'it proved not straightforward' --> ..not to be..
- p10: 'two official evaluation' --> evaluations
- p10: 'the general satisfactions' --> satisfaction
- p10: agenda question 3 'what was ... factors' --> factor
- p11: Table 4 caption: Background --> Educational background
- section 6: could be reduced placing interview answers as Supplementary material.
- p12: i think this sentence should be vice-versa like: 'The feedback suggests that working/lab sessions during the class may motivate further the non-computer scientists to improve their knowledge as well as the computer scientists to practice their skills during the course.'
- p14: 'participating citizen' --> participatory citizen

The access to the interview survey described in section 6 is not FAIR.

Nanopublication comments:

Further comments:

Indeed this is a relevant topic for the data science community, how to train data science in regard to the job market needs. I think a paper like this is appropriate as a position paper for a data science journal, and i would like that this boosts and active and necessary debate in the community. However, i would expect a bolder claim on the societal impact of training data science for a broader scope of students. This is already stated in the paper but maybe the authors could highlight their vision on the important position of data science in a digital society.

To sum up, the paper raises a novel topic of important societal impact: education of data science for future professionals, and opens an interesting discussion on training data science for students from any educational background. But the papers suffers from verbosity and some presentation issues. It needs careful reading and polishing.

By Davide Ceolin https://orcid.org/0000-0002-3357-9130

Review Details

---

Reviewer has chosen not to be Anonymous

Summary of paper in a few sentences:

This paper discusses the experiences and lesson learnt regarding the design and teaching of a post-graduate cross-disciplinary data science course at a top-class university.

Reasons to accept:

This is a pleasant paper that focuses on an important aspect related to data science, i.e., its education. While the paper focuses mostly on educational issues, the paper illustrates the endeavor to the establishment of educational methodologies for teaching data science, especially to non-computer scientists is still in progress.
Moreover, a large part of the paper is devoted to the presentation and analysis of a data science course run at a top-class university.
Therefore, I believe that the paper can contribute to triggering a discussion on a topic which is highly relevant to the data science community.

Reasons to reject:

I do not think that the paper deserves a rejection, but I do believe that it would highly benefit from
- first of all, the ‘position’ of the author in this position paper is, in my opinion, obfuscated by their experience. While the latter is extremely valuable to support the authors’ opinions and findings, I would highlight these more;
- the paper is centered around the course that the author ran together with his colleagues. This direct experience is important, but I miss a bit of comparison with other similar courses. For example minor courses and other courses dedicated to non-computer scientists from other institutes;
- regarding the multidisciplinary aspect of data science and the challenge of balancing the computer science point of view with those from other disciplines, I would like to see a comparison with the Web Science curricula. Also in the Web Science community, a comparable discussion has been promoted, given that also Web Science is a multidisciplinary and emerging discipline. I wonder how similar and how diverse the two resulting disciplines (and curricula) are;
- in the introduction, the author states “Given the evident lack of plurality and interest for data scientists in the job market”. It seems to me that while there may be lack of plurality, the job market is increasingly interested in data scientists (but I may have a wrong perception about this). Nevertheless, I wonder if the course described contributes to tackling this problem. Given that the course has been running for three years, would it be possible to evaluate if teaching data science in this manner to non-computer scientists actually contributed to their career development?
- Sections 2 and 3 would probably benefit from a classification of the course with respect to Bloom's and Dee Fink's taxonomies;
- lastly, this paper focuses on teaching data science in a multidisciplinary context, where the multidisciplinary aspect comes from the students' background. I wonder if (and how) the lessons learnt can be used also to bring multidisciplinarity to the courses taught in the context of ‘core data science’, like the big data, data mining and machine learning courses mentioned: would it be possible/advisable to bring more multidisciplinary aspects also in these courses (or in the curricula they belong to)?

Nanopublication comments:

Further comments:

Minor Comments:

"Note that several groundbreaking discoveries in the areas of complex networks and biology [...]": please, cite (at least some of) them.
[last paragraph of Section 2]: For instance [...] for instance. Please, revise.