By Marieke van Erp
Reviewer has chosen not to be AnonymousOverall Impression:
UndecidedTechnical Quality of the paper:
Limited noveltyData availability:
Not all used and produced data are FAIR and openly available in established data repositories; authors need to fix thisLength of the manuscript:
The length of this manuscript is about right
Summary of paper in a few sentences:
The paper presents an approach and experiments for ontology-based topic detection and labelling where the topic labels are constructed through use of the DBpedia hierarchy. This should lead to more human-interpretable topic labels. The approach is evaluated on two different datasets.
Reasons to accept:
Interesting approach to topic labelling
Reasons to reject:
The authors state that this paper is an extension of their 2015 ICMLA paper but the contributions of this paper were already largely covered in that paper. Tables 4, 9 and 10 and figure 8 are already present in that paper and tables 7 and 8 seem to be extended somewhat. Furthermore, the results presented in Table 10 are completely identical. It is therefore unclear to me what exactly the present paper adds to the state-of-the-art besides some additional examples, some more related work and a more elaborate explanation of the approach. I recommend that the authors are more specific about their additional contributions, and would advise them to evaluate their approach on additional datasets.
One drawback of the approach seems that a preselection of ontology classes needs to be made to base the classification on (section 7.1), how well does this generalise to other domains and how labour-intensive is this? How was this pre-selection made? Did you experiment with different selection methods?
Furthermore, it would be good to share the data and system to the research community to give them an opportunity to build upon the work.
- "In this paper we will use collapsed Gibbs sampling procedure for OntoLDA topic model" -> it would be good if the authors motivate why they chose this approach
- What was the inter annotator agreement on the quantitative evaluation?
- The DBpedia ontology at present contains 685 classes (http://wiki.dbpedia.org/services-resources/ontology) where do the 5,000,000 concepts come from that is mentioned in section 7.1?
- There are some typos and missing or misplaced articles in the text, the paper would benefit from a thorough proofread.