OntoLDA: An Ontology-based Topic Model for Automatic Topic Labeling

Tracking #: 492-1472

Authors:

	Name	ORCID
	Mehdi Allahyari	https://orcid.org/0000-0002-0230-5110
	Seyedamin Pouriyeh	https://orcid.org/0000-0002-5746-2914
	Krys Kochut	https://orcid.org/0000-0002-9831-9620
	Hamid Arabnia	https://orcid.org/0000-0003-3943-0094

Responsible editor:

Victor de Boer

Submission Type:

Research Paper

Abstract:

Topic models, which frequently represent topics as multinomial distributions over words, have been extensively used for discovering latent topics in text corpora. Topic labeling, which aims to assign meaningful labels for discovered topics, has recently gained significant attention. In this paper, we argue that the quality of topic labeling can be improved by considering ontology concepts rather than words alone, in contrast to previous works in this area, which usually represent topics via groups of words selected from topics. We have created: (1) a topic model that integrates ontological concepts with topic models in a single framework, where each topic is represented as a multinomial distribution over concepts and each concept is a multinomial distribution over words, and (2) a topic labeling method based on the ontological meaning of the concepts included in the discovered topics. In selecting the best topic labels, we rely on the semantic relatedness of the concepts and their ontological classifications. The results of our experiments conducted on two different data sets show that introducing ontological concepts as additional, richer features between topics and words and describing topics in terms of concepts offers an effective method for generating meaningful labels for the discovered topics.

Manuscript:

ds-paper-492.pdf

Data repository URLs:

none

Date of Submission:

Friday, June 23, 2017

Date of Decision:

Tuesday, July 18, 2017

Nanopublication URLs:

Decision:

Reject

Solicited Reviews:

Review #1 submitted on 06/Jul/2017

By Robert Hoehndorf ORCID logo

https://orcid.org/0000-0001-8149-5890

Review Details

Reviewer has chosen not to be Anonymous

Overall Impression: Bad
Suggested Decision: Reject
Technical Quality of the paper: Unable to judge
Presentation: Average
Reviewer`s confidence: High
Significance: Low significance
Background: Unable to judge
Novelty: Lack of novelty
Data availability: Not all used and produced data are FAIR and openly available in established data repositories; authors need to fix this
Length of the manuscript: This manuscript is too long for what it presents and should therefore be considerably shortened (below the general length limit)

Summary of paper in a few sentences:

The manuscript describes the OntoLDA method for topic modeling using classes from ontologies. The manuscript repeats previous results on the same topic.

Reasons to accept:

This is an interesting topic, and there is a potential to further extend OntoLDA, for example by including semantics of Description Logics more thoroughly.

Reasons to reject:

This manuscript is virtually identical, both in text and content, to a published manuscript of (some of) the authors, "Automatic Topic Labeling using Ontology-based Topic Models". I cannot see any novel contribution here that is not already present in the published manuscript. The "contributions" statement lists three contributions, of which (1) and (2) are taken verbatim from the previous manuscript, and (3) is paraphrased. The "Conclusions" sections are identical, and so is a large part of the manuscript text.
The authors need to clearly state their /novel/ contribution here, and structure their manuscript around this contribution. If the authors extended their method (something they have not done, as far as I can tell from the manuscript), they need to clearly state this as their contribution. If the authors applied their method to novel use cases or application domains (something that has not been done either), they should focus on what they have learned from these experiments.

Experimental data and results of applying the OntoLDA method should be deposited in a data repository, according to the guidelines of this journal.

Nanopublication comments:

Further comments:

Review #2 submitted on 09/Jul/2017

By Marieke van Erp ORCID logo

https://orcid.org/0000-0001-9195-8203

Review Details

Reviewer has chosen not to be Anonymous

Overall Impression: Average
Suggested Decision: Undecided
Technical Quality of the paper: Average
Presentation: Average
Reviewer`s confidence: High
Significance: Moderate significance
Background: Reasonable
Novelty: Limited novelty
Data availability: Not all used and produced data are FAIR and openly available in established data repositories; authors need to fix this
Length of the manuscript: The length of this manuscript is about right

Summary of paper in a few sentences:

The paper presents an approach and experiments for ontology-based topic detection and labelling where the topic labels are constructed through use of the DBpedia hierarchy. This should lead to more human-interpretable topic labels. The approach is evaluated on two different datasets.

Reasons to accept:

Interesting approach to topic labelling

Reasons to reject:

The authors state that this paper is an extension of their 2015 ICMLA paper but the contributions of this paper were already largely covered in that paper. Tables 4, 9 and 10 and figure 8 are already present in that paper and tables 7 and 8 seem to be extended somewhat. Furthermore, the results presented in Table 10 are completely identical. It is therefore unclear to me what exactly the present paper adds to the state-of-the-art besides some additional examples, some more related work and a more elaborate explanation of the approach. I recommend that the authors are more specific about their additional contributions, and would advise them to evaluate their approach on additional datasets.

One drawback of the approach seems that a preselection of ontology classes needs to be made to base the classification on (section 7.1), how well does this generalise to other domains and how labour-intensive is this? How was this pre-selection made? Did you experiment with different selection methods?

Furthermore, it would be good to share the data and system to the research community to give them an opportunity to build upon the work.

Nanopublication comments:

Further comments:

- "In this paper we will use collapsed Gibbs sampling procedure for OntoLDA topic model" -> it would be good if the authors motivate why they chose this approach
- What was the inter annotator agreement on the quantitative evaluation?
- The DBpedia ontology at present contains 685 classes (http://wiki.dbpedia.org/services-resources/ontology) where do the 5,000,000 concepts come from that is mentioned in section 7.1?

- There are some typos and missing or misplaced articles in the text, the paper would benefit from a thorough proofread.

2 Comments

About extension guidelines

Submitted by Victor de Boer on Tue, 07/18/2017 - 04:39

(This comment is based on a email-exchange, which I copy here for transparency).

After checking with the editors-in-chief, we came to the following preliminary guidelines (they will actually be made explicit on the submission page in the near future, so thanks for kickstarting this!)

Extensions of previous work in general are ok
The nature as an extended version must be clearly stated and a clarification on how this is an extension of previous work must be provided
There should be at least 25% new content
The new content should cover significant additional elements (not just explaining the same things with more words)

Based on this, preliminary guidelines were published on the datascience journal author page:
https://datasciencehub.net/content/guidelines-authors

Meta-Review by Editor

Submitted by Tobias Kuhn on Tue, 07/18/2017 - 06:06

Your manuscript was carefully reviewed by two reviewers, who both point out that although this is an interesting paper and an interesting approach, the main problem is that the current version is too similar to the previously published paper "Automatic Topic Labeling using Ontology-based Topic Models". Based on this consensus, I decided not to wait for a third review to come in, but to decide on these two reviews.

Most journals, including Data Science accept extended versions of workshop- or conference papers, provided that a) the paper clearly states the nature as an extended version and a clarification on how this is an extension of previous work and b) there is a significant amount of new content and additional elements (not just explaining the same things with more words). After discussion in the editorial board, it was decided to make these guidelines explicit on the journal's information pages. The minimum amount of new content was determined at 25%.

I therefore would like to encourage you to resubmit this paper to this journal, if you are able to address these issues, extend the paper significantly and address the other issues mentioned in the reviews.

Victor de Boer (http://orcid.org/0000-0001-9079-039X)