A framework for automated rating of online reviews against the underlying topics

Even though the most online review systems offer star rating in addition to free text reviews, this only applies to the overall review. However, different users may have different preferences in relation to different aspects of a product or a service and may struggle to extract relevant information from a massive amount of consumer reviews available online. In this paper, we present a framework for extracting prevalent topics from online reviews and automatically rating them on a 5-star scale. It consists of five modules, including linguistic pre-processing, topic modelling, text classification, sentiment analysis, and rating. Topic modelling is used to extract prevalent topics, which are then used to classify individual sentences against these topics. A state-of-the-art word embedding method is used to measure the sentiment of each sentence. The two types of information associated with each sentence -- its topic and sentiment -- are combined to aggregate the sentiment associated with each topic. The overall topic sentiment is then projected onto the 5-star rating scale. We use a dataset of Airbnb online reviews to demonstrate a proof of concept. The proposed framework is simple and fully unsupervised. It is also domain independent, and, therefore, applicable to any other domains of products and services.

We present a framework for rating online reviews, which extracts the underlying topics automatically and rates each review against these topics. The framework consists of five modules, including

  1. linguistic pre-processing,
  2. topic modeling,
  3. text classification,
  4. sentiment analysis
  5. rating.

The following subsections provide details about each module.

The following picture provides an example of topic-related ratings for a given review. Different topics are highlighted in different colors. Each sentence is tagged with its sentiment score and topic classification at the end. The overall ratings of the given review in terms of location and amenities were calculated as 4-stars and 3-stars respectively.


For original pdf download from ACM, click here

Views: 1737


You need to be a member of Data Science Central to add comments!

Join Data Science Central

Comment by AI on October 23, 2017 at 1:21pm

Hi V Sai Venkat:

We operate under an assumption that the rating is correlated with the sentiment strength. To calculate the overall sentiment, each sentence is analyzed separately using the weighted word embeddings method.

The sentiment score indicates the polarity of the sentence: the first and third sentences are positive, the second sentence is negative. The sentiment score also reflects the strength of the overall sentiment, e.g. the first sentence and the third sentence are both positive, but the sentiment of the first sentence is stronger than that of the third sentence.

Once the topic model has been extracted from a corpus of reviews, each sentence is classified into an appropriate topic. To rate a whole review against the given topics, we used the sentiment of all sentences associated with each topic. To rate a review from on a 5-star scale (1 star being very negative and 5 star being very positive), we first normalize the sentiment score of each sentence as follows.

more details here: https://dl.acm.org/citation.cfm?id=3077291

Comment by V Sai Venkat on October 20, 2017 at 4:13pm
Whqt is the model behind projecting the sentiment score to rating scale?

© 2021   TechTarget, Inc.   Powered by

Badges  |  Report an Issue  |  Privacy Policy  |  Terms of Service