SentiBench - a benchmark comparison of state-of-the-practice sentiment analysis methods.
No Thumbnail Available
Date
2016
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
In the last few years thousands of scientific papers have investigated sentiment
analysis, several startups that measure opinions on real data have emerged and a
number of innovative products related to this theme have been developed. There are
multiple methods for measuring sentiments, including lexical-based and supervised
machine learning methods. Despite the vast interest on the theme and wide
popularity of some methods, it is unclear which one is better for identifying the
polarity (i.e., positive or negative) of a message. Accordingly, there is a strong need to
conduct a thorough apple-to-apple comparison of sentiment analysis methods, as
they are used in practice, across multiple datasets originated from different data
sources. Such a comparison is key for understanding the potential limitations,
advantages, and disadvantages of popular methods. This article aims at filling this gap
by presenting a benchmark comparison of twenty-four popular sentiment analysis
methods (which we call the state-of-the-practice methods). Our evaluation is based
on a benchmark of eighteen labeled datasets, covering messages posted on social
networks, movie and product reviews, as well as opinions and comments in news
articles. Our results highlight the extent to which the prediction performance of these
methods varies considerably across datasets. Aiming at boosting the development of
this research area, we open the methods’ codes and datasets used in this article,
deploying them in a benchmark system, which provides an open API for accessing
and comparing sentence-level sentiment analysis methods.
Description
Keywords
Sentiment analysis, Benchmark, Methods evaluation
Citation
RIBEIRO, F. N. et al. SentiBench - a benchmark comparison of state-of-the-practice sentiment analysis methods. EPJ Data Science, v. 5, p. 1-29, 2016. Disponível em: <https://epjdatascience.springeropen.com/track/pdf/10.1140/epjds/s13688-016-0085-1?site=epjdatascience.springeropen.com>. Acesso em: 02 out. 2017.