Who Are We Listening to? Detecting User-generated Content (UGC) on the Web

Egger, Marc; Lang, Andre; Schoder, Detlef

doi:10.18151/7217308

Cited by 1 publication

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Following Egger et al (2015), we measure the agreement of the raters on each sentence, also known as inter-rater reliability. Additionally, the free-marginal multirater kappa was computed (Randolph, 2010).…”

Section: Data Codingmentioning

confidence: 99%

See 1 more Smart Citation

What Do Customers Say About My Products? Benchmarking Machine Learning Models for Need Identification

Stahlmann,

Ettrich,

Kurka

et al. 2023

Proceedings of the Annual Hawaii International Conference on System Sciences

View full text Add to dashboard Cite

Needmining is the process of extracting customer needs from user-generated content by classifying it as either informative or uninformative regarding need content. Contemporary studies achieve this by utilizing machine learning. However, models found in the literature cannot be compared to each other because they use private data for training and testing. This study benchmarks all previously suggested needmining models including CNN, SVM, RNN, and RoBERTa. To ensure an unbiased comparison, this study samples and annotates a dataset of customer reviews for products from 4 different categories from amazon. Henceforth, the dataset is publicly available and serves as a gold-set for future needmining benchmarks. RoBERTa outperformed other classifiers and seems to be best suited for needmining. The relevance of this study is reinforced by the fact that this benchmark creates a different hierarchy between models than otherwise suggested by comparing the results of previous studies.

show abstract

Section: Data Codingmentioning

confidence: 99%

“…For the free-marginal multirater kappa the distribution of the labels does not need to be known in advance. The metric is defined between -1 and 1, where -1 denotes a below-stochastic match, 0 equals a stochastic hit, and 1 represents a perfect match (Egger et al, 2015).…”

Section: Data Codingmentioning

confidence: 99%