Boosting Performance in Data Science Competition Using Topic-Driven Analytics: Evidence From Recommendation System Design on Kaggle

Abstract

Research developments in the recommendation system and electronic commerce literature present more accurate and comprehensive recommendation system solutions. However, while these developments add new features to the recommendation systems, the question of whether a novel solution would excel in practice remains. Open innovation and crowdsourcing platforms are becoming an arena for designers to test their solutions in business competitions. We show how structural topical modeling identifies topical themes that improve contestant performance using forum message data during the competition period. Our topic modeling analysis identifies technological and business issues that emerge in recommendation system development. An econometric framework further investigates the link between topic distribution and performance. The multi-period difference-in-differences estimator reports no significant statistical relation when linking all message communications to the performance. However, topic-dominant and topic-dispersed messages are both found to positively and significantly impact performance. Our result shows that structural topical modeling has an essential role to critically examine the most valuable message links to boost performance. Stakeholders may prioritize the messages with specific topics and/or a mixture of topics. We provide research and practical implications for researchers, business analysts, developers, and managers to improve their experiences when engaging in recommendation system design on platforms.