RecoBERT: A Catalog Language Model for Text-Based Recommendations

Malkiel, Itzik; Barkan, Oren; Caciularu, Avi; Razin, Noam; Katz, Ori; Koenigstein, Noam

doi:10.48550/arxiv.2009.13292

Cited by 1 publication

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Penha and Hauff [38] shows that off-theshelf pretrained BERT has both collaborative-and content-based knowledge stored in its parameters about the content of items to recommend; futhermore, fine-tuned BERT is highly effective in distinguishing relevant responses and irrelevant responses. ReX-Plug [19] exploits pretrained LMs to produce high-quality explainable recommendations by generating synthetic reviews on behalf of the user, and RecoBERT [32] builds upon BERT and introduces a technique for self-supervised pre-training of catalogue-based language models for text-based item recommendations.…”

Section: Crss and Lmsmentioning

confidence: 99%

“…Numerous studies have shown that these transformer-based LMs such as BERT [12], RoBERTa [30] and GPT [40] pretrained on large corpora can learn universal language representations and are extraordinarily powerful for many downstream tasks via fine-tuning [39]. Recently, CRSs have started to leverage pretrained LMs for their ability to semantically interpret a wide range of preference statement variations and have demonstrated their potential to build a variety of strong CRSs [19,32,38].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Unintended Bias in Language Model-driven Conversational Recommendation

Shen¹,

Li²,

Bouadjenek³

et al. 2022

Preprint

View full text Add to dashboard Cite

Conversational Recommendation Systems (CRSs) have recently started to leverage pretrained language models (LM) such as BERT for their ability to semantically interpret a wide range of preference statement variations. However, pretrained LMs are well-known to be prone to intrinsic biases in their training data, which may be exacerbated by biases embedded in domain-specific language data (e.g., user reviews) used to fine-tune LMs for CRSs. We study a recently introduced LM-driven recommendation backbone (termed LMRec) of a CRS to investigate how unintended bias -i.e., language variations such as name references or indirect indicators of sexual orientation or location that should not affect recommendations -manifests in significantly shifted price and category distributions of restaurant recommendations. The alarming results we observe strongly indicate that LMRec has learned to reinforce harmful stereotypes through its recommendations. For example, offhand mention of names associated with the black community significantly lowers the price distribution of recommended restaurants, while offhand mentions of common male-associated names lead to an increase in recommended alcohol-serving establishments. These and many related results presented in this work raise a red flag that advances in the language handling capability of LM-driven CRSs do not come without significant challenges related to mitigating unintended bias in future deployed CRS assistants with a potential reach of hundreds of millions of end users.

show abstract

Section: Crss and Lmsmentioning

confidence: 99%