Human alignment of neural network representations

Alignment with human representations supports robust few-shot learning

“…There are numerous names, definitions, measures, and uses of this form of alignment across various fields, including cognitive science, neuroscience, and machine learning. Some of the other names include latent space alignment (Tucker et al, 2022), concept(ual) alignment (Stolk et al, 2016;Muttenthaler et al, 2022), system alignment (Goldstone & Rogosky, 2002;Roads & Love, 2020;Aho et al, 2022), representational similarity analysis (RSA) (Kriegeskorte et al, 2008), and model alignment (Marjieh et al, 2022b). Shepard (1980) proposed that human representations can be recovered by using behavioral data to measure the similarity of a set of stimuli and then finding embeddings that satisfy those similarity associations using methods like multidimensional scaling (MDS).…”

“…Several recent studies have also attempted to identify what design choices lead to improved representational alignment in models (Kumar et al, 2022;Muttenthaler et al, 2022;Fel et al, 2022), although Moschella et al (2022) found that even with variation in design choices, many models trained on the same dataset end up learning similar 'relative representations' (embeddings projected into a relational form like a similarity matrix), or in other words, converge to the same representational space. Tucker et al (2022) showed that representational alignment emerges not only in static settings like image classification, but also dynamic reinforcement learning tasks involving human-robot interaction.…”

“…Although several recent papers have proposed ways to measure (Marjieh et al, 2022a;, explain (Muttenthaler et al, 2022;Kumar et al, 2022), and even improve (Peterson et al, 2018;Fel et al, 2022) the representational alignment of models, few have focused on studying the downstream impact of a model being representationally aligned with humans, and many studies simply rely on the intuition that better alignment leads to better performance to justify pursuing increased alignment. While there is recent evidence to suggest that alignment may help humans learn across domains and perform zero-shot generalization (Aho et al, 2022), there is also evidence to suggest that alignment may not always be beneficial for models, with models scoring low on alignment metrics achieving higher performance on downstream tasks like image classification (Kumar et al, 2022;Muttenthaler et al, 2022;Fel et al, 2022). Our goal in this paper is to conduct an in-depth study into the downstream effects of representational alignment.…”

“…We define representational alignment as the degree to which the latent representations of a model match the latent representations of humans for the same set of stimuli, and refer to models that are representationally aligned with humans as being "human-aligned." Several recent papers have proposed ways to measure (Marjieh et al, 2022a;, explain (Muttenthaler et al, 2022;Kumar et al, 2022), and even improve (Peterson et al, 2018;Fel et al, 2022) the representational alignment of models. However, many models that score low on these alignment metrics still have high performance on downstream tasks like image classification (Kumar et al, 2022;Muttenthaler et al, 2022;Fel et al, 2022).…”

“…Several recent papers have proposed ways to measure (Marjieh et al, 2022a;, explain (Muttenthaler et al, 2022;Kumar et al, 2022), and even improve (Peterson et al, 2018;Fel et al, 2022) the representational alignment of models. However, many models that score low on these alignment metrics still have high performance on downstream tasks like image classification (Kumar et al, 2022;Muttenthaler et al, 2022;Fel et al, 2022).…”