Cross-modal object recognition and dynamic weighting of sensory inputs in a fish

Abstract

Most animals use multiple sensory modalities to obtain information about objects in their environment. There is a clear adaptive advantage to being able to recognize objects cross-modally and spontaneously (without prior training with the sense being tested) as this increases the flexibility of a multisensory system, allowing an animal to perceive its world more accurately and react to environmental changes more rapidly. So far, spontaneous cross-modal object recognition has only been shown in a few mammalian species, raising the question as to whether such a high-level function may be associated with complex mammalian brain structures, and therefore absent in animals lacking a cerebral cortex. Here we use an objectdiscrimination paradigm based on operant conditioning to show, for the first time to our knowledge, that a nonmammalian vertebrate, the weakly electric fish Gnathonemus petersii, is capable of performing spontaneous cross-modal object recognition and that the sensory inputs are weighted dynamically during this task. We found that fish trained to discriminate between two objects with either vision or the active electric sense, were subsequently able to accomplish the task using only the untrained sense. Furthermore we show that crossmodal object recognition is influenced by a dynamic weighting of the sensory inputs. The fish weight object-related sensory inputs according to their reliability, to minimize uncertainty and to enable an optimal integration of the senses. Our results show that spontaneous cross-modal object recognition and dynamic weighting of sensory inputs are present in a nonmammalian vertebrate. multisensory integration | perception | sensory transfer | sensory reliability | sensory conflict

Electrosensory capture during multisensory discrimination of nearby objects in the weakly electric fish Gnathonemus petersii

“…The uni-modal tests correspond with the results of fish that were trained only with the active electric sense27. Thus, electrosensory capture is independent of possible training effects, which might have influenced the results of the electrically trained fish.…”

“…In an earlier study27, we showed that G. petersii, which were trained to discriminate between two objects using either only the active electric sense or only vision, were capable of spontaneous cross-modal object recognition. Furthermore, when trained only with the active electric sense, electrolocation dominated over vision during object discrimination at short range.…”

“…This organisation of the photoreceptors improves vision under dim light and within turbid water but comes with the cost of a relatively low spatial resolution (minimal visual angle of about 3°)25. Since this visual system has a high temporal resolution26, it mainly functions to detect fast movement of bigger objects such as predators but it also enables the fish to discriminate between objects27.…”

“…Furthermore, when trained only with the active electric sense, electrolocation dominated over vision during object discrimination at short range. With increasing object distance this dominance of the active electric sense diminished, suggesting that the sensory inputs were weighted dynamically according to their reliability27. During these experiments the fish could use only single senses to learn the task, which might have influenced the hierarchy of the senses through learning.…”

Waveform sensitivity of electroreceptors in the pulse weakly electric fish Gymnotus omarorum

“…The resulting changes in the transcutaneous electric field are transduced by cutaneous electroreceptors and encoded as train of spikes in primary electroreceptive afferents (Bullock et al, 1961;Fessard and Szabo, 1961;Lissmann, 1958;Szabo and Fessard, 1974;Wright, 1958). This information is further processed in the brain stem (Aumentado-Armstrong et al, 2015;Bell and Maler, 2005;Heiligenberg and Rose, 1985;Kawasaki, 2005) and telencephalon (Harvey Girard et al, 2010;Trinh et al, 2016), giving origin to reflex Post and von der Emde, 1999), stereotyped (Heiligenberg, 1991;Kawasaki, 2005;Moller, 1995) and learned behaviors (Jun et al, 2014;Schumacher et al, 2016;Walton and Moller, 2010).…”