Recognizing others' social interactions is a crucial human ability. Using simple stimuli, previous studies have shown that social interactions are processed in the superior temporal sulcus (STS) and the medial prefrontal cortex (mPFC). However, it remains unknown to what extent social interaction selectivity is observed for real world stimuli where social interactions covary with other perceptual and social information, such as faces, voices, and theory of mind. By using a naturalistic fMRI movie paradigm and advanced machine learning methods, we found that broad social-affective features predict neural responses in social brain regions, including the STS and mPFC. However, only the STS showed robust and unique selectivity specifically to social interactions, independent from other covarying features. This selectivity generalized across two separate fMRI datasets. These findings suggest that naturalistic social interaction perception recruits dedicated neural circuity in the STS and is a critical dimension of human social understanding.