Women are underrepresented in physical science, technology, engineering, and mathematics (pSTEM) at the undergraduate level and beyond, and there is mounting evidence that such disparities are driven by cultural and environmental influences. This study explored one underresearched environmental context—perceptions of peer connectedness within the STEM classroom—and its concurrent and longitudinal relations with outcomes of importance for undergraduate women in pSTEM, including academic major self‐efficacy, belongingness in their major, belongingness at the university, STEM interests, and course grade. First‐ and second‐year women (N = 258) who were interested in majoring in a pSTEM field self‐selected to participate in a longitudinal study across two semesters. Students' perceptions of peer connectedness in their STEM classroom had positive concurrent and longitudinal relations with academic self‐efficacy, major belonging, and university belonging (but not interest in STEM or course grade). In longitudinal analyses that controlled for earlier levels of the outcome variables, perceived peer connectedness in the classroom was also associated with greater feelings of belonging in the major the following semester, but not with self‐efficacy, university belonging, or STEM interest. These results suggest that, by fostering connections among students in the classroom, instructors may be able to increase women STEM students' broader sense of belonging within their majors.