The early life environment markedly influences brain and behavioral development, with adverse experiences associated with increased risk of anxiety and depressive phenotypes, particularly in females. Indeed, early life adversity (ELA) in humans (i.e., caregiver deprivation, maltreatment) and rodents (i.e., maternal separation, resource scarcity) is associated with sex-specific emergence of anxious and depressive behaviors. Although these disorders show clear sex differences in humans, little attention has been paid toward evaluating sex as a biological variable in models of affective dysfunction; however, recent rodent work suggests sex-specific effects. Two widely used rodent models of ELA approximate caregiver deprivation (i.e., maternal separation) and resource scarcity (i.e., limited bedding). While these approaches model aspects of ELA experienced in humans, they span different portions of the pre-weaning developmental period and may therefore differentially contribute to underlying mechanistic risk. This is borne out in the literature, where evidence suggests differences in trajectories of behavior depending on the type of ELA and/or sex; however, the neural underpinning of these differences is not well understood. Because anxiety and depression are thought to involve dysregulation in the balance of excitatory and inhibitory signaling in ELA-vulnerable brain regions (e.g., prefrontal cortex, amygdala, hippocampus), outcomes are likely driven by alterations in local and/or circuit-specific inhibitory activity. The most abundant GABAergic subtypes in the brain, accounting for approximately 40% of inhibitory neurons, contain the calcium-binding protein Parvalbumin (PV). As PV-expressing neurons have perisomatic and proximal dendritic targets on pyramidal neurons, they are well-positioned to regulate excitatory/inhibitory balance. Recent evidence suggests that PV outcomes following ELA are sex, age, and region-specific and may be influenced by the type and timing of ELA. Here, we suggest the possibility of a combined role of PV and sex hormones driving differences in behavioral outcomes associated with affective dysfunction following ELA. This review evaluates the literature across models of ELA to characterize neural (PV) and behavioral (anxiety- and depressive-like) outcomes as a function of sex and age. Additionally, we detail a putative mechanistic role of PV on ELA-related outcomes and discuss evidence suggesting hormone influences on PV expression/function which may help to explain sex differences in ELA outcomes.