Past studies identify Five Factor Model (FFM) domains that are characteristic of Borderline Personality Disorder (BPD), including those associated with specific BPD symptoms, at a between-person level. The present study replicated these between-person associations and extended past research by assessing whether the FFM explains within-person variance in the manifestation of momentary BPD symptoms in the presence or absence of close social contact (CSC). We measured CSC and the BPD core symptoms negative affectivity, impulsivity, and interpersonal problems in 74 BPD patients and in a clinical control group of 40 depressed patients over the course of 28 days, six times a day. The FFM domains showed specificity in predicting momentary BPD symptoms and interacted with CSC in doing so. In particular, for BPD individuals only, momentary impulsivity and interpersonal problems were associated with higher Neuroticism and Extraversion and lower Agreeableness, and these associations were especially strong in situations involving CSC. Negative affectivity was predicted by Neuroticism for both groups of individuals, and this association was generally unaffected by CSC. Overall, experiencing CSC was positively associated with momentary BPD symptoms. Thus, both the FFM and CSC were associated with BPD patients’ experience of symptoms in everyday life. Furthermore, specific FFM trait domains were particularly impactful in contexts where BPD symptoms are more likely to be manifested, providing further evidence that person-by-situation interactions are important for understanding BPD symptoms in the moment.