There is limited psychological literature about refugees in sub-Saharan Africa. Using a qualitative study approach, this study sought to answer the question “What factors influence the psychological well-being/frustration of refugees, and Ugandan nationals/host community?” Data was collected through 9 Focus Group Discussions (FGDs) with N = 54 participants (5 FGDs refugees and 4 FGDs Ugandans), with 6 participants in each FGD. They included males and females with Adults, youth, and school-going adolescents. Thematic analysis was used to analyze data. The FGDs, frequently reported on what contributed to their psychological well-being/frustration. These included food availability, family separations and death, good security in the refugee camp, provision of health services, access to free education, the role of mental health and psychosocial support (MHPSS), unfavorable farmland, availability of employment and income-generating activities, collaboration and peer support and delayed resettlement within or outside Uganda. Recommendations are made to provide community-based psychological interventions to increase self-reliance and reduce dependence on the few highly skilled MHPSS providers, setting up more youth collaboration activities to strengthen peaceful co-existence among refugees themselves and also between refugees and host community, designing and implementing sensitization programs regarding resettlement to overcome the aspect of over expectations and anxieties due to delay or failure in successful resettlement. This study contributes to the literature on refugees’ specific current concerns as well as those of the host community., which may require specific interventions to refugees’/nationals’ challenges.