Background Migrants and refugees have important health needs and face inequalities in their health status. Health care delivery to this patient group has become a challenging public health focus in high income countries. This paper summarizes current knowledge on health care delivery to migrants and refugees in high-income countries from multiple perspectives. Methods We performed a systematic literature review including primary source qualitative and quantitative studies between 2000 and 2017. Articles were excluded if the study setting was in low- or middle-income countries or focused on skilled migration. Quality assessment was done for qualitative and quantitative studies separately. Predefined variables were extracted in a standardized form. Authors were approached to provide missing information. Results Of 185 identified articles, 35 were included in the final analysis. We identified three main topics of challenges in health care delivery: communication, continuity of care and confidence. All but one study included at least one of the three main topics and in 21/35 (60%) all three topics were mentioned. We further developed the 3C model and elaborated the interrelatedness of the three topics. Additional topics identified showed that the specific regional context with legal, financial, geographical and cultural aspects is important and further influences the 3C model. Conclusions The 3C model gives a simple and comprehensive, patient-centered summary of key challenges in health care delivery for refugees and migrants. This concept is relevant to support clinicians in their day to day practice and in guiding stakeholders in priority setting for refugee and migrant health policies. Electronic supplementary material The online version of this article (10.1186/s12889-019-7049-x) contains supplementary material, which is available to authorized users.