Health systems worldwide are facing unprecedented pressure as the needs and expectations of patients increase and get ever more complicated. The global health system is thus,forced to leverage on every opportunity, including artificial intelligence (AI), to provide care that is consistent with patients’ needs. Meanwhile, there are serious concerns about how AI tools could threaten patients’ rights and safety. Therefore, this study maps available evidence,between January 1, 2010 to September 30, 2023, on the perceived threats posed by the usage of AI tools in healthcare on patients’ rights and safety. We deployed guidelines based on that of Tricco et al. to conduct a comprehensive search of literature from Nature, PubMed, Scopus, ScienceDirect, Dimensions, Ebsco Host, ProQuest, JStore, Semantic Scholar, Taylor & Francis, Emeralds, World Health Organisation, and Google Scholar. In keeping with the inclusion and exclusions thresholds, 14 peer reviewed articles were included in this study. We report that there is potential for breach of patients’ privacy, prejudice of race, culture, gender, social status, and that AI is also subject to errors of commission and omission. Additionally, existing regulations appeared inadequate to define standards for the use of AI tools in healthcare. Our findings have some critical implications for the achieving the Sustainable Development Goals (SDGs) 3.8, 11.7, and 16. We recommend that national governments should lead the rollout of AI tools in healthcare, key actors in the healthcare industry should contribute to developing policies on AI use in healthcare, and governments in developing countries should invest and sponsor research into AI in their healthcare system.