This study aims to comprehensively investigate the current application of augmented reality (AR) in mathematics education through meticulous bibliometric and content analysis of existing academic literature. The primary objectives involve providing an overview of the current landscape and categorizing literature related to AR's role in teaching and learning mathematics. To achieve this, three main research questions are addressed. The first examines prevalent research themes in augmented reality in mathematics learning, shedding light on core focus areas. The second identifies and analyzes emerging research topics, reflecting the evolving nature of AR in mathematics education. The third explores potential future directions, providing insights into unexplored avenues. The study contributes to understanding how AR is utilized in mathematics education, aiding educators, researchers, and policymakers in recognizing key interests and potential areas for exploration. Findings reveal evolving research topics, including geometry, problem-solving skills, apps, and interventions associated with AR. Emerging areas, such as effective approaches and suitable mediums for integrating AR to enhance mathematical literacy skills, are highlighted. The study suggests the dynamic nature of the field, emphasizing the need for continued development in geometry and exploration of affective approaches and tailored AR-based teaching for enhanced mathematical literacy skills.