As biodiversity decreases worldwide, the development of effective techniques to track changes in ecological communities becomes an urgent challenge. Together with other emerging methods in ecology, acoustic indices are increasingly being used as novel tools for rapid biodiversity assessment. These indices are based on mathematical formulae that summarise the acoustic features of audio samples, with the aim of extracting meaningful ecological information from soundscapes. However, the application of this automated method has revealed conflicting results across the literature, with conceptual and empirical controversies regarding its primary assumption: a correlation between acoustic and biological diversity. After more than a decade of research, we still lack a statistically informed synthesis of the power of acoustic indices that elucidates whether they effectively function as proxies for biological diversity. Here, we reviewed studies testing the relationship between diversity metrics (species abundance, species richness, species diversity, abundance of sounds, and diversity of sounds) and the 11 most commonly used acoustic indices. From 34 studies, we extracted 364 effect sizes that quantified the magnitude of the direct link between acoustic and biological estimates and conducted a meta-analysis. Overall, acoustic indices had a moderate positive relationship with the diversity metrics (r = 0.33, CI [0.23, 0.43]), and showed an inconsistent performance, with highly variable effect sizes both within and among studies. Over time, studies have been increasingly disregarding the validation of the acoustic estimates and those examining this link have been progressively reporting smaller effect sizes. Some of the studied indices [acoustic entropy index (H), normalised difference soundscape index (NDSI), and acoustic complexity index (ACI)] performed better in retrieving biological information, with abundance of sounds (number of sounds from identified or unidentified species) being the best estimated diversity facet of local communities. We found no effect of the type of monitored environment (terrestrial versus aquatic) and the procedure for extracting biological information (acoustic versus non-acoustic) on the performance of acoustic indices, suggesting certain potential to generalise their application across research contexts. We also identified common statistical issues and knowledge gaps that remain to be addressed in future research, such as a high rate of pseudoreplication and multiple unexplored combinations of metrics, taxa, and regions. Our findings confirm the limitations of acoustic indices to efficiently quantify alpha biodiversity and highlight that caution is necessary when using them as surrogates of diversity metrics, especially if employed as single predictors. Although these tools are able partially to capture changes in diversity metrics, endorsing to some extent the rationale behind acoustic indices and suggesting them as promising bases for future developments, they are far from being direct pr...