The application of <scp>AMSTAR2</scp> in 32 overviews of systematic reviews of interventions for mental and behavioural disorders: A cross‐sectional study

Santis, Karina Karolina De; Lorenz, Robert C.; Lakeberg, Meret; Matthias, Katja

doi:10.1002/jrsm.1532

Cited by 26 publications

(39 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although this scoping review focused on evaluation strategies rather than outcomes of such evaluations, we performed an appraisal of the 30 systematic reviews included in our study with the AMSTAR2 tool. We found that the overall methodological quality of systematic reviews of digital interventions for physical activity promotion needs improvement, which has already been suggested in the context of other health interventions [ 77 - 79 ]. The overall confidence in the results of systematic reviews of digital interventions for physical activity promotion could be improved by better adherence to established reporting guidelines for systematic reviews and the prospective registration of review protocols.…”

Section: Discussionmentioning

confidence: 81%

Evaluation of Digital Interventions for Physical Activity Promotion: Scoping Review

Santis¹,

Jahnel²,

Matthias³

et al. 2022

JMIR Public Health Surveill

Self Cite

View full text Add to dashboard Cite

Background Digital interventions are interventions supported by digital tools or technologies, such as mobile apps, wearables, or web-based software. Digital interventions in the context of public health are specifically designed to promote and improve health. Recent reviews have shown that many digital interventions target physical activity promotion; however, it is unclear how such digital interventions are evaluated. Objective We aimed to investigate evaluation strategies in the context of digital interventions for physical activity promotion using a scoping review of published reviews. We focused on the target (ie, user outcomes or tool performance), methods (ie, tool data or self-reported data), and theoretical frameworks of the evaluation strategies. Methods A protocol for this study was preregistered and published. From among 300 reviews published up to March 19, 2021 in Medline, PsycINFO, and CINAHL databases, 40 reviews (1 rapid, 9 scoping, and 30 systematic) were included in this scoping review. Two authors independently performed study selection and data coding. Consensus was reached by discussion. If applicable, data were coded quantitatively into predefined categories or qualitatively using definitions or author statements from the included reviews. Data were analyzed using either descriptive statistics, for quantitative data (relative frequencies out of all studies), or narrative synthesis focusing on common themes, for qualitative data. Results Most reviews that were included in our scoping review were published in the period from 2019 to 2021 and originated from Europe or Australia. Most primary studies cited in the reviews included adult populations in clinical or nonclinical settings, and focused on mobile apps or wearables for physical activity promotion. The evaluation target was a user outcome (efficacy, acceptability, usability, feasibility, or engagement) in 38 of the 40 reviews or tool performance in 24 of the 40 reviews. Evaluation methods relied upon objective tool data (in 35/40 reviews) or other data from self-reports or assessments (in 28/40 reviews). Evaluation frameworks based on behavior change theory, including goal setting, self-monitoring, feedback on behavior, and educational or motivational content, were mentioned in 22 out of 40 reviews. Behavior change theory was included in the development phases of digital interventions according to the findings of 20 out of 22 reviews. Conclusions The evaluation of digital interventions is a high priority according to the reviews included in this scoping review. Evaluations of digital interventions, including mobile apps or wearables for physical activity promotion, typically target user outcomes and rely upon objective tool data. Behavior change theory may provide useful guidance not only for development of digital interventions but also for the evaluation of user outcomes in the context of physical activity promotion. Future research should investigate factors that could improve the efficacy of digital interventions and the standardization of terminology and reporting in this field. International Registered Report Identifier (IRRID) RR2-10.2196/35332

show abstract

Section: Discussionmentioning

confidence: 81%

Evaluation of Digital Interventions for Physical Activity Promotion: Scoping Review

Santis¹,

Jahnel²,

Matthias³

et al. 2022

JMIR Public Health Surveill

Self Cite

View full text Add to dashboard Cite

show abstract

“…This step was necessary to prevent any research waste. According to meta-research [19][20][21], many reviews of interventions in the health context are redundant because they either address the same aims as those addressed by other existing reviews or cite the same primary studies. This problem is so extensive that some reviews do not include any unique primary studies that are not cited in other reviews, and there are as many reviews as or even more reviews than there are primary studies in some fields [19][20][21].…”

Section: Protocol and Registrationmentioning

confidence: 99%

“…According to meta-research [19][20][21], many reviews of interventions in the health context are redundant because they either address the same aims as those addressed by other existing reviews or cite the same primary studies. This problem is so extensive that some reviews do not include any unique primary studies that are not cited in other reviews, and there are as many reviews as or even more reviews than there are primary studies in some fields [19][20][21]. The procedure of checking if a new review is required prior to study registration may be especially necessary in rapidly developing fields, such as digitally supported interventions, or when addressing commonly investigated outcomes, such as PA promotion.…”

Section: Protocol and Registrationmentioning

confidence: 99%

“…Systematic reviews that do not fulfill both of these items will receive critically low confidence ratings. These two items address the most common limitations in systematic reviews of interventions in the health context [20], and they were selected according to a fast and frugal decision tree for the critical appraisal of systematic reviews [25].…”

Section: Xsl • Fomentioning

confidence: 99%

“…In addition to deriving the overall confidence ratings, the AMSTAR2 scores will be used for 2 meta-research studies. Both studies will be performed because the overall confidence ratings on AMSTAR2 alone poorly discriminate among systematic reviews of various interventions in the health context [20].…”

Section: Xsl • Fomentioning

confidence: 99%

See 2 more Smart Citations

Evaluation of Digital Interventions for Physical Activity Promotion: Protocol for a Scoping Review

Santis¹,

Jahnel²,

Mergenthal³

et al. 2022

JMIR Res Protoc

Self Cite

View full text Add to dashboard Cite

Background Digital interventions (DIs) could support physical activity (PA) promotion, according to recent reviews. However, it remains unclear if and how DIs for PA promotion are evaluated; thus, it is unclear if they support behavior change in real-world settings. A mapping of evidence from published reviews is required to focus on the evaluation of DIs for PA promotion. Objective The aim of our study is to investigate evaluation strategies for any outcome in the context of DIs for PA promotion by conducting a scoping review of published reviews. Methods Our scoping review adheres to the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines. The information sources include bibliographic databases (MEDLINE, PsycINFO, and CINAHL) and the bibliographies of the selected studies. The electronic search strategy was developed and conducted in collaboration with an experienced database specialist. The electronic search was conducted in English with no limits up to March 19, 2021, for sources with the terms digital intervention AND evaluation AND physical activity in titles or abstracts. After deduplication, 300 reviews selected from 4912 search results were assessed for eligibility by 2 authors working independently. The inclusion criteria were (1) healthy or clinical samples (population), (2) DIs for PA promotion (intervention), (3) comparisons to any other intervention or no intervention (comparison), (4) evaluation strategies (methods, results, or frameworks) for any outcome in the context of DIs for PA promotion (outcome), and (5) any published review (study type). According to the consensus reached during a discussion, 40 reviews met the inclusion criteria—36 from the electronic search and 4 from the manual search of the bibliographies of the 36 reviews. All reviews reported the evaluation strategies for any outcomes in the context of DIs for PA promotion in healthy or clinical samples. Data coding and the quality appraisal of systematic reviews are currently being performed independently by 2 authors. Results Our scoping review includes data from 40 published reviews (1 rapid review, 9 scoping reviews, and 30 systematic reviews). The focus of data coding is on evaluation strategies in the context of DIs for PA promotion and on the critical appraisal of the included systematic reviews. The final consensus regarding all data is expected in early 2022. Conclusions Interventions for PA promotion that are supported by digital technologies require evaluation to ensure their efficacy in real-world settings. Our scoping review is needed because it addresses novel objectives that focus on such evaluations and are not answered in the published reviews identified in our search. The evaluation strategies addressing DIs for PA promotion will be mapped to synthesize the results that have been reported in published reviews so far. International Registered Report Identifier (IRRID) DERR1-10.2196/35332

show abstract

The application of AMSTAR2 in 32 overviews of systematic reviews of interventions for mental and behavioural disorders: A cross‐sectional study

Santis

Lorenz

Lakeberg

et al. 2021

Research Synthesis Methods

Self Cite

View full text Add to dashboard Cite

A measurement tool to assess systematic reviews, version 2' (AMSTAR2) is a 16item tool to critically appraise systematic reviews (SRs) of healthcare interventions.This study aimed to assess the methods and outcomes of AMSTAR2 appraisals in overviews of SRs of interventions for mental and behavioural disorders. The crosssectional study was conducted using 32 overviews of SRs selected from three electronic databases in January 2021. Data items included overview and SR characteristics and AMSTAR2 appraisal methods and outcomes. Data were extracted by two authors independently and narratively synthesised using descriptive statistics (means ± SD and relative frequencies). SR characteristics were compared based on AMSTAR2 appraisal outcomes using chi-square tests. The 32 overviews appraised SRs of predominantly non-pharmacological interventions for mental disorders. AMSTAR2 appraisals were reported as confidence ratings in 25/32 overviews or individual item scores in 24/32 overviews. Most SRs/overview were non-Cochrane (mean = 94%), included RCTs only (mean = 77%) and were published before AMSTAR2 release (mean = 79%). The confidence ratings derived in 25 overviews for 349 SRs were predominantly critically low (68%). Confidence ratings were similar for SRs with RCTs only versus RCTs+non-RCTs or SRs published before versus after AMSTAR2 release, while Cochrane SRs received more high+moderate than low+critically low confidence ratings (p < 0.01). Confidence ratings derived based on AMSTAR2 do not differentiate among SRs of healthcare interventions except for Cochrane SRs that fulfil the criteria for high confidence ratings. AMSTAR2 items should be consulted to avoid common weaknesses in future SRs.

show abstract

The application of AMSTAR2 in 32 overviews of systematic reviews of interventions for mental and behavioural disorders: A cross‐sectional study

Cited by 26 publications

References 33 publications

Evaluation of Digital Interventions for Physical Activity Promotion: Scoping Review

Evaluation of Digital Interventions for Physical Activity Promotion: Scoping Review

Evaluation of Digital Interventions for Physical Activity Promotion: Protocol for a Scoping Review

The application of AMSTAR2 in 32 overviews of systematic reviews of interventions for mental and behavioural disorders: A cross‐sectional study

Contact Info

Product

Resources

About