Maintaining Curated Document Databases Using a Learning to Rank Model: The ORRCA Experience

Muhammad, Iqra; Bollegala, Danushka; Coenen, Frans; Gamble, Carrol; Kearney, Anna; Williamson, Paula

doi:10.1007/978-3-030-63799-6_26

Cited by 3 publications

(2 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Even where articles were within the scope of this review, authors used considerably different criteria for assessing retention, such as outcome data return or availability, attendance at a clinic visit, participant status at a specific point in follow up or completion of all required research activities. Going forward, ORRCA2 search strategies will be optimised based on the identified literature, and text mining will be utilised in order to efficiently maintain the resource 40,41 . However, we recommend that in the absence of a universally agreed definitions, authors must clearly describe their criteria and approach to the measurement of retention to allow readers to assess their relevance better.…”

Section: Discussionmentioning

confidence: 99%

“…Going forward, ORRCA2 search strategies will be optimised based on the identified literature, and text mining will be utilised in order to efficiently maintain the resource. 40 , 41 However, we recommend that in the absence of a universally agreed definitions, authors must clearly describe their criteria and approach to the measurement of retention to allow readers to assess their relevance better.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Developing an online, searchable database to systematically map and organise current literature on retention research (ORRCA2)

et al. 2021

Self Cite

View full text Add to dashboard Cite

Background Addressing recruitment and retention challenges in trials is a key priority for methods research, but navigating the literature is difficult and time-consuming. In 2016, ORRCA ( www.orrca.org.uk ) launched a free, searchable database of recruitment research that has been widely accessed and used to support the update of systematic reviews and the selection of recruitment strategies for clinical trials. ORRCA2 aims to create a similar database to map the growing volume and importance of retention research. Methods Searches of Medline (Ovid), CINAHL, PsycINFO, Scopus, Web of Science Core Collection and the Cochrane Library, restricted to English language and publications up to the end of 2017. Hand searches of key systematic reviews were undertaken and randomised evaluations of recruitment interventions within the ORRCA database on 1 October 2020 were also reviewed for any secondary retention outcomes. Records were screened by title and abstract before obtaining the full text of potentially relevant articles. Studies reporting or evaluating strategies, methods and study designs to improve retention within healthcare research were eligible. Case reports describing retention challenges or successes and studies evaluating participant reported reasons for withdrawal or losses were also included. Studies assessing adherence to treatments, attendance at appointments outside of research and statistical analysis methods for missing data were excluded. Eligible articles were categorised into one of the following evidence types: randomised evaluations, non-randomised evaluations, application of retention strategies without evaluation and observations of factors affecting retention. Articles were also mapped against a retention domain framework. Additional data were extracted on research outcomes, methods and host study context. Results Of the 72,904 abstracts screened, 4,364 full texts were obtained, and 1,167 articles were eligible. Of these, 165 (14%) were randomised evaluations, 99 (8%) non-randomised evaluations, 319 (27%) strategies without evaluation and 584 (50%) observations of factors affecting retention. Eighty-four percent (n = 979) of studies assessed the numbers of participants retained, 27% (n = 317) assessed demographic differences between retained and lost participants, while only 4% (n = 44) assessed the cost of retention strategies. The most frequently reported domains within the 165 studies categorised as ‘randomised evaluations of retention strategies’ were participant monetary incentives (32%), participant reminders and prompts (30%), questionnaire design (30%) and data collection location and method (26%). Conclusion ORRCA2 builds on the success of ORRCA extending the database to organise the growing volume of retention research. Less than 15% of articles were randomised evaluations of retention strategies. Mapping of the literature highlights several areas for future research such as the role of research sites, clinical staff and study design in enhancing retention. Future studies should also include cost–benefit analysis of retention strategies.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Developing an online, searchable database to systematically map and organise current literature on retention research (ORRCA2)

et al. 2021

Self Cite

View full text Add to dashboard Cite

show abstract

Document Ranking for Curated Document Databases Using BERT and Knowledge Graph Embeddings: Introducing GRAB-Rank

Muhammad

Bollegala

Coenen

et al. 2021

Big Data Analytics and Knowledge Discovery

View full text Add to dashboard Cite

The overlap between randomised evaluations of recruitment and retention interventions: An updated review of recruitment (Online Resource for Recruitment in Clinical triAls) and retention (Online Resource for Retention in Clinical triAls) literature

Kearney,

Butlin,

Coffey

et al. 2024

Clinical Trials

View full text Add to dashboard Cite

Background The Online Resource for Recruitment in Clinical triAls (ORRCA) and the Online Resource for Retention in Clinical triAls (ORRCA2) were established to organise and map the literature addressing participant recruitment and retention within clinical research. The two databases are updated on an ongoing basis using separate but parallel systematic reviews. However, recruitment and retention of research participants is widely acknowledged to be interconnected. While interventions aimed at addressing recruitment challenges can impact retention and vice versa, it is not clear how well they are simultaneously considered within methodological research. This study aims to report the recent update of ORRCA and ORRCA2 with a special emphasis on assessing crossover of the databases and how frequently randomised studies of methodological interventions measure the impact on both recruitment and retention outcomes. Methods Two parallel systematic reviews were conducted in line with previously reported methods updating ORRCA (recruitment) and ORRCA2 (retention) with publications from 2018 and 2019. Articles were categorised according to their evidence type (randomised evaluation, non-randomised evaluation, application and observation) and against the recruitment and retention domain frameworks. Articles categorised as randomised evaluations were compared to identify studies appearing in both databases. For randomised studies that were only in one database, domain categories were used to assess whether the methodological intervention was likely to impact on the alternate construct. For example, whether a recruitment intervention might also impact retention. Results In total, 806 of 17,767 articles screened for the recruitment database and 175 of 18,656 articles screened for the retention database were added as result of the update. Of these, 89 articles were classified as ‘randomised evaluation’, of which 6 were systematic reviews and 83 were randomised evaluations of methodological interventions. Ten of the randomised studies assessed recruitment and retention and were included in both databases. Of the randomised studies only in the recruitment database, 48/55 (87%) assessed the content or format of participant information which could have an impact on retention. Of the randomised studies only in the retention database, 6/18 (33%) assessed monetary incentives, 4/18 (22%) assessed data collection location and methods and 3/18 (17%) assessed non-monetary incentives, all of which could have an impact on recruitment. Conclusion Only a small proportion of randomised studies of methodological interventions assessed the impact on both recruitment and retention despite having a potential impact on both outcomes. Where possible, an integrated approach analysing both constructs should be the new standard for these types of evaluations to ensure that improvements to recruitment are not at the expense of retention and vice versa.

show abstract

Maintaining Curated Document Databases Using a Learning to Rank Model: The ORRCA Experience

Cited by 3 publications

References 15 publications

Developing an online, searchable database to systematically map and organise current literature on retention research (ORRCA2)

Developing an online, searchable database to systematically map and organise current literature on retention research (ORRCA2)

Document Ranking for Curated Document Databases Using BERT and Knowledge Graph Embeddings: Introducing GRAB-Rank

The overlap between randomised evaluations of recruitment and retention interventions: An updated review of recruitment (Online Resource for Recruitment in Clinical triAls) and retention (Online Resource for Retention in Clinical triAls) literature

Contact Info

Product

Resources

About