Development and validation of multivariable machine learning algorithms to predict risk of cancer in symptomatic patients referred urgently from primary care: a diagnostic accuracy study

Savage, Richard S.; Messenger, Mike; Neal, Richard D; Ferguson, Rosie; Johnston, C.; Lloyd, Katherine; Neal, Matthew D.; Sansom, Nigel; Selby, Peter J.; Sharma, Nisha; Shinkins, Bethany; Skinner, James R.; Tully, Giles; Duffy, Sean; Hall, Geoff

doi:10.1136/bmjopen-2021-053590

Cited by 8 publications

(12 citation statements)

References 11 publications

(17 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The use of the PinPoint test as a prioritisation test for referrals (i.e. use-case (2)), in addition to its role as a rule-out test, also deserves further exploration(12). Ideally future studies should consider the comparative utility of alternative rule-out and/or priority-setting referral strategies, so that the most clinically- and cost-effective option may be selected.…”

Section: Discussionmentioning

confidence: 99%

“…This study included data on a total of n=14,021 patients referred along the TWW pathway for breast cancer at LTHT, between 2011-16 [study development set] and 2017-19 [study validation set]). Full details of this study are now available in a published manuscript (12). Other key parameters for the Pinpoint arm were altered within scenario and secondary analyses, as outlined further below.…”

Section: Methodsmentioning

confidence: 99%

“…The maximum number of each clinic activity able to be undertaken at any given time was based on the number of rooms (each with one available imaging device) available at the LTHT clinic (four study are now available in a published manuscript (12). Other key parameters for the Pinpoint arm were altered within scenario and secondary analyses, as outlined further below.…”

Section: Sequence Of Clinic Activities Undertakenmentioning

confidence: 99%

“…fast-tracking referrals for these patients). Thus far, early diagnostic accuracy evidence suggests that for use-case (1) the test could avoid 20% of unnecessary referrals, thus freeing up secondary care resources to focus on expediated diagnosis for those most at risk (12).…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

An exploratory assessment of the impact of a novel risk assessment test on breast cancer clinic waiting times and workflow: a discrete event simulation model

Smith

Frempong

Sharma

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

BackgroundBreast cancer clinics across the UK have long been struggling to cope with high demand. Novel risk prediction tools – such as the PinPoint test – could help to reduce unnecessary clinic referrals. Using early data on the expected accuracy of the test, we explore the potential impact of PinPoint on: (a) the percentage of patients meeting the two-week referral target, and (b) the number of clinic ‘overspill’ appointments generated.MethodsA simulation model was built to reflect the annual flow of patients through a single UK clinic. Due to current uncertainty around the exact impact of PinPoint testing on standard care, two primary scenarios were assessed. Scenario 1 assumed complete GP adherence to testing, with only non-referred cancerous cases returning for delayed referral. Scenario 2 assumed GPs would overrule 20% of low-risk results, and that 10% of non-referred non-cancerous cases would also return for delayed referral. A range of sensitivity analyses were conducted to explore the impact of key uncertainties on the model results. Service reconfiguration scenarios, removing individual weekly clinics from the clinic schedule, were also explored.ResultsUnder standard care, 66.3% (95% CI: 66.0 to 66.5) of patients met the referral target, with 1,685 (1,648 to 1,722) overspill appointments. Under both PinPoint scenarios, >98% of patients met the referral target, with overspill appointments reduced to between 727 (707 to 746) [Scenario 1] and 886 (861 to 911) [Scenario 2]. The reduced clinic demand was sufficient to allow removal of one weekly low-capacity clinic [N=10], and the results were robust to sensitivity analyses.ConclusionsThe findings from this early analysis indicate that risk prediction tools could have the potential to alleviate pressure on cancer clinics, and are expected to have increased utility in the wake of heightened pressures resulting from the COVID-19 pandemic. Further research is required to validate these findings with real world evidence; evaluate the broader clinical and economic impact of the test; and to determine outcomes and risks for patients deemed to be low-risk on the PinPoint test and therefore not initially referred.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Sequence Of Clinic Activities Undertakenmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

An exploratory assessment of the impact of a novel risk assessment test on breast cancer clinic waiting times and workflow: a discrete event simulation model

Smith

Frempong

Sharma

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…fast-tracking referrals for these patients). Thus far, early diagnostic accuracy evidence suggests that for use-case (A) the test could avoid 20% of unnecessary referrals, thus freeing up secondary care resources to focus on expediated diagnosis for those most at risk [ 12 ].…”

Section: Introductionmentioning

confidence: 99%

An exploratory assessment of the impact of a novel risk assessment test on breast cancer clinic waiting times and workflow: a discrete event simulation model

Smith

Frempong

Sharma

et al. 2022

BMC Health Serv Res

View full text Add to dashboard Cite

Background Breast cancer clinics across the UK have long been struggling to cope with high demand. Novel risk prediction tools – such as the PinPoint test – could help to reduce unnecessary clinic referrals. Using early data on the expected accuracy of the test, we explore the potential impact of PinPoint on: (a) the percentage of patients meeting the two-week referral target, and (b) the number of clinic ‘overspill’ appointments generated (i.e. patients having to return to the clinic to complete their required investigations). Methods A simulation model was built to reflect the annual flow of patients through a single UK clinic. Due to current uncertainty around the exact impact of PinPoint testing on standard care, two primary scenarios were assessed. Scenario 1 assumed complete GP adherence to testing, with only non-referred cancerous cases returning for delayed referral. Scenario 2 assumed GPs would overrule 20% of low-risk results, and that 10% of non-referred non-cancerous cases would also return for delayed referral. A range of sensitivity analyses were conducted to explore the impact of key uncertainties on the model results. Service reconfiguration scenarios, removing individual weekly clinics from the clinic schedule, were also explored. Results Under standard care, 66.3% (95% CI: 66.0 to 66.5) of patients met the referral target, with 1,685 (1,648 to 1,722) overspill appointments. Under both PinPoint scenarios, > 98% of patients met the referral target, with overspill appointments reduced to between 727 (707 to 746) [Scenario 1] and 886 (861 to 911) [Scenario 2]. The reduced clinic demand was sufficient to allow removal of one weekly low-capacity clinic [N = 10], and the results were robust to sensitivity analyses. Conclusion The findings from this early analysis indicate that risk prediction tools could have the potential to alleviate pressure on cancer clinics, and are expected to have increased utility in the wake of heightened pressures resulting from the COVID-19 pandemic. Further research is required to validate these findings with real world evidence; evaluate the broader clinical and economic impact of the test; and to determine outcomes and risks for patients deemed to be low-risk on the PinPoint test and therefore not initially referred.

show abstract

The National Health Service urgent cancer referral pathway for suspected urological cancers: early economic evaluation of a risk prediction test

Cocco,

Smith,

Neal

et al. 2024

Int J Technol Assess Health Care

Self Cite

View full text Add to dashboard Cite

Objectives In the UK, the number of patients urgently referred for suspected cancer is increasing, and providers are struggling to cope with demand. We explore the potential cost-effectiveness of a new risk prediction test – the PinPoint test – to triage and prioritize patients urgently referred with suspected urological cancers. Methods Two simulation models were developed to reflect the diagnostic pathways for patients with (i) suspected prostate cancer, and (ii) bladder or kidney cancer, comparing the PinPoint test to current practice. An early economic analysis was conducted from a UK National Health Service (NHS) perspective. The primary outcomes were the percentage of individuals seen within 2 weeks and health care costs. An exploratory analysis was conducted to understand the potential impact of the Pinpoint test on quality-adjusted life years gained. Results Across both models and applications, the PinPoint test led to more individuals with urological cancer being seen within 2 weeks. Using PinPoint only to prioritize patients led to increased costs overall, whereas using PinPoint to both triage and prioritize patients led to cost savings. The estimated impact on life years gained/lost was very small and highly uncertain. Conclusions Using the PinPoint test to prioritize urgent referrals meant that more individuals with urological cancer were seen within 2 weeks, but at additional cost to the NHS. If used as a triage and prioritization tool, the PinPoint test shortens wait times for referred individuals and is cost saving. More data on the impact of short-term delays to diagnosis on health-related quality of life is needed.

show abstract

Development and validation of multivariable machine learning algorithms to predict risk of cancer in symptomatic patients referred urgently from primary care: a diagnostic accuracy study

Cited by 8 publications

References 11 publications

An exploratory assessment of the impact of a novel risk assessment test on breast cancer clinic waiting times and workflow: a discrete event simulation model

An exploratory assessment of the impact of a novel risk assessment test on breast cancer clinic waiting times and workflow: a discrete event simulation model

An exploratory assessment of the impact of a novel risk assessment test on breast cancer clinic waiting times and workflow: a discrete event simulation model

The National Health Service urgent cancer referral pathway for suspected urological cancers: early economic evaluation of a risk prediction test

Contact Info

Product

Resources

About