Collaboration between a human group and artificial intelligence can improve prediction of multiple sclerosis course: a proof-of-principle study

Tacchella, Andrea; Romano, Silvia; Ferraldeschi, Michela; Salvetti, Marco; Zaccaria, Andrea; Crisanti, A.; Grassi, Francesca

doi:10.12688/f1000research.13114.1

Cited by 29 publications

(22 citation statements)

References 24 publications

(23 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Different problems have been addressed, such as the classification of disease phase at the time of analysis [19][20][21] or evaluation of the probability of transition from Clinically Isolated Syndrome (CIS) to definite multiple sclerosis within 1 to 3 years [22][23][24]. Others have attempted to derive predictions on the course of individual patients or have investigated the variables that best predict disease evolution in time [25][26][27][28].…”

Section: Introductionmentioning

confidence: 99%

Considering patient clinical history impacts performance of machine learning models in predicting course of multiple sclerosis

et al. 2020

Self Cite

View full text Add to dashboard Cite

Multiple Sclerosis (MS) progresses at an unpredictable rate, but predictions on the disease course in each patient would be extremely useful to tailor therapy to the individual needs. We explore different machine learning (ML) approaches to predict whether a patient will shift from the initial Relapsing-Remitting (RR) to the Secondary Progressive (SP) form of the disease, using only "real world" data available in clinical routine. The clinical records of 1624 outpatients (207 in the SP phase) attending the MS service of Sant'Andrea hospital, Rome, Italy, were used. Predictions at 180, 360 or 720 days from the last visit were obtained considering either the data of the last available visit (Visit-Oriented setting), comparing four classical ML methods (Random Forest, Support Vector Machine, K-Nearest Neighbours and AdaBoost) or the whole clinical history of each patient (History-Oriented setting), using a Recurrent Neural Network model, specifically designed for historical data. Missing values were handled by removing either all clinical records presenting at least one missing parameter (Feature-saving approach) or the 3 clinical parameters which contained missing values (Record-saving approach). The performances of the classifiers were rated using common indicators, such as Recall (or Sensitivity) and Precision (or Positive predictive value). In the visit-oriented setting, the Record-saving approach yielded Recall values from 70% to 100%, but low Precision (5% to 10%), which however increased to 50% when considering only predictions for which the model returned a probability above a given "confidence threshold". For the History-oriented setting, both indicators increased as prediction time lengthened, reaching values of 67% (Recall) and 42% (Precision) at 720 days. We show how "real world" data can be effectively used to forecast the evolution of MS, leading to high Recall values and propose innovative approaches to improve Precision towards clinically useful values.

show abstract

Section: Introductionmentioning

confidence: 99%

Considering patient clinical history impacts performance of machine learning models in predicting course of multiple sclerosis

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…Understanding the conditions increasing (or decreasing) the performance of collectives is thus of key importance for several applied contexts. For example, in diagnostic decision-making, it is important to understand how to compose groups of doctors [1,22,23], and how to combine doctors with machine learning algorithms [27] to achieve collective intelligence.…”

Section: Introductionmentioning

confidence: 99%

Individuals fail to reap the collective benefits of diversity because of over-reliance on personal information

Tump

Wolf

Krause

et al. 2018

J. R. Soc. Interface.

View full text Add to dashboard Cite

Collective intelligence refers to the ability of groups to outperform individuals in solving cognitive tasks. Although numerous studies have demonstrated this effect, the mechanisms underlying collective intelligence remain poorly understood. Here, we investigate diversity in cue beliefs as a mechanism potentially promoting collective intelligence. In our experimental study, human groups observed a sequence of cartoon characters, and classified each character as a cooperator or defector based on informative and uninformative cues. Participants first made an individual decision. They then received social information consisting of their group members' decisions before making a second decision. Additionally, individuals reported their beliefs about the cues. Our results showed that individuals made better decisions after observing the decisions of others. Interestingly, individuals developed different cue beliefs, including many wrong ones, despite receiving identical information. Diversity in cue beliefs, however, did not predict collective improvement. Using simulations, we found that diverse collectives did provide better social information, but that individuals failed to reap those benefits because they relied too much on personal information. Our results highlight the potential of belief diversity for promoting collective intelligence, but suggest that this potential often remains unexploited because of over-reliance on personal information.

show abstract

“…0: still in RR phase; 1: transitioned to SP phase. DOI: 10.5256/f1000research.13114.d188355 ( Tacchella et al , 2017a )…”

Section: Data Availabilitymentioning

confidence: 99%

“…The numbering of Clinical reports is the same used in Dataset 1 . DOI: 10.5256/f1000research.13114.d188357 ( Tacchella et al , 2017c )…”

Section: Data Availabilitymentioning

confidence: 99%

Collaboration between a human group and artificial intelligence can improve prediction of multiple sclerosis course: a proof-of-principle study

et al. 2018

Self Cite

View full text Add to dashboard Cite

Background: Multiple sclerosis has an extremely variable natural course. In most patients, disease starts with a relapsing-remitting (RR) phase, which proceeds to a secondary progressive (SP) form. The duration of the RR phase is hard to predict, and to date predictions on the rate of disease progression remain suboptimal. This limits the opportunity to tailor therapy on an individual patient's prognosis, in spite of the choice of several therapeutic options. Approaches to improve clinical decisions, such as collective intelligence of human groups and machine learning algorithms are widely investigated. Methods: Medical students and a machine learning algorithm predicted the course of disease on the basis of randomly chosen clinical records of patients that attended at the Multiple Sclerosis service of Sant'Andrea hospital in Rome. Results: A significant improvement of predictive ability was obtained when predictions were combined with a weight that depends on the consistence of human (or algorithm) forecasts on a given clinical record. Conclusions: In this work we present proof-of-principle that human-machine hybrid predictions yield better prognoses than machine learning algorithms or groups of humans alone. To strengthen and generalize this preliminary result, we propose a crowdsourcing initiative to collect prognoses by physicians on an expanded set of patients.

show abstract

Collaboration between a human group and artificial intelligence can improve prediction of multiple sclerosis course: a proof-of-principle study

Cited by 29 publications

References 24 publications

Considering patient clinical history impacts performance of machine learning models in predicting course of multiple sclerosis

Considering patient clinical history impacts performance of machine learning models in predicting course of multiple sclerosis

Individuals fail to reap the collective benefits of diversity because of over-reliance on personal information

Collaboration between a human group and artificial intelligence can improve prediction of multiple sclerosis course: a proof-of-principle study

Contact Info

Product

Resources

About