Labeling images with facial emotion and the potential for pediatric healthcare

Kalantarian, Haik; Jedoui, Khaled; Washington, Peter; Tariq, Qandeel; Dunlap, Kaiti; Schwartz, Jessey; Wall, Dennis P.

doi:10.1016/j.artmed.2019.06.004

Cited by 87 publications

(53 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…However, most classifiers are biased towards neurotypical adults and can fail to generalise to children with ASD. To address this, Kalantarian et al 36 , 37 presented a framework for semi-automatic label frame extraction to crowdsource labelled emotion data from children. The labels consist of six emotions: disgust, neutral, surprise, scared, angry and happy.…”

Section: Resultsmentioning

confidence: 99%

Computer vision in autism spectrum disorder research: a systematic review of published studies from 2009 to 2019

et al. 2020

View full text Add to dashboard Cite

The current state of computer vision methods applied to autism spectrum disorder (ASD) research has not been well established. Increasing evidence suggests that computer vision techniques have a strong impact on autism research. The primary objective of this systematic review is to examine how computer vision analysis has been useful in ASD diagnosis, therapy and autism research in general. A systematic review of publications indexed on PubMed, IEEE Xplore and ACM Digital Library was conducted from 2009 to 2019. Search terms included [‘autis*’ AND (‘computer vision’ OR ‘behavio* imaging’ OR ‘behavio* analysis’ OR ‘affective computing’)]. Results are reported according to PRISMA statement. A total of 94 studies are included in the analysis. Eligible papers are categorised based on the potential biological/behavioural markers quantified in each study. Then, different computer vision approaches that were employed in the included papers are described. Different publicly available datasets are also reviewed in order to rapidly familiarise researchers with datasets applicable to their field and to accelerate both new behavioural and technological work on autism research. Finally, future research directions are outlined. The findings in this review suggest that computer vision analysis is useful for the quantification of behavioural/biological markers which can further lead to a more objective analysis in autism research.

show abstract

Section: Resultsmentioning

confidence: 99%

Computer vision in autism spectrum disorder research: a systematic review of published studies from 2009 to 2019

et al. 2020

View full text Add to dashboard Cite

show abstract

“…There are several limitations of the present study and fruitful avenues for future work. More structured videos, such as those collected in home smartphone autism interventions [ 15 , 16 , 17 , 18 , 19 ], may yield more consistent video difficulty levels due to the standardization of collected videos. Mobile therapeutics in conjunction with crowdsourcing may be leveraged toward longitudinal outcome tracking of symptoms [ 7 ].…”

Section: Discussionmentioning

confidence: 99%

“…Since autism consists of a largely behavioral phenotype, video data are a particularly powerful and rich means of capturing the range of social symptoms a child may exhibit in a fast and virtually cost-free manner. Accurate diagnoses and behavioral classifications have been inferred from categorical ordinal labels extracted by untrained humans from the short video clips [ 5 , 6 , 7 , 8 , 9 , 10 ], which are recorded by digital mobile and wearable interventions during use by the child or administering parent [ 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 ]. Such a process can be scaled through crowdsourcing platforms, which allow distributed workers from around the globe to perform short on-demand tasks.…”

Section: Introductionmentioning

confidence: 99%

Precision Telemedicine through Crowdsourced Machine Learning: Testing Variability of Crowd Workers for Video-Based Autism Feature Recognition

Washington

Leblanc

Dunlap

et al. 2020

JPM

Self Cite

View full text Add to dashboard Cite

Mobilized telemedicine is becoming a key, and even necessary, facet of both precision health and precision medicine. In this study, we evaluate the capability and potential of a crowd of virtual workers—defined as vetted members of popular crowdsourcing platforms—to aid in the task of diagnosing autism. We evaluate workers when crowdsourcing the task of providing categorical ordinal behavioral ratings to unstructured public YouTube videos of children with autism and neurotypical controls. To evaluate emerging patterns that are consistent across independent crowds, we target workers from distinct geographic loci on two crowdsourcing platforms: an international group of workers on Amazon Mechanical Turk (MTurk) (N = 15) and Microworkers from Bangladesh (N = 56), Kenya (N = 23), and the Philippines (N = 25). We feed worker responses as input to a validated diagnostic machine learning classifier trained on clinician-filled electronic health records. We find that regardless of crowd platform or targeted country, workers vary in the average confidence of the correct diagnosis predicted by the classifier. The best worker responses produce a mean probability of the correct class above 80% and over one standard deviation above 50%, accuracy and variability on par with experts according to prior studies. There is a weak correlation between mean time spent on task and mean performance (p= 0.358, p = 0.005). These results demonstrate that while the crowd can produce accurate diagnoses, there are intrinsic differences in crowdworker ability to rate behavioral features. We propose a novel strategy for recruitment of crowdsourced workers to ensure high quality diagnostic evaluations of autism, and potentially many other pediatric behavioral health conditions. Our approach represents a viable step in the direction of crowd-based approaches for more scalable and affordable precision medicine.

show abstract

“…Five studies used crowdsourcing to develop artificial intelligence projects [53][54][55][56][57]. Four of these studies annotated medical data to train machine learning algorithms [53,[55][56][57]. One study found that a three-phase crowdsourcing challenge contest could be used to develop an artificial intelligence algorithm to segment lung tumors for radiation therapy [54].…”

Section: Synthesizing Evidencementioning

confidence: 99%

“…We also found that crowdsourcing may be useful in the development of artificial intelligence projects. Four studies annotated medical data in order to train machine learning algorithms [53,[55][56][57]. Especially as crowdsourcing solicits input from large numbers of people, the resulting big data may provide a platform for machine learning.…”

Section: Strengths and Limitations Of Studymentioning

confidence: 99%

Crowdsourcing in health and medical research: a systematic review

et al. 2020

View full text Add to dashboard Cite

Background: Crowdsourcing is used increasingly in health and medical research. Crowdsourcing is the process of aggregating crowd wisdom to solve a problem. The purpose of this systematic review is to summarize quantitative evidence on crowdsourcing to improve health. Methods: We followed Cochrane systematic review guidance and systematically searched seven databases up to September 4th 2019. Studies were included if they reported on crowdsourcing and related to health or medicine. Studies were excluded if recruitment was the only use of crowdsourcing. We determined the level of evidence associated with review findings using the GRADE approach. Results: We screened 3508 citations, accessed 362 articles, and included 188 studies. Ninety-six studies examined effectiveness, 127 examined feasibility, and 37 examined cost. The most common purposes were to evaluate surgical skills (17 studies), to create sexual health messages (seven studies), and to provide layperson cardiopulmonary resuscitation (CPR) out-of-hospital (six studies). Seventeen observational studies used crowdsourcing to evaluate surgical skills, finding that crowdsourcing evaluation was as effective as expert evaluation (low quality). Four studies used a challenge contest to solicit human immunodeficiency virus (HIV) testing promotion materials and increase HIV testing rates (moderate quality), and two of the four studies found this approach saved money. Three studies suggested that an interactive technology system increased rates of layperson initiated CPR out-ofhospital (moderate quality). However, studies analyzing crowdsourcing to evaluate surgical skills and laypersoninitiated CPR were only from high-income countries. Five studies examined crowdsourcing to inform artificial intelligence projects, most often related to annotation of medical data. Crowdsourcing was evaluated using different outcomes, limiting the extent to which studies could be pooled. Conclusions: Crowdsourcing has been used to improve health in many settings. Although crowdsourcing is effective at improving behavioral outcomes, more research is needed to understand effects on clinical outcomes and costs. More research is needed on crowdsourcing as a tool to develop artificial intelligence systems in medicine.

show abstract

Labeling images with facial emotion and the potential for pediatric healthcare

Cited by 87 publications

References 32 publications

Computer vision in autism spectrum disorder research: a systematic review of published studies from 2009 to 2019

Computer vision in autism spectrum disorder research: a systematic review of published studies from 2009 to 2019

Precision Telemedicine through Crowdsourced Machine Learning: Testing Variability of Crowd Workers for Video-Based Autism Feature Recognition

Crowdsourcing in health and medical research: a systematic review

Contact Info

Product

Resources

About