Using a Machine Learning Tool to Support High‐Stakes Decisions in Child Protection

Abstract

Machine learning decision support tools have become popular in a range of social domains including healthcare, criminal justice, and child welfare. But the design of these tools often fails to consider the potentially complex interactions that happen between the tools and humans. This lack of human‐centered design is one reason that so few tools are actually deployed, and even if they are, struggle to achieve impact. In this article we present the example of the Allegheny Family Screening Tool, a machine learning model used since 2016 to support hotline screening of child maltreatment referrals. We describe aspects of human‐centered design that contributed to the successful deployment of this tool, including agency leadership and ownership, transparency by design, ethical oversight, community engagement, and social license. Finally, we identify potential next‐steps to encourage greater integration of human‐centered design into the development and implementation of machine learning decision support tools.