Productivity and Selection of Human Capital with Machine Learning

Chalfin, Aaron; Danieli, Oren; Hillis, Andrew; Jelveh, Zubin; Luca, Michael; Ludwig, Jens; Mullainathan, Sendhil

doi:10.1257/aer.p20161029

Cited by 149 publications

(109 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The decision of which teacher to hire, however, requires a prediction: the use of information available at time of hiring to forecast individual teacher quality (Kane and Staiger 2008;Dobbie 2011;Jacob et al 2016). Chalfin et al (2016) provide some preliminary evidence of how machine learning may improve predictive accuracy in these and other personnel decisions. Chandler, Levitt, and List (2011) predict highest-risk youth so that mentoring interventions can be appropriately targeted.…”

Section: Prediction In Policymentioning

confidence: 99%

Machine Learning: An Applied Econometric Approach

Mullainathan

Spiess

2017

Journal of Economic Perspectives

Self Cite

1,304

803

View full text Add to dashboard Cite

Machines are increasingly doing “intelligent” things. Face recognition algorithms use a large dataset of photos labeled as having a face or not to estimate a function that predicts the presence y of a face from pixels x. This similarity to econometrics raises questions: How do these new empirical tools fit with what we know? As empirical economists, how can we use them? We present a way of thinking about machine learning that gives it its own place in the econometric toolbox. Machine learning not only provides new tools, it solves a different problem. Specifically, machine learning revolves around the problem of prediction, while many economic applications revolve around parameter estimation. So applying machine learning to economics requires finding relevant tasks. Machine learning algorithms are now technically easy to use: you can download convenient packages in R or Python. This also raises the risk that the algorithms are applied naively or their output is misinterpreted. We hope to make them conceptually easier to use by providing a crisper understanding of how these algorithms work, where they excel, and where they can stumble—and thus where they can be most usefully applied.

show abstract

Section: Prediction In Policymentioning

confidence: 99%

Machine Learning: An Applied Econometric Approach

Mullainathan

Spiess

2017

Journal of Economic Perspectives

Self Cite

1,304

803

View full text Add to dashboard Cite

show abstract

“…5.1 Ex post policy potential Figure 5 shows the ex post policy potential 11 The training data therefore grows over time. for each evaluation period.…”

Section: Counterfactual Policy Outcomesmentioning

confidence: 99%

Battling Antibiotic Resistance: Can Machine Learning Improve Prescribing?

Ribers

Ullrich

2019

SSRN Journal

View full text Add to dashboard Cite

Antibiotic resistance constitutes a major health threat. Predicting bacterial causes of infections is key to reducing antibiotic misuse, a leading driver of antibiotic resistance. We train a machine learning algorithm on administrative and microbiological laboratory data from Denmark to predict diagnostic test outcomes for urinary tract infections. Based on predictions, we develop policies to improve prescribing in primary care, highlighting the relevance of physician expertise and policy implementation when patient distributions vary over time. The proposed policies delay antibiotic prescriptions for some patients until test results are known and give them instantly to others. We find that machine learning can reduce antibiotic use by 7.42 percent without reducing the number of treated bacterial infections. As Denmark is one of the most conservative countries in terms of antibiotic use, this result is likely to be a lower bound of what can be achieved elsewhere.that results arrive with a delay of several days, corresponding to nearly a complete course of antibiotic treatment. Machine learning predictions promise a so far unavailable instantaneous bacterial risk assessment. We train a machine learning algorithm, a random forest, on high-dimensional, administrative data from Denmark in 2010-2012 to predict the risk of bacterial presence in laboratory test results from patient samples in primary care, the main source of antibiotic prescribing. 3 The outcome variable, an indicator variable taking the value of one when bacteria are isolated in a patient sample, is based on the microbiological test result physicians receive several days after sending in a sample. The covariates in the prediction model include each individual patient's medical outpatient claims histories, past antibiotic prescriptions, past microbiological test results, a rich set of personal characteristics such as gender, age, detailed employment status and type, education, income, civil status and more, as well as the same information on each individuals' household members. We find that machine learning applied to these data is highly capable of predicting realizations of bacterial UTI in out of sample patient test results.The relevant criterion on which our predictions need to be evaluated, however, is whether or not they can be used to improve human expert decision making. For this purpose, we model prescription decisions as a trade-off between the social cost of prescribing, i.e. promoting resistance, and the health benefits of antibiotic treatment. We build on Kleinberg et al. (2018), who use machine learning predicted risk of defendants committing a crime to show the potential improvements of judges' bail decisions. The model allows us to evaluate reassignment of antibiotic treatment based on the algorithm's prediction of risk, that is, delaying prescriptions for low risk patients until test results are available and instantly giving prescriptions to high risk patients. This reassignment is similar in spirit to Currie and MacLeod (2017) who eva...

show abstract

“…Many important decisions hinge on a prediction: managers assess future productivity for hiring; lenders forecast repayment; doctors form diagnostic and prognostic estimates; even economics PhD admissions committees assess future success (Athey et al, 2007; Chalfin et al, 2016). These predictions can be imperfect since they may rely on limited experience and faulty mental models and probabilistic reasoning.…”

Section: Introductionmentioning

confidence: 99%

“…In many other applications, such biases could loom even larger. For example, colleges admitting students, police deciding where to patrol, or firms hiring employees all maximize a complex set of preferences (Chalfin et al, 2016). Outperforming the decision maker on the single dimension we predict need not imply the decision maker is mispredicting, or that we can improve their decisions.…”

Section: Introductionmentioning

confidence: 99%

Human Decisions and Machine Predictions*

Kleinberg

Lakkaraju

Leskovec

et al. 2017

The Quarterly Journal of Economics

Self Cite

484

555

View full text Add to dashboard Cite

Can machine learning improve human decision making? Bail decisions provide a good test case. Millions of times each year, judges make jail-or-release decisions that hinge on a prediction of what a defendant would do if released. The concreteness of the prediction task combined with the volume of data available makes this a promising machine-learning application. Yet comparing the algorithm to judges proves complicated. First, the available data are generated by prior judge decisions. We only observe crime outcomes for released defendants, not for those judges detained. This makes it hard to evaluate counterfactual decision rules based on algorithmic predictions. Second, judges may have a broader set of preferences than the variable the algorithm predicts; for instance, judges may care specifically about violent crimes or about racial inequities. We deal with these problems using different econometric strategies, such as quasi-random assignment of cases to judges. Even accounting for these concerns, our results suggest potentially large welfare gains: one policy simulation shows crime reductions up to 24.7% with no change in jailing rates, or jailing rate reductions up to 41.9% with no increase in crime rates. Moreover, all categories of crime, including violent crimes, show reductions; and these gains can be achieved while simultaneously reducing racial disparities. These results suggest that while machine learning can be valuable, realizing this value requires integrating these tools into an economic framework: being clear about the link between predictions and decisions; specifying the scope of payoff functions; and constructing unbiased decision counterfactuals. JEL Codes: C10 (Econometric and statistical methods and methodology), C55 (Large datasets: Modeling and analysis), K40 (Legal procedure, the legal system, and illegal behavior)

show abstract

Productivity and Selection of Human Capital with Machine Learning

Cited by 149 publications

References 8 publications

Machine Learning: An Applied Econometric Approach

Machine Learning: An Applied Econometric Approach

Battling Antibiotic Resistance: Can Machine Learning Improve Prescribing?

Human Decisions and Machine Predictions*

Contact Info

Product

Resources

About