An Economic Perspective on Algorithmic Fairness

Rambachan, Ashesh; Kleinberg, Jon; Ludwig, Jens; Mullainathan, Sendhil

doi:10.1257/pandp.20201036

Cited by 51 publications

(33 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Given the known possible biases with historical data, we investigated whether protocols related to the labeling of the training data could have an impact on performance. 65 , 66…”

Section: Methodsmentioning

confidence: 99%

Topic classification of electric vehicle consumer experiences with transformer-based deep learning

Marchetto

Dharur

et al. 2021

Patterns

View full text Add to dashboard Cite

Summary The transportation sector is a major contributor to greenhouse gas (GHG) emissions and is a driver of adverse health effects globally. Increasingly, government policies have promoted the adoption of electric vehicles (EVs) as a solution to mitigate GHG emissions. However, government analysts have failed to fully utilize consumer data in decisions related to charging infrastructure. This is because a large share of EV data is unstructured text, which presents challenges for data discovery. In this article, we deploy advances in transformer-based deep learning to discover topics of attention in a nationally representative sample of user reviews. We report classification accuracies greater than 91% (F1 scores of 0.83), outperforming previously leading algorithms in this domain. We describe applications of these deep learning models for public policy analysis and large-scale implementation. This capability can boost intelligence for the EV charging market, which is expected to grow to US$27.6 billion by 2027.

show abstract

“…Given the known possible biases with historical data, we investigated whether protocols related to the labeling of the training data could have an impact on performance. 65 , 66…”

Section: Methodsmentioning

confidence: 99%

Topic classification of electric vehicle consumer experiences with transformer-based deep learning

Marchetto

Dharur

et al. 2021

Patterns

View full text Add to dashboard Cite

show abstract

“…Researchers at LinkedIn showed that the Atkinson index could be used to promote more equitable design choices when used in A/B testing (Saint-Jacques et al 2020). More generally, there has been much work by economists and social scientists to frame the questions surrounding algorithmic fairness (Rambachan et al 2020;Cowgill and Tucker 2020). One particularly relevant work emphasizes that equality and power are often better ways to frame questions of algorithmic harms, and it provides a basis for new metrics for computing quantities relevant to these issues from economic theory (Kasy and Abebe 2021).…”

Section: Prior Workmentioning

confidence: 99%

Measuring Disparate Outcomes of Content Recommendation Algorithms with Distributional Inequality Metrics

Lazovich¹,

Belli²,

Gonzales³

et al. 2022

Preprint

View full text Add to dashboard Cite

The harmful impacts of algorithmic decision systems have recently come into focus, with many examples of systems such as machine learning (ML) models amplifying existing societal biases. Most metrics attempting to quantify disparities resulting from ML algorithms focus on differences between groups, dividing users based on demographic identities and comparing model performance or overall outcomes between these groups. However, in industry settings, such information is often not available, and inferring these characteristics carries its own risks and biases. Moreover, typical metrics that focus on a single classifier's output ignore the complex network of systems that produce outcomes in real-world settings. In this paper, we evaluate a set of metrics originating from economics, distributional inequality metrics, and their ability to measure disparities in content exposure in a production recommendation system, the Twitter algorithmic timeline. We define desirable criteria for metrics to be used in an operational setting, specifically by ML practitioners. We characterize different types of engagement with content on Twitter using these metrics, and use these results to evaluate the metrics with respect to the desired criteria. We show that we can use these metrics to identify content suggestion algorithms that contribute more strongly to skewed outcomes between users. Overall, we conclude that these metrics can be useful tools for understanding disparate outcomes in online social networks.

show abstract

“…Dwork et al (2012) calls for the idea that similar individuals should be treated similarly, which requires an appropriate measure of similarity. Group-based fairness requires that algorithms have equal errors rate across groups defined by protected attributes such as race and gender (Hardt et al 2016, Kleinberg et al 2018, Zafar et al 2019, Rambachan et al 2020, Mitchell et al 2021. Popular fairness criteria include demographic parity, equal of opportunity, and equalized odds, to name a few.…”

Section: Algorithmic Fairnessmentioning

confidence: 99%

Impossibility of Collective Intelligence

Muandet¹

2022

Preprint

View full text Add to dashboard Cite

Democratization of AI involves training and deploying machine learning models across heterogeneous and potentially massive environments. Diversity of data opens up a number of possibilities to advance AI systems, but also introduces pressing concerns such as privacy, security, and equity that require special attention. This work shows that it is theoretically impossible to design a rational learning algorithm that has the ability to successfully learn across heterogeneous environments, which we decoratively call collective intelligence (CI). By representing learning algorithms as choice correspondences over a hypothesis space, we are able to axiomatize them with essential properties. Unfortunately, the only feasible algorithm compatible with all of the axioms is the standard empirical risk minimization (ERM) which learns arbitrarily from a single environment. Our impossibility result reveals informational incomparability between environments as one of the foremost obstacles for researchers who design novel algorithms that learn from multiple environments, which sheds light on prerequisites for success in critical areas of machine learning such as out-of-distribution generalization, federated learning, algorithmic fairness, and multi-modal learning.

show abstract

An Economic Perspective on Algorithmic Fairness

Cited by 51 publications

References 6 publications

Topic classification of electric vehicle consumer experiences with transformer-based deep learning

Topic classification of electric vehicle consumer experiences with transformer-based deep learning

Measuring Disparate Outcomes of Content Recommendation Algorithms with Distributional Inequality Metrics

Impossibility of Collective Intelligence

Contact Info

Product

Resources

About