Classification models are often used to make decisions that affect humans: whether to approve a loan application, extend a job offer, or provide insurance. In such applications, individuals should have the ability to change the decision of the model. When a person is denied a loan by a credit scoring model, for example, they should be able to change the input variables of the model in a way that will guarantee approval. Otherwise, this person will be denied the loan so long as the model is deployed, and -more importantlywill lack agency over a decision that affects their livelihood.In this paper, we propose to audit a linear classification model in terms of recourse, which we define as the ability of a person to change the decision of the model through actionable input variables (e.g., income vs. gender, age, or marital status). We present an integer programming toolkit to: (i) measure the feasibility and difficulty of recourse in a target population; and (ii) generate a list of actionable changes for an individual to obtain a desired outcome. We demonstrate how our tools can inform practitioners, policymakers, and consumers by auditing credit scoring models built using real-world datasets. Our results illustrate how recourse can be significantly impacted by common modeling practices, and motivate the need to guarantee recourse as a policy objective for regulation in algorithmic decision-making. arXiv:1809.06514v1 [stat.ML]
Converging investigations on the part of multiple agencies/agents have provided overwhelming evidence for Russian interference in the 2016 U.S. presidential election. As a part (and consequence) of recent reports, multiple datasets that capture actions taken by actors of the Internet Research Agency (IRA), have been released to the public. In the cur-rent paper, we present and abridged report of several preliminary forensic analyses of Facebook ad data and Twitter troll accounts that were run by the IRA during the election cycle. Through the use of language analysis, we characterize the evolution of IRA content over the course of the election cycle, providing a basis for understanding how left- and right-leaning ideologies were differentially targeted to spread enmity among the American electorate. Additionally, through an analysis of syntactic constructions, we find that the content produced by the IRA on Twitter was linguistically unique from a control sample of English-speaking Twitter accounts. Altogether, our findings suggest that the IRA’s operations were largely unsophisticated and “low-budget” in nature, with no serious attempts at point-of-origin obfuscation being taken.
As labeling schemas evolve over time, small differences can render datasets following older schemas unusable. This prevents researchers from building on top of previous annotation work and results in the existence, in discourse learning in particular, of many small classimbalanced datasets. In this work, we show that a multitask learning approach can combine discourse datasets from similar and diverse domains to improve discourse classification. We show an improvement of 4.9% Micro F1-score over current state-of-the-art benchmarks on the NewsDiscourse dataset, one of the largest discourse datasets recently published, due in part to label correlations across tasks, which improve performance for underrepresented classes. We also offer an extensive review of additional techniques proposed to address resource-poor problems in NLP, and show that none of these approaches can improve classification accuracy in our setting 1 .
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.