Gain-Some-Lose-Some: Reliable Quantification Under General Dataset Shift

This chapter sets the stage for the rest of the book by introducing notions fundamental to quantification, such as class proportions, class distributions and their estimation, dataset shift, and the various subtypes of dataset shift which are relevant to the quantification endeavour. In this chapter we also argue why using classification techniques for estimating class distributions is suboptimal, and we then discuss why learning to quantify has evolved as a task of its own, rather than remaining a by-product of classification.

show abstract

The Case for Quantification

Esuli

Fabris

Moreo

et al. 2023

The Information Retrieval Series

View full text Add to dashboard Cite

show abstract

Applications of Quantification

Esuli

Fabris

Moreo

et al. 2023

The Information Retrieval Series

View full text Add to dashboard Cite

This chapter provides the motivation for what is to come in the rest of the book by describing the applications that quantification has been put at, ranging from improving classification accuracy in domain adaptation, to measuring and improving the fairness of classification systems with respect to a sensitive attribute, to supporting research and development in fields that are usually more concerned with aggregate data than with individual data, such as the social sciences, political science, epidemiology, market research, ecological modelling, and others.

show abstract

Evaluation of Quantification Algorithms

Esuli

Fabris²,

Moreo³

et al. 2023

The Information Retrieval Series

View full text Add to dashboard Cite

In this chapter we discuss the experimental evaluation of quantification systems. We look at evaluation measures for the various types of quantification systems (binary, single-label multiclass, multi-label multiclass, ordinal), but also at evaluation protocols for quantification, that essentially consist in ways to extract multiple testing samples for use in quantification evaluation from a single classification test set. The chapter ends with a discussion on how to perform model selection (i.e., hyperparameter optimization) in a quantification-specific way.

show abstract

scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.

Contact Info

hi@scite.ai

10624 S. Eastern Ave., Ste. A-614

Henderson, NV 89052, USA

Blog Terms and Conditions API Terms Privacy Policy Contact Cookie Preferences Do Not Sell or Share My Personal Information

Made with 💙 for researchers

Part of the Research Solutions Family.

Gain-Some-Lose-Some: Reliable Quantification Under General Dataset Shift

Cited by 8 publications

References 8 publications

The Case for Quantification

The Case for Quantification

Applications of Quantification

Evaluation of Quantification Algorithms

Contact Info

Product

Resources

About