Probability of default estimation, with a reject option

Coenen, Lize; Abdullah, Ahmed K. A.; Guns, Tias

doi:10.1109/dsaa49011.2020.00058

Cited by 6 publications

(2 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Instead, we simply assume that the predictor is likely to be inaccurate on data points that are highly dissimilar to those samples in the training data. This yields a model-agnostic approach where r models the training data using, for example, a one-class model such as a Gaussian-mixture [20] or a One-Class Support Vector Machine (OCSVM) [3,18]. During deployment, r only passes samples to h that are similar to those found in the training data.…”

Section: Related Work On Machine Learning With a Reject Optionmentioning

confidence: 99%

Know Your Limits: Machine Learning with Rejection for Vehicle Engineering

Hendrickx

Meert

Cornelis

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

New vehicle designs need to be tested in representative driving scenarios to evaluate their durability. Because these tests are costly, only a limited number of them can be performed. These have traditionally been selected using rules of thumb, which are not always applicable to modern vehicles. Hence, there is a need to ensure that vehicle tests are aligned with their real-world usage. One possibility for obtaining a broad real-world usage overview is to exploit the data collected by sensors embedded in production vehicles. But these do not produce the detailed data needed to derive the metrics computed using expensive sensors during testing. Therefore it is necessary to correlate the low-end sensor measurements available in production vehicles with the relevant metrics acquired using high-end sensors during testing. Machine learning is a promising avenue for doing this. The key challenge is that vehicles will be used "in the wild" in many scenarios that were not encountered in the controlled testing environment, and it is unlikely that learned models will perform reliably in these previously unseen environments. We overcome this challenge by allowing learned models to abstain from making a prediction when unexpected vehicle usage is identified. We propose a general framework that combines standard machine learning with novelty detection to identify previously unseen situations. We illustrate our framework's potential on data we collected from a large-scale roadroughness analysis use case. Empirically, our approach can identify novel road types in the wild and by doing so it yields better performance.

show abstract

Section: Related Work On Machine Learning With a Reject Optionmentioning

confidence: 99%

Know Your Limits: Machine Learning with Rejection for Vehicle Engineering

Hendrickx

Meert

Cornelis

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…The novel observations can be identified as lying in low density areas [3] or outside a boundary encapsulating the training data [10,13]. Alternatively, an anomaly detector can be used to identify the deviating data such as the k-nearest neighbor outlier detector [1] or isolation forest [6]. All these methods only look at the independent variable, without assessing the accuracy of the classifier.…”

Section: Introductionmentioning

confidence: 99%

A novel reject option applied to sleep stage scoring

Pias¹,

Meert²,

Verbraecken³

et al. 2023

Proceedings of the 2023 SIAM International Conference on Data Mining (SDM)

View full text Add to dashboard Cite

Sleep stage scoring is an essential component of diagnosing sleep disorders. Unfortunately, it is a time-intensive task that requires clinical experts to annotate an entire night's recording for each patient. Therefore, machine learned models offer the potential to alleviate this burden by automating this task. While learned models achieve acceptable accuracy on curated data, these models still produce highly inaccurate scorings for certain patients when deployed in medical centers. This is because particular subsets of the population may not be adequately represented in the data used to train the model. For example, data are not easily accessible (e.g., a given age group like children) or are hard or impossible to collect (e.g., patients with a rare disease or previously unknown pathology). This creates trust issues as incorrect scorings can have severe consequences such as undetected diseases. To address this, we propose augmenting an existing model with a reject option which enables it to abstain from making predictions if the model is at an elevated risk of making a mistake. We show that traditional rejection frameworks can systematically be too cautious in certain circumstances and abstain even when the model can make good predictions. We propose a solution by considering both the data distribution and the model predictions. We demonstrate the efficacy of our method on a real-world sleep scoring use case. Moreover, we found that our approach leads to improved performance on several publicly available benchmarks.

show abstract

Algorithmic discrimination in the credit domain: what do we know about it?

2023

View full text Add to dashboard Cite

The widespread usage of machine learning systems and econometric methods in the credit domain has transformed the decision-making process for evaluating loan applications. Automated analysis of credit applications diminishes the subjectivity of the decision-making process. On the other hand, since machine learning is based on past decisions recorded in the financial institutions’ datasets, the process very often consolidates existing bias and prejudice against groups defined by race, sex, sexual orientation, and other attributes. Therefore, the interest in identifying, preventing, and mitigating algorithmic discrimination has grown exponentially in many areas, such as Computer Science, Economics, Law, and Social Science. We conducted a comprehensive systematic literature review to understand (1) the research settings, including the discrimination theory foundation, the legal framework, and the applicable fairness metric; (2) the addressed issues and solutions; and (3) the open challenges for potential future research. We explored five sources: ACM Digital Library, Google Scholar, IEEE Digital Library, Springer Link, and Scopus. Following inclusion and exclusion criteria, we selected 78 papers written in English and published between 2017 and 2022. According to the meta-analysis of this literature survey, algorithmic discrimination has been addressed mainly by looking at the CS, Law, and Economics perspectives. There has been great interest in this topic in the financial area, especially the discrimination in providing access to the mortgage market and differential treatment (different fees, number of parcels, and interest rates). Most attention has been devoted to the potential discrimination due to bias in the dataset. Researchers are still only dealing with direct discrimination, addressed by algorithmic fairness, while indirect discrimination (structural discrimination) has not received the same attention.

show abstract

Probability of default estimation, with a reject option

Cited by 6 publications

References 22 publications

Know Your Limits: Machine Learning with Rejection for Vehicle Engineering

Know Your Limits: Machine Learning with Rejection for Vehicle Engineering

A novel reject option applied to sleep stage scoring

Algorithmic discrimination in the credit domain: what do we know about it?

Contact Info

Product

Resources

About