Submodular Combinatorial Information Measures with Applications in Machine Learning

Iyer, Rishabh; Khargonkar, Ninad; Bilmes, Jeff; Asnani, Himanshu

doi:10.48550/arxiv.2006.15412

Cited by 4 publications

(6 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…are some examples [8]. Submodular Mutual Information (Smi): Given a set of items A, Q ⊆ V, the submodular mutual information (MI) [5,7] is defined as…”

Section: Preliminariesmentioning

confidence: 99%

See 1 more Smart Citation

BASIL: Balanced Active Semi-supervised Learning for Class Imbalanced Datasets

Kothawade¹,

Reddy²,

Ramakrishnan³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Current semi-supervised learning (SSL) methods assume a balance between the number of data points available for each class in both the labeled and the unlabeled data sets. However, there naturally exists a class imbalance in most real-world datasets. It is known that training models on such imbalanced datasets leads to biased models, which in turn lead to biased predictions towards the more frequent classes. This issue is further pronounced in SSL methods, as they would use this biased model to obtain psuedo-labels (on the unlabeled data) during training. In this paper, we tackle this problem by attempting to select a balanced labeled dataset for SSL that would result in an unbiased model. Unfortunately, acquiring a balanced labeled dataset from a class imbalanced distribution in one shot is challenging. We propose Basil (Balanced Active Semi-supervIsed Learning), a novel algorithm that optimizes the submodular mutual information (SMI) functions in a per-class fashion to gradually select a balanced dataset in an active learning loop. Importantly, our technique can be efficiently used to improve the performance of any SSL method. Our experiments on Path-MNIST and Organ-MNIST medical datasets for a wide array of SSL methods show the effectiveness of Basil. Furthermore, we observe that Basil outperforms the state-of-the-art diversity and uncertainty based active learning methods since the SMI functions select a more balanced dataset.

show abstract

“…are some examples [8]. Submodular Mutual Information (Smi): Given a set of items A, Q ⊆ V, the submodular mutual information (MI) [5,7] is defined as…”

Section: Preliminariesmentioning

confidence: 99%

“…For balanced subset selection via Basil we use the recently introduced Smi functions in [7,5] and their extensions introduced in [14] as acquisition functions. Note that we only use a subset of functions presented in [14], that are the most scalable, for per-class selection of data points.…”

Section: Examples Of Smi Functionsmentioning

confidence: 99%

BASIL: Balanced Active Semi-supervised Learning for Class Imbalanced Datasets

Kothawade¹,

Reddy²,

Ramakrishnan³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…are some examples [9]. Submodular Mutual Information (Smi): Given a set of items A, Q ⊆ V, the submodular mutual information (MI) [6,8] is defined as…”

Section: Preliminariesmentioning

confidence: 99%

CLINICAL: Targeted Active Learning for Imbalanced Medical Image Classification

Kothawade¹,

Atharv²,

Iyer³

et al. 2022

Preprint

View full text Add to dashboard Cite

Training deep learning models on medical datasets that perform well for all classes is a challenging task. It is often the case that a suboptimal performance is obtained on some classes due to the natural class imbalance issue that comes with medical data. An effective way to tackle this problem is by using targeted active learning, where we iteratively add data points that belong to the rare classes, to the training data. However, existing active learning methods are ineffective in targeting rare classes in medical datasets. In this work, we propose Clinical (targeted aCtive Learning for ImbalaNced medICal imAge cLassification) a framework that uses submodular mutual information functions as acquisition functions to mine critical data points from rare classes. We apply our framework to a wide-array of medical imaging datasets on a variety of real-world class imbalance scenarios -namely, binary imbalance and long-tail imbalance. We show that Clinical outperforms the state-of-the-art active learning methods by acquiring a diverse set of data points that belong to the rare classes.

show abstract

“…Wang et al [191] apply the 3-increasing condition to develop a variation of the continuous greedy algorithm, and Chen et al [44] study functions with alternating monotonicity conditions, something we will also encounter. Measures of redundancy [68] and analogues of mutual information [94] also fit the form of second derivatives and are highly relevant to works on 3-increasing functions.…”

Section: 51d Higher-order Monotonicity and Derivativesmentioning

confidence: 99%

Sensor Planning for Large Numbers of Robots

Corah

2021

Preprint

View full text Add to dashboard Cite

For my parents, David and Barbara, who supported and encouraged my interest in robotics at every step of this journey. I also have many to thank in the Resilient Intelligent Systems Lab. My officemates, Ellen and Derek were ever-present companions. Ellen, I believe our fish tank set new standards for office decor as well as undergraduate achievement. Derek, aside from being my office compatriot, you were also an early mentor when I was still an intern. Thank you for that time. And, Vibhav, you were my replacement, but I preferred being students together. Kshitij and Cormac, we collaborated on some of the best work that is not in this thesis. Thank you Curt for keeping my robots alive before I learned to avoid them. And, thanks Wennie for doing the hard work to develop systems for exploration and mapping. Vishnu, thank you for demonstrating the first feasible solution to the graduation problem. Your leadership was invaluable. Shaurya, thank you for being someone who I can talk to about research and everything else. Also, John, the pandemic has taken our pizza, and I want it back. Thanks to everyone else in the lab who has made my time here brighter, including Alex, Xuning, Aditya, Arjav, Mosam, Moses, Tim, Matt, Mike, Erik, and anyone else who I may have forgotten. Also, Karen Widmaier, you put in a good deal of hard work for our lab, and you make our hearts warmer.Many thanks to my friends in the Robotics Institute. Achal and Radhika, Dhruv and Cara, and now I am compelled to mention Xuning again, thank you for the adventures that brought so much joy to this time. Shushman, you are a caring friend. I did not introduce you to my family; I introduced my family to you. Thank you Reuben, you speak softly, but your words carry great weight. Nick, you have done great things for the RI, but you also do great things for frogs.Thanks to everyone else who contributed to my time here. Rachel Burcin, I am only one of a vast number of people who have been significantly impacted by your tireless enthusiasm administrating RISS. I could not keep up with you. Reid, you welcomed me to CMU twice, and I hope to someday become more like you. Thank you Michael Erdmann, you allowed me to babble about submodular functions in front of 16-811. Thanks Sankalp, your words in front of same class introduced me to same topic. My thanks also to Roie for validating my interest in 3-increasing functions and for our too infrequent exchanges on the topic.Thank you Nate for guiding my journey in robotics at CMU. You accepted my stubbornness and pushed me to excellence. Thanks also to the rest of my committee, Anupam, Katia, and Mac for direction and advice. I hope these first few pages are to your liking; for the next hundred, well I am not Asimov.

show abstract

Submodular Combinatorial Information Measures with Applications in Machine Learning

Cited by 4 publications

References 29 publications

BASIL: Balanced Active Semi-supervised Learning for Class Imbalanced Datasets

BASIL: Balanced Active Semi-supervised Learning for Class Imbalanced Datasets

CLINICAL: Targeted Active Learning for Imbalanced Medical Image Classification

Sensor Planning for Large Numbers of Robots

Contact Info

Product

Resources

About