Adaptive Margin Diversity Regularizer for Handling Data Imbalance in Zero-Shot SBIR

Dutta, Titir; Singh, Anurag; Biswas, Soma

doi:10.1007/978-3-030-58558-7_21

Cited by 21 publications

(18 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…IV. RESULTS AND DISCUSSIONS We compare the performance of the proposed model with some of the existing state-of-the-art frameworks of [9]- [12], [14], [17], [18], [21], [34]. We also lay down the performances of some of the notable works in SBIR that uses the same datasets to show how the proposed model which solves a more challenging ZS-SBIR problem achieves comparable performance.…”

Section: Sketchy-ext Tu Berlin-extmentioning

confidence: 99%

“…The Doo-dle2search [10] uses a triplet architecture and uses gradient reversal layers to enforce learning domain agnostic features from image and sketches. On the other hand, [17] pointed out that the fall in the performance attributed to the imbalance in the class-wise samples can be tackled by using a novel adaptive margin regularizer. While the above-mentioned frameworks try to improve on the accuracy of the model by synthesizing a real-valued feature space, models such as [9], [16], [19] propose a hashed feature space which makes the retrieval a highly efficient process.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Zero-Shot Sketch Based Image Retrieval using Graph Transformer

Sumrit¹,

Chaudhuri²,

Banerjee³

2022

Preprint

View full text Add to dashboard Cite

The performance of a zero-shot sketch-based image retrieval (ZS-SBIR) task is primarily affected by two challenges. The substantial domain gap between image and sketch features needs to be bridged, while at the same time the side information has to be chosen tactfully. Existing literature has shown that varying the semantic side information greatly affects the performance of ZS-SBIR. To this end, we propose a novel graph transformer based zero-shot sketch-based image retrieval (GTZSR) framework for solving ZS-SBIR tasks which uses a novel graph transformer to preserve the topology of the classes in the semantic space and propagates the context-graph of the classes within the embedding features of the visual space. To bridge the domain gap between the visual features, we propose minimizing the Wasserstein distance between images and sketches in a learned domain-shared space. We also propose a novel compatibility loss that further aligns the two visual domains by bridging the domain gap of one class with respect to the domain gap of all other classes in the training set. Experimental results obtained on the extended Sketchy, TU-Berlin, and QuickDraw datasets exhibit sharp improvements over the existing state-ofthe-art methods in both ZS-SBIR and generalized ZS-SBIR.

show abstract

Section: Sketchy-ext Tu Berlin-extmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Zero-Shot Sketch Based Image Retrieval using Graph Transformer

Sumrit¹,

Chaudhuri²,

Banerjee³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…So in the training sequence, we need to carefully take care of this fact by either using the weighted average of the number of samples in each class distribution or cleverly select the batches while training the network. Some of the commonly used strategies for handling data imbalance in literature are based on re-balancing the dataset and cost-sensitive learning of classifier [51]. Naive-over and undersampling [52], selective decontamination [53], SMOTE [54], GAN based augmentation [55] are some of the commonly used re-balancing techniques.…”

Section: B Training Data Imbalancementioning

confidence: 99%

“…Naive-over and undersampling [52], selective decontamination [53], SMOTE [54], GAN based augmentation [55] are some of the commonly used re-balancing techniques. The cons-sensitive learning approach involves adding focal loss [56] and diversity regularizers [51].…”

Section: B Training Data Imbalancementioning

confidence: 99%

Interband Retrieval and Classification Using the Multilabeled Sentinel-2 BigEarthNet Archive

Chaudhuri

Dey

Datcu

et al. 2021

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Conventional remote sensing data analysis techniques have a significant bottleneck of operating on a selectively chosen small-scale dataset. Availability of an enormous volume of data demands handling large-scale, diverse data, which have been made possible with neural network-based architectures. This work exploits the contextual information capturing ability of deep neural networks (DNNs), particularly investigating multispectral band properties from Sentinel-2 image patches. Besides, an increase in the spatial resolution often leads to non-linear mixing of land-cover types within a target resolution cell. We recognize this fact and group the bands according to their spatial resolutions, and propose a classification and retrieval framework. We design a representation learning framework for classifying the multi-spectral data by first utilizing all the bands and then using the grouped bands according to their spatial resolutions. We also propose a novel triplet-loss function for multi-labeled images and use it to design an inter-band group retrieval framework. We demonstrate its effectiveness over the conventional tripletloss function. Finally, we present a comprehensive discussion of the obtained results. We thoroughly analyze the performance of the band groups on various land-cover and land-use areas from agro-forestry regions, water bodies, and human-made structures. Experimental results for the classification and retrieval framework on the benchmarked BigEarthNet dataset exhibit marked improvements over existing studies.

show abstract

“…The use of LSTM to forecast inflows has not been attempted to the best of our knowledge. Considerable success has been achieved with the use of machine learning techniques which are able to solve wide range of problems in optimization and operations research [3,4] (Singh, 2019 [5,6](dutta 2020). Recently attempts have been made for successful determination of precipitation but naive deep learning based architectures are difficult to train them to learn and investigate temporal correlation over arbitrary length.…”

Section: Introductionmentioning

confidence: 99%

Optimal Reservoir Operations using Long Short-Term Memory Network

Singh,

Singh

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Reliable forecast of inflows to the reservoir is a key factor in the optimal operation of reservoirs. Real-time operation of the reservoir based on forecasts of inflows can lead to substantial economic gains. However, the forecast of inflow is an intricate task as it has to incorporate impacts of climate and hydrological changes. Therefore, the major objective of the present work is to develop a novel approach based on long short-term memory (LSTM) for the forecast of inflows. Real-time inflow forecast, in other words, daily inflow at the reservoir helps in efficient operation of water resources. Also, daily variations in the release can be monitored efficiently and reliability of operation is improved. This work proposes a naive anomaly detection algorithm baseline based on LSTM. In other words, a strong baseline to forecast flood and drought for any deep learning-based prediction model. The practicality of the approach has been demonstrated using the observed daily data of the past 20 years from Bhakra Dam in India. The results of the simulations conducted herein clearly indicate the supremacy of the LSTM approach over the traditional methods of forecasting.Although, experiments are run on data from Bhakra Dam Reservoir in India, LSTM model and anomaly detection algorithm are general purpose and can be applied to any basin with minimal changes. A distinct practical advantage of the LSTM method presented herein is that it can adequately simulate non-stationarity and non-linearity in the historical data.

show abstract

Adaptive Margin Diversity Regularizer for Handling Data Imbalance in Zero-Shot SBIR

Cited by 21 publications

References 30 publications

Zero-Shot Sketch Based Image Retrieval using Graph Transformer

Zero-Shot Sketch Based Image Retrieval using Graph Transformer

Interband Retrieval and Classification Using the Multilabeled Sentinel-2 BigEarthNet Archive

Optimal Reservoir Operations using Long Short-Term Memory Network

Contact Info

Product

Resources

About