Significance of Softmax-based Features in Comparison to Distance Metric Learning-based Features

Horiguchi, Shota; Ikami, Daiki; Aizawa, Kiyoharu

doi:10.1109/tpami.2019.2911075

Cited by 53 publications

(32 citation statements)

References 30 publications

(77 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition to OSV, these two categories of loss functions were also studied for different classification tasks, and the similar comparative results were reported (Horiguchi, Ikami, & Aizawa, 2016;Janocha & Czarnecki, 2017). The work (Horiguchi et al, 2016) confirmed this result by performing a fair comparison between Cross-Entropy (CE) loss as a classification loss and several state-of-the-art metric learning losses.…”

Section: Introductionsupporting

confidence: 57%

Multi-representational learning for Offline Signature Verification using Multi-Loss Snapshot Ensemble of CNNs

Masoudnia

Mersa

Araabi

et al. 2019

Expert Systems with Applications

View full text Add to dashboard Cite

Offline Signature Verification (OSV) is a challenging pattern recognition task, especially in the presence of skilled forgeries that are not available during training. This study aims to tackle its challenges and meet the substantial need for generalization for OSV by examining different loss functions for Convolutional Neural Network (CNN). We adopt our new approach to OSV by asking two questions: 1. which classification loss provides more generalization for feature learning in OSV? , and 2. How integration of different losses into a unified multi-loss function lead to an improved learning framework?These questions are studied based on analysis of three loss functions, including cross entropy, Cauchy-Schwarz divergence, and hinge loss. According to complementary features of these losses, we combine them into a dynamic multi-loss function and propose a novel ensemble framework for simultaneous use of them in CNN. Our proposed Multi-Loss Snapshot Ensemble (MLSE) consists of several sequential trials. In each trial, a dominant loss function is selected from the multi-loss set, and the remaining losses act as a regularizer. Different trials learn diverse representations for each input based on signature identification task. This multi-representation set is then employed for the verification task. An ensemble of SVMs is trained on these representations, and their decisions are finally combined according to the selection of most generalizable SVM for each user.We conducted two sets of experiments based on two different protocols of OSV, i.e., writer-dependent and writer-independent on three signature datasets: GPDS-Synthetic, MCYT, and UT-SIG. Based on the writerdependent OSV protocol, On UT-SIG, we achieved 6.17% Equal Error Rate (EER) which showed substantial improvement over the best EER in the literature, 9.61%. Our method surpassed state-of-the-arts by 2.5% on GPDS-Synthetic, achieving 6.13%. Our result on MCYT was also comparable to the best previous results.The second set of experiments examined the robustness of our proposed method to the arrival of new users enrolled in the OSV system based on the writer-independent protocol. The results also confirmed that our proposed system efficiently performed the verification of new users enrolled in the OSV system.

show abstract

Section: Introductionsupporting

confidence: 57%

Multi-representational learning for Offline Signature Verification using Multi-Loss Snapshot Ensemble of CNNs

Masoudnia

Mersa

Araabi

et al. 2019

Expert Systems with Applications

View full text Add to dashboard Cite

show abstract

“…In particular, several sampling strategies are widely investigated to improve the performance, such as hard mining [16], semihard mining [35], smart mining [13] and so on. In comparison, softmax embedding achieves competitive performance without sampling requirement [18]. Supervised learning has achieved superior performance on various tasks, but they still rely on enough annotated data.…”

Section: Related Workmentioning

confidence: 99%

Unsupervised Embedding Learning via Invariant and Spreading Instance Feature

Zhang

Yuen³

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

523

440

View full text Add to dashboard Cite

This paper studies the unsupervised embedding learning problem, which requires an effective similarity measurement between samples in low-dimensional embedding space. Motivated by the positive concentrated and negative separated properties observed from category-wise supervised learning, we propose to utilize the instance-wise supervision to approximate these properties, which aims at learning data augmentation invariant and instance spreadout features. To achieve this goal, we propose a novel instance based softmax embedding method, which directly optimizes the 'real' instance features on top of the softmax function. It achieves significantly faster learning speed and higher accuracy than all existing methods. The proposed method performs well for both seen and unseen testing categories with cosine similarity. It also achieves competitive performance even without pre-trained network over samples from fine-grained categories.

show abstract

“…Finally, a Fully Connected layer flattens the previous layer volume into a feature vector and then an output layer which computes the scores (confidence or probabilities) for the output classes/features through a dense network. This output is then passed to a regression function such as Softmax [12], for example, which maps everything to a vector whose elements sum up to one [7].…”

Section: Advantages Of Deep Learningmentioning

confidence: 99%

Deep Learning vs. Traditional Computer Vision

Mahony

Campbell

Carvalho

et al. 2019

Advances in Intelligent Systems and Computing

677

383

View full text Add to dashboard Cite

Deep Learning has pushed the limits of what was possible in the domain of Digital Image Processing. However, that is not to say that the traditional computer vision techniques which had been undergoing progressive development in years prior to the rise of DL have become obsolete. This paper will analyse the benefits and drawbacks of each approach. The aim of this paper is to promote a discussion on whether knowledge of classical computer vision techniques should be maintained. The paper will also explore how the two sides of computer vision can be combined. Several recent hybrid methodologies are reviewed which have demonstrated the ability to improve computer vision performance and to tackle problems not suited to Deep Learning. For example, combining traditional computer vision techniques with Deep Learning has been popular in emerging domains such as Panoramic Vision and 3D vision for which Deep Learning models have not yet been fully optimised.

show abstract

Significance of Softmax-based Features in Comparison to Distance Metric Learning-based Features

Cited by 53 publications

References 30 publications

Multi-representational learning for Offline Signature Verification using Multi-Loss Snapshot Ensemble of CNNs

Multi-representational learning for Offline Signature Verification using Multi-Loss Snapshot Ensemble of CNNs

Unsupervised Embedding Learning via Invariant and Spreading Instance Feature

Deep Learning vs. Traditional Computer Vision

Contact Info

Product

Resources

About