Generalized Bilinear Deep Convolutional Neural Networks for Multimodal Biometric Identification

Abstract

In this paper, we propose to employ a bank of modalitydedicated Convolutional Neural Networks (CNNs), fuse, train, and optimize them together for person classification tasks. A modality-dedicated CNN is used for each modality to extract modality-specific features. We demonstrate that, rather than spatial fusion at the convolutional layers, the fusion can be performed on the outputs of the fully-connected layers of the modality-specific CNNs without any loss of performance and with significant reduction in the number of parameters. We show that, using multiple CNNs with multimodal fusion at the feature-level, we significantly outperform systems that use unimodal representation. We study weighted feature, bilinear, and compact bilinear feature-level fusion algorithms for multimodal biometric person identification. Finally, We propose generalized compact bilinear fusion algorithm to deploy both the weighted feature fusion and compact bilinear schemes. We provide the results for the proposed algorithms on three challenging databases: CMU Multi-PIE, BioCop, and BIOMDATA.

Sequential Sampling for Optimal Bayesian Classification of Sequencing Count Data

“…Future works can include expansion of this sampling method to the hierarchical Gaussian-Poisson framework introduced in [3] for modeling read count data, and also implementing the proposed sampling method on negative binomial models fit on real RNA-Seq datasets. Also employing convolutional neural networks as fully data-driven architectures to merge feature extraction and classification procedures like in [36], [37] can be another area for potential research.…”

Adversarial Examples to Fool Iris Recognition Systems

“…Recently, deep learning models have outperformed the classical machine learning models in a variety of application areas, including biometrics [26,27,25]. However, these models are vulnerable to a small perturbation in the input image.…”

ID Preserving Generative Adversarial Network for Partial Latent Fingerprint Reconstruction

“…Second, the generator should preserve the identification information embedded in minutiae and ridge patterns. To extract and preserve the identification information we developed a method that is inspired by perceptual loss [4,7,12,17] and multi-level feature abstraction [27,28,13,10]. For this purpose, we separately trained a deep Siamese CNN as a fingerprint verifier.…”