Hyper-class augmented and regularized deep learning for fine-grained image classification

Sun

Computational Intelligence and Neuroscience

2017

508

212

Automatic and accurate estimation of disease severity is essential for food security, disease management, and yield loss prediction. Deep learning, the latest breakthrough in computer vision, is promising for fine-grained disease severity classification, as the method avoids the labor-intensive feature engineering and threshold-based segmentation. Using the apple black rot images in the PlantVillage dataset, which are further annotated by botanists with four severity stages as ground truth, a series of deep convolutional neural networks are trained to diagnose the severity of the disease. The performances of shallow networks trained from scratch and deep models fine-tuned by transfer learning are evaluated systemically in this paper. The best model is the deep VGG16 model trained with transfer learning, which yields an overall accuracy of 90.4% on the hold-out test set. The proposed deep learning model may have great potential in disease control for modern agriculture.

Section: Related Workmentioning

confidence: 99%

Automatic Image-Based Plant Disease Severity Estimation Using Deep Learning

Sun

Computational Intelligence and Neuroscience

2017

508

212

2020 IEEE 14th International Conference on Semantic Computing (ICSC)

“…Hyper-class augmented deep learning model has been shown to work well for fine-grained image classification. Instead of fine tuning a convolutional neural network (CNN) , [17] suggests a hyper-class augmentation formulated as multitask learning in order to boost the recognition task. Similarly, we include frame element classification in parallel with answer classification.…”

Section: Our Vqa Modelmentioning

confidence: 99%

Augmenting Visual Question Answering with Semantic Frame Information in a Multitask Learning Approach

Alizadeh

Eugenio

2020

Visual Question Answering (VQA) concerns providing answers to Natural Language questions about images. Several deep neural network approaches have been proposed to model the task in an end-to-end fashion. Whereas the task is grounded in visual processing, if the question focuses on events described by verbs, the language understanding component becomes crucial. Our hypothesis is that models should be aware of verb semantics, as expressed via semantic role labels, argument types, and/or frame elements. Unfortunately, no VQA dataset exists that includes verb semantic information. Our first contribution is a new VQA dataset (imSituVQA) that we built by taking advantage of the imSitu annotations. The imSitu dataset consists of images manually labeled with semantic frame elements, mostly taken from FrameNet. Second, we propose a multitask CNN-LSTM VQA model that learns to classify the answers as well as the semantic frame elements. Our experiments show that semantic frame element classification helps the VQA system avoid inconsistent responses and improves performance.

“…Fine-grained categorization and retrieval are two related but different tasks. Fine-grained categorization has been extensively investigated and various algorithms have achieved outstanding performance [5,6,7,8,9,10,11,12,13]. In contrast, fine-grained image search is under-studied and requires distinguishing subtle differences given a potentially growing database.…”

Section: Related Workmentioning

confidence: 99%

Adversarial Learning for Fine-Grained Image Search

Lin

Yang

2019 IEEE International Conference on Multimedia and Expo (ICME)

et al. 2019

Fine-grained image search is still a challenging problem due to the difficulty in capturing subtle differences regardless of pose variations of objects from fine-grained categories. In practice, a dynamic inventory with new fine-grained categories adds another dimension to this challenge. In this work, we propose an end-to-end network, called FGGAN, that learns discriminative representations by implicitly learning a geometric transformation from multi-view images for fine-grained image search. We integrate a generative adversarial network (GAN) that can automatically handle complex view and pose variations by converting them to a canonical view without any predefined transformations. Moreover, in an open-set scenario, our network is able to better match images from unseen and unknown fine-grained categories. Extensive experiments on two public datasets and a newly collected dataset have demonstrated the outstanding robust performance of the proposed FG-GAN in both closed-set and open-set scenarios, providing as much as 10% relative improvement compared to baselines.