Deep Virtual Networks for Memory Efficient Inference of Multiple Tasks

Kim, Eunwoo; Ahn, Chanho; Torr, Philip H. S.; Oh, Songhwai

doi:10.1109/cvpr.2019.00282

Cited by 10 publications

(5 citation statements)

References 29 publications

(87 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Likewise, [22] fuses the features from the different tasks to train them jointly. The other one learns multiple tasks using a single shared architecture [11], [17], [23]- [26]. [11], [23] utilize the auxiliary tasks to rich the feature representation.…”

Section: Related Work a Multi-task Learningmentioning

confidence: 99%

Resource-Efficient Multi-Task Deep Learning Using a Multi-Path Network

2022

Self Cite

View full text Add to dashboard Cite

Multi-task learning (MTL) improves learning efficiency compared to the single-task counterpart in that it performs multiple tasks at the same time. Due to the nature, it can achieve generalized performance as well as alleviate overfitting. However, it does not efficiently perform resource-aware inference from a single trained architecture. To address the issue, we aim to build a learning framework that minimizes the cost to infer tasks under different memory budgets. To this end, we propose a multi-path network with a self-auxiliary learning strategy. The multi-path structure contains task-specific paths in a backbone network, where a lower-level path predicts earlier with a smaller number of parameters. To alleviate the performance degradation from earlier predictions, a self-auxiliary learning strategy is presented. The selfauxiliary tasks convey task-specific knowledge to the main tasks to compensate for the performance leak. We evaluate the proposed method on an extensive set of multi-task learning scenarios, including multiple tasks learning, hierarchical learning, and curriculum learning. The proposed method outperforms existing multi-task learning competitors for most scenarios about by a margin of 1% ∼ 2% accuracy on average while consuming 30% ∼ 60% smaller computational cost.INDEX TERMS Multi-task learning, resource-efficient learning, multi-path network.

show abstract

Section: Related Work a Multi-task Learningmentioning

confidence: 99%

Resource-Efficient Multi-Task Deep Learning Using a Multi-Path Network

2022

Self Cite

View full text Add to dashboard Cite

show abstract

“…There are several recent studies [11,23,24] that proposed a network structure in which parameters can be efficiently shared across tasks. Other approaches [15,16,22] suggest a single architecture which includes multiple internal networks (or models) so that they can assign different models to multiple tasks without increasing the parameters. However, they use a fixed model structure for each task and it requires expert efforts to assign the model to each task.…”

Section: Related Workmentioning

confidence: 99%

“…However, a single shared model for multiple tasks may cause performance degradation when associ-ated tasks are less relevant [29]. To avoid this issue, recent approaches [15,16] proposed a network architecture which can contain several sub-models to assign the them to multiple tasks. Despite their attempts for MTL, they require human efforts to construct sub-models from the network architecture and assign the model to each task.…”

Section: Introductionmentioning

confidence: 99%

Deep Elastic Networks With Model Selection for Multi-Task Learning

Ahn

Kim

2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

Self Cite

View full text Add to dashboard Cite

In this work, we consider the problem of instance-wise dynamic network model selection for multi-task learning. To this end, we propose an efficient approach to exploit a compact but accurate model in a backbone architecture for each instance of all tasks. The proposed method consists of an estimator and a selector. The estimator is based on a backbone architecture and structured hierarchically. It can produce multiple different network models of different configurations in a hierarchical structure. The selector chooses a model dynamically from a pool of candidate models given an input instance. The selector is a relatively small-size network consisting of a few layers, which estimates a probability distribution over the candidate models when an input instance of a task is given. Both estimator and selector are jointly trained in a unified learning framework in conjunction with a sampling-based learning strategy, without additional computation steps. We demonstrate the proposed approach for several image classification tasks compared to existing approaches performing model selection or learning multiple tasks. Experimental results show that our approach gives not only outstanding performance compared to other competitors but also the versatility to perform instance-wise model selection for multiple tasks.

show abstract

“…Mallya et al studied a method for performing multiple tasks in a single deep neural network by iteratively pruning and packing the network parameters [24]. Kim et al proposed a novel architecture containing multiple networks of different configurations termed deep virtual networks with respect to different tasks and memory budgets [25]. Recently, multi-task learning with DCNNs have also been studied and applied to face attributes prediction.…”

Section: Related Workmentioning

confidence: 99%

Multi-Task Learning Using Task Dependencies for Face Attributes Prediction

Fan

Kim

et al. 2019

Applied Sciences

View full text Add to dashboard Cite

Face attributes prediction has an increasing amount of applications in human–computer interaction, face verification and video surveillance. Various studies show that dependencies exist in face attributes. Multi-task learning architecture can build a synergy among the correlated tasks by parameter sharing in the shared layers. However, the dependencies between the tasks have been ignored in the task-specific layers of most multi-task learning architectures. Thus, how to further boost the performance of individual tasks by using task dependencies among face attributes is quite challenging. In this paper, we propose a multi-task learning using task dependencies architecture for face attributes prediction and evaluate the performance with the tasks of smile and gender prediction. The designed attention modules in task-specific layers of our proposed architecture are used for learning task-dependent disentangled representations. The experimental results demonstrate the effectiveness of our proposed network by comparing with the traditional multi-task learning architecture and the state-of-the-art methods on Faces of the world (FotW) and Labeled faces in the wild-a (LFWA) datasets.

show abstract

Deep Virtual Networks for Memory Efficient Inference of Multiple Tasks

Cited by 10 publications

References 29 publications

Resource-Efficient Multi-Task Deep Learning Using a Multi-Path Network

Resource-Efficient Multi-Task Deep Learning Using a Multi-Path Network

Deep Elastic Networks With Model Selection for Multi-Task Learning

Multi-Task Learning Using Task Dependencies for Face Attributes Prediction

Contact Info

Product

Resources

About