Online multimedia retrieval on CPU–GPU platforms with adaptive work partition

Souza, Rafael Martins de; Fernandes, André Gustavo de Araújo; Teixeira, Thiago; Teodoro, George; Ferreira, Renato

doi:10.1016/j.jpdc.2020.10.001

Cited by 10 publications

(3 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In other recent works, we have developed strategies to deal with varying query loads [33,36]. First, we parallelized Hypercurves [36] to allocate the processing in CPU and GPU under varying loads to minimize response times.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Large-Scale Response-Aware Online ANN Search in Dynamic Datasets

Andrade

Barreiros

Rocha

et al. 2023

Preprint

View full text Add to dashboard Cite

Similarity search is an key operation in Content-based multimedia retrieval(CBMR) applications. Online CBMR applications, which is the focus of thiswork, have to search in large and dynamic datasets that are updated during theexecution while offering low response times. Additionally, these applications aresubmitted to workloads that vary at runtime. The computing demands in thisscenario exceeds the processing power of a single computer, motivating the large-scale machines in the domain. Thus, in this work, we proposed a distributedmemory parallelization of similarity search that addresses the mentioned chal-lenges. Our solution employs the efficient Inverted File System with AsymmetricDistance Computation algorithm (IVFADC) algorithm as the baseline, which isextended here to support dynamic datasets. Further, we developed a dynamicresource management algorithm, called Multi-Stream Adaptation (MS-ADAPT),that is executed at run-time to change the computing resource assignment withthe goal of minimizing response times. We have evaluated our system solutionwith multiple data partitioning strategies using up to 160 compute nodes anda dataset with 344 billion multimedia descriptors. It demonstrated superlinearscalability for our Spatial-Aware data partition algorithms and MS-ADAPT out-performed the best static approach (oracle) by reducing the response times upto 32× on high-load cases.

show abstract

Section: Related Workmentioning

confidence: 99%

“…First, we parallelized Hypercurves [36] to allocate the processing in CPU and GPU under varying loads to minimize response times. We have further developed a distributed memory version of IVFADC [33] targeting GPU. The latter work tunes at run-time the number of queries bundled for execution with the GPU targeting response time reduction.…”

Section: Related Workmentioning

confidence: 99%

Large-Scale Response-Aware Online ANN Search in Dynamic Datasets

Andrade

Barreiros

Rocha

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…The recent advances and trends of MMDL are from Audio-visual speech recognition (AVSR) [114], multimedia content indexing and retrieval [6,89], understanding human multimodal behaviors during social interaction, multimodal emotion recognition [14,19], image and video captioning [9,55], Visual Question-Answering (VQA) [66], multimedia retrieval [90] to health analysis [110] and so on. In this article, we analyzed the latest MMDL models to propose typical models and techniques for advancing the field forward.…”

Section: Introductionmentioning

confidence: 99%

A Review on Methods and Applications in Multimodal Deep Learning

Summaira¹,

Li²,

Shoib³

et al. 2022

Preprint

View full text Add to dashboard Cite

Deep Learning has implemented a wide range of applications and has become increasingly popular in recent years. The goal of multimodal deep learning (MMDL) is to create models that can process and link information using various modalities. Despite the extensive development made for unimodal learning, it still cannot cover all the aspects of human learning. Multimodal learning helps to understand and analyze better when various senses are engaged in the processing of information. This paper focuses on multiple types of modalities, i.e., image, video, text, audio, body gestures, facial expressions, and physiological signals. Detailed analysis of the baseline approaches and an in-depth study of recent advancements during the last five years (2017 to 2021) in multimodal deep learning applications has been provided. A fine-grained taxonomy of various multimodal deep learning methods is proposed, elaborating on different applications in more depth. Lastly, main issues are highlighted separately for each domain, along with their possible future research directions.CCS Concepts: • Computing methodologies → Machine learning; • Information systems → Multimedia and multimodal retrieval.

show abstract