Deep Dynamic Neural Networks for Multimodal Gesture Segmentation and Recognition

Wu, Di; Pigou, Lionel; Kindermans, Pieter-Jan; Le, Nam; Shao, Ling; Dambre, Joni; Odobez, Jean-Marc

doi:10.1109/tpami.2016.2537340

Cited by 401 publications

(209 citation statements)

References 52 publications

(82 reference statements)

Supporting

Mentioning

198

Contrasting

Unclassified

Order By: Relevance

“…Others, such as [10], [11], [25], [26], [27], accept a more relaxed hash coding restriction that heterogeneous data representing common objects share similar binary codes which means the Hamming distance of their binary codes, should be small enough. Some other interesting methods could be found [23], [28], [29], [30], [31], [32].…”

Section: Related Workmentioning

confidence: 99%

Hetero-Manifold Regularisation for Cross-Modal Hashing

Zheng

Tang²,

Shao

2018

IEEE Trans. Pattern Anal. Mach. Intell.

Self Cite

View full text Add to dashboard Cite

Abstract-Recently, cross-modal search has attracted considerable attention but remains a very challenging task because of the integration complexity and heterogeneity of the multi-modal data. To address both challenges, in this paper, we propose a novel method termed hetero-manifold regularisation (HMR) to supervise the learning of hash functions for efficient cross-modal search. A hetero-manifold integrates multiple sub-manifolds defined by homogeneous data with the help of cross-modal supervision information. Taking advantages of the hetero-manifold, the similarity between each pair of heterogeneous data could be naturally measured by three order random walks on this hetero-manifold. Furthermore, a novel cumulative distance inequality defined on the hetero-manifold is introduced to avoid the computational difficulty induced by the discreteness of hash codes. By using the inequality, cross-modal hashing is transformed into a problem of hetero-manifold regularised support vector learning. Therefore, the performance of cross-modal search can be significantly improved by seamlessly combining the integrated information of the hetero-manifold and the strong generalisation of the support vector machine. Comprehensive experiments show that the proposed HMR achieve advantageous results over the state-of-the-art methods in several challenging cross-modal tasks.

show abstract

Section: Related Workmentioning

confidence: 99%

Hetero-Manifold Regularisation for Cross-Modal Hashing

Zheng

Tang²,

Shao

2018

IEEE Trans. Pattern Anal. Mach. Intell.

Self Cite

View full text Add to dashboard Cite

show abstract

“…The authors found this feature learning algorithm is surprisingly successful an applied to detect image objects. The authors in the paper [2] describe the hundreds of thousands of unlabelled videos from the web to learn visual representation of those videos it helps tracking visually provides the super vision that means two patches connected by a track should have similar visual representation in deep feature space since they probably using deep dynamic neural networks for multimodal gesture segmentation and recognition [3]. The author proposed semi supervised hierarchical dynamic framework based on Hidden Markov model (HMM) for simultaneous gesture segmentation and recognition where skeleton joint information, depth and RGB images, are the multimodal input observation.…”

Section: Literature Surveymentioning

confidence: 99%

“…This model will be used in the later stage for detection of objects and arriving features inside images uploaded by user.This paper utilizes python implementation [3] for CNN.…”

Section: B Creating Training Datamentioning

confidence: 99%

An Efficient CNN a deep learning approach applied on the image matching context

Bushanam¹,

Reddy²

2018

IJET

View full text Add to dashboard Cite

Image matching is a quite challenging task to identify matching images in the data. There are multiple methods in computer vision techniques such as histogram-based algorithms, colour or edge based algorithms, textual based features, SIFT and Surf algorithms which will help to identify similar images. Here in our paper we are addressing an industrial problem to provide the better solution where US multinational courier delivery service facing challenges in delivering the products where labels/tags and bar codes of the products are missed while delivering to the customers and customers comes with the product image and with some information about the product. The job is to map the user or customer product information with the existing missed products. The advances in computer science and availability of GPU Machines, the problem will be addressed, and solutions can be automated using deep learning approaches. The paper describes the solution of matching the solution accurately and comparing the solution with the existing classical computer vision algorithms.

show abstract

“…Yan and Shao [38] also used deep learning technique to estimate image blur blindly. For more reference about the application of deep learning, see [39], [40] Taking advantages of deep learning, these image based algorithms offer more promising superresolution estimations than most of patch based algorithms. However, the huge burden of training a convolutional neural network makes these image based algorithms time-consuming during the training process.…”

Section: A Brief Review On Single-image Super-resolutionmentioning

confidence: 99%

Pairwise Operator Learning for Patch-Based Single-Image Super-Resolution

Tang

Shao

2017

IEEE Trans. on Image Process.

Self Cite

View full text Add to dashboard Cite

Abstract-Motivated by the fact that image patches could be inherently represented by matrices, single-image super-resolution is treated as a problem of learning regression operators in a matrix space in this paper. The regression operators that map low-resolution image patches to high-resolution image patches are generally defined by left and right multiplication operators. The pairwise operators are respectively used to extract the raw and column information of low-resolution image patches for recovering high-resolution estimations. The patch based regression algorithm possesses three favorable properties. Firstly, the proposed super-resolution algorithm is efficient during both training and testing, because image patches are treated as matrices. Secondly, the data storage requirement of the optimal pairwise operator is far less than most popular single-image super-resolution algorithms because only two small sized matrices need to be stored. Lastly, the super-resolution performance is competitive with most popular single-image super-resolution algorithms because both raw and column information of image patches is considered. Experimental results show the efficiency and effectiveness of the proposed patch-based single-image superresolution algorithm. IndexTerms-Single-image super-resolution, matrix space, matrix-value operator regression, left and right multiplication operators.

show abstract

Deep Dynamic Neural Networks for Multimodal Gesture Segmentation and Recognition

Cited by 401 publications

References 52 publications

Hetero-Manifold Regularisation for Cross-Modal Hashing

Hetero-Manifold Regularisation for Cross-Modal Hashing

An Efficient CNN a deep learning approach applied on the image matching context

Pairwise Operator Learning for Patch-Based Single-Image Super-Resolution

Contact Info

Product

Resources

About