MS-ASL: A Large-Scale Data Set and Benchmark for Understanding American Sign Language

Joze, Hamid Reza Vaezi; Koller, Oscar

doi:10.48550/arxiv.1812.01053

Cited by 17 publications

(26 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We notice that an unpublished paper[32] aims at providing an ASL dataset containing 1,000 glosses. Since the dataset is not released, we cannot evaluate the quality and the usefulness of the dataset.…”

mentioning

confidence: 99%

Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparison

Opazo

et al. 2020

2020 IEEE Winter Conference on Applications of Computer Vision (WACV)

321

239

View full text Add to dashboard Cite

Vision-based sign language recognition aims at helping the hearing-impaired people to communicate with others. However, most existing sign language datasets are limited to a small number of words. Due to the limited vocabulary size, models learned from those datasets cannot be applied in practice. In this paper, we introduce a new largescale Word-Level American Sign Language (WLASL) video dataset, containing more than 2000 words performed by over 100 signers. This dataset will be made publicly available to the research community. To our knowledge,it is by far the largest public ASL dataset to facilitate word-level sign recognition research.Based on this new large-scale dataset, we are able to experiment several deep learning methods for word-level sign recognition and evaluate their performances in large scale scenarios. Specifically we implement and compare two different models,i.e., (i) holistic visual appearance based approach, and (ii) 2D human pose based approach. Both models are valuable baselines that will benefit the community for method benchmarking. Moreover, we also propose a novel pose-based temporal graph convolution networks (Pose-TGCN) that models spatial and temporal dependencies in human pose trajectories simultaneously, which has further boosted the performance of the pose-based method. Our results show that pose-based and appearance-based models achieve comparable performances up to 62.63% at top-10 accuracy on 2,000 words/glosses, demonstrating the validity and challenges of our dataset. We will make the large-scale dataset, as well as our baseline deep models, freely available on github.

show abstract

mentioning

confidence: 99%

Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparison

Opazo

et al. 2020

2020 IEEE Winter Conference on Applications of Computer Vision (WACV)

321

239

View full text Add to dashboard Cite

show abstract

“…Recently, 3D-CNNs show remarkable performances on modeling the spatio-temporal patterns simultaneously in video frames. Joze et al [2] released a large-scale American Sign Language (ASL) dataset, namely MS-ASL, and provided baseline methods including 2D-CNN-LSTM and 3D-CNN based models. A 3D-CNN based I3D model [19] achieved the best result with a large improvement to the 2D-CNN based model.…”

Section: Related Workmentioning

confidence: 99%

“…Therefore, we made some modifications in the architecture; we replaced the 2 FC layers with a global average pooling, and also we inserted a batch normalization (BN) layer after all the convolutional layers. Compared to the C3D model, the I3D model has fewer parameters and achieved better results; recently, it has been used on new sign language datasets, e.g., WLASL [4], MS-ASL [2], and BSL-1K [40]. In [40], it was observed that I3D achieved higher accuracy on two ASL datasets when pretrained on a new largescale British Sign Language dataset, BSL-1K, compared to Kinetics-400.…”

Section: D-cnn Modelsmentioning

confidence: 99%

“…I3D that is pretrained on BSL-1K, R(2+1)D-18, and S3D models all achieve above 89% recognition accuracy. We choose the I3D model as the baseline to use in our follow-up experiments, since many recent state-of-the-art works in SLR also chose I3D as their baseline, e.g., WLASL [4] and MS-ASL [2]. In the follow up works, we include the self-attention mechanisms to the selected 3D baseline model, i.e.…”

Section: Experiments On Autslmentioning

confidence: 99%

“…Although continuous SLR is more challenging, isolated SLR is still an active research area. Particularly, the recently released new large-scale datasets provide unique challenges to the community [2][3][4][5][6][7]. According to the survey in [8], there is an exponential growth in isolated SLR studies between 1983-2020.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Using Motion History Images with 3D Convolutional Networks in Isolated Sign Language Recognition

Sincan,

Keles

2021

Preprint

View full text Add to dashboard Cite

Sign language recognition using computational models is a challenging problem that requires simultaneous spatio-temporal modeling of the multiple sources, i.e. faces, hands, body etc. In this paper, we propose an isolated sign language recognition model based on a model trained using Motion History Images (MHI) that are generated from RGB video frames. RGB-MHI images represent spatio-temporal summary of each sign video effectively in a single RGB image. We propose two different approaches using this model. In the first approach, we use RGB-MHI model as a motion-based spatial attention module integrated in a 3D-CNN architecture. In the second approach, we use RGB-MHI model features directly with a late fusion technique with the features of a 3D-CNN model. We perform extensive experiments on two recently released large-scale isolated sign language datasets, namely AUTSL and BosphorusSign22k datasets. Our experiments show that our models, which use only RGB data, can compete with the state-of-the-art models in the literature that use multi-modal data.

show abstract

From rule-based models to deep learning transformers architectures for natural language processing and sign language translation systems: survey, taxonomy and performance evaluation

Shahin,

Ismail

2024

Artif Intell Rev

View full text Add to dashboard Cite

MS-ASL: A Large-Scale Data Set and Benchmark for Understanding American Sign Language

Cited by 17 publications

References 0 publications

Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparison

Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparison

Using Motion History Images with 3D Convolutional Networks in Isolated Sign Language Recognition

From rule-based models to deep learning transformers architectures for natural language processing and sign language translation systems: survey, taxonomy and performance evaluation

Contact Info

Product

Resources

About