A Survey on Large-Scale Machine Learning

Wang, Meng; Fu, Weijie; He, Xiangnan; Hao, Shijie; Wu, Xindong

doi:10.1109/tkde.2020.3015777

Cited by 58 publications

(24 citation statements)

References 161 publications

(216 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Finally, according to the competent and comprehensive survey on large-scale machine learning in Ref. [40], [41], we may consider designing a simplified model with the characteristics of quantum computing under the premise of ensuring the performance of the models on NLP tasks and reducing the computational cost of the model when dealing with large-scale datasets. We can also pay more attention to the collision of quantum mechanics and deep learning in other NLP tasks such as question answer matching, machine translation and so on.…”

Section: Results and Comparisonsmentioning

confidence: 99%

Two End-to-End Quantum-Inspired Deep Neural Networks for Text Classification

Shi

Lai

et al. 2023

IEEE Trans. Knowl. Data Eng.

View full text Add to dashboard Cite

In linguistics, the uncertainty of context due to polysemy is widespread, which attracts much attention. Quantum-inspired complex word embedding based on Hilbert space plays an important role in natural language processing (NLP), which fully leverages the similarity between quantum states and word tokens. A word containing multiple meanings could correspond to a single quantum particle which may exist in several possible states, and a sentence could be analogous to the quantum system where particles interfere with each other. Motivated by quantum-inspired complex word embedding, interpretable complex-valued word embedding (ICWE) is proposed to design two end-to-end quantum-inspired deep neural networks (ICWE-QNN and CICWE-QNN representing convolutional complexvalued neural network based on ICWE) for binary text classification. They have the proven feasibility and effectiveness in the application of NLP and can solve the problem of text information loss in CE-Mix [1] model caused by neglecting the important linguistic features of text, since linguistic feature extraction is presented in our model with deep learning algorithms, in which gated recurrent unit (GRU) extracts the sequence information of sentences, attention mechanism makes the model focus on important words in sentences and convolutional layer captures the local features of projected matrix. The model ICWE-QNN can avoid random combination of word tokens and CICWE-QNN fully considers textual features of the projected matrix. Experiments conducted on five benchmarking classification datasets demonstrate our proposed models have higher accuracy than the compared traditional models including CaptionRep BOW, DictRep BOW and Paragram-Phrase, and they also have great performance on F1-score. Eespecially, CICWE-QNN model has higher accuracy than the quantum-inspired model CE-Mix as well for four datasets including SST, SUBJ, CR and MPQA. It is a meaningful and effictive exploration to design quantum-inspired deep neural networks to promote the performance of text classification.

show abstract

Section: Results and Comparisonsmentioning

confidence: 99%

Two End-to-End Quantum-Inspired Deep Neural Networks for Text Classification

Shi

Lai

et al. 2023

IEEE Trans. Knowl. Data Eng.

View full text Add to dashboard Cite

show abstract

“…In future work, we plan to develop a semi-supervised scheme that leverages the self-supervised learning signals and manual annotation together to improve the performance of initiative discrimination of KS. Second, compared to previous methods, the introduced initiative discriminator increases the computational burden of our model during training and inference [49]. We plan to design a simple but effictive initiative discriminator to improve the efficiency.…”

Section: Conclusion and Discussionmentioning

confidence: 99%

Initiative-Aware Self-Supervised Learning for Knowledge-Grounded Conversations

Chen

Ren

Chen

et al. 2021

Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

View full text Add to dashboard Cite

In the knowledge-grounded conversation (KGC) task systems aim to produce more informative responses by leveraging external knowledge. KGC includes a vital part, knowledge selection, where conversational agents select the appropriate knowledge to be incorporated in the next response. Mixed initiative is an intrinsic feature of conversations where the user and the system can both take the initiative in suggesting new conversational directions. Knowledge selection can be driven by the user's initiative or by the system's initiative. For the former, the system usually selects knowledge according to the current user utterance that contains new topics or questions posed by the user; for the latter, the system usually selects knowledge according to the previously selected knowledge. No previous study has considered the mixed-initiative characteristics of knowledge selection to improve its performance.In this paper, we propose a mixed-initiative knowledge selection method (MIKe) for KGC, which explicitly distinguishes between user-initiative and system-initiative knowledge selection. Specifically, we introduce two knowledge selectors to model both of them separately, and design a novel initiative discriminator to discriminate the initiative type of knowledge selection at each conversational turn. A challenge for training MIKe is that we usually have no labels for indicating initiative. To tackle this challenge, we devise an initiative-aware self-supervised learning scheme that helps MIKe to learn to discriminate the initiative type via a self-supervised task. Experimental results on two datasets show that MIKe significantly outperforms state-of-the-art methods in terms of both automatic and human evaluations, indicating that it can select more appropriate knowledge and generate more informative and engaging responses.

show abstract

“…In addition, in order to make better use of the discriminative information expressed by visual features, we also analyze the effects of classifiers with different structures. (4) We also conduct visual experiments on synthetic visual features from unseen classes by t-SNE [30], which intuitively proves the effective generation ability of our model.…”

Section: Introductionmentioning

confidence: 89%

“…In recent years, deep learning [1][2][3][4] has achieved great success in a wide range of computer vision and machine learning tasks [5], including face recognition, emotion classification, and visual question answering. In most cases, these deep learning models are more effective than human beings in many aspects, because they can observe potential information that may be ignored by human eyes in pictures.…”

Section: Introductionmentioning

confidence: 99%

Dual Generative Network with Discriminative Information for Generalized Zero-Shot Learning

Zhao

Liu

2021

Complexity

View full text Add to dashboard Cite

Zero-shot learning is dedicated to solving the classification problem of unseen categories, while generalized zero-shot learning aims to classify the samples selected from both seen classes and unseen classes, in which “seen” and “unseen” classes indicate whether they can be used in the training process, and if so, they indicate seen classes, and vice versa. Nowadays, with the promotion of deep learning technology, the performance of zero-shot learning has been greatly improved. Generalized zero-shot learning is a challenging topic that has promising prospects in many realistic scenarios. Although the zero-shot learning task has made gratifying progress, there is still a strong deviation between seen classes and unseen classes in the existing methods. Recent methods focus on learning a unified semantic-aligned visual representation to transfer knowledge between two domains, while ignoring the intrinsic characteristics of visual features which are discriminative enough to be classified by itself. To solve the above problems, we propose a novel model that uses the discriminative information of visual features to optimize the generative module, in which the generative module is a dual generation network framework composed of conditional VAE and improved WGAN. Specifically, the model uses the discrimination information of visual features, according to the relevant semantic embedding, synthesizes the visual features of unseen categories by using the learned generator, and then trains the final softmax classifier by using the generated visual features, thus realizing the recognition of unseen categories. In addition, this paper also analyzes the effect of the additional classifiers with different structures on the transmission of discriminative information. We have conducted a lot of experiments on six commonly used benchmark datasets (AWA1, AWA2, APY, FLO, SUN, and CUB). The experimental results show that our model outperforms several state-of-the-art methods for both traditional as well as generalized zero-shot learning.

show abstract

A Survey on Large-Scale Machine Learning

Cited by 58 publications

References 161 publications

Two End-to-End Quantum-Inspired Deep Neural Networks for Text Classification

Two End-to-End Quantum-Inspired Deep Neural Networks for Text Classification

Initiative-Aware Self-Supervised Learning for Knowledge-Grounded Conversations

Dual Generative Network with Discriminative Information for Generalized Zero-Shot Learning

Contact Info

Product

Resources

About