Application of Multimodal NLP Instruction Combined with Speech Recognition in Oral English Practice

Abstract

In order to study the application of multimodal NLP instruction combined with speech recognition based on hybrid deep learning in oral English practice, firstly, the basic principle of speech recognition technology is introduced. The concept of hidden Markov model and three key algorithms are explained, and its simulation and implementation in speech recognition application are realized. The architecture and key technologies of the system are introduced. Then, it introduces the specific application of deep learning in NLP. Finally, Chinese teachers with oral English teaching experience participate in the recording. The effective reading time of each person is 65 minutes, and the reading sentences are 3100 sentences. The total number of people is 80 (40 men and 40 women). The sentences cover 1595 spoken English words. Conduct oral English training. The experimental results show that the recognition accuracy decreases by about 2%, but the recognition speed increases by 10 times. In addition, the scoring accuracy is equivalent to that of the platform system. The accuracy of this method in instruction classification is increased, which verifies the feasibility and effectiveness of this method. In the future, attention mechanism will be used to expand this method.

Natural Language Processing in Teacher Training : a systematic review

“…For instance, JILL BURSTEIN et al used Language MuseSM (LM), a web-based application based on NLP, to raise language awareness among teachers and support them in designing language-based instructional support for English language learners (ELLs) [32]. Meanwhile, Jin Xu and Tong Li researched the application of multimodal NLP guidance and speech recognition technology in English oral practice [33]. Pascual Pérez-Paredes et al studied the role of mobile-assisted language learning (MALL), data-driven learning (DDL), and natural language processing technology (NLPT) as open educational resources (OERs) in second language education [34].…”

Research on the Application of ERNIE Bot in College English Teaching

“…ERNIE Bot, a cutting-edge technology in the field of artificial intelligence, has recently emerged as a promising tool for enhancing language learning experiences [1][2][3]. Like ChatGPT, ERNIE Bot uses the power of natural language processing and deep learning to attract learners in a more interactive and personalized way [4][5][6][7]. This technology is designed to simulate human-like conversations, enabling learners to practice their language skills in a realistic and immersive environment.…”