Weighted Automata Extraction from Recurrent Neural Networks via Regression on State Spaces

Okudono, Takamasa; Waga, Masaki; Sekiyama, Taro; Hasuo, Ichiro

doi:10.1609/aaai.v34i04.5977

Cited by 33 publications

(25 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recurrent neural network (RNN) [29] is an extension of traditional feedforward neural network. It has the memory function of inputting information.…”

Section: Deep Learningmentioning

confidence: 99%

Research on Online Learner Modeling and Course Recommendation Based on Emotional Factors

Wang

2022

Scientific Programming

View full text Add to dashboard Cite

With the popularization of online education and the idea of learning anytime and anywhere, more and more learners search and learn courses of various disciplines through online learning platforms to meet their personal knowledge needs. With the increase of the number of courses, it is difficult for learners to find the courses they want quickly and accurately; that is, they encounter the problems of information overload and cognitive maze. Therefore, how to recommend personalized courses for learners according to their preferences has become one of the important problems that need to be solved urgently to improve the service quality of courses in online learning platforms. Therefore, in order to improve the accuracy of course recommendation, it is necessary to build an accurate and complete learner model. In order to improve the application effect of recommendation, this paper focuses on the recommendation method of emotional factors to improve the recommendation efficiency of learning resources. The traditional recommendation model is a method based on the user’s purchase behavior and historical information. However, in the emotional factors, the effect of traditional recommendation is limited. This paper proposes a recommendation method based on emotional factors, which may consider the emotional and psychological factors of scholars according to the learning content of learners. The experimental results show that the learner model incorporating learners’ affection can reflect learners’ preferences more accurately, and the use of deep neural factorization machine for curriculum recommendation can effectively improve the accuracy of curriculum recommendation.

show abstract

“…Recurrent neural network (RNN) [29] is an extension of traditional feedforward neural network. It has the memory function of inputting information.…”

Section: Deep Learningmentioning

confidence: 99%

Research on Online Learner Modeling and Course Recommendation Based on Emotional Factors

Wang

2022

Scientific Programming

View full text Add to dashboard Cite

show abstract

“…Eisner (2001) describes an algorithm for estimating probabilities in a finite-state transducer from data using EM-based methods. Weiss, Goldberg, and Yahav (2019) and Okudono et al (2020) provide adaptations to the Weiss, Goldberg, and Yahav (2018) DFA extraction algorithm to yield weighted automata.…”

Section: Related Workmentioning

confidence: 99%

Approximating Probabilistic Models as Weighted Finite Automata

Suresh

Roark

Riley

et al. 2021

Computational Linguistics

View full text Add to dashboard Cite

Weighted finite automata (WFA) are often used to represent probabilistic models, such as n-gram language models, since, among other things, they are efficient for recognition tasks in time and space. The probabilistic source to be represented as a WFA, however, may come in many forms. Given a generic probabilistic model over sequences, we propose an algorithm to approximate it as a weighted finite automaton such that the Kullback-Leibler divergence between the source model and the WFA target model is minimized. The proposed algorithm involves a counting step and a difference of convex optimization step, both of which can be performed efficiently.We demonstrate the usefulness of our approach on various tasks, including distilling n-gram models from neural models, building compact language models, and building open-vocabulary character models. The algorithms used for these experiments are available in an open-source software library.

show abstract

“…Recent work [ 22 ] also shows the equivalence between a 2-RNN with linear hidden activation and weighted automata. Related work [ 23 , 24 ] used weighted automata to explore the relationships between deep learning and grammatical inference. Since a 2-RNN has a 3D tensor weight, computation is more intensive.…”

Section: Preliminariesmentioning

confidence: 99%

An Entropy Metric for Regular Grammar Classification and Learning with Recurrent Neural Networks

Zhang

Wang

Giles

2021

Entropy

View full text Add to dashboard Cite

Recently, there has been a resurgence of formal language theory in deep learning research. However, most research focused on the more practical problems of attempting to represent symbolic knowledge by machine learning. In contrast, there has been limited research on exploring the fundamental connection between them. To obtain a better understanding of the internal structures of regular grammars and their corresponding complexity, we focus on categorizing regular grammars by using both theoretical analysis and empirical evidence. Specifically, motivated by the concentric ring representation, we relaxed the original order information and introduced an entropy metric for describing the complexity of different regular grammars. Based on the entropy metric, we categorized regular grammars into three disjoint subclasses: the polynomial, exponential and proportional classes. In addition, several classification theorems are provided for different representations of regular grammars. Our analysis was validated by examining the process of learning grammars with multiple recurrent neural networks. Our results show that as expected more complex grammars are generally more difficult to learn.

show abstract

Weighted Automata Extraction from Recurrent Neural Networks via Regression on State Spaces

Cited by 33 publications

References 16 publications

Research on Online Learner Modeling and Course Recommendation Based on Emotional Factors

Research on Online Learner Modeling and Course Recommendation Based on Emotional Factors

Approximating Probabilistic Models as Weighted Finite Automata

An Entropy Metric for Regular Grammar Classification and Learning with Recurrent Neural Networks

Contact Info

Product

Resources

About