WordTokenizers.jl: Basic tools for tokenizing natural language in Julia

Kaushal, Ayush; White, Lyndon; Innes, Mike; Kumar, Rohit

doi:10.21105/joss.01956

Cited by 6 publications

(3 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Tokenizing is the process of separating each word in a sentence [15]. In this case, the question that sent to the chatbot will be processed into a word/token…”

Section: Pre-processingmentioning

confidence: 99%

Chatbot-Based Movie Recommender System Using POS Tagging

Nugraha

Baizal

Richasdy

2022

bits

View full text Add to dashboard Cite

The movie recommender system is a technology designed to make it easier for users to provide recommendations quickly and among the many pieces of information. Because the number of movies is huge, it causes a person to be confused in determining the choice of the movie to watch. Many movie recommending systems have been developed, but users cannot interact intensively. Based on these problems, we developed a chatbot-based conversational recommender system, which can interact intensively with the system. The developed chatbot uses normal language handling to permit the framework to comprehend what the user enters as natural language. POS Tagging is used to find tags in the form of movie titles with patterns in the POS Tagging model. However, the algorithm of those used on POS Tagging does not pay attention to the sentence entity, so the predicted title must correspond to the provisions of POS Tagging. Multinomial Naive Bayes looks for similarities of user input to datasets on intents. The dataset with the highest probability value or almost equal to the sentence entered by the user can be used as a response to user input. The test results of the chatbot application showed that the match rate between response and user input was 89.1%. Thus, the developed chatbot can be used well to provide practical and interactive movie recommendations.

show abstract

“…Tokenizing is the process of separating each word in a sentence [15]. In this case, the question that sent to the chatbot will be processed into a word/token…”

Section: Pre-processingmentioning

confidence: 99%

Chatbot-Based Movie Recommender System Using POS Tagging

Nugraha

Baizal

Richasdy

2022

bits

View full text Add to dashboard Cite

show abstract

“…Bert multitask model predicts using the [CLS] representation from Bert (Devlin et al, 2019). We also build an LSTM model (Hochreiter and Schmidhuber, 1997) with GloVe embedding (Pennington et al, 2014), and twitter-tokenization using Word- Tokenizers package (Kaushal et al, 2020).…”

Section: Slot-fillingmentioning

confidence: 99%

Winners at W-NUT 2020 Shared Task-3: Leveraging Event Specific and Chunk Span information for Extracting COVID Entities from Tweets

Kaushal

Vaidhya

2020

Proceedings of the Sixth Workshop on Noisy User-Generated Text (W-Nut 2020)

Self Cite

View full text Add to dashboard Cite

Twitter has acted as an important source of information during disasters and pandemic, especially during the times of COVID-19. In this paper, we describe our system entry for WNUT 2020 Shared Task-3. The task was aimed at automating the extraction of a variety of COVID-19 related events from Twitter, such as individuals who recently contracted the virus, someone with symptoms who were denied testing and believed remedies against the infection. The system consists of separate multi-task models for slot-filling subtasks and sentence-classification subtasks while leveraging the useful sentence-level information for the corresponding event. The system uses COVID-Twitter-Bert with attention-weighted pooling of candidate slot-chunk features to capture the useful information chunks. The system ranks 1st at the leader-board with F1 of 0.6598, without using any ensembles or additional datasets. The code and trained models are available at this https url 1 .

show abstract

“…As computers play an increasing role, there is a pressing desire to be able to manipulate them in a natural way. People can interact with machines in the same way that they interact with each other in everyday life (voice, gestures, and expressions) [8][9][10][11][12]. As a result, there has been an increasing amount of research in recent years on interacting with computers through multiple modalities, such as simple voice, gesture, and expression interaction.…”

Section: Introductionmentioning

confidence: 99%

Research on Semisupervised Affective Interaction Technology in Immersive Virtual Teaching Environment

Tang

Zhang

2022

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

The separation of time and space in immersive virtual teaching makes students unable to realize emotional communication, which may affect students’ mental health. In recent years, the use of affective computing technology to solve the problem of affective loss in distance education has become a key research topic. In order to realize the problem of emotion interaction in immersive virtual teaching, a semisupervised support vector machine- (SVM-) based affective interaction model was proposed. First, the natural language sequences of students in the virtual teaching environment are preprocessed using a statistical-based framing method, and mutual information and expected cross-entropy are used as feature selection methods. Then, a vector space model based on TF/IDF feature term weights is proposed to implement the feature vector representation of natural language sequences. Finally, after the constructed sentiment space, a semisupervised SVM is employed as the classifier to complete the affective interaction computation. The experimental results of emotion classification show that the proposed model is able to determine and understand the emotional state more accurately than other traditional models and significantly improves the training speed. In addition, the proposed model can provide emotional encouragement or emotional compensation according to the specific emotional state of the learner.

show abstract

WordTokenizers.jl: Basic tools for tokenizing natural language in Julia

Cited by 6 publications

References 2 publications

Chatbot-Based Movie Recommender System Using POS Tagging

Chatbot-Based Movie Recommender System Using POS Tagging

Winners at W-NUT 2020 Shared Task-3: Leveraging Event Specific and Chunk Span information for Extracting COVID Entities from Tweets

Research on Semisupervised Affective Interaction Technology in Immersive Virtual Teaching Environment

Contact Info

Product

Resources

About