Proactive Personality Measurement Using Item Response Theory and Social Media Text Mining

Abstract

This prospective study was designed to propose a novel method of assessing proactive personality by combining text mining technology and Item Response Theory (IRT) to measure proactive personality more efficiently. We got freely expressed texts (essay question text dataset and social media text dataset) and item response data on the topic of proactive personality from 901 college students. To enhance validity and reliability, three different approaches were employed in the study. In Method 1, we used item response data to develop a proactive personality evaluation model based on IRT. In Method 2, we used freely expressed texts to develop a proactive personality evaluation model based on text mining. In Method 3, we utilized the text mining results as the prior information for the IRT estimation and built a proactive personality evaluation model combining text mining and IRT. Finally, we evaluated those three approaches via the confusion matrix indicators. The major result revealed that (1) the combined method based on essay question text, micro-blog text with pre-estimated IRT parameters performed the highest accuracy of 0.849; (2) the combined method using essay question text and pre-estimated IRT parameters performed the highest sensitivity of 0.821; (3) the text classification method based on essay question text had the best performance on the specificity of 0.959; and (4) if the models were considered comprehensively, the combined method using essay question text, micro-blog text, and pre-estimated IRT parameters achieved the best performance. Thus, we concluded that the novel combined method was significantly better than the other two traditional methods based on IRT and text mining.

A new modification and application of item response theory‐based feature selection for different machine learning tasks

“…As summarized above, our review of the literature showed that combining ML and IRT fields is one of the hot topics day by day in the fields of education, psychology, and computer science. This combination often aims to solve cold-start problem in learning ability evaluation, 28,36 classifier/instance hardness evaluation, 27,[29][30][31]33,37 suitable dataset selection, 38 estimation of IRT parameters, 32,35,39 FS, 7,[24][25][26] feature extension, 40 and other purposes especially ones 7,41,42 that use information between IRT models and text mining tasks. This brief summarization shows that using IRT models as feature selectors is also attracting the interest of the research community, especially in the field of computer science.…”