Applying Big Data visualization to detect trends in 30 years of performance reports

Abstract

Evaluators worldwide are dealing with a growing amount of unstructured electronic data, predominantly in textual format. Currently, evaluators analyze textual Big Data primarily using traditional content analysis methods based on keyword search, a practice that is limited to iterating over predefined concepts. But what if evaluators cannot define the necessary keywords for their analysis? Often we should examine trends in the way certain organizations have been operating, while our raw data are gigabytes of documents generated by that organization over decades. The problem is that in many cases we do not know what exactly we need to look for. In such cases, traditional analytical machinery would not provide an adequate solution within reasonable time—instead, heavy-lifting Big Data Science should be applied. We propose an automated, quantitative, user-friendly methodology based on text mining, machine learning, and data visualization, which assists researchers and evaluation practitioners to reveal trends, trajectories, and interrelations between bits and pieces of textual information in order to support evaluation. Our system automatically extracts a large amount of descriptive terminology for a particular domain in a given language, finds semantic connections between documents based on the extracted terminology, visualizes the entire document repository as a graph of semantic connections, and leads the user to the areas on that graph where most interesting trends can be observed. This article exemplifies the new method on 1700 performance reports, showing that the method can be used successfully, supplying evaluators with highly important information which cannot be revealed using other methods. Such exploratory exercise is vital as a preliminary exploratory phase for evaluations involving unstructured Big Data, after which a range of evaluation methods can be applied. We argue that our system can be successfully implemented on any domain evaluated.

Evaluation criteria for artificial intelligence

“…Presumably to remedy this state of affairs, scholars have advanced three related threads: Some have outlined arguments for why it is important to integrate AI in evaluation (Bamberger, 2016; Hejnowicz & Chaplowe, 2021; Leeuw, 2020; York & Bamberger, 2020). Examples of potential uses of techniques grounded in one of AI's branches are being published, which offer case examples from which evaluation pracitioners can learn (see, e.g., Cintron & Montrosse‐Moorhead, 2021; Raveh et al., 2020; Roy & Rambo‐Hernandez, 2021). Others have highlighted critical questions about the use of AI in evaluation (Leeuw, 2020; Picciotto, 2020).…”

Establishing a Novel Algorithm for Highly Responsive Storage Space Allocation Based on NAR and Improved NSGA‐III

“…However, some of these assumptions are not accurate and are not supported by massive data inputs [33,34]. Current well-documented theoretical studies have shown that data generated from various industry sectors, when fully applied, will significantly improve management effectiveness and economic performance [35,36]. NAR artificial neural networks, with easily adjustable parameters, can provide demand forecasting with high accuracy and with improved NSGA-III based on the good points set theory, resulting in a highly responsive space allocation strategy.…”

Intelligent Recognition and Teaching of English Fuzzy Texts Based on Fuzzy Computing and Big Data

“…Charitopoulos et al point out that the SVD rank reduction algorithm does not guarantee the nonnegativity of the matrix decomposition, so nonnegative matrix factorization (NMF) becomes a sensible choice for decomposing the PPMI, which guarantees the nonnegativity of the dimensional approximate reduction of the semantic relationship of the word-context information and is more consistent with the semantic relationship hypothesis [7]. Raveh et al demonstrated that matrix factorization methods are consistent with the word vectors of neural network language models in terms of task performance, such that PPMI-SVD matrix decomposition is equivalent to skipgram (SGNS) based on negative sampling [8]. The neural network language model has an inherent advantage over factorization methods, which have more flexible super parameter settings [9].…”