Data-Driven Monitoring in Community Based Management of Sam Children Using Psychometric Techniques: An Operational Framework

Joshi, Ankur; Pakhare, Abhijit; Nair, Sivaja K.; Revadi, G; Chouhan, Manoj; Pandey, Deepak; Kokane, Arun

doi:10.7759/cureus.18589

Cited by 3 publications

(4 citation statements)

References 22 publications

(29 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…During the DF season, individuals who suspect they may have DF can click on the link provided to determine their DF risk based on the 11 identified symptoms. They can also view the APP [ 65 ] in action through demonstrations using an MP4 video as shown on the QR-code in Figure 9 .…”

Section: Resultsmentioning

confidence: 99%

“…Four visualizations in RaschOnline [59] were applied to present item features and person responses, including Wright Map [62] with groups, differential item functioning (DIF) [63] using forest plots, [64] item characteristic curves (ICCs), [65,66] and KIDMAP. [67] The method of drawing these visualizations refers to the manual of RaschOnline.…”

Section: Methodsmentioning

confidence: 99%

“…[60] Through the Newton-Raphson iteration method [61] and the person estimate and SE( θ) in Equations ( 1) and ( 5), RaschOnline [59] was programed and developed. Four visualizations in RaschOnline [59] were applied to present item features and person responses, including Wright Map [62] with groups, differential item functioning (DIF) [63] using forest plots, [64] item characteristic curves (ICCs), [65,66] and KIDMAP. [67] The method of drawing these visualizations refers to the manual of RaschOnline.…”

Section: Rasch Analysis Of Item Features and Person Responses Using R...mentioning

confidence: 99%

“…Details about DIF are in Supplemental Digital Content 2, http://links.lww.com/MD/I679. (4) The ICC [64,65] is a plot of the probability of the examinee answering a question correctly against his or her underlying abilities on the trait being measured. [69] The ICC is based on item response theory: the curve is bounded between 0 and 1, monotonically increases and is commonly referred to as a logistic function.…”

Section: Four Tasks Required To Achievementioning

confidence: 99%

See 3 more Smart Citations

Detecting dengue fever in children using online Rasch analysis to develop algorithms for parents: An APP development and usability study

Chow

Chien

et al. 2023

Medicine

View full text Add to dashboard Cite

Background: Dengue fever (DF) is a significant public health concern in Asia. However, detecting the disease using traditional dichotomous criteria (i.e., absent vs present) can be extremely difficult. Convolutional neural networks (CNNs) and artificial neural networks (ANNs), due to their use of a large number of parameters for modeling, have shown the potential to improve prediction accuracy (ACC). To date, there has been no research conducted to understand item features and responses using online Rasch analysis. To verify the hypothesis that a combination of CNN, ANN, K-nearest-neighbor algorithm (KNN), and logistic regression (LR) can improve the ACC of DF prediction for children, further research is required. Methods: We extracted 19 feature variables related to DF symptoms from 177 pediatric patients, of whom 69 were diagnosed with DF. Using the RaschOnline technique for Rasch analysis, we examined 11 variables for their statistical significance in predicting the risk of DF. Based on 2 sets of data, 1 for training (80%) and the other for testing (20%), we calculated the prediction ACC by comparing the areas under the receiver operating characteristic curve (AUCs) between DF + and DF− in both sets. In the training set, we compared 2 scenarios: the combined scheme and individual algorithms. Results: Our findings indicate that visual displays of DF data are easily interpreted using Rasch analysis; the k-nearest neighbors algorithm has a lower AUC (<0.50); LR has a relatively higher AUC (0.70); all 3 algorithms have an almost equal AUC (=0.68), which is smaller than the individual algorithms of Naive Bayes, LR in raw data, and Naive Bayes in normalized data; and we developed an app to assist parents in detecting DF in children during the dengue season. Conclusion: The development of an LR-based APP for the detection of DF in children has been completed. To help patients, family members, and clinicians differentiate DF from other febrile illnesses at an early stage, an 11-item model is proposed for developing the APP.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Rasch Analysis Of Item Features and Person Responses Using R...mentioning

confidence: 99%

Section: Four Tasks Required To Achievementioning

confidence: 99%

See 2 more Smart Citations

Detecting dengue fever in children using online Rasch analysis to develop algorithms for parents: An APP development and usability study

Chow

Chien

et al. 2023

Medicine

View full text Add to dashboard Cite

show abstract

Assessing ChatGPT’s Capability for Multiple Choice Questions Using RaschOnline: Observational Study

Chow¹,

Cheng²,

Chien³

et al. 2024

JMIR Form Res

View full text Add to dashboard Cite

Background ChatGPT (OpenAI), a state-of-the-art large language model, has exhibited remarkable performance in various specialized applications. Despite the growing popularity and efficacy of artificial intelligence, there is a scarcity of studies that assess ChatGPT’s competence in addressing multiple-choice questions (MCQs) using KIDMAP of Rasch analysis—a website tool used to evaluate ChatGPT’s performance in MCQ answering. Objective This study aims to (1) showcase the utility of the website (Rasch analysis, specifically RaschOnline), and (2) determine the grade achieved by ChatGPT when compared to a normal sample. Methods The capability of ChatGPT was evaluated using 10 items from the English tests conducted for Taiwan college entrance examinations in 2023. Under a Rasch model, 300 simulated students with normal distributions were simulated to compete with ChatGPT’s responses. RaschOnline was used to generate 5 visual presentations, including item difficulties, differential item functioning, item characteristic curve, Wright map, and KIDMAP, to address the research objectives. Results The findings revealed the following: (1) the difficulty of the 10 items increased in a monotonous pattern from easier to harder, represented by logits (–2.43, –1.78, –1.48, –0.64, –0.1, 0.33, 0.59, 1.34, 1.7, and 2.47); (2) evidence of differential item functioning was observed between gender groups for item 5 (P=.04); (3) item 5 displayed a good fit to the Rasch model (P=.61); (4) all items demonstrated a satisfactory fit to the Rasch model, indicated by Infit mean square errors below the threshold of 1.5; (5) no significant difference was found in the measures obtained between gender groups (P=.83); (6) a significant difference was observed among ability grades (P<.001); and (7) ChatGPT’s capability was graded as A, surpassing grades B to E. Conclusions By using RaschOnline, this study provides evidence that ChatGPT possesses the ability to achieve a grade A when compared to a normal sample. It exhibits excellent proficiency in answering MCQs from the English tests conducted in 2023 for the Taiwan college entrance examinations.

show abstract

Assessing ChatGPT’s Capability for Multiple Choice Questions Using RaschOnline: Observational Study (Preprint)

Chow¹,

Chien²,

Chou³

2023

Preprint

View full text Add to dashboard Cite

BACKGROUND ChatGPT, a new large language model (LLM) developed by OpenAI, has demonstrated impressive performance in several specialized applications. In spite of the rising popularity and performance of artificial intelligence, there are a limited number of studies that evaluate ChatGPT’s capability for multiple choice questions (NCQs) using KIDMAP of Rasch analysis (i.e., an online tool called KIDMAP, which is used to evaluate the performance of ChatGPT in answering MCQs). OBJECTIVE Study objectives were to (1) demonstrate the use of online Rasch analysis (namely, RaschOnline) and (2) determine the ChatGPT’s grade compared to a normal sample. METHODS ChatGPT capability was evaluated using ten items from Taiwan college entrance examinations for the year 2023. Under a Rasch model, 300 virtual students with normal distributions were simulated and generated to compete with ChatGPR's responses. A total of five visual presentations were created using RaschOnline (e.g., item difficulties, differential item functioning (DIF), item characteristic curve, Wright map, and KIDMAP) to answer the research questions outlined in the objectives. RESULTS The results indicated that (1) the difficulty of the ten items monotonously increased from easier to harder (i.e., -2.43, -1.78, -1.48, -0.64, -0.1, 0.33, 0.59, 1.34, 1.7, and 2.47 logits); (2) there was evidence of DIF between gender groups for item 5(p=0.042); (3) item 5 fits the Rasch model rather well (p=0.61); (4) all of the items fit the Rasch model, as indicated by Infit mean square errors below the threshold of 1.5; (5) there was no significant difference in the measures obtained between gender groups(p=0.832); (6) a significant difference was observed among ability grades(P< 0.001); and (7)ChatGPT's capability is graded as A, surpassing grades B to E. CONCLUSIONS With RaschOnline, we demonstrate that ChatGPT is capable of scoring a grade A compared to a normal sample, with an excellent ability to answer MCQs of English tests for the year 2023 on Taiwan college entrance examinations.

show abstract

Data-Driven Monitoring in Community Based Management of Sam Children Using Psychometric Techniques: An Operational Framework

Cited by 3 publications

References 22 publications

Detecting dengue fever in children using online Rasch analysis to develop algorithms for parents: An APP development and usability study

Detecting dengue fever in children using online Rasch analysis to develop algorithms for parents: An APP development and usability study

Assessing ChatGPT’s Capability for Multiple Choice Questions Using RaschOnline: Observational Study

Assessing ChatGPT’s Capability for Multiple Choice Questions Using RaschOnline: Observational Study (Preprint)

Contact Info

Product

Resources

About