Psychologist in a Pocket: Lexicon Development and Content Validation of a Mobile-Based App for Depression Screening

Abstract

BackgroundLanguage reflects the state of one’s mental health and personal characteristics. It also reveals preoccupations with a particular schema, thus possibly providing insights into psychological conditions. Using text or lexical analysis in exploring depression, negative schemas and self-focusing tendencies may be depicted. As mobile technology has become highly integrated in daily routine, mobile devices have the capacity for ecological momentary assessment (EMA), specifically the experience sampling method (ESM), where behavior is captured in real-time or closer in time to experience in one’s natural environment. Extending mobile technology to psychological health could augment initial clinical assessment, particularly of mood disturbances, such as depression and analyze daily activities, such as language use in communication. Here, we present the process of lexicon generation and development and the initial validation of Psychologist in a Pocket (PiaP), a mobile app designed to screen signs of depression through text analysis.ObjectiveThe main objectives of the study are (1) to generate and develop a depressive lexicon that can be used for screening text-input in mobile apps to be used in the PiaP; and (2) to conduct content validation as initial validation.MethodsThe first phase of our research focused on lexicon development. Words related to depression and its symptoms based on the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) and in the ICD-10 Classification of Mental and Behavioural Disorders: Clinical Descriptions and Diagnostic Guidelines classification systems were gathered from focus group discussions with Filipino college students, interviews with mental health professionals, and the review of established scales for depression and other related constructs.ResultsThe lexicon development phase yielded a database consisting of 13 categories based on the criteria depressive symptoms in the DSM-5 and ICD-10. For the draft of the depression lexicon for PiaP, we were able to gather 1762 main keywords and 9655 derivatives of main keywords. In addition, we compiled 823,869 spelling variations. Keywords included negatively-valenced words like “sad”, “unworthy”, or “tired” which are almost always accompanied by personal pronouns, such as “I”, “I’m” or “my” and in Filipino, “ako” or “ko”. For the content validation, only keywords with CVR equal to or more than 0.75 were included in the depression lexicon test-run version. The mean of all CVRs yielded a high overall CVI of 0.90. A total of 1498 main keywords, 8911 derivatives of main keywords, and 783,140 spelling variations, with a total of 793, 553 keywords now comprise the test-run version.ConclusionsThe generation of the depression lexicon is relatively exhaustive. The breadth of keywords used in text analysis incorporates the characteristic expressions of depression and its related constructs by a particular culture and age group. A content-validated mobile health app, PiaP may help augment a more effective and early detect...

Validation of an mHealth App for Depression Screening and Monitoring (Psychologist in a Pocket): Correlational Study and Concurrence Analysis

“…In this paper, only the results from stage 3 of the tripartite model are presented and discussed (see [19] for stages 1 and 2).…”

“…The study’s construct is depressive symptoms . It is characterized by negatively valenced words (words that describe unpleasant emotions) grouped according to 1 of the PiaP 13 symptoms based on a prior-developed lexicon and the frequency of first-person pronoun usage (see Cheng, et al [19] and Ramos, Cheng et al [27] for the development of the mentioned lexicon).…”

“…PiaP’s basic assumptions are as follows: (1) Everyday language—its usage, content, and themes—is a reliable indicator of the state of one’s mental health; (2) Individuals tend to reveal personal information when using electronic media; and 3) Depressed or depression-prone individuals tend to self-focus and to ruminate on the negative aspects of their lives. PiaP aims at detecting changes in the nature of electronic text inputs through a lexicon of words in English and Tagalog related to depression, which were developed using both top-down and bottom-up processes (see [19] for app details and [18] for technical details). Sources for the lexicon were (1) symptom classification systems of the Diagnostic and Statistical Manual of Mental Disorders (DSM)-5 criteria for major depressive disorder and the International Statistical Classification of Diseases and Related Health Problems–10 criteria for depressive disorder, (2) focus group discussions, (3) interviews with mental health professionals, and (4) established psychological tests.…”

Pengembangan Instrumen Literasi Digital Sekolah, Siswa Dan Guru

“…Kelebihan dari angket literasi digital sekolah ini yaitu memiliki 83% butir pertanyaan dengan tingkat validitas yang tinggi, sehingga sudah dapat langsung digunakan untuk mengukur tingkat literasi digital. Cheng et al (2016) mengungkapkan bahwa validasi isi merupakan salah satu prosedur psikometrik yang menghitung kemampuan suatu alat tes untuk mengukur. Selain itu, bentuk pertanyaan tertutup lebih memudahkan responden dalam menjawab tanpa memberikan pendapat yang dapat mempengaruhi peneliti (Reja, Manfreda, & Hlebec, 2003).…”

Development, Reliability, and Validity of Open-ended Test to Measure Student’s Digital Literacy Skill

“…Content validation is one the psychometric procedures that index a test's validity or its ability to measure what it purports to measure (Cheng et al, 2016). It involves a panel of subject matter ‚experts‛ rating items into one of three categories: ‚essential,‛ ‚useful, but not essential,‛ or ‚not necessary.‛ Items deemed ‚essential‛ by a critical number of panel members are then included within the final instrument, with items failing to achieve this critical level discarded (Ayre & Scally, 2014).…”