Identifying women with postdelivery posttraumatic stress disorder using natural language processing of personal childbirth narratives

Bartal, Alon; Jagodnik, Kathleen M.; CHAN, Ms Sabrina J.; BABU, Ms Mrithula S.; Dekel, Sharon

doi:10.1016/j.ajogmf.2022.100834

Cited by 16 publications

(27 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We used the HuggingFace repository [53] with Python coding. The models that we compared (Table 4) were evaluated using two Evaluation Methods on the dataset published in [44].…”

Section: Resultsmentioning

confidence: 99%

“…However, the examination of Table 4 Comparative performance analysis with different embeddings in Model #3 (Appendix A). The average performance results of the 10-fold cross-validation process conducted on the same analyzed dataset that was used in [44] are presented. OpenAI's embeddings in Model #3, outperform all other embeddings in Model #3, demonstrating superior ability in identifying CB-PTSD using narrative data only.…”

Section: Discussionmentioning

confidence: 99%

“…Data preparation. We discarded narratives with < 30 words from the dataset [44,45]. To handle the imbalance in the analyzed dataset (due to the small representation of cases with PCL-5 ≥ 31), we randomly sampled the majority Class 0 to fit the size of the minority Class 1.…”

Section: Discussionmentioning

confidence: 99%

“…The gpt-3. To identify similar or contextually relevant narratives, in Step #3 we adopted the approach used in our previous work [44]. This approach analyzes pairs of narratives To avoid overfitting, we stopped training when there was no loss improvement for 3 consecutive epochs.…”

Section: Narrative Analysismentioning

confidence: 99%

“…AUC F1 score Sensitivity (Recall) Specificity Model #3 0.80 0.82 0.81 0.72 all-mpnet-base-v2 [44] 0.75 0.76 0.80 0.70 mental-roberta-base [6] 0.67 0.71 0.80 0.55 mental-xlnet-base-cased [7] 0.65 0.70 0.80 0.50 Bio ClinicalBERT [20] 0.65 0.63 0.60 0.70 biogpt [21] 0.62 0.59 0.55 0.70 mental-bert-base-uncased [6] 0.63 0.58 0.60 0.70 these technologies for mental health assessment remains in early stages, with previous findings about the utility of ChatGPT being mixed, and previous reports suggesting that ML models trained on application-specific corpuses of text might be necessary for accurate model performance. This study sought to explore the performance of different variations of ChatGPT for the purpose of identifying probable cases of childbirthrelated post-traumatic stress disorder (CB-PTSD) using the childbirth narratives of postpartum women.…”

Section: Modelmentioning

confidence: 99%

See 4 more Smart Citations

ChatGPT Demonstrates Potential for Identifying Psychiatric Disorders: Application to Childbirth-Related Post-Traumatic Stress Disorder

Bartal,

Jagodnik,

Chan

et al. 2023

Preprint

Self Cite

View full text Add to dashboard Cite

Free-text analysis using Machine Learning (ML)-based Natural Language Processing (NLP) shows promise for diagnosing psychiatric conditions. Chat Generative Pre-trained Transformer (ChatGPT) has demonstrated initial feasibility for this purpose; however, this work remains preliminary, and whether it can accurately assess mental illness remains to be determined. This study examines ChatGPT’s utility to identify post-traumatic stress disorder following childbirth (CB-PTSD), a maternal postpartum mental illness affecting millions of women annually, with no standard screening protocol. We explore ChatGPT’s potential to screen for CB-PTSD by analyzing maternal childbirth narratives as the sole data source. By developing an ML model that utilizes ChatGPT’s knowledge, we identify CB-PTSD via narrative classification. Our model outperformed (F1 score: 0.82) ChatGPT and six previously published large language models (LLMs) trained on mental health or clinical domains data, suggesting that ChatGPT can be harnessed to identify CB-PTSD. Our modeling approach can be generalized to assess other mental health disorders.

show abstract

“…We used the HuggingFace repository [53] with Python coding. The models that we compared (Table 4) were evaluated using two Evaluation Methods on the dataset published in [44].…”

Section: Resultsmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Narrative Analysismentioning

confidence: 99%