No abstract

BACKGROUND Automatic speech recognition (ASR) technology is increasingly being used for transcription in clinical contexts. Although there are numerous HIPAA-compliant transcription services using ASR, few studies have compared the word error rate (WER) between different transcription services among different diagnostic groups in a mental health setting. There has also been little research into the types of words ASR transcriptions mistakenly generate or omit. OBJECTIVE This study compared the WER of three ASR transcription services (Amazon Transcribe, Zoom/Otter AI, and Whisper/Open AI) in interviews across three different clinical categories (controls, participants experiencing depression, and participants experiencing a variety of other mental health conditions). These ASR transcription services were also compared to a commercial human transcription service, REV. Words that were either included or excluded by the error in the transcripts were systematically analyzed by their Linguistic Inquiry and Word Count (LIWC) categories. METHODS Participants completed a one-time research psychiatric interview, which was recorded on a secure server. Transcriptions created by the research team were used as the gold standard from which WER was calculated. The interviewees were categorized into either the control group (N = 19), the major depressive disorder (MDD) group (N = 22), or the other group (N = 24) using the Mini-International Neuropsychiatric Interview. The total sample included 65 participants. Brunner-Munzel tests were used for comparing independent sets such as the diagnostic groupings, and Wilcoxon signed-rank tests were used for correlated samples when comparing the total sample between different transcriptions services. RESULTS There were significant differences between each ASR transcription service WER (P < .001). Amazon Transcribe’s output exhibited significantly lower WERs compared to the Zoom/Otter AI and Whisper/Open AI ASR. ASR performances did not significantly differ across the three different clinical categories within each service (P > 0.05). A comparison between the human transcription service output from REV and the best-performing ASR (Amazon Transcribe) demonstrated a significant difference, with REV having a slightly lower median WER (7.6% versus 8.9%). Heatmaps and spider plots were used to visualize the most common errors in LIWC categories, which were found to be within three overarching categories: Conversation, Cognition, and Function. CONCLUSIONS Overall, these results indicate that the WER between manual and automated transcription services is narrowing as ASR services advance. These advances, coupled with decreased cost and time in receiving transcriptions, may make ASR transcriptions a more viable option within healthcare settings. However, more research is required to determine if errors in specific types of words impact the analysis and utility of these transcriptions, particularly for specific applications and in a variety of populations in terms of clinical diagnosis, literacy level, accent, and cultural origin.

The study described herein is a continuation of our work in which we developed a methodology to identify small foci of transduced cells following rectal challenge of rhesus macaques with a non-replicative luciferase reporter virus. In the current study, the wild-type virus was added to the inoculation mix and twelve rhesus macaques were necropsied 2-4 days after the rectal challenge to study the changes in infected cell phenotype as the infection progressed. Relying on luciferase reporter we noted that both anus and rectum tissues are susceptible to the virus as early as 48h after the challenge. Small regions of the tissue containing luciferase-positive foci were further analyzed microscopically and were found to also contain cells infected by wild-type virus. Phenotypic analysis of the Env and Gag positive cells in these tissues revealed the virus can infect diverse cell populations, including but not limited to Th17 T cells, non Th17 T cells, immature dendritic cells, and myeloid-like cells. The proportions of the infected cell types, however, did not vary much during the first four days of infection when anus and rectum tissues were examined together. Nonetheless, when the same data was analyzed on a tissue-specific basis, we found significant changes in infected cell phenotypes over the course of infection. For anal tissue, a statistically significant increase in infection was observed for Th17 T cells and myeloid-like cells, while in the rectum, the non-Th17 T cells showed the biggest temporal increase, also of statistical significance.

No abstract