A pilot study on the evaluation of medical student documentation: assessment of SOAP notes

Seo, Ji Hyun; Kong, Hyun-Hee; Im, Sun Ju; Roh, HyeRin; Kim, Do Kyong; Bae, Hwa ok; Oh, Young Rim

doi:10.3946/kjme.2016.26

Cited by 20 publications

(16 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The notes in these consultations were often very brief, and the completeness and quality of information varied across entries. This is similar in studies from routine primary care [ 25 ]; however, in OOH care, this is likely to be worse, making it more difficult to develop mining models. This reflects the reality of medical practice and the limitations of real-world data.…”

Section: Discussionsupporting

confidence: 54%

Predicting COVID-19 Symptoms From Free Text in Medical Records Using Artificial Intelligence: Feasibility Study

Olmen¹,

Nooten²,

Philips³

et al. 2022

JMIR Med Inform

View full text Add to dashboard Cite

Background Electronic medical records have opened opportunities to analyze clinical practice at large scale. Structured registries and coding procedures such as the International Classification of Primary Care further improved these procedures. However, a large part of the information about the state of patient and the doctors’ observations is still entered in free text fields. The main function of those fields is to report the doctor’s line of thought, to remind oneself and his or her colleagues on follow-up actions, and to be accountable for clinical decisions. These fields contain rich information that can be complementary to that in coded fields, and until now, they have been hardly used for analysis. Objective This study aims to develop a prediction model to convert the free text information on COVID-19–related symptoms from out of hours care electronic medical records into usable symptom-based data that can be analyzed at large scale. Methods The design was a feasibility study in which we examined the content of the raw data, steps and methods for modelling, as well as the precision and accuracy of the models. A data prediction model for 27 preidentified COVID-19–relevant symptoms was developed for a data set derived from the database of primary-care out-of-hours consultations in Flanders. A multiclass, multilabel categorization classifier was developed. We tested two approaches, which were (1) a classical machine learning–based text categorization approach, Binary Relevance, and (2) a deep neural network learning approach with BERTje, including a domain-adapted version. Ethical approval was acquired through the Institutional Review Board of the Institute of Tropical Medicine and the ethics committee of the University Hospital of Antwerpen (ref 20/50/693). Results The sample set comprised 3957 fields. After cleaning, 2313 could be used for the experiments. Of the 2313 fields, 85% (n=1966) were used to train the model, and 15% (n=347) for testing. The normal BERTje model performed the best on the data. It reached a weighted F1 score of 0.70 and an exact match ratio or accuracy score of 0.38, indicating the instances for which the model has identified all correct codes. The other models achieved respectable results as well, ranging from 0.59 to 0.70 weighted F1. The Binary Relevance method performed the best on the data without a frequency threshold. As for the individual codes, the domain-adapted version of BERTje performs better on several of the less common objective codes, while BERTje reaches higher F1 scores for the least common labels especially, and for most other codes in general. Conclusions The artificial intelligence model BERTje can reliably predict COVID-19–related information from medical records using text mining from the free text fields generated in primary care settings. This feasibility study invites researchers to examine further possibilities to use primary...

show abstract

Section: Discussionsupporting

confidence: 54%

Predicting COVID-19 Symptoms From Free Text in Medical Records Using Artificial Intelligence: Feasibility Study

Olmen¹,

Nooten²,

Philips³

et al. 2022

JMIR Med Inform

View full text Add to dashboard Cite

show abstract

“…Completeness of EMR is one of the most frequently discussed issues that may limit the use of EMR data for clinical and population health research [15]. This issue was reported to be particularly relevant in the evaluation of medical student documentation [16]; however, no systematic assessment of the completeness of EMR in dental student clinics has been performed.…”

Section: Introductionmentioning

confidence: 99%

Completeness of Electronic Dental Records in a Student Clinic: Retrospective Analysis

2019

View full text Add to dashboard Cite

Background A well-designed, adequately documented, and properly maintained patient record is an important tool for quality assurance and care continuity. Good clinical documentation skills are supposed to be a fundamental part of dental student training. Objective The goal of this study was to assess the completeness of electronic patient records in a student clinic. Methods Completeness of patient records was assessed using comparative review of validated cases of alveolar osteitis treated between August 2011 and May 2017 in a student clinic at Columbia University College of Dental Medicine, New York, USA. Based on a literature review, population-based prevalence of nine most frequently mentioned symptoms, signs, and treatment procedures of alveolar osteitis was identified. Completeness of alveolar osteitis records was assessed by comparison of population-based prevalence and frequency of corresponding items in the student documentation. To obtain all alveolar osteitis cases, we ran a query on the electronic dental record, which included all cases with diagnostic code Z1820 or any variation of the phrases “dry socket” and “alveolar osteitis” in the notes. The resulting records were manually reviewed to definitively confirm alveolar osteitis and to extract all index items. Results Overall, 296 definitive cases of alveolar osteitis were identified. Only 22% (64/296) of cases contained a diagnostic code. Comparison of the frequency of the nine index categories in the validated alveolar osteitis cases between the student clinic and the population showed the following results: severe pain: 94% (279/296) vs 100% (430/430); bare bone/missing blood clot: 27% (80/296) vs 74% (35/47) to 100% (329/329); malodor: 7% (22/296) vs 33%-50% (18/54); radiating pain to the ear: 8% (24/296) vs 56% (30/54); lymphadenopathy: 1% (3/296) vs 9% (5/54); inflammation: 14% (42/296) vs 50% (27/54); debris: 12% (36/296) vs 87% (47/54); alveolar osteitis site noted: 96% (283/296) vs 100% (430/430; accepted documentation requirement); and anesthesia during debridement: 77% (20/24) vs 100% (430/430; standard of anesthetization prior to debridement). Conclusions There was a significant discrepancy between the index category frequency in alveolar osteitis cases documented by dental students and in the population (reported in peer-reviewed literature). More attention to clinical documentation skills is warranted in dental student training.

show abstract

“…[13] A study done in Korea in 2014 to assess the completeness of medical records, only 27.4% of the records were found to be completely documented especially for items related to the patient's symptoms and examination . [14] Higher figures were revealed in our study especially in the surgical, medical and Gynecology/Obstetric departments. A big defect was found in documentation completeness level for the Progress/follow up sheets mainly in the Gynecology/Obstetric words (93%) followed by the surgical word (75%).…”

Section: Discussionmentioning

confidence: 46%

Assessment of the documentation completeness level of the medical records in Basrah General Hospital

Hussein¹,

Majeed²,

Mahmoud³

et al. 2018

MJBU

View full text Add to dashboard Cite

Background: Medical records documentation is an important legal and professional requirement for all health professionals. They include information which describes all aspects of patient's care. But, despite the importance of medical records to support better quality service provided at the health facilities, incomplete documentation is very common all over the world. Objective of the study: to assess the documentation completeness level of the medical records in the different inpatient wards of Basrah General Hospital. Methodology: The study was a descriptive cross-sectional one. Medical records of 268 inpatients from Basrah General Hospital during June 2016 were included from four departments of the hospital (medicine, surgery, pediatrics and obstetrics and gynecology). A standard Iraqi Ministry of Health inpatient medical record with a two-level scoring system for assessing the level of documentation completeness were used in the study. Results: the overall documentation level for the medical records included in the study was generally poor in 78% of the records. Surgical department was found to be the worse in documenting patient's notes related to medical history, while Gynecology and Obstetrics department was found to be the worst in documenting the medical examination assessment and the physician's notes related to the patient's state and details of any improvement / deterioration of his/her condition. Conclusions and recommendations: The present study confirmed obvious incompleteness of documenting medical data for inpatient records in Basrah General Hospital especially in the general surgery, internal medicine and Gynecology and Obstetrics words. This is specifically found for the Physician notes (patient's state and details of any improvement/deterioration of the condition) and the Clinical pharmaceutical sheet. A hospital based quality improvement project to improve the medical record documentation completion is highly recommended to be implemented by the Quality Assurance Unit of Basrah Directorate of Health.

show abstract

A pilot study on the evaluation of medical student documentation: assessment of SOAP notes

Cited by 20 publications

References 5 publications

Predicting COVID-19 Symptoms From Free Text in Medical Records Using Artificial Intelligence: Feasibility Study

Predicting COVID-19 Symptoms From Free Text in Medical Records Using Artificial Intelligence: Feasibility Study

Completeness of Electronic Dental Records in a Student Clinic: Retrospective Analysis

Assessment of the documentation completeness level of the medical records in Basrah General Hospital

Contact Info

Product

Resources

About