An automated parsing routine was written for extracting the »site«, »diagnostic«, and »modifier« components of the diagnostic statements of the diagnostic summary of surgical pathology reports. Such parsed reports appear to be suitable for input into an information retrieval system for the surgical pathology reports.Data was input through a key-to-tape device producing a computer compatible magnetic tape with a record size of 870 bytes. The statements were parsed through syntactic and morphological analysis utilizing the common prepositions, the common punctuations and the morphemal constructions common in medical terms. (A total of sixty-two delimiters were used). Certain suffix transformations were performed, converting some »site« adjective to »site« nouns, and some »diagnostic« nouns to »site« nouns. 1,108 diagnostic statements were processed with an error rate of 9.3% for the latest version on the last 493 statements.
The errors studied are misspellings and typographical errors made by the physician house staff, surgical pathologists, and secretary/typists of a large teaching hospital. The 6,019 errors studies were encountered in the compilation of a LEXICON now containing 24,135 medical and non-medical terms (including errors) from Tissue Examination Request Forms and Surgical Pathology Reports. An automated error correction algorithm was sought to reduce the tedious task of manual encoding of errors, and eliminate the need for storing errors occupying 24.9% of the LEXICON storage space. The errors were classified into 23 types, and it was found that 84.2% of the errors were in the 11 first order categories.Existing error correction algorithms were analyzed with respect to possible application to our medical sample. Two were selected for experimentation, the Baskin-Selfridge algorithm and SOUNDEX. Results showed that Baskin-Selfridge worked quite well, but was too slow to be applied singularly. SOUNDEX was reasonable in speed, but had too many mismatches to be applied singularly in a non-interactive application. SOUNDEX was modified phonologically and with respect to code length in various ways and some experimental data showed improvements.The optimal design for the medical LEXICON sample appears to be a two-step process. The modified version of SOUNDEX will quickly select the most likely corrections for the error (experimental average is 2.38 choices/error). Then the Baskin-Selfridge will decide which, if any, is the actual correct form of the error. By only considering a very small number of choices, the time required for the Baskin-Selfridge algorithm becomes trivial.On the basis of experimental results, it is estimated that this combination will reduce manual encoding of errors by 60—70% and reduce the storage required for the LEXICON by approximately 15%.
With the objective of providing easier access to pathology specimens, slides and kodachromes with linkage to x-ray and the remainder of the patient’s medical records, an automated natural language parsing routine, based on dictionary look-up, was written for Surgical Pathology document-pairs, each consisting of a Request for Examination (authored by clinicians) and its corresponding report (authored by pathologists). These documents were input to the system in free-text English without manual editing or coding.Two types of indices were prepared. The first was an »inverted« file, available for on-line retrieval, for display of the content of the document-pairs, frequency counts of cases or listing of cases in table format. Retrievable items are patient’s and specimen’s identification data, date of operation, name of clinician and pathologist, etc. The English content of the operative procedure, clinical findings and pathologic diagnoses can be retrieved through logical combination of key words. The second type of index was a catalog. Three catalog files — »operation«, »clinical«, and »pathology« — were prepared by alphabetization of lines formed by the rotation of phrases, headed by keywords. These keywords were automatically selected and standardized by the parsing routine and the phrases were extracted from each sentence of each input document. Over 2,500 document-pairs have been entered and are currently being utilized for purpose of medical education.
A free text data collection system has been developed at the University of Illinois utilizing single word, syntax free dictionary lookup to process data for retrieval. The source document for the system is the Surgical Pathology Request and Report form. To date 12,653 documents have been entered into the system.The free text data was used to create an IRS (Information Retrieval System) database. A program to interrogate this database has been developed to numerically coded operative procedures. A total of 16,519 procedures records were generated. One and nine tenths percent of the procedures could not be fitted into any procedures category; 6.1% could not be specifically coded, while 92% were coded into specific categories. A system of PL/1 programs has been developed to facilitate manual editing of these records, which can be performed in a reasonable length of time (1 week). This manual check reveals that these 92% were coded with precision = 0.931 and recall = 0.924. Correction of the readily correctable errors could improve these figures to precision = 0.977 and recall = 0.987. Syntax errors were relatively unimportant in the overall coding process, but did introduce significant error in some categories, such as when right-left-bilateral distinction was attempted.The coded file that has been constructed will be used as an input file to a gynecological disease/PAP smear correlation system. The outputs of this system will include retrospective information on the natural history of selected diseases and a patient log providing information to the clinician on patient follow-up.Thus a free text data collection system can be utilized to produce numerically coded files of reasonable accuracy. Further, these files can be used as a source of useful information both for the clinician and for the medical researcher.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
customersupport@researchsolutions.com
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.