Analyzing PACS Usage Patterns by Means of Process Mining: Steps Toward a More Detailed Workflow Analysis in Radiology

Abstract

In this paper, statistical analysis and techniques from process mining are employed to analyze interaction patterns originating from radiologists reading medical images in a picture archiving and communication system (PACS). Event logs from 1 week of data, corresponding to 567 cases of single-view chest radiographs read by 14 radiologists, were analyzed. Statistical analysis showed that the numbers of commands and command types used by the radiologists per case only have a slightly positive correlation with the time to read a case (0.31 and 0.55, respectively). Further, one way ANOVA showed that the factors time of day, radiologist and specialty were significant for the number of commands per case, whereas radiologist was also significant for the number of command types, but with no significance of any of the factors on time to read. Applying process mining to the event logs of all users showed that a seemingly "simple" examination (single-view chest radiographs) can be associated with a highly complex interaction process. However, repeating the process discovery on each individual radiologist revealed that the initially discovered complex interaction process consists of one group of radiologists with individually well-structured interaction processes and a second smaller group of users with progressively more complex usage patterns. Future research will focus on metrics to describe derived interaction processes in order to investigate if one set of interaction patterns can be considered as more efficient than another set when reading radiological images in a PACS.

Factors Affecting Radiologist’s PACS Usage

“…For ease of statistical analysis, PACS usage is quantified by four high-level metrics: number of commands, number of command classes, bigram repetitiveness, and time to read. Based upon earlier results [12], we hypothesize that the factors radiologist and examination category are the only two factors that have a significant effect on PACS usage as described by all four metrics and that the interaction effect between these two factors would also significantly affect PACS usage.…”

“…Extended analysis is required to assess whether these differences are due to preference or dependent on, for example, differences in the actual case load. Based upon previous work related to analysis of PACS usage as recorded by command logs [12], a number of precautions were taken in order to ensure a high quality of the analyzed data. These include, for example, the filtering of command traces to only include those where it remains highly likely that only a single radiologist has interpreted and dictated the corresponding examinations.…”

“…Although discovering underlying usage patterns from PACS usage logs has been successfully implemented earlier using process mining [12], we did not include it here since process mining currently lacks standards for comparison of different usage patterns when described by separate process models. However, we do intend to revisit these data to compare differences in derived process models, once standardized methods and metrics for this purpose have been defined, with some initial suggestions already available [18][19][20].…”

“…For TimeOfWeek, the timestamp of the first command in the command trace was used to determine time of week. Four categorical values were used, daytime/ nighttime weekday/weekend, same as in [12]. The variable Week was simply set to the week number of the recorded data but in groups of two, i.e., 1-2, 3-4, 5-6, and 7-8.…”

“…For example, studies have evaluated the effect of different input devices on radiologists' efficiency [1][2][3], impact of employed display devices [4], or general ergonomic and environmental aspects of the reading context [5][6][7][8]. A very limited set of research studies [9][10][11][12] have focused directly on PACS usage itself.…”