Probabilistic master lists: integration of patient records from different databases when unique patient identifier is missing

Abstract

We show how Bayesian probability models can be used to integrate two databases, one of which does not have a key for uniquely identifying clients (e.g., social security number or medical record number). The analyst selects a set of imperfect identifiers (last visit diagnosis, first name, etc.). The algorithm assesses the likelihood ratio associated with the identifier from the database of known cases. It estimates the probability that two records belong to the same client from the likelihood ratios. As it proceeds in examining various identifiers, it accounts for inter-dependencies among them by allowing overlapping and redundant identifiers to be used. We test that the procedure is effective by examining data from the Medical Expenditure Panel Survey (MEPS) Population Characteristics data set, a publicly available data set. We randomly selected 1,000 cases for training data set--these constituted the known cases. The algorithm was used to identify if 100 cases not in the training data set would be misclassified in terms of being a case in the training set or a new case. With 12 fields as identifiers, all 100 cases were correctly classified as new cases. We also selected 100 known cases from the training set and asked the algorithm to classify these cases. Again, all 100 cases were correctly classified. Less accurate results were obtained when the training data set was too small (e.g., less than 100 records) or the number of fields used as identifiers was too small (e.g., less than seven fields). In a test of performance of the algorithm, when the ratio of testing to training data set exceeds 4 to 1, the accuracy of the algorithm exceeded 90% of cases. As the ratio increases, the accuracy of algorithm improves further. These data suggest the accuracy of our automated and mathematical procedure to merge data from two different data sets without the presence of a unique identifier. The algorithm uses imperfect and overlapping clues to re-identify cases from information not typically considered to be a patient identifier.

Facilitating Cancer Epidemiologic Efforts in Cleveland via Creation of Longitudinal De-Duplicated Patient Data Sets

“…For example, different EHR systems may have different date formats for patient's DOB; (iii) the accuracy of the matching algorithm: What is the possibility of the algorithm returning a false-positive match or missing true matches? There has been some specific matching algorithms to match patients (15,16); however, the patient data that we have to compare is binary data to which these algorithms cannot be applied. We believe that DeDeRE provides a novel approach for working with this type of patient data.…”

Gestion décentralisée des documents médicaux des patients. Un système de recherche et d'accès aux données

“…Une littérature abondante entoure ce sujet (Cooper et al, 2007 ;Alemi et al, 2007 ;Ceusters et al, 2005 ;Netter et al, 2003 ;Mays et al, 2002 ;Kruse et al, 2001 ;Omar Bagayoko et al, 2008 ;Quantin et al, 2007a ;Quantin et al, 2007b ;Quantin et al, 2007c), que ce soit au niveau national ou des différents pays de la communauté européenne pour ne limiter notre sujet qu'à ce territoire. La question principale est en fait de déterminer à quelle échelle il est nécessaire de disposer d'une identification unique du patient, au niveau régional (comme en Italie ou en Espagne), national (comme aux Pays-Bas, au Royaume-Uni et en France) ou au niveau de l'Europe ?…”

“…Dans la perspective d'une interopérabilité des systèmes d'information et donc des identifiants au niveau international, comme il est notamment envisagé dans le cadre de l'Europe (Cooper et al, 2007 ;Alemi et al, 2007 ;Ceusters et al, 2005 ;Allaert et al, 2003 ;Quantin et al, 2007d), l'utilisation d'un numéro d'identifiant à valeur régionale ou nationale paraît à proscrire. La biométrie est parfois proposée comme un moyen de lier le patient à ses propres données médicales, car elle présente l'avantage de fournir un numéro d'identification très spécifique du patient et intégrable dans tous les systèmes d'information, quel que soit le pays.…”