Secure pseudonymisation for privacy-preserving probabilistic record linkage

Abstract

Record linkage is becoming an increasingly important tool in many areas of research -particularly medical research, where the relevant data often reside in more than one location. In the absence of a reliable and unique identifier probabilistic approaches to linkage are often employed. This linkage generally exploits the information contained in the fields that are common to a record pair. In classical record linkage the values in common fields are simply compared for equality. As values might contain typographical (or other) errors the performance of classical record linkage can often be significantly improved if similarities between value pairs are also exploited.In applications where the data used for matching must be kept private the raw values are replaced by pseudonyms. For better linkage performance these pseudonyms should also convey information regarding similarities. Existing approaches are often based on Bloom filters, yet these are susceptible to attack. Secure schemes based on Bloom filters inevitably involve additional security measures. Here we introduce a new scheme that produces pseudonyms that are far more secure than Bloom filters. It can be used a drop-in replacement for many schemes that use Bloom filters. The new scheme allows similarity scores to be estimated from pairs of pseudonyms with negligible bias and with known variance for a given similarity score.©

Secure and Accurate Two-Step Hash Encoding for Privacy-Preserving Record Linkage

“…We show that our encoding technique is significantly more secure than popular Bloom filter encoding [8,13], which is known to be vulnerable to cryptanalysis attacks [2,6,14]. We evaluate our technique on large real databases, which validates that, compared to existing PPRL encodings [8,11], this technique can efficiently and accurately calculate similarities between encoded values while providing privacy against cryptanalysis attacks.…”

“…However, as we discuss in Sect. 5, this tabulation hash encoding approach [11] has significantly higher computational complexity compared to BF encoding. As a result, for encoding very large databases the practical use of this approach is questionable.…”

“…Smith [11] recently proposed a tabulation Min-hash based approach which showed both improved similarity and privacy protection on small data sets. To generate a bit vector of length l, first l hash tables each containing random 64bit strings are generated.…”

“…Following earlier BF work in PPRL, we set the BF parameters as l =1000 bits, k = 30, and q = 2 [8,13]. The second baseline is the tabulation hash (TH) encoding proposed by Smith [11]. Following [11], we used 8 tabulation keys each of 64 bits length to generate one bit array of length l = 1000 bits to encode the attribute values in a record.…”

“…2, we propose a novel efficient encoding technique that can be used to securely encode sensitive values while facilitating accurate similarity calculations between encoded values. Unlike existing encoding techniques [8,11], the aim of our technique is to provide privacy guarantees against recently proposed re-identification attacks [2,14] while at the same time allowing the accurate and efficient calculation of similarities between encoded values.…”