Address standardization with latent semantic association

Abstract

Address standardization is a very challenging task in data cleansing. To provide better customer relationship management and business intelligence for customer-oriented cooperates, millions of free-text addresses need to be converted to a standard format for data integration, de-duplication and householding. Existing commercial tools usually employ lots of hand-craft, domain-specific rules and reference data dictionary of cities, states etc. These rules work better for the region they are designed. However, rule-based methods usually require more human efforts to rewrite these rules for each new domain since address data are very irregular and varied with countries and regions. Supervised learning methods usually are more adaptable than rule-based approaches. However, supervised methods need large-scale labeled training data. It is a labor-intensive and time-consuming task to build a large-scale annotated corpus for each target domain. For minimizing human efforts and the size of labeled training data set, we present a free-text address standardization method with latent semantic association (LaSA). LaSA model is constructed to capture latent semantic association among words from the unlabeled corpus. The original term space of the target domain is projected to a concept space using LaSA model at first, then the address standardization model is active learned from LaSA features and informative samples. The proposed method effectively captures the data distribution of the domain. Experimental results on large-scale English and Chinese corpus show that the proposed method significantly enhances the performance of standardization with less efforts and training data.

A classification of data quality assessment and improvement methods

“…For instance in (Kothari et al, 2010) a new address standardization technique for countries such as India is shown which is able to deal with substantial variants in address structures within a country. Also for addresses, (Guo et al, 2009) proposes latent semantic association instead of the traditional rule and dictionary-based approaches for data standardization. The data standardization step is typically applied before matching and data consolidation algorithms are applied to match records with the intent to find duplicates.…”

DeepAM: Deep Semantic Address Representation for Address Matching

“…-Company. We also crawl addresses on two food review websites 2 3 and one company information query website 4 . This database contains 10k company addresses.…”

“…The process of translating manually written addresses into a certain digital format is known as address standardization. There are also some researches on Address Standardization [4,6,8]. A method based on trie-tree and finite state machine is proposed in [10] which focuses on the problem of inaccurate word segmentation.…”

Towards the automation of address identification

“…The aim of this model is to extract hidden (unknown) information from a string of visible parameters. Particularly novel is the work of Guo et al (2009), which analyzes postal addresses using a model of Latent Semantic Association (LaSA). LaSA model is built to minimize the human efforts and the size of the control data.…”

“…Some tests to solve the problem of normalization of addresses were done years ago, but the greatest difficulty was the necessary computing power, not very developed at that time (Fernández et al, 1993). Current processors overcome this difficulty and, besides, new studies emerge every day analyzing the feasibility of different algorithms for data management (Navarro et al, 2003;Patman and Thompson, 2003;Christen and Belacic, 2005;Guo et al, 2009). Although these studies are not applied to Bibliometrics, they employ different techniques that can be used for present and future improvements.…”