Attribute classification using feature analysis

Abstract

The basis of many systems that integrate data from multiple sources is a set of correspondences between source schemata and a target schema. Correspondences express a relationship between sets of source attributes, possibly from multiple sources, and a set of target attributes. Clio is an integration tool that assists users in dening value correspondences between attributes [1].In real life scenarios there may be many sources and the source relations may have many attributes. Users can get lost and might miss or be unable to nd some correspondences. Also, in many real life schemata the attribute names reveal little or nothing about the semantics of the data values. Only the data values in the attribute columns can convey the semantic meaning of the attribute. Our work relieves users of the problems of too many attributes and meaningless attribute names, by automatically suggesting correspondences between source and target attributes. For each attribute, we analyze the data values and derive a set of features. The overall feature set forms the characteristic signature of an attribute. There are more likely to be correspondences between attributes with similar signatures than between others. Our results show that a properly chosen small set of domain-independent features can mostly capture structural information of attributes. We describe the problem of nding corresponding attributes as a classication problem, and use the Naïve Bayes classier to decide upon most likely matches. Several dierent domains served to test the features and Naïve Bayes classier: a set of three bibliography databases, data from three real estate Web sites, a commercial semiconductor manufacturers database collecting, and an insurance database. The experiments showed an expected strong dependency between column size and accuracy. For all domains we observed satisfying misclassication rates below 5% for > 250 training data and > 16 test data values. Numerical attributes. For numerical attributes we use a non-Boolean feature set: To best model the range and distribution of the values of an attribute, we choose 18 quantile features for our implementationthe 10%, 20%, to 90% quantiles of each data column, and the 10%, 20%, to 90% quantiles of each data column after removing all data of value zero.To decide upon most likely correspondences, we introduce a new quantile-based classication method. As before, we generate one signature for each numerical attribute from the training column. We then generate the signature for the numerical test attribute to be classied. We evaluate the quantile-based classication methods by applying them to two real world databases and a synthetic data set. The rst database is a collection of numerical attributes taken from a large biochemical database; the second is an insurance database. Finally, we generated several data columns with synthetic data and only slightly diering distributions. We achieved on average misclassication rates below 5% for > 64 training and test data values. Summary. The techniques described a...

Validating Multi-column Schema Matchings by Type

“…In their scheme, the data is effectively treated as categorical. Other related works in this area include the work of He, Chang and Han [11] on schema matching for the deep web and work on using distributional signatures for value mapping by Kang et al [12] and Naumann et al [13]. For related work in the Al community, we refer the reader to the survey by Doan and Halevy [2].…”

Matching large schemas: Approaches and evaluation

“…Previous generic tools include Cupid, SimilarityFlooding, and Clio [34][35][36]. As discussed, COMA++ is also generic and supports several schema languages, including XSD, OWL, and relational schemas, and it is able to deal with complex distributed XML schemas.…”

Profiling relational data: a survey

“…Many schema matching systems perform data profiling to create attribute features, such as data type, average value length, and patterns, to compare feature vectors and align those attributes with the best matching ones [98,109].…”