Proxying for Unobservable Variables With Internet Document-Frequency

Abstract

The internet contains billions of documents. We show that document frequencies in large decentralized textual databases can capture the cross-sectional variation in the occurrence frequencies of social phenomena. We characterize the econometric conditions under which such proxying is likely. We also propose using recently-introduced internet search volume indexes as proxies for fundamental locational traits, and discuss their advantages and limitations. We then successfully proxy for a number of economic and demographic variables in US cities and states. We further obtain document-frequency measures of corruption by country and US state and replicate the econometric results of previous research studying its covariates. Finally, we provide the first measure of corruption in American cities. Poverty, population size, service-sector orientation, and ethnic fragmentation are shown to predict higher levels of corruption in urban America. (JEL: H00, J11, C81, B40, D73)

Market-Based Lobbying: Evidence from Advertising Spending in Italy

“…(Our interpretation would still account for this shift) In Columns 1 and 2 of Table 9 we replicate the baseline time-series and diff-in-diff regressions in a weighted OLS regression, with the addition of firm fixed effects. 26 The results are nearly identical to the ones in the benchmark specifications, indicating that the results are not due to a compositional shift.…”

“…A more important role is played by weighting by total advertising spending, since the results are larger for sectors with higher spending on television advertising. We also document that the results are similar when using an alternative measure of regulation based upon the occurrence of internet content of industry names together with words indicating regulation, as developed in Saiz and Simonsohn (2013). 4 We also test for a dynamic version of the quid-pro-quo.…”

“…We now analyze whether the shift in advertising revenue 26 The weights are given by the advertising expenditure for that firm in that quarter. The firm-level panel, unlike the sector-level results, does not include all firms, just the 810 largest firms (see the Data section).…”

“…Low on the list instead are industries such as housing, alcohol, food, leisure, apparel, and personal items. 15 As an alternative measure of regulation, we capture dispersed information on the internet using the methodology of Saiz and Simonsohn (2013). Namely, we count the number of search results containing the industry name and a word indicating regulation, and divide by the number of search results for the industry name.…”

Isolated Capital Cities, Accountability, and Corruption: Evidence from US States

“…First, we follow Saiz and Simonsohn (2013) in building a measure from an online search, using the Exalead tool, for the term "corruption" close to the name of each state (performed in 2009).…”