The Billion Prices Project: Using Online Prices for Measurement and Research

Abstract

A large and growing share of retail prices all over the world are posted online on the websites of retailers. This is a massive and (until recently) untapped source of retail price information. Our objective with the Billion Prices Project, created at MIT in 2008, is to experiment with these new sources of information to improve the computation of traditional economic indicators, starting with the Consumer Price Index. We also seek to understand whether online prices have distinct dynamics, their advantages and disadvantages, and whether they can serve as reliable source of information for economic research. The word “billion” in Billion Prices Project was simply meant to express our desire to collect a massive amount of prices, though we in fact reached that number of observations in less than two years. By 2010, we were collecting 5 million prices every day from over 300 retailers in 50 countries. We describe the methodology used to compute online price indexes and show how they co-move with consumer price indexes in most countries. We also use our price data to study price stickiness, and to investigate the “law of one price” in international economics. Finally we describe how the Billion Prices Project data are publicly shared and discuss why data collection is an important endeavor that macro- and international economists should pursue more often.

Statistical Challenges of Administrative and Transaction Data

“…A variant of this uses Web‐scraped price collection, being explored by various national statistical institutes. For example, the billion prices project (Cavallo and Rigobon, ) seeks to collect massive amounts of price data from Web sites. Apart from its vast coverage (‘big data’) this means that much more timely estimates can be obtained much more cheaply than by traditional methods.…”

“…Apart from its vast coverage (‘big data’) this means that much more timely estimates can be obtained much more cheaply than by traditional methods. Note, however, that Cavallo and Rigobon () did not describe this as an alternative to traditional methods, but as a complement.…”

Forty Years of Price Transmission Research in the Food Industry: Insights, Challenges and Prospects

“…Scanner data also permit the analysis of prices recorded on‐line and at the individual store level offering a wealth of new information. The Billion Prices Project at the Massachusetts Institute of Technology began recording scanner data in a number of countries in 2008 and surpassed its ‘billion’ label in under two years (see Cavallo and Rigobon, ).…”

(Un)beliveable wages? An analysis of minimum wage policies in Europe from a living wage perspective

“…The lack of official data sources is partly filled by several online tools. Perhaps the most famous of these tools is the MIT Billion Prices Project that collects online prices online to estimate price indices in 50 countries since 2008, which have achieved remarkable similarity with the official statistics (Cavallo and Rigobon 2016;Cavallo 2017). Nonetheless, this dataset is limited to supermarket prices and as such cannot be used to estimate LW, which necessarily contains items like housing, which are not traded in supermarkets.…”