“…With merely thousands of consumers per experiment, these 600 experiments individually lack sta-tistical power-the majority were statistically insignificant-but collectively demonstrate a significant ad effect in a meta-study (Lodish et al, 1995;Hu et al, 2007). Kalyanam et al (2015) demonstrate that product-category search advertising increases offline retail sales using a meta-study of 15 experiments and 13 retailers. Kalyanam et al (2015) vary advertising at the DMA level then compare sales at treated stores to similar counterparts.…”
Section: Literature Reviewmentioning
confidence: 94%
“…Kalyanam et al (2015) demonstrate that product-category search advertising increases offline retail sales using a meta-study of 15 experiments and 13 retailers. Kalyanam et al (2015) vary advertising at the DMA level then compare sales at treated stores to similar counterparts. Again, these studies are collectively significant though only half are individually significant.…”
Section: Literature Reviewmentioning
confidence: 97%
“…We are interested in measuring the effect on online and in-store sales, thoug this is a harder estimation problem, because this allows us to evaluate the short-run return on investment for the campaign. Another solution to the power problem is to study settings with large ad effects like in online search where users are often seeking out competing advertisers with an intent to purchase (Sahni, 2015a;Kalyanam et al, 2015), though not if the users already have an advertiser in mind (Blake et al, 2015). Instead, we are interested in the effects of image ads on users who receive ads while browsing unrelated content; our efforts to increase power are therefore crucial.…”
Yahoo! Research partnered with a nationwide retailer to study the effects of online display advertising on both online and in-store purchases. We use a randomized field experiment on 3 million Yahoo! users who are also past customers of the retailer. We find statistically significant evidence that the retailer ads increase sales 3.6% relative to the control group. We show that control ads boost measurement precision by identifying and removing the half of in-campaign sales data that is unaffected by the ads. Less data gives us 31% more precision in our estimates-equivalent to increasing our sample to 5.3 million users. By contrast, we only improve precision by 5% when we include additional covariate data to reduce the residual variance in our experimental regression. The covariate-adjustment strategy disappoints despite exceptional consumer-level data including demographics, ad exposure levels, and two years' worth of past purchase history.
“…With merely thousands of consumers per experiment, these 600 experiments individually lack sta-tistical power-the majority were statistically insignificant-but collectively demonstrate a significant ad effect in a meta-study (Lodish et al, 1995;Hu et al, 2007). Kalyanam et al (2015) demonstrate that product-category search advertising increases offline retail sales using a meta-study of 15 experiments and 13 retailers. Kalyanam et al (2015) vary advertising at the DMA level then compare sales at treated stores to similar counterparts.…”
Section: Literature Reviewmentioning
confidence: 94%
“…Kalyanam et al (2015) demonstrate that product-category search advertising increases offline retail sales using a meta-study of 15 experiments and 13 retailers. Kalyanam et al (2015) vary advertising at the DMA level then compare sales at treated stores to similar counterparts. Again, these studies are collectively significant though only half are individually significant.…”
Section: Literature Reviewmentioning
confidence: 97%
“…We are interested in measuring the effect on online and in-store sales, thoug this is a harder estimation problem, because this allows us to evaluate the short-run return on investment for the campaign. Another solution to the power problem is to study settings with large ad effects like in online search where users are often seeking out competing advertisers with an intent to purchase (Sahni, 2015a;Kalyanam et al, 2015), though not if the users already have an advertiser in mind (Blake et al, 2015). Instead, we are interested in the effects of image ads on users who receive ads while browsing unrelated content; our efforts to increase power are therefore crucial.…”
Yahoo! Research partnered with a nationwide retailer to study the effects of online display advertising on both online and in-store purchases. We use a randomized field experiment on 3 million Yahoo! users who are also past customers of the retailer. We find statistically significant evidence that the retailer ads increase sales 3.6% relative to the control group. We show that control ads boost measurement precision by identifying and removing the half of in-campaign sales data that is unaffected by the ads. Less data gives us 31% more precision in our estimates-equivalent to increasing our sample to 5.3 million users. By contrast, we only improve precision by 5% when we include additional covariate data to reduce the residual variance in our experimental regression. The covariate-adjustment strategy disappoints despite exceptional consumer-level data including demographics, ad exposure levels, and two years' worth of past purchase history.
“…They apply a difference-in-difference (DID) model and show a significant increase in offline purchase with display advertising. Kalyanam et al (2018) also conduct a large-scale field experiment on Google.com and find a positive cross-channel effect of online marketing on the offline retail in-store purchase.…”
Use a unique dataset collected from a large classified ads website, we empirically examine the effect of the offline call intensity on the online consumer purchase probability of digital services and the carryover effect of the call intensity. We find that the online consumer purchase probability is increasing in the call intensity but at a decreasing rate. We further demonstrate that the decreasing rate is sizable enough that the relationship between the online consumer purchase probability and the call intensity is an inverted U‐shaped curve. In addition, there exists a strong carryover effect where the online call intensity in the past 4 weeks does not fade away but has a positive effect on recent consumer purchases. Our estimations show that both too much and too little call intensity will result in considerably worse outcomes. As compared with the call intensity at the optimal level, too much call intensity potentially reduces the online consumer purchase probability by 29.11%, and too little call intensity potentially reduces the online consumer purchase probability by 54.90%. Furthermore, making calls every week for 4 weeks can increase consumer purchase probability by 10.69 times as compared with just initiating calls with the consumers.
“…Sometimes the unit of randomization is at the geographic market level, rather than the individual consumer. In these cases, researchers have often relied on quasi-experiments or matched market tests to generate suitable treatment and control groups (Vaver and Koehler, 2011;Kalyanam et al 2017).…”
Section: Experimental Studies Of Advertising Effectsmentioning
Digital advertising markets are growing and attracting increased scrutiny. This paper explores four market inefficiencies that remain poorly understood: ad effect measurement, frictions between and within advertising channel members, ad blocking and ad fraud. These topics are not unique to digital advertising, but each manifests in new ways in markets for digital ads. We identify relevant findings in the academic literature, recent developments in practice, and promising topics for future research. * Corresponding authors: zskatona@berkeley.edu , kcwilbur@ucsd.edu 1 Digital advertising markets have offered unprecedented innovations to marketers. Businesses can now advertise to finely targeted sets of individuals with customized commercial messages at specific locations and times in a variety of formats. Compared to traditional advertising, digital ads promise better targeting and relevance, personalized ad content, programmatic sales based on real-time auctions, and measurement of the co-occurrence of individual consumer ad exposures with a variety of online and offline response behaviors. These features have fundamentally altered marketers' spending: digital advertising revenues reached $108B in 2018, up 117% over 2014 (IAB 2019), with expectations to grow 19% and surpass cumulative traditional advertising revenues in 2019 (eMarketer 2019a). Digital advertising markets sell a wide variety of search and display advertising opportunities to marketers. Although digital advertising is 25 years old, market structures are still changing rapidly. For example, a census of marketing technology firms showed an increase from 150 in 2011 to 7,040 in 2019 (Brinker 2019). The IAB Tech Lab recently introduced a series of broad-based initiatives, including a new real-time bidding standard, and Google moved from second-price to first-price auctions for display ads. Publishers have introduced a variety of ad formats, with spending typically following consumer attention and media usage: after initial growth in desktop display and search, recent growth has been more concentrated in social networking, video, audio, and mobile ads.Yet there are indicators that unregulated markets for digital advertising have experienced problems. The E.U. has fined Google more than $9 billion in three antitrust cases and the U.S.Federal Trade Commission fined Facebook $5 billion after it broke a 2012 consent order (Case 19-cv-2184). Prominent politicians have criticized the industry and proposed structural reforms.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.