Structural Clustering of Machine-Generated Mail

Avigdor-Elgrabli, Noa; Cwalinski, Mark; Castro, Dotan Di; Gamzu, Iftah; Grabovitch-Zuyev, Irena; Lewin-Eytan, Liane; Maarek, Yoelle

doi:10.1145/2983323.2983350

Cited by 12 publications

(4 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It has been widely used in information extraction over structured web pages [10,25]. For emails, multiple algorithms for template induction have been described [5,11,21] along with applications in information extraction [3,20,31], email threading [5], and hierarchical classification [37]. This paper uses techniques previously described in the literature [31], but is the first description of an online template induction system.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Online template induction for machine-generated emails

et al. 2019

View full text Add to dashboard Cite

In emails, information abounds. Whether it be a bill reminder, a hotel confirmation, or a shipping notification, our emails contain useful bits of information that enable a number of applications. Most of this email traffic is machine-generated, sent from a business to a human. These business-to-consumer emails are typically instantiated from a set of email templates, and discovering these templates is a key step in enabling a variety of intelligent experiences. Existing email information extraction systems typically separate information extraction into two steps: an offline template discovery process (called template induction) that is periodically run on a sample of emails, and an online email annotation process that applies discovered templates to emails as they arrive. Since information extraction requires an email's template to be known, any delay in discovering a newly created template causes missed extractions, lowering the overall extraction coverage. In this paper, we present a novel system called Crusher that discovers templates completely online, reducing template discovery delay from a week (for the existing MapReduce-based batch system) to minutes. Furthermore, Crusher has a resource consumption footprint that is significantly smaller than the existing batch system. We also report on the surprising lesson we learned that conventional stream processing systems do not present a good framework on which to build Crusher. Crusher delivers an order of magnitude more throughput than a prototype built using a stream processing engine. We hope that these lessons help designers of stream processing systems accommodate a broader range of applications like online template induction in the future.

show abstract

Section: Related Workmentioning

confidence: 99%

“…Rather, 90% of email traffic consists of machine-generated business-to-consumer (B2C) emails instantiated from a collection of a few million email templates, as illustrated in Figure 1. Existing email mining systems take advantage of this templatic nature of emails to extract information from emails using a two-step process [11,20,21,31].…”

Section: Introductionmentioning

confidence: 99%

Online template induction for machine-generated emails

et al. 2019

View full text Add to dashboard Cite

show abstract

“…It has been widely used in information extraction over structured web pages [3,23]. For emails, multiple algorithms for template induction have been described [2,4] along with applications like email threading [2] and hierarchical classification [44]. A technique has also been suggested for plain text emails [35] where data is not explicitly structured.…”

Section: Related Workmentioning

confidence: 99%

“…Unlike the general web, which is predominantly public, emails are private. Thus, to date, relatively little research has been presented on information extraction over this domain [4,46]. Learning and iterating is a much more difficult task when privacy must be preserved [13].…”

mentioning

confidence: 99%

Anatomy of a Privacy-Safe Large-Scale Information Extraction System Over Email

Sheng

Tata

Wendt

et al. 2018

Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery &Amp; Data Mining

View full text Add to dashboard Cite

Extracting structured data from emails can enable several assistive experiences, such as reminding the user when a bill payment is due, answering queries about the departure time of a booked flight, or proactively surfacing an emailed discount coupon while the user is at that store. This paper presents Juicer, a system for extracting information from email that is serving over a billion Gmail users daily. We describe how the design of the system was informed by three key principles: scaling to a planet-wide email service, isolating the complexity to provide a simple experience for the developer, and safeguarding the privacy of users (our team and the developers we support are not allowed to view any single email). We describe the design tradeoffs made in building this system, the challenges faced and the approaches used to tackle them. We present case studies of three extraction tasks implemented on this platform-bill reminders, commercial offers, and hotel reservations-to illustrate the effectiveness of the platform despite challenges unique to each task. Finally, we outline several areas of ongoing research in largescale machine-learned information extraction from email.

show abstract

Template Trees: Extracting Actionable Information from Machine Generated Emails

Agarwal

Singh

2018

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Structural Clustering of Machine-Generated Mail

Cited by 12 publications

References 37 publications

Online template induction for machine-generated emails

Online template induction for machine-generated emails

Anatomy of a Privacy-Safe Large-Scale Information Extraction System Over Email

Template Trees: Extracting Actionable Information from Machine Generated Emails

Contact Info

Product

Resources

About