Developing Systems for Real-Time Streaming Analysis

Michalak, Sarah; DuBois, Andrew; DuBois, D. F.; Wiel, Scott Vander; Hogden, John

doi:10.1080/10618600.2012.657144

Cited by 12 publications

(8 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Effectively, all the prior data, D t−1 are summarized in θ t . This choice means that the computations are bounded by the dimension of θ and the time required to update θ instead of growing as a function of t. Note that this effectively forces users to implement an online policy (Michalak, DuBois, DuBois, Wiel, and Hogden 2012) as the complete dataset D t−1 is not revisited at subsequent interactions. Making a request to StreamingBandit's setreward endpoint containing a JSON object including either the advice_id or a complete description of {x t , a t , p t }, and the reward r t , allows one to update θ t +1 and subsequently to influence the actions selected at t + 1.…”

Section: Basic Usagementioning

confidence: 99%

StreamingBandit: Experimenting with Bandit Policies

Kruijswijk¹,

Emden²,

Parvinen³

et al. 2020

J. Stat. Soft.

View full text Add to dashboard Cite

A large number of statistical decision problems in the social sciences and beyond can be framed as a (contextual) multi-armed bandit problem. However, it is notoriously hard to develop and evaluate policies that tackle these types of problems, and to use such policies in applied studies. To address this issue, this paper introduces StreamingBandit, a Python web application for developing and testing bandit policies in field studies. StreamingBandit can sequentially select treatments using (online) policies in real time. Once StreamingBandit is implemented in an applied context, different policies can be tested, altered, nested, and compared. StreamingBandit makes it easy to apply a multitude of bandit policies for sequential allocation in field experiments, and allows for the quick development and re-use of novel policies. In this article, we detail the implementation logic of StreamingBandit and provide several examples of its use.

show abstract

Section: Basic Usagementioning

confidence: 99%

StreamingBandit: Experimenting with Bandit Policies

Kruijswijk¹,

Emden²,

Parvinen³

et al. 2020

J. Stat. Soft.

View full text Add to dashboard Cite

show abstract

“…This section is aimed at discussion and analysis of existing research on data streams and tries to highlight significant challenges and issues in this domain. In [1] the authors have discussed the issue of power reduction of radio frequency interference (RFI) effect on the high-energy impulses to find it and receive it from different sources. For this the purpose radio astronomy has been used along with an example of Allen Telescope Array (ATA) to collect data.…”

Section: Application Of Data Mining In Crime Detectionmentioning

confidence: 99%

Challenges and Issues in DATA Stream: A Review

Arif¹,

Alam²,

Hussain³

2015

IJHIT

View full text Add to dashboard Cite

Data stream is a continuous, time varying, massive and infinitely ordered sequence of data elements. The streaming data are fast changing with time, it is impossible to acquire all the elements in a data stream. Therefore, each data element should be examined at most once in data streams. Memory usage for mining data stream should be limited due to the new data elements are continuously generated from the streams. It is essential to ensure that newly arrived stream should be immediately available whenever it is requested made this task much challenging and necessary for fraud detection in stream, taking out knowledge, for business improvement and other applications where data arrived in stream. This paper tries to highlight important issues and research challenges of data stream by means of a comprehensive review.

show abstract

“…Streaming data, consisting of indefinitely and possibly time-evolving sequences, are becoming ubiquitous in many branches of science (Chu et al, 2007;Michalak et al, 2012). The omnipresence of streaming data poses new challenges for statistics and machine learning.…”

Section: Introductionmentioning

confidence: 99%

“…Streaming algorithms provide a computationally efficient way to deal with continuous data streams by summarizing all historic data into a limited set of parameters. With the current growth of available data the development of reliable streaming algorithms whose behavior is well understood is highly important (Michalak et al, 2012). For a more formal description of streaming (or online) learning see Bottou (1998).…”

Section: Introductionmentioning

confidence: 99%

RStorm: Developing and Testing Streaming Algorithms in R

Kaptein¹

2014

The R Journal

View full text Add to dashboard Cite

Streaming data, consisting of indefinitely evolving sequences, are becoming ubiquitous in many branches of science and in various applications. Computer scientists have developed streaming applications such as Storm and the S4 distributed stream computing platform 1 to deal with data streams. However, in current production packages testing and evaluating streaming algorithms is cumbersome. This paper presents RStorm for the development and evaluation of streaming algorithms analogous to these production packages, but implemented fully in R. RStorm allows developers of streaming algorithms to quickly test, iterate, and evaluate various implementations of streaming algorithms. The paper provides both a canonical computer science example, the streaming word count, and examples of several statistical applications of RStorm.

show abstract

Developing Systems for Real-Time Streaming Analysis

Cited by 12 publications

References 27 publications

StreamingBandit: Experimenting with Bandit Policies

StreamingBandit: Experimenting with Bandit Policies

Challenges and Issues in DATA Stream: A Review

RStorm: Developing and Testing Streaming Algorithms in R

Contact Info

Product

Resources

About