Adaptive and sequential experiment design is a well-studied area in numerous domains. We survey and synthesize the work of the online statistical learning paradigm referred to as multi-armed bandits integrating the existing research as a resource for a certain class of online experiments. We first explore the traditional stochastic model of a multi-armed bandit, then explore a taxonomic scheme of complications to that model, for each complication relating it to a specific requirement or consideration of the experiment design context. Finally, at the end of the paper, we present a table of known bounds of regret for all studied algorithms providing both perspectives for future theoretical work and a decision-making tool for practitioners looking for theoretical guarantees. Primary 62K99, 62L05; secondary 68T05. Keywords and phrases: multi-armed bandits, adaptive experiments, sequential experiment design, online experiment design. * Loeppky and Lawrence were partly supported by Natural Sciences and Engineering Research Council of Canada Discovery Grants, grant numbers RGPIN-2015-03895 and RGPIN-341202-12 respectively. 1 imsart-generic ver. 2011/11/15 file: mab-ss-survey.tex date: November 4,

With a very modest investment in computer hardware and the open source local data manger (LDM) software from UCAR's Unidata Program Center, an individual researcher can receive a variety of NEXRAD Level III gridded rainfall products, and the unprocessed Level II data in real-time from most NEXRAD radars. Additionally, the National Climatic Data Center has vast archives of these products and Level II data. Still, significant obstacles remain in order to unlock the full potential of the data. One set of obstacles is related to effective management of multi-terrabyte data sets: storing, compressing, and backing up. A second set of obstacles, for hydrologists and hydrometeorologists in particular, is that the NEXRAD Level III products are not well suited for application in hydrology. There is a strong need for the generation of highquality products directly from the Level II data with well-documented steps that include quality control, removal of false echoes, rainfall estimation algorithms with variety of corrections, coordinate conversion and georeferencing, conversion to a convenient data format(s), and integration with GIS. For hydrologists it is imperative that these procedures are basin-centered as opposed to radar-centered. Thirdly, the amount of data present in a multi-year, multi-radar dataset is such that simple cataloging and indexing of the data is not sufficient. Rather, sophisticated metadata extraction and management techniques are required. The authors describe and discuss the Hydro-NEXRAD software system that addresses the above three challenges. With support from the National Science Foundation through its ITR program, the authors are developing a basin-centered framework for addressing all these issues in a comprehensive manner, tailored specifically for use of NEXRAD data in hydrology and hydrometeorology. Through a flexible web interface users can search a large metadata database base, managed by a World Environmental and Water Resources Congress 2007: Restoring Our Natural Habitat

No abstract

Real-time heuristic search algorithms satisfy a constant bound on the amount of planning per action, independent of problem size. As a result, they scale up well as problems become larger. This property would make them well suited for video games where Artificial Intelligence controlled agents must react quickly to user commands and to other agents' actions. On the downside, real-time search algorithms employ learning methods that frequently lead to poor solution quality and cause the agent to appear irrational by re-visiting the same problem states repeatedly. The situation changed recently with a new algorithm, D LRTA*, which attempted to eliminate learning by automatically selecting subgoals. D LRTA* is well poised for video games, except it has a complex and memory-demanding pre-computation phase during which it builds a database of subgoals. In this paper, we propose a simpler and more memory-efficient way of pre-computing subgoals thereby eliminating the main obstacle to applying state-of-the-art real-time search methods in video games. The new algorithm solves a number of randomly chosen problems off-line, compresses the solutions into a series of subgoals and stores them in a database. When presented with a novel problem on-line, it queries the database for the most similar previously solved case and uses its subgoals to solve the problem. In the domain of pathfinding on four large video game maps, the new algorithm delivers solutions eight times better while using 57 times less memory and requiring 14% less pre-computation time.