All for One and One for All

Abstract

A s scientific instruments become ever more powerful, from orbiting observatories to genomesequencing machines, they are making their fields data-rich but analysis-poor. Ground-based telescopes in digital sky surveys are currently pouring several hundred terabytes (10 12 bytes) of data per year into dozens of archives, enough to keep astronomers busy for decades. The four satellites of NASA's Earth Observing System currently beam down 1000 terabytes annually, far more than earth scientists can hope to calibrate and analyze. And looming on the horizon is the Large Hadron Collider, the world's largest physics experiment, now under construction at CERN, Europe's particle physics lab near Geneva. Soon after it comes online in 2007, each of the five detectors will be spewing out several petabytes (10 15 bytes) of data-about a million DVDs' worth-every year. These and similar outpourings of information are overwhelming the available computing power. Few researchers have access to the powerful supercomputers that could make inroads into such vast data sets, so they are trying to be more creative. Some are parceling big computing jobs into small work packages and distributing them to underused computers on the Internet. With this strategy, insurmountable tasks may soon become manageable. One approach to such "distributed computing" was pioneered by computer scientists working with SETI, the Search for Extraterrestrial Intelligence. The phenomenally successful SETI@home program now makes use of the idle computer time of millions of ordinary computer users, working as a screen saver to quietly crunch away at radio-signal data from deep space. As John Bohannon describes on p. 810, the same screensaver technique is now being used by a wide array of researchers studying everything from climate change to gravitational waves and protein folding. Bohannon also delves into the strange tribal world (p. 812) of the "crunchers": computer enthusiasts whose goal is to become the most prolific processors of data for various screen-saver research projects. And on p. 813, Mark Buchanan samples a piece of computer navel gazing: a distributed computing project to study the geography of the Internet itself. Another way of distributing both data and computing power, known as grid computing, taps the Internet to put petabyte processing on every researcher's desktop. On p. 814, Foster highlights the development of a lingua franca of grid computing: a set of standardized interfaces and protocols that permits researchers to work across the Web. Hey and Trefethen (p. 818) describe the U.K.-based e-Science program to design plug-and-play grid technologies for a range of disciplines. And Buetow (p. 822) outlines the ways in which cyberinfrastructure can weld together the vastly different styles of biomedical research. For all the excitement, however, there are disturbing trends in the directions being taken by funding agencies that have historically been involved with driving the Internet revolution. In their Editorial (p. 757), Lazowska and Patter...

Internet-based virtual computing environment (iVCE): Concepts and architecture

“…In the past ten years, many projects for grid computing have been launched, such as TeraGrid, DataGrid and e-Science, to explore the methods to build a new computing environment on the Internet [1] . Afterward, the OGSA (Open Grid Services Architecture) [11] and the WSRF specification [12] combine grid computing with web services.…”

“…With the fast development and wide application of computing and network technologies, Internet has become an important information infrastructure for modern society [1][2][3] . Nowadays there are vast kinds of resources over Internet, including instruments, information and applications, etc.…”

A Service-Oriented Architecture for Proactive Geospatial Information Services

“…The contrast between the increasing ability to acquire data and insufficient abilities to process data becomes more serious. The problem "data-rich but analysis-poor" [8] is worsening. The fundamental reason behind this problem is not due to the lack of processing approaches to deal with those tremendous datasets, but the shortage of effective sharing, aggregation, and cooperation service mechanisms to handle the sensors, data, and processes focusing on user's complex tasks in near real-time [9].…”

“…Human/citizen sensor network is a new paradigm to strengthen data collection by Internet or Web-enabled mobile sensors and powerful computing ability. Humans as citizens on the ubiquitous Web, acting as sensors and sharing their observations and views using mobile devices and Web 2.0 services [7], i.e., "all for one and one for all" [8].…”

The emergence of spatial cyberinfrastructure

“…Additionally, making scientific discoveries requires the computational ability to synthesize and analyze very large datasets that are integrated across biological, physical, and social sciences and engineering and across the science-technology interface, where Hey et al (5) name "data-intensive science" as the "fourth paradigm." Indeed, CI has become more than just hardware and software but its own evolving area of research in the realm of data-intensive science and digital libraries (5-9), with many countries investing hundreds of millions of dollars in CI research and development (10,11) and calls coming from diverse scientific communities arguing the urgent need for further levels of CI investment (12,13). Hey et al (5) point out that, although we have attained high-performance computing at affordable cost and have made good progress on simulation tools, many challenges remain in effectively integrating multiple field observatories containing thousands of instruments, involving millions of users and petabytes of data, built on a true data grid with the ability to analyze data on that grid with sophisticated data analysis.…”