Unlocking the Potential of NEXRAD Data through NOAA’s Big Data Partnership

Abstract

The National Oceanic and Atmospheric Administration’s (NOAA) Big Data Partnership (BDP) was established in April 2015 through cooperative research agreements between NOAA and selected commercial and academic partners. The BDP is investigating how the value inherent in NOAA’s data may be leveraged to broaden their utilization through modern cloud infrastructures and advanced “big data” techniques. NOAA’s Next Generation Weather Radar (NEXRAD) data were identified as an ideal candidate for such collaborative efforts. NEXRAD Level II data are valuable yet challenging to utilize in their entirety, and recent advances in weather radar science can be applied to both the archived and real-time data streams. NOAA’s National Centers for Environmental Information (NCEI) transferred the complete NEXRAD Level II historical archive, originating in 1991, through North Carolina State University’s Cooperative Institute for Climate and Satellites (CICS-NC) to interested BDP collaborators. Amazon Web Services (AWS) has received and made freely available the complete archived Level II data through its AWS platform. AWS then partnered with Unidata/University Corporation for Atmospheric Research (UCAR) to establish a real-time NEXRAD feed, thereby providing on-demand dissemination of both archived and current data seamlessly through the same access mechanism by October 2015. To organize, verify, and utilize the NEXRAD data on its platform, AWS further partnered with the Climate Corporation. This collective effort among federal government, private industry, and academia has already realized a number of new and novel applications that employ NOAA’s NEXRAD data, at no net cost to the U.S. taxpayer. The volume of accessed NEXRAD data, including this new AWS platform service, has increased by 130%, while the amount of data delivered by NOAA/NCEI has decreased by 50%.

MistNet: Measuring historical bird migration in the US using archived weather radar data and convolutional neural networks

“…We downloaded radar scans for training from Amazon Web Services (Ansari et al, ). The scans were selected from all 161 radar stations in spring (April and May) and fall (September and October) from 2014 through 2016.…”

“…Each radar scans its surroundings every 4 to 10 min. The radars and data collection are standardized, and essentially all of the data—over 200 million individual files—has been archived over more than 25 years (Ansari et al, ). It is well known that these radars regularly detect birds and can be used to quantify their movements (Gauthreaux & Belser, ).…”

Revealing patterns of nocturnal migration using the European weather radar network

“…Through several initiatives the data has also become more accessible. In the US the long-term national archive of NEXRAD weather data is available as open data (Ansari et al 2018) and in Europe the recent COST Action 'ENRAM' (European Network for the Radar surveillance of Animal Movement) is working with the European Meteorological Services Network (EUMETNET) which runs OPERA (Operational Program for the Exchange of Weather Radar Information; Huuskonen et al 2014) to make the data from the European weather radar network available to biologists. The European effort involves the standardization of data collection; data formats and storage.…”

Field validation of radar systems for monitoring bird migration

“…The use of WRs to monitor animal movements have so far mainly been explored in continental US and Europe, but it has the potential to be used in other countries with extensive WR networks, like Russia, China and India. In the US the entire data archive of all 143 continental NEXRAD WR stations are publicly available (Ansari et al., ) and in Europe the European Network for the Radar Surveillance of Animal Movement together with the Operational Program for the Exchange of Weather Radar Information (OPERA) is in the process of making bird profiles from European WRs available (Shamoun‐Baranes et al., ).…”