Processing requirements at NASA's AMSR-E SIPS * have evolved considerably throughout the life span of the mission, as the SIPS has moved from mission testing and early experimental data processing to full operations supporting both forward processing and reprocessing. Working in close collaboration with the University of Alabama in Huntsville's Information Technology and Systems Center (ITSC), the SIPS has developed a robust and flexible hardware and software framework that supports a variety of data streams and processing requirements. Today the SIPS maintains four operational processing environments in addition to development and integration test areas:• Routine operations environment for forward processing of AMSR-E data upon acquisition, • Late processing environment to handle occasional swaths of data that arrive after gridded products have been created for their time period, • Reprocessing environment for large-scale reprocessing efforts of individual data products or the entire product suite, • A special processing environment to manage special requests by the AMSR-E science team, such as evaluation of new algorithms with seasonal data. In addition, ITSC personnel have integrated science data subsetting and browse imagery generation into the SIPS processing flow, enabling the scientists to quickly and easily target parameters of interest. While similar in structure, each of these environments and their target data streams have unique requirements that would normally necessitate customized software. This paper will describe the evolution of the SIPS, the resulting processing and distribution framework, and plans to meet future requirements.
Issues associated with failed data transfers can be complicated and inconvenient, not only for end users expecting timely products, but also for the data managers who are responsible for recovering from failures. Working in close collaboration with engineers at the University of Alabama in Huntsville's Information Technology and Systems Center, the AMSR-E SIPS has evolved a local implementation of the Product Delivery Record Server from a simple ECS-compliant data exchange system to a smart automated message handling system that also executes routine operational error recovery procedures. While automated data exchange mechanisms have been the norm for quite some time, the AMSR-E SIPS has deployed unique technologies in our local implementation of the PDRS by integrating it with and automating some routine data management functions, such as process scheduling, restaging of certain classes of failed data transfers, and statistics reporting. This paper discusses the evolution of the PDRS design and architecture, highlights the unique features of this flexible processing system that has contributed to reduced operating costs, and describes our plans to meet future requirements.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.