Integrating network-awareness and network-management into PhEDEx: first results from the ANSE project

Abstract

ANSE Collaboration †PhEDEx is the data placement and management tool for the CMS experiment at the LHC. It controls the large-scale data-flows on the WAN across the experiment, typically handling 1 PB of data per week. While robust, its architecture is now ten years old and has yet to fully adapt to today's production environment, an environment in which the network is the fastest and most reliable component. The ANSE (Advanced Network Services for Experiments) project, in the context of CMS, aims to greatly improve PhEDEx' network awareness for smart source selection, as well as to integrate bandwidth provisioning capabilities in the data transfer management. Both parts require a good knowledge of the network status, topology and of course, access to useful and up-to-date performance metrics. One of the first steps towards this goal involved the identification of a mechanism for informing PhEDEx about independent network performance metrics. Methods for providing these metrics have been prototyped and verified in a LAN testbed using fake data transfer requests. This mechanism is already directly usable by CMS in their production environment. Currently, the ANSE-PhEDEx testbed is spread over many servers at a number of sites. It is composed of several machines dedicated to PhEDEx site agents, one server holding the PhEDEx central agents, a central database and one server which contains the PhEDEx website and dataservice. Some of the site nodes have additional attached storage nodes. In this paper, we present the work that has been done in ANSE for PhEDEx. This includes performance measurements using the Fast Data Transfer (FDT) tool and the extension of the PhEDEx agent that downloads files to a site to allow it to control the network via creation and use of dynamic circuits. We present the results of our tests using these new features, on highspeed WAN circuits ranging from a few Gbps to 40Gbps and detail the development done within PhEDEx itself. Finally, the paper will also describe the future plans for the project.