FTS3, the service responsible for globally distributing the majority of the LHC data across the WLCG infrastructure, is now available to everybody. Already integrated into LHC experiment frameworks, a new Web interface now makes the FTS3's transfer technology directly available to end users. In this contribution we describe this intuitive new interface, "WebFTS", which allows users to easily schedule and manage large data transfers right from the browser, profiting from a service which has been proven at the scale of petabytes per month. We will shed light on new development activities to extend FTS3 transfers capabilities outside Grid boundaries with support of non-Grid endpoints like Dropbox and S3. We also describe the latest changes integrated into the transfer engine itself, such as new data management operations like deletions and staging files from archive, all of which may be accessed through our standardscompliant REST API. For the Service Manager, we explain such features as the service's horizontal scalability, advanced monitoring and its "zero configuration" approach to deployment made possible by specialised transfer optimisation logic. For the Data Manager, we will present new tools for management of FTS3 transfer parameters like limits for bandwidth and max active file transfers per endpoint and VO, user and endpoint banning and powerful command line tools. We finish by describing the impact of extending WebFTS's captivating graphical interface with support of Federated Identity technologies, thus demonstrating the use of Grid resources without the burden of X.509 certificate management. In this manner we show how FTS3 can cover the needs of wide range of parties from casual users to high-load services.
The various Euclid imaging surveys will become a reference for studies of galaxy morphology by delivering imaging over an unprecedented area of 15 000 square degrees with high spatial resolution. In order to understand the capabilities of measuring morphologies from Euclid-detected galaxies and to help implement measurements in the pipeline of the Organisational Unit MER of the Euclid Science Ground Segment, we have conducted the Euclid Morphology Challenge, which we present in two papers. While the companion paper focusses on the analysis of photometry, this paper assesses the accuracy of the parametric galaxy morphology measurements in imaging predicted from within the Euclid Wide Survey. We evaluate the performance of five state-of-the-art surface-brightness-fitting codes, DeepLeGATo, Galapagos-2, Morfometryka, ProFitand SourceXtractor++ , on a sample of about 1.5 million simulated galaxies (350 000 above 5σ) resembling reduced observations with the Euclid VIS and NIR instruments. The simulations include analytic Sérsic profiles with one and two components, as well as more realistic galaxies generated with neural networks. We find that, despite some code-specific differences, all methods tend to achieve reliable structural measurements (< 10% scatter on ideal Sérsic simulations) down to an apparent magnitude of about I E = 23 in one component and I E = 21 in two components, which correspond to a signal-to-noise ratio of approximately 1 and 5, respectively. We also show that when tested on non-analytic profiles, the results are typically degraded by a factor of 3, driven by systematics. We conclude that the official Euclid Data Releases will deliver robust structural parameters for at least 400 million galaxies in the Euclid Wide Survey by the end of the mission. We find that a key factor for explaining the different behaviour of the codes at the faint end is the set of adopted priors for the various structural parameters.
Recently the importance of clustering storage nodes across site boundaries is becoming more clear, thanks also to the recent ongoing initiatives in the CMS and ATLAS experiments. These approaches are supposed to promote simplicity in accessing the data and offering new possibilities for resilience and data placement strategies, that may also lead to a better utilization of the available CPU slots. Here we report on work that seeks to exploit the federation potential of redirectable protocols like HTTP/WebDAV and build a dynamic, scalable, persistency-free system that offers a unique view of the storage and metadata ensemble and the possibility of integration of other compatible resources such as those from cloud providers. The challenge, here undertaken by the providers of dCache and DPM, partially in the context of EMI-data, and pragmatically open to any other Grid and Cloud storage solutions, is to build such a system while being able to accommodate name translations from existing catalogues (e.g. LFCs), experiment-based metadata catalogues, or stateless algorithmic name translations, also known as "trivial file catalogues". Other technical challenges that will determine the success of this initiative include performance, latency and scalability, and the ability to create worldwide storage federations that are able to redirect clients to repositories that they can efficiently access, for instance trying to choose the endpoints that are closer or applying other criteria. One of the key requirements is to use standard clients (provided by OS'es or open source distributions, e.g. Web browsers) to access an already aggregated system. This was accomplished by exploiting the possibilities offered by the DMLite framework, in order to integrate with existing frontends like the Apache server. We believe that the features of a loosely coupled federation of open-protocols-based storage elements will open many possibilities of evolving the current computing models without disrupting them, and, at the same time, can operate with the existing infrastructures, follow their evolution path and add storage centers that can be acquired as a third-party service.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.