User Environment Tracking and Problem Detection with XALT

Agrawal, Kapil; Fahey, Mark R.; McLay, Robert; James, Doug L.

doi:10.1109/hust.2014.6

Cited by 44 publications

(13 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This would allow us detecting similar problems that experience different users and workgroups. This work is planned to be done using XALT tool designed for collecting job-level information about used libraries, modules and executables [10].…”

Section: Discussionmentioning

confidence: 99%

Deep Analysis of Job State Statistics on Lomonosov-2 Supercomputer

2018

JSFI

View full text Add to dashboard Cite

It is a common knowledge that the increasingly growing capabilities of HPC systems are always limited by a number of efficiency related issues. The reasons can be very different: hardware failures, incorrect job scheduling, peculiarities of algorithm, chosen programming technology specifics, etc. Most of these issues can be detected after precise analysis, but is a very resourceful way to study every application run. Therefore we performed less complicated analysis of the whole supercomputer job flow. In this paper we share our experience of analyzing user applications job states assigned by the SLURM resource manager that is used on the Lomonosov-2 system at Supercomputing center of Lomonosov Moscow State University. The statistics on job states was collected and it revealed that the ratio of correctly finished jobs (with the COMPLETED state) was rather low. The jobs owners were asked if the distribution of their jobs states is normal regarding their applications. This user feedback was processed, and some new ways of efficiency gain were revealed as the result.

show abstract

Section: Discussionmentioning

confidence: 99%

Deep Analysis of Job State Statistics on Lomonosov-2 Supercomputer

2018

JSFI

View full text Add to dashboard Cite

show abstract

“…System statistics are collected by collectd and nmond monitoring systems and then processed by Octotron [7] (anomaly detection). Data about actually compiled and used applications and computational packages are collected by XALT software [8].…”

Section: System Software and Programming Systemsmentioning

confidence: 99%

Supercomputer Lomonosov-2: Large Scale, Deep Monitoring and Fine Analytics for the User Community

2019

JSFI

182

View full text Add to dashboard Cite

The huge number of hardware and software components, together with a large number of parameters affecting the performance of each parallel application, makes ensuring the efficiency of a large scale supercomputer extremely difficult. In this situation, all basic parameters of the supercomputer should be constantly monitored, as well as many decisions about its functioning should be made by special software automatically. In this paper we describe the tight connection between complexity of modern large high performance computing systems and special techniques and tools required to ensure their efficiency in practice. The main subsystems of the developed complex (Octoshell, DiMMoN, Octotron, JobDigest, and an expert software system to bring fine analytics on parallel applications and the entire supercomputer to users and sysadmins) are actively operated on the large supercomputer systems at Lomonosov Moscow State University. A brief description of the architecture of Lomonosov-2 supercomputer is presented, and questions showing both a wide variety of emerging complex issues and the need for an integrated approach to solving the problem of effectively supporting large supercomputer systems are discussed.

show abstract

“…that are actually called by the application. This capability becomes increasingly useful vendor-supplied libraries such as the Intel MKL, the Cray LibSci, or ACML combine multiple canonical numerical libraries (BLAS, LAPACK, Fourier 1 Universally unique identifier, a 128-bit value used to uniquely identify information (see e.g. : https://tools.ietf.org/html/rfc4122.html) 2 JSON is an open standard date interchange format, see http://json.org/ Transforms, etc) into a single package.…”

Section: Function Trackingmentioning

confidence: 99%

“…XALT is a tool that collects accurate, detailed, and continuous job-level and link-time data and stores that data in a database; all the data collection is transparent to the users [1]. XALT tracks libraries and object files linked by the application.…”

Section: Introductionmentioning

confidence: 99%

Library Function Tracking with XALT

Budiardja

Agrawal

Fahey

et al. 2016

Proceedings of the XSEDE16 Conference on Diversity, Big Data, and Science at Scale

Self Cite

View full text Add to dashboard Cite

XALT is a tracking tool that collects accurate, detailed, and continuous job-level and link-time data. XALT stores that data in a database and ensures that all the data collection is transparent to the users. XALT tracks libraries and object files linked by the application. A recent feature improvement in XALT enable it to also track external subroutines and functions called by an application. This paper describes the function-tracking implementation in XALT and showcases the kind of data and analysis that becomes available from this new feature. A recently developed web-based interface to XALT database is also described, allowing the staffs of a supercomputing center to more easily understand software usage on their compute resources.

show abstract

User Environment Tracking and Problem Detection with XALT

Cited by 44 publications

References 4 publications

Deep Analysis of Job State Statistics on Lomonosov-2 Supercomputer

Deep Analysis of Job State Statistics on Lomonosov-2 Supercomputer

Supercomputer Lomonosov-2: Large Scale, Deep Monitoring and Fine Analytics for the User Community

Library Function Tracking with XALT

Contact Info

Product

Resources

About