Performance optimization in a High Throughput Computing environment
Project name
Performance optimization in a High Throughput Computing environmentProject description
Profiling of computing resources respect to WLCG experiment workloads is a crucial factor to select the most effective resources and to be able to optimise their usage.
There is a rich amount of data collected by the CERN and WLCG monitoring infrastructures just waiting to be turned into useful information. This data covers all the areas of the computing activity such as (real and/or virtual) machine monitoring, storage, network, batch system performance, experiment job monitoring.
Data gathered by those systems contain great intrinsic value, however information needs to be extracted and understood through a predictive data analytics process. The final purpose of this process is to support decisions and improve the efficiency and the reliability of the related services.
For instance, with the adoption of the remote access of data it becomes mandatory to understand the impact of this approach to the job efficiency. Here the interplay of network and CPU effects, as well as the resource usage from multi VOs needs to be studied and understood. An interesting topic of study is the performance of job processing at the WLCG distributed T0 center, which is physically split between Computer Centers in Meyrin and Wigner. The goal of the project will be to understand the difference in the performance and to suggest possible optimization.

The work will be conducted in close contact with the experts (CERN analytics working group, system managers, developers) and will provide a deep insight into the computing infrastructure of a WLCG datacenter, its design, technical requirements and operational challenges.