Agile Infrastructure at CERN: Moving 9'000 servers into a private cloud (Videoconferência)
CERN, the big particle physics lab close to Geneva (Switzerland), is home to the Large Hadron Collider (LHC) and its four data-intensive experiments. In 2012, a new particle was detected that was identified in 2013 as the long-sought Higgs boson. The experiments deliver some
50 Petabytes of data per year to the CERN computer centre that runs almost 14'000 servers in order to process them. For the next years, data rates will grow, and so will the number of servers. In order to cope with this increase without corresponding increase of personnel, CERN reviewed the way it runs its computer centre, and launched the "Agile Infrastructure" project with the aim of organising services into horizontal layers; moving to a private cloud is an integral part of the strategy. Major work areas are configuration, cloud computing, and monitoring. Other areas such as hardware provisioning workflow and local workload management system are being refurbished as well. Whenever possible, existing free, open-source solutions have been selected as components for well-confined functionality, with CERN adding just a bit of glue to make the components work together, with the aim of being able to easily replace a component, should the chosen one not perform to expectations. Cornerstones of the new system include Puppet and PuppetDB, Foreman, Openstack, ElasticSearch and Kibana, Flume and others. As of August 2015, almost 18'000 servers (physical or virtual) are managed with Puppet, and almost 5'000 physical servers have been moved into the private cloud that comprises 14'000 VMs and more than 130'000 cores. The presentation will discuss the motivation, describe the current choices of tools, report on experience made so far, and give an outlook of what is expected to come.