Stephen Trofinoff (CSCS) presents how to schedule GRES resources with SLURM.
Blog
Archive for the ‘CSCS’ Category
hpc-ch Forum on GPU – Video on Scheduling GRES resources With SLURM
Wednesday, November 23rd, 2011hpc-ch Forum on GPU – Video on Installation and Operational Needs of Multi-purpose GPU Clusters
Tuesday, November 22nd, 2011CSCS has experience of operating multi-purpose GPU clusters for a diverse project and service portfolio.
Vincenzo Annaloro (CSCS) presents details on system installation and operation for two clusters: a production and experimental testbed that serves as HP2C prototype system and a multi-purpose visualization and GPU development cluster.
Lake Water to Cool Supercomputers at CSCS
Tuesday, November 15th, 2011High-performance computing centres use a great deal of electricity. In order to run its new computer centre in Lugano-Cornaredo as energy-efficiently and cost-effectively as possible, CSCS is using the natural resource of Lake Lugano to cool its supercomputers and the new building.
A high-performance computing centre like the CSCS (Swiss National Supercomputing Centre) uses as much electricity every day as a small town. About a third of this electricity is used for cooling. If supercomputers are not constantly cooled, they overheat and, in the worst case scenario, may be damaged.
Three pumps move up to 760 litres of water per second
In conventional computing centres, compressors generate cold water to cool the computers. To save energy, CSCS will make use of a natural cooling resource: water extracted from Lake Lugano at 45 meters depth at a temperature of 6 degrees Celsius. This will greatly reduce the amount of electricity used by CSCS for cooling.
To operate the cooling system, a pumping station has been built on the lake shore in Parco Ciani, in the centre of Lugano. This is connected by a pipe to the 13 tonne, six metre high suction baskets which have been immersed at a depth of over forty metres. Through this pipe, three pumps in the pumping station will pump up to 760 litres of water per second. 460 litres will go to CSCS for cooling purposes, while up to 300 litres will be available to AIL for its water reservoir, which has yet to be built. On its way to CSCS, the water will climb 30 metres and cover a distance of 2.8 kilometres.

n the labyrinth of the pumping station: five metres beneath the lawns of Parco Ciani, over 700 litres of water per second are pumped out of Lake Lugano. (Picture: CSCS)
High-tech underground control centre
A steel ladder leads down to the high-tech underground control centre of the pumping station. The electrical infrastructure has been installed on steel mesh platforms about 2.5 metres above the ground, underneath this – covering a surface area of about 200 square metres –are the three pumps, a further backup pump, and two shock absorbing containers which will protect the pipeline from damage in the event of a sudden pressure surge.
On two sides of the pumping station, two massive pipes pierce the metre-thick walls. The pipeline connecting CSCS to the pumping station is 80 centimetres in diameter. On its way to CSCS, the water´s temperature will increase by up to 0.5°C. At the arrival point at CSCS, the lake water circuit and the internal cooling water circuit meet in man-sized heat exchangers. The internal water cooling circuit delivers the water, which has now reached 8 -9°C, to the supercomputers for cooling.

The suction strainers for the lake water pipe, just before they were lowered 45 metres into Lake Lugano. (Picture: CSCS)
One pumping operation for double the cooling effect
Once the water has passed through this first cooling circuit, it has been heated up by eight degrees. The now 16 to 17 °C is sent through a further heat exchanger, connected to a second cooling circuit. This mid-temperature circuit cools the air in the housings for the computers and hard drives of lesser energy density, that can therefore be cooled with water that is less cold. This means that with one pumping operation, two cooling is supplied to two circuits to cool two types of system.
The cold water pipe is designed to cool supercomputers of up to 14 megawatts on the first cooling circuit. The second circuit can cool a further seven megawatts of computers. The more the second circuit is used, the higher the waste heat absorbed by the water and so the more useful it is to the local industrial works who will be able to use it.
Before the lake water returns to the lake, it passes through a stilling basin which can hold 120 cubic metres. The basin collects the water and makes sure that it flows freely down the return pipe back to the lake at a constant pressure and with no need for further power to be used. On the contrary, the plan is to use the energy generated as it falls to produce electricity. That is why connections for a microturbine have been provided in the pumping station.
So as not to affect the ecological balance of the lake, the water going back into the lake must never exceed 25 degrees Celsius. To ensure that this is always the case, a back-mixing funnel has been fitted which will add cold water if necessary.
(Article by Simone Ulmer, CSCS)
Switzerland at SC11 in Seattle
Friday, November 11th, 2011Once again CSCS together with several other representatives of Swiss universities and institutes goes to Supercomputing 2011. This year conference is held in Seattle, WA from November 12th to 18th. The Swiss contributions to Supercomuting are various and in several categories. Moreover, CSCS is involved in panels committees, in several Birds-of-Feather sessions, and has several papers submitted.
Once again there will be a Swiss Booth at the exhibition organized together with hpc-ch (booth number 5615).
Herein is a more detailed list of the contributions:
- Scientific Visualization Showcase, “The Reversing Magnetic Field of Planet Earth” by Jean M. Favre (CSCS), Jon Rotvig (ETHZ), Andrew Jackson (ETHZ): This showcase will be presented at SC11 in a museum/art exhibit-style environment so that attendees can experience and enjoy the latest in science and engineering HPC results expressed through state-of-the-art visualization technologies.
- Birds-of-Feather (BOF) are informal group of discussions where experts in different disciplines are meeting to discuss and exchange important ideas. CSCS is involved in two of such sesssions :
- Scalable Heterogeneous Computing Forum, Jeff Vetter and Sadaf Alam (CSCS)
- Parallel Communication and I/O Interfaces for Accelerators: HPC Advisory Council Special Interest Group, Gilad Shainer and Sadaf Alam (CSCS)
- “rCUDA demonstration at Mellanox’s buzz” by Antonio Penya at the exhibit floor
- Poster: “Autotuning for communication bound scientific kernels” by Adrian Tineo (CSCS) and Sadaf Alam (CSCS) at the PGAS booth at SC11
- Paper: “Parallel I/O and the Metadata Wall” by Sadaf Alam, Hussein N El-Harake, Kristopher Howard, Neil Stringfellow and Fabio Verzelloni will be presented at the 6th Parallel Data Storage Workshop (PDSW’11)
- Paper:”Towards autotuning by alternating communication methods” by Adrian Tineo, Sadaf Alam and Thomas Schulthess will be presented at the 2nd International Workshop on Modeling, Benchmarking and Simulation of High Performance Computing Systems (PMBS11)
- CSCS Director, Thomas Schulthess and Sadaf Alam are both in the Technical Program Committee
- SC11 proceedings chair: Sadaf Alam
- Thomas Schulthess, member of the Gordon Bell Committee
CSCS User Day – Video “Microscopic origins of complex behavior in carbon and sodium” by Rustam Khaliullin
Wednesday, November 9th, 2011We terminate our series of videos from the CSCS User day with an invited presentation of Rustan Khaliullin.
Rustan is from the group led by Michele Parrinello, a professor at the ETH Zurich and the Universitą della Svizzera italiana, concluded the series of scientific lectures at the CSCS User Day 2011 with a presentation of the latest research breakthrough in the simulation of molecular dynamic processes. The team has succeeded in using molecular dynamic simulations to show how diamonds are formed from graphite under high pressure (see the full presentation).

