Programming GPU Devices using OpenACC Directives on the Cray XK6
March, 6-7  2012
CSCS in Manno

The registration fee is of CHF 150 including the coffee breaks.

Registration and agenda »

Contents

Attendees of this HP2C training event will learn about the Cray XK6 hybrid multi-core and GPU architecture and its programming environment.

They will learn about the OpenACC directives, which were designed to help users develop and port applications to run on heterogeneous systems. They will have an understanding on how to use the Cray Performance tools to identify “hot areas” in the code to focus the use of OpenACC directives. They will have the opportunity to experiment the OpenACC directives with the Cray Compilation Environment (CCE). In addition, they will learn about the Cray scientific libraries for accelerators and will learn and experiment Allinea’s DDT and Cray’s Performance Tools for debugging and performance tuning of heterogeneous applications on the Cray XK6 systems.

Attendees are encouraged to bring in their own applications and codes for the hands-on sessions.  Experts from Cray PE, OpenACC and libsci development and performance tools and Allinea DDT debugger will be present at the meeting for discussions and feedback.  We also invite current users who have their applications running successfully on the Cray XK6 system to present brief user experience talks.

Agenda

- Welcome
- Overview of the Cray XK6 system
- Introduction to Cray XK6 Programming Environment
- Support for GPU application development and execution

• GPU development environments (CUDA C & Fortran, OpenCL & OpenACC from Cray & PGI)
• GPU accelerated libraries
• Message passing communication (MPI)

- Introduction to OpenACC
- Development cycle of application porting

• Static analysis of the application
• Find hot loops
• Scoping Analysis
• Add OpenMP
• Create OpenACC regions from OpenMP regions

- Using libsci_acc
- Debugging
- Performance tuning

• Profile application
• Using the accelerator hardware counters
• Analysis of data transfers
• Add data regions

The HPC Advisory Council and the Swiss National Supercomputing Centre will host  again the HPC Advisory Council Switzerland Conference 2012 in the

Lugano Convention Centre, Lugano, Switzerland,
March 13-15, 2012

The conference will focus on High-Performance Computing education, training (including hands-on) and overview of new developments. The conference will include the following sections per day:

• High Speed Networks
• High Performance and Parallel I/O
• Communication libraries: MPI, SHMEM, PGAS
• GPU computing, CUDA, OpenCL
• Big Data
• Advanced topics / Technologies / development / the road to Exascale
• Hands-on: clustering, network, troubleshooting, tuning, optimizations

It will bring together system managers, researchers, developers, computational scientists, students and industry affiliates for cross-training and to discuss recent HPC developments and future advancements.

Additional information »

Register now »

Brian tells us that the HPC Advisory Board is an open community that was started in 2008. Now, just three years later there are 270 members worldwide, a variety of ISV, OEM, independent Hardware vendors, as well as all the different end users both in academia and in research. All members come together for a common goal: being able to take advantage of HPC technologies and make them very easy to use.

According to Brian, CSCS is an important partner for the HPC Advisory Board for a number of reasons. CSCS has been the first center of excellence outside the USA, followed by China. These centers are basically providing tools and outbound activities to be able to promote HPC to users of all kinds. In the last two years the HPC Advisory Board together with CSCS have organized together three days hands-on workshops in Lugano.

The third annual workshop will be organized again with CSCS and will take place on March 13-15, 2012 in Lugano. The same place has been already booked  for the fourth workshop in 2013. This is a great workshop not only for people in Switzerland but also in the surrounding countries to get in and have nice three days of heavy information in terms of MPI, parallel programming languages, GPU, storage technology, network topologies, and some hand-on clustering. At the end there will also be a little competition and some great prizes.

Brian also recognizes the importance of hpc-ch as media partner for the HPC Advisory Council. hpc-ch covers not only the workshop in Lugano but also the workshop in the ISC time frame.

We look forward to meeting Brian and the HPC Advisory Board in March 2012 in Lugano.

We started by discussing the similarities between supercomputers (clusters) and his Java motorcycle he was exposing at his booth (yes, there are similarities). Rich told us about some highlights of the conference like the interactive Earth displayed at the NOAA booth. The discussion then continued to what makes HPC so interesting for us: Are these the latest GPU models or the applications of supercomputing to better understand the world?

Finally we closed the discussion looking at the HPC initiatives in Switzerland (CSCS has been the first center deploying a Cray XK6). Finally we said goodbye until March 2012 in Lugano at the HPC Advisory Workshop.

Torsten Hoefler is internationally regarded as one of the leading young scientists in the field of high-performance computing. At the University of Illinois, he is currently involved in the development of one of the world’s most efficient supercomputers. His research interests focus on system design, programming and efficiency analysis. Torsten Hoefler will provide the Department of Computer Science, the research focus “Scientific Computing and Simulation” and the CSCS (Swiss National Supercomputing Centre) with important stimuli.

According to Torsten’s blog he had the choice between “Juelich [as] *the* top supercomputing center in Europe and ETH [as] *the* top research university in (mainland) Europe (with people like Einstein as alumni)”. Torsten also adds “It was a very hard choice and I took some time to make it final”. Finally he concludes that “Zurich is probably one of the most beautiful cities in Europe and definitely one of (if not the) most expensive city . Public transit is just a dream, I believe one really doesn’t need a car around the city”.

We are happy to welcome Torsten to HPC in Switzerland!

The Mayo Clinic is a large clinical practice for medicine and is specialized on medical research and medical education. The new Cray XMT is being used to tackle different kinds of biomedical and bioinformatic problems that have been very challenging using traditional computer architectures.

In one research project Bob is going to work with the electronic medical records on a million of patients with millions of events. The researchers assume that there are secrets buried in the information that is very challenging to extract using traditional data mining techniques. One example is the discovery that weight loss drug Fen-phen can lead to heart valve damage. The relationship has accidentally been discovered by a Mayo clinic data analyst. A supercomputer could have helped researcher find earlier such a relationship.

There are other interesting and very valuable insights in the medical data that can not be mined out using a traditional relational data base management system. The Cray XMT is very good at graph analytics and can be used to find patterns that are very difficult to discover otherwise. The algorithms and the necessary data structures are being developed together with Cray and the Pacific Northwest National Laboratory. The first results of this common effort are expected in about one year from now.

At Mayo the Cray XMT is also being used to look at different ways to assemble the genomic sequences from the sequence data and to analyze the differences between individuals and the species references. Another possible application is free form text analysis. The clinic has a lot of free form text that is recorded as part of medical records that can be analyzed using the XMT to draw new conclusions.

CSCS is also deploying one of the first Cray XMT worldwide that is made available to research from different scientific fields.

Download the slides of the presentation (PDF) »

Download the slides of the presentation (PDF) »

Vincenzo Annaloro (CSCS) presents details on system installation and operation for two clusters: a production and experimental testbed that serves as HP2C prototype system and a multi-purpose visualization and GPU development cluster.

Download the slides of the presentation (PDF) »

A high-performance computing centre like the CSCS (Swiss National Supercomputing Centre) uses as much electricity every day as a small town. About a third of this electricity is used for cooling. If supercomputers are not constantly cooled, they overheat and, in the worst case scenario, may be damaged.

Three pumps move up to 760 litres of water per second
In conventional computing centres, compressors generate cold water to cool the computers. To save energy, CSCS will make use of a natural cooling resource: water extracted from Lake Lugano at 45 meters depth at a temperature of 6 degrees Celsius. This will greatly reduce the amount of electricity used by CSCS for cooling.

To operate the cooling system, a pumping station has been built on the lake shore in Parco Ciani, in the centre of Lugano. This is connected by a pipe to the 13 tonne, six metre high suction baskets which have been immersed at a depth of over forty metres. Through this pipe, three pumps in the pumping station will pump up to 760 litres of water per second. 460 litres will go to CSCS for cooling purposes, while up to 300 litres will be available to AIL for its water reservoir, which has yet to be built. On its way to CSCS, the water will climb 30 metres and cover a distance of 2.8 kilometres.

n the labyrinth of the pumping station: five metres beneath the lawns of Parco Ciani, over 700 litres of water per second are pumped out of Lake Lugano. (Picture: CSCS)

High-tech underground control centre
A steel ladder leads down to the high-tech underground control centre of the pumping station. The electrical infrastructure has been installed on steel mesh platforms about 2.5 metres above the ground, underneath this – covering a surface area of about 200 square metres –are the three pumps, a further backup pump, and two shock absorbing containers which will protect the pipeline from damage in the event of a sudden pressure surge.

On two sides of the pumping station, two massive pipes pierce the metre-thick walls. The pipeline connecting CSCS to the pumping station is 80 centimetres in diameter. On its way to CSCS, the water’s temperature will increase by up to 0.5°C. At the arrival point at CSCS, the lake water circuit and the internal cooling water circuit meet in man-sized heat exchangers. The internal water cooling circuit delivers the water, which has now reached 8 -9°C, to the supercomputers for cooling.

The suction strainers for the lake water pipe, just before they were lowered 45 metres into Lake Lugano. (Picture: CSCS)

One pumping operation for double the cooling effect
Once the water has passed through this first cooling circuit, it has been heated up by eight degrees. The now 16 to 17 °C is sent through a further heat exchanger, connected to a second cooling circuit. This mid-temperature circuit cools the air in the housings for the computers and hard drives of lesser energy density, that can therefore be cooled with water that is less cold. This means that with one pumping operation, two cooling is supplied to two circuits to cool two types of system.

The cold water pipe is designed to cool supercomputers of up to 14 megawatts on the first cooling circuit. The second circuit can cool  a further seven megawatts of computers. The more the second circuit is used, the higher the waste heat absorbed by the water and so the more useful it is to the local industrial works who will be able to use it.

Before the lake water returns to the lake, it passes through a stilling basin which can hold 120 cubic metres. The basin collects the water and makes sure that it flows freely down the return pipe back to the lake at a constant pressure and with no need for further power to be used. On the contrary, the plan is to use the energy generated as it falls to produce electricity. That is why connections for a microturbine have been provided in the pumping station.

So as not to affect the ecological balance of the lake, the water going back into the lake must never exceed 25 degrees Celsius. To ensure that this is always the case, a back-mixing funnel has been fitted which will add cold water if necessary.

(Article by Simone Ulmer, CSCS)

Additional reading: CSCS Media Release »