Overview
Get an introduction to parallel programming using C and similar languages
This course gives beginner programmers an introduction to parallel programming.
Parallel programming describes the breaking down of a larger problem into smaller steps. Instructions are delivered to multiple processors, which will execute necessary calculations in parallel – hence the name.
On this course, you will explore the fundamentals of parallel programming using C, C++, Python, Fortran, Cuda/OpenCL or similar programming languages.
Learn to use OpenMP and MPI for parallel programming
On this course, you will:
- Learn how to use OpenMP (Open Multi-Processing) for incremental parallelisation within a single multicore shared memory paradigm
- Learn how to use the MPI (Message Passing Interface) message-passing standard to control communication between two processes, subroutines, or functions within a program
- Get an introduction to the OpenMPI library project, used by TOP500 supercomputers, which aims to create a single world-class open source MPI implementation based on best practice
- Learn how to express numerical problems in parallel programming paradigms, gaining an awareness of potential design and performance pitfalls in heterogeneous architectures Get an introduction to hybrid and accelerated paradigms (Cuda, OpenCL)
This course is delivered by the LECAD Laboratory at the University of Ljubljana, as part of PRACE (Partnership for Advanced Computing in Europe). The LECAD Laboratory emphasises the teaching of parallel programming alongside leading European HPC (high performance computing) centres.
This course is part of an introduction to programming from PRACE, targeting beginner programmers. It follows on from the Supercomputing course.
This course is hands-on. Some knowledge of C, C++, Fortran, Python, or similar programming languages would be useful.
Syllabus
- Introduction to parallel programming
- Intro to parallel programming
- OpenMP overview
- MPI overview
- Accelerators overview
- OpenMP
- Getting started with OpenMP
- Worksharing directives
- Data environment and combined constructs
- Tasking
- Message Passing Interface
- Communicators and communication in MPI
- Types of communication in MPI
- Collective communications
- Advanced Collective operations
- MPI Continued
- Advanced communication in MPI
- MPI + OpenMP
- User derived datatypes
- Parallel File I/O
- Beyond OpenMP and MPI – GPU parallelisation
- GPUs: Introduction and Architecture
- GPUs: Execution and programming models, Introduction to CUDA and OpenCL
- CUDA and OpenCL step-by-step
- Advanced examples on GPUs: Numerical integration
- Improving performance of GPU codes