Author Peter Pacheco uses a tutorial approach to show students how to develop effective parallel programs with MPI, Pthreads, and OpenMP. The first undergraduate text to directly address compiling and running parallel programs on the new multi-core and cluster architecture, An Introduction to Parallel Programming explains how to design, debug, and evaluate the performance of distributed and shared-memory programs. User-friendly exercises teach students how to compile, run and modify example programs.

• Takes a tutorial approach, starting with small programming examples and building progressively to more challenging examples
• Focuses on designing, debugging and evaluating the performance of distributed and shared-memory programs
• Explains how to develop parallel programs using MPI, Pthreads, and OpenMP programming models

How can we use this book to help us write parallel programs?

First, when you’re interested in high-performance, whether you’re writing serial or parallel programs, you need to know a little bit about the systems you’re working with--both hardware and software. In Chapter 2, we’ll give an overview of parallel hardware and software. In order to understand this discussion, it will be necessary to review some information on serial hardware and software. Much of the material in Chapter 2 won’t be needed when we’re getting started, so you might want to skim some of this material, and refer back to it occasionally when you’re reading later chapters.

The heart of the book is contained in Chapters 3 through 6. Chapters 3, 4, and 5 provide a very elementary introduction to programming parallel systems using C and MPI, Pthreads, and OpenMP, respectively. The only prerequisite for reading these chapters is a knowledge of C programming. We’ve tried to make these chapters independent of each other, and you should be able to read them in any order. However, in order to make them independent, we did find it necessary to repeat some material. So if you’ve read one of the three chapters, and you go on to read another, be prepared to skim over some of the material in the new chapter.

Chapter 6 puts together all we’ve learned in the preceding chapters, and develops two fairly large programs in both a shared- and a distributed-memory setting. However, it should be possible to read much of this even if you’ve only read one of Chapters 3, 4, or 5. The last chapter, Chapter 7, provides a few suggestions for further study on parallel programming.

Read a sample chapter from An Introduction to Parallel Programming

"Pacheco succeeds in introducing the reader to the key issues and considerations in parallel programming. The simplicity of the examples allows the reader to focus on parallel programming aspects rather than application logic. Including both MPI and Pthreads/OpenMP is a good way to illustrate the differences between message passing and shared-memory programming models. The discussions about analyzing the scalability and efficiency of the resulting parallel programs present a key aspect of developing real parallel programs. Finally, working through the same examples using all three facilities helps make this even more concrete." --W. Hu, ComputingReviews.com

"This is a well-written book, appropriately targeted at junior undergraduates. Being easily digestible, it makes the difficult task of parallel programming come across a lot less daunting than I have seen in other texts. Admittedly, it is light on theory; however, the most memorable lessons in parallel programming are those learned from mistakes made. With over 100 programming exercises, learning opportunities abound." --Bernard Kuc, ACM’s Computing Reviews.com

"With the coming of multicore processors and the cloud, parallel computing is most certainly not a niche area off in a corner of the computing world. Parallelism has become central to the efficient use of resources, and this new textbook by Peter Pacheco will go a long way toward introducing students early in their academic careers to both the art and practice of parallel computing." --Duncan Buell, Department of Computer Science and Engineering, University of South Carolina

"An Introduction to Parallel Programming illustrates fundamental programming principles in the increasingly important area of shared memory programming using Pthreads and OpenMP and distributed memory programming using MPI. More importantly, it emphasizes good programming practices by indicating potential performance pitfalls. These topics are presented in the context of a variety of disciplines including computer science, physics and mathematics. The chapters include numerous programming exercises that range from easy to very challenging. This is an ideal book for students or professionals looking to learn parallel programming skills or to refresh their knowledge." --Leigh Little, Department of Computational Science, The College at Brockport, The State University of New York

"An Introduction to Parallel Programming is a well written, comprehensive book on the field of parallel computing. Students and practitioners alike will appreciate the relevant, up-to-date information. Peter Pacheco’s very accessible writing style combined with numerous interesting examples keeps the reader’s attention. In a field that races forward at a dizzying pace, this book hangs on for the wild ride covering the ins and outs of parallel hardware and software." --Kathy J. Liszka, Department of Computer Science, University of Akron

"Parallel computing is the future and this book really helps introduce this complicated subject with practical and useful examples." --Andrew N. Sloss FBCS, Consultant Engineer, ARM, Author of ARM System Developer’s Guide.