| Category | Difficulty |
|---|---|
| HW | 7 |
| Exams | 9 |
| Project | 6 |
Parallel is often called the most worth it class at CMU in CS. With the large breadth of content you learn, very modern tech you use in labs, and the ability to explore any application you'd like in your final project, there's a lot to like about this class.
The homeworks in this class are pretty fun. Besides the first HW, which gives an introduction to the types of parallel code out there, the other HWs are structured as
- Problem statement
- Technology you must use
- Goals for how good your parallel program is
As long as you and your partner can come up with a good approach and you give yourself sufficient time to debug it, you should be able to complete your assignment on time.
The only caveat is that sometimes your approach might not work. Therefore, it's important to come up with a good solution at the start, rather than implement a crappy program and have to redo it.
One of the hardest labs is the openMP lab, which is about message passing. The difficult part comes in that message passing is extremely slow, so communication is very expensive. A tip for this lab, while obvious, is to try to make your algorithm communicate as little as possible. And I mean it, like you might consider only 2 times of communication, once at the start of the program, and once at the end. This means you have to give all the workers enough info at the start to do all the work it needs until the end. Of course, you can have intermediary communication, but it will cost you time.
Because there is so much breadth of material in this class, exams can be very difficult, since you are almost guaranteed to come across material on the exam you didn't quite study all the way through. Therefore, unless you have a ton of time to study beforehand, make sure you can work through the exam quickly, so you can beef up your answers a bit more for partial credit. Exams are also quite long, so you will feel some time pressure as you work through it.
You're usually given a 3inx3in notecard, which is basically nothing, so you have to study up pretty well for the exam.
Some of the hardest concepts you'll be tested on are
- Synchronization constructs
- How to put
FENCESto make your multi-threaded code work properly - Guaranteeing thread safety with mutexes and barriers
- How to put
- Analyzing the efficiency of your parallel program with various metrics
- Understanding when to use a certain parallel approach over another
The final project is a fun time for you to explore an application of parallel. You can pretty much choose any space you want. The common themes for projects revolve around either multi-threading/processing with CPUs, or GPU threading. Some people choose to do mobile projects, other people choose to do web/server ones.
The project can be quite bland though, if you limit the technology you use. A very typical project is to take a game, maybe like chess, and parallelizing some sort of move tree search to make a chess bot. The hardware used is your own laptop. When you limit yourself to the hardware you have, the applications you can do are a lot less. Even using a slightly unfamiliar device like your phone might should be more interesting.
Some devices to look into:
- Computers with a lot of CPUs (clusters, AWS)
- GPUs (AWS machines)
- iPhone (Android is probably fine too, but not as fun to program)
- Raspberry Pi
- FPGA
With the Github Education Pack you already get AWS credits and a bunch more services, so I would explore that. CMU can also reimburse some of your projects expenses.
- The first homework is deceivingly difficult. It's very easy to get things working, but there are very subtle points in analyzing the parallelism of your code that you can lose points for. It's a good idea to consult a TA about your HW1 even if you feel pretty confident about it.
- As mentioned in Homework, the
openMPlab can be very difficult because of how expensive communication is. Try to start this early and be prepared for your first approach to get rekt. - Choose cool hardware to do parallel processing for your project. You'll learn more, be able to do more exciting things, and get a better grade.
- There's often a large waitlist for this class because of its popularity. If you have a good registration time, you might consider snagging a spot in this class.