Parallel and Distributed Computing

Learning Outcomes

Knowledge and Understanding

Having successfully completed this module, you will be able to demonstrate knowledge and understanding of:

• Analytically provide upper bounds on acceleration due to parallelism
• Explain the strengths and weaknesses of packet oriented and connection oriented systems
• Describe common internet networking protocols

Subject Specific Practical Skills

Having successfully completed this module you will be able to:

• Use standard networking protocols to communicate between software on two different machines
• Use multi-threading primitives to parallelise a shared-memory program
• Use GPU parallelism to accelerate a simple iteration space

Transferable and Generic Skills

Having successfully completed this module you will be able to:

• Describe the pros and cons of distributed systems for system reliability
• Explain how network encryption can improve system security
• Identify the environmental impact of parallel computing systems

Subject Specific Intellectual and Research Skills

Having successfully completed this module you will be able to:

• Analyse programs to identify opportunities for introducing concurrency and parallelism
• Decompose a high-level software system into loosely coupled portions
• Identify data races in shared data structures and protect them using threading primitives

Networking concepts:

• Connection vs connectionless approaches
• Streams vs routed packets
• Reliable/unreliable delivery
• Addresses and naming

Practical networking:

• IP networks (main protocols and services, _not_ the deep implementation)
• Sockets
• Clients and servers
• HW to SW channels

Parallel programming concepts:

• Multiple threads
• Race conditions and atomics
• Parallel computing using iteration spaces
• Introduction to GPUs

Distributed computing:

• Combining networks and parallel processing
• Load balancing
• MPI
• Map-reduce
• Efficiency and power consumption of distributed compute

• Lectures
• Guided self-study
• Labs as part of the AICE Lab Programme which will cover practical aspects

Summative

This is how we’ll formally assess what you have learned in this module.