FAQs and Resources Parallel, Concurrent, and/or Distributed Computing?

There is a bit of confusion (or lack of clarity) in the computing space with the terms "concurrent computing", "parallel computing", and "distributed computing". They do overlap, and there are some distinctions.

Concurrent computing describes the process with which multiple workloads get operated on at the same time. For example, your computer maybe burning a DVD, browsing the web and doing virus checking at the same time. All these workloads are running concurrently - making visible progress from the user's perspective during any given second. But underneath, the workload may be time-multiplexing sequentially on the same core in the same processor in the same system, or running on multiple cores or multiple processors or multiple systems at the same time.

Parallel computing is a subset of concurrent computing, where the multiple workloads really are operating on different hardware resources at the same time. The different resources could be different cores or different processors or different systems.

Distributed computing is a subset of parallel computing where the execution takes place on computer systems that are distributed. This is usually motivated by the distributed nature of the location of data, where the computation can take place close to where the data is located.

Resource Links:

The HPC & GPU Supercomputing Group is a group for the application of cutting-edge HPC & GPU supercomputing technology to cutting-edge business problems.
HPC/GPU Supercomputing Group of Silicon Valley

Increasingly, you can leverage GPU power for any computationally-intense operation. GPU Computing Gems, edited by Wen-mei Hwu, provides a wealth of tested, proven GPU techniques. Chapter 37. Efficient Automatic Speech Recognition on the GPU is authored by Jike Chong.
GPU Computing Gems, the Emerald Edition

In taking a fresh approach to the longstanding parallel computing problem, Berkeley Par Lab's research agenda will be driven by compelling applications developed by domain experts.
The Parallel Computing Laboratory at University of California, Berkeley