Parallel programming is a computer programming technique that provides for the execution of operations in parallel, either within a single computer, or across a number of systems. In the latter case, the term distributed computing is used. A drawing of the everyday computer. ... Computer programming (often simply programming) is the craft of implementing one or more interrelated abstract algorithms using a particular programming language to produce a concrete computer program. ... In computer science, distributed computing studies the coordinated use of physically distributed computers. ...

Certain kinds of parallel programming can be accomplished in the lambda calculus, e.g., the parallel evaluation of arguments. However, the lambda calculus cannot in general express the concurrency of the Actor model and process calculi. For example the lambda calculus cannot express the concurrency of a shared financial account in which deposits are performed concurrently. However the Actor model and process calculi can express the parallelism in the lambda calculus. In computer science, the Actor model, first published in 1973, is a mathematical model of concurrent computation. ... The Process calculi (or process algebras) are a diverse family of related approaches to formally modelling concurrent systems. ... In computer science, the Actor model, first published in 1973, is a mathematical model of concurrent computation. ... The Process calculi (or process algebras) are a diverse family of related approaches to formally modelling concurrent systems. ...

In parallel programming, single tasks are split into a number of subtasks that can be computed relatively independently and then aggregated to form a single coherent solution. Parallel programming is most effective for tasks that can be easily broken down into independent tasks such as purely mathematical problems, e.g. factorization. Multiprocessor machines can often achieve better performance by taking advantage of this kind of programming. Multiprocessing is traditionally known as the use of multiple concurrent processes in a system as opposed to a single process at any one instant. ...

One way to achieve parallel programming is through distributed computing, which is a method of information processing in which work is performed by separate computers linked through a communications network. In computer science, distributed computing studies the coordinated use of physically distributed computers. ...

Parallel programming often relies on specialized algorithms, which allow problems to be split up into pieces. However, not all algorithms can be optimized to run in a distributed environment, often leading to different performance issues from single processor systems.

Major issues stem from trying to prevent concurrent processes from interfering with each other. Consider the following algorithm for a checking account:

1. bool withdraw(int withdrawal) {
2. if( balance > withdrawal ) {
3. balance -= withdrawal;
4. return true;
5. } else return false;
6. }

Suppose the balance=500, and two processes both call withdraw(300), and withdraw(350), concurrently. If line 2 in both operations executes before line 3, in both cases balance>withdrawal. However, more will end up being withdrawn than the balance. These sorts of issues require the use of Concurrency control, or non-blocking algorithms. In computer science -- more specifically, in the field of databases -- concurrency control is a method used to ensure that database transactions are executed in a safe manner (i. ... In concurrent programming, non-blocking algorithms are those algorithms designed to avoid requiring a critical section. ...

Pioneers in the field of concurrent programming include Edsger Dijkstra and C. A. R. Hoare. Portrait of Edsger Dijkstra (courtesy Brian Randell) Edsger Wybe Dijkstra (Rotterdam, May 11, 1930 – Nuenen, August 6, 2002) was a Dutch computer scientist. ... Sir Charles Antony Richard Hoare (Tony Hoare or C.A.R. Hoare, born January 11, 1934) is a British computer scientist, probably best known for the development of Quicksort, the worlds most widely used sorting algorithm, in 1960. ...

See also

• Parallel computing
• Parallel programming model
• Concurrent programming language
• Critical sections
• Mutual exclusion
• Synchronization
• Computer multitasking
• Multithreading
• Concurrency control
• Coroutines
• Parallel processor
• Completion ports
• Parawiki

Parallel computing is the simultaneous execution of the same task (split up and specially adapted) on multiple processors in order to obtain results faster. ... A parallel programming model is a set of software technologies to express parallel algorithms and match applications with the underlying parallel systems. ... Concurrent programming languages are programming languages that use language constructs for concurrency. ... In computer programming a critical section is a piece of code that can only be executed by one process or thread at a time. ... Mutual exclusion (often abbreviated to mutex) algorithms are used in concurrent programming to avoid the concurrent use of un-shareable resources by pieces of computer code called critical sections. ... Synchronization is coordination with respect to time. ... In computing, multitasking is a method by which multiple tasks, also known as processes, share common processing resources such as a CPU. In the case of a computer with a single CPU, only one task is said to be running at any point in time, meaning that the CPU is... Many programming languages, operating systems, and other software development environments support what are called threads of execution. ... In computer science -- more specifically, in the field of databases -- concurrency control is a method used to ensure that database transactions are executed in a safe manner (i. ... Coroutines are program components like subroutines. ... Parallel computing is the simultaneous execution of the same task (split up and specially adapted) on multiple processors in order to obtain faster results. ... // Introduction The Parawiki is a wiki dedicated to provide a single online resource for all questions regarding parallel computing. ...

External links

• "Multiprocessor Optimizations: Fine-Tuning Concurrent Access to Large Data Collections" by Ian Emmons
• "The Free Lunch Is Over: A Fundamental Turn Toward Concurrency in Software" by Herb Sutter
• Citations from CiteSeer
• Peterson algorithm