FACTOID # 59: People might eat oats when they're hungry, but people from Hungary don't eat oats.
 
 Home   Encyclopedia   Statistics   Countries A-Z   Flags   Maps   Education   Forum   FAQ   About 
 
WHAT'S NEW
RECENT ARTICLES
More Recent Articles »
 

SEARCH ALL

FACTS & STATISTICS    Advanced view

Search encyclopedia, statistics and forums:

 

 

(* = Graphable)

 

 


Encyclopedia > Data parallelism

Data parallelism (also known as loop-level parallelism) is a form of parallelization of computing across multiple processors in parallel computing environments. Data parallelism focuses on distributing the data across different parallel computing nodes. It contrasts to task parallelism as another form of parallelism. Parallel computing is the simultaneous execution of the same task (split up and specially adapted) on multiple processors in order to obtain results faster. ... CPU redirects here. ... Parallel computing is the simultaneous execution of the same task (split up and specially adapted) on multiple processors in order to obtain results faster. ... Task Parallelism is a form of parallelization of computer code. ...

Contents

Description

In a multiprocessor system executing a single set of instructions (SIMD), data parallelism is achieved when each processor performs the same task on different pieces of distributed data. In some situations, a single execution thread controls operations on all pieces of data. In others, different threads control the operation, but they execute the same code.-1...


For instance, if we are running code on a 2-processor system (CPUs A and B) in a parallel environment, and we wish to do a task on some data D, it is possible to tell CPU A to do that task on one part of D and CPU B on another part simultaneously, thereby reducing the runtime of the execution. The data can be assigned using conditional statements as described below. As a specific example, consider adding two matrices. In a data parallel implementation, CPU A could add all elements from the top half of the matrices, while CPU B could add all elements from the bottom half of the matrices. Since the two processors work in parallel, the job of performing matrix addition would take one half the time of performing the same operation in serial using one CPU alone. CPU can stand for: in computing: Central processing unit in journalism: Commonwealth Press Union in law enforcement: Crime prevention unit in software: Critical patch update, a type of software patch distributed by Oracle Corporation in Macleans College is often known as Ash Lim. ... Parallel may refer to: Parallel (geometry) Parallel (latitude), an imaginary east-west line circling a globe Parallelism (grammar), a balance of two or more similar words, phrases, or clauses Parallel (manga), a shōnen manga by Toshihiko Kobayashi Parallel (video), a video album by R.E.M. The Parallel, an... In computer science, runtime or run time describes the operation of a computer program, the duration of its execution, from beginning to termination (compare compile time). ... In computer science, conditional statements are a vital part of a programming language. ...


Data parallelism emphasizes the distributed (parallelized) nature of the data, as opposed to the processing (task parallelism). Most real programs fall somewhere on a continuum between Task parallelism and Data parallelism. Task Parallelism is a form of parallelization of computer code. ...


Example

The pseudocode below illustrates data parallelism: Pseudocode (derived from pseudo and code) is a compact and informal high-level description of a computer programming algorithm that uses the structural conventions of some programming language, but typically omits details that are not essential for the understanding of the algorithm, such as subroutines, variable declarations and system-specific... 

 program: ... if CPU="a" then low_limit=1 upper_limit=50 else if CPU="b" then low_limit=51 upper_limit=100 end if do i = low_limit , upper_limit Task on d(i) end do ... end program

The goal of the program is to do some task on the data array "d" of size 100 (for example). If we write the code as above and launch it on a 2-processor system, then the runtime environment will execute it as follows.

  • In a SIMD system, both CPUs will execute the code.
  • In a parallel environment, both will have access to "d".
  • A mechanism is presumed to be in place whereby each CPU will create its own copy of "low_limit" and "upper_limit" that is independent of the other
  • The "if" clause differentiates between the CPUs. CPU "a" will read true on the "if" and CPU "b" will read true on the "else if", thus having their own values of "low_limit" and "upper_limit"
  • Now, both CPUs execute "Task on d(i)", but since each cpu has different values of the "limits", they operate on different parts of "d" simultaneously, thereby distributing the task among themselves. Obviously, this will be faster than doing it on a single CPU.

Code executed by CPU "a":-1... CPU can stand for: in computing: Central processing unit in journalism: Commonwealth Press Union in law enforcement: Crime prevention unit in software: Critical patch update, a type of software patch distributed by Oracle Corporation in Macleans College is often known as Ash Lim. ... 

 program: ... low_limit=1 upper_limit=50 do i = low_limit , upper_limit Task on d(i) end do ... end program

Code executed by CPU "b":

 program: ... low_limit=51 upper_limit=100 do i = low_limit , upper_limit Task on d(i) end do ... end program

This concept can now be generalized to any number of processors.


However, when the number of processors is more, it will be useful to code the above in the following way:


...

 low_limit = cpuid; /* cpuid ranges from 0 to (NCPUS-1) */
for(i=low_limit; i<100; i+=NCPUS)
{
 operate data[i]
}

...


If you consider this code on a 2 processor case, CPU A (cpuid 0) will operate on even entries and CPU B(cpuid 1) will operate on odd entries.


References

This article needs cleanup. ... Guy Lewis Steele, Jr. ... Communications of the ACM (CACM) is the flagship monthly magazine of the Association for Computing Machinery. ...

See also

 

COMMENTARY     


Share your thoughts, questions and commentary here
Your name
Your comments
Please enter the 5-letter protection code

Want to know more?
Search encyclopedia, statistics and forums:
 


Lesson Plans | Student Area | Student FAQ | Reviews | Press Releases |  Feeds | Contact
The Wikipedia article included on this page is licensed under the GFDL.
Images may be subject to relevant owners' copyright.
All other elements are (c) copyright NationMaster.com 2003-5. All Rights Reserved.
Usage implies agreement with terms.