NationMaster - Encyclopedia: Master theorem

In the analysis of algorithms, the master theorem, which is a specific case of the Akra-Bazzi theorem, provides a cookbook solution in asymptotic terms for recurrence relations of types that occur in practice. It was popularized by the canonical algorithms textbook Introduction to Algorithms by Cormen, Leiserson, Rivest, and Stein, which introduces and proves it in sections 4.3 and 4.4, respectively. Nevertheless, not all recurrence relations can be solved with the use of the master theorem. To analyze an algorithm is to determine the amount of resources (such as time and storage) necessary to execute it. ... In computer science, the Akra-Bazzi method, or Akra-Bazzi theorem, is used to analyze the asymptotic behavior of the mathematical recurrences that appear in the analysis of divide and conquer algorithms where the sub-problems have substantially different sizes. ... In mathematics and applications, particularly the analysis of algorithms, asymptotic analysis is a method of classifying limiting behaviour, by concentrating on some trend. ... In mathematics, a recurrence relation, also known as a difference equation, is an equation which defines a sequence recursively: each term of the sequence is defined as a function of the preceding terms. ... Cover of the second edition Introduction to Algorithms is a book by Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein. ... Thomas H. Cormen is the co-author of Introduction to Algorithms, along with Charles Leiserson, Ron Rivest, and Cliff Stein. ... Charles E. Leiserson is a computer scientist, specializing in the theory of parallel computing and distributed computing, and particularly practical applications thereof; as part of this effort, he developed the Cilk multithreaded language. ... Election People This box: Professor Ronald Lorin Rivest (born 1947, Schenectady, New York) is a cryptographer, and is the Andrew and Erna Viterbi Professor of Computer Science at MITs Department of Electrical Engineering and Computer Science (CSAIL). ... Clifford Stein is a computer scientist, currently working as a professor at Columbia University in New York, NY. He earned his BSE from Princeton University in 1987, a MS from Massachusetts Institute of Technology in 1989, and a PhD from Massachusetts Institute of Technology in 1992. ...

1 Generic form
2 Case 1
- 2.1 Generic form
- 2.2 Example
3 Case 2
- 3.1 Generic form
- 3.2 Example
4 Case 3
- 4.1 Generic form
- 4.2 Example
5 See also
6 References
7 External Links

Generic form

The master theorem concerns recurrence relations of the form:

$T(n) = a Tleft(frac{n}{b}right) + f(n) ;;;; mbox{where} ;; a geq 1 , b > 1.$

In the application to the analysis of a recursive algorithm, the constants and function take on the following significance:

n is the size of the problem.
a is the number of subproblems in the recursion.
n/b is the size of each subproblem. (Here it is assumed that all subproblems are essentially the same size.)
f (n) is the cost of the work done outside the recursive calls, which includes the cost of dividing the problem and the cost of merging the solutions to the subproblems.

It is possible to determine an asymptotic tight bound in these three cases:

Case 1

Generic form

If it is true that:

$f(n) in mathcal{O}left( n^{log_b a - epsilon} right)$ for some constant

ε > 0

it follows that:

$T(n) in Thetaleft( n^{log_b a} right).$

Example

$T(n) = 8 Tleft(frac{n}{2}right) + 1000n^2$

As you can see in the formula above, the variables get the following values:

$a = 8 ,$ , $b = 2 ,$ , $f(n) = 1000n^2 ,$ , $log_b a = log_2 8 = 3 ,$

Now you have to check that the following equation holds:

$f(n) in mathcal{O}left( n^{log_b a - epsilon} right)$

If you insert the values from above, you get:

$1000n^2 in mathcal{O}left( n^{3 - epsilon} right)$

If you choose $ε$ = 1, you get:

$1000n^2 in mathcal{O}left( n^{3 - 1} right) = mathcal{O}left( n^{2} right)$

Since this equation holds, the first case of the master theorem applies to the given recurrence relation, thus resulting in the conclusion:

$T(n) in Thetaleft( n^{log_b a} right).$

If you insert the values from above, you finally get:

$T(n) in Thetaleft( n^{3} right).$

Thus the given recurrence relation T(n) was in Θ(n³)

Case 2

Generic form

If it is true that:

$f(n) in Thetaleft( n^{log_b a}(log n)^k right)$ for some $k geq 0$

it follows that:

$T(n) in Thetaleft( n^{log_b a}(log n)^{k+1}right).$

Example

$T(n) = 2 Tleft(frac{n}{2}right) + 10n$

As you can see in the formula above the variables get the following values:

$a = 2 ,$ , $b = 2 ,$ , $k = 0 ,$ , $f(n) = 10n ,$ , $log_b a = log_2 2 = 1 ,$

Now you have to check that the following equation holds:

$f(n) in Thetaleft( n^{log_b a} right)$

If you insert the values from above, you get:

$10n in Thetaleft( n^{1} right) = Thetaleft( n right)$

Since this equation holds, the second case of the master theorem applies to the given recurrence relation, thus resulting in the conclusion:

$T(n) in Thetaleft( n^{log_b a} log nright).$

If you insert the values from above, you finally get:

$T(n) in Thetaleft( n log nright).$

Thus the given recurrence relation T(n) was in Θ(n log n).

Case 3

Generic form

If it is true that:

$f(n) in Omegaleft( n^{log_b a + epsilon} right)$ for some constant

ε > 0

and if it is also true that:

$a fleft( frac{n}{b} right) le c f(n)$ for some constant

c < 1

and sufficiently large n

it follows that:

$T(n) in Theta(f(n)).$

Example

$T(n) = 2 Tleft(frac{n}{2}right) + n^2$

As you can see in the formula above the variables get the following values:

$a = 2 ,$ , $b = 2 ,$ , $f(n) = n^2 ,$ , $log_b a = log_2 2 = 1 ,$

Now you have to check that the following equation holds:

$f(n) in Omegaleft( n^{log_b a + epsilon} right)$

If you insert the values from above, and choose $ε$ = 1, you get:

$n^2 in Omegaleft( n^{1 + 1} right) = Omegaleft( n^2 right)$

Since this equation holds, you have to check the second condition, namely if it is true that:

$a fleft( frac{n}{b} right) le c f(n)$

If you insert once more the values from above, you get:

$2 left( frac{n}{2} right)^2$ $le c n^2 Leftrightarrow frac{1}{2} n^2 le cn^2$

If you choose $c = frac{1}{2}$ , it is true that:

$frac{1}{2} n^2 le frac{1}{2} n^2$ $forall n ge 1$

So it follows:

$T(n) in Theta(f(n)).$

If you insert once more the necessary values, you get:

$T(n) in Theta(n^2).$

Thus the given recurrence relation T(n) was in Θ(n²), that complies with the f (n) of the original formula.

References

Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein. Introduction to Algorithms, Second Edition. MIT Press and McGraw-Hill, 2001. ISBN 0-262-03293-7. Sections 4.3 (The master method) and 4.4 (Proof of the master theorem), pp.73–90.

Thomas H. Cormen is the co-author of Introduction to Algorithms, along with Charles Leiserson, Ron Rivest, and Cliff Stein. ... Charles E. Leiserson is a computer scientist, specializing in the theory of parallel computing and distributed computing, and particularly practical applications thereof; as part of this effort, he developed the Cilk multithreaded language. ... Professor Ron Rivest Professor Ronald Linn Rivest (born 1947, Schenectady, New York) is a cryptographer, and is the Viterbi Professor of Computer Science at MITs Department of Electrical Engineering and Computer Science. ... Clifford Stein is a computer scientist, currently working as a professor at Columbia University in New York, NY. He earned his BSE from Princeton University in 1987, a MS from Massachusetts Institute of Technology in 1989, and a PhD from Massachusetts Institute of Technology in 1992. ... Cover of the second edition Introduction to Algorithms is a book by Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein. ...

External Links

Stony Brook CS Dept Notes for an explanation of why the Master Theorem works

Categories: Asymptotic analysis | Mathematical theorems | Recurrence relations

COMMENTARY

Share your thoughts, questions and commentary here

Want to know more?
Search encyclopedia, statistics and forums:

Feeds | Contact
The Wikipedia article included on this page is licensed under the GFDL.
Images may be subject to relevant owners' copyright.
All other elements are (c) copyright NationMaster.com 2003-5. All Rights Reserved.
Usage implies agreement with terms.

Contents

Generic form GA_googleFillSlot("encyclopedia_square");

Case 1

Generic form

Example

Case 2

Generic form

Example

Case 3

Generic form

Example

See also

References

External Links

Generic form