We then prove T(n)≤cnlgn for some suitable c>0 (to find c)
T(n)=2T(⌊n/2⌋)+n≤2(c⌊n/2⌋lg⌊n/2⌋)+n=cnlg(n/2)+n=cnlgn−cnlg2+n≤cnlgnsubstituting inductive assumptionby removing floor, as ⌊n⌋≤nand we want to find upper boundas lg(a/b)=lga−lgbif c≥1 and n≥0
With respect to the last step, we know that cnlgn−cnlg2+n≤cnlgn if and only if cnlg2+n is non-zero and positive.
Therefore, we add the constraint c≥1,n≥0
We must also have c≥2 to satisfy the base cases, so n0=2 and c=2 will suffice\
1.1.3 - Substitution Method - Guessing
Consider the following recurrence:
T(n)=2T(⌊n/2⌋)+17+n
The difference between:
T(⌊n/2⌋)andT(⌊n/2⌋)+17
Since the difference is small, we can guess the same solution.
We can then guess loose upper and lower bounds, then refine:
T(n)=Ω(n)T(n)=O(n2)
Try to lower the upper bound and raise the lower bound until convergence to get a sufficiently tight bound
1.1.4 - Substitution Method - Strengthening the Guess
🌱 We can “Strengthen the Guess” by removing a lower-order term.
Consider:
T(n)=T(⌊n/2⌋)+T(⌈n/2⌉)+1
We guess T(n)=O(n)
In our inductive step, we assume that T(⌊n/2⌋)≤c⌊n/2⌋ and T(⌈n/2⌉)≤c⌈n/2⌉ and attempt to prove T(n)≤cn
We add the conditions (to the right) if __ > 1 to ensure that we don’t hit the base case and therefore can keep expanding.
We want to develop a generic form for the expression above, if we were to expand it some arbitrary amount of times, i.
We can continue to expand, if the condition ⌊4i−1n⌋>1 holds true.
=n+3⌊n/4⌋+32⌊n/42⌋+⋯+3iT(⌊n/4i⌋)if⌊4i−1n⌋>1
For which value of i do we stop expanding the recurrence?
For the smallest value of i, such that ⌊n/4i⌋≤1 (i.e., when we hit the base case)
Imprecisely Assuming that T(n)=Θ(1) for n≤1, we stop expanding when
≡≡⌊n/4i⌋≈14i≈ni≈log4n
Therefore, we expand out the occurrence roughly log4n times.
More precisely, we must have i≤⌈log4n⌉ since ⌊n/4⌈log4n⌉⌋≤1
From this, it follows that:
T(n)=n+3⌊n/4⌋+32⌊n/42⌋+⋯+3iT(⌊n/4i⌋)≤n+3⌊n/4⌋+32⌊n/42⌋+⋯+3⌈log4n⌉Θ(1)≤n+43n+4232n+433n3+⋯+Θ(3log4n)≤ni=0∑∞(43)i+Θ(3log4n)=1−431+O(3log4n)=4n+O(nlog43)=4n+O(n)=O(n)if ⌊4i=1n⌋>1as i≤⌈log4n⌉n=⌊n/4i⌋⇒T(n)=Θ(1)as finding upper boundas a summationusing sum formulaas alogb(n)=nlogb(a)as log4n<1
Observe that when we are trying to find the upper bound, we can drop the floors and ceilings.
Additionally, though the summation is not infinite, the infinite summation is an upper bound on it.
The iteration tree for the recurrence T(n)=n=3T(⌊n/4⌋) is given as shown below - we can perform the same iterative analysis as above by constructing a recurrence tree.
Once again, we can repeat this analysis until we see a pattern start to develop.
To determine what summation describes, we need to determine two things:
When do we stop expanding the summation (described by the height of the tree) - in this case, this is given by ⌈log4n⌉
How much work is performed at each level of the tree.
In this, the height of the tree is essentially how many iterations we perform, and the amount of work done at each level is the value we are summing in each iteration
1.2.2 - Iteration Method Notes
This technique has a few issues:
Lots of math
You have to work out all of the terms, when to stop, and to sum what you get
Sometimes you can only start the iteration, and guess the answer
If you guess correctly, you can abandon the maths and revert to substitution
Floors and ceilings can be troublesome
Workaround - Assume that n has the correct form to delete them (as in the previous example, where n=4k
Technical abuse often work well (OK if the function is well-behaved)
Start trying to work out the solution if they weren’t there, and come back and ‘fix it up’
Taking n=i, we have T(n−i)=T(0)∈Θ(1) (reached the base case) and so:
T(n)=j=0∑n−12j+2nT(0)=2n−1+2nΘ(1)=Θ(2n)
1.3 - Master Method
Need to memorise 3 cases for solving some recurrences of the form:
T(n)=aT(n/b)+f(n)
where n/b can be ⌊n/b⌋ or ⌈n/b⌉
In each case, we compare nlogba with f(n)
nlogba is polynomially larger than f(n) → Solution is Θ(nlogba)
nlogba is same tight asymptotic bound as f(n) → Solution is Θ(nlogbalgn)
f(n) is polynomially larger thannlogba and f(n) is regular → Solution is Θ(f(n))
1.3.1 - Polynomially Larger Than
A function f(n) is polynomially larger thang(n) when
f(n)∈Ω(g(n)×nϵ)for some ϵ>0
Equivalently:
g(n)f(n)∈Ω(nϵ)for some ϵ>0
In which f(n) is polynomially larger than g(n) by polynomial nϵ
Example:
Is f(n)=n2 polynomially larger than g(n)=n3/2?
g(n)f(n)=n3/2n2=n2−3/2=n1/2
Yes, as n2∈Ω(n3/2×n1/2)
Is f(n)=log2n polynomially larger than g(n)=1?
No, as log2n∈Ω(1×nϵ) for any ϵ>0
log2n is larger than, but it is not polynomially larger than
1.3.2 - Regularity
Function f(n) is regular when:
af(n/b)<cf(n)for some c<1
Most functions satisfy regularity, but still need to check
1.3.3 - Formal Definition of Master Method
Given recurrences of the form:
T(n)=aT(bn)+f(n)
case 1T(n)∈Θ(nlogba)iff(n)∈O(nlogba−ϵ)for some ϵ>0case 2T(n)∈Θ(nlogbalgn)iff(n)∈Θ(nlogba)for some ϵ>0case 3T(n)∈Θ(f(n))iff(n)∈Ω(nlogba+ϵ)for some ϵ>1and af(bn)≤cf(n)for some c<1
We can re-define the master method, using the division syntax.
case 1T(n)∈Θ(nlogba)ifnlogba−ϵf(n)∈O(n−ϵ)for some ϵ>0case 2T(n)∈Θ(nlogbalgn)ifnlogbaf(n)∈Θ(1)for some ϵ>0case 3T(n)∈Θ(f(n))ifnlogba+ϵf(n)∈Ω(nϵ)for some ϵ>1and af(bn)≤cf(n)for some c<1
1.3.4 - Application of the Master Theorem:
Recurrence: T(n)=2T(2n)+f(n)where f(n)∈Θ(n)
The master method works on recurreneces of the form
T(n)=aT(bn)+f(n)
a=2,b=2
nlogbaf(n)∈Θ(nlog22n)=Θ(n1n)=Θ(1)
Therefore, the second case applies, and the solution is Θ(nlog22lgn)=Θ(nlgn)
Recurrence: T(n)=1T(2n)+f(n)where f(n)∈Θ(1)
The master method works on recurreneces of the form
T(n)=aT(bn)+f(n)
a=1,b=2
nlogbaf(n)∈Θ(nlog211)=Θ(11)=Θ(1)
Therefore, the second case applies, and the solution is Θ(nlog21lgn)=Θ(n0lgn)=Θ(lgn)
Recurrence: T(n)=7T(2n)+f(n)where f(n)∈Θ(n2)
The master method works on recurreneces of the form
T(n)=aT(bn)+f(n)
a=7,b=2
nlogbaf(n)∈Θ(nlog27n2)=Θ(n−(log27−2))
Additionally, ϵ=log27−2>0
Since 4<7<8, we know that log24<log27<log28 and therefore, 2<log27<3
Hence, case 1 applies and T(n)∈Θ(nlogba)=Θ(nlog27)≈Θ(n2.81)
This is slightly better than the average solution of matrix multiplication, which is Θ(n3)
Recurrence: T(n)=4T(2n)+f(n)where f(n)∈Θ(n3)
The master method works on recurrences of the form
T(n)=aT(bn)+f(n)
a=4,b=2
nlogbaf(n)∈Θ(nlog24n3)=Θ(n2n3)=Θ(n)
Case 3 of the master method applies, as long as we can satisfy the regularity condition.
af(n/b)≤cf(n)for some c<1
That is,
af(bn)=4(2n)3=48n3=21n3≤cn3
For some choice of c=21<1
1.3 - Extended Master Method
Fills in a gap in the Master Theorem
If f(n)∈Θ(nlogbalgkn)
f(n) is larger, but not polynomially larger
Solution is Θ(nlogbalgk+1n)
The above case 2 is a special case of this, when k=0