2. Amortized Analysis
What is Amortized Analysis ?
In amortized analysis, the time required to perform
a sequence of operations is averaged over all the
operations performed.
ļ· No involvement of probability
ļ·Average performance on a sequence of operations,
even some operation is expensive.
ļ·Guarantee average performance of each operation
among the sequence in worst case.
Amortized analysis is not just an analysis tool, it is also a way of
thinking about designing algorithms.
2017/05/15 2Amortized Analysis
3. Amortized Analysis
Methods of Amortized Analysis
ļ± Accounting Method: we overcharge
some operations early and use them
to as prepaid charge later.
ļ± Aggregate Method: we determine an
upper bound T(n) on the total sequence
of n operations. The cost of each will
then be T(n)/n.
ļ± Potential Method: we maintain credit
as potential energy associated with
the structure as a whole.
2017/05/15 3Amortized Analysis
4. Amortized Analysis
1. Aggregate Method
Show that for all n, a sequence of n operations take
worst-case time T(n) in total
In the worst case, the average cost, or amortized cost ,
per operation is T(n)/n.
The amortized cost applies to each operation, even
when there are several types of operations in the
sequence.
2017/05/15 4Amortized Analysis
6. Amortized Analysis
Sequence of n push, pop, Multipop operations
ļ±Worst-case cost of Multipop is O(n)
ļ±Have n operations
ļ±Therefore, worst-case cost of sequence is O(n2
)
Observations
ļ±Each object can be popped only once per time that itās
pushed
ļ±Have <= n pushes => <= n pops, including those in
Multipop
ļ±Therefore total cost = O(n)
ļ±Average over n operations => O(1) per operation on
average
Notice that no probability involved
ā¦ā¦. Aggregate Analysis: Stack Example
2017/05/15 6Amortized Analysis
7. Amortized Analysis
2. Accounting Method
Charge i th operation a fictitious amortized cost Äi,
where $1 pays for 1 unit of work (i.e., time).
ļ±Assign different charges (amortized cost ) to different
operations
ļ§ Some are charged more than actual cost
ļ§ Some are charged less
This fee is consumed to perform the operation.
Any amount not immediately consumed is stored in the
bank for use by subsequent operations.
The bank balance (the credit) must not go negative!
We must ensure that
for all n.
āā
==
ā¤
n
i
i
n
i
i cc
11
Ė
Thus, the total amortized costs provide an upper
bound on the total true costs.
2017/05/15 7Amortized Analysis
8. Amortized Analysis
3 ops:
Push(S,x) Pop(S) Multi-pop(S,k)
ā¢Assigned
cost:
2 0 0
ā¢Actual cost: 1 1 min(|S|,k)
Push(S,x) pays for possible later pop of x.
ā¦.. Accounting Method: Stack Example
2017/05/15 8Amortized Analysis
9. Amortized Analysis
ā¦.. Accounting Method: Stack Example
When pushing an object, pay $2
ļ±$1 pays for the push
ļ±$1 is prepayment for it being popped by either pop
or Multipop
ļ±Since each object has $1, which is credit, the
credit can never go negative
ļ±Therefore, total amortized cost = O(n), is an upper
bound on total actual cost
2017/05/15 9Amortized Analysis
10. Amortized Analysis
ā¦.. Accounting Method: Binary Counter
k-bit Binary Counter: A[0..kā1]
INCREMENT(A)
1. i ā 0
2. while i < length[A] and A[i] = 1
3. do A[i] ā 0 ā³ reset a bit
4. i ā i + 1
5. if i < length[A]
6. then A[i] ā 1 ā³ set a bit
ā ā = ā
=
1
0 2][k
i
i
iAx
Introduction
2017/05/15 10Amortized Analysis
11. Amortized Analysis
Consider a sequence of n increments. The
worst-case time to execute one increment is
Ī(k). Therefore, the worst-case time for n
increments is n Ā· Ī(k) = Ī(nā k).
WRONG! In fact, the worst-case cost for n
increments is only Ī(n) āŖ Ī(nā k).
Letās see why. Note: Youād be correct
if youād said O(nā k).
But, itās an overestimate.
ā¦.. Accounting Method: Binary Counter
2017/05/15 11Amortized Analysis
14. Amortized Analysis
Example:
Charge an amortized cost of $2 every time a bit is set from 0 to 1
ā¢ $1 pays for the actual bit setting.
ā¢ $1 is stored for later re-setting (from 1 to 0).
At any point, every 1 bit in the counter has $1 on itā¦ that pays for
resetting it. (reset is āfreeā)
0 0 0 1$1
0 1$1
0
0 0 0 1$1
0 1$1
1$1
0 0 0 1$1
1$1
0 0
Cost = $2
Cost = $2
ā¦.. Accounting Method: Binary Counter
2017/05/15 14Amortized Analysis
15. Amortized Analysis
When Incrementing,
ļ±Amortized cost for line 3 = $0
ļ±Amortized cost for line 6 = $2
Amortized cost for INCREMENT(A) = $2
Amortized cost for n INCREMENT(A) = $2n =O(n)
INCREMENT(A)
1. i ā 0
2. while i < length[A] and A[i] = 1
3. do A[i] ā 0 ā³ reset a bit
4. i ā i + 1
5. if i < length[A]
6. then A[i] ā 1 ā³ set a bit
ā¦.. Accounting Method: Binary Counter
2017/05/15 15Amortized Analysis
16. Amortized Analysis
Accounting Method vs. Aggregate Method
Aggregate Method :
First analyze entire sequence
Then calculate amortized cost per operation
2017/05/15 16Amortized Analysis
Accounting method:
First assign amortized cost per operation
Check that they are valid .
Then compute cost of entire sequence of
operations
17. Amortized Analysis
3. Potential Method
IDEA: View the bank account as the
potential energy (as in physics) of the
dynamic set.
FRAMEWORK:
Start with an initial data structure D0.
Operation i transforms Diā1 to Di.
The cost of operation i is ci.
Define a potential function Ī¦ : {Di} ā R,
such that Ī¦(D0 ) = 0 and Ī¦(Di ) ā„ 0 for all i.
The amortized cost Äi with respect to Ī¦ is
defined to be Äi = ci + Ī¦(Di) ā Ī¦(Diā1).
2017/05/15 17Amortized Analysis
18. Amortized Analysis
Like the accounting method, but think of the
credit as potential stored with the entire data
structure.
ļ± Accounting method stores credit with
specific objects while potential method
stores potential in the data structure as a
whole.
ļ± Can release potential to pay for future
operations
Most flexible of the amortized analysis
methods .
ā¦.. Potential Method
2017/05/15 18Amortized Analysis
19. Amortized Analysis
Äi = ci + Ī¦(Di) ā Ī¦(Diā1)
potential difference āĪ¦i
ļ± If āĪ¦i > 0, then Äi > ci. Operation i
stores work in the data structure for later
use.
ļ± If āĪ¦i < 0, then Äi < ci. The data
structure delivers up stored work to help
pay for operation i.
ā¦.. Potential Method
2017/05/15 19Amortized Analysis
20. Amortized Analysis
The total amortized cost of n operations is
Summing both sides telescopically.
ā¦.. Potential Method
)()( 0
1
DDc n
n
i
i Ī¦āĪ¦+= ā=
( )āā =
ā
=
Ī¦āĪ¦+=
n
i
iii
n
i
i DDcc
1
1
1
)()(Ė
ā=
ā„
n
i
ic
1
since Ī¦(Dn) ā„ 0 and Ī¦(D0 ) = 0.
2017/05/15 20Amortized Analysis
22. Amortized Analysis
ā¦.. Potential Method: Binary Counter
Define the potential of the counter after the ith
operation
by Ī¦(Di) = bi, the number of 1ās in the counter after the ith
operation.
Note:
ā¢ Ī¦(D0 ) = 0,
ā¢ Ī¦(Di) ā„ 0 for all i.
Example:
0 0 0 1 0 1 0
0 0 0 1$1
0 1$1
0 Accounting method)(
2017/05/15 22Amortized Analysis
23. Amortized Analysis
ā¦.. Potential Method
The amortized cost of the i th INCREMENT is
Äi = ci + Ī¦(Di) ā Ī¦(Diā1)
= (ti + 1) + (1 ā ti)
= 2
Assume ith INCREMENT resets ti bits (in line 3).
Actual cost ci = (ti + 1)
Number of 1ās after ith operation: bi = biā1 ā ti + 1
Therefore, n INCREMENTs cost Ī(n) in the worst case.
2017/05/15 23Amortized Analysis
24. Amortized Analysis
ā¦.. Dynamic Tables
Problem: if too many items inserted, table may be too
small
Implementing a table (e.g., hash table) for dynamic
data, want to make it small as possible
Idea: allocate more memory as needed
2017/05/15 24Amortized Analysis
25. Amortized Analysis
In some applications:
ļ±We don't know how many objects will be stored in a table.
Similarly, if many objects are deleted from the table:
ļ±It may be worthwhile to reallocate the table with a smaller size .
This problem is called
ļ±Dynamically Expanding and Contracting a table.
ā¦.. Dynamic Tables: Why Dynamic Tables?
2017/05/15 25Amortized Analysis
26. Amortized Analysis
Using amortized analysis we will show that :
ļ±The amortized cost of insertion and deletion is O(1).
ļ±Even though the actual cost of an operation is large when it triggers an
expansion or a contraction.
We will also show how to guarantee that:
ļ±The unused space in a dynamic table never exceeds a constant fraction of
the total space.
Operations on Dynamic Table
ļ±TABLE-INSERT: Inserts into the table an item that occupies a single slot.
ļ±TABLE-DELETE: Removes an item from the table & frees its slot.
Load Factor: For empty its 1 and for non empty its given below
ā¦.. Dynamic Tables:
2017/05/15 26Amortized Analysis
tabletheof)(
tablein thestoreditemsofNumber
)(
slotsofnumbersize
T =Ī±
27. Amortized Analysis
Assumption:
ļ±Table is allocated as an array of slots.
A table fills up when:
ļ±All slots have been used.
ļ±Equivalently, when its load factor becomes 1.
Table-Expansion occurs when:
ļ±An item is to be inserted into a full table.
A Common Heuristic:
ļ±Allocate a new table that has twice as many slots as the old one.
Hence, insertions are performed if only :
ā¦.. Insertion-Only Dynamic Tables:
2017/05/15 27Amortized Analysis
1 / 2 ā¤ Ī±(T) ā¤ 1
28. Amortized Analysis
Table Insert:
ā¦.. Insertion-Only Dynamic Tables:
2017/05/15 28Amortized Analysis
1 / 2 ā¤ Ī±(T) ā¤ 1
TABLE-INSERT (T, x)
1. if size[T] = 0 then
2. allocate table[T] with 1 slot
3. size[T] ā 1
4. if num[T] = size[T] then
5. allocate new-table with 2.size[T] slots
6. copy all items in table[T] into new-table
7. free table[T]
8. table[T] ā new-table[T]
9. size[T] ā 2.size[T]
10. insert x into table[T]
11. num[T] ā num[T] + 1
12. end
table[T] : pointer to block
of table storage
num[T] : number of
items in the table
size[T] : total number of
slots in the table
Initially, table is empty,
so num[T] = size[T] = 0
29. ā Let ci = cost of ith insert
ā ci = i if i-1 is exact power of 2, 1 otherwise
ā Example:
ā Operation Table Size Cost
Insert(1) 1 1 1
Amortized Analysis
2017/05/15 27Amortized Analysis
30. ā Let ci = cost of ith insert
ā ci = i if i-1 is exact power of 2, 1 otherwise
ā Example:
ā Operation Table Size Cost
Insert(1) 1 1 1
Insert(2) 2 1 + 1 2
Amortized Analysis
2017/05/15 28Amortized Analysis
31. ā Let ci = cost of ith insert
ā ci = i if i-1 is exact power of 2, 1 otherwise
ā Example:
ā Operation Table Size Cost
Insert(1) 1 1 1
Insert(2) 2 1 + 1 2
Insert(3) 4 1 + 2 3
Amortized Analysis
2017/05/15 29Amortized Analysis
32. ā Let ci = cost of ith insert
ā ci = i if i-1 is exact power of 2, 1 otherwise
ā Example:
ā Operation Table Size Cost
Insert(1) 1 1 1
Insert(2) 2 1 + 1 2
Insert(3) 4 1 + 2 3
Insert(4) 4 1 4
Amortized Analysis
2017/05/15 30Amortized Analysis
33. Let ci = cost of ith insert
ci = i if i-1 is exact power of 2, 1 otherwise
Example:
ā Operation Table Size Cost
Insert(1) 1 1 1
Insert(2) 2 1 + 1 2
Insert(3) 4 1 + 2 3
Insert(4) 4 1 4
Insert(5) 8 1 + 4 5
Amortized Analysis
2017/05/15 31Amortized Analysis
34. ā Let ci = cost of ith insert
ā ci = i if i-1 is exact power of 2, 1 otherwise
ā Example:
ā Operation Table Size Cost
Insert(1) 1 1 1
Insert(2) 2 1 + 1 2
Insert(3) 4 1 + 2 3
Insert(4) 4 1 4
Insert(5) 8 1 + 4 5
Insert(6) 8 1 6
Amortized Analysis
2017/05/15 32Amortized Analysis
35. ā Let ci = cost of ith insert
ā ci = i if i-1 is exact power of 2, 1 otherwise
ā Example:
ā Operation Table Size Cost
Insert(1) 1 1 1
Insert(2) 2 1 + 1 2
Insert(3) 4 1 + 2 3
Insert(4) 4 1 4
Insert(5) 8 1 + 4 5
Insert(6) 8 1 6
Insert(7) 8 1 7
Amortized Analysis
2017/05/15 33Amortized Analysis
36. ā Let ci = cost of ith insert
ā ci = i if i-1 is exact power of 2, 1 otherwise
ā Example:
ā Operation Table Size Cost
Insert(1) 1 1 1
Insert(2) 2 1 + 1 2
Insert(3) 4 1 + 2 3
Insert(4) 4 1 4
Insert(5) 8 1 + 4 5
Insert(6) 8 1 6
Insert(7) 8 1 7
Insert(8) 8 1 8
Amortized Analysis
2017/05/15 34Amortized Analysis
37. ā Let ci = cost of ith insert
ā ci = i if i-1 is exact power of 2, 1 otherwise
ā Example:
ā Operation Table Size Cost
Insert(1) 1 1 1
Insert(2) 2 1 + 1 2
Insert(3) 4 1 + 2
Insert(4) 4 1
Insert(5) 8 1 + 4
Insert(6) 8 1
Insert(7) 8 1
Insert(8) 8 1
Insert(9) 16 1 + 8
1
2
3
4
5
6
7
8
9
Amortized Analysis
2017/05/15 35Amortized Analysis
38. Amortized Analysis
1. Aggregate Method
ļ§Table is initially empty.
ļ§Observe:
o i-th operation causes an expansion only when i-1
is an exact power of 2.
2017/05/15 38Amortized Analysis
ļ£³
ļ£²
ļ£±
=
otherwise1
2ofpowerexactanisif ii
ci
39. Amortized Analysis
ā¦Aggregate Method
ļ§Therefore the total cost of n TABLE-INSERT
operations
ļ§ The amortized cost of a single operation is 3n/n=3 = O(1)
2017/05/15 39Amortized Analysis
ļ£° ļ£»
nnnnnc
n
j
j
n
j
j
n
i
i 3222
lg
0
lg
01
=+=+<+= āāā ===
i 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 ...
ci
1 2 3 1 5 1 1 1 9 1 1 1 1 1 1 1 17 1 1 1 ...
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 ...
Expansion
cost
1 2 4 8 16
40. Amortized Analysis
Assign the following amortized costs
ā Table-Expansion : 0
ā Insertion of a new item : 3
Insertion of a new item
a) 1 (as an actual cost) for inserting itself into the table
b) 1 (as a credit) for moving itself in the next expansion
c) 1 (as a credit) for moving another item (in the next
expansion) that has already moved in the last expansion
2017/05/15 40Amortized Analysis
ā¦ Accounting Method: Dynamic Tables
41. Amortized Analysis
Size of the table: M
Immediately after an expansion (just before the insertion)
num[T] = M/2 and size[T] = M where M is a power of 2.
Table contains no credits
2017/05/15 41Amortized Analysis
ā¦Accounting Method : Dynamic Tables
43. Amortized Analysis
M/2th Insertion
Thus, by the time the table contains M items and
is full
ā each item in the table has $1 of credit to pay for its
move during the next expansion
2017/05/15 43Amortized Analysis
ā¦Accounting Method : Dynamic Tables
44. Amortized Analysis
Define a potential function Ī¦ that is
ā 0 immediately after an expansion
ā builds to the table size by the time table becomes full
Next expansion can be paid for by the potential.
One possible Ī¦ is:
Ī¦(T) = 2*num[T] āsize[T]
Immediately after an expansion
size[T] = 2*num[T] ā Ī¦(T) = 0
Immediately before an expansion
size[T] = num[T] ā Ī¦(T) = num[T]
The initial value for the potential is 0
2017/05/15 44Amortized Analysis
ā¦ā¦. Potential Method: Dynamic Tables
45. Amortized Analysis
Definition of Ī¦:
Since the table is at least half full
(i.e. num[T] ā„ size[T] / 2)
Ī¦(T) is always nonnegative.
Thus, the sum of the amortized cost of n TABLE-INSERT
operations is an upper bound on the sum of the actual
costs.
2017/05/15 45Amortized Analysis
ā¦ā¦. Potential Method: Dynamic Tables
46. Amortized Analysis
Analysis of i-th Table Insert:
numi : num[T] after the i-th operation
sizei : size[T] after the i-th operation
Ī¦i : Potential after the i-th operation
Initially we have numi = sizei = Ī¦i = 0
Note that, numi = numi-1 +1 always hold.
2017/05/15 46Amortized Analysis
ā¦ā¦. Potential Method: Dynamic Tables
47. Amortized Analysis
Insertion without Expansion:
sizei = sizei ā 1 ; ci = 1
Insertion with Expansion:
sizei = 2.sizei ā 1 and sizei-1 =numi ā 1 =numi - 1
2017/05/15 47Amortized Analysis
ā¦ā¦. Potential Method: Dynamic Tables
48. Amortized Analysis
Adding Delete Operation:
TABLE-DELETE: Remove the specified item from the
table. It is often desirable to contract the table.
In table contraction, we would like to preserve two
properties
ļ§ Load factor of dynamic table is bounded below by a
constant
ļ§ Amortized cost of a table operation is bounded above
by a constant
We assume that the cost can be measured in terms of
elementary insertions and deletions
2017/05/15 48Amortized Analysis
ā¦ā¦. Potential Method: Dynamic Tables
49. Amortized Analysis
A natural strategy for expansion and contraction
ā¢ Double the table size when an item is inserted into a
full table
ā¢ Halve the size when a deletion would cause
< 1 / 2
This strategy guarantees
Unfortunately, it can cause the amortized cost of an
operation to be quite large
2017/05/15 49Amortized Analysis
ā¦ā¦. Potential Method: Dynamic Tables
)(TĪ±
1)(
2
1
ā¤ā¤ TĪ±
50. Amortized Analysis
Worst-Case for Ī±(T) ā„ Ā½
Consider the following worst case scenario
ā We perform n operations on an empty table
where n is a power of 2
ā First n/2 operations are all insertions , cost a
total of Ī(n)
at the end: we have num[T] = size[T] = n/2
ā Second n/2 operations repeat the sequence
I D D I
that is I D D I I D D I I D D I ...
2017/05/15 50Amortized Analysis
ā¦ā¦. Potential Method: Dynamic Tables
51. Amortized Analysis
Worst-Case for Ī±(T) ā„ Ā½
2017/05/15 51Amortized Analysis
ā¦ā¦. Potential Method: Dynamic Tables
Example: n=16
In the second n/2 operations
ā The first INSERT cause an expansion
ā Two further DELETEs cause contraction
ā Two further INSERTs cause expansion ... and so on
Hence there are n/8 expansions and n/8 contractions
The cost of each expansion and contraction is ā n/2
52. Amortized Analysis
Worst-Case for Ī±(T) ā„ Ā½
Thus the total cost of n operations is Ī(n2
) since
ā First n/2 operations : 3n
ā Second n/2 operations : (n/4)*(n/2)=n2
/8
The amortized cost of an operation is Ī(n)
The difficulty with this strategy is
ā After an expansion, we do not perform enough
deletions to pay for a contraction
ā After a contraction, we do not perform enough
insertions to pay for an expansion
2017/05/15 52Amortized Analysis
ā¦ā¦. Potential Method: Dynamic Tables
53. Amortized Analysis
Improving Expansion ā Contraction
We can improve upon this strategy by allowing Ī±(T)
to drop below Ā½
We continue to double the table size when an item
is inserted into a full table
But, we halve the table size (perform contraction)
when a deletion causes Ī±(T) < Ā¼ rather than Ī±(T)
< Ā½ ,
Therefore, Ā¼ ā¤ Ī±(T) ā¤ 1
2017/05/15 53Amortized Analysis
ā¦ā¦. Potential Method: Dynamic Tables
54. Amortized Analysis
Improving Expansion ā Contraction
Hence after an expansion, Ī±(T) = Ā½ , thus at least
half of the items in the table must be deleted
before a contraction can occur.
Similarly, after a contraction Ī±(T) = Ā½ , thus the
number of items in the table must be doubled by
insertions before an expansion can occur.
2017/05/15 54Amortized Analysis
ā¦ā¦. Potential Method: Dynamic Tables
55. Amortized Analysis
Define the potential function as follows
ā¢ Ī¦(T) = 0 immediately after an expansion or
contraction
ā¢ Recall that, Ī±(T) = Ā½ immediately after and
expansion or contraction,
therefore the potential should build up to num[T]
as Ī±(T) increases to 1 or decreases to Ā¼
ā¢ So that the next expansion or contraction can be
paid by the potential.
2017/05/15 55Amortized Analysis
ā¦ā¦. Potential Method: Dynamic Tables
56. Amortized Analysis
Description of New Ī¦: One such Ī¦ is
ā¢ Ī¦(T) = 0 when Ī± = Ā½
ā¢ Ī¦(T) = num[T] when Ī± = Ā¼
ā¢ Ī¦ (T)= 0 for an empty table (num[T] = size[T]=0)
ā¢ Ī¦ is always nonnegative
2017/05/15 56Amortized Analysis
ā¦ā¦. Potential Method: Dynamic Tables
ļ£“
ļ£³
ļ£“
ļ£²
ļ£±
<ā
ā„ā
=Ī¦
2
1
(T)if][
2
size[T]
2
1
(T)if][][2
)(
Ī±
Ī±
Tnum
TsizeTnum
T
61. ļ± Thus, the amortized cost of a TABLE-DELETE operation is at
most 2
ļ± Since the amortized cost of each operation is bounded above by
a constant
ļ± The actual time for any sequence of n operations on a Dynamic
Table is O(n)
Amortized Analysis
ā¦.. Dynamic Tables
2017/05/15 25Amortized Analysis