В рамках C/C++/Embedded місяця у GlobalLogic нещодавно відбувся Online TechTalk "Computational Complexity of the Algorithm: When Time Fades Away" Під час заходу спікер розглянув такі питання:  - Чому стандартна концепція часу є неприйнятною для оцінки алгоритму? - Асимптотична складність алгоритму. - Байка про мавпу, яка сортує масив. - Порівняння елементарних алгоритмів, визначення асимптотичної складності. More details and video: https://bit.ly/3kfRU6T

1. 1
Confidential

2. 2
Confidential
Computational complexity of the algorithm:
When time fades away
Oleksandr Basalkevych
Senior Software Developer

3. 3
Confidential
Agenda
1. Why can't we use the common concept of time?
2. The asymptotic complexity of algorithm.
3. A story about monkey, who can sort an array.
4. Comparison of elementary algorithms, determination of complexity.
5. Q&A.
6. Quiz

4. 4
Confidential
Gifts

5. 5
Confidential
Algorithm evaluation criteria
• Execution time
• Used memory

6. 6
Confidential
6
Why can't we use the common
concept of time?

7. 7
Confidential
Execution time
We cannot use usual concept of “time” (like milliseconds, seconds) because
different CPUs have different clock frequency.
CPU1
2 x 10 op/s
7
CPU2
5 x 10 op/s
8
To complete, algorithm A needs to execute 10 elementary operations
8
THEN
T = 5s
CPU1
T = 0.2s
CPU2
But we are talking about the same algorithm…

8. 8
Confidential
8
The asymptotic complexity of
algorithm

9. 9
Confidential
Computational Complexity of an algorithm is the
amount of resources required for running it.
• O-notation (big O notation)
𝑂 𝑔 𝑛 = { 𝑓 𝑛 : 𝑡ℎ𝑒𝑟𝑒 𝑒𝑥𝑖𝑠𝑡
𝑝𝑜𝑠𝑖𝑡𝑖𝑣𝑒 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡𝑠 𝒄 𝑎𝑛𝑑 𝒏𝟎
𝑠𝑢𝑐ℎ 𝑡ℎ𝑎𝑡
0 ≤ 𝑓 𝑛 ≤ 𝒄𝑔 𝑛 𝑓𝑜𝑟 𝑎𝑙𝑙 𝑛 ≥ 𝒏𝟎 }

10. 10
Confidential
Computational Complexity

11. 11
Confidential
Computational Complexity

12. 12
Confidential
Computational Complexity
Let’s assume that the CPU performs approximately 10 elementary operations per
second.
6

13. 13
Confidential
13
A story about monkey, who can sort
an array.

14. 14
Confidential
Monkey with cards
Even monkey can sort an array… with O(N!) complexity.

15. 15
Confidential
15
Comparison of elementary algorithms,
determination of complexity.

16. 16
Confidential
O-notation expression simplification
• Ignore constants
• Greater power has much greater contribution
So,
𝑂 2𝑁 + 10 = 𝑂(𝑁)
𝑂(
𝑁
2
) = 𝑂(𝑁)
𝑂 𝑁2
+ 𝑁 = 𝑂(𝑁2
)
𝑂
𝑁3
2
+ 5𝑁2 +
𝑁
3
+ 1 = ?
𝑂(𝑁3)

17. 17
Confidential
Let’s practice: Reverse array
A: 3 18 1 21 48 26 reverse A: 26 48 21 1 18 3
N = length(A)
for i : [0 … N/2 - 1]
left = i
right = N – i - 1
temp = A[left]
A[left] = A[right]
A[right] = temp
2
N/2 times:
1
3
2
3
2
𝑂 2 +
𝑁
2
1 + 3 + 2 + 3 + 2 = 𝑂
11
2
𝑁 + 2 = 𝑶(𝑵)

18. 18
Confidential
Let’s practice: Element search
A: 3 18 1 21 48 26
res = -1
N = length(A)
for i : [0 … N-1]
if A[i] is equal to <Element>
res = i
break
1
2
N times in the worst case:
2
1
𝑂 1 + 2 + 2𝑁 + 1 = 𝑂 2𝑁 + 4 = 𝑶(𝑵)

19. 19
Confidential
Binary search
A: 4 5 10 15 20 25
• The array must be sorted
30 35 40 45 50 58 65 80 98
• Lets find 20 using binary search algorithm

20. 20
Confidential
Binary search: Searching for 20
A: 4 5 10 15 20 25 30 35 40 45 50 58 65 80 98
left(0) middle(7) right(14)
A: 4 5 10 15 20 25 30 35 40 45 50 58 65 80 98
left(0) middle(3) right(7)
A: 4 5 10 15 20 25 30 35 40 45 50 58 65 80 98
left(3) middle(5) right(7)
A: 4 5 10 15 20 25 30 35 40 45 50 58 65 80 98
left middle right

21. 21
Confidential
Binary search: Searching for 38
A: 4 5 10 15 20 25 30 35 40 45 50 58 65 80 98
left(0) middle(7) right(14)
A: 4 5 10 15 20 25 30 35 40 45 50 58 65 80 98
left(7) middle(10) right(14)
A: 4 5 10 15 20 25 30 35 40 45 50 58 65 80 98
left(7) middle right(10)
A: 4 5 10 15 20 25 30 35 40 45 50 58 65 80 98
left(7) right(8) Not found: right – left = 1

22. 22
Confidential
Binary search: Complexity
• Informal explanation. Each time we reduce the array search area by 2
times and perform some O(1) operations. E.g. having array of 16 elements,
we reduce the problem to 8, 4, 2, 1 elements step by step. Assume the
length of array is N. How many steps are we going to have?
𝑂(log2 𝑁)
• For formal proof refer to Cormen T., Leiserson C., Rivest R., Stein C.
“Introduction to Algorithms”

23. 23
Confidential
23
It’s Quiz Time!

24. 24
Confidential
It’s Quiz Time!
Join at: www.kahoot.it
Game PIN: XXXXXXX

25. 25
Confidential
Example 1
𝑂(𝑁)
N = array.length

26. 26
Confidential
Example 2
𝑂(𝑁2
)
N = array.length

27. 27
Confidential
Example 3
𝑂(𝑁2
)
N = array.length
• • • • •
• • • •
• • •
• •
•

28. 28
Confidential
Example 4
𝑂(𝑁𝑀)
N = arrayA.length
M = arrayB.length

29. 29
Confidential
Example 5
𝑂(𝑁𝑀)
N = arrayA.length
M = arrayB.length

30. 30
Confidential
Example 6
𝑂(𝑁2
+ 𝑀)
N = arrayA.length
M = arrayB.length

31. 31
Thank You