xinyin025/hello-algo
1
0
Fork 0
mirror of https://github.com/krahets/hello-algo.git synced 2026-06-28 00:24:21 +00:00
Code Issues Packages Projects Releases Wiki Activity

Revisit the English version (#1835)

Browse Source 
* Review the English version using Claude-4.5.

* Update mkdocs.yml

* Align the section titles.

* Bug fixes
This commit is contained in:
Yudong Jin
2025-12-30 17:54:01 +08:00
committed by GitHub
parent 091afd38b4
commit 45e1295241
106 changed files with 4195 additions and 3398 deletions
Download Patch File Download Diff File Expand all files Collapse all files
en/docs/chapter_computational_complexity/time_complexity.md
+284 -172
View File
@@ -1,22 +1,22 @@
# Time complexity
The runtime can intuitively assess the efficiency of an algorithm. How can we accurately estimate the runtime of a piece of an algorithm?
Runtime can intuitively and accurately reflect the efficiency of an algorithm. If we want to accurately estimate the runtime of a piece of code, how should we proceed?
1. **Determining the Running Platform**: This includes hardware configuration, programming language, system environment, etc., all of which can affect the efficiency of code execution.
2. **Evaluating the Run Time for Various Computational Operations**: For instance, an addition operation `+` might take 1 ns, a multiplication operation `*` might take 10 ns, a print operation `print()` might take 5 ns, etc.
3. **Counting All the Computational Operations in the Code**: Summing the execution times of all these operations gives the total run time.
1. **Determine the running platform**, including hardware configuration, programming language, system environment, etc., as these factors all affect code execution efficiency.
2. **Evaluate the runtime required for various computational operations**, for example, an addition operation `+` requires 1 ns, a multiplication operation `*` requires 10 ns, a print operation `print()` requires 5 ns, etc.
3. **Count all computational operations in the code**, and sum the execution times of all operations to obtain the runtime.
For example, consider the following code with an input size of $n$:
For example, in the following code, the input data size is $n$:
=== "Python"
```python title=""
# Under an operating platform
# On a certain running platform
def algorithm(n: int):
a = 2 # 1 ns
a = a + 1 # 1 ns
a = a * 2 # 10 ns
# Cycle n times
# Loop n times
for _ in range(n): # 1 ns
print(0) # 5 ns
```
@@ -24,13 +24,13 @@ For example, consider the following code with an input size of $n$:
=== "C++"
```cpp title=""
// Under a particular operating platform
// On a certain running platform
void algorithm(int n) {
int a = 2; // 1 ns
a = a + 1; // 1 ns
a = a * 2; // 10 ns
// Loop n times
for (int i = 0; i < n; i++) { // 1 ns , every round i++ is executed
for (int i = 0; i < n; i++) { // 1 ns
cout << 0 << endl; // 5 ns
}
}
@@ -39,13 +39,13 @@ For example, consider the following code with an input size of $n$:
=== "Java"
```java title=""
// Under a particular operating platform
// On a certain running platform
void algorithm(int n) {
int a = 2; // 1 ns
a = a + 1; // 1 ns
a = a * 2; // 10 ns
// Loop n times
for (int i = 0; i < n; i++) { // 1 ns , every round i++ is executed
for (int i = 0; i < n; i++) { // 1 ns
System.out.println(0); // 5 ns
}
}
@@ -54,13 +54,13 @@ For example, consider the following code with an input size of $n$:
=== "C#"
```csharp title=""
// Under a particular operating platform
// On a certain running platform
void Algorithm(int n) {
int a = 2; // 1 ns
a = a + 1; // 1 ns
a = a * 2; // 10 ns
// Loop n times
for (int i = 0; i < n; i++) { // 1 ns , every round i++ is executed
for (int i = 0; i < n; i++) { // 1 ns
Console.WriteLine(0); // 5 ns
}
}
@@ -69,7 +69,7 @@ For example, consider the following code with an input size of $n$:
=== "Go"
```go title=""
// Under a particular operating platform
// On a certain running platform
func algorithm(n int) {
a := 2 // 1 ns
a = a + 1 // 1 ns
@@ -84,7 +84,7 @@ For example, consider the following code with an input size of $n$:
=== "Swift"
```swift title=""
// Under a particular operating platform
// On a certain running platform
func algorithm(n: Int) {
var a = 2 // 1 ns
a = a + 1 // 1 ns
@@ -99,13 +99,13 @@ For example, consider the following code with an input size of $n$:
=== "JS"
```javascript title=""
// Under a particular operating platform
// On a certain running platform
function algorithm(n) {
var a = 2; // 1 ns
a = a + 1; // 1 ns
a = a * 2; // 10 ns
// Loop n times
for(let i = 0; i < n; i++) { // 1 ns , every round i++ is executed
for(let i = 0; i < n; i++) { // 1 ns
console.log(0); // 5 ns
}
}
@@ -114,13 +114,13 @@ For example, consider the following code with an input size of $n$:
=== "TS"
```typescript title=""
// Under a particular operating platform
// On a certain running platform
function algorithm(n: number): void {
var a: number = 2; // 1 ns
a = a + 1; // 1 ns
a = a * 2; // 10 ns
// Loop n times
for(let i = 0; i < n; i++) { // 1 ns , every round i++ is executed
for(let i = 0; i < n; i++) { // 1 ns
console.log(0); // 5 ns
}
}
@@ -129,13 +129,13 @@ For example, consider the following code with an input size of $n$:
=== "Dart"
```dart title=""
// Under a particular operating platform
// On a certain running platform
void algorithm(int n) {
int a = 2; // 1 ns
a = a + 1; // 1 ns
a = a * 2; // 10 ns
// Loop n times
for (int i = 0; i < n; i++) { // 1 ns , every round i++ is executed
for (int i = 0; i < n; i++) { // 1 ns
print(0); // 5 ns
}
}
@@ -144,13 +144,13 @@ For example, consider the following code with an input size of $n$:
=== "Rust"
```rust title=""
// Under a particular operating platform
// On a certain running platform
fn algorithm(n: i32) {
let mut a = 2; // 1 ns
a = a + 1; // 1 ns
a = a * 2; // 10 ns
// Loop n times
for _ in 0..n { // 1 ns for each round i++
for _ in 0..n { // 1 ns
println!("{}", 0); // 5 ns
}
}
@@ -159,13 +159,13 @@ For example, consider the following code with an input size of $n$:
=== "C"
```c title=""
// Under a particular operating platform
// On a certain running platform
void algorithm(int n) {
int a = 2; // 1 ns
a = a + 1; // 1 ns
a = a * 2; // 10 ns
// Loop n times
for (int i = 0; i < n; i++) { // 1 ns , every round i++ is executed
for (int i = 0; i < n; i++) { // 1 ns
printf("%d", 0); // 5 ns
}
}
@@ -174,13 +174,37 @@ For example, consider the following code with an input size of $n$:
=== "Kotlin"
```kotlin title=""
// On a certain running platform
fun algorithm(n: Int) {
var a = 2 // 1 ns
a = a + 1 // 1 ns
a = a * 2 // 10 ns
// Loop n times
for (i in 0..<n) { // 1 ns
println(0) // 5 ns
}
}
```
=== "Ruby"
```ruby title=""
# On a certain running platform
def algorithm(n)
a = 2 # 1 ns
a = a + 1 # 1 ns
a = a * 2 # 10 ns
# Loop n times
(0...n).each do # 1 ns
puts 0 # 5 ns
end
end
```
=== "Zig"
```zig title=""
// Under a particular operating platform
// On a certain running platform
fn algorithm(n: usize) void {
var a: i32 = 2; // 1 ns
a += 1; // 1 ns
@@ -192,19 +216,19 @@ For example, consider the following code with an input size of $n$:
}
```
Using the above method, the run time of the algorithm can be calculated as $(6n + 12)$ ns:
According to the above method, the algorithm's runtime can be obtained as $(6n + 12)$ ns:
$$
1 + 1 + 10 + (1 + 5) \times n = 6n + 12
$$
However, in practice, **counting the run time of an algorithm is neither practical nor reasonable**. First, we don't want to tie the estimated time to the running platform, as algorithms need to run on various platforms. Second, it's challenging to know the run time for each type of operation, making the estimation process difficult.
In reality, however, **counting an algorithm's runtime is neither reasonable nor realistic**. First, we do not want to tie the estimated time to the running platform, because algorithms need to run on various different platforms. Second, it is difficult to know the runtime of each type of operation, which brings great difficulty to the estimation process.
## Assessing time growth trend
## Counting time growth trends
Time complexity analysis does not count the algorithm's run time, **but rather the growth trend of the run time as the data volume increases**.
Time complexity analysis does not count the algorithm's runtime, **but rather counts the growth trend of the algorithm's runtime as the data volume increases**.
Let's understand this concept of "time growth trend" with an example. Assume the input data size is $n$, and consider three algorithms `A`, `B`, and `C`:
The concept of "time growth trend" is rather abstract; let us understand it through an example. Suppose the input data size is $n$, and given three algorithms `A`, `B`, and `C`:
=== "Python"
@@ -438,7 +462,41 @@ Let's understand this concept of "time growth trend" with an example. Assume the
=== "Kotlin"
```kotlin title=""
// Time complexity of algorithm A: constant order
fun algoritm_A(n: Int) {
println(0)
}
// Time complexity of algorithm B: linear order
fun algorithm_B(n: Int) {
for (i in 0..<n){
println(0)
}
}
// Time complexity of algorithm C: constant order
fun algorithm_C(n: Int) {
for (i in 0..<1000000) {
println(0)
}
}
```
=== "Ruby"
```ruby title=""
# Time complexity of algorithm A: constant order
def algorithm_A(n)
puts 0
end
# Time complexity of algorithm B: linear order
def algorithm_B(n)
(0...n).each { puts 0 }
end
# Time complexity of algorithm C: constant order
def algorithm_C(n)
(0...1_000_000).each { puts 0 }
end
```
=== "Zig"
@@ -464,23 +522,23 @@ Let's understand this concept of "time growth trend" with an example. Assume the
}
```
The figure below shows the time complexities of these three algorithms.
The figure below shows the time complexity of the above three algorithm functions.
- Algorithm `A` has just one print operation, and its run time does not grow with $n$. Its time complexity is considered "constant order."
- Algorithm `B` involves a print operation looping $n$ times, and its run time grows linearly with $n$. Its time complexity is "linear order."
- Algorithm `C` has a print operation looping 1,000,000 times. Although it takes a long time, it is independent of the input data size $n$. Therefore, the time complexity of `C` is the same as `A`, which is "constant order."
- Algorithm `A` has only $1$ print operation, and the algorithm's runtime does not grow as $n$ increases. We call the time complexity of this algorithm "constant order".
- In algorithm `B`, the print operation needs to loop $n$ times, and the algorithm's runtime grows linearly as $n$ increases. The time complexity of this algorithm is called "linear order".
- In algorithm `C`, the print operation needs to loop $1000000$ times. Although the runtime is very long, it is independent of the input data size $n$. Therefore, the time complexity of `C` is the same as `A`, still "constant order".
![Time growth trend of algorithms a, b, and c](time_complexity.assets/time_complexity_simple_example.png)
![Time growth trends of algorithms A, B, and C](time_complexity.assets/time_complexity_simple_example.png)
Compared to directly counting the run time of an algorithm, what are the characteristics of time complexity analysis?
Compared to directly counting the algorithm's runtime, what are the characteristics of time complexity analysis?
- **Time complexity effectively assesses algorithm efficiency**. For instance, algorithm `B` has linearly growing run time, which is slower than algorithm `A` when $n > 1$ and slower than `C` when $n > 1,000,000$. In fact, as long as the input data size $n$ is sufficiently large, a "constant order" complexity algorithm will always be better than a "linear order" one, demonstrating the essence of time growth trend.
- **Time complexity analysis is more straightforward**. Obviously, the running platform and the types of computational operations are irrelevant to the trend of run time growth. Therefore, in time complexity analysis, we can simply treat the execution time of all computational operations as the same "unit time," simplifying the "computational operation run time count" to a "computational operation count." This significantly reduces the complexity of estimation.
- **Time complexity has its limitations**. For example, although algorithms `A` and `C` have the same time complexity, their actual run times can be quite different. Similarly, even though algorithm `B` has a higher time complexity than `C`, it is clearly superior when the input data size $n$ is small. In these cases, it's difficult to judge the efficiency of algorithms based solely on time complexity. Nonetheless, despite these issues, complexity analysis remains the most effective and commonly used method for evaluating algorithm efficiency.
- **Time complexity can effectively evaluate algorithm efficiency**. For example, the runtime of algorithm `B` grows linearly; when $n > 1$ it is slower than algorithm `A`, and when $n > 1000000$ it is slower than algorithm `C`. In fact, as long as the input data size $n$ is sufficiently large, an algorithm with "constant order" complexity will always be superior to one with "linear order" complexity, which is precisely the meaning of time growth trend.
- **The derivation method for time complexity is simpler**. Obviously, the running platform and the types of computational operations are both unrelated to the growth trend of the algorithm's runtime. Therefore, in time complexity analysis, we can simply treat the execution time of all computational operations as the same "unit time", thus simplifying "counting computational operation runtime" to "counting the number of computational operations", which greatly reduces the difficulty of estimation.
- **Time complexity also has certain limitations**. For example, although algorithms `A` and `C` have the same time complexity, their actual runtimes differ significantly. Similarly, although algorithm `B` has a higher time complexity than `C`, when the input data size $n$ is small, algorithm `B` is clearly superior to algorithm `C`. In such cases, it is often difficult to judge the efficiency of algorithms based solely on time complexity. Of course, despite the above issues, complexity analysis remains the most effective and commonly used method for evaluating algorithm efficiency.
## Asymptotic upper bound
## Asymptotic upper bound of functions
Consider a function with an input size of $n$:
Given a function with input size $n$:
=== "Python"
@@ -489,7 +547,7 @@ Consider a function with an input size of $n$:
a = 1 # +1
a = a + 1 # +1
a = a * 2 # +1
# Cycle n times
# Loop n times
for i in range(n): # +1
print(0) # +1
```
@@ -502,7 +560,7 @@ Consider a function with an input size of $n$:
a = a + 1; // +1
a = a * 2; // +1
// Loop n times
for (int i = 0; i < n; i++) { // +1 (execute i ++ every round)
for (int i = 0; i < n; i++) { // +1 (i++ is executed each round)
cout << 0 << endl; // +1
}
}
@@ -516,7 +574,7 @@ Consider a function with an input size of $n$:
a = a + 1; // +1
a = a * 2; // +1
// Loop n times
for (int i = 0; i < n; i++) { // +1 (execute i ++ every round)
for (int i = 0; i < n; i++) { // +1 (i++ is executed each round)
System.out.println(0); // +1
}
}
@@ -530,7 +588,7 @@ Consider a function with an input size of $n$:
a = a + 1; // +1
a = a * 2; // +1
// Loop n times
for (int i = 0; i < n; i++) { // +1 (execute i ++ every round)
for (int i = 0; i < n; i++) { // +1 (i++ is executed each round)
Console.WriteLine(0); // +1
}
}
@@ -572,7 +630,7 @@ Consider a function with an input size of $n$:
a += 1; // +1
a *= 2; // +1
// Loop n times
for(let i = 0; i < n; i++){ // +1 (execute i ++ every round)
for(let i = 0; i < n; i++){ // +1 (i++ is executed each round)
console.log(0); // +1
}
}
@@ -586,7 +644,7 @@ Consider a function with an input size of $n$:
a += 1; // +1
a *= 2; // +1
// Loop n times
for(let i = 0; i < n; i++){ // +1 (execute i ++ every round)
for(let i = 0; i < n; i++){ // +1 (i++ is executed each round)
console.log(0); // +1
}
}
@@ -600,7 +658,7 @@ Consider a function with an input size of $n$:
a = a + 1; // +1
a = a * 2; // +1
// Loop n times
for (int i = 0; i < n; i++) { // +1 (execute i ++ every round)
for (int i = 0; i < n; i++) { // +1 (i++ is executed each round)
print(0); // +1
}
}
@@ -615,7 +673,7 @@ Consider a function with an input size of $n$:
a = a * 2; // +1
// Loop n times
for _ in 0..n { // +1 (execute i ++ every round)
for _ in 0..n { // +1 (i++ is executed each round)
println!("{}", 0); // +1
}
}
@@ -629,16 +687,38 @@ Consider a function with an input size of $n$:
a = a + 1; // +1
a = a * 2; // +1
// Loop n times
for (int i = 0; i < n; i++) { // +1 (execute i ++ every round)
for (int i = 0; i < n; i++) { // +1 (i++ is executed each round)
printf("%d", 0); // +1
}
}
}
```
=== "Kotlin"
```kotlin title=""
fun algorithm(n: Int) {
var a = 1 // +1
a = a + 1 // +1
a = a * 2 // +1
// Loop n times
for (i in 0..<n) { // +1 (i++ is executed each round)
println(0) // +1
}
}
```
=== "Ruby"
```ruby title=""
def algorithm(n)
a = 1 # +1
a = a + 1 # +1
a = a * 2 # +1
# Loop n times
(0...n).each do # +1
puts 0 # +1
end
end
```
=== "Zig"
@@ -649,56 +729,58 @@ Consider a function with an input size of $n$:
a += 1; // +1
a *= 2; // +1
// Loop n times
for (0..n) |_| { // +1 (execute i ++ every round)
for (0..n) |_| { // +1 (i++ is executed each round)
std.debug.print("{}\n", .{0}); // +1
}
}
```
Given a function that represents the number of operations of an algorithm as a function of the input size $n$, denoted as $T(n)$, consider the following example:
Let the number of operations of the algorithm be a function of the input data size $n$, denoted as $T(n)$. Then the number of operations of the above function is:
$$
T(n) = 3 + 2n
$$
Since $T(n)$ is a linear function, its growth trend is linear, and therefore, its time complexity is of linear order, denoted as $O(n)$. This mathematical notation, known as <u>big-O notation</u>, represents the <u>asymptotic upper bound</u> of the function $T(n)$.
$T(n)$ is a linear function, indicating that its runtime growth trend is linear, and therefore its time complexity is linear order.
In essence, time complexity analysis is about finding the asymptotic upper bound of the "number of operations $T(n)$". It has a precise mathematical definition.
We denote the time complexity of linear order as $O(n)$. This mathematical symbol is called <u>big-$O$ notation</u>, representing the <u>asymptotic upper bound</u> of the function $T(n)$.
!!! note "Asymptotic Upper Bound"
Time complexity analysis essentially calculates the asymptotic upper bound of "the number of operations $T(n)$", which has a clear mathematical definition.
If there exist positive real numbers $c$ and $n_0$ such that for all $n > n_0$, $T(n) \leq c \cdot f(n)$, then $f(n)$ is considered an asymptotic upper bound of $T(n)$, denoted as $T(n) = O(f(n))$.
!!! note "Asymptotic upper bound of functions"
As shown in the figure below, calculating the asymptotic upper bound involves finding a function $f(n)$ such that, as $n$ approaches infinity, $T(n)$ and $f(n)$ have the same growth order, differing only by a constant factor $c$.
If there exist positive real numbers $c$ and $n_0$ such that for all $n > n_0$, we have $T(n) \leq c \cdot f(n)$, then $f(n)$ can be considered as an asymptotic upper bound of $T(n)$, denoted as $T(n) = O(f(n))$.
As shown in the figure below, calculating the asymptotic upper bound is to find a function $f(n)$ such that when $n$ tends to infinity, $T(n)$ and $f(n)$ are at the same growth level, differing only by a constant coefficient $c$.
![Asymptotic upper bound of a function](time_complexity.assets/asymptotic_upper_bound.png)
## Calculation method
## Derivation method
While the concept of asymptotic upper bound might seem mathematically dense, you don't need to fully grasp it right away. Let's first understand the method of calculation, which can be practiced and comprehended over time.
The asymptotic upper bound has a bit of mathematical flavor. If you feel you haven't fully understood it, don't worry. We can first master the derivation method, and gradually grasp its mathematical meaning through continuous practice.
Once $f(n)$ is determined, we obtain the time complexity $O(f(n))$. But how do we determine the asymptotic upper bound $f(n)$? This process generally involves two steps: counting the number of operations and determining the asymptotic upper bound.
According to the definition, after determining $f(n)$, we can obtain the time complexity $O(f(n))$. So how do we determine the asymptotic upper bound $f(n)$? Overall, it is divided into two steps: first count the number of operations, then determine the asymptotic upper bound.
### Step 1: counting the number of operations
### Step 1: Count the number of operations
This step involves going through the code line by line. However, due to the presence of the constant $c$ in $c \cdot f(n)$, **all coefficients and constant terms in $T(n)$ can be ignored**. This principle allows for simplification techniques in counting operations.
For code, count from top to bottom line by line. However, since the constant coefficient $c$ in $c \cdot f(n)$ above can be of any size, **coefficients and constant terms in the number of operations $T(n)$ can all be ignored**. According to this principle, the following counting simplification techniques can be summarized.
1. **Ignore constant terms in $T(n)$**, as they do not affect the time complexity being independent of $n$.
2. **Omit all coefficients**. For example, looping $2n$, $5n + 1$ times, etc., can be simplified to $n$ times since the coefficient before $n$ does not impact the time complexity.
3. **Use multiplication for nested loops**. The total number of operations equals the product of the number of operations in each loop, applying the simplification techniques from points 1 and 2 for each loop level.
1. **Ignore constants in $T(n)$**. Because they are all independent of $n$, they do not affect time complexity.
2. **Omit all coefficients**. For example, looping $2n$ times, $5n + 1$ times, etc., can all be simplified as $n$ times, because the coefficient before $n$ does not affect time complexity.
3. **Use multiplication for nested loops**. The total number of operations equals the product of the number of operations in the outer and inner loops, with each layer of loop still able to apply techniques `1.` and `2.` separately.
Given a function, we can use these techniques to count operations:
Given a function, we can use the above techniques to count the number of operations:
=== "Python"
```python title=""
def algorithm(n: int):
a = 1 # +0 (trick 1)
a = a + n # +0 (trick 1)
# +n (technique 2)
a = 1 # +0 (Technique 1)
a = a + n # +0 (Technique 1)
# +n (Technique 2)
for i in range(5 * n + 1):
print(0)
# +n*n (technique 3)
# +n*n (Technique 3)
for i in range(2 * n):
for j in range(n + 1):
print(0)
@@ -708,13 +790,13 @@ Given a function, we can use these techniques to count operations:
```cpp title=""
void algorithm(int n) {
int a = 1; // +0 (trick 1)
a = a + n; // +0 (trick 1)
// +n (technique 2)
int a = 1; // +0 (Technique 1)
a = a + n; // +0 (Technique 1)
// +n (Technique 2)
for (int i = 0; i < 5 * n + 1; i++) {
cout << 0 << endl;
}
// +n*n (technique 3)
// +n*n (Technique 3)
for (int i = 0; i < 2 * n; i++) {
for (int j = 0; j < n + 1; j++) {
cout << 0 << endl;
@@ -727,13 +809,13 @@ Given a function, we can use these techniques to count operations:
```java title=""
void algorithm(int n) {
int a = 1; // +0 (trick 1)
a = a + n; // +0 (trick 1)
// +n (technique 2)
int a = 1; // +0 (Technique 1)
a = a + n; // +0 (Technique 1)
// +n (Technique 2)
for (int i = 0; i < 5 * n + 1; i++) {
System.out.println(0);
}
// +n*n (technique 3)
// +n*n (Technique 3)
for (int i = 0; i < 2 * n; i++) {
for (int j = 0; j < n + 1; j++) {
System.out.println(0);
@@ -746,13 +828,13 @@ Given a function, we can use these techniques to count operations:
```csharp title=""
void Algorithm(int n) {
int a = 1; // +0 (trick 1)
a = a + n; // +0 (trick 1)
// +n (technique 2)
int a = 1; // +0 (Technique 1)
a = a + n; // +0 (Technique 1)
// +n (Technique 2)
for (int i = 0; i < 5 * n + 1; i++) {
Console.WriteLine(0);
}
// +n*n (technique 3)
// +n*n (Technique 3)
for (int i = 0; i < 2 * n; i++) {
for (int j = 0; j < n + 1; j++) {
Console.WriteLine(0);
@@ -765,13 +847,13 @@ Given a function, we can use these techniques to count operations:
```go title=""
func algorithm(n int) {
a := 1 // +0 (trick 1)
a = a + n // +0 (trick 1)
// +n (technique 2)
a := 1 // +0 (Technique 1)
a = a + n // +0 (Technique 1)
// +n (Technique 2)
for i := 0; i < 5 * n + 1; i++ {
fmt.Println(0)
}
// +n*n (technique 3)
// +n*n (Technique 3)
for i := 0; i < 2 * n; i++ {
for j := 0; j < n + 1; j++ {
fmt.Println(0)
@@ -784,13 +866,13 @@ Given a function, we can use these techniques to count operations:
```swift title=""
func algorithm(n: Int) {
var a = 1 // +0 (trick 1)
a = a + n // +0 (trick 1)
// +n (technique 2)
var a = 1 // +0 (Technique 1)
a = a + n // +0 (Technique 1)
// +n (Technique 2)
for _ in 0 ..< (5 * n + 1) {
print(0)
}
// +n*n (technique 3)
// +n*n (Technique 3)
for _ in 0 ..< (2 * n) {
for _ in 0 ..< (n + 1) {
print(0)
@@ -803,13 +885,13 @@ Given a function, we can use these techniques to count operations:
```javascript title=""
function algorithm(n) {
let a = 1; // +0 (trick 1)
a = a + n; // +0 (trick 1)
// +n (technique 2)
let a = 1; // +0 (Technique 1)
a = a + n; // +0 (Technique 1)
// +n (Technique 2)
for (let i = 0; i < 5 * n + 1; i++) {
console.log(0);
}
// +n*n (technique 3)
// +n*n (Technique 3)
for (let i = 0; i < 2 * n; i++) {
for (let j = 0; j < n + 1; j++) {
console.log(0);
@@ -822,13 +904,13 @@ Given a function, we can use these techniques to count operations:
```typescript title=""
function algorithm(n: number): void {
let a = 1; // +0 (trick 1)
a = a + n; // +0 (trick 1)
// +n (technique 2)
let a = 1; // +0 (Technique 1)
a = a + n; // +0 (Technique 1)
// +n (Technique 2)
for (let i = 0; i < 5 * n + 1; i++) {
console.log(0);
}
// +n*n (technique 3)
// +n*n (Technique 3)
for (let i = 0; i < 2 * n; i++) {
for (let j = 0; j < n + 1; j++) {
console.log(0);
@@ -841,13 +923,13 @@ Given a function, we can use these techniques to count operations:
```dart title=""
void algorithm(int n) {
int a = 1; // +0 (trick 1)
a = a + n; // +0 (trick 1)
// +n (technique 2)
int a = 1; // +0 (Technique 1)
a = a + n; // +0 (Technique 1)
// +n (Technique 2)
for (int i = 0; i < 5 * n + 1; i++) {
print(0);
}
// +n*n (technique 3)
// +n*n (Technique 3)
for (int i = 0; i < 2 * n; i++) {
for (int j = 0; j < n + 1; j++) {
print(0);
@@ -860,15 +942,15 @@ Given a function, we can use these techniques to count operations:
```rust title=""
fn algorithm(n: i32) {
let mut a = 1; // +0 (trick 1)
a = a + n; // +0 (trick 1)
let mut a = 1; // +0 (Technique 1)
a = a + n; // +0 (Technique 1)
// +n (technique 2)
// +n (Technique 2)
for i in 0..(5 * n + 1) {
println!("{}", 0);
}
// +n*n (technique 3)
// +n*n (Technique 3)
for i in 0..(2 * n) {
for j in 0..(n + 1) {
println!("{}", 0);
@@ -881,13 +963,13 @@ Given a function, we can use these techniques to count operations:
```c title=""
void algorithm(int n) {
int a = 1; // +0 (trick 1)
a = a + n; // +0 (trick 1)
// +n (technique 2)
int a = 1; // +0 (Technique 1)
a = a + n; // +0 (Technique 1)
// +n (Technique 2)
for (int i = 0; i < 5 * n + 1; i++) {
printf("%d", 0);
}
// +n*n (technique 3)
// +n*n (Technique 3)
for (int i = 0; i < 2 * n; i++) {
for (int j = 0; j < n + 1; j++) {
printf("%d", 0);
@@ -899,22 +981,50 @@ Given a function, we can use these techniques to count operations:
=== "Kotlin"
```kotlin title=""
fun algorithm(n: Int) {
var a = 1 // +0 (Technique 1)
a = a + n // +0 (Technique 1)
// +n (Technique 2)
for (i in 0..<5 * n + 1) {
println(0)
}
// +n*n (Technique 3)
for (i in 0..<2 * n) {
for (j in 0..<n + 1) {
println(0)
}
}
}
```
=== "Ruby"
```ruby title=""
def algorithm(n)
a = 1 # +0 (Technique 1)
a = a + n # +0 (Technique 1)
# +n (Technique 2)
(0...(5 * n + 1)).each do { puts 0 }
# +n*n (Technique 3)
(0...(2 * n)).each do
(0...(n + 1)).each do { puts 0 }
end
end
```
=== "Zig"
```zig title=""
fn algorithm(n: usize) void {
var a: i32 = 1; // +0 (trick 1)
a = a + @as(i32, @intCast(n)); // +0 (trick 1)
var a: i32 = 1; // +0 (Technique 1)
a = a + @as(i32, @intCast(n)); // +0 (Technique 1)
// +n (technique 2)
// +n (Technique 2)
for(0..(5 * n + 1)) |_| {
std.debug.print("{}\n", .{0});
}
// +n*n (technique 3)
// +n*n (Technique 3)
for(0..(2 * n)) |_| {
for(0..(n + 1)) |_| {
std.debug.print("{}\n", .{0});
@@ -923,48 +1033,50 @@ Given a function, we can use these techniques to count operations:
}
```
The formula below shows the counting results before and after simplification, both leading to a time complexity of $O(n^2)$:
The following formula shows the counting results before and after using the above techniques; both derive a time complexity of $O(n^2)$.
$$
\begin{aligned}
T(n) & = 2n(n + 1) + (5n + 1) + 2 & \text{Complete Count (-.-|||)} \newline
T(n) & = 2n(n + 1) + (5n + 1) + 2 & \text{Complete count (-.-|||)} \newline
& = 2n^2 + 7n + 3 \newline
T(n) & = n^2 + n & \text{Simplified Count (o.O)}
T(n) & = n^2 + n & \text{Simplified count (o.O)}
\end{aligned}
$$
### Step 2: determining the asymptotic upper bound
### Step 2: Determine the asymptotic upper bound
**The time complexity is determined by the highest order term in $T(n)$**. This is because, as $n$ approaches infinity, the highest order term dominates, rendering the influence of other terms negligible.
**Time complexity is determined by the highest-order term in $T(n)$**. This is because as $n$ tends to infinity, the highest-order term will play a dominant role, and the influence of other terms can be ignored.
The following table illustrates examples of different operation counts and their corresponding time complexities. Some exaggerated values are used to emphasize that coefficients cannot alter the order of growth. When $n$ becomes very large, these constants become insignificant.
The table below shows some examples, where some exaggerated values are used to emphasize the conclusion that "coefficients cannot shake the order". When $n$ tends to infinity, these constants become insignificant.
<p align="center"> Table: Time complexity for different operation counts </p>
<p align="center"> Table <id> &nbsp; Time complexities corresponding to different numbers of operations </p>
| Operation Count $T(n)$ | Time Complexity $O(f(n))$ |
| ---------------------- | ------------------------- |
| $100000$ | $O(1)$ |
| $3n + 2$ | $O(n)$ |
| $2n^2 + 3n + 2$ | $O(n^2)$ |
| $n^3 + 10000n^2$ | $O(n^3)$ |
| $2^n + 10000n^{10000}$ | $O(2^n)$ |
| Number of Operations $T(n)$ | Time Complexity $O(f(n))$ |
| ---------------------- | -------------------- |
| $100000$ | $O(1)$ |
| $3n + 2$ | $O(n)$ |
| $2n^2 + 3n + 2$ | $O(n^2)$ |
| $n^3 + 10000n^2$ | $O(n^3)$ |
| $2^n + 10000n^{10000}$ | $O(2^n)$ |
## Common types of time complexity
## Common types
Let's consider the input data size as $n$. The common types of time complexities are shown in the figure below, arranged from lowest to highest:
Let the input data size be $n$. Common time complexity types are shown in the figure below (arranged in order from low to high).
$$
\begin{aligned}
& O(1) < O(\log n) < O(n) < O(n \log n) < O(n^2) < O(2^n) < O(n!) \newline
& \text{Constant} < \text{Log} < \text{Linear} < \text{Linear-Log} < \text{Quadratic} < \text{Exp} < \text{Factorial}
O(1) < O(\log n) < O(n) < O(n \log n) < O(n^2) < O(2^n) < O(n!) \newline
\text{Constant order} < \text{Logarithmic order} < \text{Linear order} < \text{Linearithmic order} < \text{Quadratic order} < \text{Exponential order} < \text{Factorial order}
\end{aligned}
$$
![Common types of time complexity](time_complexity.assets/time_complexity_common_types.png)
![Common time complexity types](time_complexity.assets/time_complexity_common_types.png)
### Constant order $O(1)$
Constant order means the number of operations is independent of the input data size $n$. In the following function, although the number of operations `size` might be large, the time complexity remains $O(1)$ as it's unrelated to $n$:
The number of operations in constant order is independent of the input data size $n$, meaning it does not change as $n$ changes.
In the following function, although the number of operations `size` may be large, since it is independent of the input data size $n$, the time complexity remains $O(1)$:
```src
[file]{time_complexity}-[class]{}-[func]{constant}
@@ -972,23 +1084,23 @@ Constant order means the number of operations is independent of the input data s
### Linear order $O(n)$
Linear order indicates the number of operations grows linearly with the input data size $n$. Linear order commonly appears in single-loop structures:
The number of operations in linear order grows linearly relative to the input data size $n$. Linear order typically appears in single-layer loops:
```src
[file]{time_complexity}-[class]{}-[func]{linear}
```
Operations like array traversal and linked list traversal have a time complexity of $O(n)$, where $n$ is the length of the array or list:
Operations such as traversing arrays and traversing linked lists have a time complexity of $O(n)$, where $n$ is the length of the array or linked list:
```src
[file]{time_complexity}-[class]{}-[func]{array_traversal}
```
It's important to note that **the input data size $n$ should be determined based on the type of input data**. For example, in the first example, $n$ represents the input data size, while in the second example, the length of the array $n$ is the data size.
It is worth noting that **the input data size $n$ should be determined according to the type of input data**. For example, in the first example, the variable $n$ is the input data size; in the second example, the array length $n$ is the data size.
### Quadratic order $O(n^2)$
Quadratic order means the number of operations grows quadratically with the input data size $n$. Quadratic order typically appears in nested loops, where both the outer and inner loops have a time complexity of $O(n)$, resulting in an overall complexity of $O(n^2)$:
The number of operations in quadratic order grows quadratically relative to the input data size $n$. Quadratic order typically appears in nested loops, where both the outer and inner loops have a time complexity of $O(n)$, resulting in an overall time complexity of $O(n^2)$:
```src
[file]{time_complexity}-[class]{}-[func]{quadratic}
@@ -996,9 +1108,9 @@ Quadratic order means the number of operations grows quadratically with the inpu
The figure below compares constant order, linear order, and quadratic order time complexities.
![Constant, linear, and quadratic order time complexities](time_complexity.assets/time_complexity_constant_linear_quadratic.png)
![Time complexities of constant, linear, and quadratic orders](time_complexity.assets/time_complexity_constant_linear_quadratic.png)
For instance, in bubble sort, the outer loop runs $n - 1$ times, and the inner loop runs $n-1$, $n-2$, ..., $2$, $1$ times, averaging $n / 2$ times, resulting in a time complexity of $O((n - 1) n / 2) = O(n^2)$:
Taking bubble sort as an example, the outer loop executes $n - 1$ times, and the inner loop executes $n-1$, $n-2$, $\dots$, $2$, $1$ times, averaging $n / 2$ times, resulting in a time complexity of $O((n - 1) n / 2) = O(n^2)$:
```src
[file]{time_complexity}-[class]{}-[func]{bubble_sort}
@@ -1006,107 +1118,107 @@ For instance, in bubble sort, the outer loop runs $n - 1$ times, and the inner l
### Exponential order $O(2^n)$
Biological "cell division" is a classic example of exponential order growth: starting with one cell, it becomes two after one division, four after two divisions, and so on, resulting in $2^n$ cells after $n$ divisions.
Biological "cell division" is a typical example of exponential order growth: the initial state is $1$ cell, after one round of division it becomes $2$, after two rounds it becomes $4$, and so on; after $n$ rounds of division there are $2^n$ cells.
The figure below and code simulate the cell division process, with a time complexity of $O(2^n)$:
The figure below and the following code simulate the cell division process, with a time complexity of $O(2^n)$. Note that the input $n$ represents the number of division rounds, and the return value `count` represents the total number of divisions.
```src
[file]{time_complexity}-[class]{}-[func]{exponential}
```
![Exponential order time complexity](time_complexity.assets/time_complexity_exponential.png)
![Time complexity of exponential order](time_complexity.assets/time_complexity_exponential.png)
In practice, exponential order often appears in recursive functions. For example, in the code below, it recursively splits into two halves, stopping after $n$ divisions:
In actual algorithms, exponential order often appears in recursive functions. For example, in the following code, it recursively splits in two, stopping after $n$ splits:
```src
[file]{time_complexity}-[class]{}-[func]{exp_recur}
```
Exponential order growth is extremely rapid and is commonly seen in exhaustive search methods (brute force, backtracking, etc.). For large-scale problems, exponential order is unacceptable, often requiring dynamic programming or greedy algorithms as solutions.
Exponential order growth is very rapid and is common in exhaustive methods (brute force search, backtracking, etc.). For problems with large data scales, exponential order is unacceptable and typically requires dynamic programming or greedy algorithms to solve.
### Logarithmic order $O(\log n)$
In contrast to exponential order, logarithmic order reflects situations where "the size is halved each round." Given an input data size $n$, since the size is halved each round, the number of iterations is $\log_2 n$, the inverse function of $2^n$.
In contrast to exponential order, logarithmic order reflects the situation of "reducing to half each round". Let the input data size be $n$. Since it is reduced to half each round, the number of loops is $\log_2 n$, which is the inverse function of $2^n$.
The figure below and code simulate the "halving each round" process, with a time complexity of $O(\log_2 n)$, commonly abbreviated as $O(\log n)$:
The figure below and the following code simulate the process of "reducing to half each round", with a time complexity of $O(\log_2 n)$, abbreviated as $O(\log n)$:
```src
[file]{time_complexity}-[class]{}-[func]{logarithmic}
```
![Logarithmic order time complexity](time_complexity.assets/time_complexity_logarithmic.png)
![Time complexity of logarithmic order](time_complexity.assets/time_complexity_logarithmic.png)
Like exponential order, logarithmic order also frequently appears in recursive functions. The code below forms a recursive tree of height $\log_2 n$:
Like exponential order, logarithmic order also commonly appears in recursive functions. The following code forms a recursion tree of height $\log_2 n$:
```src
[file]{time_complexity}-[class]{}-[func]{log_recur}
```
Logarithmic order is typical in algorithms based on the divide-and-conquer strategy, embodying the "split into many" and "simplify complex problems" approach. It's slow-growing and is the most ideal time complexity after constant order.
Logarithmic order commonly appears in algorithms based on the divide-and-conquer strategy, embodying the algorithmic thinking of "dividing into many" and "simplifying complexity". It grows slowly and is the ideal time complexity second only to constant order.
!!! tip "What is the base of $O(\log n)$?"
Technically, "splitting into $m$" corresponds to a time complexity of $O(\log_m n)$. Using the logarithm base change formula, we can equate different logarithmic complexities:
To be precise, "dividing into $m$" corresponds to a time complexity of $O(\log_m n)$. And through the logarithmic base change formula, we can obtain time complexities with different bases that are equal:
$$
O(\log_m n) = O(\log_k n / \log_k m) = O(\log_k n)
$$
This means the base $m$ can be changed without affecting the complexity. Therefore, we often omit the base $m$ and simply denote logarithmic order as $O(\log n)$.
That is to say, the base $m$ can be converted without affecting the complexity. Therefore, we usually omit the base $m$ and denote logarithmic order simply as $O(\log n)$.
### Linear-logarithmic order $O(n \log n)$
### Linearithmic order $O(n \log n)$
Linear-logarithmic order often appears in nested loops, with the complexities of the two loops being $O(\log n)$ and $O(n)$ respectively. The related code is as follows:
Linearithmic order commonly appears in nested loops, where the time complexities of the two layers of loops are $O(\log n)$ and $O(n)$ respectively. The relevant code is as follows:
```src
[file]{time_complexity}-[class]{}-[func]{linear_log_recur}
```
The figure below demonstrates how linear-logarithmic order is generated. Each level of a binary tree has $n$ operations, and the tree has $\log_2 n + 1$ levels, resulting in a time complexity of $O(n \log n)$.
The figure below shows how linearithmic order is generated. Each level of the binary tree has a total of $n$ operations, and the tree has $\log_2 n + 1$ levels, resulting in a time complexity of $O(n \log n)$.
![Linear-logarithmic order time complexity](time_complexity.assets/time_complexity_logarithmic_linear.png)
![Time complexity of linearithmic order](time_complexity.assets/time_complexity_logarithmic_linear.png)
Mainstream sorting algorithms typically have a time complexity of $O(n \log n)$, such as quicksort, mergesort, and heapsort.
Mainstream sorting algorithms typically have a time complexity of $O(n \log n)$, such as quicksort, merge sort, and heap sort.
### Factorial order $O(n!)$
Factorial order corresponds to the mathematical problem of "full permutation." Given $n$ distinct elements, the total number of possible permutations is:
Factorial order corresponds to the mathematical "permutation" problem. Given $n$ distinct elements, find all possible permutation schemes; the number of schemes is:
$$
n! = n \times (n - 1) \times (n - 2) \times \dots \times 2 \times 1
$$
Factorials are typically implemented using recursion. As shown in the code and the figure below, the first level splits into $n$ branches, the second level into $n - 1$ branches, and so on, stopping after the $n$th level:
Factorials are typically implemented using recursion. As shown in the figure below and the following code, the first level splits into $n$ branches, the second level splits into $n - 1$ branches, and so on, until the $n$-th level when splitting stops:
```src
[file]{time_complexity}-[class]{}-[func]{factorial_recur}
```
![Factorial order time complexity](time_complexity.assets/time_complexity_factorial.png)
![Time complexity of factorial order](time_complexity.assets/time_complexity_factorial.png)
Note that factorial order grows even faster than exponential order; it's unacceptable for larger $n$ values.
Note that because when $n \geq 4$ we always have $n! > 2^n$, factorial order grows faster than exponential order, and is also unacceptable for large $n$.
## Worst, best, and average time complexities
**The time efficiency of an algorithm is often not fixed but depends on the distribution of the input data**. Assume we have an array `nums` of length $n$, consisting of numbers from $1$ to $n$, each appearing only once, but in a randomly shuffled order. The task is to return the index of the element $1$. We can draw the following conclusions:
**The time efficiency of an algorithm is often not fixed, but is related to the distribution of the input data**. Suppose we input an array `nums` of length $n$, where `nums` consists of numbers from $1$ to $n$, with each number appearing only once, but the element order is randomly shuffled. The task is to return the index of element $1$. We can draw the following conclusions.
- When `nums = [?, ?, ..., 1]`, that is, when the last element is $1$, it requires a complete traversal of the array, **achieving the worst-case time complexity of $O(n)$**.
- When `nums = [1, ?, ?, ...]`, that is, when the first element is $1$, no matter the length of the array, no further traversal is needed, **achieving the best-case time complexity of $\Omega(1)$**.
- When `nums = [?, ?, ..., 1]`, i.e., when the last element is $1$, it requires a complete traversal of the array, **reaching worst-case time complexity $O(n)$**.
- When `nums = [1, ?, ?, ...]`, i.e., when the first element is $1$, no matter how long the array is, there is no need to continue traversing, **reaching best-case time complexity $\Omega(1)$**.
The "worst-case time complexity" corresponds to the asymptotic upper bound, denoted by the big $O$ notation. Correspondingly, the "best-case time complexity" corresponds to the asymptotic lower bound, denoted by $\Omega$:
The "worst-case time complexity" corresponds to the function's asymptotic upper bound, denoted using big-$O$ notation. Correspondingly, the "best-case time complexity" corresponds to the function's asymptotic lower bound, denoted using $\Omega$ notation:
```src
[file]{worst_best_time_complexity}-[class]{}-[func]{find_one}
```
It's important to note that the best-case time complexity is rarely used in practice, as it is usually only achievable under very low probabilities and might be misleading. **The worst-case time complexity is more practical as it provides a safety value for efficiency**, allowing us to confidently use the algorithm.
It is worth noting that we rarely use best-case time complexity in practice, because it can usually only be achieved with a very small probability and may be somewhat misleading. **The worst-case time complexity is more practical because it gives a safety value for efficiency**, allowing us to use the algorithm with confidence.
From the above example, it's clear that both the worst-case and best-case time complexities only occur under "special data distributions," which may have a small probability of occurrence and may not accurately reflect the algorithm's run efficiency. In contrast, **the average time complexity can reflect the algorithm's efficiency under random input data**, denoted by the $\Theta$ notation.
From the above example, we can see that both worst-case and best-case time complexities only occur under "special data distributions", which may have a very small probability of occurrence and may not truly reflect the algorithm's running efficiency. In contrast, **average time complexity can reflect the algorithm's running efficiency under random input data**, denoted using the $\Theta$ notation.
For some algorithms, we can simply estimate the average case under a random data distribution. For example, in the aforementioned example, since the input array is shuffled, the probability of element $1$ appearing at any index is equal. Therefore, the average number of loops for the algorithm is half the length of the array $n / 2$, giving an average time complexity of $\Theta(n / 2) = \Theta(n)$.
For some algorithms, we can simply derive the average case under random data distribution. For example, in the above example, since the input array is shuffled, the probability of element $1$ appearing at any index is equal, so the algorithm's average number of loops is half the array length $n / 2$, giving an average time complexity of $\Theta(n / 2) = \Theta(n)$.
However, calculating the average time complexity for more complex algorithms can be quite difficult, as it's challenging to analyze the overall mathematical expectation under the data distribution. In such cases, we usually use the worst-case time complexity as the standard for judging the efficiency of the algorithm.
But for more complex algorithms, calculating average time complexity is often quite difficult, because it is hard to analyze the overall mathematical expectation under data distribution. In this case, we usually use worst-case time complexity as the criterion for judging algorithm efficiency.
!!! question "Why is the $\Theta$ symbol rarely seen?"
Possibly because the $O$ notation is more commonly spoken, it is often used to represent the average time complexity. However, strictly speaking, this practice is not accurate. In this book and other materials, if you encounter statements like "average time complexity $O(n)$", please understand it directly as $\Theta(n)$.
This may be because the $O$ symbol is too catchy, so we often use it to represent average time complexity. But strictly speaking, this practice is not standard. In this book and other materials, if you encounter expressions like "average time complexity $O(n)$", please understand it directly as $\Theta(n)$.