xinyin025/hello-algo
1
0
Fork 0
mirror of https://github.com/krahets/hello-algo.git synced 2026-06-28 00:24:21 +00:00
Code Issues Packages Projects Releases Wiki Activity

Revisit the English version (#1835)

Browse Source 
* Review the English version using Claude-4.5.

* Update mkdocs.yml

* Align the section titles.

* Bug fixes
This commit is contained in:
Yudong Jin
2025-12-30 17:54:01 +08:00
committed by GitHub
parent 091afd38b4
commit 45e1295241
106 changed files with 4195 additions and 3398 deletions
Download Patch File Download Diff File Expand all files Collapse all files
en/docs/chapter_computational_complexity/index.md
+3 -3
View File
@@ -1,9 +1,9 @@
# Complexity analysis
# Complexity Analysis
![Complexity analysis](../assets/covers/chapter_complexity_analysis.jpg)
!!! abstract
Complexity analysis is like a space-time navigator in the vast universe of algorithms.
Complexity analysis is like a space-time guide in the vast universe of algorithms.
It guides us in exploring deeper within the dimensions of time and space, seeking more elegant solutions.
It leads us to explore deeply within the two dimensions of time and space, seeking more elegant solutions.
en/docs/chapter_computational_complexity/iteration_and_recursion.md
+81 -81
View File
@@ -1,77 +1,77 @@
# Iteration and recursion
# Iteration and Recursion
In algorithms, the repeated execution of a task is quite common and is closely related to the analysis of complexity. Therefore, before delving into the concepts of time complexity and space complexity, let's first explore how to implement repetitive tasks in programming. This involves understanding two fundamental programming control structures: iteration and recursion.
In algorithms, repeatedly executing a task is very common and closely related to complexity analysis. Therefore, before introducing time complexity and space complexity, let's first understand how to implement repeated task execution in programs, namely the two basic program control structures: iteration and recursion.
## Iteration
<u>Iteration</u> is a control structure for repeatedly performing a task. In iteration, a program repeats a block of code as long as a certain condition is met until this condition is no longer satisfied.
<u>Iteration</u> is a control structure for repeatedly executing a task. In iteration, a program repeatedly executes a segment of code under certain conditions until those conditions are no longer satisfied.
### For loops
### for Loop
The `for` loop is one of the most common forms of iteration, and **it's particularly suitable when the number of iterations is known in advance**.
The `for` loop is one of the most common forms of iteration, **suitable for use when the number of iterations is known in advance**.
The following function uses a `for` loop to perform a summation of $1 + 2 + \dots + n$, with the sum being stored in the variable `res`. It's important to note that in Python, `range(a, b)` creates an interval that is inclusive of `a` but exclusive of `b`, meaning it iterates over the range from $a$ up to $b−1$.
The following function implements the summation $1 + 2 + \dots + n$ based on a `for` loop, with the sum result recorded using the variable `res`. Note that in Python, `range(a, b)` corresponds to a "left-closed, right-open" interval, with the traversal range being $a, a + 1, \dots, b-1$:
```src
[file]{iteration}-[class]{}-[func]{for_loop}
```
The figure below represents this sum function.
The figure below shows the flowchart of this summation function.
![Flowchart of the sum function](iteration_and_recursion.assets/iteration.png)
![Flowchart of the summation function](iteration_and_recursion.assets/iteration.png)
The number of operations in this summation function is proportional to the size of the input data $n$, or in other words, it has a linear relationship. **This "linear relationship" is what time complexity describes**. This topic will be discussed in more detail in the next section.
The number of operations in this summation function is proportional to the input data size $n$, or has a "linear relationship". In fact, **time complexity describes precisely this "linear relationship"**. Related content will be introduced in detail in the next section.
### While loops
### while Loop
Similar to `for` loops, `while` loops are another approach for implementing iteration. In a `while` loop, the program checks a condition at the beginning of each iteration; if the condition is true, the execution continues, otherwise, the loop ends.
Similar to the `for` loop, the `while` loop is also a method for implementing iteration. In a `while` loop, the program first checks the condition in each round; if the condition is true, it continues execution, otherwise it ends the loop.
Below we use a `while` loop to implement the sum $1 + 2 + \dots + n$.
Below we use a `while` loop to implement the summation $1 + 2 + \dots + n$:
```src
[file]{iteration}-[class]{}-[func]{while_loop}
```
**`while` loops provide more flexibility than `for` loops**, especially since they allow for custom initialization and modification of the condition variable at each step.
**The `while` loop has greater flexibility than the `for` loop**. In a `while` loop, we can freely design the initialization and update steps of the condition variable.
For example, in the following code, the condition variable $i$ is updated twice each round, which would be inconvenient to implement with a `for` loop.
For example, in the following code, the condition variable $i$ is updated twice per round, which is not convenient to implement using a `for` loop:
```src
[file]{iteration}-[class]{}-[func]{while_loop_ii}
```
Overall, **`for` loops are more concise, while `while` loops are more flexible**. Both can implement iterative structures. Which one to use should be determined based on the specific requirements of the problem.
Overall, **`for` loops have more compact code, while `while` loops are more flexible**; both can implement iterative structures. The choice of which to use should be determined based on the requirements of the specific problem.
### Nested loops
### Nested Loops
We can nest one loop structure within another. Below is an example using `for` loops:
We can nest one loop structure inside another. Below is an example using `for` loops:
```src
[file]{iteration}-[class]{}-[func]{nested_for_loop}
```
The figure below represents this nested loop.
The figure below shows the flowchart of this nested loop.
![Flowchart of the nested loop](iteration_and_recursion.assets/nested_iteration.png)
![Flowchart of nested loops](iteration_and_recursion.assets/nested_iteration.png)
In such cases, the number of operations of the function is proportional to $n^2$, meaning the algorithm's runtime and the size of the input data $n$ has a 'quadratic relationship.'
In this case, the number of operations of the function is proportional to $n^2$, or the algorithm's running time has a "quadratic relationship" with the input data size $n$.
We can further increase the complexity by adding more nested loops, each level of nesting effectively "increasing the dimension," which raises the time complexity to "cubic," "quartic," and so on.
We can continue adding nested loops, where each nesting is a "dimension increase", raising the time complexity to "cubic relationship", "quartic relationship", and so on.
## Recursion
<u>Recursion</u> is an algorithmic strategy where a function solves a problem by calling itself. It primarily involves two phases:
<u>Recursion</u> is an algorithmic strategy that solves problems by having a function call itself. It mainly consists of two phases.
1. **Calling**: This is where the program repeatedly calls itself, often with progressively smaller or simpler arguments, moving towards the "termination condition."
2. **Returning**: Upon triggering the "termination condition," the program begins to return from the deepest recursive function, aggregating the results of each layer.
1. **Descend**: The program continuously calls itself deeper, usually passing in smaller or more simplified parameters, until reaching a "termination condition".
2. **Ascend**: After triggering the "termination condition", the program returns layer by layer from the deepest recursive function, aggregating the result of each layer.
From an implementation perspective, recursive code mainly includes three elements.
From an implementation perspective, recursive code mainly consists of three elements.
1. **Termination Condition**: Determines when to switch from "calling" to "returning."
2. **Recursive Call**: Corresponds to "calling," where the function calls itself, usually with smaller or more simplified parameters.
3. **Return Result**: Corresponds to "returning," where the result of the current recursion level is returned to the previous layer.
1. **Termination condition**: Used to determine when to switch from "descending" to "ascending".
2. **Recursive call**: Corresponds to "descending", where the function calls itself, usually with smaller or more simplified parameters.
3. **Return result**: Corresponds to "ascending", returning the result of the current recursion level to the previous layer.
Observe the following code, where simply calling the function `recur(n)` can compute the sum of $1 + 2 + \dots + n$:
Observe the following code. We only need to call the function `recur(n)` to complete the calculation of $1 + 2 + \dots + n$:
```src
[file]{recursion}-[class]{}-[func]{recur}
@@ -79,116 +79,116 @@ Observe the following code, where simply calling the function `recur(n)` can com
The figure below shows the recursive process of this function.
![Recursive process of the sum function](iteration_and_recursion.assets/recursion_sum.png)
![Recursive process of the summation function](iteration_and_recursion.assets/recursion_sum.png)
Although iteration and recursion can achieve the same results from a computational standpoint, **they represent two entirely different paradigms of thinking and problem-solving**.
Although from a computational perspective, iteration and recursion can achieve the same results, **they represent two completely different paradigms for thinking about and solving problems**.
- **Iteration**: Solves problems "from the bottom up." It starts with the most basic steps, and then repeatedly adds or accumulates these steps until the task is complete.
- **Recursion**: Solves problems "from the top down." It breaks down the original problem into smaller sub-problems, each of which has the same form as the original problem. These sub-problems are then further decomposed into even smaller sub-problems, stopping at the base case whose solution is known.
- **Iteration**: Solves problems "bottom-up". Starting from the most basic steps, these steps are then repeatedly executed or accumulated until the task is complete.
- **Recursion**: Solves problems "top-down". The original problem is decomposed into smaller subproblems that have the same form as the original problem. These subproblems continue to be decomposed into even smaller subproblems until reaching the base case (where the solution is known).
Let's take the earlier example of the summation function, defined as $f(n) = 1 + 2 + \dots + n$.
Taking the above summation function as an example, let the problem be $f(n) = 1 + 2 + \dots + n$.
- **Iteration**: In this approach, we simulate the summation process within a loop. Starting from $1$ and traversing to $n$, we perform the summation operation in each iteration to eventually compute $f(n)$.
- **Recursion**: Here, the problem is broken down into a sub-problem: $f(n) = n + f(n-1)$. This decomposition continues recursively until reaching the base case, $f(1) = 1$, at which point the recursion terminates.
- **Iteration**: Simulates the summation process in a loop, traversing from $1$ to $n$, performing the summation operation in each round to obtain $f(n)$.
- **Recursion**: Decomposes the problem into the subproblem $f(n) = n + f(n-1)$, continuously decomposing (recursively) until terminating at the base case $f(1) = 1$.
### Call stack
### Call Stack
Every time a recursive function calls itself, the system allocates memory for the newly initiated function to store local variables, the return address, and other relevant information. This leads to two primary outcomes.
Each time a recursive function calls itself, the system allocates memory for the newly opened function to store local variables, call addresses, and other information. This leads to two consequences.
- The function's context data is stored in a memory area called "stack frame space" and is only released after the function returns. Therefore, **recursion generally consumes more memory space than iteration**.
- Recursive calls introduce additional overhead. **Hence, recursion is usually less time-efficient than loops.**
- The function's context data is stored in a memory area called "stack frame space", which is not released until the function returns. Therefore, **recursion usually consumes more memory space than iteration**.
- Recursive function calls incur additional overhead. **Therefore, recursion is usually less time-efficient than loops**.
As shown in the figure below, there are $n$ unreturned recursive functions before triggering the termination condition, indicating a **recursion depth of $n$**.
As shown in the figure below, before the termination condition is triggered, there are $n$ unreturned recursive functions existing simultaneously, with a **recursion depth of $n$**.
![Recursion call depth](iteration_and_recursion.assets/recursion_sum_depth.png)
In practice, the depth of recursion allowed by programming languages is usually limited, and excessively deep recursion can lead to stack overflow errors.
In practice, the recursion depth allowed by programming languages is usually limited, and excessively deep recursion may lead to stack overflow errors.
### Tail recursion
### Tail Recursion
Interestingly, **if a function performs its recursive call as the very last step before returning,** it can be optimized by the compiler or interpreter to be as space-efficient as iteration. This scenario is known as <u>tail recursion</u>.
Interestingly, **if a function makes the recursive call as the very last step before returning**, the function can be optimized by the compiler or interpreter to have space efficiency comparable to iteration. This case is called <u>tail recursion</u>.
- **Regular recursion**: In standard recursion, when the function returns to the previous level, it continues to execute more code, requiring the system to save the context of the previous call.
- **Tail recursion**: Here, the recursive call is the final operation before the function returns. This means that upon returning to the previous level, no further actions are needed, so the system does not need to save the context of the previous level.
- **Regular recursion**: When a function returns to the previous level, it needs to continue executing code, so the system needs to save the context of the previous layer's call.
- **Tail recursion**: The recursive call is the last operation before the function returns, meaning that after returning to the previous level, there is no need to continue executing other operations, so the system does not need to save the context of the previous layer's function.
For example, in calculating $1 + 2 + \dots + n$, we can make the result variable `res` a parameter of the function, thereby achieving tail recursion:
Taking the calculation of $1 + 2 + \dots + n$ as an example, we can set the result variable `res` as a function parameter to implement tail recursion:
```src
[file]{recursion}-[class]{}-[func]{tail_recur}
```
The execution process of tail recursion is shown in the figure below. Comparing regular recursion and tail recursion, the point of the summation operation is different.
The execution process of tail recursion is shown in the figure below. Comparing regular recursion and tail recursion, the execution point of the summation operation is different.
- **Regular recursion**: The summation operation occurs during the "returning" phase, requiring another summation after each layer returns.
- **Tail recursion**: The summation operation occurs during the "calling" phase, and the "returning" phase only involves returning through each layer.
- **Regular recursion**: The summation operation is performed during the "ascending" process, requiring an additional summation operation after each layer returns.
- **Tail recursion**: The summation operation is performed during the "descending" process; the "ascending" process only needs to return layer by layer.
![Tail recursion process](iteration_and_recursion.assets/tail_recursion_sum.png)
!!! tip
Note that many compilers or interpreters do not support tail recursion optimization. For example, Python does not support tail recursion optimization by default, so even if the function is in the form of tail recursion, it may still encounter stack overflow issues.
Please note that many compilers or interpreters do not support tail recursion optimization. For example, Python does not support tail recursion optimization by default, so even if a function is in tail recursive form, it may still encounter stack overflow issues.
### Recursion tree
### Recursion Tree
When dealing with algorithms related to "divide and conquer", recursion often offers a more intuitive approach and more readable code than iteration. Take the "Fibonacci sequence" as an example.
When dealing with algorithmic problems related to "divide and conquer", recursion often provides a more intuitive approach and more readable code than iteration. Taking the "Fibonacci sequence" as an example.
!!! question
Given a Fibonacci sequence $0, 1, 1, 2, 3, 5, 8, 13, \dots$, find the $n$th number in the sequence.
Given a Fibonacci sequence $0, 1, 1, 2, 3, 5, 8, 13, \dots$, find the $n$-th number in the sequence.
Let the $n$th number of the Fibonacci sequence be $f(n)$, it's easy to deduce two conclusions:
Let the $n$-th number of the Fibonacci sequence be $f(n)$. Two conclusions can be easily obtained.
- The first two numbers of the sequence are $f(1) = 0$ and $f(2) = 1$.
- Each number in the sequence is the sum of the two preceding ones, that is, $f(n) = f(n - 1) + f(n - 2)$.
- Each number in the sequence is the sum of the previous two numbers, i.e., $f(n) = f(n - 1) + f(n - 2)$.
Using the recursive relation, and considering the first two numbers as termination conditions, we can write the recursive code. Calling `fib(n)` will yield the $n$th number of the Fibonacci sequence:
Following the recurrence relation to make recursive calls, with the first two numbers as termination conditions, we can write the recursive code. Calling `fib(n)` will give us the $n$-th number of the Fibonacci sequence:
```src
[file]{recursion}-[class]{}-[func]{fib}
```
Observing the above code, we see that it recursively calls two functions within itself, **meaning that one call generates two branching calls**. As illustrated in the figure below, this continuous recursive calling eventually creates a <u>recursion tree</u> with a depth of $n$.
Observing the above code, we recursively call two functions within the function, **meaning that one call produces two call branches**. As shown in the figure below, such continuous recursive calling will eventually produce a <u>recursion tree</u> with $n$ levels.
![Fibonacci sequence recursion tree](iteration_and_recursion.assets/recursion_tree.png)
![Recursion tree of the Fibonacci sequence](iteration_and_recursion.assets/recursion_tree.png)
Fundamentally, recursion embodies the paradigm of "breaking down a problem into smaller sub-problems." This divide-and-conquer strategy is crucial.
Fundamentally, recursion embodies the paradigm of "decomposing a problem into smaller subproblems", and this divide-and-conquer strategy is crucial.
- From an algorithmic perspective, many important strategies like searching, sorting, backtracking, divide-and-conquer, and dynamic programming directly or indirectly use this way of thinking.
- From a data structure perspective, recursion is naturally suited for dealing with linked lists, trees, and graphs, as they are well suited for analysis using the divide-and-conquer approach.
- From an algorithmic perspective, many important algorithmic strategies such as searching, sorting, backtracking, divide and conquer, and dynamic programming directly or indirectly apply this way of thinking.
- From a data structure perspective, recursion is naturally suited for handling problems related to linked lists, trees, and graphs, because they are well-suited for analysis using divide-and-conquer thinking.
## Comparison
## Comparison of the Two
Summarizing the above content, the following table shows the differences between iteration and recursion in terms of implementation, performance, and applicability.
Summarizing the above content, as shown in the table below, iteration and recursion differ in implementation, performance, and applicability.
<p align="center"> Table: Comparison of iteration and recursion characteristics </p>
<p align="center"> Table <id> &nbsp; Comparison of iteration and recursion characteristics </p>
| | Iteration | Recursion |
| ----------------- | ----------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- |
| Approach | Loop structure | Function calls itself |
| Time Efficiency | Generally higher efficiency, no function call overhead | Each function call generates overhead |
| Memory Usage | Typically uses a fixed size of memory space | Accumulative function calls can use a substantial amount of stack frame space |
| Suitable Problems | Suitable for simple loop tasks, intuitive and readable code | Suitable for problem decomposition, like trees, graphs, divide-and-conquer, backtracking, etc., concise and clear code structure |
| | Iteration | Recursion |
| -------------- | -------------------------------------------------------- | -------------------------------------------------------------------------------------- |
| Implementation | Loop structure | Function calls itself |
| Time efficiency | Generally more efficient, no function call overhead | Each function call incurs overhead |
| Memory usage | Usually uses a fixed amount of memory space | Accumulated function calls may use a large amount of stack frame space |
| Suitable problems | Suitable for simple loop tasks, with intuitive and readable code | Suitable for subproblem decomposition, such as trees, graphs, divide and conquer, backtracking, etc., with concise and clear code structure |
!!! tip
If you find the following content difficult to understand, consider revisiting it after reading the "Stack" chapter.
If you find the following content difficult to understand, you can review it after reading the "Stack" chapter.
So, what is the intrinsic connection between iteration and recursion? Taking the above recursive function as an example, the summation operation occurs during the recursion's "return" phase. This means that the initially called function is the last to complete its summation operation, **mirroring the "last in, first out" principle of a stack**.
What is the intrinsic relationship between iteration and recursion? Taking the above recursive function as an example, the summation operation is performed during the "ascending" phase of recursion. This means that the function called first actually completes its summation operation last, **and this working mechanism is similar to the "last-in, first-out" principle of stacks**.
Recursive terms like "call stack" and "stack frame space" hint at the close relationship between recursion and stacks.
In fact, recursive terminology such as "call stack" and "stack frame space" already hints at the close relationship between recursion and stacks.
1. **Calling**: When a function is called, the system allocates a new stack frame on the "call stack" for that function, storing local variables, parameters, return addresses, and other data.
2. **Returning**: When a function completes execution and returns, the corresponding stack frame is removed from the "call stack," restoring the execution environment of the previous function.
1. **Descend**: When a function is called, the system allocates a new stack frame on the "call stack" for that function to store the function's local variables, parameters, return address, and other data.
2. **Ascend**: When the function completes execution and returns, the corresponding stack frame is removed from the "call stack", restoring the execution environment of the previous function.
Therefore, **we can use an explicit stack to simulate the behavior of the call stack**, thus transforming recursion into an iterative form:
Therefore, **we can use an explicit stack to simulate the behavior of the call stack**, thus transforming recursion into iterative form:
```src
[file]{recursion}-[class]{}-[func]{for_loop_recur}
```
Observing the above code, when recursion is transformed into iteration, the code becomes more complex. Although iteration and recursion can often be transformed into each other, it's not always advisable to do so for two reasons:
Observing the above code, when recursion is transformed into iteration, the code becomes more complex. Although iteration and recursion can be converted into each other in many cases, it may not be worthwhile to do so for the following two reasons.
- The transformed code may become more challenging to understand and less readable.
- For some complex problems, simulating the behavior of the system's call stack can be quite challenging.
- The transformed code may be more difficult to understand and less readable.
- For some complex problems, simulating the behavior of the system call stack can be very difficult.
In conclusion, **whether to choose iteration or recursion depends on the specific nature of the problem**. In programming practice, it's crucial to weigh the pros and cons of both and choose the most suitable approach for the situation at hand.
In summary, **choosing between iteration and recursion depends on the nature of the specific problem**. In programming practice, it is crucial to weigh the pros and cons of both and choose the appropriate method based on the context.
en/docs/chapter_computational_complexity/performance_evaluation.md
+26 -26
View File
@@ -1,49 +1,49 @@
# Algorithm efficiency assessment
# Algorithm Efficiency Evaluation
In algorithm design, we pursue the following two objectives in sequence.
In algorithm design, we pursue the following two levels of objectives sequentially.
1. **Finding a Solution to the Problem**: The algorithm should reliably find the correct solution within the specified range of inputs.
2. **Seeking the Optimal Solution**: For the same problem, multiple solutions might exist, and we aim to find the most efficient algorithm possible.
1. **Finding a solution to the problem**: The algorithm must reliably obtain the correct solution within the specified input range.
2. **Seeking the optimal solution**: Multiple solutions may exist for the same problem, and we hope to find an algorithm that is as efficient as possible.
In other words, under the premise of being able to solve the problem, algorithm efficiency has become the main criterion for evaluating an algorithm, which includes the following two dimensions.
In other words, under the premise of being able to solve the problem, algorithm efficiency has become the primary evaluation criterion for measuring the quality of algorithms. It includes the following two dimensions.
- **Time efficiency**: The speed at which an algorithm runs.
- **Space efficiency**: The size of the memory space occupied by an algorithm.
- **Time efficiency**: The length of time the algorithm runs.
- **Space efficiency**: The size of memory space the algorithm occupies.
In short, **our goal is to design data structures and algorithms that are both fast and memory-efficient**. Effectively assessing algorithm efficiency is crucial because only then can we compare various algorithms and guide the process of algorithm design and optimization.
In short, **our goal is to design data structures and algorithms that are "both fast and memory-efficient"**. Effectively evaluating algorithm efficiency is crucial, because only in this way can we compare various algorithms and guide the algorithm design and optimization process.
There are mainly two methods of efficiency assessment: actual testing and theoretical estimation.
Efficiency evaluation methods are mainly divided into two types: actual testing and theoretical estimation.
## Actual testing
## Actual Testing
Suppose we have algorithms `A` and `B`, both capable of solving the same problem, and we need to compare their efficiencies. The most direct method is to use a computer to run these two algorithms, monitor and record their runtime and memory usage. This assessment method reflects the actual situation, but it has significant limitations.
Suppose we now have algorithm `A` and algorithm `B`, both of which can solve the same problem, and we need to compare the efficiency of these two algorithms. The most direct method is to find a computer, run these two algorithms, and monitor and record their running time and memory usage. This evaluation approach can reflect the real situation, but it also has considerable limitations.
On one hand, **it's difficult to eliminate interference from the testing environment**. Hardware configurations can affect algorithm performance. For example, an algorithm with a high degree of parallelism is better suited for running on multi-core CPUs, while an algorithm that involves intensive memory operations performs better with high-performance memory. The test results of an algorithm may vary across different machines. This means testing across multiple machines to calculate average efficiency becomes impractical.
On one hand, **it is difficult to eliminate interference factors from the testing environment**. Hardware configuration affects the performance of algorithms. For example, if an algorithm has a high degree of parallelism, it is more suitable for running on multi-core CPUs; if an algorithm has intensive memory operations, it will perform better on high-performance memory. In other words, the test results of an algorithm on different machines may be inconsistent. This means we need to test on various machines and calculate average efficiency, which is impractical.
On the other hand, **conducting a full test is very resource-intensive**. Algorithm efficiency varies with input data size. For example, with smaller data volumes, algorithm `A` might run faster than `B`, but with larger data volumes, the test results may be the opposite. Therefore, to draw convincing conclusions, we need to test a wide range of input data sizes, which requires excessive computational resources.
On the other hand, **conducting complete testing is very resource-intensive**. As the input data volume changes, the algorithm will exhibit different efficiencies. For example, when the input data volume is small, the running time of algorithm `A` is shorter than algorithm `B`; but when the input data volume is large, the test results may be exactly the opposite. Therefore, to obtain convincing conclusions, we need to test input data of various scales, which requires a large amount of computational resources.
## Theoretical estimation
## Theoretical Estimation
Due to the significant limitations of actual testing, we can consider evaluating algorithm efficiency solely through calculations. This estimation method is known as <u>asymptotic complexity analysis</u>, or simply <u>complexity analysis</u>.
Since actual testing has considerable limitations, we can consider evaluating algorithm efficiency through calculations alone. This estimation method is called <u>asymptotic complexity analysis</u>, or <u>complexity analysis</u> for short.
Complexity analysis reflects the relationship between the time and space resources required for algorithm execution and the size of the input data. **It describes the trend of growth in the time and space required by the algorithm as the size of the input data increases**. This definition might sound complex, but we can break it down into three key points to understand it better.
Complexity analysis can reflect the relationship between the time and space resources required for algorithm execution and the input data scale. **It describes the growth trend of the time and space required for algorithm execution as the input data scale increases**. This definition is somewhat convoluted, so we can break it down into three key points to understand.
- "Time and space resources" correspond to <u>time complexity</u> and <u>space complexity</u>, respectively.
- "As the size of input data increases" means that complexity reflects the relationship between algorithm efficiency and the volume of input data.
- "The trend of growth in time and space" indicates that complexity analysis focuses not on the specific values of runtime or space occupied, but on the "rate" at which time or space increases.
- "As the input data scale increases" means that complexity reflects the relationship between algorithm running efficiency and input data scale.
- "Growth trend of time and space" indicates that complexity analysis focuses not on the specific values of running time or occupied space, but on how "fast" time or space grows.
**Complexity analysis overcomes the disadvantages of actual testing methods**, reflected in the following aspects:
**Complexity analysis overcomes the drawbacks of the actual testing method**, reflected in the following aspects.
- It does not require actually running the code, making it more environmentally friendly and energy efficient.
- It is independent of the testing environment and applicable to all operating platforms.
- It can reflect algorithm efficiency under different data volumes, especially in the performance of algorithms with large data volumes.
- It does not need to actually run the code, making it more environmentally friendly and energy-efficient.
- It is independent of the testing environment, and the analysis results are applicable to all running platforms.
- It can reflect algorithm efficiency at different data volumes, especially algorithm performance at large data volumes.
!!! tip
If you're still confused about the concept of complexity, don't worry. We will cover it in detail in subsequent chapters.
If you are still confused about the concept of complexity, don't worry—we will introduce it in detail in subsequent chapters.
Complexity analysis provides us with a "ruler" to evaluate the efficiency of an algorithm, enabling us to measure the time and space resources required to execute it and compare the efficiency of different algorithms.
Complexity analysis provides us with a "ruler" for evaluating algorithm efficiency, allowing us to measure the time and space resources required to execute a certain algorithm and compare the efficiency between different algorithms.
Complexity is a mathematical concept that might be abstract and challenging for beginners. From this perspective, complexity analysis might not be the most suitable topic to introduce first. However, when discussing the characteristics of a particular data structure or algorithm, it's hard to avoid analyzing its speed and space usage.
Complexity is a mathematical concept that may be relatively abstract for beginners, with a relatively high learning difficulty. From this perspective, complexity analysis may not be very suitable as the first content to be introduced. However, when we discuss the characteristics of a certain data structure or algorithm, it is difficult to avoid analyzing its running speed and space usage.
In summary, it is recommended to develop a basic understanding of complexity analysis before diving deep into data structures and algorithms, **so that you can perform complexity analysis on simple algorithms**.
In summary, it is recommended that before diving deep into data structures and algorithms, **you first establish a preliminary understanding of complexity analysis so that you can complete complexity analysis of simple algorithms**.
en/docs/chapter_computational_complexity/space_complexity.md
+290 -195
View File
@@ -1,130 +1,129 @@
# Space complexity
# Space Complexity
<u>Space complexity</u> is used to measure the growth trend of the memory space occupied by an algorithm as the amount of data increases. This concept is very similar to time complexity, except that "running time" is replaced with "occupied memory space".
<u>Space complexity</u> measures the growth trend of memory space occupied by an algorithm as the data size increases. This concept is very similar to time complexity, except that "running time" is replaced with "occupied memory space".
## Space related to algorithms
## Algorithm-Related Space
The memory space used by an algorithm during its execution mainly includes the following types.
The memory space used by an algorithm during execution mainly includes the following types.
- **Input space**: Used to store the input data of the algorithm.
- **Temporary space**: Used to store variables, objects, function contexts, and other data during the algorithm's execution.
- **Output space**: Used to store the output data of the algorithm.
Generally, the scope of space complexity statistics includes both "Temporary Space" and "Output Space".
In general, the scope of space complexity statistics is "temporary space" plus "output space".
Temporary space can be further divided into three parts.
- **Temporary data**: Used to save various constants, variables, objects, etc., during the algorithm's execution.
- **Stack frame space**: Used to save the context data of the called function. The system creates a stack frame at the top of the stack each time a function is called, and the stack frame space is released after the function returns.
- **Instruction space**: Used to store compiled program instructions, which are usually negligible in actual statistics.
- **Stack frame space**: Used to save the context data of called functions. The system creates a stack frame at the top of the stack each time a function is called, and the stack frame space is released after the function returns.
- **Instruction space**: Used to save compiled program instructions, which are usually ignored in actual statistics.
When analyzing the space complexity of a program, **we typically count the Temporary Data, Stack Frame Space, and Output Data**, as shown in the figure below.
When analyzing the space complexity of a program, **we usually count three parts: temporary data, stack frame space, and output data**, as shown in the following figure.
![Space types used in algorithms](space_complexity.assets/space_types.png)
![Algorithm-related space](space_complexity.assets/space_types.png)
The relevant code is as follows:
The related code is as follows:
=== "Python"
```python title=""
class Node:
"""Classes"""
"""Class"""
def __init__(self, x: int):
self.val: int = x # node value
self.next: Node | None = None # reference to the next node
self.val: int = x # Node value
self.next: Node | None = None # Reference to the next node
def function() -> int:
"""Functions"""
# Perform certain operations...
"""Function"""
# Perform some operations...
return 0
def algorithm(n) -> int: # input data
A = 0 # temporary data (constant, usually in uppercase)
b = 0 # temporary data (variable)
node = Node(0) # temporary data (object)
c = function() # Stack frame space (call function)
return A + b + c # output data
def algorithm(n) -> int: # Input data
A = 0 # Temporary data (constant, usually represented by uppercase letters)
b = 0 # Temporary data (variable)
node = Node(0) # Temporary data (object)
c = function() # Stack frame space (function call)
return A + b + c # Output data
```
=== "C++"
```cpp title=""
/* Structures */
/* Structure */
struct Node {
int val;
Node *next;
Node(int x) : val(x), next(nullptr) {}
};
/* Functions */
/* Function */
int func() {
// Perform certain operations...
// Perform some operations...
return 0;
}
int algorithm(int n) { // input data
const int a = 0; // temporary data (constant)
int b = 0; // temporary data (variable)
Node* node = new Node(0); // temporary data (object)
int c = func(); // stack frame space (call function)
return a + b + c; // output data
int algorithm(int n) { // Input data
const int a = 0; // Temporary data (constant)
int b = 0; // Temporary data (variable)
Node* node = new Node(0); // Temporary data (object)
int c = func(); // Stack frame space (function call)
return a + b + c; // Output data
}
```
=== "Java"
```java title=""
/* Classes */
/* Class */
class Node {
int val;
Node next;
Node(int x) { val = x; }
}
/* Functions */
/* Function */
int function() {
// Perform certain operations...
// Perform some operations...
return 0;
}
int algorithm(int n) { // input data
final int a = 0; // temporary data (constant)
int b = 0; // temporary data (variable)
Node node = new Node(0); // temporary data (object)
int c = function(); // stack frame space (call function)
return a + b + c; // output data
int algorithm(int n) { // Input data
final int a = 0; // Temporary data (constant)
int b = 0; // Temporary data (variable)
Node node = new Node(0); // Temporary data (object)
int c = function(); // Stack frame space (function call)
return a + b + c; // Output data
}
```
=== "C#"
```csharp title=""
/* Classes */
class Node {
int val;
/* Class */
class Node(int x) {
int val = x;
Node next;
Node(int x) { val = x; }
}
/* Functions */
/* Function */
int Function() {
// Perform certain operations...
// Perform some operations...
return 0;
}
int Algorithm(int n) { // input data
const int a = 0; // temporary data (constant)
int b = 0; // temporary data (variable)
Node node = new(0); // temporary data (object)
int c = Function(); // stack frame space (call function)
return a + b + c; // output data
int Algorithm(int n) { // Input data
const int a = 0; // Temporary data (constant)
int b = 0; // Temporary data (variable)
Node node = new(0); // Temporary data (object)
int c = Function(); // Stack frame space (function call)
return a + b + c; // Output data
}
```
=== "Go"
```go title=""
/* Structures */
/* Structure */
type node struct {
val int
next *node
@@ -134,26 +133,26 @@ The relevant code is as follows:
func newNode(val int) *node {
return &node{val: val}
}
/* Functions */
/* Function */
func function() int {
// Perform certain operations...
// Perform some operations...
return 0
}
func algorithm(n int) int { // input data
const a = 0 // temporary data (constant)
b := 0 // temporary storage of data (variable)
newNode(0) // temporary data (object)
c := function() // stack frame space (call function)
return a + b + c // output data
func algorithm(n int) int { // Input data
const a = 0 // Temporary data (constant)
b := 0 // Temporary data (variable)
newNode(0) // Temporary data (object)
c := function() // Stack frame space (function call)
return a + b + c // Output data
}
```
=== "Swift"
```swift title=""
/* Classes */
/* Class */
class Node {
var val: Int
var next: Node?
@@ -163,99 +162,99 @@ The relevant code is as follows:
}
}
/* Functions */
/* Function */
func function() -> Int {
// Perform certain operations...
// Perform some operations...
return 0
}
func algorithm(n: Int) -> Int { // input data
let a = 0 // temporary data (constant)
var b = 0 // temporary data (variable)
let node = Node(x: 0) // temporary data (object)
let c = function() // stack frame space (call function)
return a + b + c // output data
func algorithm(n: Int) -> Int { // Input data
let a = 0 // Temporary data (constant)
var b = 0 // Temporary data (variable)
let node = Node(x: 0) // Temporary data (object)
let c = function() // Stack frame space (function call)
return a + b + c // Output data
}
```
=== "JS"
```javascript title=""
/* Classes */
/* Class */
class Node {
val;
next;
constructor(val) {
this.val = val === undefined ? 0 : val; // node value
this.next = null; // reference to the next node
this.val = val === undefined ? 0 : val; // Node value
this.next = null; // Reference to the next node
}
}
/* Functions */
/* Function */
function constFunc() {
// Perform certain operations
// Perform some operations
return 0;
}
function algorithm(n) { // input data
const a = 0; // temporary data (constant)
let b = 0; // temporary data (variable)
const node = new Node(0); // temporary data (object)
const c = constFunc(); // Stack frame space (calling function)
return a + b + c; // output data
function algorithm(n) { // Input data
const a = 0; // Temporary data (constant)
let b = 0; // Temporary data (variable)
const node = new Node(0); // Temporary data (object)
const c = constFunc(); // Stack frame space (function call)
return a + b + c; // Output data
}
```
=== "TS"
```typescript title=""
/* Classes */
/* Class */
class Node {
val: number;
next: Node | null;
constructor(val?: number) {
this.val = val === undefined ? 0 : val; // node value
this.next = null; // reference to the next node
this.val = val === undefined ? 0 : val; // Node value
this.next = null; // Reference to the next node
}
}
/* Functions */
/* Function */
function constFunc(): number {
// Perform certain operations
// Perform some operations
return 0;
}
function algorithm(n: number): number { // input data
const a = 0; // temporary data (constant)
let b = 0; // temporary data (variable)
const node = new Node(0); // temporary data (object)
const c = constFunc(); // Stack frame space (calling function)
return a + b + c; // output data
function algorithm(n: number): number { // Input data
const a = 0; // Temporary data (constant)
let b = 0; // Temporary data (variable)
const node = new Node(0); // Temporary data (object)
const c = constFunc(); // Stack frame space (function call)
return a + b + c; // Output data
}
```
=== "Dart"
```dart title=""
/* Classes */
/* Class */
class Node {
int val;
Node next;
Node(this.val, [this.next]);
}
/* Functions */
/* Function */
int function() {
// Perform certain operations...
// Perform some operations...
return 0;
}
int algorithm(int n) { // input data
const int a = 0; // temporary data (constant)
int b = 0; // temporary data (variable)
Node node = Node(0); // temporary data (object)
int c = function(); // stack frame space (call function)
return a + b + c; // output data
int algorithm(int n) { // Input data
const int a = 0; // Temporary data (constant)
int b = 0; // Temporary data (variable)
Node node = Node(0); // Temporary data (object)
int c = function(); // Stack frame space (function call)
return a + b + c; // Output data
}
```
@@ -264,56 +263,102 @@ The relevant code is as follows:
```rust title=""
use std::rc::Rc;
use std::cell::RefCell;
/* Structures */
/* Structure */
struct Node {
val: i32,
next: Option<Rc<RefCell<Node>>>,
}
/* Constructor */
/* Create Node structure */
impl Node {
fn new(val: i32) -> Self {
Self { val: val, next: None }
}
}
/* Functions */
fn function() -> i32 {
// Perform certain operations...
/* Function */
fn function() -> i32 {
// Perform some operations...
return 0;
}
fn algorithm(n: i32) -> i32 { // input data
const a: i32 = 0; // temporary data (constant)
let mut b = 0; // temporary data (variable)
let node = Node::new(0); // temporary data (object)
let c = function(); // stack frame space (call function)
return a + b + c; // output data
fn algorithm(n: i32) -> i32 { // Input data
const a: i32 = 0; // Temporary data (constant)
let mut b = 0; // Temporary data (variable)
let node = Node::new(0); // Temporary data (object)
let c = function(); // Stack frame space (function call)
return a + b + c; // Output data
}
```
=== "C"
```c title=""
/* Functions */
/* Function */
int func() {
// Perform certain operations...
// Perform some operations...
return 0;
}
int algorithm(int n) { // input data
const int a = 0; // temporary data (constant)
int b = 0; // temporary data (variable)
int c = func(); // stack frame space (call function)
return a + b + c; // output data
int algorithm(int n) { // Input data
const int a = 0; // Temporary data (constant)
int b = 0; // Temporary data (variable)
int c = func(); // Stack frame space (function call)
return a + b + c; // Output data
}
```
=== "Kotlin"
```kotlin title=""
/* Class */
class Node(var _val: Int) {
var next: Node? = null
}
/* Function */
fun function(): Int {
// Perform some operations...
return 0
}
fun algorithm(n: Int): Int { // Input data
val a = 0 // Temporary data (constant)
var b = 0 // Temporary data (variable)
val node = Node(0) // Temporary data (object)
val c = function() // Stack frame space (function call)
return a + b + c // Output data
}
```
=== "Ruby"
```ruby title=""
### Class ###
class Node
attr_accessor :val # Node value
attr_accessor :next # Reference to the next node
def initialize(x)
@val = x
end
end
### Function ###
def function
# Perform some operations...
0
end
### Algorithm ###
def algorithm(n) # Input data
a = 0 # Temporary data (constant)
b = 0 # Temporary data (variable)
node = Node.new(0) # Temporary data (object)
c = function # Stack frame space (function call)
a + b + c # Output data
end
```
=== "Zig"
@@ -322,16 +367,16 @@ The relevant code is as follows:
```
## Calculation method
## Calculation Method
The method for calculating space complexity is roughly similar to that of time complexity, with the only change being the shift of the statistical object from "number of operations" to "size of used space".
The calculation method for space complexity is roughly the same as for time complexity, except that the statistical object is changed from "number of operations" to "size of space used".
However, unlike time complexity, **we usually only focus on the worst-case space complexity**. This is because memory space is a hard requirement, and we must ensure that there is enough memory space reserved under all input data.
Unlike time complexity, **we usually only focus on the worst-case space complexity**. This is because memory space is a hard requirement, and we must ensure that sufficient memory space is reserved for all input data.
Consider the following code, the term "worst-case" in worst-case space complexity has two meanings.
Observe the following code. The "worst case" in worst-case space complexity has two meanings.
1. **Based on the worst input data**: When $n < 10$, the space complexity is $O(1)$; but when $n > 10$, the initialized array `nums` occupies $O(n)$ space, thus the worst-case space complexity is $O(n)$.
2. **Based on the peak memory used during the algorithm's execution**: For example, before executing the last line, the program occupies $O(1)$ space; when initializing the array `nums`, the program occupies $O(n)$ space, hence the worst-case space complexity is $O(n)$.
1. **Based on the worst input data**: When $n < 10$, the space complexity is $O(1)$; but when $n > 10$, the initialized array `nums` occupies $O(n)$ space, so the worst-case space complexity is $O(n)$.
2. **Based on the peak memory during algorithm execution**: For example, before executing the last line, the program occupies $O(1)$ space; when initializing the array `nums`, the program occupies $O(n)$ space, so the worst-case space complexity is $O(n)$.
=== "Python"
@@ -443,10 +488,10 @@ Consider the following code, the term "worst-case" in worst-case space complexit
```rust title=""
fn algorithm(n: i32) {
let a = 0; // O(1)
let b = [0; 10000]; // O(1)
let a = 0; // O(1)
let b = [0; 10000]; // O(1)
if n > 10 {
let nums = vec![0; n as usize]; // O(n)
let nums = vec![0; n as usize]; // O(n)
}
}
```
@@ -465,7 +510,23 @@ Consider the following code, the term "worst-case" in worst-case space complexit
=== "Kotlin"
```kotlin title=""
fun algorithm(n: Int) {
val a = 0 // O(1)
val b = IntArray(10000) // O(1)
if (n > 10) {
val nums = IntArray(n) // O(n)
}
}
```
=== "Ruby"
```ruby title=""
def algorithm(n)
a = 0 # O(1)
b = Array.new(10000) # O(1)
nums = Array.new(n) if n > 10 # O(n)
end
```
=== "Zig"
@@ -474,22 +535,22 @@ Consider the following code, the term "worst-case" in worst-case space complexit
```
**In recursive functions, stack frame space must be taken into count**. Consider the following code:
**In recursive functions, it is necessary to count the stack frame space**. Observe the following code:
=== "Python"
```python title=""
def function() -> int:
# Perform certain operations
# Perform some operations
return 0
def loop(n: int):
"""Loop O(1)"""
"""Loop has space complexity of O(1)"""
for _ in range(n):
function()
def recur(n: int):
"""Recursion O(n)"""
"""Recursion has space complexity of O(n)"""
if n == 1:
return
return recur(n - 1)
@@ -499,16 +560,16 @@ Consider the following code, the term "worst-case" in worst-case space complexit
```cpp title=""
int func() {
// Perform certain operations
// Perform some operations
return 0;
}
/* Cycle O(1) */
/* Loop has space complexity of O(1) */
void loop(int n) {
for (int i = 0; i < n; i++) {
func();
}
}
/* Recursion O(n) */
/* Recursion has space complexity of O(n) */
void recur(int n) {
if (n == 1) return;
recur(n - 1);
@@ -519,16 +580,16 @@ Consider the following code, the term "worst-case" in worst-case space complexit
```java title=""
int function() {
// Perform certain operations
// Perform some operations
return 0;
}
/* Cycle O(1) */
/* Loop has space complexity of O(1) */
void loop(int n) {
for (int i = 0; i < n; i++) {
function();
}
}
/* Recursion O(n) */
/* Recursion has space complexity of O(n) */
void recur(int n) {
if (n == 1) return;
recur(n - 1);
@@ -539,16 +600,16 @@ Consider the following code, the term "worst-case" in worst-case space complexit
```csharp title=""
int Function() {
// Perform certain operations
// Perform some operations
return 0;
}
/* Cycle O(1) */
/* Loop has space complexity of O(1) */
void Loop(int n) {
for (int i = 0; i < n; i++) {
Function();
}
}
/* Recursion O(n) */
/* Recursion has space complexity of O(n) */
int Recur(int n) {
if (n == 1) return 1;
return Recur(n - 1);
@@ -559,18 +620,18 @@ Consider the following code, the term "worst-case" in worst-case space complexit
```go title=""
func function() int {
// Perform certain operations
// Perform some operations
return 0
}
/* Cycle O(1) */
/* Loop has space complexity of O(1) */
func loop(n int) {
for i := 0; i < n; i++ {
function()
}
}
/* Recursion O(n) */
/* Recursion has space complexity of O(n) */
func recur(n int) {
if n == 1 {
return
@@ -584,18 +645,18 @@ Consider the following code, the term "worst-case" in worst-case space complexit
```swift title=""
@discardableResult
func function() -> Int {
// Perform certain operations
// Perform some operations
return 0
}
/* Cycle O(1) */
/* Loop has space complexity of O(1) */
func loop(n: Int) {
for _ in 0 ..< n {
function()
}
}
/* Recursion O(n) */
/* Recursion has space complexity of O(n) */
func recur(n: Int) {
if n == 1 {
return
@@ -608,16 +669,16 @@ Consider the following code, the term "worst-case" in worst-case space complexit
```javascript title=""
function constFunc() {
// Perform certain operations
// Perform some operations
return 0;
}
/* Cycle O(1) */
/* Loop has space complexity of O(1) */
function loop(n) {
for (let i = 0; i < n; i++) {
constFunc();
}
}
/* Recursion O(n) */
/* Recursion has space complexity of O(n) */
function recur(n) {
if (n === 1) return;
return recur(n - 1);
@@ -628,16 +689,16 @@ Consider the following code, the term "worst-case" in worst-case space complexit
```typescript title=""
function constFunc(): number {
// Perform certain operations
// Perform some operations
return 0;
}
/* Cycle O(1) */
/* Loop has space complexity of O(1) */
function loop(n: number): void {
for (let i = 0; i < n; i++) {
constFunc();
}
}
/* Recursion O(n) */
/* Recursion has space complexity of O(n) */
function recur(n: number): void {
if (n === 1) return;
return recur(n - 1);
@@ -648,16 +709,16 @@ Consider the following code, the term "worst-case" in worst-case space complexit
```dart title=""
int function() {
// Perform certain operations
// Perform some operations
return 0;
}
/* Cycle O(1) */
/* Loop has space complexity of O(1) */
void loop(int n) {
for (int i = 0; i < n; i++) {
function();
}
}
/* Recursion O(n) */
/* Recursion has space complexity of O(n) */
void recur(int n) {
if (n == 1) return;
recur(n - 1);
@@ -668,17 +729,17 @@ Consider the following code, the term "worst-case" in worst-case space complexit
```rust title=""
fn function() -> i32 {
// Perform certain operations
// Perform some operations
return 0;
}
/* Cycle O(1) */
/* Loop has space complexity of O(1) */
fn loop(n: i32) {
for i in 0..n {
function();
}
}
/* Recursion O(n) */
void recur(n: i32) {
/* Recursion has space complexity of O(n) */
fn recur(n: i32) {
if n == 1 {
return;
}
@@ -690,16 +751,16 @@ Consider the following code, the term "worst-case" in worst-case space complexit
```c title=""
int func() {
// Perform certain operations
// Perform some operations
return 0;
}
/* Cycle O(1) */
/* Loop has space complexity of O(1) */
void loop(int n) {
for (int i = 0; i < n; i++) {
func();
}
}
/* Recursion O(n) */
/* Recursion has space complexity of O(n) */
void recur(int n) {
if (n == 1) return;
recur(n - 1);
@@ -709,7 +770,41 @@ Consider the following code, the term "worst-case" in worst-case space complexit
=== "Kotlin"
```kotlin title=""
fun function(): Int {
// Perform some operations
return 0
}
/* Loop has space complexity of O(1) */
fun loop(n: Int) {
for (i in 0..<n) {
function()
}
}
/* Recursion has space complexity of O(n) */
fun recur(n: Int) {
if (n == 1) return
return recur(n - 1)
}
```
=== "Ruby"
```ruby title=""
def function
# Perform some operations
0
end
### Loop has space complexity of O(1) ###
def loop(n)
(0...n).each { function }
end
### Recursion has space complexity of O(n) ###
def recur(n)
return if n == 1
recur(n - 1)
end
```
=== "Zig"
@@ -718,35 +813,35 @@ Consider the following code, the term "worst-case" in worst-case space complexit
```
The time complexity of both `loop()` and `recur()` functions is $O(n)$, but their space complexities differ.
The time complexity of both functions `loop()` and `recur()` is $O(n)$, but their space complexities are different.
- The `loop()` function calls `function()` $n$ times in a loop, where each iteration's `function()` returns and releases its stack frame space, so the space complexity remains $O(1)$.
- The recursive function `recur()` will have $n$ instances of unreturned `recur()` existing simultaneously during its execution, thus occupying $O(n)$ stack frame space.
- The function `loop()` calls `function()` $n$ times in a loop. In each iteration, `function()` returns and releases its stack frame space, so the space complexity remains $O(1)$.
- The recursive function `recur()` has $n$ unreturned `recur()` instances existing simultaneously during execution, thus occupying $O(n)$ stack frame space.
## Common types
## Common Types
Let the size of the input data be $n$, the figure below displays common types of space complexities (arranged from low to high).
Let the input data size be $n$. The following figure shows common types of space complexity (arranged from low to high).
$$
\begin{aligned}
& O(1) < O(\log n) < O(n) < O(n^2) < O(2^n) \newline
& \text{Constant} < \text{Logarithmic} < \text{Linear} < \text{Quadratic} < \text{Exponential}
O(1) < O(\log n) < O(n) < O(n^2) < O(2^n) \newline
\text{Constant} < \text{Logarithmic} < \text{Linear} < \text{Quadratic} < \text{Exponential}
\end{aligned}
$$
![Common types of space complexity](space_complexity.assets/space_complexity_common_types.png)
### Constant order $O(1)$
### Constant Order $O(1)$
Constant order is common in constants, variables, objects that are independent of the size of input data $n$.
Constant order is common in constants, variables, and objects whose quantity is independent of the input data size $n$.
Note that memory occupied by initializing variables or calling functions in a loop, which is released upon entering the next cycle, does not accumulate over space, thus the space complexity remains $O(1)$:
It should be noted that memory occupied by initializing variables or calling functions in a loop is released when entering the next iteration, so it does not accumulate space, and the space complexity remains $O(1)$:
```src
[file]{space_complexity}-[class]{}-[func]{constant}
```
### Linear order $O(n)$
### Linear Order $O(n)$
Linear order is common in arrays, linked lists, stacks, queues, etc., where the number of elements is proportional to $n$:
@@ -754,50 +849,50 @@ Linear order is common in arrays, linked lists, stacks, queues, etc., where the
[file]{space_complexity}-[class]{}-[func]{linear}
```
As shown in the figure below, this function's recursive depth is $n$, meaning there are $n$ instances of unreturned `linear_recur()` function, using $O(n)$ size of stack frame space:
As shown in the following figure, the recursion depth of this function is $n$, meaning that there are $n$ unreturned `linear_recur()` functions existing simultaneously, using $O(n)$ stack frame space:
```src
[file]{space_complexity}-[class]{}-[func]{linear_recur}
```
![Recursive function generating linear order space complexity](space_complexity.assets/space_complexity_recursive_linear.png)
![Linear order space complexity generated by recursive function](space_complexity.assets/space_complexity_recursive_linear.png)
### Quadratic order $O(n^2)$
### Quadratic Order $O(n^2)$
Quadratic order is common in matrices and graphs, where the number of elements is quadratic to $n$:
Quadratic order is common in matrices and graphs, where the number of elements is quadratically related to $n$:
```src
[file]{space_complexity}-[class]{}-[func]{quadratic}
```
As shown in the figure below, the recursive depth of this function is $n$, and in each recursive call, an array is initialized with lengths $n$, $n-1$, $\dots$, $2$, $1$, averaging $n/2$, thus overall occupying $O(n^2)$ space:
As shown in the following figure, the recursion depth of this function is $n$, and an array is initialized in each recursive function with lengths of $n$, $n-1$, $\dots$, $2$, $1$, with an average length of $n / 2$, thus occupying $O(n^2)$ space overall:
```src
[file]{space_complexity}-[class]{}-[func]{quadratic_recur}
```
![Recursive function generating quadratic order space complexity](space_complexity.assets/space_complexity_recursive_quadratic.png)
![Quadratic order space complexity generated by recursive function](space_complexity.assets/space_complexity_recursive_quadratic.png)
### Exponential order $O(2^n)$
### Exponential Order $O(2^n)$
Exponential order is common in binary trees. Observe the figure below, a "full binary tree" with $n$ levels has $2^n - 1$ nodes, occupying $O(2^n)$ space:
Exponential order is common in binary trees. Observe the following figure: a "full binary tree" with $n$ levels has $2^n - 1$ nodes, occupying $O(2^n)$ space:
```src
[file]{space_complexity}-[class]{}-[func]{build_tree}
```
![Full binary tree generating exponential order space complexity](space_complexity.assets/space_complexity_exponential.png)
![Exponential order space complexity generated by full binary tree](space_complexity.assets/space_complexity_exponential.png)
### Logarithmic order $O(\log n)$
### Logarithmic Order $O(\log n)$
Logarithmic order is common in divide-and-conquer algorithms. For example, in merge sort, an array of length $n$ is recursively divided in half each round, forming a recursion tree of height $\log n$, using $O(\log n)$ stack frame space.
Logarithmic order is common in divide-and-conquer algorithms. For example, merge sort: given an input array of length $n$, each recursion divides the array in half from the midpoint, forming a recursion tree of height $\log n$, using $O(\log n)$ stack frame space.
Another example is converting a number to a string. Given a positive integer $n$, its number of digits is $\log_{10} n + 1$, corresponding to the length of the string, thus the space complexity is $O(\log_{10} n + 1) = O(\log n)$.
Another example is converting a number to a string. Given a positive integer $n$, it has $\lfloor \log_{10} n \rfloor + 1$ digits, i.e., the corresponding string length is $\lfloor \log_{10} n \rfloor + 1$, so the space complexity is $O(\log_{10} n + 1) = O(\log n)$.
## Balancing time and space
## Trading Time for Space
Ideally, we aim for both time complexity and space complexity to be optimal. However, in practice, optimizing both simultaneously is often difficult.
Ideally, we hope that both the time complexity and space complexity of an algorithm can reach optimal. However, in practice, optimizing both time complexity and space complexity simultaneously is usually very difficult.
**Lowering time complexity usually comes at the cost of increased space complexity, and vice versa**. The approach of sacrificing memory space to improve algorithm speed is known as "space-time tradeoff"; the reverse is known as "time-space tradeoff".
**Reducing time complexity usually comes at the cost of increasing space complexity, and vice versa**. The approach of sacrificing memory space to improve algorithm execution speed is called "trading space for time"; conversely, it is called "trading time for space".
The choice depends on which aspect we value more. In most cases, time is more precious than space, so "space-time tradeoff" is often the more common strategy. Of course, controlling space complexity is also very important when dealing with large volumes of data.
The choice of which approach depends on which aspect we value more. In most cases, time is more precious than space, so "trading space for time" is usually the more common strategy. Of course, when the data volume is very large, controlling space complexity is also very important.
en/docs/chapter_computational_complexity/summary.md
+29 -23
View File
@@ -1,49 +1,55 @@
# Summary
### Key review
### Key Review
**Algorithm Efficiency Assessment**
- Time efficiency and space efficiency are the two main criteria for assessing the merits of an algorithm.
- We can assess algorithm efficiency through actual testing, but it's challenging to eliminate the influence of the test environment, and it consumes substantial computational resources.
- Complexity analysis can overcome the disadvantages of actual testing. Its results are applicable across all operating platforms and can reveal the efficiency of algorithms at different data scales.
- Time efficiency and space efficiency are the two primary evaluation metrics for measuring algorithm performance.
- We can evaluate algorithm efficiency through actual testing, but it is difficult to eliminate the influence of the testing environment, and it consumes substantial computational resources.
- Complexity analysis can eliminate the drawbacks of actual testing, with results applicable to all running platforms, and it can reveal algorithm efficiency under different data scales.
**Time Complexity**
- Time complexity measures the trend of an algorithm's running time with the increase in data volume, effectively assessing algorithm efficiency. However, it can fail in certain cases, such as with small input data volumes or when time complexities are the same, making it challenging to precisely compare the efficiency of algorithms.
- Worst-case time complexity is denoted using big-$O$ notation, representing the asymptotic upper bound, reflecting the growth level of the number of operations $T(n)$ as $n$ approaches infinity.
- Calculating time complexity involves two steps: first counting the number of operations, then determining the asymptotic upper bound.
- Common time complexities, arranged from low to high, include $O(1)$, $O(\log n)$, $O(n)$, $O(n \log n)$, $O(n^2)$, $O(2^n)$, and $O(n!)$, among others.
- The time complexity of some algorithms is not fixed and depends on the distribution of input data. Time complexities are divided into worst, best, and average cases. The best case is rarely used because input data generally needs to meet strict conditions to achieve the best case.
- Average time complexity reflects the efficiency of an algorithm under random data inputs, closely resembling the algorithm's performance in actual applications. Calculating average time complexity requires accounting for the distribution of input data and the subsequent mathematical expectation.
- Time complexity is used to measure the trend of algorithm runtime as data volume increases. It can effectively evaluate algorithm efficiency, but may fail in certain situations, such as when the input data volume is small or when time complexities are identical, making it impossible to precisely compare algorithm efficiency.
- Worst-case time complexity is represented using Big $O$ notation, corresponding to the asymptotic upper bound of a function, reflecting the growth level of the number of operations $T(n)$ as $n$ approaches positive infinity.
- Deriving time complexity involves two steps: first, counting the number of operations, then determining the asymptotic upper bound.
- Common time complexities arranged from low to high include $O(1)$, $O(\log n)$, $O(n)$, $O(n \log n)$, $O(n^2)$, $O(2^n)$, and $O(n!)$.
- The time complexity of some algorithms is not fixed, but rather depends on the distribution of input data. Time complexity is divided into worst-case, best-case, and average-case time complexity. Best-case time complexity is rarely used because input data generally needs to satisfy strict conditions to achieve the best case.
- Average time complexity reflects the algorithm's runtime efficiency under random data input, and is closest to the algorithm's performance in practical applications. Calculating average time complexity requires statistical analysis of input data distribution and the combined mathematical expectation.
**Space Complexity**
- Space complexity, similar to time complexity, measures the trend of memory space occupied by an algorithm with the increase in data volume.
- The relevant memory space used during the algorithm's execution can be divided into input space, temporary space, and output space. Generally, input space is not included in space complexity calculations. Temporary space can be divided into temporary data, stack frame space, and instruction space, where stack frame space usually affects space complexity only in recursive functions.
- We usually focus only on the worst-case space complexity, which means calculating the space complexity of the algorithm under the worst input data and at the worst moment of operation.
- Common space complexities, arranged from low to high, include $O(1)$, $O(\log n)$, $O(n)$, $O(n^2)$, and $O(2^n)$, among others.
- Space complexity serves a similar purpose to time complexity, used to measure the trend of algorithm memory usage as data volume increases.
- The memory space related to algorithm execution can be divided into input space, temporary space, and output space. Typically, input space is not included in space complexity calculations. Temporary space can be divided into temporary data, stack frame space, and instruction space, where stack frame space usually affects space complexity only in recursive functions.
- We typically only focus on worst-case space complexity, which is the space complexity of an algorithm under worst-case input data and worst-case runtime.
- Common space complexities arranged from low to high include $O(1)$, $O(\log n)$, $O(n)$, $O(n^2)$, and $O(2^n)$.
### Q & A
**Q**: Is the space complexity of tail recursion $O(1)$?
Theoretically, the space complexity of a tail-recursive function can be optimized to $O(1)$. However, most programming languages (such as Java, Python, C++, Go, C#) do not support automatic optimization of tail recursion, so it's generally considered to have a space complexity of $O(n)$.
Theoretically, the space complexity of tail recursive functions can be optimized to $O(1)$. However, most programming languages (such as Java, Python, C++, Go, C#, etc.) do not support automatic tail recursion optimization, so the space complexity is generally considered to be $O(n)$.
**Q**: What is the difference between the terms "function" and "method"?
**Q**: What is the difference between the terms function and method?
A <u>function</u> can be executed independently, with all parameters passed explicitly. A <u>method</u> is associated with an object and is implicitly passed to the object calling it, able to operate on the data contained within an instance of a class.
A <u>function</u> can be executed independently, with all parameters passed explicitly. A <u>method</u> is associated with an object, is implicitly passed to the object that invokes it, and can operate on data contained in class instances.
Here are some examples from common programming languages:
The following examples use several common programming languages for illustration.
- C is a procedural programming language without object-oriented concepts, so it only has functions. However, we can simulate object-oriented programming by creating structures (struct), and functions associated with these structures are equivalent to methods in other programming languages.
- C is a procedural programming language without object-oriented concepts, so it only has functions. However, we can simulate object-oriented programming by creating structures (struct), and functions associated with structures are equivalent to methods in other programming languages.
- Java and C# are object-oriented programming languages where code blocks (methods) are typically part of a class. Static methods behave like functions because they are bound to the class and cannot access specific instance variables.
- C++ and Python support both procedural programming (functions) and object-oriented programming (methods).
**Q**: Does the "Common Types of Space Complexity" figure reflect the absolute size of occupied space?
**Q**: Does the diagram for "common space complexity types" reflect the absolute size of occupied space?
No, the figure shows space complexities, which reflect growth trends, not the absolute size of the occupied space.
No, the diagram shows space complexity, which reflects growth trends rather than the absolute size of occupied space.
If you take $n = 8$, you might find that the values of each curve don't correspond to their functions. This is because each curve includes a constant term, intended to compress the value range into a visually comfortable range.
Assuming $n = 8$, you might find that the values of each curve do not correspond to the functions. This is because each curve contains a constant term used to compress the value range into a visually comfortable range.
In practice, since we usually don't know the "constant term" complexity of each method, it's generally not possible to choose the best solution for $n = 8$ based solely on complexity. However, for $n = 8^5$, it's much easier to choose, as the growth trend becomes dominant.
In practice, because we generally do not know what the "constant term" complexity of each method is, we usually cannot select the optimal solution for $n = 8$ based on complexity alone. But for $n = 8^5$, the choice is straightforward, as the growth trend already dominates.
**Q**: Are there situations where algorithms are designed to sacrifice time (or space) based on actual use cases?
In practical applications, most situations choose to sacrifice space for time. For example, with database indexes, we typically choose to build B+ trees or hash indexes, occupying substantial memory space in exchange for efficient queries of $O(\log n)$ or even $O(1)$.
In scenarios where space resources are precious, time may be sacrificed for space. For example, in embedded development, device memory is precious, and engineers may forgo using hash tables and choose to use array sequential search to save memory usage, at the cost of slower searches.
en/docs/chapter_computational_complexity/time_complexity.md
+284 -172
View File
@@ -1,22 +1,22 @@
# Time complexity
The runtime can intuitively assess the efficiency of an algorithm. How can we accurately estimate the runtime of a piece of an algorithm?
Runtime can intuitively and accurately reflect the efficiency of an algorithm. If we want to accurately estimate the runtime of a piece of code, how should we proceed?
1. **Determining the Running Platform**: This includes hardware configuration, programming language, system environment, etc., all of which can affect the efficiency of code execution.
2. **Evaluating the Run Time for Various Computational Operations**: For instance, an addition operation `+` might take 1 ns, a multiplication operation `*` might take 10 ns, a print operation `print()` might take 5 ns, etc.
3. **Counting All the Computational Operations in the Code**: Summing the execution times of all these operations gives the total run time.
1. **Determine the running platform**, including hardware configuration, programming language, system environment, etc., as these factors all affect code execution efficiency.
2. **Evaluate the runtime required for various computational operations**, for example, an addition operation `+` requires 1 ns, a multiplication operation `*` requires 10 ns, a print operation `print()` requires 5 ns, etc.
3. **Count all computational operations in the code**, and sum the execution times of all operations to obtain the runtime.
For example, consider the following code with an input size of $n$:
For example, in the following code, the input data size is $n$:
=== "Python"
```python title=""
# Under an operating platform
# On a certain running platform
def algorithm(n: int):
a = 2 # 1 ns
a = a + 1 # 1 ns
a = a * 2 # 10 ns
# Cycle n times
# Loop n times
for _ in range(n): # 1 ns
print(0) # 5 ns
```
@@ -24,13 +24,13 @@ For example, consider the following code with an input size of $n$:
=== "C++"
```cpp title=""
// Under a particular operating platform
// On a certain running platform
void algorithm(int n) {
int a = 2; // 1 ns
a = a + 1; // 1 ns
a = a * 2; // 10 ns
// Loop n times
for (int i = 0; i < n; i++) { // 1 ns , every round i++ is executed
for (int i = 0; i < n; i++) { // 1 ns
cout << 0 << endl; // 5 ns
}
}
@@ -39,13 +39,13 @@ For example, consider the following code with an input size of $n$:
=== "Java"
```java title=""
// Under a particular operating platform
// On a certain running platform
void algorithm(int n) {
int a = 2; // 1 ns
a = a + 1; // 1 ns
a = a * 2; // 10 ns
// Loop n times
for (int i = 0; i < n; i++) { // 1 ns , every round i++ is executed
for (int i = 0; i < n; i++) { // 1 ns
System.out.println(0); // 5 ns
}
}
@@ -54,13 +54,13 @@ For example, consider the following code with an input size of $n$:
=== "C#"
```csharp title=""
// Under a particular operating platform
// On a certain running platform
void Algorithm(int n) {
int a = 2; // 1 ns
a = a + 1; // 1 ns
a = a * 2; // 10 ns
// Loop n times
for (int i = 0; i < n; i++) { // 1 ns , every round i++ is executed
for (int i = 0; i < n; i++) { // 1 ns
Console.WriteLine(0); // 5 ns
}
}
@@ -69,7 +69,7 @@ For example, consider the following code with an input size of $n$:
=== "Go"
```go title=""
// Under a particular operating platform
// On a certain running platform
func algorithm(n int) {
a := 2 // 1 ns
a = a + 1 // 1 ns
@@ -84,7 +84,7 @@ For example, consider the following code with an input size of $n$:
=== "Swift"
```swift title=""
// Under a particular operating platform
// On a certain running platform
func algorithm(n: Int) {
var a = 2 // 1 ns
a = a + 1 // 1 ns
@@ -99,13 +99,13 @@ For example, consider the following code with an input size of $n$:
=== "JS"
```javascript title=""
// Under a particular operating platform
// On a certain running platform
function algorithm(n) {
var a = 2; // 1 ns
a = a + 1; // 1 ns
a = a * 2; // 10 ns
// Loop n times
for(let i = 0; i < n; i++) { // 1 ns , every round i++ is executed
for(let i = 0; i < n; i++) { // 1 ns
console.log(0); // 5 ns
}
}
@@ -114,13 +114,13 @@ For example, consider the following code with an input size of $n$:
=== "TS"
```typescript title=""
// Under a particular operating platform
// On a certain running platform
function algorithm(n: number): void {
var a: number = 2; // 1 ns
a = a + 1; // 1 ns
a = a * 2; // 10 ns
// Loop n times
for(let i = 0; i < n; i++) { // 1 ns , every round i++ is executed
for(let i = 0; i < n; i++) { // 1 ns
console.log(0); // 5 ns
}
}
@@ -129,13 +129,13 @@ For example, consider the following code with an input size of $n$:
=== "Dart"
```dart title=""
// Under a particular operating platform
// On a certain running platform
void algorithm(int n) {
int a = 2; // 1 ns
a = a + 1; // 1 ns
a = a * 2; // 10 ns
// Loop n times
for (int i = 0; i < n; i++) { // 1 ns , every round i++ is executed
for (int i = 0; i < n; i++) { // 1 ns
print(0); // 5 ns
}
}
@@ -144,13 +144,13 @@ For example, consider the following code with an input size of $n$:
=== "Rust"
```rust title=""
// Under a particular operating platform
// On a certain running platform
fn algorithm(n: i32) {
let mut a = 2; // 1 ns
a = a + 1; // 1 ns
a = a * 2; // 10 ns
// Loop n times
for _ in 0..n { // 1 ns for each round i++
for _ in 0..n { // 1 ns
println!("{}", 0); // 5 ns
}
}
@@ -159,13 +159,13 @@ For example, consider the following code with an input size of $n$:
=== "C"
```c title=""
// Under a particular operating platform
// On a certain running platform
void algorithm(int n) {
int a = 2; // 1 ns
a = a + 1; // 1 ns
a = a * 2; // 10 ns
// Loop n times
for (int i = 0; i < n; i++) { // 1 ns , every round i++ is executed
for (int i = 0; i < n; i++) { // 1 ns
printf("%d", 0); // 5 ns
}
}
@@ -174,13 +174,37 @@ For example, consider the following code with an input size of $n$:
=== "Kotlin"
```kotlin title=""
// On a certain running platform
fun algorithm(n: Int) {
var a = 2 // 1 ns
a = a + 1 // 1 ns
a = a * 2 // 10 ns
// Loop n times
for (i in 0..<n) { // 1 ns
println(0) // 5 ns
}
}
```
=== "Ruby"
```ruby title=""
# On a certain running platform
def algorithm(n)
a = 2 # 1 ns
a = a + 1 # 1 ns
a = a * 2 # 10 ns
# Loop n times
(0...n).each do # 1 ns
puts 0 # 5 ns
end
end
```
=== "Zig"
```zig title=""
// Under a particular operating platform
// On a certain running platform
fn algorithm(n: usize) void {
var a: i32 = 2; // 1 ns
a += 1; // 1 ns
@@ -192,19 +216,19 @@ For example, consider the following code with an input size of $n$:
}
```
Using the above method, the run time of the algorithm can be calculated as $(6n + 12)$ ns:
According to the above method, the algorithm's runtime can be obtained as $(6n + 12)$ ns:
$$
1 + 1 + 10 + (1 + 5) \times n = 6n + 12
$$
However, in practice, **counting the run time of an algorithm is neither practical nor reasonable**. First, we don't want to tie the estimated time to the running platform, as algorithms need to run on various platforms. Second, it's challenging to know the run time for each type of operation, making the estimation process difficult.
In reality, however, **counting an algorithm's runtime is neither reasonable nor realistic**. First, we do not want to tie the estimated time to the running platform, because algorithms need to run on various different platforms. Second, it is difficult to know the runtime of each type of operation, which brings great difficulty to the estimation process.
## Assessing time growth trend
## Counting time growth trends
Time complexity analysis does not count the algorithm's run time, **but rather the growth trend of the run time as the data volume increases**.
Time complexity analysis does not count the algorithm's runtime, **but rather counts the growth trend of the algorithm's runtime as the data volume increases**.
Let's understand this concept of "time growth trend" with an example. Assume the input data size is $n$, and consider three algorithms `A`, `B`, and `C`:
The concept of "time growth trend" is rather abstract; let us understand it through an example. Suppose the input data size is $n$, and given three algorithms `A`, `B`, and `C`:
=== "Python"
@@ -438,7 +462,41 @@ Let's understand this concept of "time growth trend" with an example. Assume the
=== "Kotlin"
```kotlin title=""
// Time complexity of algorithm A: constant order
fun algoritm_A(n: Int) {
println(0)
}
// Time complexity of algorithm B: linear order
fun algorithm_B(n: Int) {
for (i in 0..<n){
println(0)
}
}
// Time complexity of algorithm C: constant order
fun algorithm_C(n: Int) {
for (i in 0..<1000000) {
println(0)
}
}
```
=== "Ruby"
```ruby title=""
# Time complexity of algorithm A: constant order
def algorithm_A(n)
puts 0
end
# Time complexity of algorithm B: linear order
def algorithm_B(n)
(0...n).each { puts 0 }
end
# Time complexity of algorithm C: constant order
def algorithm_C(n)
(0...1_000_000).each { puts 0 }
end
```
=== "Zig"
@@ -464,23 +522,23 @@ Let's understand this concept of "time growth trend" with an example. Assume the
}
```
The figure below shows the time complexities of these three algorithms.
The figure below shows the time complexity of the above three algorithm functions.
- Algorithm `A` has just one print operation, and its run time does not grow with $n$. Its time complexity is considered "constant order."
- Algorithm `B` involves a print operation looping $n$ times, and its run time grows linearly with $n$. Its time complexity is "linear order."
- Algorithm `C` has a print operation looping 1,000,000 times. Although it takes a long time, it is independent of the input data size $n$. Therefore, the time complexity of `C` is the same as `A`, which is "constant order."
- Algorithm `A` has only $1$ print operation, and the algorithm's runtime does not grow as $n$ increases. We call the time complexity of this algorithm "constant order".
- In algorithm `B`, the print operation needs to loop $n$ times, and the algorithm's runtime grows linearly as $n$ increases. The time complexity of this algorithm is called "linear order".
- In algorithm `C`, the print operation needs to loop $1000000$ times. Although the runtime is very long, it is independent of the input data size $n$. Therefore, the time complexity of `C` is the same as `A`, still "constant order".
![Time growth trend of algorithms a, b, and c](time_complexity.assets/time_complexity_simple_example.png)
![Time growth trends of algorithms A, B, and C](time_complexity.assets/time_complexity_simple_example.png)
Compared to directly counting the run time of an algorithm, what are the characteristics of time complexity analysis?
Compared to directly counting the algorithm's runtime, what are the characteristics of time complexity analysis?
- **Time complexity effectively assesses algorithm efficiency**. For instance, algorithm `B` has linearly growing run time, which is slower than algorithm `A` when $n > 1$ and slower than `C` when $n > 1,000,000$. In fact, as long as the input data size $n$ is sufficiently large, a "constant order" complexity algorithm will always be better than a "linear order" one, demonstrating the essence of time growth trend.
- **Time complexity analysis is more straightforward**. Obviously, the running platform and the types of computational operations are irrelevant to the trend of run time growth. Therefore, in time complexity analysis, we can simply treat the execution time of all computational operations as the same "unit time," simplifying the "computational operation run time count" to a "computational operation count." This significantly reduces the complexity of estimation.
- **Time complexity has its limitations**. For example, although algorithms `A` and `C` have the same time complexity, their actual run times can be quite different. Similarly, even though algorithm `B` has a higher time complexity than `C`, it is clearly superior when the input data size $n$ is small. In these cases, it's difficult to judge the efficiency of algorithms based solely on time complexity. Nonetheless, despite these issues, complexity analysis remains the most effective and commonly used method for evaluating algorithm efficiency.
- **Time complexity can effectively evaluate algorithm efficiency**. For example, the runtime of algorithm `B` grows linearly; when $n > 1$ it is slower than algorithm `A`, and when $n > 1000000$ it is slower than algorithm `C`. In fact, as long as the input data size $n$ is sufficiently large, an algorithm with "constant order" complexity will always be superior to one with "linear order" complexity, which is precisely the meaning of time growth trend.
- **The derivation method for time complexity is simpler**. Obviously, the running platform and the types of computational operations are both unrelated to the growth trend of the algorithm's runtime. Therefore, in time complexity analysis, we can simply treat the execution time of all computational operations as the same "unit time", thus simplifying "counting computational operation runtime" to "counting the number of computational operations", which greatly reduces the difficulty of estimation.
- **Time complexity also has certain limitations**. For example, although algorithms `A` and `C` have the same time complexity, their actual runtimes differ significantly. Similarly, although algorithm `B` has a higher time complexity than `C`, when the input data size $n$ is small, algorithm `B` is clearly superior to algorithm `C`. In such cases, it is often difficult to judge the efficiency of algorithms based solely on time complexity. Of course, despite the above issues, complexity analysis remains the most effective and commonly used method for evaluating algorithm efficiency.
## Asymptotic upper bound
## Asymptotic upper bound of functions
Consider a function with an input size of $n$:
Given a function with input size $n$:
=== "Python"
@@ -489,7 +547,7 @@ Consider a function with an input size of $n$:
a = 1 # +1
a = a + 1 # +1
a = a * 2 # +1
# Cycle n times
# Loop n times
for i in range(n): # +1
print(0) # +1
```
@@ -502,7 +560,7 @@ Consider a function with an input size of $n$:
a = a + 1; // +1
a = a * 2; // +1
// Loop n times
for (int i = 0; i < n; i++) { // +1 (execute i ++ every round)
for (int i = 0; i < n; i++) { // +1 (i++ is executed each round)
cout << 0 << endl; // +1
}
}
@@ -516,7 +574,7 @@ Consider a function with an input size of $n$:
a = a + 1; // +1
a = a * 2; // +1
// Loop n times
for (int i = 0; i < n; i++) { // +1 (execute i ++ every round)
for (int i = 0; i < n; i++) { // +1 (i++ is executed each round)
System.out.println(0); // +1
}
}
@@ -530,7 +588,7 @@ Consider a function with an input size of $n$:
a = a + 1; // +1
a = a * 2; // +1
// Loop n times
for (int i = 0; i < n; i++) { // +1 (execute i ++ every round)
for (int i = 0; i < n; i++) { // +1 (i++ is executed each round)
Console.WriteLine(0); // +1
}
}
@@ -572,7 +630,7 @@ Consider a function with an input size of $n$:
a += 1; // +1
a *= 2; // +1
// Loop n times
for(let i = 0; i < n; i++){ // +1 (execute i ++ every round)
for(let i = 0; i < n; i++){ // +1 (i++ is executed each round)
console.log(0); // +1
}
}
@@ -586,7 +644,7 @@ Consider a function with an input size of $n$:
a += 1; // +1
a *= 2; // +1
// Loop n times
for(let i = 0; i < n; i++){ // +1 (execute i ++ every round)
for(let i = 0; i < n; i++){ // +1 (i++ is executed each round)
console.log(0); // +1
}
}
@@ -600,7 +658,7 @@ Consider a function with an input size of $n$:
a = a + 1; // +1
a = a * 2; // +1
// Loop n times
for (int i = 0; i < n; i++) { // +1 (execute i ++ every round)
for (int i = 0; i < n; i++) { // +1 (i++ is executed each round)
print(0); // +1
}
}
@@ -615,7 +673,7 @@ Consider a function with an input size of $n$:
a = a * 2; // +1
// Loop n times
for _ in 0..n { // +1 (execute i ++ every round)
for _ in 0..n { // +1 (i++ is executed each round)
println!("{}", 0); // +1
}
}
@@ -629,16 +687,38 @@ Consider a function with an input size of $n$:
a = a + 1; // +1
a = a * 2; // +1
// Loop n times
for (int i = 0; i < n; i++) { // +1 (execute i ++ every round)
for (int i = 0; i < n; i++) { // +1 (i++ is executed each round)
printf("%d", 0); // +1
}
}
}
```
=== "Kotlin"
```kotlin title=""
fun algorithm(n: Int) {
var a = 1 // +1
a = a + 1 // +1
a = a * 2 // +1
// Loop n times
for (i in 0..<n) { // +1 (i++ is executed each round)
println(0) // +1
}
}
```
=== "Ruby"
```ruby title=""
def algorithm(n)
a = 1 # +1
a = a + 1 # +1
a = a * 2 # +1
# Loop n times
(0...n).each do # +1
puts 0 # +1
end
end
```
=== "Zig"
@@ -649,56 +729,58 @@ Consider a function with an input size of $n$:
a += 1; // +1
a *= 2; // +1
// Loop n times
for (0..n) |_| { // +1 (execute i ++ every round)
for (0..n) |_| { // +1 (i++ is executed each round)
std.debug.print("{}\n", .{0}); // +1
}
}
```
Given a function that represents the number of operations of an algorithm as a function of the input size $n$, denoted as $T(n)$, consider the following example:
Let the number of operations of the algorithm be a function of the input data size $n$, denoted as $T(n)$. Then the number of operations of the above function is:
$$
T(n) = 3 + 2n
$$
Since $T(n)$ is a linear function, its growth trend is linear, and therefore, its time complexity is of linear order, denoted as $O(n)$. This mathematical notation, known as <u>big-O notation</u>, represents the <u>asymptotic upper bound</u> of the function $T(n)$.
$T(n)$ is a linear function, indicating that its runtime growth trend is linear, and therefore its time complexity is linear order.
In essence, time complexity analysis is about finding the asymptotic upper bound of the "number of operations $T(n)$". It has a precise mathematical definition.
We denote the time complexity of linear order as $O(n)$. This mathematical symbol is called <u>big-$O$ notation</u>, representing the <u>asymptotic upper bound</u> of the function $T(n)$.
!!! note "Asymptotic Upper Bound"
Time complexity analysis essentially calculates the asymptotic upper bound of "the number of operations $T(n)$", which has a clear mathematical definition.
If there exist positive real numbers $c$ and $n_0$ such that for all $n > n_0$, $T(n) \leq c \cdot f(n)$, then $f(n)$ is considered an asymptotic upper bound of $T(n)$, denoted as $T(n) = O(f(n))$.
!!! note "Asymptotic upper bound of functions"
As shown in the figure below, calculating the asymptotic upper bound involves finding a function $f(n)$ such that, as $n$ approaches infinity, $T(n)$ and $f(n)$ have the same growth order, differing only by a constant factor $c$.
If there exist positive real numbers $c$ and $n_0$ such that for all $n > n_0$, we have $T(n) \leq c \cdot f(n)$, then $f(n)$ can be considered as an asymptotic upper bound of $T(n)$, denoted as $T(n) = O(f(n))$.
As shown in the figure below, calculating the asymptotic upper bound is to find a function $f(n)$ such that when $n$ tends to infinity, $T(n)$ and $f(n)$ are at the same growth level, differing only by a constant coefficient $c$.
![Asymptotic upper bound of a function](time_complexity.assets/asymptotic_upper_bound.png)
## Calculation method
## Derivation method
While the concept of asymptotic upper bound might seem mathematically dense, you don't need to fully grasp it right away. Let's first understand the method of calculation, which can be practiced and comprehended over time.
The asymptotic upper bound has a bit of mathematical flavor. If you feel you haven't fully understood it, don't worry. We can first master the derivation method, and gradually grasp its mathematical meaning through continuous practice.
Once $f(n)$ is determined, we obtain the time complexity $O(f(n))$. But how do we determine the asymptotic upper bound $f(n)$? This process generally involves two steps: counting the number of operations and determining the asymptotic upper bound.
According to the definition, after determining $f(n)$, we can obtain the time complexity $O(f(n))$. So how do we determine the asymptotic upper bound $f(n)$? Overall, it is divided into two steps: first count the number of operations, then determine the asymptotic upper bound.
### Step 1: counting the number of operations
### Step 1: Count the number of operations
This step involves going through the code line by line. However, due to the presence of the constant $c$ in $c \cdot f(n)$, **all coefficients and constant terms in $T(n)$ can be ignored**. This principle allows for simplification techniques in counting operations.
For code, count from top to bottom line by line. However, since the constant coefficient $c$ in $c \cdot f(n)$ above can be of any size, **coefficients and constant terms in the number of operations $T(n)$ can all be ignored**. According to this principle, the following counting simplification techniques can be summarized.
1. **Ignore constant terms in $T(n)$**, as they do not affect the time complexity being independent of $n$.
2. **Omit all coefficients**. For example, looping $2n$, $5n + 1$ times, etc., can be simplified to $n$ times since the coefficient before $n$ does not impact the time complexity.
3. **Use multiplication for nested loops**. The total number of operations equals the product of the number of operations in each loop, applying the simplification techniques from points 1 and 2 for each loop level.
1. **Ignore constants in $T(n)$**. Because they are all independent of $n$, they do not affect time complexity.
2. **Omit all coefficients**. For example, looping $2n$ times, $5n + 1$ times, etc., can all be simplified as $n$ times, because the coefficient before $n$ does not affect time complexity.
3. **Use multiplication for nested loops**. The total number of operations equals the product of the number of operations in the outer and inner loops, with each layer of loop still able to apply techniques `1.` and `2.` separately.
Given a function, we can use these techniques to count operations:
Given a function, we can use the above techniques to count the number of operations:
=== "Python"
```python title=""
def algorithm(n: int):
a = 1 # +0 (trick 1)
a = a + n # +0 (trick 1)
# +n (technique 2)
a = 1 # +0 (Technique 1)
a = a + n # +0 (Technique 1)
# +n (Technique 2)
for i in range(5 * n + 1):
print(0)
# +n*n (technique 3)
# +n*n (Technique 3)
for i in range(2 * n):
for j in range(n + 1):
print(0)
@@ -708,13 +790,13 @@ Given a function, we can use these techniques to count operations:
```cpp title=""
void algorithm(int n) {
int a = 1; // +0 (trick 1)
a = a + n; // +0 (trick 1)
// +n (technique 2)
int a = 1; // +0 (Technique 1)
a = a + n; // +0 (Technique 1)
// +n (Technique 2)
for (int i = 0; i < 5 * n + 1; i++) {
cout << 0 << endl;
}
// +n*n (technique 3)
// +n*n (Technique 3)
for (int i = 0; i < 2 * n; i++) {
for (int j = 0; j < n + 1; j++) {
cout << 0 << endl;
@@ -727,13 +809,13 @@ Given a function, we can use these techniques to count operations:
```java title=""
void algorithm(int n) {
int a = 1; // +0 (trick 1)
a = a + n; // +0 (trick 1)
// +n (technique 2)
int a = 1; // +0 (Technique 1)
a = a + n; // +0 (Technique 1)
// +n (Technique 2)
for (int i = 0; i < 5 * n + 1; i++) {
System.out.println(0);
}
// +n*n (technique 3)
// +n*n (Technique 3)
for (int i = 0; i < 2 * n; i++) {
for (int j = 0; j < n + 1; j++) {
System.out.println(0);
@@ -746,13 +828,13 @@ Given a function, we can use these techniques to count operations:
```csharp title=""
void Algorithm(int n) {
int a = 1; // +0 (trick 1)
a = a + n; // +0 (trick 1)
// +n (technique 2)
int a = 1; // +0 (Technique 1)
a = a + n; // +0 (Technique 1)
// +n (Technique 2)
for (int i = 0; i < 5 * n + 1; i++) {
Console.WriteLine(0);
}
// +n*n (technique 3)
// +n*n (Technique 3)
for (int i = 0; i < 2 * n; i++) {
for (int j = 0; j < n + 1; j++) {
Console.WriteLine(0);
@@ -765,13 +847,13 @@ Given a function, we can use these techniques to count operations:
```go title=""
func algorithm(n int) {
a := 1 // +0 (trick 1)
a = a + n // +0 (trick 1)
// +n (technique 2)
a := 1 // +0 (Technique 1)
a = a + n // +0 (Technique 1)
// +n (Technique 2)
for i := 0; i < 5 * n + 1; i++ {
fmt.Println(0)
}
// +n*n (technique 3)
// +n*n (Technique 3)
for i := 0; i < 2 * n; i++ {
for j := 0; j < n + 1; j++ {
fmt.Println(0)
@@ -784,13 +866,13 @@ Given a function, we can use these techniques to count operations:
```swift title=""
func algorithm(n: Int) {
var a = 1 // +0 (trick 1)
a = a + n // +0 (trick 1)
// +n (technique 2)
var a = 1 // +0 (Technique 1)
a = a + n // +0 (Technique 1)
// +n (Technique 2)
for _ in 0 ..< (5 * n + 1) {
print(0)
}
// +n*n (technique 3)
// +n*n (Technique 3)
for _ in 0 ..< (2 * n) {
for _ in 0 ..< (n + 1) {
print(0)
@@ -803,13 +885,13 @@ Given a function, we can use these techniques to count operations:
```javascript title=""
function algorithm(n) {
let a = 1; // +0 (trick 1)
a = a + n; // +0 (trick 1)
// +n (technique 2)
let a = 1; // +0 (Technique 1)
a = a + n; // +0 (Technique 1)
// +n (Technique 2)
for (let i = 0; i < 5 * n + 1; i++) {
console.log(0);
}
// +n*n (technique 3)
// +n*n (Technique 3)
for (let i = 0; i < 2 * n; i++) {
for (let j = 0; j < n + 1; j++) {
console.log(0);
@@ -822,13 +904,13 @@ Given a function, we can use these techniques to count operations:
```typescript title=""
function algorithm(n: number): void {
let a = 1; // +0 (trick 1)
a = a + n; // +0 (trick 1)
// +n (technique 2)
let a = 1; // +0 (Technique 1)
a = a + n; // +0 (Technique 1)
// +n (Technique 2)
for (let i = 0; i < 5 * n + 1; i++) {
console.log(0);
}
// +n*n (technique 3)
// +n*n (Technique 3)
for (let i = 0; i < 2 * n; i++) {
for (let j = 0; j < n + 1; j++) {
console.log(0);
@@ -841,13 +923,13 @@ Given a function, we can use these techniques to count operations:
```dart title=""
void algorithm(int n) {
int a = 1; // +0 (trick 1)
a = a + n; // +0 (trick 1)
// +n (technique 2)
int a = 1; // +0 (Technique 1)
a = a + n; // +0 (Technique 1)
// +n (Technique 2)
for (int i = 0; i < 5 * n + 1; i++) {
print(0);
}
// +n*n (technique 3)
// +n*n (Technique 3)
for (int i = 0; i < 2 * n; i++) {
for (int j = 0; j < n + 1; j++) {
print(0);
@@ -860,15 +942,15 @@ Given a function, we can use these techniques to count operations:
```rust title=""
fn algorithm(n: i32) {
let mut a = 1; // +0 (trick 1)
a = a + n; // +0 (trick 1)
let mut a = 1; // +0 (Technique 1)
a = a + n; // +0 (Technique 1)
// +n (technique 2)
// +n (Technique 2)
for i in 0..(5 * n + 1) {
println!("{}", 0);
}
// +n*n (technique 3)
// +n*n (Technique 3)
for i in 0..(2 * n) {
for j in 0..(n + 1) {
println!("{}", 0);
@@ -881,13 +963,13 @@ Given a function, we can use these techniques to count operations:
```c title=""
void algorithm(int n) {
int a = 1; // +0 (trick 1)
a = a + n; // +0 (trick 1)
// +n (technique 2)
int a = 1; // +0 (Technique 1)
a = a + n; // +0 (Technique 1)
// +n (Technique 2)
for (int i = 0; i < 5 * n + 1; i++) {
printf("%d", 0);
}
// +n*n (technique 3)
// +n*n (Technique 3)
for (int i = 0; i < 2 * n; i++) {
for (int j = 0; j < n + 1; j++) {
printf("%d", 0);
@@ -899,22 +981,50 @@ Given a function, we can use these techniques to count operations:
=== "Kotlin"
```kotlin title=""
fun algorithm(n: Int) {
var a = 1 // +0 (Technique 1)
a = a + n // +0 (Technique 1)
// +n (Technique 2)
for (i in 0..<5 * n + 1) {
println(0)
}
// +n*n (Technique 3)
for (i in 0..<2 * n) {
for (j in 0..<n + 1) {
println(0)
}
}
}
```
=== "Ruby"
```ruby title=""
def algorithm(n)
a = 1 # +0 (Technique 1)
a = a + n # +0 (Technique 1)
# +n (Technique 2)
(0...(5 * n + 1)).each do { puts 0 }
# +n*n (Technique 3)
(0...(2 * n)).each do
(0...(n + 1)).each do { puts 0 }
end
end
```
=== "Zig"
```zig title=""
fn algorithm(n: usize) void {
var a: i32 = 1; // +0 (trick 1)
a = a + @as(i32, @intCast(n)); // +0 (trick 1)
var a: i32 = 1; // +0 (Technique 1)
a = a + @as(i32, @intCast(n)); // +0 (Technique 1)
// +n (technique 2)
// +n (Technique 2)
for(0..(5 * n + 1)) |_| {
std.debug.print("{}\n", .{0});
}
// +n*n (technique 3)
// +n*n (Technique 3)
for(0..(2 * n)) |_| {
for(0..(n + 1)) |_| {
std.debug.print("{}\n", .{0});
@@ -923,48 +1033,50 @@ Given a function, we can use these techniques to count operations:
}
```
The formula below shows the counting results before and after simplification, both leading to a time complexity of $O(n^2)$:
The following formula shows the counting results before and after using the above techniques; both derive a time complexity of $O(n^2)$.
$$
\begin{aligned}
T(n) & = 2n(n + 1) + (5n + 1) + 2 & \text{Complete Count (-.-|||)} \newline
T(n) & = 2n(n + 1) + (5n + 1) + 2 & \text{Complete count (-.-|||)} \newline
& = 2n^2 + 7n + 3 \newline
T(n) & = n^2 + n & \text{Simplified Count (o.O)}
T(n) & = n^2 + n & \text{Simplified count (o.O)}
\end{aligned}
$$
### Step 2: determining the asymptotic upper bound
### Step 2: Determine the asymptotic upper bound
**The time complexity is determined by the highest order term in $T(n)$**. This is because, as $n$ approaches infinity, the highest order term dominates, rendering the influence of other terms negligible.
**Time complexity is determined by the highest-order term in $T(n)$**. This is because as $n$ tends to infinity, the highest-order term will play a dominant role, and the influence of other terms can be ignored.
The following table illustrates examples of different operation counts and their corresponding time complexities. Some exaggerated values are used to emphasize that coefficients cannot alter the order of growth. When $n$ becomes very large, these constants become insignificant.
The table below shows some examples, where some exaggerated values are used to emphasize the conclusion that "coefficients cannot shake the order". When $n$ tends to infinity, these constants become insignificant.
<p align="center"> Table: Time complexity for different operation counts </p>
<p align="center"> Table <id> &nbsp; Time complexities corresponding to different numbers of operations </p>
| Operation Count $T(n)$ | Time Complexity $O(f(n))$ |
| ---------------------- | ------------------------- |
| $100000$ | $O(1)$ |
| $3n + 2$ | $O(n)$ |
| $2n^2 + 3n + 2$ | $O(n^2)$ |
| $n^3 + 10000n^2$ | $O(n^3)$ |
| $2^n + 10000n^{10000}$ | $O(2^n)$ |
| Number of Operations $T(n)$ | Time Complexity $O(f(n))$ |
| ---------------------- | -------------------- |
| $100000$ | $O(1)$ |
| $3n + 2$ | $O(n)$ |
| $2n^2 + 3n + 2$ | $O(n^2)$ |
| $n^3 + 10000n^2$ | $O(n^3)$ |
| $2^n + 10000n^{10000}$ | $O(2^n)$ |
## Common types of time complexity
## Common types
Let's consider the input data size as $n$. The common types of time complexities are shown in the figure below, arranged from lowest to highest:
Let the input data size be $n$. Common time complexity types are shown in the figure below (arranged in order from low to high).
$$
\begin{aligned}
& O(1) < O(\log n) < O(n) < O(n \log n) < O(n^2) < O(2^n) < O(n!) \newline
& \text{Constant} < \text{Log} < \text{Linear} < \text{Linear-Log} < \text{Quadratic} < \text{Exp} < \text{Factorial}
O(1) < O(\log n) < O(n) < O(n \log n) < O(n^2) < O(2^n) < O(n!) \newline
\text{Constant order} < \text{Logarithmic order} < \text{Linear order} < \text{Linearithmic order} < \text{Quadratic order} < \text{Exponential order} < \text{Factorial order}
\end{aligned}
$$
![Common types of time complexity](time_complexity.assets/time_complexity_common_types.png)
![Common time complexity types](time_complexity.assets/time_complexity_common_types.png)
### Constant order $O(1)$
Constant order means the number of operations is independent of the input data size $n$. In the following function, although the number of operations `size` might be large, the time complexity remains $O(1)$ as it's unrelated to $n$:
The number of operations in constant order is independent of the input data size $n$, meaning it does not change as $n$ changes.
In the following function, although the number of operations `size` may be large, since it is independent of the input data size $n$, the time complexity remains $O(1)$:
```src
[file]{time_complexity}-[class]{}-[func]{constant}
@@ -972,23 +1084,23 @@ Constant order means the number of operations is independent of the input data s
### Linear order $O(n)$
Linear order indicates the number of operations grows linearly with the input data size $n$. Linear order commonly appears in single-loop structures:
The number of operations in linear order grows linearly relative to the input data size $n$. Linear order typically appears in single-layer loops:
```src
[file]{time_complexity}-[class]{}-[func]{linear}
```
Operations like array traversal and linked list traversal have a time complexity of $O(n)$, where $n$ is the length of the array or list:
Operations such as traversing arrays and traversing linked lists have a time complexity of $O(n)$, where $n$ is the length of the array or linked list:
```src
[file]{time_complexity}-[class]{}-[func]{array_traversal}
```
It's important to note that **the input data size $n$ should be determined based on the type of input data**. For example, in the first example, $n$ represents the input data size, while in the second example, the length of the array $n$ is the data size.
It is worth noting that **the input data size $n$ should be determined according to the type of input data**. For example, in the first example, the variable $n$ is the input data size; in the second example, the array length $n$ is the data size.
### Quadratic order $O(n^2)$
Quadratic order means the number of operations grows quadratically with the input data size $n$. Quadratic order typically appears in nested loops, where both the outer and inner loops have a time complexity of $O(n)$, resulting in an overall complexity of $O(n^2)$:
The number of operations in quadratic order grows quadratically relative to the input data size $n$. Quadratic order typically appears in nested loops, where both the outer and inner loops have a time complexity of $O(n)$, resulting in an overall time complexity of $O(n^2)$:
```src
[file]{time_complexity}-[class]{}-[func]{quadratic}
@@ -996,9 +1108,9 @@ Quadratic order means the number of operations grows quadratically with the inpu
The figure below compares constant order, linear order, and quadratic order time complexities.
![Constant, linear, and quadratic order time complexities](time_complexity.assets/time_complexity_constant_linear_quadratic.png)
![Time complexities of constant, linear, and quadratic orders](time_complexity.assets/time_complexity_constant_linear_quadratic.png)
For instance, in bubble sort, the outer loop runs $n - 1$ times, and the inner loop runs $n-1$, $n-2$, ..., $2$, $1$ times, averaging $n / 2$ times, resulting in a time complexity of $O((n - 1) n / 2) = O(n^2)$:
Taking bubble sort as an example, the outer loop executes $n - 1$ times, and the inner loop executes $n-1$, $n-2$, $\dots$, $2$, $1$ times, averaging $n / 2$ times, resulting in a time complexity of $O((n - 1) n / 2) = O(n^2)$:
```src
[file]{time_complexity}-[class]{}-[func]{bubble_sort}
@@ -1006,107 +1118,107 @@ For instance, in bubble sort, the outer loop runs $n - 1$ times, and the inner l
### Exponential order $O(2^n)$
Biological "cell division" is a classic example of exponential order growth: starting with one cell, it becomes two after one division, four after two divisions, and so on, resulting in $2^n$ cells after $n$ divisions.
Biological "cell division" is a typical example of exponential order growth: the initial state is $1$ cell, after one round of division it becomes $2$, after two rounds it becomes $4$, and so on; after $n$ rounds of division there are $2^n$ cells.
The figure below and code simulate the cell division process, with a time complexity of $O(2^n)$:
The figure below and the following code simulate the cell division process, with a time complexity of $O(2^n)$. Note that the input $n$ represents the number of division rounds, and the return value `count` represents the total number of divisions.
```src
[file]{time_complexity}-[class]{}-[func]{exponential}
```
![Exponential order time complexity](time_complexity.assets/time_complexity_exponential.png)
![Time complexity of exponential order](time_complexity.assets/time_complexity_exponential.png)
In practice, exponential order often appears in recursive functions. For example, in the code below, it recursively splits into two halves, stopping after $n$ divisions:
In actual algorithms, exponential order often appears in recursive functions. For example, in the following code, it recursively splits in two, stopping after $n$ splits:
```src
[file]{time_complexity}-[class]{}-[func]{exp_recur}
```
Exponential order growth is extremely rapid and is commonly seen in exhaustive search methods (brute force, backtracking, etc.). For large-scale problems, exponential order is unacceptable, often requiring dynamic programming or greedy algorithms as solutions.
Exponential order growth is very rapid and is common in exhaustive methods (brute force search, backtracking, etc.). For problems with large data scales, exponential order is unacceptable and typically requires dynamic programming or greedy algorithms to solve.
### Logarithmic order $O(\log n)$
In contrast to exponential order, logarithmic order reflects situations where "the size is halved each round." Given an input data size $n$, since the size is halved each round, the number of iterations is $\log_2 n$, the inverse function of $2^n$.
In contrast to exponential order, logarithmic order reflects the situation of "reducing to half each round". Let the input data size be $n$. Since it is reduced to half each round, the number of loops is $\log_2 n$, which is the inverse function of $2^n$.
The figure below and code simulate the "halving each round" process, with a time complexity of $O(\log_2 n)$, commonly abbreviated as $O(\log n)$:
The figure below and the following code simulate the process of "reducing to half each round", with a time complexity of $O(\log_2 n)$, abbreviated as $O(\log n)$:
```src
[file]{time_complexity}-[class]{}-[func]{logarithmic}
```
![Logarithmic order time complexity](time_complexity.assets/time_complexity_logarithmic.png)
![Time complexity of logarithmic order](time_complexity.assets/time_complexity_logarithmic.png)
Like exponential order, logarithmic order also frequently appears in recursive functions. The code below forms a recursive tree of height $\log_2 n$:
Like exponential order, logarithmic order also commonly appears in recursive functions. The following code forms a recursion tree of height $\log_2 n$:
```src
[file]{time_complexity}-[class]{}-[func]{log_recur}
```
Logarithmic order is typical in algorithms based on the divide-and-conquer strategy, embodying the "split into many" and "simplify complex problems" approach. It's slow-growing and is the most ideal time complexity after constant order.
Logarithmic order commonly appears in algorithms based on the divide-and-conquer strategy, embodying the algorithmic thinking of "dividing into many" and "simplifying complexity". It grows slowly and is the ideal time complexity second only to constant order.
!!! tip "What is the base of $O(\log n)$?"
Technically, "splitting into $m$" corresponds to a time complexity of $O(\log_m n)$. Using the logarithm base change formula, we can equate different logarithmic complexities:
To be precise, "dividing into $m$" corresponds to a time complexity of $O(\log_m n)$. And through the logarithmic base change formula, we can obtain time complexities with different bases that are equal:
$$
O(\log_m n) = O(\log_k n / \log_k m) = O(\log_k n)
$$
This means the base $m$ can be changed without affecting the complexity. Therefore, we often omit the base $m$ and simply denote logarithmic order as $O(\log n)$.
That is to say, the base $m$ can be converted without affecting the complexity. Therefore, we usually omit the base $m$ and denote logarithmic order simply as $O(\log n)$.
### Linear-logarithmic order $O(n \log n)$
### Linearithmic order $O(n \log n)$
Linear-logarithmic order often appears in nested loops, with the complexities of the two loops being $O(\log n)$ and $O(n)$ respectively. The related code is as follows:
Linearithmic order commonly appears in nested loops, where the time complexities of the two layers of loops are $O(\log n)$ and $O(n)$ respectively. The relevant code is as follows:
```src
[file]{time_complexity}-[class]{}-[func]{linear_log_recur}
```
The figure below demonstrates how linear-logarithmic order is generated. Each level of a binary tree has $n$ operations, and the tree has $\log_2 n + 1$ levels, resulting in a time complexity of $O(n \log n)$.
The figure below shows how linearithmic order is generated. Each level of the binary tree has a total of $n$ operations, and the tree has $\log_2 n + 1$ levels, resulting in a time complexity of $O(n \log n)$.
![Linear-logarithmic order time complexity](time_complexity.assets/time_complexity_logarithmic_linear.png)
![Time complexity of linearithmic order](time_complexity.assets/time_complexity_logarithmic_linear.png)
Mainstream sorting algorithms typically have a time complexity of $O(n \log n)$, such as quicksort, mergesort, and heapsort.
Mainstream sorting algorithms typically have a time complexity of $O(n \log n)$, such as quicksort, merge sort, and heap sort.
### Factorial order $O(n!)$
Factorial order corresponds to the mathematical problem of "full permutation." Given $n$ distinct elements, the total number of possible permutations is:
Factorial order corresponds to the mathematical "permutation" problem. Given $n$ distinct elements, find all possible permutation schemes; the number of schemes is:
$$
n! = n \times (n - 1) \times (n - 2) \times \dots \times 2 \times 1
$$
Factorials are typically implemented using recursion. As shown in the code and the figure below, the first level splits into $n$ branches, the second level into $n - 1$ branches, and so on, stopping after the $n$th level:
Factorials are typically implemented using recursion. As shown in the figure below and the following code, the first level splits into $n$ branches, the second level splits into $n - 1$ branches, and so on, until the $n$-th level when splitting stops:
```src
[file]{time_complexity}-[class]{}-[func]{factorial_recur}
```
![Factorial order time complexity](time_complexity.assets/time_complexity_factorial.png)
![Time complexity of factorial order](time_complexity.assets/time_complexity_factorial.png)
Note that factorial order grows even faster than exponential order; it's unacceptable for larger $n$ values.
Note that because when $n \geq 4$ we always have $n! > 2^n$, factorial order grows faster than exponential order, and is also unacceptable for large $n$.
## Worst, best, and average time complexities
**The time efficiency of an algorithm is often not fixed but depends on the distribution of the input data**. Assume we have an array `nums` of length $n$, consisting of numbers from $1$ to $n$, each appearing only once, but in a randomly shuffled order. The task is to return the index of the element $1$. We can draw the following conclusions:
**The time efficiency of an algorithm is often not fixed, but is related to the distribution of the input data**. Suppose we input an array `nums` of length $n$, where `nums` consists of numbers from $1$ to $n$, with each number appearing only once, but the element order is randomly shuffled. The task is to return the index of element $1$. We can draw the following conclusions.
- When `nums = [?, ?, ..., 1]`, that is, when the last element is $1$, it requires a complete traversal of the array, **achieving the worst-case time complexity of $O(n)$**.
- When `nums = [1, ?, ?, ...]`, that is, when the first element is $1$, no matter the length of the array, no further traversal is needed, **achieving the best-case time complexity of $\Omega(1)$**.
- When `nums = [?, ?, ..., 1]`, i.e., when the last element is $1$, it requires a complete traversal of the array, **reaching worst-case time complexity $O(n)$**.
- When `nums = [1, ?, ?, ...]`, i.e., when the first element is $1$, no matter how long the array is, there is no need to continue traversing, **reaching best-case time complexity $\Omega(1)$**.
The "worst-case time complexity" corresponds to the asymptotic upper bound, denoted by the big $O$ notation. Correspondingly, the "best-case time complexity" corresponds to the asymptotic lower bound, denoted by $\Omega$:
The "worst-case time complexity" corresponds to the function's asymptotic upper bound, denoted using big-$O$ notation. Correspondingly, the "best-case time complexity" corresponds to the function's asymptotic lower bound, denoted using $\Omega$ notation:
```src
[file]{worst_best_time_complexity}-[class]{}-[func]{find_one}
```
It's important to note that the best-case time complexity is rarely used in practice, as it is usually only achievable under very low probabilities and might be misleading. **The worst-case time complexity is more practical as it provides a safety value for efficiency**, allowing us to confidently use the algorithm.
It is worth noting that we rarely use best-case time complexity in practice, because it can usually only be achieved with a very small probability and may be somewhat misleading. **The worst-case time complexity is more practical because it gives a safety value for efficiency**, allowing us to use the algorithm with confidence.
From the above example, it's clear that both the worst-case and best-case time complexities only occur under "special data distributions," which may have a small probability of occurrence and may not accurately reflect the algorithm's run efficiency. In contrast, **the average time complexity can reflect the algorithm's efficiency under random input data**, denoted by the $\Theta$ notation.
From the above example, we can see that both worst-case and best-case time complexities only occur under "special data distributions", which may have a very small probability of occurrence and may not truly reflect the algorithm's running efficiency. In contrast, **average time complexity can reflect the algorithm's running efficiency under random input data**, denoted using the $\Theta$ notation.
For some algorithms, we can simply estimate the average case under a random data distribution. For example, in the aforementioned example, since the input array is shuffled, the probability of element $1$ appearing at any index is equal. Therefore, the average number of loops for the algorithm is half the length of the array $n / 2$, giving an average time complexity of $\Theta(n / 2) = \Theta(n)$.
For some algorithms, we can simply derive the average case under random data distribution. For example, in the above example, since the input array is shuffled, the probability of element $1$ appearing at any index is equal, so the algorithm's average number of loops is half the array length $n / 2$, giving an average time complexity of $\Theta(n / 2) = \Theta(n)$.
However, calculating the average time complexity for more complex algorithms can be quite difficult, as it's challenging to analyze the overall mathematical expectation under the data distribution. In such cases, we usually use the worst-case time complexity as the standard for judging the efficiency of the algorithm.
But for more complex algorithms, calculating average time complexity is often quite difficult, because it is hard to analyze the overall mathematical expectation under data distribution. In this case, we usually use worst-case time complexity as the criterion for judging algorithm efficiency.
!!! question "Why is the $\Theta$ symbol rarely seen?"
Possibly because the $O$ notation is more commonly spoken, it is often used to represent the average time complexity. However, strictly speaking, this practice is not accurate. In this book and other materials, if you encounter statements like "average time complexity $O(n)$", please understand it directly as $\Theta(n)$.
This may be because the $O$ symbol is too catchy, so we often use it to represent average time complexity. But strictly speaking, this practice is not standard. In this book and other materials, if you encounter expressions like "average time complexity $O(n)$", please understand it directly as $\Theta(n)$.