mirror of
https://github.com/krahets/hello-algo.git
synced 2026-07-13 15:56:05 +00:00
build
This commit is contained in:
@@ -6,7 +6,7 @@ comments: true
|
||||
|
||||
In computers, all data is stored in binary form, and character `char` is no exception. To represent characters, we need to establish a "character set" that defines a one-to-one correspondence between each character and binary numbers. With a character set, computers can convert binary numbers to characters by looking up the table.
|
||||
|
||||
## 3.4.1 Ascii Character Set
|
||||
## 3.4.1 ASCII Character Set
|
||||
|
||||
<u>ASCII code</u> is the earliest character set, with the full name American Standard Code for Information Interchange. It uses 7 binary bits (the lower 7 bits of one byte) to represent a character, and can represent a maximum of 128 different characters. As shown in Figure 3-6, ASCII code includes uppercase and lowercase English letters, numbers 0 ~ 9, some punctuation marks, and some control characters (such as newline and tab).
|
||||
|
||||
@@ -18,7 +18,7 @@ However, **ASCII code can only represent English**. With the globalization of co
|
||||
|
||||
Worldwide, a batch of EASCII character sets suitable for different regions have appeared successively. The first 128 characters of these character sets are unified as ASCII code, and the last 128 characters are defined differently to adapt to the needs of different languages.
|
||||
|
||||
## 3.4.2 Gbk Character Set
|
||||
## 3.4.2 GBK Character Set
|
||||
|
||||
Later, people found that **EASCII still could not provide enough characters for many languages**. For example, there are nearly one hundred thousand Chinese characters, and several thousand are used in everyday life. In 1980, the China National Standardization Administration released the <u>GB2312</u> character set, which included 6,763 Chinese characters, basically meeting the needs of computer processing for Chinese.
|
||||
|
||||
@@ -30,7 +30,7 @@ With the vigorous development of computer technology, character sets and encodin
|
||||
|
||||
Researchers of that era thought: **If a sufficiently complete character set were released to include all languages and symbols in the world, wouldn't that solve problems in cross-language environments and eliminate garbled text**? Driven by this idea, a large and comprehensive character set, Unicode, was born.
|
||||
|
||||
<u>Unicode</u> is called "统一码" (Unified Code) in Chinese and can theoretically accommodate over one million characters. It is committed to including characters from around the world into a unified character set, providing a universal character set to handle and display various language texts, reducing garbled character problems caused by different encoding standards.
|
||||
<u>Unicode</u>, or Unified Code, can theoretically accommodate over one million characters. It is committed to including characters from around the world into a unified character set, providing a universal character set to handle and display various language texts, reducing garbled character problems caused by different encoding standards.
|
||||
|
||||
Since its release in 1991, Unicode has continuously expanded to include new languages and characters. As of September 2022, Unicode has included 149,186 characters, including characters, symbols, and even emojis from various languages. In practical storage and encoding schemes for this vast character set, commonly used characters often occupy 2 bytes, while some rare characters occupy 3 bytes or even 4 bytes.
|
||||
|
||||
@@ -44,7 +44,7 @@ For the above problem, **a straightforward solution is to store all characters a
|
||||
|
||||
However, ASCII code has already proven to us that encoding English only requires 1 byte. If the above scheme is adopted, the size of English text will be twice that under ASCII encoding, which is very wasteful of memory space. Therefore, we need a more efficient Unicode encoding method.
|
||||
|
||||
## 3.4.4 Utf-8 Encoding
|
||||
## 3.4.4 UTF-8 Encoding
|
||||
|
||||
Currently, UTF-8 has become the most widely used Unicode encoding method internationally. **It is a variable-length encoding** that uses 1 to 4 bytes to represent a character, depending on the complexity of the character. ASCII characters only require 1 byte, Latin and Greek letters require 2 bytes, commonly used Chinese characters require 3 bytes, and some other rare characters require 4 bytes.
|
||||
|
||||
@@ -53,7 +53,7 @@ The encoding rules of UTF-8 are not complicated and can be divided into the foll
|
||||
- For 1-byte characters, set the highest bit to $0$, and set the remaining 7 bits to the Unicode code point. It is worth noting that ASCII characters occupy the first 128 code points in the Unicode character set. That is to say, **UTF-8 encoding is backward compatible with ASCII code**. This means we can use UTF-8 to parse very old ASCII code text.
|
||||
- For characters with a length of $n$ bytes (where $n > 1$), set the highest $n$ bits of the first byte to $1$, and set the $(n + 1)$-th bit to $0$; starting from the second byte, set the highest 2 bits of each byte to $10$; use all remaining bits to fill in the Unicode code point of the character.
|
||||
|
||||
Figure 3-8 shows the UTF-8 encoding corresponding to "Hello算法". It can be observed that since the highest $n$ bits are all set to $1$, the system can determine that the character length is $n$ by counting the leading $1$ bits.
|
||||
Figure 3-8 shows the UTF-8 encoding corresponding to "Hello 算法". It can be observed that since the highest $n$ bits are all set to $1$, the system can determine that the character length is $n$ by counting the leading $1$ bits.
|
||||
|
||||
But why set the highest 2 bits of all other bytes to $10$? In fact, this $10$ can serve as a check symbol. Assuming the system starts parsing text from an incorrect byte, the $10$ at the beginning of the byte can help the system quickly determine an anomaly.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user