LangBot/docs/en/plugin/tech-info.md

Technical Information

Below is the technical information about the implementation principles of LangBot. Please read carefully before contributing.

This document is the technical documentation for LangBot 2.x and is outdated. For version 3.x, please refer directly to the source code.
Please read the OpenAI API documentation at https://beta.openai.com/docs/ first. The following information assumes that you are familiar with the relevant features of OpenAI models and their interface invocation methods.

Terminology

Includes the terminology involved in the OpenAI API and the naming of concepts in the project.
The terms in parentheses are the corresponding terms in the program, while those without parentheses are abstract concepts.

Model

AI model. The content generated by calling OpenAI's interface in the program is all generated by OpenAI's models.

Tokens

Tokens as defined by OpenAI. ASCII characters are 1 token, while others are 2 tokens.

Prompt

i. The prompt used when calling OpenAI's text completion model. The model interface will return a response based on the prompt. The program will encapsulate the conversation content to generate the prompt. The prompts for calling the text completion model are marked with user_name (default is You, can be modified in the configuration file) and bot_name (default is Bot, can be modified in the configuration file) to identify the conversation roles for the model. Here is an example:

You: The weather is nice today.
Bot: I'm glad you like today's weather :)
You: Thank you.
Bot: You're welcome :)

For the program implementation of the completion model, please refer to the Implementation section below.

ii. The prompt used when calling OpenAI's drawing model. The model will draw based on the prompt and return the image URL.

Object

The program treats a single person or a single QQ group as an object. The object and the model are the two parties in a conversation during a session.

Session

Sessions are only valid for the text completion function; the drawing function does not have the concept of a session. Each object uses the same session, and there are only two roles in a session: the object and the model. Therefore, all people in the group will be treated as the same role to converse with the model.

The essence of the program obtaining a reply is text completion. Since the conversation needs to maintain context, the program will send the conversation history between the model and the object as a prompt to OpenAI's interface to obtain a reply that fits the previous context. However, the prompt for OpenAI's text completion interface has a length limit (the default text-davinci-003 has a limit of 4096 tokens), so the concept of session is added to manage the prompt content sent to the interface.

The session's lifespan can be set in config.py, with a default of 20 minutes. After the session expires, it will be stored in the database and reset. The next time the object initiates a conversation, a new session will be started.

Default Prompt, Personality (default_prompt)

The preset conversation information for each session can be set in config.py. The program will write the following content to the prompt when each session is created:

You: <preset information>
Bot: Okay

Implementation

QQ Bot

Program Path: pkg.qqbot

• pkg.qqbot.manager contains QQBotManager, which implements functions such as receiving messages, calling the OpenAI module to process messages, reporting to the audit module to record usage, and provides methods for notifying administrators and sending messages for other modules to call.
• pkg.qqbot.filter provides operations related to sensitive word filtering.
• pkg.qqbot.process provides unified processing logic for private messages and group messages.

Mirai and YiriMirai are used as the framework for Python to interact with QQ. For details, please refer to their documentation.
At startup, YiriMirai's function is called to create a bot object, which is used by the program to interact with QQ through Mirai. The upper-level program calls this bot object's methods to process messages.
Since YiriMirai currently cannot shut down the bot, the same bot object is maintained before and after hot reloading, meaning that the QQ bot's related configuration (QQ number, adapter, etc.) information does not support hot reloading.

Database

Program Path: pkg.database

• pkg.database.manager contains DatabaseManager, which encapsulates many methods for calling the database for other modules to use.

SQLite is used as the database, storing all historical session information of objects, api-key cost information, and api-key usage information.

OpenAI Interaction

Program Path: pkg.openai

• pkg.openai.manager contains the OpenAIInteract class, which encapsulates OpenAI's text completion Completion API and drawing API for the bot module to call. After a successful interface call, it reports the usage information of the current api-key to the audit module.
• pkg.openai.keymgr implements multi-api-key management, where the exceeded variable records api-key overage errors at runtime and provides api-key switching functionality based on overage records.
• pkg.openai.pricing records the cost information of each model for estimating costs when calling the interface. The cost estimation function is no longer tied to api-key switching; api-keys are only switched when the interface call reports an overage error.
• pkg.openai.session contains Session for session management.

Utils Module

Context Module

Saves the objects from the aforementioned modules and allows each module to retrieve objects from here to call their methods.

Hot Reload Function

pkg.utils.reloader

Before reloading, save all objects in the context, execute the program shutdown process in main.py, use the reload function of importlib to reload all modules (including the configuration file and any new modules), restore the context after reloading, and execute the program startup process.
All modules will recreate their objects, but the bot object in the QQ bot module will not be recreated. This is because the shutdown method provided by YiriMirai cannot be used, meaning that the QQ bot configuration in config.py does not support hot reloading.

Hot Update Function

pkg.utils.updater

Use the dulwich library to perform a pull operation to fetch the latest source code from the remote repository and perform a hot reload to load the latest code.