refactor(persistence): freeze legacy DB migration chain, drop dbm026

The legacy pkg/persistence/migrations (DBMigration / dbmXXX) system now
coexists with Alembic but accepts no new migrations — all new schema
changes go through Alembic.

- remove dbm026_llm_model_context_length (superseded by Alembic
  0005_add_llm_context_length, which makes the identical change)
- cap required_database_version at 25 (legacy chain dbm001-025 kept
  read-only to upgrade pre-existing 3.x DBs to the Alembic baseline)
- add migrations/README.md documenting the freeze
- document the Alembic-only policy and revision-id/idempotency rules in
  AGENTS.md
This commit is contained in:
RockChinQ
2026-06-13 05:26:08 -04:00
parent 672abfe95d
commit a2c6c8201b
4 changed files with 52 additions and 44 deletions
+8
View File
@@ -125,6 +125,14 @@ uv run python -m langbot.pkg.persistence.alembic_runner autogenerate "descriptio
Review and edit the generated script before committing. Migrations execute automatically on startup. `autogenerate` detects schema changes (add/drop columns, tables, type changes) but **data migrations** (e.g. mutating JSON field contents) must be hand-written into the generated script. `env.py` sets `render_as_batch=True`, so SQLite's ALTER TABLE limits are handled automatically — no need to branch per database. More in the wiki ["开发配置"](https://docs.langbot.app/zh/develop/dev-config#数据库迁移). Review and edit the generated script before committing. Migrations execute automatically on startup. `autogenerate` detects schema changes (add/drop columns, tables, type changes) but **data migrations** (e.g. mutating JSON field contents) must be hand-written into the generated script. `env.py` sets `render_as_batch=True`, so SQLite's ALTER TABLE limits are handled automatically — no need to branch per database. More in the wiki ["开发配置"](https://docs.langbot.app/zh/develop/dev-config#数据库迁移).
When writing a migration, follow these rules:
- **Revision id ≤ 32 characters.** PostgreSQL stores `alembic_version.version_num` as `varchar(32)`; a longer id raises `StringDataRightTruncationError` at runtime. Prefer short, descriptive ids like `0005_add_llm_context_length`.
- **Guard every operation against missing tables/columns.** Fresh installs build the schema via `create_all()` and then stamp the Alembic baseline, so a migration may run against a table that already has the change — or, in tests, against an empty database. Check `inspector.get_table_names()` / `inspector.get_columns(...)` before `add_column` / `drop_column`, mirroring the existing migrations.
- **Keep a single linear head.** Chain `down_revision` to the current head; do not create branches. Run the migration tests after adding one: `uv run pytest tests/integration/persistence/ -q` (the PostgreSQL test needs a running PG via `TEST_POSTGRES_URL`).
> **Legacy migration system (deprecated — do not extend).** The old 3.x migration system under `src/langbot/pkg/persistence/migrations/` (`DBMigration` subclasses in `dbmXXX_*.py`, run from `pkg/persistence/mgr.py`) is **frozen**. Do **not** add new `dbmXXX_*.py` files. The chain is capped at `required_database_version = 25` (`pkg/utils/constants.py`); those files only exist to upgrade pre-existing 3.x databases up to the Alembic baseline and are kept read-only. All new schema changes go through Alembic.
## Some Principles ## Some Principles
- Keep it simple, stupid. - Keep it simple, stupid.
@@ -0,0 +1,36 @@
# Legacy migrations (DEPRECATED — do not add new files here)
This directory holds the **legacy 3.x database migration system**
(`DBMigration` subclasses in `dbmXXX_*.py`, registered via
`@migration.migration_class(N)` and run from `pkg/persistence/mgr.py`).
**This system is frozen. Do not add new `dbmXXX_*.py` migrations.**
The chain is capped at version 25 (`required_database_version = 25` in
`pkg/utils/constants.py`). These files exist only to upgrade pre-existing
3.x databases up to the Alembic baseline (`0001_baseline`). Removing them
would break in-place upgrades from old installations, so they are kept
read-only.
## All new schema changes use Alembic
Migrations now live in `pkg/persistence/alembic/versions/`. To create one:
```bash
uv run python -m langbot.pkg.persistence.alembic_runner autogenerate "description of your change"
```
(requires `data/config.yaml` to exist). Review and edit the generated
script before committing — Alembic migrations run automatically on startup
and must be idempotent and guard against missing tables (the test suite
runs them against empty databases).
### Rules for Alembic revision ids
- Keep the revision id **≤ 32 characters** — PostgreSQL stores
`alembic_version.version_num` as `varchar(32)` and will raise
`StringDataRightTruncationError` on overflow.
- Guard every `op` call against a missing table / missing column
(`inspector.get_table_names()` / `inspector.get_columns()`); fresh
installs create the schema via `create_all()` and stamp the baseline,
so migrations may run against tables that already match or do not exist.
@@ -1,42 +0,0 @@
import sqlalchemy
from .. import migration
@migration.migration_class(26)
class DBMigrateLLMModelContextLength(migration.DBMigration):
"""Add context_length column to LLM models"""
async def upgrade(self):
columns = await self._get_columns('llm_models')
if 'context_length' not in columns:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('ALTER TABLE llm_models ADD COLUMN context_length INTEGER')
)
async def downgrade(self):
columns = await self._get_columns('llm_models')
if 'context_length' not in columns:
return
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('ALTER TABLE llm_models DROP COLUMN IF EXISTS context_length')
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('ALTER TABLE llm_models DROP COLUMN context_length')
)
async def _get_columns(self, table_name: str) -> set[str]:
if self.ap.persistence_mgr.db.name == 'postgresql':
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("""
SELECT column_name FROM information_schema.columns
WHERE table_name = :table_name
"""),
{'table_name': table_name},
)
return {row[0] for row in result.fetchall()}
result = await self.ap.persistence_mgr.execute_async(sqlalchemy.text(f'PRAGMA table_info({table_name})'))
return {row[1] for row in result.fetchall()}
+8 -2
View File
@@ -2,8 +2,14 @@ import langbot
semantic_version = f'v{langbot.__version__}' semantic_version = f'v{langbot.__version__}'
required_database_version = 26 required_database_version = 25
"""Tag the version of the database schema, used to check if the database needs to be migrated""" """Tag the version of the legacy (3.x) database schema migration chain.
Frozen at 25: the legacy ``pkg/persistence/migrations`` system (DBMigration /
dbmXXX_*.py) is deprecated and no longer accepts new migrations. All schema
changes from here on are managed by Alembic (see
``pkg/persistence/alembic/versions``). This value only gates the one-time
upgrade of pre-existing 3.x databases up to the Alembic baseline."""
debug_mode = False debug_mode = False