mirror of
https://github.com/langbot-app/LangBot.git
synced 2026-09-15 06:17:14 +00:00
Compare commits
26 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 8e3d541876 | |||
| 60bb67f025 | |||
| cbe6844342 | |||
| 940895c541 | |||
| 9b130680ca | |||
| d26d0635c5 | |||
| 58cde8c022 | |||
| 9eb8683997 | |||
| 19526e1400 | |||
| dfde9578c1 | |||
| ec5b8cc8a8 | |||
| 137bb4fdb3 | |||
| 273b8839b9 | |||
| 9db6650274 | |||
| b594cf23e4 | |||
| 45d77c3926 | |||
| 1ea9cd3f6f | |||
| ff6ad6adc2 | |||
| ce6b647fe7 | |||
| 485113ae43 | |||
| 1ba3c1ec72 | |||
| fc1c998434 | |||
| 267232c24f | |||
| d6443b10bc | |||
| 0577689da4 | |||
| 0f216a0d4d |
@@ -0,0 +1,111 @@
|
||||
# Discord release announcements
|
||||
|
||||
This independent workflow announces new stable LangBot releases in the channel
|
||||
selected by a dedicated Discord incoming webhook. It does not change the existing
|
||||
release/build workflows, edit releases, run a persistent service, poll, or backfill.
|
||||
Announcements run on publication, independently of artifact builds finishing.
|
||||
|
||||
## Setup and read-only validation
|
||||
|
||||
1. In the intended community **announcement channel**, create a dedicated incoming
|
||||
webhook (Channel Settings → Integrations → Webhooks). Copy its URL; do not reuse
|
||||
a webhook belonging to another automation.
|
||||
2. In `langbot-app/LangBot` → Settings → Secrets and variables → Actions, create the
|
||||
**repository secret** `DISCORD_RELEASE_WEBHOOK_URL`. Its value must be exactly
|
||||
`https://discord.com/api/webhooks/<id>/<token>` — no query, trailing slash,
|
||||
API-version segment, or alternate domain. Treat the entire URL as a password.
|
||||
3. Once this workflow is on `master`, open Actions → **Discord Release Announcement**
|
||||
→ Run workflow, choosing `master`. Alternatively:
|
||||
|
||||
```sh
|
||||
gh workflow run discord-release.yml --repo langbot-app/LangBot --ref master
|
||||
```
|
||||
|
||||
4. Inspect **Validate webhook (GET only, no message)**. It checks webhook type `1`
|
||||
and reports `guild_id` and `channel_id`; compare both with the intended server
|
||||
and channel using Discord Developer Mode → Copy ID. The secret determines the
|
||||
destination; no channel ID is guessed or overridden. The URL/token is never
|
||||
logged. Dispatch cannot send a test message or announce an old release, even
|
||||
when run again. Missing/invalid secrets fail validation clearly; offline tests
|
||||
do not need secrets.
|
||||
|
||||
GET validation confirms the webhook's identity, not delivery or notification
|
||||
permissions. Verify those on the first genuine release. `mention_everyone=true`
|
||||
confirms Discord parsed the mention; it cannot prove every member received a push
|
||||
notification (member/server notification settings still apply).
|
||||
|
||||
## Activation and message
|
||||
|
||||
The workflow and `.github/discord-release/` helper **must be in the commit targeted
|
||||
by each new release tag**. Merging to `master` does not enable announcements for
|
||||
old tags whose commits lack these files. Manual dispatch becomes available when
|
||||
the workflow is on the default branch. Only publish release tags from trusted,
|
||||
reviewed commits: release workflows execute that tag's code with the secret.
|
||||
|
||||
Only `release` events with action `published`, `draft=false`, and
|
||||
`prerelease=false` can send. Drafts and prereleases are skipped; release edits do
|
||||
not trigger announcements. The helper requires the repository to be exactly
|
||||
`langbot-app/LangBot`, a stable `vX.Y.Z` tag (ASCII digits, at most 64 characters),
|
||||
and its exact canonical GitHub release URL. Other naming schemes fail closed.
|
||||
|
||||
Example message (the version and URL come from the validated event file):
|
||||
|
||||
```text
|
||||
@everyone LangBot v4.10.11 is now available!
|
||||
Release notes: https://github.com/langbot-app/LangBot/releases/tag/v4.10.11
|
||||
```
|
||||
|
||||
The release title/body is never copied. There is one literal `@everyone`, explicit
|
||||
`allowed_mentions.parse=["everyone"]`, empty user/role allowlists, and no reply
|
||||
mention. TTS and notification-suppressing flags are disabled. Requests use HTTPS
|
||||
only to `discord.com`, an explicit User-Agent, and no redirects or automatic
|
||||
retries. After a webhook identity GET, one `POST ?wait=true` obtains a message ID;
|
||||
an exact `/messages/<id>` GET verifies its ID, webhook/channel, content,
|
||||
`mention_everyone=true`, and empty user/role mention arrays before success.
|
||||
|
||||
## Repeat guard and manual recovery
|
||||
|
||||
Production sending requires **`GITHUB_RUN_ATTEMPT == "1"`**. Any Actions rerun
|
||||
(including “Re-run failed jobs”) refuses to POST and requires manual reconciliation,
|
||||
even if the first attempt failed before sending. Read-only dispatch may be rerun.
|
||||
|
||||
This is a practical repeat guard, **not durable exactly-once delivery**. It cannot
|
||||
prevent duplicates from a separate new run/event (for example deleting/recreating
|
||||
a release), separate automation, or manual posting. It stores no durable dedupe
|
||||
state and never modifies the release to mark delivery.
|
||||
|
||||
If a POST times out, returns an error, or readback fails, the message may already
|
||||
exist. The workflow fails rather than blindly sending again. A returned message ID
|
||||
is included in the safe error when available. A runner termination can also leave
|
||||
an ambiguous send without that log line.
|
||||
|
||||
1. Inspect the announcement channel and the failed run logs. Locate the canonical
|
||||
release link and, if available, the returned message ID. A failed verification
|
||||
does **not** mean the message was absent.
|
||||
2. If present, reconcile the existing message/mention problem manually; do not
|
||||
rerun, create another release event, or send a duplicate ping.
|
||||
3. If an operator has positively confirmed no message exists, fix the secret or
|
||||
permission issue and use read-only dispatch to validate configuration. A
|
||||
maintainer may then post the announcement manually once in Discord and record
|
||||
the message link in the incident/run notes. Do not override the attempt guard
|
||||
or delete/recreate a release to force recovery.
|
||||
4. If absence cannot be established, pause and reconcile rather than resending.
|
||||
|
||||
To stop future sends, disable **Discord Release Announcement** in Actions. Rotate
|
||||
or delete the dedicated Discord webhook if the URL is exposed, and update the
|
||||
secret before validation. No rollback of release artifacts is involved.
|
||||
|
||||
## Local checks
|
||||
|
||||
Requires Python 3.11+ and the standard library only:
|
||||
|
||||
```sh
|
||||
python3 -m unittest discover -s .github/discord-release -p 'test_*.py' -v
|
||||
python3 -m py_compile .github/discord-release/announce.py .github/discord-release/test_announce.py
|
||||
```
|
||||
|
||||
Tests exercise policy, CLI/event-file handling, mention payloads, hostile inputs,
|
||||
HTTP failures, exact message readback, and refusal to retry. Only the HTTPS
|
||||
transport is mocked for Discord tests; no live Discord requests or messages are
|
||||
made. Changes to this directory or its workflow run the offline tests on push and
|
||||
pull request; tests also gate release sending and read-only dispatch validation.
|
||||
@@ -0,0 +1,162 @@
|
||||
"""Announce only first-attempt stable releases; dispatch is read-only validation."""
|
||||
|
||||
import http.client
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import sys
|
||||
|
||||
REPOSITORY = 'langbot-app/LangBot'
|
||||
RELEASE_PREFIX = f'https://github.com/{REPOSITORY}/releases/tag/'
|
||||
RECONCILE = (
|
||||
'Do not resend or bypass the run-attempt guard; manual reconciliation is required. '
|
||||
'Inspect the announcement channel and workflow logs before any manual recovery '
|
||||
'(see .github/discord-release/README.md).'
|
||||
)
|
||||
|
||||
|
||||
class AnnouncementError(Exception):
|
||||
"""A safe, operator-facing error containing no webhook URL or response body."""
|
||||
|
||||
|
||||
def release_payload(event, attempt):
|
||||
"""Return a bounded, mention-safe payload, or None for draft/preview releases."""
|
||||
if not isinstance(event, dict) or event.get('action') != 'published':
|
||||
raise AnnouncementError('Only release.published events are accepted.')
|
||||
repository = event.get('repository')
|
||||
if not isinstance(repository, dict) or repository.get('full_name') != REPOSITORY:
|
||||
raise AnnouncementError('Unexpected release repository.')
|
||||
release = event.get('release')
|
||||
if not isinstance(release, dict) or any(type(release.get(key)) is not bool for key in ('draft', 'prerelease')):
|
||||
raise AnnouncementError('Invalid release flags.')
|
||||
if release['draft'] or release['prerelease']:
|
||||
return None
|
||||
if attempt != '1':
|
||||
raise AnnouncementError(f'Release reruns or missing run attempts are refused. {RECONCILE}')
|
||||
tag = release.get('tag_name')
|
||||
if not isinstance(tag, str) or len(tag) > 64 or not re.fullmatch(r'v[0-9]+\.[0-9]+\.[0-9]+', tag):
|
||||
raise AnnouncementError('Expected a stable release tag in vX.Y.Z format (at most 64 characters).')
|
||||
url = RELEASE_PREFIX + tag
|
||||
if release.get('html_url') != url:
|
||||
raise AnnouncementError('Release URL must be the canonical LangBot release URL matching its tag.')
|
||||
return {
|
||||
'content': f'@everyone LangBot {tag} is now available!\nRelease notes: {url}',
|
||||
'allowed_mentions': {'parse': ['everyone'], 'users': [], 'roles': [], 'replied_user': False},
|
||||
'tts': False,
|
||||
'flags': 0,
|
||||
}
|
||||
|
||||
|
||||
def is_snowflake(value):
|
||||
return isinstance(value, str) and re.fullmatch(r'[0-9]{1,20}', value) is not None
|
||||
|
||||
|
||||
class DiscordWebhook:
|
||||
def __init__(self, url):
|
||||
if not url:
|
||||
raise AnnouncementError('DISCORD_RELEASE_WEBHOOK_URL is missing. Set the repository Actions secret.')
|
||||
match = re.fullmatch(r'https://discord\.com(/api/webhooks/([0-9]{1,20})/[A-Za-z0-9_-]+)', url)
|
||||
if not match:
|
||||
raise AnnouncementError('Invalid webhook URL; expected https://discord.com/api/webhooks/<id>/<token>.')
|
||||
self.path, self.id = match.groups()
|
||||
|
||||
def _request(self, method, suffix='', payload=None):
|
||||
# Direct HTTPS, default certificate verification, no proxies or redirect/retry machinery.
|
||||
connection = http.client.HTTPSConnection('discord.com', timeout=20)
|
||||
try:
|
||||
body = json.dumps(payload).encode('utf-8') if payload is not None else None
|
||||
connection.request(
|
||||
method,
|
||||
self.path + suffix,
|
||||
body=body,
|
||||
headers={'Content-Type': 'application/json', 'User-Agent': 'LangBot-Release-Announcements/1.0'},
|
||||
)
|
||||
response = connection.getresponse()
|
||||
if response.status != 200:
|
||||
raise AnnouncementError(f'Discord {method} returned HTTP {response.status}; no retry was attempted.')
|
||||
raw = response.read(1_048_577)
|
||||
if len(raw) > 1_048_576:
|
||||
raise AnnouncementError('Discord response exceeded the size limit.')
|
||||
return json.loads(raw)
|
||||
except (OSError, http.client.HTTPException, ValueError, UnicodeError):
|
||||
# Exceptions and bodies can contain the token; never print them or chain them.
|
||||
raise AnnouncementError(
|
||||
f'Discord {method} failed or returned invalid JSON; no retry was attempted.'
|
||||
) from None
|
||||
finally:
|
||||
connection.close()
|
||||
|
||||
def validate(self):
|
||||
"""GET only: verify an incoming webhook and return safe identifying fields."""
|
||||
webhook = self._request('GET')
|
||||
if (
|
||||
not isinstance(webhook, dict)
|
||||
or type(webhook.get('type')) is not int
|
||||
or webhook['type'] != 1
|
||||
or webhook.get('id') != self.id
|
||||
or not is_snowflake(webhook.get('guild_id'))
|
||||
or not is_snowflake(webhook.get('channel_id'))
|
||||
):
|
||||
raise AnnouncementError('Expected an incoming (type 1) webhook with matching ID and guild/channel IDs.')
|
||||
return {key: webhook[key] for key in ('id', 'type', 'guild_id', 'channel_id')}
|
||||
|
||||
def send(self, payload):
|
||||
"""One POST, followed by exact message GET; never automatically retry a send."""
|
||||
webhook = self.validate()
|
||||
message_id = None
|
||||
try:
|
||||
sent = self._request('POST', '?wait=true', payload)
|
||||
if not isinstance(sent, dict) or not is_snowflake(sent.get('id')):
|
||||
raise AnnouncementError('Discord did not return a valid message ID.')
|
||||
message_id = sent['id']
|
||||
saved = self._request('GET', f'/messages/{message_id}')
|
||||
if (
|
||||
not isinstance(saved, dict)
|
||||
or saved.get('id') != message_id
|
||||
or saved.get('webhook_id') != self.id
|
||||
or saved.get('channel_id') != webhook['channel_id']
|
||||
or saved.get('content') != payload['content']
|
||||
or saved.get('mention_everyone') is not True
|
||||
or saved.get('mentions') != []
|
||||
or saved.get('mention_roles') != []
|
||||
):
|
||||
raise AnnouncementError('Discord message readback did not match content, identity, or mentions.')
|
||||
except AnnouncementError as error:
|
||||
reference = f' Returned message ID: {message_id}.' if message_id else ''
|
||||
raise AnnouncementError(f'Delivery not confirmed. {error}{reference} {RECONCILE}') from None
|
||||
return message_id
|
||||
|
||||
|
||||
def main(env=None):
|
||||
env = os.environ if env is None else env
|
||||
try:
|
||||
if env.get('GITHUB_REPOSITORY') != REPOSITORY:
|
||||
raise AnnouncementError('This workflow is restricted to langbot-app/LangBot.')
|
||||
name = env.get('GITHUB_EVENT_NAME')
|
||||
if name == 'workflow_dispatch':
|
||||
webhook = DiscordWebhook(env.get('DISCORD_RELEASE_WEBHOOK_URL')).validate()
|
||||
print(
|
||||
f'Validated incoming webhook: guild_id={webhook["guild_id"]} channel_id={webhook["channel_id"]}. No message sent.'
|
||||
)
|
||||
return 0
|
||||
if name != 'release':
|
||||
raise AnnouncementError('Only release and workflow_dispatch events are accepted by this helper.')
|
||||
try:
|
||||
event = json.loads(Path(env.get('GITHUB_EVENT_PATH', '')).read_text(encoding='utf-8'))
|
||||
except (OSError, ValueError, UnicodeError):
|
||||
raise AnnouncementError('Cannot read a valid JSON release event from GITHUB_EVENT_PATH.') from None
|
||||
payload = release_payload(event, env.get('GITHUB_RUN_ATTEMPT'))
|
||||
if payload is None:
|
||||
print('Skipped draft or prerelease; no message sent.')
|
||||
return 0
|
||||
message_id = DiscordWebhook(env.get('DISCORD_RELEASE_WEBHOOK_URL')).send(payload)
|
||||
print(f'Announcement verified by exact message readback: message_id={message_id}.')
|
||||
return 0
|
||||
except AnnouncementError as error:
|
||||
print(f'Error: {error}', file=sys.stderr)
|
||||
return 1
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,427 @@
|
||||
"""Offline contract tests; no Discord credentials or network required."""
|
||||
|
||||
import contextlib
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
try:
|
||||
import announce
|
||||
except ModuleNotFoundError:
|
||||
announce = None
|
||||
|
||||
WEBHOOK = 'https://discord.com/api/webhooks/123456789012345678/fixture_token-ONLY'
|
||||
WEBHOOK_ID = '123456789012345678'
|
||||
GUILD_ID = '234567890123456789'
|
||||
CHANNEL_ID = '345678901234567890'
|
||||
MESSAGE_ID = '456789012345678901'
|
||||
REPO = 'langbot-app/LangBot'
|
||||
URL = f'https://github.com/{REPO}/releases/tag/v4.10.11'
|
||||
CONTENT = f'@everyone LangBot v4.10.11 is now available!\nRelease notes: {URL}'
|
||||
|
||||
|
||||
def event():
|
||||
return {
|
||||
'action': 'published',
|
||||
'repository': {'full_name': REPO},
|
||||
'release': {
|
||||
'draft': False,
|
||||
'prerelease': False,
|
||||
'tag_name': 'v4.10.11',
|
||||
'html_url': URL,
|
||||
'name': 'Hostile @everyone <@123> $(touch /tmp/unsafe)',
|
||||
'body': '@everyone @here <@123> <@&456> `hostile`',
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def metadata():
|
||||
return {'id': WEBHOOK_ID, 'type': 1, 'guild_id': GUILD_ID, 'channel_id': CHANNEL_ID}
|
||||
|
||||
|
||||
def message():
|
||||
return {
|
||||
'id': MESSAGE_ID,
|
||||
'webhook_id': WEBHOOK_ID,
|
||||
'channel_id': CHANNEL_ID,
|
||||
'content': CONTENT,
|
||||
'mention_everyone': True,
|
||||
'mentions': [],
|
||||
'mention_roles': [],
|
||||
}
|
||||
|
||||
|
||||
class BaseTest(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.assertIsNotNone(announce, 'The release announcement helper must exist')
|
||||
|
||||
|
||||
class PolicyTests(BaseTest):
|
||||
def test_payload_has_one_literal_everyone_and_no_untrusted_body(self):
|
||||
payload = announce.release_payload(event(), '1')
|
||||
self.assertEqual(payload['content'], CONTENT)
|
||||
self.assertEqual(json.dumps(payload).count('@everyone'), 1)
|
||||
self.assertEqual(
|
||||
payload['allowed_mentions'],
|
||||
{
|
||||
'parse': ['everyone'],
|
||||
'users': [],
|
||||
'roles': [],
|
||||
'replied_user': False,
|
||||
},
|
||||
)
|
||||
self.assertIs(payload['tts'], False)
|
||||
self.assertEqual(payload['flags'], 0)
|
||||
|
||||
def test_drafts_and_prereleases_are_skipped(self):
|
||||
for flag in ('draft', 'prerelease'):
|
||||
with self.subTest(flag=flag):
|
||||
value = event()
|
||||
value['release'][flag] = True
|
||||
self.assertIsNone(announce.release_payload(value, '1'))
|
||||
|
||||
def test_only_published_action_is_accepted(self):
|
||||
for action in ('edited', 'created', 'released', 'deleted', '', None):
|
||||
with self.subTest(action=action):
|
||||
value = event()
|
||||
value['action'] = action
|
||||
with self.assertRaises(announce.AnnouncementError):
|
||||
announce.release_payload(value, '1')
|
||||
|
||||
def test_reruns_and_missing_attempt_refuse_manual_reconciliation(self):
|
||||
for attempt in ('2', '3', '', None, '01', '0', '1\n'):
|
||||
with self.subTest(attempt=attempt):
|
||||
with self.assertRaisesRegex(announce.AnnouncementError, 'manual reconciliation'):
|
||||
announce.release_payload(event(), attempt)
|
||||
|
||||
def test_repository_must_match_exactly(self):
|
||||
for repo in ('evil/LangBot', 'langbot-app/langbot', None):
|
||||
value = event()
|
||||
value['repository']['full_name'] = repo
|
||||
with self.assertRaises(announce.AnnouncementError):
|
||||
announce.release_payload(value, '1')
|
||||
|
||||
def test_hostile_and_noncanonical_tags_are_rejected(self):
|
||||
for tag in (
|
||||
'v1.2.3 @everyone',
|
||||
'v1.2.3\n',
|
||||
'v1.2.3/../../x',
|
||||
'v1.2.3?x=y',
|
||||
'$(id)',
|
||||
'v1.2.3-rc.1',
|
||||
'v1.2.3',
|
||||
'v1.2.3%0a',
|
||||
'<@123>',
|
||||
'v1.2.' + '3' * 100,
|
||||
'',
|
||||
None,
|
||||
123,
|
||||
):
|
||||
with self.subTest(tag=tag):
|
||||
value = event()
|
||||
value['release']['tag_name'] = tag
|
||||
value['release']['html_url'] = f'https://github.com/{REPO}/releases/tag/{tag}'
|
||||
with self.assertRaises(announce.AnnouncementError):
|
||||
announce.release_payload(value, '1')
|
||||
|
||||
def test_release_url_must_be_canonical_and_match_tag(self):
|
||||
for url in (
|
||||
'https://evil.example/tag/v4.10.11',
|
||||
URL + '?x=y',
|
||||
URL + '#anchor',
|
||||
URL + '/',
|
||||
URL.replace('v4.10.11', 'v4.10.12'),
|
||||
URL.replace('github.com', 'github.com@evil.example'),
|
||||
URL.replace('https:', 'http:'),
|
||||
URL + '\n',
|
||||
None,
|
||||
):
|
||||
with self.subTest(url=url):
|
||||
value = event()
|
||||
value['release']['html_url'] = url
|
||||
with self.assertRaises(announce.AnnouncementError):
|
||||
announce.release_payload(value, '1')
|
||||
|
||||
def test_malformed_events_fail_closed(self):
|
||||
for value in (None, [], {}, {'release': []}, {'repository': None}):
|
||||
with self.subTest(value=value):
|
||||
with self.assertRaises(announce.AnnouncementError):
|
||||
announce.release_payload(value, '1')
|
||||
for flag in ('draft', 'prerelease'):
|
||||
for bad in (None, 'false', 0, 1):
|
||||
value = event()
|
||||
value['release'][flag] = bad
|
||||
with self.assertRaises(announce.AnnouncementError):
|
||||
announce.release_payload(value, '1')
|
||||
|
||||
|
||||
class DiscordTests(BaseTest):
|
||||
def setUp(self):
|
||||
super().setUp()
|
||||
self.patch = patch('announce.http.client.HTTPSConnection')
|
||||
self.connection_class = self.patch.start()
|
||||
self.addCleanup(self.patch.stop)
|
||||
self.connection = self.connection_class.return_value
|
||||
|
||||
def respond(self, *values):
|
||||
responses = []
|
||||
for value in values:
|
||||
response = MagicMock()
|
||||
response.status = 200
|
||||
response.read.return_value = json.dumps(value).encode()
|
||||
responses.append(response)
|
||||
self.connection.getresponse.side_effect = responses
|
||||
|
||||
def methods(self):
|
||||
return [call.args[0] for call in self.connection.request.call_args_list]
|
||||
|
||||
def test_webhook_validation_is_get_only_and_reports_ids(self):
|
||||
self.respond(metadata())
|
||||
result = announce.DiscordWebhook(WEBHOOK).validate()
|
||||
self.assertEqual(result, metadata())
|
||||
self.assertEqual(self.methods(), ['GET'])
|
||||
self.assertEqual(
|
||||
self.connection.request.call_args.args[:2], ('GET', f'/api/webhooks/{WEBHOOK_ID}/fixture_token-ONLY')
|
||||
)
|
||||
self.connection_class.assert_called_with('discord.com', timeout=20)
|
||||
self.connection.close.assert_called_once()
|
||||
|
||||
def test_invalid_webhook_urls_are_rejected_before_network(self):
|
||||
for url in (
|
||||
'',
|
||||
None,
|
||||
WEBHOOK + '/',
|
||||
WEBHOOK + '?wait=true',
|
||||
WEBHOOK + '#x',
|
||||
WEBHOOK + '\n',
|
||||
' ' + WEBHOOK,
|
||||
WEBHOOK.replace('https:', 'http:'),
|
||||
WEBHOOK.replace('discord.com', 'discord.com.evil.example'),
|
||||
WEBHOOK.replace('discord.com', 'discord.com@evil.example'),
|
||||
WEBHOOK.replace('discord.com', 'discord.com:443'),
|
||||
WEBHOOK.replace('/api/', '/api/v10/'),
|
||||
WEBHOOK.replace(WEBHOOK_ID, 'abc'),
|
||||
WEBHOOK + '/../../x',
|
||||
WEBHOOK.replace('fixture_token-ONLY', 'a%2Fb'),
|
||||
):
|
||||
with self.subTest(url=url):
|
||||
with self.assertRaises(announce.AnnouncementError):
|
||||
announce.DiscordWebhook(url)
|
||||
self.connection_class.assert_not_called()
|
||||
|
||||
def test_webhook_metadata_requires_incoming_type_and_ids(self):
|
||||
invalid = [
|
||||
None,
|
||||
[],
|
||||
{},
|
||||
dict(metadata(), type=2),
|
||||
dict(metadata(), type=True),
|
||||
dict(metadata(), id='999'),
|
||||
dict(metadata(), channel_id=None),
|
||||
dict(metadata(), guild_id='::error::hostile'),
|
||||
]
|
||||
for value in invalid:
|
||||
with self.subTest(value=value):
|
||||
self.respond(value)
|
||||
with self.assertRaises(announce.AnnouncementError):
|
||||
announce.DiscordWebhook(WEBHOOK).validate()
|
||||
self.assertNotIn('POST', self.methods())
|
||||
|
||||
def test_send_waits_and_reads_back_exact_returned_message(self):
|
||||
self.respond(metadata(), message(), message())
|
||||
result = announce.DiscordWebhook(WEBHOOK).send(announce.release_payload(event(), '1'))
|
||||
self.assertEqual(result, MESSAGE_ID)
|
||||
self.assertEqual(self.methods(), ['GET', 'POST', 'GET'])
|
||||
calls = self.connection.request.call_args_list
|
||||
self.assertEqual(calls[1].args[:2], ('POST', f'/api/webhooks/{WEBHOOK_ID}/fixture_token-ONLY?wait=true'))
|
||||
self.assertEqual(json.loads(calls[1].kwargs['body']), announce.release_payload(event(), '1'))
|
||||
self.assertEqual(
|
||||
calls[2].args[:2], ('GET', f'/api/webhooks/{WEBHOOK_ID}/fixture_token-ONLY/messages/{MESSAGE_ID}')
|
||||
)
|
||||
|
||||
def test_readback_must_match_content_mentions_and_identity(self):
|
||||
for field, bad in (
|
||||
('content', 'wrong'),
|
||||
('mention_everyone', False),
|
||||
('mention_everyone', 1),
|
||||
('mentions', [{'id': '123'}]),
|
||||
('mention_roles', ['123']),
|
||||
('id', '999'),
|
||||
('channel_id', '999'),
|
||||
('webhook_id', '999'),
|
||||
):
|
||||
with self.subTest(field=field, bad=bad):
|
||||
self.connection.reset_mock()
|
||||
self.respond(metadata(), message(), dict(message(), **{field: bad}))
|
||||
with self.assertRaisesRegex(announce.AnnouncementError, 'manual reconciliation'):
|
||||
announce.DiscordWebhook(WEBHOOK).send(announce.release_payload(event(), '1'))
|
||||
self.assertEqual(self.methods().count('POST'), 1)
|
||||
|
||||
def test_missing_readback_fields_fail_closed(self):
|
||||
for field in message():
|
||||
value = message()
|
||||
del value[field]
|
||||
self.respond(metadata(), message(), value)
|
||||
with self.assertRaises(announce.AnnouncementError):
|
||||
announce.DiscordWebhook(WEBHOOK).send(announce.release_payload(event(), '1'))
|
||||
|
||||
def test_unsafe_post_message_id_never_becomes_get_path(self):
|
||||
for value in (None, {}, dict(message(), id='../evil'), dict(message(), id='123?x=y')):
|
||||
self.connection.reset_mock()
|
||||
self.respond(metadata(), value)
|
||||
with self.assertRaisesRegex(announce.AnnouncementError, 'manual reconciliation'):
|
||||
announce.DiscordWebhook(WEBHOOK).send(announce.release_payload(event(), '1'))
|
||||
self.assertEqual(self.methods(), ['GET', 'POST'])
|
||||
|
||||
def test_post_failure_never_retries_and_never_logs_secret(self):
|
||||
for status in (301, 302, 307, 308, 400, 401, 403, 429, 500, 204):
|
||||
with self.subTest(status=status):
|
||||
self.connection.reset_mock()
|
||||
self.respond(metadata(), message())
|
||||
responses = list(self.connection.getresponse.side_effect)
|
||||
responses[1].status = status
|
||||
self.connection.getresponse.side_effect = responses
|
||||
with self.assertRaisesRegex(announce.AnnouncementError, 'manual reconciliation') as caught:
|
||||
announce.DiscordWebhook(WEBHOOK).send(announce.release_payload(event(), '1'))
|
||||
self.assertNotIn('fixture_token', str(caught.exception))
|
||||
self.assertEqual(self.methods(), ['GET', 'POST'])
|
||||
|
||||
def test_ambiguous_timeout_never_retries_or_echoes_exception(self):
|
||||
self.respond(metadata())
|
||||
first = next(self.connection.getresponse.side_effect)
|
||||
self.connection.getresponse.side_effect = [first, TimeoutError(WEBHOOK)]
|
||||
with self.assertRaisesRegex(announce.AnnouncementError, 'manual reconciliation') as caught:
|
||||
announce.DiscordWebhook(WEBHOOK).send(announce.release_payload(event(), '1'))
|
||||
self.assertNotIn('fixture_token', str(caught.exception))
|
||||
self.assertEqual(self.methods(), ['GET', 'POST'])
|
||||
|
||||
def test_malformed_json_response_is_sanitized(self):
|
||||
self.respond(metadata())
|
||||
response = next(self.connection.getresponse.side_effect)
|
||||
response.read.return_value = WEBHOOK.encode()
|
||||
self.connection.getresponse.side_effect = [response]
|
||||
with self.assertRaises(announce.AnnouncementError) as caught:
|
||||
announce.DiscordWebhook(WEBHOOK).validate()
|
||||
self.assertNotIn('fixture_token', str(caught.exception))
|
||||
|
||||
def test_get_redirect_is_not_followed(self):
|
||||
self.respond(metadata())
|
||||
response = next(self.connection.getresponse.side_effect)
|
||||
response.status = 302
|
||||
response.getheader.return_value = 'https://evil.example/'
|
||||
self.connection.getresponse.side_effect = [response]
|
||||
with self.assertRaises(announce.AnnouncementError):
|
||||
announce.DiscordWebhook(WEBHOOK).validate()
|
||||
self.assertEqual(self.methods(), ['GET'])
|
||||
|
||||
|
||||
class EntrypointTests(BaseTest):
|
||||
def run_main(self, data=None, **overrides):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
path = Path(directory) / 'event.json'
|
||||
path.write_text(json.dumps(event() if data is None else data))
|
||||
env = {
|
||||
'GITHUB_EVENT_NAME': 'release',
|
||||
'GITHUB_EVENT_PATH': str(path),
|
||||
'GITHUB_REPOSITORY': REPO,
|
||||
'GITHUB_RUN_ATTEMPT': '1',
|
||||
'DISCORD_RELEASE_WEBHOOK_URL': WEBHOOK,
|
||||
}
|
||||
env.update(overrides)
|
||||
output = io.StringIO()
|
||||
with contextlib.redirect_stdout(output), contextlib.redirect_stderr(output):
|
||||
result = announce.main(env)
|
||||
return result, output.getvalue()
|
||||
|
||||
def test_dispatch_only_validates_even_if_event_contains_release(self):
|
||||
with patch('announce.DiscordWebhook') as client:
|
||||
client.return_value.validate.return_value = metadata()
|
||||
result, output = self.run_main(GITHUB_EVENT_NAME='workflow_dispatch')
|
||||
self.assertEqual(result, 0)
|
||||
client.return_value.validate.assert_called_once()
|
||||
client.return_value.send.assert_not_called()
|
||||
self.assertIn(GUILD_ID, output)
|
||||
self.assertIn(CHANNEL_ID, output)
|
||||
self.assertNotIn('fixture_token', output)
|
||||
|
||||
def test_production_release_sends_once(self):
|
||||
with patch('announce.DiscordWebhook') as client:
|
||||
client.return_value.send.return_value = MESSAGE_ID
|
||||
result, output = self.run_main()
|
||||
self.assertEqual(result, 0)
|
||||
client.return_value.send.assert_called_once_with(announce.release_payload(event(), '1'))
|
||||
self.assertIn(MESSAGE_ID, output)
|
||||
|
||||
def test_skipped_releases_need_no_secret_or_network(self):
|
||||
for flag in ('draft', 'prerelease'):
|
||||
value = event()
|
||||
value['release'][flag] = True
|
||||
with patch('announce.DiscordWebhook') as client:
|
||||
result, _ = self.run_main(value, DISCORD_RELEASE_WEBHOOK_URL='')
|
||||
self.assertEqual(result, 0)
|
||||
client.assert_not_called()
|
||||
|
||||
def test_rerun_never_constructs_client(self):
|
||||
with patch('announce.DiscordWebhook') as client:
|
||||
result, output = self.run_main(GITHUB_RUN_ATTEMPT='2')
|
||||
self.assertEqual(result, 1)
|
||||
self.assertIn('manual reconciliation', output)
|
||||
client.assert_not_called()
|
||||
|
||||
def test_unexpected_event_or_repository_cannot_send(self):
|
||||
for overrides in (
|
||||
{'GITHUB_EVENT_NAME': 'push'},
|
||||
{'GITHUB_EVENT_NAME': 'pull_request'},
|
||||
{'GITHUB_REPOSITORY': 'evil/LangBot'},
|
||||
):
|
||||
with patch('announce.DiscordWebhook') as client:
|
||||
result, _ = self.run_main(**overrides)
|
||||
self.assertEqual(result, 1)
|
||||
client.assert_not_called()
|
||||
|
||||
def test_missing_secret_fails_clearly_for_send_and_validation(self):
|
||||
for name in ('release', 'workflow_dispatch'):
|
||||
result, output = self.run_main(GITHUB_EVENT_NAME=name, DISCORD_RELEASE_WEBHOOK_URL='')
|
||||
self.assertEqual(result, 1)
|
||||
self.assertIn('DISCORD_RELEASE_WEBHOOK_URL is missing', output)
|
||||
|
||||
def test_cli_reads_event_file_and_redacts_invalid_input(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
path = Path(directory) / 'event.json'
|
||||
value = event()
|
||||
value['release']['tag_name'] = '::error::hostile @everyone'
|
||||
path.write_text(json.dumps(value))
|
||||
env = dict(
|
||||
os.environ,
|
||||
GITHUB_EVENT_NAME='release',
|
||||
GITHUB_EVENT_PATH=str(path),
|
||||
GITHUB_REPOSITORY=REPO,
|
||||
GITHUB_RUN_ATTEMPT='1',
|
||||
DISCORD_RELEASE_WEBHOOK_URL=WEBHOOK,
|
||||
)
|
||||
result = subprocess.run(
|
||||
[sys.executable, str(Path(__file__).with_name('announce.py'))],
|
||||
env=env,
|
||||
text=True,
|
||||
capture_output=True,
|
||||
check=False,
|
||||
)
|
||||
self.assertEqual(result.returncode, 1)
|
||||
self.assertNotIn('hostile', result.stderr)
|
||||
self.assertNotIn('fixture_token', result.stderr)
|
||||
self.assertNotIn('Traceback', result.stderr)
|
||||
|
||||
def test_unreadable_event_fails_safely(self):
|
||||
result, output = self.run_main(GITHUB_EVENT_PATH='/nonexistent/event.json')
|
||||
self.assertEqual(result, 1)
|
||||
self.assertNotIn('Traceback', output)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
@@ -0,0 +1,78 @@
|
||||
name: Build fnOS FPK
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
## 发布release的时候会自动构建
|
||||
release:
|
||||
types: [published]
|
||||
|
||||
permissions:
|
||||
contents: write
|
||||
|
||||
jobs:
|
||||
build-fnos-fpk:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v2
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
- name: Check version
|
||||
id: check_version
|
||||
run: |
|
||||
echo $GITHUB_REF
|
||||
# 如果是tag,则去掉refs/tags/前缀(与其他 release workflow 一致,版本号取 tag 名)
|
||||
if [[ $GITHUB_REF == refs/tags/* ]]; then
|
||||
echo "It's a tag"
|
||||
echo "version=$(echo $GITHUB_REF | awk -F '/' '{print $3}')" >> $GITHUB_OUTPUT
|
||||
else
|
||||
# 手动触发(workflow_dispatch):读不到 tag,使用 manifest 内维护的版本
|
||||
echo "It's not a tag"
|
||||
echo "version=$(grep '^version=' packaging/fnos/manifest | cut -d= -f2)" >> $GITHUB_OUTPUT
|
||||
fi
|
||||
|
||||
- name: Setup Node
|
||||
uses: actions/setup-node@v2
|
||||
with:
|
||||
node-version: '22'
|
||||
|
||||
- name: Setup Python
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: '3.12'
|
||||
|
||||
- name: Install build tools
|
||||
run: |
|
||||
pip install pillow
|
||||
# fnpack:飞牛官方打包 CLI(静态二进制)
|
||||
curl -fsSL -o /usr/local/bin/fnpack \
|
||||
https://static2.fnnas.com/fnpack/fnpack-1.2.3-linux-amd64
|
||||
chmod +x /usr/local/bin/fnpack
|
||||
|
||||
- name: Build FPK
|
||||
env:
|
||||
FPK_VERSION: ${{ steps.check_version.outputs.version }}
|
||||
run: |
|
||||
bash packaging/fnos/build.sh
|
||||
test -f packaging/fnos/langbot.fpk
|
||||
|
||||
- name: Upload Artifact
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: langbot-${{ steps.check_version.outputs.version }}-fnos
|
||||
path: packaging/fnos/langbot.fpk
|
||||
|
||||
- name: Upload To Release
|
||||
# 仅 release 触发时执行;手动/workflow_dispatch 触发时没有 release,
|
||||
# 且 github.event.release.tag_name 为空(否则 gh release upload 缺参数报错)
|
||||
if: github.event_name == 'release'
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
run: |
|
||||
# tag 可能已带 -fnos 后缀(如 v4.10.9-fnos),产物名统一为
|
||||
# langbot-<基础版本>-fnos.fpk,避免出现 -fnos-fnos
|
||||
VER="${{ steps.check_version.outputs.version }}"
|
||||
BASE="${VER#v}"; BASE="${BASE%-fnos}"
|
||||
cp packaging/fnos/langbot.fpk "langbot-${BASE}-fnos.fpk"
|
||||
gh release upload ${{ github.event.release.tag_name }} "langbot-${BASE}-fnos.fpk"
|
||||
@@ -0,0 +1,64 @@
|
||||
name: Discord Release Announcement
|
||||
|
||||
on:
|
||||
release:
|
||||
types: [published]
|
||||
workflow_dispatch:
|
||||
push:
|
||||
paths:
|
||||
- '.github/workflows/discord-release.yml'
|
||||
- '.github/discord-release/**'
|
||||
pull_request:
|
||||
paths:
|
||||
- '.github/workflows/discord-release.yml'
|
||||
- '.github/discord-release/**'
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
tests:
|
||||
name: Offline announcement tests
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 5
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
|
||||
with:
|
||||
persist-credentials: false
|
||||
- name: Test helper without secrets or network
|
||||
run: python3 -m unittest discover -s .github/discord-release -p 'test_*.py' -v
|
||||
|
||||
validate:
|
||||
name: Validate webhook (GET only, no message)
|
||||
if: github.repository == 'langbot-app/LangBot' && github.event_name == 'workflow_dispatch'
|
||||
needs: tests
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 5
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
|
||||
with:
|
||||
persist-credentials: false
|
||||
- name: Validate incoming webhook and report guild/channel IDs
|
||||
env:
|
||||
DISCORD_RELEASE_WEBHOOK_URL: ${{ secrets.DISCORD_RELEASE_WEBHOOK_URL }}
|
||||
run: python3 .github/discord-release/announce.py
|
||||
|
||||
announce:
|
||||
name: Announce published stable release
|
||||
if: >-
|
||||
github.repository == 'langbot-app/LangBot' &&
|
||||
github.event_name == 'release' && github.event.action == 'published' &&
|
||||
github.event.release.draft == false && github.event.release.prerelease == false
|
||||
needs: tests
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 5
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
|
||||
with:
|
||||
persist-credentials: false
|
||||
# The helper refuses GITHUB_RUN_ATTEMPT != 1 with recovery guidance.
|
||||
# Never interpolate release data into a shell command.
|
||||
- name: Send once and verify the exact Discord message
|
||||
env:
|
||||
DISCORD_RELEASE_WEBHOOK_URL: ${{ secrets.DISCORD_RELEASE_WEBHOOK_URL }}
|
||||
run: python3 .github/discord-release/announce.py
|
||||
@@ -2,6 +2,11 @@ name: Build and Publish to PyPI
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
source_ref:
|
||||
description: 'Existing release tag to publish (for example v4.10.11)'
|
||||
required: true
|
||||
type: string
|
||||
release:
|
||||
types: [published]
|
||||
|
||||
@@ -11,13 +16,39 @@ jobs:
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write # Required for trusted publishing to PyPI
|
||||
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
ref: ${{ inputs.source_ref || github.sha }}
|
||||
fetch-depth: 0
|
||||
persist-credentials: false
|
||||
|
||||
- name: Validate release source and version
|
||||
env:
|
||||
RELEASE_TAG: ${{ inputs.source_ref || github.event.release.tag_name }}
|
||||
run: |
|
||||
python3 - <<'PY'
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
import tomllib
|
||||
from pathlib import Path
|
||||
|
||||
tag = os.environ['RELEASE_TAG']
|
||||
if not re.fullmatch(r'v[0-9]+\.[0-9]+\.[0-9]+', tag):
|
||||
raise SystemExit('source_ref must be an existing release tag: vX.Y.Z')
|
||||
def revision(ref):
|
||||
return subprocess.check_output(['git', 'rev-parse', '--verify', ref], text=True).strip()
|
||||
if revision('HEAD') != revision(f'refs/tags/{tag}^{{}}'):
|
||||
raise SystemExit('Checked-out commit does not match the release tag')
|
||||
version = tomllib.loads(Path('pyproject.toml').read_text())['project']['version']
|
||||
if version != tag[1:]:
|
||||
raise SystemExit(f'Package version {version} does not match tag {tag}')
|
||||
print(f'Validated {tag} at {revision("HEAD")} (package {version})')
|
||||
PY
|
||||
|
||||
- name: Set up Node.js
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
@@ -26,9 +57,9 @@ jobs:
|
||||
- name: Build frontend
|
||||
run: |
|
||||
cd web
|
||||
npm install -g pnpm
|
||||
pnpm install
|
||||
pnpm build
|
||||
# Match the archive/Docker npm path; npm ci rejects older tags' stale npm lockfiles.
|
||||
npm install --include=optional
|
||||
npm run build
|
||||
mkdir -p ../src/langbot/web/dist
|
||||
cp -r dist ../src/langbot/web/
|
||||
|
||||
|
||||
@@ -10,12 +10,16 @@ on:
|
||||
- 'src/langbot/pkg/persistence/**'
|
||||
- 'src/langbot/pkg/entity/persistence/**'
|
||||
- 'tests/integration/persistence/**'
|
||||
- 'tests/unit_tests/api/service/test_monitoring_sessions.py'
|
||||
- '.github/workflows/test-migrations.yml'
|
||||
pull_request:
|
||||
types: [opened, synchronize, reopened, ready_for_review]
|
||||
paths:
|
||||
- 'src/langbot/pkg/persistence/**'
|
||||
- 'src/langbot/pkg/entity/persistence/**'
|
||||
- 'tests/integration/persistence/**'
|
||||
- 'tests/unit_tests/api/service/test_monitoring_sessions.py'
|
||||
- '.github/workflows/test-migrations.yml'
|
||||
|
||||
jobs:
|
||||
test-migrations-sqlite:
|
||||
@@ -80,6 +84,8 @@ jobs:
|
||||
run: >-
|
||||
uv run pytest
|
||||
tests/integration/persistence/test_migrations_postgres.py
|
||||
tests/integration/persistence/test_monitoring_postgres.py
|
||||
tests/unit_tests/api/service/test_monitoring_sessions.py::test_postgres_upgrade_rls_and_concurrent_bot_counts
|
||||
tests/integration/persistence/test_pgvector_postgres.py
|
||||
tests/integration/persistence/test_release_migration_postgres.py
|
||||
tests/integration/persistence/test_plugin_identity_migration.py
|
||||
|
||||
+12
@@ -57,3 +57,15 @@ testsdk/
|
||||
|
||||
# Next.js build cache (legacy)
|
||||
web/.next/
|
||||
web/.pnpm-home
|
||||
.tmp
|
||||
Caddyfile
|
||||
|
||||
# fnOS packaging build artifacts (packaging/fnos/build.sh)
|
||||
packaging/fnos/app/langbot/
|
||||
packaging/fnos/app/bin/
|
||||
packaging/fnos/ICON.PNG
|
||||
packaging/fnos/ICON_256.PNG
|
||||
packaging/fnos/app/ui/images/
|
||||
packaging/fnos/app/desktop/images/
|
||||
packaging/fnos/*.fpk
|
||||
|
||||
@@ -93,17 +93,6 @@ docker compose --profile all up -d
|
||||
|
||||
---
|
||||
|
||||
## Live Demo
|
||||
|
||||
**Try it now:** https://demo.langbot.dev/
|
||||
|
||||
- Email: `demo@langbot.app`
|
||||
- Password: `langbot123456`
|
||||
|
||||
_Note: Public demo environment. Do not enter sensitive information._
|
||||
|
||||
---
|
||||
|
||||
## Supported Platforms
|
||||
|
||||
| Platform | Status | Notes |
|
||||
|
||||
@@ -93,16 +93,6 @@ docker compose --profile all up -d
|
||||
|
||||
---
|
||||
|
||||
## 在线演示
|
||||
|
||||
**立即体验:** https://demo.langbot.dev/
|
||||
- 邮箱:`demo@langbot.app`
|
||||
- 密码:`langbot123456`
|
||||
|
||||
*注意:公开演示环境,请不要在其中填入任何敏感信息。*
|
||||
|
||||
---
|
||||
|
||||
## 支持的平台
|
||||
|
||||
| 平台 | 状态 | 备注 |
|
||||
|
||||
@@ -92,16 +92,6 @@ docker compose --profile all up -d
|
||||
|
||||
---
|
||||
|
||||
## Demo en Vivo
|
||||
|
||||
**Pruébelo ahora:** https://demo.langbot.dev/
|
||||
- Correo electrónico: `demo@langbot.app`
|
||||
- Contraseña: `langbot123456`
|
||||
|
||||
*Nota: Entorno de demostración público. No ingrese información confidencial.*
|
||||
|
||||
---
|
||||
|
||||
## Plataformas Soportadas
|
||||
|
||||
| Plataforma | Estado | Notas |
|
||||
|
||||
@@ -92,16 +92,6 @@ docker compose --profile all up -d
|
||||
|
||||
---
|
||||
|
||||
## Démo en Ligne
|
||||
|
||||
**Essayez maintenant :** https://demo.langbot.dev/
|
||||
- Email : `demo@langbot.app`
|
||||
- Mot de passe : `langbot123456`
|
||||
|
||||
*Note : Environnement de démonstration public. Ne saisissez pas d'informations sensibles.*
|
||||
|
||||
---
|
||||
|
||||
## Plateformes Supportées
|
||||
|
||||
| Plateforme | Statut | Notes |
|
||||
|
||||
@@ -92,16 +92,6 @@ docker compose --profile all up -d
|
||||
|
||||
---
|
||||
|
||||
## ライブデモ
|
||||
|
||||
**今すぐ試す:** https://demo.langbot.dev/
|
||||
- メール: `demo@langbot.app`
|
||||
- パスワード: `langbot123456`
|
||||
|
||||
*注意: 公開デモ環境です。機密情報を入力しないでください。*
|
||||
|
||||
---
|
||||
|
||||
## 対応プラットフォーム
|
||||
|
||||
| プラットフォーム | ステータス | 備考 |
|
||||
|
||||
@@ -92,16 +92,6 @@ docker compose --profile all up -d
|
||||
|
||||
---
|
||||
|
||||
## 라이브 데모
|
||||
|
||||
**지금 체험:** https://demo.langbot.dev/
|
||||
- 이메일: `demo@langbot.app`
|
||||
- 비밀번호: `langbot123456`
|
||||
|
||||
*참고: 공개 데모 환경입니다. 민감한 정보를 입력하지 마세요.*
|
||||
|
||||
---
|
||||
|
||||
## 지원 플랫폼
|
||||
|
||||
| 플랫폼 | 상태 | 비고 |
|
||||
|
||||
@@ -92,16 +92,6 @@ docker compose --profile all up -d
|
||||
|
||||
---
|
||||
|
||||
## Демо
|
||||
|
||||
**Попробуйте прямо сейчас:** https://demo.langbot.dev/
|
||||
- Email: `demo@langbot.app`
|
||||
- Пароль: `langbot123456`
|
||||
|
||||
*Примечание: Публичная демо-среда. Не вводите конфиденциальную информацию.*
|
||||
|
||||
---
|
||||
|
||||
## Поддерживаемые платформы
|
||||
|
||||
| Платформа | Статус | Примечания |
|
||||
|
||||
@@ -94,16 +94,6 @@ docker compose --profile all up -d
|
||||
|
||||
---
|
||||
|
||||
## 線上演示
|
||||
|
||||
**立即體驗:** https://demo.langbot.dev/
|
||||
- 信箱:`demo@langbot.app`
|
||||
- 密碼:`langbot123456`
|
||||
|
||||
*注意:公開演示環境,請不要在其中填入任何敏感資訊。*
|
||||
|
||||
---
|
||||
|
||||
## 支援的平台
|
||||
|
||||
| 平台 | 狀態 | 備註 |
|
||||
|
||||
@@ -92,16 +92,6 @@ docker compose --profile all up -d
|
||||
|
||||
---
|
||||
|
||||
## Demo trực tuyến
|
||||
|
||||
**Thử ngay:** https://demo.langbot.dev/
|
||||
- Email: `demo@langbot.app`
|
||||
- Mật khẩu: `langbot123456`
|
||||
|
||||
*Lưu ý: Môi trường demo công khai. Không nhập thông tin nhạy cảm.*
|
||||
|
||||
---
|
||||
|
||||
## Nền tảng được hỗ trợ
|
||||
|
||||
| Nền tảng | Trạng thái | Ghi chú |
|
||||
|
||||
@@ -0,0 +1,65 @@
|
||||
# ChatGPT / Codex subscription
|
||||
|
||||
LangBot's **OpenAI Codex** model provider uses **Sign in with ChatGPT** and the account's Codex entitlement. It is separate from the existing OpenAI API-key provider: subscribing to ChatGPT does not supply an OpenAI Platform API key, and API-key billing is unchanged.
|
||||
|
||||
## Connect an account
|
||||
|
||||
1. Open **Models**, choose **Add Provider**, and select **OpenAI Codex**.
|
||||
2. Enter a provider name and choose **Save and sign in**. This saves the provider before authorization, so an interrupted login can be retried from its settings.
|
||||
3. Open the OpenAI authorization link and enter the one-time code displayed in LangBot. Sign in on OpenAI's site, not in LangBot.
|
||||
4. If OpenAI asks you to enable device-code authorization, enable it in your ChatGPT account's security settings, or contact your workspace administrator.
|
||||
5. Keep the LangBot dialog open until it confirms the connection, then finish the form.
|
||||
6. Use the existing **Scan models** or **Add model** controls, test the model, and select it in a pipeline as usual. Only LLM models are supported by this provider.
|
||||
|
||||
The device-code flow also works when LangBot runs remotely or in Docker: the browser does not need to reach a localhost OAuth callback on the server. Serve the LangBot management panel over HTTPS when accessing it remotely.
|
||||
|
||||
The account's model catalog is authoritative. A model listed elsewhere or entered manually is not a guarantee that this account has access. Scan errors are reported rather than replaced with a fabricated available-model list.
|
||||
|
||||
## Reconnect and disconnect
|
||||
|
||||
Open the provider's existing settings to sign in again or disconnect. LangBot refreshes expiring access tokens automatically. A revoked or invalid refresh grant requires another sign-in; transient network failures are not proof that the grant was revoked.
|
||||
|
||||
**Disconnect** removes this provider's locally stored authorization. It does not log the account out of other applications or revoke the account globally. Canceling a pending sign-in is separate from disconnecting an existing account. Removing a provider also removes its authorization; the normal rule that models must be removed first still applies.
|
||||
|
||||
A saved provider can remain disconnected. Scanning or invoking it then returns a sign-in-required error; LangBot does not silently switch to paid API-key billing.
|
||||
|
||||
## Usage and deployment boundary
|
||||
|
||||
Calls consume the connected account's included Codex usage and remain subject to OpenAI's plan limits, model availability, workspace policies, and terms. Token counts recorded by LangBot are request usage, not a measurement of remaining subscription quota or an OpenAI invoice.
|
||||
|
||||
Use this integration for your own authorized account and trusted workflows. Third-party sign-in support is not permission to pool accounts, resell subscription quota, or redistribute one subscription as a shared API service. For a public or commercial multi-user service, use the appropriate OpenAI API or separately authorized enterprise arrangement. The provider remains a Workspace resource in LangBot: consider who can invoke its models before connecting a personal account.
|
||||
|
||||
## Credential handling and API surface
|
||||
|
||||
- OAuth credentials are stored server-side separately from provider API keys. Provider and model reads do not supply OAuth access, refresh, or ID tokens.
|
||||
- Authorization uses a fixed OpenAI origin. The Codex provider does not accept a custom base URL or manually supplied API keys.
|
||||
- Authentication controls require an authenticated LangBot browser user with `provider_secret.manage` in the selected Workspace. Pending attempts are scoped to the Workspace, provider, and initiating user.
|
||||
- Browser storage must not contain OAuth tokens. Treat the server database and its backups as sensitive application data.
|
||||
- MCP and LangBot API keys do not expose the browser-only OAuth controls. Agents may inspect configured providers and models with the existing tools, but a human connects the subscription in the management panel.
|
||||
|
||||
The provider-scoped authentication routes are under `/api/v1/provider/providers/{uuid}/codex`:
|
||||
|
||||
| Method | Suffix | Purpose |
|
||||
| --- | --- | --- |
|
||||
| GET | `/status` | Read local connection state without returning credentials |
|
||||
| POST | `/device` | Start device authorization |
|
||||
| POST | `/device/poll` | Poll the initiating user's authorization attempt |
|
||||
| DELETE | `/device/{authorization_id}` | Cancel only that pending attempt |
|
||||
| DELETE | `/auth` | Remove local authorization |
|
||||
|
||||
Use the returned polling interval and expiration time. An expired attempt must be restarted. These routes are not a general-purpose subscription-to-API gateway.
|
||||
|
||||
## References
|
||||
|
||||
- [OpenAI Codex authentication](https://developers.openai.com/codex/auth): ChatGPT versus API-key access and device-code login.
|
||||
- [Hermes Agent providers](https://hermes-agent.nousresearch.com/docs/integrations/providers/): subscription device authentication and refresh recovery.
|
||||
- [OpenClaw OpenAI provider](https://docs.openclaw.ai/providers/openai): subscription and API-key route distinctions.
|
||||
- [New API](https://github.com/QuantumNous/new-api): reference for Codex protocol compatibility; its gateway/account-pooling product model is not adopted here.
|
||||
|
||||
## 中文快速说明
|
||||
|
||||
在「模型」中添加提供商,选择 **OpenAI Codex**,填写名称并点击「保存并登录」。打开 OpenAI 授权页面,输入 LangBot 显示的一次性验证码,完成授权后回到原对话框。随后照常扫描或添加模型、测试模型,并在流水线中选择它。
|
||||
|
||||
无需填写 API Key,也无需为远程服务器配置 localhost 回调。登录中断后可以从该提供商的设置中重试;断开连接只删除 LangBot 中保存的授权。调用消耗所登录账号的 Codex 额度,受账号实际权限和 OpenAI 限制约束,不会自动转用按量付费的 OpenAI API。
|
||||
|
||||
此功能用于自己的授权账号及可信工作流,不应将个人订阅作为面向多个用户转售或共享的 API 服务。提供商仍是 LangBot 工作空间内的资源,连接个人账号前请确认模型的使用范围。
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 73 KiB |
@@ -0,0 +1,145 @@
|
||||
|
||||
LangBot 用户许可协议
|
||||
User License Agreement
|
||||
|
||||
飞牛 fnOS 平台发行版 | 最后更新:2026 年 8 月
|
||||
|
||||
感谢您使用 LangBot!本协议是您(用户)与 LangBot 开源项目(以下简称「LangBot」「我们」)之间,就您在飞牛 fnOS 平台(含飞牛 NAS 设备、飞牛 OS 及其应用中心,以下简称「平台」)上安装、运行、使用 LangBot 应用所订立的合法协议。
|
||||
|
||||
您一旦在飞牛应用中心勾选「我接受许可协议的条款」并继续安装、或以其他方式运行 LangBot,即表示您已阅读、理解并同意本协议的全部内容。如您不同意本协议,请不要安装或使用本软件。
|
||||
|
||||
---
|
||||
|
||||
一、软件性质
|
||||
|
||||
1.1 LangBot 是一款基于 LLM 的多平台智能对话机器人开源软件。主程序源代码按照 Apache License, Version 2.0 公开,您可在遵守开源协议的前提下自由使用、修改与再分发。
|
||||
|
||||
1.2 本发行版系 LangBot 社区为飞牛 fnOS 平台打包构建的自托管移植版本,与飞牛官方、飞牛硬件厂商不存在从属或关联关系。飞牛应用中心提供的分发渠道不构成对软件功能、可用性的任何担保。
|
||||
|
||||
---
|
||||
|
||||
二、使用授权
|
||||
|
||||
2.1 授予您一份有限的、非排他的、不可转让的个人使用许可:您可在一台或多台您合法拥有或管理的 fnOS 设备上安装、运行本软件,用于个人、家庭或组织内部合法用途。
|
||||
|
||||
2.2 您不得:
|
||||
(a) 将本软件用于违反中国大陆地区法律法规或您所在司法管辖区法律的用途;
|
||||
(b) 对机器人账号进行骚扰、诈骗、批量营销、发布违法违规内容等滥用行为;
|
||||
(c) 逆向工程、反编译本软件所包含的第三方二进制(uv 等),但适用法律明确允许或对应开源许可证另作规定的除外;
|
||||
(d) 试图干扰、过载或损害任何由本软件对外提供的服务或其基础设施。
|
||||
|
||||
---
|
||||
|
||||
三、服务可用性与「现状」提供
|
||||
|
||||
3.1 我们努力提供稳定可靠的软件,但**不保证运行无中断或无错误**。软件可能因维护、升级、不可抗力或其他原因暂时不可用。
|
||||
|
||||
3.2 本软件按「现状」「按可用」提供。我们不作任何明示或默示担保,包括但不限于对适销性、特定用途适用性与非侵权性的默示担保。我们不保证:
|
||||
(a) 软件将满足您的具体需求;
|
||||
(b) 运行不间断、及时、安全或无错误;
|
||||
(c) 使用获得的结果准确或可靠;
|
||||
(d) 任何错误都会被修复。
|
||||
|
||||
---
|
||||
|
||||
四、责任限制
|
||||
|
||||
在适用法律允许的最大范围内:
|
||||
(a) 我们不对任何**间接、附带、特殊、惩罚性或后果性损失**承担责任,包括但不限于利润损失、数据丢失、商誉损失、业务中断或其他无形损失,无论是否已被告知该等损害发生的可能性;
|
||||
(b) 无论基于合同、侵权、严格责任或其他任何理论,我们就本软件所引起的所有索赔,向您承担的**累计赔偿总额**不超过 15 美元(或等值当地货币)。
|
||||
|
||||
---
|
||||
|
||||
五、用户责任
|
||||
|
||||
5.1 您对通过本软件发送的所有内容和消息(包括您所配置并运行的机器人发出的消息)承担全部责任。
|
||||
|
||||
5.2 您必须遵守所有适用的法律法规,包括但不限于数据保护、隐私、消费者保护和反垃圾邮件法律。
|
||||
|
||||
5.3 您有责任保管好自己的账号凭据和各类 API Key,并**自行对重要数据进行备份**。我们对因服务中断、账号终止或系统故障等任何原因造成的数据丢失不承担责任。
|
||||
|
||||
5.4 您不得将本软件用于任何非法活动、发送垃圾信息、骚扰他人或侵害他人合法权益。
|
||||
|
||||
5.5 您使用机器人接入任何即时通讯平台(QQ、微信、飞书、钉钉、Telegram 等)前,须自行确认已获得该平台授权并遵守其开发者协议与社区规范;因违规接入导致的账号封禁、平台处罚,由您自行承担。
|
||||
|
||||
---
|
||||
|
||||
六、数据与隐私
|
||||
|
||||
6.1 本自托管版本默认情况下,所有配置、对话记录、知识库与插件数据均保存在您所安装的 fnOS 设备本地共享目录(langbot/data)中,不会被自动上传至除您显式配置的模型/服务提供商以外的任何第三方。您对自己的数据及备份负责。
|
||||
|
||||
6.2 LangBot 默认启用最少量的匿名遥测,用于帮助改进产品。详细政策见官方文档:
|
||||
https://docs.langbot.app/zh/insight/data-collection-policy
|
||||
|
||||
6.3 启用遥测时仅可能发送:查询事件(适配器类型、运行器类型、模型名称、处理耗时、版本号、匿名工作区 UUID、插件/功能使用计数、不含用户内容的错误追踪)、每日一次的工作区心跳(部署概况、资源对象数量)、完全自愿的问卷回答。
|
||||
|
||||
6.4 我们绝对不收集:消息内容、用户名/手机号/平台账号 ID、API 密钥或凭据、IP 地址、文件或媒体内容。
|
||||
|
||||
6.5 关闭方式:进入 LangBot Web 管理界面 → 设置 → Space 遥测,关闭开关;或在配置文件 data/config.yaml 中设置 `space.disable_telemetry: true`。关闭后所有功能照常运行。
|
||||
|
||||
---
|
||||
|
||||
七、不可抗力
|
||||
|
||||
因不可抗力事件导致的履约失败或延迟,我们不承担责任。不可抗力包括但不限于:自然灾害(地震、洪水、飓风等)、战争、恐怖主义或内乱、政府行为或法规、网络攻击(DDoS、勒索软件等)、第三方服务故障(AI 模型提供商、即时通讯平台、飞牛平台运行时等)、电力故障或互联网连接中断、流行病或公共卫生紧急事件。
|
||||
|
||||
---
|
||||
|
||||
八、第三方服务
|
||||
|
||||
本软件可能依赖或集成第三方服务,包括但不限于:即时通讯平台(Telegram、Discord、微信、QQ、Slack 等)、AI 模型提供商(OpenAI、Anthropic、Google、深度求索等)、飞牛平台的应用中心运行时与依赖应用(如 Node.js)。我们不对第三方服务的可用性、准确性、可靠性或安全性负责;使用第三方服务须受其各自条款约束,第三方服务的变更可能不经通知即影响本软件功能。
|
||||
|
||||
---
|
||||
|
||||
九、软件修改与终止
|
||||
|
||||
9.1 我们保留随时修改、暂停或终止软件或其中任何部分的权利,无论是否事先通知。
|
||||
9.2 我们保留随时修改本协议的权利。对重大变更我们会尽力在项目主页公告,继续使用软件即视为接受修订后的协议。
|
||||
9.3 您可以在飞牛应用中心卸载本软件。卸载时向导会询问是否保留数据,您可自主选择。终止后您继续使用软件的权利立即终止,我们无义务保留您的数据。
|
||||
|
||||
---
|
||||
|
||||
十、赔偿
|
||||
|
||||
您同意赔偿、抗辩并使 LangBot 团队及其关联贡献者、管理人员、代理人、员工免受任何及所有因以下事项引起或与之相关的索赔、损失、损害、责任、成本和费用(包括合理的律师费):
|
||||
(a) 您对软件的使用;
|
||||
(b) 您违反本协议;
|
||||
(c) 您违反任何适用的法律或法规;
|
||||
(d) 您侵犯任何第三方权利;
|
||||
(e) 您或您所配置的机器人通过本软件传输的内容。
|
||||
|
||||
---
|
||||
|
||||
十一、知识产权
|
||||
|
||||
11.1 本软件及其设计、代码、文档和品牌归 LangBot 项目所有并受知识产权法保护。
|
||||
11.2 使用本软件并不授予您对软件的任何所有权。
|
||||
11.3 您保留通过本软件创建和传输内容的所有权。
|
||||
|
||||
---
|
||||
|
||||
十二、争议解决
|
||||
|
||||
因本协议引起或与之相关的任何争议,应首先通过友好协商解决。协商在 30 日内未达成一致的,任何一方均可依适用法律规定向有管辖权的法院提起诉讼。
|
||||
|
||||
---
|
||||
|
||||
十三、可分割性
|
||||
|
||||
如本协议的任何条款被认定为不可执行或无效,该条款应在最小必要范围内予以限制或剔除,其余条款继续完全有效。
|
||||
|
||||
---
|
||||
|
||||
十四、完整协议
|
||||
|
||||
本协议连同我们的数据收集政策(https://docs.langbot.app/zh/insight/data-collection-policy)构成您与我们之间关于本软件的完整协议,并取代所有先前的协议与谅解。
|
||||
|
||||
---
|
||||
|
||||
附录:开源许可证声明
|
||||
LangBot 主程序代码依照 Apache License 2.0 发布。详细条款见:
|
||||
https://github.com/langbot-app/LangBot/blob/master/LICENSE
|
||||
或 LangBot 源码包内的 LICENSE 文件。
|
||||
|
||||
数据收集政策:https://docs.langbot.app/zh/insight/data-collection-policy
|
||||
|
||||
@@ -0,0 +1,78 @@
|
||||
# LangBot fnOS Packaging
|
||||
|
||||
This directory packages LangBot as a `.fpk` app for the fnOS App Store. It is a native deployment: no Docker involved — uv creates a Python virtual environment directly on the NAS, and Node.js v22 from the fnOS App Store provides the Box sandbox and npx MCP capabilities.
|
||||
|
||||
## Directory Structure
|
||||
|
||||
```
|
||||
packaging/fnos/
|
||||
├── manifest # App metadata (appname/version/port/dependency declarations)
|
||||
├── build.sh # One-shot build script (shared by local and CI)
|
||||
├── LICENSE
|
||||
├── config/
|
||||
│ ├── privilege # Privilege config (run-as: root)
|
||||
│ └── resource # Persistent data share declaration (langbot/data)
|
||||
├── cmd/ # Lifecycle scripts (fnOS invokes them with TRIM_* env vars)
|
||||
│ ├── main # Service start/stop manager (start/stop/status, owns PID/log)
|
||||
│ ├── install_init # Pre-install hook
|
||||
│ ├── install_callback # Post-install hook: create venv, uv sync deps, seed config.yaml port
|
||||
│ ├── upgrade_init # Pre-upgrade hook
|
||||
│ ├── upgrade_callback # Post-upgrade hook
|
||||
│ ├── uninstall_init # Pre-uninstall hook
|
||||
│ ├── uninstall_callback # Post-uninstall hook (keeps data per wizard choice)
|
||||
│ ├── config_init # Pre-config-change hook
|
||||
│ └── config_callback # Post-config-change hook (apply new port etc.)
|
||||
├── wizard/ # Install wizards (JSON forms; values passed as wizard_* env vars)
|
||||
│ ├── install # On install: Node version, web port, deployment-time notice
|
||||
│ ├── upgrade # On upgrade: Node version confirmation
|
||||
│ └── uninstall # On uninstall: whether to keep data
|
||||
├── app/
|
||||
│ ├── ui/config # Desktop entry declaration (${wizard_port} placeholder, substituted by fnOS at install)
|
||||
│ ├── desktop/langbot.main.url
|
||||
│ ├── langbot/ # [generated] repo source synced via rsync (includes web/dist)
|
||||
│ └── bin/ # [generated] offline uv binaries (x86_64/aarch64)
|
||||
├── ICON.PNG / ICON_256.PNG # [generated] derived from res/logo-blue.png
|
||||
└── langbot.fpk # [generated] final artifact
|
||||
```
|
||||
|
||||
Paths marked `[generated]` are produced by `build.sh`, ignored via `.gitignore`; everything else is a git-tracked source file.
|
||||
|
||||
## Building
|
||||
|
||||
### Locally
|
||||
|
||||
```bash
|
||||
bash packaging/fnos/build.sh
|
||||
```
|
||||
|
||||
Dependencies: python3 + Pillow, node + npm (or pnpm), fnpack (official fnOS packaging CLI, download from https://developer.fnnas.com/docs/cli/fnpack).
|
||||
|
||||
The version comes from `version=` maintained in the manifest; it can also be injected: `FPK_VERSION=4.10.10-1 bash packaging/fnos/build.sh`.
|
||||
|
||||
### CI
|
||||
|
||||
[`.github/workflows/build-fnos-fpk.yaml`](../../.github/workflows/build-fnos-fpk.yaml) triggers automatically on Release publication and uploads `langbot-<version>-fnos.fpk` to the Release; it can also be triggered manually via workflow_dispatch.
|
||||
|
||||
Version sources (consistent with the other release workflows):
|
||||
|
||||
| Trigger | Version source |
|
||||
|---|---|
|
||||
| Release/tag auto build | Tag name (`v4.10.10-1` → in-package `4.10.10-1`; build.sh strips the `v` prefix on injection) |
|
||||
| Manual workflow_dispatch / local build | Version maintained in manifest |
|
||||
|
||||
## Final Artifact
|
||||
|
||||
`langbot.fpk` (gzip + tar archive), containing:
|
||||
|
||||
- `manifest` — realigned and appended with a `checksum` field by fnpack
|
||||
- `app.tgz` — app payload (source, web/dist, uv binaries, entry configs)
|
||||
- `cmd/`, `config/`, `wizard/` — lifecycle scripts and wizards
|
||||
- `ICON.PNG`, `ICON_256.PNG`, `LICENSE`
|
||||
|
||||
The first startup after installation takes about 5-10 minutes to finish dependency deployment (uv venv + sync); after that the web admin UI is reachable via the desktop icon or `http://<NAS-IP>:<port>` (default port 5300).
|
||||
|
||||
## References
|
||||
|
||||
- fnOS developer docs: https://developer.fnnas.com/
|
||||
- fnOS app wizard: https://developer.fnnas.com/docs/core-concepts/wizard/
|
||||
- fnpack CLI: https://developer.fnnas.com/docs/cli/fnpack
|
||||
@@ -0,0 +1,9 @@
|
||||
{
|
||||
"title": "LangBot",
|
||||
"icon": "images/icon-256.png",
|
||||
"type": "url",
|
||||
"protocol": "http",
|
||||
"port": "${wizard_port}",
|
||||
"url": "/",
|
||||
"allUsers": true
|
||||
}
|
||||
@@ -0,0 +1,13 @@
|
||||
{
|
||||
".url": {
|
||||
"langbot.main": {
|
||||
"title": "LangBot",
|
||||
"icon": "images/icon-{0}.png",
|
||||
"type": "url",
|
||||
"protocol": "http",
|
||||
"port": "${wizard_port}",
|
||||
"url": "/",
|
||||
"allUsers": true
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,180 @@
|
||||
#!/usr/bin/env bash
|
||||
# packaging/fnos/build.sh - Build LangBot fnOS FPK package (in-repo version)
|
||||
# 在 LangBot 仓库内直接打包飞牛 fnOS 应用
|
||||
# Usage:
|
||||
# bash packaging/fnos/build.sh # 版本取 manifest 中 version=
|
||||
# FPK_VERSION=4.10.9 bash packaging/fnos/build.sh # 注入版本(CI 用 release tag)
|
||||
# Dependencies: python3+Pillow, node+npm, fnpack
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
SRC_ROOT="$(cd "${SCRIPT_DIR}/../.." && pwd)" # LangBot 仓库根
|
||||
FPK_DIR="${SCRIPT_DIR}"
|
||||
|
||||
echo "==> LangBot fnOS FPK builder (in-repo)"
|
||||
echo " Source root: ${SRC_ROOT}"
|
||||
echo " FPK dir: ${FPK_DIR}"
|
||||
|
||||
# --- 0. Inject version (release tag) ---
|
||||
if [ -n "${FPK_VERSION:-}" ]; then
|
||||
sed -i "s/^version=.*/version=${FPK_VERSION#v}/" "${FPK_DIR}/manifest"
|
||||
else
|
||||
# 无注入版本时自动跟进仓库主版本(pyproject.toml)
|
||||
PY_VER=$(grep -m1 '^version = ' "${SRC_ROOT}/pyproject.toml" | cut -d'"' -f2)
|
||||
if [ -n "${PY_VER}" ]; then
|
||||
sed -i "s/^version=.*/version=${PY_VER}/" "${FPK_DIR}/manifest"
|
||||
fi
|
||||
fi
|
||||
MANIFEST_VER=$(grep '^version=' "${FPK_DIR}/manifest" | cut -d= -f2)
|
||||
echo " FPK version: ${MANIFEST_VER}"
|
||||
|
||||
# --- 1. Build frontend ---
|
||||
echo "[1/5] Building frontend (web/dist)..."
|
||||
cd "${SRC_ROOT}/web"
|
||||
if command -v pnpm >/dev/null 2>&1; then
|
||||
pnpm install --frozen-lockfile 2>/dev/null || pnpm install
|
||||
pnpm build
|
||||
else
|
||||
npm install
|
||||
npx vite build
|
||||
fi
|
||||
[ -d dist ] || { echo "ERROR: web/dist missing" >&2; exit 1; }
|
||||
echo " Frontend built"
|
||||
|
||||
# --- 2. Sync source into packaging/fnos/app/langbot/ ---
|
||||
echo "[2/5] Syncing source to app/langbot/..."
|
||||
rm -rf "${FPK_DIR}/app/langbot"
|
||||
mkdir -p "${FPK_DIR}/app/langbot"
|
||||
cd "${SRC_ROOT}"
|
||||
rsync -a \
|
||||
--exclude='.git' \
|
||||
--exclude='.venv' \
|
||||
--exclude='__pycache__' \
|
||||
--exclude='*.pyc' \
|
||||
--exclude='web/node_modules' \
|
||||
--exclude='web/.vite' \
|
||||
--exclude='tests' \
|
||||
--exclude='packaging' \
|
||||
--exclude='.pytest_cache' \
|
||||
--exclude='.mypy_cache' \
|
||||
--exclude='.ruff_cache' \
|
||||
--exclude='data' \
|
||||
--exclude='*.log' \
|
||||
--exclude='.dockerignore' \
|
||||
--exclude='Dockerfile' \
|
||||
--exclude='docker/' \
|
||||
--exclude='kubernetes.yaml' \
|
||||
--exclude='.github/' \
|
||||
--exclude='docs/' \
|
||||
--exclude='examples/' \
|
||||
--exclude='res/' \
|
||||
./ "${FPK_DIR}/app/langbot/"
|
||||
|
||||
[ -d "${FPK_DIR}/app/langbot/web/dist" ] || { echo "ERROR: web/dist missing after rsync!" >&2; exit 1; }
|
||||
echo " Source synced ($(du -sh "${FPK_DIR}/app/langbot" | cut -f1))"
|
||||
|
||||
# --- 2.5 Download bundled uv binaries (offline install on NAS) ---
|
||||
echo "[2.5/5] Downloading bundled uv binaries..."
|
||||
UV_VERSION="0.12.9"
|
||||
mkdir -p "${FPK_DIR}/app/bin"
|
||||
for arch in x86_64 aarch64; do
|
||||
out="${FPK_DIR}/app/bin/uv-${arch}"
|
||||
if [ -x "${out}" ]; then
|
||||
echo " uv-${arch} already present, skip"
|
||||
continue
|
||||
fi
|
||||
tmp="$(mktemp -d)"
|
||||
if curl -sSL -o "${tmp}/uv.tar.gz" \
|
||||
"https://github.com/astral-sh/uv/releases/download/${UV_VERSION}/uv-${arch}-unknown-linux-gnu.tar.gz" \
|
||||
&& tar xzf "${tmp}/uv.tar.gz" -C "${tmp}" \
|
||||
&& cp "${tmp}/uv-${arch}-unknown-linux-gnu/uv" "${out}"; then
|
||||
chmod +x "${out}"
|
||||
echo " uv-${arch} downloaded (${UV_VERSION})"
|
||||
else
|
||||
echo " WARNING: failed to download uv for ${arch}, install will fall back to online install" >&2
|
||||
fi
|
||||
rm -rf "${tmp}"
|
||||
done
|
||||
|
||||
# --- 3. Regenerate icons from res/logo-blue.png ---
|
||||
echo "[3/5] Generating icons from res/logo-blue.png..."
|
||||
export LOGO_SRC="${SRC_ROOT}/res/logo-blue.png"
|
||||
export OUT_DIR="${FPK_DIR}"
|
||||
python3 << 'PYEOF'
|
||||
from PIL import Image
|
||||
import os, sys
|
||||
|
||||
src = os.environ.get("LOGO_SRC")
|
||||
out_dir = os.environ.get("OUT_DIR")
|
||||
if not src or not out_dir:
|
||||
print("ERROR: LOGO_SRC or OUT_DIR not set", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
if not os.path.isfile(src):
|
||||
print(f"ERROR: logo source not found: {src}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
img = Image.open(src).convert("RGBA")
|
||||
|
||||
for size, name in [(64, "ICON.PNG"), (256, "ICON_256.PNG")]:
|
||||
img.resize((size, size), Image.LANCZOS).save(os.path.join(out_dir, name))
|
||||
|
||||
ui_dir = os.path.join(out_dir, "app/ui/images")
|
||||
os.makedirs(ui_dir, exist_ok=True)
|
||||
for size in [64, 256]:
|
||||
img.resize((size, size), Image.LANCZOS).save(os.path.join(ui_dir, f"icon-{size}.png"))
|
||||
|
||||
desktop_dir = os.path.join(out_dir, "app/desktop/images")
|
||||
os.makedirs(desktop_dir, exist_ok=True)
|
||||
for size in [64, 256]:
|
||||
img.resize((size, size), Image.LANCZOS).save(os.path.join(desktop_dir, f"icon-{size}.png"))
|
||||
|
||||
print(" Icons generated")
|
||||
PYEOF
|
||||
|
||||
# --- 4. Validate structure ---
|
||||
echo "[4/5] Validating FPK structure..."
|
||||
ERRORS=0
|
||||
[ -f "${FPK_DIR}/manifest" ] || { echo " MISSING: manifest"; ERRORS=$((ERRORS+1)); }
|
||||
[ -f "${FPK_DIR}/config/privilege" ] || { echo " MISSING: config/privilege"; ERRORS=$((ERRORS+1)); }
|
||||
[ -f "${FPK_DIR}/config/resource" ] || { echo " MISSING: config/resource"; ERRORS=$((ERRORS+1)); }
|
||||
[ -f "${FPK_DIR}/ICON.PNG" ] || { echo " MISSING: ICON.PNG"; ERRORS=$((ERRORS+1)); }
|
||||
[ -f "${FPK_DIR}/ICON_256.PNG" ] || { echo " MISSING: ICON_256.PNG"; ERRORS=$((ERRORS+1)); }
|
||||
[ -f "${FPK_DIR}/app/ui/config" ] || { echo " MISSING: app/ui/config"; ERRORS=$((ERRORS+1)); }
|
||||
[ -f "${FPK_DIR}/app/ui/images/icon-64.png" ] || { echo " MISSING: app/ui/images/icon-64.png"; ERRORS=$((ERRORS+1)); }
|
||||
[ -f "${FPK_DIR}/app/ui/images/icon-256.png" ] || { echo " MISSING: app/ui/images/icon-256.png"; ERRORS=$((ERRORS+1)); }
|
||||
[ -f "${FPK_DIR}/app/desktop/langbot.main.url" ] || { echo " MISSING: app/desktop/langbot.main.url"; ERRORS=$((ERRORS+1)); }
|
||||
[ -f "${FPK_DIR}/app/desktop/images/icon-64.png" ] || { echo " MISSING: app/desktop/images/icon-64.png"; ERRORS=$((ERRORS+1)); }
|
||||
[ -f "${FPK_DIR}/app/desktop/images/icon-256.png" ] || { echo " MISSING: app/desktop/images/icon-256.png"; ERRORS=$((ERRORS+1)); }
|
||||
[ -d "${FPK_DIR}/cmd" ] || { echo " MISSING: cmd/"; ERRORS=$((ERRORS+1)); }
|
||||
[ -d "${FPK_DIR}/wizard" ] || { echo " MISSING: wizard/"; ERRORS=$((ERRORS+1)); }
|
||||
|
||||
for script in "${FPK_DIR}/cmd/"*; do
|
||||
[ -x "${script}" ] || { echo " NOT EXECUTABLE: cmd/$(basename "$script")"; ERRORS=$((ERRORS+1)); }
|
||||
done
|
||||
|
||||
if [ "${ERRORS}" -gt 0 ]; then
|
||||
echo "FAILED: ${ERRORS} validation errors" >&2
|
||||
exit 1
|
||||
fi
|
||||
echo " Structure OK"
|
||||
|
||||
# --- 5. Build FPK ---
|
||||
echo "[5/5] Building .fpk..."
|
||||
if ! command -v fnpack >/dev/null 2>&1; then
|
||||
echo "ERROR: fnpack not found in PATH." >&2
|
||||
echo " Download from https://developer.fnnas.com/docs/cli/fnpack" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
cd "${FPK_DIR}"
|
||||
fnpack build
|
||||
FPK_FILE=$(ls -t *.fpk 2>/dev/null | head -1)
|
||||
if [ -n "${FPK_FILE}" ]; then
|
||||
echo ""
|
||||
echo "==> Done! FPK: ${FPK_DIR}/${FPK_FILE}"
|
||||
echo " Size: $(du -sh "${FPK_FILE}" | cut -f1)"
|
||||
else
|
||||
echo "WARNING: fnpack finished but no .fpk found in ${FPK_DIR}" >&2
|
||||
exit 1
|
||||
fi
|
||||
Executable
+6
@@ -0,0 +1,6 @@
|
||||
#!/bin/bash
|
||||
# cmd/config_callback - post-config hook
|
||||
# Applies wizard-provided settings (e.g. Node.js version) that cmd/main reads.
|
||||
# Nothing to persist currently; cmd/main reads wizard_node_version at runtime.
|
||||
|
||||
exit 0
|
||||
Executable
+4
@@ -0,0 +1,4 @@
|
||||
#!/bin/bash
|
||||
# cmd/config_init - pre-config hook
|
||||
|
||||
exit 0
|
||||
Executable
+149
@@ -0,0 +1,149 @@
|
||||
#!/bin/bash
|
||||
# cmd/install_callback - post-install hook
|
||||
# Prefers bundled uv binary, creates Python venv, syncs deps, verifies dist.
|
||||
# Also validates the user-selected Node.js version is actually installed.
|
||||
|
||||
APP_DIR="${TRIM_APPDEST}/langbot"
|
||||
|
||||
# --- Resolve data directory ---
|
||||
# LangBot loads config CWD-relative (data/config.yaml); cmd/main replaces
|
||||
# APP_DIR/data with a symlink to this persistent dir on every start.
|
||||
DATA_DIR="${TRIM_DATA_SHARE_PATHS%%:*}"
|
||||
if [ -z "${DATA_DIR}" ]; then
|
||||
DATA_DIR="${TRIM_PKGVAR}/data"
|
||||
fi
|
||||
|
||||
cd "${APP_DIR}" || {
|
||||
echo "App directory missing after install" > "${TRIM_TEMP_LOGFILE}"
|
||||
exit 1
|
||||
}
|
||||
|
||||
# --- Ensure data directory exists ---
|
||||
mkdir -p "${DATA_DIR}/plugins" "${DATA_DIR}/box" "${DATA_DIR}/logs" 2>/dev/null || true
|
||||
|
||||
# --- Pre-seed config.yaml with the user-selected web port ---
|
||||
# Data root points at the persistent share (LANGBOT_DATA_ROOT is exported by
|
||||
# cmd/main at start), so this config survives app upgrades.
|
||||
# LangBot copies templates/config.yaml there on first boot only if missing,
|
||||
# so we seed it ourselves with the chosen port.
|
||||
PORT="${wizard_port:-5300}"
|
||||
case "${PORT}" in
|
||||
''|*[!0-9]*) PORT="5300" ;;
|
||||
esac
|
||||
TEMPLATE_FILE="${APP_DIR}/src/langbot/templates/config.yaml"
|
||||
|
||||
_patch_config() {
|
||||
local cfg_dir="$1"
|
||||
local cfg_file="${cfg_dir}/config.yaml"
|
||||
mkdir -p "${cfg_dir}" 2>/dev/null || true
|
||||
if [ ! -f "${cfg_file}" ] && [ -f "${TEMPLATE_FILE}" ]; then
|
||||
cp "${TEMPLATE_FILE}" "${cfg_file}"
|
||||
fi
|
||||
if [ -f "${cfg_file}" ]; then
|
||||
sed -i -E "/^api:/,/^[a-z_]+:/ s/^([[:space:]]*port:).*/\1 ${PORT}/" "${cfg_file}"
|
||||
sed -i "s#webhook_prefix: 'http://127\.0\.0\.1:[0-9]*'#webhook_prefix: 'http://127.0.0.1:${PORT}'#" "${cfg_file}"
|
||||
fi
|
||||
}
|
||||
|
||||
# Seed the persistent dir AND the CWD-relative APP_DIR/data (cmd/main merges
|
||||
# the latter into the persistent dir via symlink on first start, so the port
|
||||
# survives regardless of which path ends up being read).
|
||||
_patch_config "${DATA_DIR}"
|
||||
if [ "${APP_DIR}/data" != "${DATA_DIR}" ]; then
|
||||
_patch_config "${APP_DIR}/data"
|
||||
fi
|
||||
|
||||
# --- Fallback patch for desktop entry port ---
|
||||
# fnOS natively substitutes ${wizard_port} in ui/config at install time.
|
||||
# This only kicks in if the placeholder somehow survived (e.g. CLI install).
|
||||
UI_CONFIG="${TRIM_APPDEST}/ui/config"
|
||||
if [ -f "${UI_CONFIG}" ] && grep -q 'wizard_port\|{port}' "${UI_CONFIG}"; then
|
||||
sed -i "s/\${wizard_port}/${PORT}/g; s/{port}/${PORT}/g" "${UI_CONFIG}"
|
||||
fi
|
||||
|
||||
# --- Validate user-selected Node.js version is installed ---
|
||||
NODE_VERSION="${wizard_node_version:-22}"
|
||||
if [ ! -d "/var/apps/nodejs_v${NODE_VERSION}" ]; then
|
||||
echo "Node.js v${NODE_VERSION} 未安装:请先在应用中心安装 nodejs_v${NODE_VERSION},再重新安装本应用。" > "${TRIM_TEMP_LOGFILE}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# --- Python check ---
|
||||
PYTHON_BIN="python3"
|
||||
if ! command -v "${PYTHON_BIN}" >/dev/null 2>&1; then
|
||||
PYTHON_BIN="python"
|
||||
fi
|
||||
if ! command -v "${PYTHON_BIN}" >/dev/null 2>&1; then
|
||||
echo "Python not found on this system" > "${TRIM_TEMP_LOGFILE}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
PY_VER=$("${PYTHON_BIN}" -c 'import sys; print(f"{sys.version_info.major}.{sys.version_info.minor}")' 2>/dev/null)
|
||||
if [ -z "${PY_VER}" ]; then
|
||||
echo "Python 3.11+ required but not found on this system" > "${TRIM_TEMP_LOGFILE}"
|
||||
exit 1
|
||||
fi
|
||||
PY_MAJOR=$(echo "${PY_VER}" | cut -d. -f1)
|
||||
PY_MINOR=$(echo "${PY_VER}" | cut -d. -f2)
|
||||
if [ "${PY_MAJOR}" -lt 3 ] || { [ "${PY_MAJOR}" -eq 3 ] && [ "${PY_MINOR}" -lt 11 ]; }; then
|
||||
echo "Python 3.11+ required, found ${PY_VER}" > "${TRIM_TEMP_LOGFILE}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# --- Resolve uv: bundled binary first, then online fallbacks ---
|
||||
UV_BIN=""
|
||||
ARCH=$(uname -m)
|
||||
case "${ARCH}" in
|
||||
x86_64) BUNDLED_UV="${TRIM_APPDEST}/bin/uv-x86_64" ;;
|
||||
aarch64) BUNDLED_UV="${TRIM_APPDEST}/bin/uv-aarch64" ;;
|
||||
*) BUNDLED_UV="" ;;
|
||||
esac
|
||||
|
||||
if [ -n "${BUNDLED_UV}" ] && [ -x "${BUNDLED_UV}" ]; then
|
||||
mkdir -p "${TRIM_PKGVAR}/bin"
|
||||
cp "${BUNDLED_UV}" "${TRIM_PKGVAR}/bin/uv" && chmod +x "${TRIM_PKGVAR}/bin/uv"
|
||||
UV_BIN="${TRIM_PKGVAR}/bin/uv"
|
||||
fi
|
||||
|
||||
if [ -z "${UV_BIN}" ] && command -v uv >/dev/null 2>&1; then
|
||||
UV_BIN="uv"
|
||||
fi
|
||||
|
||||
if [ -z "${UV_BIN}" ]; then
|
||||
"${PYTHON_BIN}" -m pip install --user --no-cache-dir uv 2>/dev/null || \
|
||||
"${PYTHON_BIN}" -m pip install --no-cache-dir uv 2>/dev/null || \
|
||||
curl -LsSf https://astral.sh/uv/install.sh | sh 2>/dev/null || true
|
||||
export PATH="${HOME}/.local/bin:${PATH}"
|
||||
if command -v uv >/dev/null 2>&1; then
|
||||
UV_BIN="uv"
|
||||
elif [ -x "${HOME}/.local/bin/uv" ]; then
|
||||
UV_BIN="${HOME}/.local/bin/uv"
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ -z "${UV_BIN}" ]; then
|
||||
echo "无法获取 uv:内置二进制缺失且在线安装失败。请检查网络后重新安装。" > "${TRIM_TEMP_LOGFILE}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# --- Create venv via uv ---
|
||||
if [ ! -d ".venv" ]; then
|
||||
"${UV_BIN}" venv .venv --python "${PYTHON_BIN}" || {
|
||||
echo "Failed to create Python virtual environment via uv" > "${TRIM_TEMP_LOGFILE}"
|
||||
exit 1
|
||||
}
|
||||
fi
|
||||
|
||||
# --- Sync dependencies ---
|
||||
"${UV_BIN}" sync --extra seekdb || {
|
||||
echo "Dependency sync failed. Check network connectivity." > "${TRIM_TEMP_LOGFILE}"
|
||||
exit 1
|
||||
}
|
||||
|
||||
# --- Verify frontend dist ---
|
||||
if [ ! -d "web/dist" ] || [ -z "$(ls -A web/dist 2>/dev/null)" ]; then
|
||||
echo "Frontend dist missing! Web UI will not be available." > "${TRIM_TEMP_LOGFILE}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
exit 0
|
||||
Executable
+5
@@ -0,0 +1,5 @@
|
||||
#!/bin/bash
|
||||
# cmd/install_init - pre-install hook
|
||||
# Nothing special to do before extraction.
|
||||
|
||||
exit 0
|
||||
Executable
+194
@@ -0,0 +1,194 @@
|
||||
#!/bin/bash
|
||||
# cmd/main - LangBot lifecycle manager for fnOS
|
||||
# Handles start / stop / status via standalone-runtime Python process.
|
||||
# Node.js path is injected into PATH so Box sandbox npx MCP servers can run.
|
||||
|
||||
PID_FILE="${TRIM_PKGVAR}/langbot.pid"
|
||||
APP_DIR="${TRIM_APPDEST}/langbot"
|
||||
LOG_FILE="${TRIM_PKGVAR}/langbot.log"
|
||||
|
||||
# --- Locate fnOS Node.js bin path ---
|
||||
# fnOS appname-based path: /var/apps/nodejs_vXX/target/bin
|
||||
# This is a stable symlink regardless of which volume the app is on.
|
||||
NODE_VERSION="${wizard_node_version:-22}"
|
||||
NODE_BIN_DIR="/var/apps/nodejs_v${NODE_VERSION}/target/bin"
|
||||
|
||||
if [ -d "${NODE_BIN_DIR}" ]; then
|
||||
export PATH="${NODE_BIN_DIR}:${PATH}"
|
||||
fi
|
||||
|
||||
# --- Persistent data root ---
|
||||
# LangBot loads data/config.yaml CWD-RELATIVE (see core/stages/load_config.py:
|
||||
# load_yaml_config('data/config.yaml', ...)) and resolves its data root to
|
||||
# <CWD>/data in source-install mode — it does NOT honour LANGBOT_DATA_ROOT for
|
||||
# config.yaml. So the real fix is the symlink below: APP_DIR/data -> DATA_DIR.
|
||||
DATA_DIR="${TRIM_DATA_SHARE_PATHS%%:*}"
|
||||
if [ -z "${DATA_DIR}" ]; then
|
||||
DATA_DIR="${TRIM_PKGVAR}/data"
|
||||
fi
|
||||
export LANGBOT_DATA_ROOT="${DATA_DIR}"
|
||||
mkdir -p "${DATA_DIR}" 2>/dev/null || true
|
||||
|
||||
# --- Locate Python ---
|
||||
PYTHON_BIN="python3"
|
||||
! command -v "${PYTHON_BIN}" >/dev/null 2>&1 && PYTHON_BIN="python"
|
||||
|
||||
# --- Locate uv ---
|
||||
# install_callback puts the bundled uv binary at ${TRIM_PKGVAR}/bin/uv
|
||||
UV_BIN="${TRIM_PKGVAR}/bin/uv"
|
||||
if [ ! -x "${UV_BIN}" ]; then
|
||||
UV_BIN="uv"
|
||||
fi
|
||||
if ! command -v "${UV_BIN}" >/dev/null 2>&1; then
|
||||
UV_BIN="${HOME}/.local/bin/uv"
|
||||
fi
|
||||
if ! command -v "${UV_BIN}" >/dev/null 2>&1 && [ ! -x "${UV_BIN}" ]; then
|
||||
UV_BIN="${HOME}/.cargo/bin/uv"
|
||||
fi
|
||||
if ! command -v "${UV_BIN}" >/dev/null 2>&1 && [ ! -x "${UV_BIN}" ]; then
|
||||
UV_BIN="${APP_DIR}/.venv/bin/uv"
|
||||
fi
|
||||
|
||||
case $1 in
|
||||
start)
|
||||
if [ -f "${PID_FILE}" ]; then
|
||||
PID=$(cat "${PID_FILE}" | tr -d '[:space:]')
|
||||
if [ -n "${PID}" ] && kill -0 "${PID}" 2>/dev/null; then
|
||||
exit 0
|
||||
fi
|
||||
rm -f "${PID_FILE}"
|
||||
fi
|
||||
|
||||
if [ ! -d "${APP_DIR}" ]; then
|
||||
echo "LangBot app directory missing: ${APP_DIR}" > "${TRIM_TEMP_LOGFILE}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
cd "${APP_DIR}" || {
|
||||
echo "Cannot enter app directory: ${APP_DIR}" > "${TRIM_TEMP_LOGFILE}"
|
||||
exit 1
|
||||
}
|
||||
|
||||
if [ ! -d ".venv" ]; then
|
||||
echo "Python virtual environment not found. Please reinstall LangBot." > "${TRIM_TEMP_LOGFILE}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# --- Unify data location: APP_DIR/data -> symlink to persistent DATA_DIR ---
|
||||
# LangBot reads config CWD-relative (data/config.yaml), so without this a
|
||||
# fresh data/ with a default 5300 config gets recreated inside target/ on
|
||||
# every install/upgrade. The symlink keeps everything on the persistent
|
||||
# share; it is recreated here on each start (upgrades wipe target/).
|
||||
APP_DATA="${APP_DIR}/data"
|
||||
if [ -L "${APP_DATA}" ]; then
|
||||
# already a symlink; re-point if the persistent dir changed
|
||||
[ "$(readlink "${APP_DATA}")" != "${DATA_DIR}" ] && ln -sfn "${DATA_DIR}" "${APP_DATA}"
|
||||
elif [ -d "${APP_DATA}" ]; then
|
||||
# legacy real dir (created by LangBot before this fix): merge into the
|
||||
# persistent dir without overwriting newer files already there
|
||||
mkdir -p "${DATA_DIR}"
|
||||
cp -an "${APP_DATA}/." "${DATA_DIR}/" 2>/dev/null || cp -a "${APP_DATA}/." "${DATA_DIR}/"
|
||||
rm -rf "${APP_DATA}"
|
||||
ln -s "${DATA_DIR}" "${APP_DATA}"
|
||||
else
|
||||
ln -s "${DATA_DIR}" "${APP_DATA}"
|
||||
fi
|
||||
|
||||
# Ensure LangBot's actual listen port always matches what fnOS shows in
|
||||
# "应用设置 → 访问端口" (which is the single source of truth from the user's
|
||||
# POV). Two sources, checked in priority order:
|
||||
# 1. ${wizard_port} — only set during install/upgrade callbacks (not on
|
||||
# normal `start`; kept for completeness).
|
||||
# 2. target/ui/config — read the "port" field written by fnOS after
|
||||
# ${wizard_port} substitution AND any later edit the user made via
|
||||
# "应用设置 → 自定义 URL" pencil button.
|
||||
# Without this: user picks 5303 in wizard, LangBot still listens on its
|
||||
# default 5300, desktop shortcut hits 5303 → connection refused.
|
||||
CONFIG_FILE="${DATA_DIR}/config.yaml"
|
||||
_patch_port() {
|
||||
local _port="$1"
|
||||
case "${_port}" in
|
||||
''|*[!0-9]*) return ;;
|
||||
esac
|
||||
if [ ! -f "${CONFIG_FILE}" ]; then
|
||||
local _tmpl="${APP_DIR}/src/langbot/templates/config.yaml"
|
||||
[ -f "${_tmpl}" ] && cp "${_tmpl}" "${CONFIG_FILE}"
|
||||
fi
|
||||
if [ -f "${CONFIG_FILE}" ]; then
|
||||
sed -i -E "/^api:/,/^[a-z_]+:/ s/^([[:space:]]*port:).*/\1 ${_port}/" "${CONFIG_FILE}"
|
||||
sed -i "s#webhook_prefix: 'http://127\.0\.0\.1:[0-9]*'#webhook_prefix: 'http://127.0.0.1:${_port}'#" "${CONFIG_FILE}"
|
||||
fi
|
||||
}
|
||||
if [ -n "${wizard_port:-}" ]; then
|
||||
_patch_port "${wizard_port}"
|
||||
fi
|
||||
if [ -f "${TRIM_APPDEST}/ui/config" ]; then
|
||||
_port_from_ui=$(python3 -c 'import json,sys
|
||||
try:
|
||||
d = json.load(open(sys.argv[1]))
|
||||
for _name, _entry in (d.get(".url") or {}).items():
|
||||
p = _entry.get("port")
|
||||
if isinstance(p, (int, float)):
|
||||
print(int(p))
|
||||
elif isinstance(p, str) and p.isdigit():
|
||||
print(int(p))
|
||||
break
|
||||
except Exception:
|
||||
pass
|
||||
' "${TRIM_APPDEST}/ui/config" 2>/dev/null)
|
||||
if [ -n "${_port_from_ui}" ]; then
|
||||
_patch_port "${_port_from_ui}"
|
||||
fi
|
||||
fi
|
||||
|
||||
# Native deployment: no --standalone-runtime flag, LangBot spawns the
|
||||
# plugin runtime as a stdio subprocess (same as official `uv run main.py`).
|
||||
# (--standalone-runtime would require an external runtime at
|
||||
# ws://langbot_plugin_runtime:5400, which only exists in Docker Compose.)
|
||||
# --standalone-box omitted: Box sandbox defaults off, users enable via Web UI
|
||||
nohup "${UV_BIN}" run --no-sync main.py \
|
||||
> "${LOG_FILE}" 2>&1 &
|
||||
echo $! > "${PID_FILE}"
|
||||
|
||||
sleep 3
|
||||
if [ -f "${PID_FILE}" ]; then
|
||||
PID=$(cat "${PID_FILE}" | tr -d '[:space:]')
|
||||
if [ -n "${PID}" ] && kill -0 "${PID}" 2>/dev/null; then
|
||||
exit 0
|
||||
fi
|
||||
fi
|
||||
echo "LangBot failed to start. Check ${LOG_FILE}" > "${TRIM_TEMP_LOGFILE}"
|
||||
exit 1
|
||||
;;
|
||||
|
||||
stop)
|
||||
if [ -f "${PID_FILE}" ]; then
|
||||
PID=$(cat "${PID_FILE}" | tr -d '[:space:]')
|
||||
if [ -n "${PID}" ]; then
|
||||
kill "${PID}" 2>/dev/null
|
||||
for _ in 1 2 3 4 5 6 7 8 9 10; do
|
||||
kill -0 "${PID}" 2>/dev/null || break
|
||||
sleep 1
|
||||
done
|
||||
kill -9 "${PID}" 2>/dev/null
|
||||
fi
|
||||
rm -f "${PID_FILE}"
|
||||
fi
|
||||
exit 0
|
||||
;;
|
||||
|
||||
status)
|
||||
if [ -f "${PID_FILE}" ]; then
|
||||
PID=$(cat "${PID_FILE}" | tr -d '[:space:]')
|
||||
if [ -n "${PID}" ] && kill -0 "${PID}" 2>/dev/null; then
|
||||
exit 0
|
||||
fi
|
||||
rm -f "${PID_FILE}"
|
||||
fi
|
||||
exit 3
|
||||
;;
|
||||
|
||||
*)
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
Executable
+21
@@ -0,0 +1,21 @@
|
||||
#!/bin/bash
|
||||
# cmd/uninstall_callback - post-uninstall hook
|
||||
# The system preserves var/ and shares/ by default. Honor the user's
|
||||
# wizard_keep_data choice: delete data only when explicitly requested.
|
||||
|
||||
if [ "${wizard_keep_data:-yes}" = "no" ]; then
|
||||
# 应用运行数据(pid、日志等)
|
||||
if [ -n "${TRIM_PKGVAR}" ]; then
|
||||
rm -rf "${TRIM_PKGVAR:?}"/langbot.pid \
|
||||
"${TRIM_PKGVAR:?}"/langbot.log \
|
||||
"${TRIM_PKGVAR:?}"/bin 2>/dev/null || true
|
||||
fi
|
||||
|
||||
# 共享数据目录(langbot/data)
|
||||
DATA_DIR="${TRIM_DATA_SHARE_PATHS%%:*}"
|
||||
if [ -n "${DATA_DIR}" ]; then
|
||||
rm -rf "${DATA_DIR:?}" 2>/dev/null || true
|
||||
fi
|
||||
fi
|
||||
|
||||
exit 0
|
||||
Executable
+20
@@ -0,0 +1,20 @@
|
||||
#!/bin/bash
|
||||
# cmd/uninstall_init - pre-uninstall hook
|
||||
# Stop LangBot before files are removed.
|
||||
|
||||
PID_FILE="${TRIM_PKGVAR}/langbot.pid"
|
||||
|
||||
if [ -f "${PID_FILE}" ]; then
|
||||
PID=$(cat "${PID_FILE}" | tr -d '[:space:]')
|
||||
if [ -n "${PID}" ] && kill -0 "${PID}" 2>/dev/null; then
|
||||
kill "${PID}" 2>/dev/null
|
||||
for _ in 1 2 3 4 5 6 7 8 9 10; do
|
||||
kill -0 "${PID}" 2>/dev/null || break
|
||||
sleep 1
|
||||
done
|
||||
kill -9 "${PID}" 2>/dev/null
|
||||
fi
|
||||
rm -f "${PID_FILE}"
|
||||
fi
|
||||
|
||||
exit 0
|
||||
Executable
+77
@@ -0,0 +1,77 @@
|
||||
#!/bin/bash
|
||||
# cmd/upgrade_callback - post-upgrade hook
|
||||
# Re-sync dependencies after code replacement using uv.
|
||||
|
||||
APP_DIR="${TRIM_APPDEST}/langbot"
|
||||
|
||||
# Persistent data root (must match cmd/main)
|
||||
DATA_DIR="${TRIM_DATA_SHARE_PATHS%%:*}"
|
||||
[ -z "${DATA_DIR}" ] && DATA_DIR="${TRIM_PKGVAR}/data"
|
||||
|
||||
# Apply port from upgrade wizard (config persists across upgrades; this
|
||||
# only rewrites it when the user changed the value in the upgrade wizard)
|
||||
CONFIG_FILE="${DATA_DIR}/config.yaml"
|
||||
if [ -n "${wizard_port:-}" ] && [ -f "${CONFIG_FILE}" ]; then
|
||||
case "${wizard_port}" in
|
||||
''|*[!0-9]*) ;;
|
||||
*)
|
||||
sed -i -E "/^api:/,/^[a-z_]+:/ s/^([[:space:]]*port:).*/\1 ${wizard_port}/" "${CONFIG_FILE}"
|
||||
sed -i "s#webhook_prefix: 'http://127\.0\.0\.1:[0-9]*'#webhook_prefix: 'http://127.0.0.1:${wizard_port}'#" "${CONFIG_FILE}"
|
||||
;;
|
||||
esac
|
||||
fi
|
||||
|
||||
cd "${APP_DIR}" || {
|
||||
echo "App directory missing after upgrade" > "${TRIM_TEMP_LOGFILE}"
|
||||
exit 1
|
||||
}
|
||||
|
||||
# Find uv (bundled first, then PATH / ~/.local/bin / ~/.cargo/bin)
|
||||
UV_BIN="${TRIM_PKGVAR}/bin/uv"
|
||||
if [ ! -x "${UV_BIN}" ]; then
|
||||
UV_BIN="uv"
|
||||
fi
|
||||
if ! command -v "${UV_BIN}" >/dev/null 2>&1; then
|
||||
UV_BIN="${HOME}/.local/bin/uv"
|
||||
fi
|
||||
if ! command -v "${UV_BIN}" >/dev/null 2>&1 && [ ! -x "${UV_BIN}" ]; then
|
||||
UV_BIN="${HOME}/.cargo/bin/uv"
|
||||
fi
|
||||
|
||||
PYTHON_BIN="python3"
|
||||
! command -v "${PYTHON_BIN}" >/dev/null 2>&1 && PYTHON_BIN="python"
|
||||
|
||||
# Re-sync deps
|
||||
if [ -d ".venv" ]; then
|
||||
"${UV_BIN}" sync --extra seekdb 2>/dev/null || {
|
||||
echo "Dependency sync failed after upgrade" > "${TRIM_TEMP_LOGFILE}"
|
||||
exit 1
|
||||
}
|
||||
else
|
||||
# Venv was lost, recreate via uv
|
||||
if ! command -v "${UV_BIN}" >/dev/null 2>&1 && [ ! -x "${UV_BIN}" ]; then
|
||||
"${PYTHON_BIN}" -m pip install --user --no-cache-dir uv 2>/dev/null || \
|
||||
"${PYTHON_BIN}" -m pip install --no-cache-dir uv 2>/dev/null || {
|
||||
echo "Failed to install uv" > "${TRIM_TEMP_LOGFILE}"
|
||||
exit 1
|
||||
}
|
||||
export PATH="${HOME}/.local/bin:${PATH}"
|
||||
UV_BIN="uv"
|
||||
fi
|
||||
"${UV_BIN}" venv .venv --python "${PYTHON_BIN}" || {
|
||||
echo "Failed to recreate virtual environment" > "${TRIM_TEMP_LOGFILE}"
|
||||
exit 1
|
||||
}
|
||||
"${UV_BIN}" sync --extra seekdb || {
|
||||
echo "Dependency sync failed" > "${TRIM_TEMP_LOGFILE}"
|
||||
exit 1
|
||||
}
|
||||
fi
|
||||
|
||||
# Verify frontend dist still present
|
||||
if [ ! -d "web/dist" ] || [ -z "$(ls -A web/dist 2>/dev/null)" ]; then
|
||||
echo "Frontend dist missing after upgrade! Web UI will not be available." > "${TRIM_TEMP_LOGFILE}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
exit 0
|
||||
Executable
+20
@@ -0,0 +1,20 @@
|
||||
#!/bin/bash
|
||||
# cmd/upgrade_init - pre-upgrade hook
|
||||
# Stop the running LangBot process before files are replaced.
|
||||
|
||||
PID_FILE="${TRIM_PKGVAR}/langbot.pid"
|
||||
|
||||
if [ -f "${PID_FILE}" ]; then
|
||||
PID=$(cat "${PID_FILE}" | tr -d '[:space:]')
|
||||
if [ -n "${PID}" ] && kill -0 "${PID}" 2>/dev/null; then
|
||||
kill "${PID}" 2>/dev/null
|
||||
for _ in 1 2 3 4 5 6 7 8 9 10; do
|
||||
kill -0 "${PID}" 2>/dev/null || break
|
||||
sleep 1
|
||||
done
|
||||
kill -9 "${PID}" 2>/dev/null
|
||||
fi
|
||||
rm -f "${PID_FILE}"
|
||||
fi
|
||||
|
||||
exit 0
|
||||
@@ -0,0 +1,5 @@
|
||||
{
|
||||
"defaults": {
|
||||
"run-as": "root"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,9 @@
|
||||
{
|
||||
"data-share": {
|
||||
"shares": [
|
||||
{
|
||||
"name": "langbot/data"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,16 @@
|
||||
appname=langbot
|
||||
version=4.10.10
|
||||
display_name=LangBot
|
||||
desc=基于 LLM 的多平台智能对话机器人,支持 QQ、微信、飞书、钉钉、Telegram 等十余种即时通讯平台,内置 Web 管理界面和 AI Agent 能力。
|
||||
platform=all
|
||||
source=thirdparty
|
||||
maintainer=LangBot
|
||||
maintainer_url=https://langbot.app
|
||||
service_port=5300
|
||||
checkport=true
|
||||
os_min_version=0.9.0
|
||||
desktop_uidir=ui
|
||||
desktop_applaunchname=langbot.main
|
||||
ctl_stop=true
|
||||
install_dep_apps=nodejs_v22
|
||||
changelog=飞牛 fnOS 增强版首版:原生 Python 部署,依赖 Node.js v22 以启用 Box 沙箱 + npx MCP 能力
|
||||
@@ -0,0 +1,47 @@
|
||||
[
|
||||
{
|
||||
"stepTitle": "运行环境",
|
||||
"items": [
|
||||
{
|
||||
"type": "tips",
|
||||
"helpText": "LangBot 依赖 Node.js v22 运行 Box 沙箱和 npx MCP。请先在应用中心安装 Node.js v22。"
|
||||
},
|
||||
{
|
||||
"type": "select",
|
||||
"field": "wizard_node_version",
|
||||
"label": "Node.js 版本",
|
||||
"initValue": "22",
|
||||
"options": [
|
||||
{ "label": "Node.js v22 (推荐, LTS)", "value": "22" },
|
||||
{ "label": "Node.js v24", "value": "24" },
|
||||
{ "label": "Node.js v20", "value": "20" }
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"stepTitle": "访问配置",
|
||||
"items": [
|
||||
{
|
||||
"type": "text",
|
||||
"field": "wizard_port",
|
||||
"label": "Web 访问端口",
|
||||
"initValue": "5300",
|
||||
"rules": [
|
||||
{ "required": true, "message": "请输入访问端口" },
|
||||
{ "pattern": "^[0-9]+$", "message": "端口只能是数字" },
|
||||
{ "min": 1, "max": 5, "message": "端口号长度不正确" }
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"stepTitle": "安装说明",
|
||||
"items": [
|
||||
{
|
||||
"type": "tips",
|
||||
"helpText": "安装完成后,LangBot 首次启动约需 5-10 分钟完成依赖部署与初始化,部署完成后即可打开网页端使用。"
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
@@ -0,0 +1,17 @@
|
||||
[
|
||||
{
|
||||
"stepTitle": "数据保留",
|
||||
"items": [
|
||||
{
|
||||
"type": "radio",
|
||||
"field": "wizard_keep_data",
|
||||
"label": "是否保留 LangBot 数据(插件、配置、日志)",
|
||||
"initValue": "yes",
|
||||
"options": [
|
||||
{ "label": "保留数据(重新安装后可继续使用)", "value": "yes" },
|
||||
{ "label": "彻底删除全部数据", "value": "no" }
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
@@ -0,0 +1,38 @@
|
||||
[
|
||||
{
|
||||
"stepTitle": "运行环境",
|
||||
"items": [
|
||||
{
|
||||
"type": "tips",
|
||||
"helpText": "LangBot 依赖 Node.js v22 运行 Box 沙箱和 npx MCP。如需更换版本,请先在应用中心安装对应版本。"
|
||||
},
|
||||
{
|
||||
"type": "select",
|
||||
"field": "wizard_node_version",
|
||||
"label": "Node.js 版本",
|
||||
"initValue": "22",
|
||||
"options": [
|
||||
{ "label": "Node.js v22 (推荐, LTS)", "value": "22" },
|
||||
{ "label": "Node.js v24", "value": "24" },
|
||||
{ "label": "Node.js v20", "value": "20" }
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"stepTitle": "访问配置",
|
||||
"items": [
|
||||
{
|
||||
"type": "text",
|
||||
"field": "wizard_port",
|
||||
"label": "Web 访问端口",
|
||||
"initValue": "5300",
|
||||
"rules": [
|
||||
{ "required": true, "message": "请输入访问端口" },
|
||||
{ "pattern": "^[0-9]+$", "message": "端口只能是数字" },
|
||||
{ "min": 1, "max": 5, "message": "端口号长度不正确" }
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
+5
-3
@@ -1,6 +1,6 @@
|
||||
[project]
|
||||
name = "langbot"
|
||||
version = "4.10.10"
|
||||
version = "4.10.11"
|
||||
description = "Production-grade platform for building agentic IM bots"
|
||||
readme = "README.md"
|
||||
license-files = ["LICENSE"]
|
||||
@@ -70,7 +70,7 @@ dependencies = [
|
||||
"langchain-text-splitters>=1.1.2",
|
||||
"chromadb>=1.0.0,<2.0.0",
|
||||
"qdrant-client (>=1.15.1,<2.0.0)",
|
||||
"langbot-plugin==0.5.7",
|
||||
"langbot-plugin==0.5.8",
|
||||
"asyncpg>=0.30.0",
|
||||
"line-bot-sdk>=3.19.0",
|
||||
"matrix-nio>=0.25.2",
|
||||
@@ -81,6 +81,7 @@ dependencies = [
|
||||
"botocore>=1.42.39",
|
||||
"litellm>=1.0.0",
|
||||
"valkey-glide>=2.4.1,<3.0.0; sys_platform != 'win32'", # No Windows wheels are published
|
||||
"webauthn>=3.0.0",
|
||||
]
|
||||
keywords = [
|
||||
"bot",
|
||||
@@ -109,7 +110,8 @@ classifiers = [
|
||||
|
||||
[project.optional-dependencies]
|
||||
seekdb = [
|
||||
"pyseekdb==1.1.0.post3",
|
||||
"pyseekdb==1.4.0.post1",
|
||||
"pylibseekdb==1.4.0; sys_platform == 'linux' or (sys_platform == 'darwin' and platform_machine == 'arm64')",
|
||||
]
|
||||
|
||||
[project.urls]
|
||||
|
||||
@@ -88,6 +88,38 @@ already have a default pipeline.
|
||||
4. Use `list_*` tools to discover, then `get_*` / `create_*` / `update_*` /
|
||||
`delete_*` as needed.
|
||||
|
||||
## ChatGPT / Codex subscription providers
|
||||
|
||||
`list_model_providers` can return the `openai-codex` requester. Its OAuth
|
||||
credentials are server-only and are not provider API keys. Never ask a user
|
||||
to paste ChatGPT access tokens, refresh tokens, or a Codex auth cache into an
|
||||
MCP tool or model configuration.
|
||||
|
||||
A human connects or disconnects the subscription through **Models → provider
|
||||
settings** in the LangBot web UI. The provider-scoped `/codex/*` authentication
|
||||
routes deliberately require a browser-user session and are not exposed as MCP
|
||||
tools or authorized by a LangBot API key. Once connected, models are managed
|
||||
and selected through the normal provider/model workflow. A disconnected
|
||||
provider must be reauthorized; do not silently replace it with API-key billing.
|
||||
|
||||
See [ChatGPT / Codex subscription](../../../docs/CODEX_SUBSCRIPTION.md) for setup,
|
||||
usage limits, and the personal-account versus shared-service boundary.
|
||||
|
||||
## Provider deletion
|
||||
|
||||
The curated MCP surface currently lists providers but has no provider-deletion
|
||||
tool. In the web UI, **Edit Provider → Delete** asks for confirmation before
|
||||
removing that provider and all its LLM, embedding, and rerank models. This is
|
||||
irreversible; never interpret a request to edit a provider as authorization to
|
||||
delete it.
|
||||
|
||||
The equivalent HTTP operation is
|
||||
`DELETE /api/v1/provider/providers/{uuid}?cascade=true`, requiring
|
||||
`resource.manage` in the authenticated Workspace. Omitting `cascade` preserves
|
||||
the existing refusal to delete providers that still have models. Cloud-managed
|
||||
providers remain protected. Cascade deletion removes stored Codex authorization
|
||||
state as well; it is not the same operation as disconnecting an account.
|
||||
|
||||
## Implementation & maintenance (for LangBot developers)
|
||||
|
||||
- Server: `src/langbot/pkg/api/mcp/server.py` (FastMCP). Tools call the service
|
||||
|
||||
@@ -25,7 +25,10 @@ CLI uses. Create one in your Space account (Profile → Personal Access Tokens),
|
||||
then send it as a Bearer token:
|
||||
|
||||
```
|
||||
Authorization: Bearer lbpat_...uests without a valid PAT get `401 Unauthorized`.
|
||||
Authorization: Bearer <your-pat>
|
||||
```
|
||||
|
||||
Requests without a valid PAT get `401 Unauthorized`.
|
||||
|
||||
## Client configuration
|
||||
|
||||
@@ -66,6 +69,36 @@ All tools are read-only.
|
||||
state (available, unprobed, unavailable), then Space recommendation. Each
|
||||
item includes `availability.up`, `last_probed_at`, latency, and HTTP status.
|
||||
|
||||
## Runner usage recommendations
|
||||
|
||||
Use `search_plugins` with `runner_usage: "agent"` for Agent, pipeline, and
|
||||
setup-wizard recommendations, or `runner_usage: "event"` for event processors.
|
||||
The component kind remains `Runner`. Only these two exact values are accepted;
|
||||
omit the optional field to preserve unfiltered browsing.
|
||||
|
||||
```json
|
||||
{"query":"", "runner_usage":"agent", "page":1, "page_size":100}
|
||||
```
|
||||
|
||||
Plugin results include `latest_version` and `runner_usages: string[]`, the
|
||||
explicit union of usages in that latest installable version. Only recommend a
|
||||
plugin when this array explicitly contains the target usage. Missing, empty,
|
||||
malformed, or unknown usages must never mean agent-compatible. Event-only
|
||||
plugins must never enter Agent recommendations. Empty filtered results are
|
||||
valid while legacy packages await corrected releases; never remove the filter
|
||||
to fill a recommendation list.
|
||||
|
||||
REST callers use `runner_usage` on both
|
||||
`POST /api/v1/marketplace/extensions/search` and the compatibility
|
||||
`POST /api/v1/marketplace/plugins/search`; preserve it during fallback. Add
|
||||
`"type_filter":"plugin", "component_filter":"Runner"` on the unified endpoint.
|
||||
Usage is ANDed with other filters before pagination and `total`; MCP/Skill items
|
||||
do not match. Invalid REST values return HTTP 400.
|
||||
|
||||
Open the same filter in the webpage:
|
||||
`https://space.langbot.app/market?type=plugin&component=Runner&runner_usage=agent`
|
||||
(or `runner_usage=event`). Switch All / Agent / Event in the Runner usage row.
|
||||
|
||||
## Implementation & maintenance (for Space developers)
|
||||
|
||||
- Server: `internal/controller/mcp/server.go` (official Go MCP SDK
|
||||
|
||||
@@ -15,6 +15,7 @@ from ....workspace.collaboration import MembershipPermissionError, WorkspaceColl
|
||||
from ....workspace.errors import WorkspaceNotFoundError
|
||||
from ....cloud.entitlements import EntitlementUnavailableError
|
||||
from ....core.errors import TaskCapacityError
|
||||
from ....provider.modelmgr.codex_errors import CodexProviderError
|
||||
from ..authz import (
|
||||
AuthenticationDeniedError,
|
||||
AuthorizationError,
|
||||
@@ -247,6 +248,8 @@ class RouterGroup(abc.ABC):
|
||||
return await f(*args, **kwargs)
|
||||
|
||||
except Exception as e: # 自动 500
|
||||
if isinstance(e, CodexProviderError):
|
||||
return self.http_status(e.status_code, e.error_code, str(e))
|
||||
if isinstance(e, AuthorizationError):
|
||||
return self.http_status(e.status_code, e.error_code, str(e))
|
||||
if isinstance(e, WorkspaceNotFoundError):
|
||||
|
||||
@@ -5,6 +5,7 @@ import quart
|
||||
|
||||
from ...authz import Permission
|
||||
from ...context import RequestContext
|
||||
from ...service.monitoring_traffic import get_traffic_series
|
||||
from .. import group
|
||||
|
||||
|
||||
@@ -377,6 +378,14 @@ class MonitoringRouterGroup(group.RouterGroup):
|
||||
|
||||
return self.success(
|
||||
data={
|
||||
'traffic': await get_traffic_series(
|
||||
self.ap,
|
||||
request_context,
|
||||
bot_ids=bot_ids or None,
|
||||
pipeline_ids=pipeline_ids or None,
|
||||
start_time=start_time,
|
||||
end_time=end_time,
|
||||
),
|
||||
'overview': overview,
|
||||
'messages': messages,
|
||||
'llmCalls': llm_calls,
|
||||
@@ -405,6 +414,7 @@ class MonitoringRouterGroup(group.RouterGroup):
|
||||
session_id,
|
||||
start_time=start_time,
|
||||
end_time=end_time,
|
||||
bot_id=quart.request.args.get('botId'),
|
||||
)
|
||||
|
||||
# Always return success with the analysis data
|
||||
|
||||
@@ -9,6 +9,80 @@ from .query import resolve_include_secret
|
||||
@group.group_class('models/providers', '/api/v1/provider/providers')
|
||||
class ModelProvidersRouterGroup(group.RouterGroup):
|
||||
async def initialize(self) -> None:
|
||||
# Subscription authorization is an interactive, browser-user-only surface.
|
||||
@self.route(
|
||||
'/<provider_uuid>/codex/status',
|
||||
methods=['GET'],
|
||||
auth_type=group.AuthType.USER_TOKEN,
|
||||
permission=Permission.PROVIDER_SECRET_MANAGE,
|
||||
)
|
||||
async def codex_status(provider_uuid: str, request_context: RequestContext):
|
||||
try:
|
||||
return self.success(
|
||||
data=await self.ap.provider_service.codex_auth.status(request_context, provider_uuid)
|
||||
)
|
||||
except ValueError as exc:
|
||||
return self.http_status(400, -1, str(exc))
|
||||
|
||||
@self.route(
|
||||
'/<provider_uuid>/codex/device',
|
||||
methods=['POST'],
|
||||
auth_type=group.AuthType.USER_TOKEN,
|
||||
permission=Permission.PROVIDER_SECRET_MANAGE,
|
||||
)
|
||||
async def codex_device(provider_uuid: str, request_context: RequestContext):
|
||||
try:
|
||||
return self.success(
|
||||
data=await self.ap.provider_service.codex_auth.start(request_context, provider_uuid)
|
||||
)
|
||||
except ValueError as exc:
|
||||
return self.http_status(400, -1, str(exc))
|
||||
|
||||
@self.route(
|
||||
'/<provider_uuid>/codex/device/poll',
|
||||
methods=['POST'],
|
||||
auth_type=group.AuthType.USER_TOKEN,
|
||||
permission=Permission.PROVIDER_SECRET_MANAGE,
|
||||
)
|
||||
async def codex_poll(provider_uuid: str, request_context: RequestContext):
|
||||
body = await quart.request.get_json()
|
||||
if not isinstance(body, dict):
|
||||
return self.http_status(400, -1, 'JSON object required')
|
||||
try:
|
||||
return self.success(
|
||||
data=await self.ap.provider_service.codex_auth.poll(
|
||||
request_context, provider_uuid, body.get('authorization_id')
|
||||
)
|
||||
)
|
||||
except ValueError as exc:
|
||||
return self.http_status(400, -1, str(exc))
|
||||
|
||||
@self.route(
|
||||
'/<provider_uuid>/codex/auth',
|
||||
methods=['DELETE'],
|
||||
auth_type=group.AuthType.USER_TOKEN,
|
||||
permission=Permission.PROVIDER_SECRET_MANAGE,
|
||||
)
|
||||
async def codex_disconnect(provider_uuid: str, request_context: RequestContext):
|
||||
try:
|
||||
await self.ap.provider_service.codex_auth.disconnect(request_context, provider_uuid)
|
||||
return self.success()
|
||||
except ValueError as exc:
|
||||
return self.http_status(400, -1, str(exc))
|
||||
|
||||
@self.route(
|
||||
'/<provider_uuid>/codex/device/<authorization_id>',
|
||||
methods=['DELETE'],
|
||||
auth_type=group.AuthType.USER_TOKEN,
|
||||
permission=Permission.PROVIDER_SECRET_MANAGE,
|
||||
)
|
||||
async def codex_cancel(provider_uuid: str, authorization_id: str, request_context: RequestContext):
|
||||
try:
|
||||
await self.ap.provider_service.codex_auth.cancel(request_context, provider_uuid, authorization_id)
|
||||
return self.success()
|
||||
except ValueError as exc:
|
||||
return self.http_status(400, -1, str(exc))
|
||||
|
||||
@self.route(
|
||||
'',
|
||||
methods=['GET'],
|
||||
@@ -95,7 +169,15 @@ class ModelProvidersRouterGroup(group.RouterGroup):
|
||||
)
|
||||
async def _(provider_uuid: str, request_context: RequestContext) -> str:
|
||||
try:
|
||||
await self.ap.provider_service.delete_provider(request_context, provider_uuid)
|
||||
cascade_values = quart.request.args.getlist('cascade')
|
||||
if cascade_values:
|
||||
if len(cascade_values) != 1 or cascade_values[0] not in ('true', 'false'):
|
||||
return self.http_status(400, -1, 'cascade must be a single true or false value')
|
||||
await self.ap.provider_service.delete_provider(
|
||||
request_context, provider_uuid, cascade=cascade_values[0] == 'true'
|
||||
)
|
||||
else:
|
||||
await self.ap.provider_service.delete_provider(request_context, provider_uuid)
|
||||
return self.success()
|
||||
except ValueError as e:
|
||||
return self.http_status(400, -1, str(e))
|
||||
|
||||
@@ -1,7 +1,12 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import quart
|
||||
import argon2
|
||||
import asyncio
|
||||
import datetime
|
||||
import hmac
|
||||
import time
|
||||
import typing
|
||||
import uuid
|
||||
from urllib.parse import parse_qs, urlsplit
|
||||
|
||||
@@ -11,6 +16,33 @@ from ...context import RequestContext
|
||||
from .....cloud.launch import SpaceLaunchError
|
||||
from ...service.user import ControlPlaneDirectoryRequiredError, PublicRegistrationClosedError
|
||||
|
||||
# Fixed-window admission quota for the unauthenticated reset-password endpoint (#2392).
|
||||
# The admission check and slot bump share ONE synchronous critical section with no await
|
||||
# points, so concurrent bursts within a single event loop cannot slip past accounting.
|
||||
# Every admitted attempt consumes quota (regardless of success), which throttles both the
|
||||
# legacy 24-bit keyspace exhaustion and brute-force on modern high-entropy keys.
|
||||
# NOTE: this state is process-local; multi-worker deployments need a shared limiter upstream.
|
||||
_MAX_RESET_ATTEMPTS_PER_WINDOW = 5
|
||||
_RESET_WINDOW_SECONDS = 15 * 60
|
||||
|
||||
_reset_password_state: dict = {'window_started_at': 0.0, 'attempts': 0}
|
||||
|
||||
|
||||
def _admit_reset_attempt(now: float) -> bool:
|
||||
"""Atomically reserve one reset-password admission slot.
|
||||
|
||||
Must stay await-free: running to completion without suspension makes the
|
||||
check-and-increment atomic under the single-threaded event loop.
|
||||
"""
|
||||
st = _reset_password_state
|
||||
if now - st['window_started_at'] >= _RESET_WINDOW_SECONDS:
|
||||
st['window_started_at'] = now
|
||||
st['attempts'] = 0
|
||||
if st['attempts'] >= _MAX_RESET_ATTEMPTS_PER_WINDOW:
|
||||
return False
|
||||
st['attempts'] += 1
|
||||
return True
|
||||
|
||||
|
||||
@group.group_class('user', '/api/v1/user')
|
||||
class UserRouterGroup(group.RouterGroup):
|
||||
@@ -35,6 +67,22 @@ class UserRouterGroup(group.RouterGroup):
|
||||
|
||||
return redirect_uri
|
||||
|
||||
def _extract_origin_and_rp_id(self, json_data: dict[str, typing.Any] | None = None) -> tuple[str, str]:
|
||||
origin = ''
|
||||
if json_data and isinstance(json_data, dict):
|
||||
origin = json_data.get('origin', '')
|
||||
if not origin:
|
||||
origin = quart.request.headers.get('Origin', '')
|
||||
if not origin:
|
||||
origin = quart.request.headers.get('Referer', '')
|
||||
if not origin:
|
||||
origin = quart.request.url_root.rstrip('/')
|
||||
|
||||
parsed = urlsplit(origin)
|
||||
rp_id = parsed.hostname or 'localhost'
|
||||
clean_origin = f'{parsed.scheme}://{parsed.netloc}' if parsed.scheme and parsed.netloc else origin.rstrip('/')
|
||||
return clean_origin, rp_id
|
||||
|
||||
async def initialize(self) -> None:
|
||||
@self.route('/init', methods=['GET', 'POST'], auth_type=group.AuthType.NONE)
|
||||
async def _() -> str:
|
||||
@@ -81,6 +129,12 @@ class UserRouterGroup(group.RouterGroup):
|
||||
|
||||
@self.route('/reset-password', methods=['POST'], auth_type=group.AuthType.NONE)
|
||||
async def _() -> str:
|
||||
# Admit (or reject) BEFORE touching the body or any service call (#2392):
|
||||
# rejecting requests never reach the slow path, and quota accounting happens
|
||||
# synchronously at entry, closing the post-await race of burst requests.
|
||||
if not _admit_reset_attempt(time.monotonic()):
|
||||
return self.http_status(429, -1, 'Too many attempts, try again later')
|
||||
|
||||
json_data = await quart.request.json
|
||||
|
||||
user_email = json_data['user']
|
||||
@@ -98,7 +152,18 @@ class UserRouterGroup(group.RouterGroup):
|
||||
if user_obj is None:
|
||||
return self.http_status(400, -1, 'User not found')
|
||||
|
||||
if recovery_key != self.ap.instance_config.data['system']['recovery_key']:
|
||||
stored_key = self.ap.instance_config.data['system']['recovery_key']
|
||||
try:
|
||||
key_matches = (
|
||||
isinstance(recovery_key, str)
|
||||
and isinstance(stored_key, str)
|
||||
and hmac.compare_digest(recovery_key.encode(), stored_key.encode())
|
||||
)
|
||||
except UnicodeEncodeError:
|
||||
# JSON can contain lone surrogates, which are not valid UTF-8.
|
||||
key_matches = False
|
||||
|
||||
if not key_matches:
|
||||
return self.http_status(403, -1, 'Invalid recovery key')
|
||||
|
||||
await self.ap.user_service.reset_password(user_email, new_password)
|
||||
@@ -341,6 +406,8 @@ class UserRouterGroup(group.RouterGroup):
|
||||
capabilities['password_login_enabled'] = False
|
||||
capabilities['authenticated_invitation_acceptance_enabled'] = cloud_mode
|
||||
capabilities['invitation_registration_enabled'] = not cloud_mode
|
||||
capabilities['passkey_login_enabled'] = True
|
||||
capabilities['passkey_supported'] = True
|
||||
return self.success(data={'initialized': True, **capabilities})
|
||||
|
||||
@self.route('/set-password', methods=['POST'], auth_type=group.AuthType.USER_TOKEN)
|
||||
@@ -431,6 +498,182 @@ class UserRouterGroup(group.RouterGroup):
|
||||
except Exception:
|
||||
raise
|
||||
|
||||
@self.route('/passkey/register/options', methods=['POST'], auth_type=group.AuthType.USER_TOKEN)
|
||||
async def _(user_email: str) -> str:
|
||||
"""Generate WebAuthn registration options for current account."""
|
||||
allow_modify_login_info = self.ap.instance_config.data.get('system', {}).get(
|
||||
'allow_modify_login_info', True
|
||||
)
|
||||
if not allow_modify_login_info:
|
||||
return self.http_status(403, -1, 'Modifying login info is disabled')
|
||||
|
||||
user_obj = await self.ap.user_service.get_user_by_email(user_email)
|
||||
if user_obj is None:
|
||||
return self.http_status(404, -1, 'User not found')
|
||||
|
||||
json_data = (await quart.request.json) or {}
|
||||
origin, rp_id = self._extract_origin_and_rp_id(json_data)
|
||||
|
||||
try:
|
||||
options, challenge_token = await self.ap.user_service.generate_passkey_registration_options(
|
||||
account_uuid=user_obj.uuid,
|
||||
rp_id=rp_id,
|
||||
origin=origin,
|
||||
rp_name='LangBot',
|
||||
)
|
||||
return self.success(data={'options': options, 'challenge_token': challenge_token})
|
||||
except Exception as e:
|
||||
return self.fail(1, str(e))
|
||||
|
||||
@self.route('/passkey/register/verify', methods=['POST'], auth_type=group.AuthType.USER_TOKEN)
|
||||
async def _(user_email: str) -> str:
|
||||
"""Verify WebAuthn registration response and save credential."""
|
||||
allow_modify_login_info = self.ap.instance_config.data.get('system', {}).get(
|
||||
'allow_modify_login_info', True
|
||||
)
|
||||
if not allow_modify_login_info:
|
||||
return self.http_status(403, -1, 'Modifying login info is disabled')
|
||||
|
||||
user_obj = await self.ap.user_service.get_user_by_email(user_email)
|
||||
if user_obj is None:
|
||||
return self.http_status(404, -1, 'User not found')
|
||||
|
||||
json_data = await quart.request.json
|
||||
challenge_token = json_data.get('challenge_token')
|
||||
credential = json_data.get('credential') or json_data.get('response')
|
||||
name = json_data.get('name')
|
||||
|
||||
if not challenge_token or not credential:
|
||||
return self.fail(1, 'Missing challenge_token or credential')
|
||||
|
||||
try:
|
||||
cred = await self.ap.user_service.verify_and_save_passkey_registration(
|
||||
challenge_token=challenge_token,
|
||||
credential_data=credential,
|
||||
name=name,
|
||||
)
|
||||
return self.success(
|
||||
data={
|
||||
'uuid': cred.uuid,
|
||||
'name': cred.name,
|
||||
'created_at': cred.created_at.isoformat() if cred.created_at else None,
|
||||
}
|
||||
)
|
||||
except Exception as e:
|
||||
return self.fail(1, str(e))
|
||||
|
||||
@self.route('/passkey/auth/options', methods=['POST'], auth_type=group.AuthType.NONE)
|
||||
async def _() -> str:
|
||||
"""Generate WebAuthn authentication options for passkey login."""
|
||||
json_data = (await quart.request.json) or {}
|
||||
email = json_data.get('email')
|
||||
origin, rp_id = self._extract_origin_and_rp_id(json_data)
|
||||
|
||||
try:
|
||||
options, challenge_token = await self.ap.user_service.generate_passkey_authentication_options(
|
||||
rp_id=rp_id,
|
||||
origin=origin,
|
||||
email=email,
|
||||
)
|
||||
return self.success(data={'options': options, 'challenge_token': challenge_token})
|
||||
except Exception as e:
|
||||
return self.fail(1, str(e))
|
||||
|
||||
@self.route('/passkey/auth/verify', methods=['POST'], auth_type=group.AuthType.NONE)
|
||||
async def _() -> str:
|
||||
"""Verify WebAuthn authentication response and log in."""
|
||||
json_data = await quart.request.json
|
||||
challenge_token = json_data.get('challenge_token')
|
||||
credential = json_data.get('credential') or json_data.get('response')
|
||||
|
||||
if not challenge_token or not credential:
|
||||
return self.fail(1, 'Missing challenge_token or credential')
|
||||
|
||||
try:
|
||||
token, user_obj = await self.ap.user_service.verify_passkey_authentication(
|
||||
challenge_token=challenge_token,
|
||||
credential_data=credential,
|
||||
)
|
||||
return self.success(
|
||||
data={
|
||||
'token': token,
|
||||
'user': user_obj.user,
|
||||
}
|
||||
)
|
||||
except Exception as e:
|
||||
return self.fail(1, str(e))
|
||||
|
||||
@self.route('/passkeys', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
|
||||
async def _(user_email: str) -> str:
|
||||
"""List registered passkeys for the current user."""
|
||||
user_obj = await self.ap.user_service.get_user_by_email(user_email)
|
||||
if user_obj is None:
|
||||
return self.http_status(404, -1, 'User not found')
|
||||
|
||||
passkeys = await self.ap.user_service.get_user_passkeys(user_obj.uuid)
|
||||
return self.success(
|
||||
data=[
|
||||
{
|
||||
'uuid': pk.uuid,
|
||||
'name': pk.name,
|
||||
'aaguid': pk.aaguid,
|
||||
'transports': pk.transports,
|
||||
'backed_up': pk.backed_up,
|
||||
'created_at': pk.created_at.isoformat() if pk.created_at else None,
|
||||
'last_used_at': pk.last_used_at.isoformat() if pk.last_used_at else None,
|
||||
}
|
||||
for pk in passkeys
|
||||
]
|
||||
)
|
||||
|
||||
@self.route('/passkey/<passkey_uuid>', methods=['PATCH'], auth_type=group.AuthType.USER_TOKEN)
|
||||
async def _(user_email: str, passkey_uuid: str) -> str:
|
||||
"""Rename a registered passkey."""
|
||||
allow_modify_login_info = self.ap.instance_config.data.get('system', {}).get(
|
||||
'allow_modify_login_info', True
|
||||
)
|
||||
if not allow_modify_login_info:
|
||||
return self.http_status(403, -1, 'Modifying login info is disabled')
|
||||
|
||||
user_obj = await self.ap.user_service.get_user_by_email(user_email)
|
||||
if user_obj is None:
|
||||
return self.http_status(404, -1, 'User not found')
|
||||
|
||||
json_data = await quart.request.json
|
||||
name = (json_data.get('name') or '').strip()
|
||||
if not name:
|
||||
return self.fail(1, 'Passkey name cannot be empty')
|
||||
|
||||
updated = await self.ap.user_service.rename_user_passkey(
|
||||
account_uuid=user_obj.uuid,
|
||||
passkey_uuid=passkey_uuid,
|
||||
new_name=name,
|
||||
)
|
||||
if not updated:
|
||||
return self.http_status(404, -1, 'Passkey not found')
|
||||
return self.success(data={'uuid': updated.uuid, 'name': updated.name})
|
||||
|
||||
@self.route('/passkey/<passkey_uuid>', methods=['DELETE'], auth_type=group.AuthType.USER_TOKEN)
|
||||
async def _(user_email: str, passkey_uuid: str) -> str:
|
||||
"""Delete/revoke a registered passkey."""
|
||||
allow_modify_login_info = self.ap.instance_config.data.get('system', {}).get(
|
||||
'allow_modify_login_info', True
|
||||
)
|
||||
if not allow_modify_login_info:
|
||||
return self.http_status(403, -1, 'Modifying login info is disabled')
|
||||
|
||||
user_obj = await self.ap.user_service.get_user_by_email(user_email)
|
||||
if user_obj is None:
|
||||
return self.http_status(404, -1, 'User not found')
|
||||
|
||||
deleted = await self.ap.user_service.delete_user_passkey(
|
||||
account_uuid=user_obj.uuid,
|
||||
passkey_uuid=passkey_uuid,
|
||||
)
|
||||
if not deleted:
|
||||
return self.http_status(404, -1, 'Passkey not found')
|
||||
return self.success()
|
||||
|
||||
async def _handle_space_direct_launch(
|
||||
self,
|
||||
launch_assertion: str,
|
||||
|
||||
@@ -446,15 +446,19 @@ class MCPService:
|
||||
persisted_session = runtime_mcp_session
|
||||
|
||||
async def _refresh_and_report() -> None:
|
||||
needs_start = persisted_session.status == MCPSessionStatus.ERROR or persisted_session.session is None
|
||||
if needs_start:
|
||||
await persisted_session.start()
|
||||
else:
|
||||
try:
|
||||
await persisted_session.refresh()
|
||||
except Exception:
|
||||
try:
|
||||
needs_start = (
|
||||
persisted_session.status == MCPSessionStatus.ERROR or persisted_session.session is None
|
||||
)
|
||||
if needs_start:
|
||||
await persisted_session.start()
|
||||
ctx.metadata['runtime_info'] = persisted_session.get_runtime_info_dict()
|
||||
else:
|
||||
try:
|
||||
await persisted_session.refresh()
|
||||
except Exception:
|
||||
await persisted_session.start()
|
||||
finally:
|
||||
ctx.metadata['runtime_info'] = persisted_session.get_runtime_info_dict()
|
||||
|
||||
coroutine = _refresh_and_report()
|
||||
else:
|
||||
@@ -471,8 +475,11 @@ class MCPService:
|
||||
async def _run_and_cleanup() -> None:
|
||||
try:
|
||||
await test_session.start()
|
||||
ctx.metadata['runtime_info'] = test_session.get_runtime_info_dict()
|
||||
finally:
|
||||
# start() raises for a failed connection. Preserve the
|
||||
# terminal runtime state so the UI can render actionable
|
||||
# failure phases such as OAuth-required.
|
||||
ctx.metadata['runtime_info'] = test_session.get_runtime_info_dict()
|
||||
try:
|
||||
await test_session.shutdown()
|
||||
except Exception as exc:
|
||||
|
||||
@@ -29,6 +29,19 @@ _DEFAULT_CLEANUP_BATCHES_PER_TABLE = 4
|
||||
_HARD_MAX_CLEANUP_BATCHES_PER_TABLE = 100
|
||||
|
||||
|
||||
def _normalize_user_id(value: str | int | None) -> str | None:
|
||||
"""Convert numeric platform IDs before binding a VARCHAR with asyncpg.
|
||||
|
||||
Opaque string IDs (including whitespace and leading zeros) and missing
|
||||
IDs must remain unchanged. Do not silently stringify unsupported objects.
|
||||
"""
|
||||
if value is None or isinstance(value, str):
|
||||
return value
|
||||
if isinstance(value, int) and not isinstance(value, bool):
|
||||
return str(value)
|
||||
raise TypeError('user_id must be a string, integer, or None')
|
||||
|
||||
|
||||
def _workspace_transaction(method):
|
||||
"""Run an explicit service entrypoint in one Workspace transaction."""
|
||||
|
||||
@@ -281,19 +294,21 @@ class MonitoringService:
|
||||
for _batch_number in range(max_batches):
|
||||
|
||||
async def delete_batch() -> tuple[int, int]:
|
||||
key_columns = list(model_cls.__table__.primary_key.columns)
|
||||
select_result = await self.ap.persistence_mgr.execute_async(
|
||||
sqlalchemy.select(pk_column)
|
||||
sqlalchemy.select(*key_columns)
|
||||
.where(model_cls.workspace_uuid == workspace_uuid, ts_column < cutoff)
|
||||
.limit(batch_size)
|
||||
)
|
||||
pk_values = list(select_result.scalars().all())
|
||||
pk_values = [tuple(row) for row in select_result.all()]
|
||||
if not pk_values:
|
||||
return 0, 0
|
||||
|
||||
delete_result = await self.ap.persistence_mgr.execute_async(
|
||||
sqlalchemy.delete(model_cls).where(
|
||||
model_cls.workspace_uuid == workspace_uuid,
|
||||
pk_column.in_(pk_values),
|
||||
sqlalchemy.tuple_(*key_columns).in_(pk_values),
|
||||
ts_column < cutoff,
|
||||
)
|
||||
)
|
||||
return len(pk_values), int(delete_result.rowcount or 0)
|
||||
@@ -415,7 +430,7 @@ class MonitoringService:
|
||||
status: str = 'success',
|
||||
level: str = 'info',
|
||||
platform: str | None = None,
|
||||
user_id: str | None = None,
|
||||
user_id: str | int | None = None,
|
||||
user_name: str | None = None,
|
||||
runner_name: str | None = None,
|
||||
variables: str | None = None,
|
||||
@@ -437,7 +452,7 @@ class MonitoringService:
|
||||
'status': status,
|
||||
'level': level,
|
||||
'platform': platform,
|
||||
'user_id': user_id,
|
||||
'user_id': _normalize_user_id(user_id),
|
||||
'user_name': user_name,
|
||||
'runner_name': runner_name,
|
||||
'variables': variables,
|
||||
@@ -610,7 +625,7 @@ class MonitoringService:
|
||||
pipeline_id: str,
|
||||
pipeline_name: str,
|
||||
platform: str | None = None,
|
||||
user_id: str | None = None,
|
||||
user_id: str | int | None = None,
|
||||
user_name: str | None = None,
|
||||
) -> None:
|
||||
"""Record a new session"""
|
||||
@@ -622,17 +637,29 @@ class MonitoringService:
|
||||
'bot_name': bot_name,
|
||||
'pipeline_id': pipeline_id,
|
||||
'pipeline_name': pipeline_name,
|
||||
'message_count': 0,
|
||||
'message_count': 1,
|
||||
'start_time': datetime.datetime.now(datetime.timezone.utc).replace(tzinfo=None),
|
||||
'last_activity': datetime.datetime.now(datetime.timezone.utc).replace(tzinfo=None),
|
||||
'is_active': True,
|
||||
'platform': platform,
|
||||
'user_id': user_id,
|
||||
'user_id': _normalize_user_id(user_id),
|
||||
'user_name': user_name,
|
||||
}
|
||||
|
||||
model = persistence_monitoring.MonitoringSession
|
||||
dialect = self.ap.persistence_mgr.get_db_engine().dialect.name
|
||||
insert = postgresql_dialect.insert if dialect == 'postgresql' else sqlite_dialect.insert
|
||||
statement = insert(model).values(session_data)
|
||||
await self.ap.persistence_mgr.execute_async(
|
||||
sqlalchemy.insert(persistence_monitoring.MonitoringSession).values(session_data)
|
||||
statement.on_conflict_do_update(
|
||||
index_elements=['workspace_uuid', 'bot_id', 'session_id'],
|
||||
set_={
|
||||
'message_count': model.message_count + 1,
|
||||
'last_activity': statement.excluded.last_activity,
|
||||
'pipeline_id': statement.excluded.pipeline_id,
|
||||
'pipeline_name': statement.excluded.pipeline_name,
|
||||
},
|
||||
)
|
||||
)
|
||||
|
||||
@_workspace_transaction
|
||||
@@ -642,6 +669,7 @@ class MonitoringService:
|
||||
session_id: str,
|
||||
pipeline_id: str | None = None,
|
||||
pipeline_name: str | None = None,
|
||||
bot_id: str | None = None,
|
||||
) -> bool:
|
||||
"""Update session last activity time and increment message count.
|
||||
|
||||
@@ -651,6 +679,9 @@ class MonitoringService:
|
||||
True if session was found and updated, False if session doesn't exist.
|
||||
"""
|
||||
workspace_uuid = self._require_write_context(context)
|
||||
bot_id = bot_id if bot_id is not None else context.bot_uuid
|
||||
if not bot_id:
|
||||
raise ValueError('Session activity requires a bot_id')
|
||||
update_values = {
|
||||
'last_activity': datetime.datetime.now(datetime.timezone.utc).replace(tzinfo=None),
|
||||
'message_count': persistence_monitoring.MonitoringSession.message_count + 1,
|
||||
@@ -667,6 +698,7 @@ class MonitoringService:
|
||||
.where(
|
||||
persistence_monitoring.MonitoringSession.workspace_uuid == workspace_uuid,
|
||||
persistence_monitoring.MonitoringSession.session_id == session_id,
|
||||
persistence_monitoring.MonitoringSession.bot_id == bot_id,
|
||||
)
|
||||
.values(update_values)
|
||||
)
|
||||
@@ -769,13 +801,13 @@ class MonitoringService:
|
||||
message_conditions.append(persistence_monitoring.MonitoringMessage.timestamp >= start_time)
|
||||
llm_conditions.append(persistence_monitoring.MonitoringLLMCall.timestamp >= start_time)
|
||||
embedding_conditions.append(persistence_monitoring.MonitoringEmbeddingCall.timestamp >= start_time)
|
||||
session_conditions.append(persistence_monitoring.MonitoringSession.start_time >= start_time)
|
||||
session_conditions.append(persistence_monitoring.MonitoringSession.last_activity >= start_time)
|
||||
|
||||
if end_time:
|
||||
message_conditions.append(persistence_monitoring.MonitoringMessage.timestamp <= end_time)
|
||||
llm_conditions.append(persistence_monitoring.MonitoringLLMCall.timestamp <= end_time)
|
||||
embedding_conditions.append(persistence_monitoring.MonitoringEmbeddingCall.timestamp <= end_time)
|
||||
session_conditions.append(persistence_monitoring.MonitoringSession.start_time <= end_time)
|
||||
session_conditions.append(persistence_monitoring.MonitoringSession.last_activity <= end_time)
|
||||
|
||||
# Total messages
|
||||
message_query = sqlalchemy.select(sqlalchemy.func.count(persistence_monitoring.MonitoringMessage.id))
|
||||
@@ -1272,9 +1304,9 @@ class MonitoringService:
|
||||
if pipeline_ids:
|
||||
conditions.append(persistence_monitoring.MonitoringSession.pipeline_id.in_(pipeline_ids))
|
||||
if start_time:
|
||||
conditions.append(persistence_monitoring.MonitoringSession.start_time >= start_time)
|
||||
conditions.append(persistence_monitoring.MonitoringSession.last_activity >= start_time)
|
||||
if end_time:
|
||||
conditions.append(persistence_monitoring.MonitoringSession.start_time <= end_time)
|
||||
conditions.append(persistence_monitoring.MonitoringSession.last_activity <= end_time)
|
||||
if user_query and user_query.strip():
|
||||
user_pattern = f'%{user_query.strip()}%'
|
||||
conditions.append(
|
||||
@@ -1376,6 +1408,7 @@ class MonitoringService:
|
||||
session_id: str,
|
||||
start_time: datetime.datetime | None = None,
|
||||
end_time: datetime.datetime | None = None,
|
||||
bot_id: str | None = None,
|
||||
) -> dict:
|
||||
"""Get bounded session details with full statistics computed in SQL."""
|
||||
workspace_uuid = require_workspace_uuid(context)
|
||||
@@ -1385,8 +1418,13 @@ class MonitoringService:
|
||||
persistence_monitoring.MonitoringSession.workspace_uuid == workspace_uuid,
|
||||
persistence_monitoring.MonitoringSession.session_id == session_id,
|
||||
)
|
||||
session_result = await self.ap.persistence_mgr.execute_async(session_query)
|
||||
session_row = session_result.first()
|
||||
if bot_id is not None:
|
||||
session_query = session_query.where(persistence_monitoring.MonitoringSession.bot_id == bot_id)
|
||||
session_result = await self.ap.persistence_mgr.execute_async(session_query.limit(2))
|
||||
session_rows = session_result.all()
|
||||
if len(session_rows) > 1:
|
||||
return {'session_id': session_id, 'found': False, 'ambiguous': True}
|
||||
session_row = session_rows[0] if session_rows else None
|
||||
|
||||
if not session_row:
|
||||
return {
|
||||
@@ -1395,6 +1433,7 @@ class MonitoringService:
|
||||
}
|
||||
|
||||
session = session_row[0] if isinstance(session_row, tuple) else session_row
|
||||
bot_id = session.bot_id
|
||||
|
||||
message_stats_result = await self.ap.persistence_mgr.execute_async(
|
||||
sqlalchemy.select(
|
||||
@@ -1422,6 +1461,7 @@ class MonitoringService:
|
||||
).where(
|
||||
persistence_monitoring.MonitoringMessage.workspace_uuid == workspace_uuid,
|
||||
persistence_monitoring.MonitoringMessage.session_id == session_id,
|
||||
persistence_monitoring.MonitoringMessage.bot_id == bot_id,
|
||||
)
|
||||
)
|
||||
message_stats = message_stats_result.one()
|
||||
@@ -1460,6 +1500,7 @@ class MonitoringService:
|
||||
).where(
|
||||
persistence_monitoring.MonitoringLLMCall.workspace_uuid == workspace_uuid,
|
||||
persistence_monitoring.MonitoringLLMCall.session_id == session_id,
|
||||
persistence_monitoring.MonitoringLLMCall.bot_id == bot_id,
|
||||
)
|
||||
)
|
||||
llm_stats = llm_stats_result.one()
|
||||
@@ -1486,12 +1527,14 @@ class MonitoringService:
|
||||
).where(
|
||||
persistence_monitoring.MonitoringToolCall.workspace_uuid == workspace_uuid,
|
||||
persistence_monitoring.MonitoringToolCall.session_id == session_id,
|
||||
persistence_monitoring.MonitoringToolCall.bot_id == bot_id,
|
||||
)
|
||||
)
|
||||
tool_stats = tool_stats_result.one()
|
||||
tool_conditions = [
|
||||
persistence_monitoring.MonitoringToolCall.workspace_uuid == workspace_uuid,
|
||||
persistence_monitoring.MonitoringToolCall.session_id == session_id,
|
||||
persistence_monitoring.MonitoringToolCall.bot_id == bot_id,
|
||||
]
|
||||
if start_time is not None:
|
||||
tool_conditions.append(persistence_monitoring.MonitoringToolCall.timestamp >= start_time)
|
||||
@@ -1520,6 +1563,7 @@ class MonitoringService:
|
||||
.where(
|
||||
persistence_monitoring.MonitoringError.workspace_uuid == workspace_uuid,
|
||||
persistence_monitoring.MonitoringError.session_id == session_id,
|
||||
persistence_monitoring.MonitoringError.bot_id == bot_id,
|
||||
)
|
||||
.order_by(persistence_monitoring.MonitoringError.timestamp.desc())
|
||||
.limit(detail_limit + 1)
|
||||
@@ -2004,9 +2048,9 @@ class MonitoringService:
|
||||
if pipeline_ids:
|
||||
conditions.append(persistence_monitoring.MonitoringSession.pipeline_id.in_(pipeline_ids))
|
||||
if start_time:
|
||||
conditions.append(persistence_monitoring.MonitoringSession.start_time >= start_time)
|
||||
conditions.append(persistence_monitoring.MonitoringSession.last_activity >= start_time)
|
||||
if end_time:
|
||||
conditions.append(persistence_monitoring.MonitoringSession.start_time <= end_time)
|
||||
conditions.append(persistence_monitoring.MonitoringSession.last_activity <= end_time)
|
||||
|
||||
query = sqlalchemy.select(persistence_monitoring.MonitoringSession).order_by(
|
||||
persistence_monitoring.MonitoringSession.last_activity.desc()
|
||||
@@ -2040,6 +2084,7 @@ class MonitoringService:
|
||||
|
||||
# ========== Feedback Methods ==========
|
||||
|
||||
@_workspace_transaction
|
||||
async def record_feedback(
|
||||
self,
|
||||
context: ExecutionContext,
|
||||
@@ -2054,7 +2099,7 @@ class MonitoringService:
|
||||
session_id: str | None = None,
|
||||
message_id: str | None = None,
|
||||
stream_id: str | None = None,
|
||||
user_id: str | None = None,
|
||||
user_id: str | int | None = None,
|
||||
platform: str | None = None,
|
||||
) -> str | None:
|
||||
"""Record user feedback (like/dislike) from AI Bot conversation.
|
||||
@@ -2110,7 +2155,7 @@ class MonitoringService:
|
||||
'session_id': session_id,
|
||||
'message_id': message_id,
|
||||
'stream_id': stream_id,
|
||||
'user_id': user_id,
|
||||
'user_id': _normalize_user_id(user_id),
|
||||
'platform': platform,
|
||||
}
|
||||
dialect_name = self.ap.persistence_mgr.get_db_engine().dialect.name
|
||||
|
||||
@@ -0,0 +1,83 @@
|
||||
"""Bounded traffic aggregation, independent of record-list pagination."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import datetime
|
||||
import typing
|
||||
|
||||
import sqlalchemy
|
||||
|
||||
from ....entity.persistence.monitoring import MonitoringLLMCall, MonitoringMessage
|
||||
from .tenant import TenantContext, require_workspace_uuid
|
||||
|
||||
if typing.TYPE_CHECKING:
|
||||
from ....core.app import Application
|
||||
|
||||
MAX_TRAFFIC_POINTS = 1000
|
||||
|
||||
|
||||
async def get_traffic_series(
|
||||
ap: Application,
|
||||
context: TenantContext,
|
||||
*,
|
||||
bot_ids: list[str] | None = None,
|
||||
pipeline_ids: list[str] | None = None,
|
||||
start_time: datetime.datetime | None = None,
|
||||
end_time: datetime.datetime | None = None,
|
||||
) -> dict:
|
||||
"""Count all matching records in UTC buckets, returning at most 1000 points."""
|
||||
workspace_uuid = require_workspace_uuid(context)
|
||||
bucket = 'hour' if start_time and end_time and end_time - start_time <= datetime.timedelta(days=7) else 'day'
|
||||
step = datetime.timedelta(hours=1) if bucket == 'hour' else datetime.timedelta(days=1)
|
||||
postgres = ap.persistence_mgr.get_db_engine().dialect.name == 'postgresql'
|
||||
points: dict[datetime.datetime, dict[str, int]] = {}
|
||||
truncated = False
|
||||
for model, field in ((MonitoringMessage, 'messages'), (MonitoringLLMCall, 'llm_calls')):
|
||||
timestamp = model.timestamp
|
||||
if postgres:
|
||||
time_bucket = sqlalchemy.func.date_trunc(bucket, timestamp)
|
||||
else:
|
||||
pattern = '%Y-%m-%dT%H:00:00' if bucket == 'hour' else '%Y-%m-%dT00:00:00'
|
||||
time_bucket = sqlalchemy.func.strftime(pattern, timestamp)
|
||||
conditions = [model.workspace_uuid == workspace_uuid]
|
||||
if bot_ids:
|
||||
conditions.append(model.bot_id.in_(bot_ids))
|
||||
if pipeline_ids:
|
||||
conditions.append(model.pipeline_id.in_(pipeline_ids))
|
||||
if start_time is not None:
|
||||
conditions.append(timestamp >= start_time)
|
||||
if end_time is not None:
|
||||
conditions.append(timestamp <= end_time)
|
||||
statement = (
|
||||
sqlalchemy.select(time_bucket.label('bucket'), sqlalchemy.func.count(model.id).label('count'))
|
||||
.where(*conditions)
|
||||
.group_by(time_bucket)
|
||||
.order_by(time_bucket)
|
||||
.limit(MAX_TRAFFIC_POINTS + 1)
|
||||
)
|
||||
result = await ap.persistence_mgr.execute_async(statement)
|
||||
rows = result.all()
|
||||
truncated = truncated or len(rows) > MAX_TRAFFIC_POINTS
|
||||
for timestamp_value, count in rows[:MAX_TRAFFIC_POINTS]:
|
||||
key = (
|
||||
datetime.datetime.fromisoformat(timestamp_value)
|
||||
if isinstance(timestamp_value, str)
|
||||
else timestamp_value
|
||||
)
|
||||
points.setdefault(key, {'messages': 0, 'llm_calls': 0})[field] = int(count)
|
||||
|
||||
def floor(value: datetime.datetime) -> datetime.datetime:
|
||||
return value.replace(minute=0, second=0, microsecond=0, **({'hour': 0} if bucket == 'day' else {}))
|
||||
|
||||
first = floor(start_time) if start_time is not None else min(points, default=None)
|
||||
last = floor(end_time) if end_time is not None else max(points, default=None)
|
||||
series = []
|
||||
if first is not None and last is not None:
|
||||
cursor = first
|
||||
while cursor <= last and len(series) < MAX_TRAFFIC_POINTS:
|
||||
series.append(
|
||||
{'timestamp': cursor.isoformat() + 'Z', **points.get(cursor, {'messages': 0, 'llm_calls': 0})}
|
||||
)
|
||||
cursor += step
|
||||
truncated = truncated or cursor <= last
|
||||
return {'bucket': bucket, 'points': series, 'truncated': truncated}
|
||||
@@ -1,5 +1,6 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import uuid
|
||||
import traceback
|
||||
|
||||
@@ -7,8 +8,10 @@ import sqlalchemy
|
||||
|
||||
from ....cloud.model_catalog import LANGBOT_MODELS_PROVIDER_REQUESTER
|
||||
from ....core import app
|
||||
from ....core.task_boundary import create_detached_task
|
||||
from ....entity.persistence import model as persistence_model
|
||||
from ....workspace.errors import WorkspaceNotFoundError
|
||||
from ....provider.modelmgr.codex_auth import CodexAuth, REQUESTER as CODEX_REQUESTER, validate_config
|
||||
from .secrets import contains_secret_placeholder, redact_secrets, restore_secret_placeholders
|
||||
from .tenant import TenantContext, require_workspace_uuid, scope_statement
|
||||
|
||||
@@ -20,6 +23,8 @@ class ModelProviderService:
|
||||
|
||||
def __init__(self, ap: app.Application) -> None:
|
||||
self.ap = ap
|
||||
self.codex_auth = CodexAuth(ap)
|
||||
self._deletion_tasks: set[asyncio.Task[None]] = set()
|
||||
|
||||
def _is_cloud_runtime(self) -> bool:
|
||||
mode = getattr(self.ap.persistence_mgr, 'mode', None)
|
||||
@@ -116,14 +121,30 @@ class ModelProviderService:
|
||||
provider_data = provider_data.copy()
|
||||
if self._system_requester_is_reserved(provider_data.get('requester')):
|
||||
raise ValueError('space-chat-completions is reserved for the Cloud-managed LangBot Models provider')
|
||||
validate_config(provider_data)
|
||||
provider_data['uuid'] = str(uuid.uuid4())
|
||||
provider_data['workspace_uuid'] = require_workspace_uuid(context)
|
||||
provider_data['api_keys'] = self._normalize_api_keys(
|
||||
restore_secret_placeholders(provider_data.get('api_keys'), sensitive=True)
|
||||
)
|
||||
await self.ap.persistence_mgr.execute_async(
|
||||
sqlalchemy.insert(persistence_model.ModelProvider).values(**provider_data)
|
||||
)
|
||||
if provider_data.get('requester') == CODEX_REQUESTER:
|
||||
async with self.ap.persistence_mgr.tenant_uow(provider_data['workspace_uuid']):
|
||||
await self.ap.persistence_mgr.execute_async(
|
||||
sqlalchemy.insert(persistence_model.ModelProvider).values(**provider_data)
|
||||
)
|
||||
await self.ap.persistence_mgr.execute_async(
|
||||
sqlalchemy.insert(persistence_model.CodexCredential).values(
|
||||
workspace_uuid=provider_data['workspace_uuid'],
|
||||
provider_uuid=provider_data['uuid'],
|
||||
payload={},
|
||||
version=0,
|
||||
lease_until=0,
|
||||
)
|
||||
)
|
||||
else:
|
||||
await self.ap.persistence_mgr.execute_async(
|
||||
sqlalchemy.insert(persistence_model.ModelProvider).values(**provider_data)
|
||||
)
|
||||
|
||||
# load to runtime
|
||||
runtime_provider = await self.ap.model_mgr.load_provider(context, provider_data)
|
||||
@@ -138,6 +159,17 @@ class ModelProviderService:
|
||||
raise ValueError('space-chat-completions is reserved for the Cloud-managed LangBot Models provider')
|
||||
provider_data.pop('uuid', None)
|
||||
provider_data.pop('workspace_uuid', None)
|
||||
if {'requester', 'base_url', 'api_keys'} & provider_data.keys():
|
||||
current = await self.get_provider(context, provider_uuid, include_secret=True)
|
||||
if current is None:
|
||||
raise WorkspaceNotFoundError('Provider not found')
|
||||
if CODEX_REQUESTER in (current.get('requester'), provider_data.get('requester')):
|
||||
if provider_data.get('requester', current.get('requester')) != current.get('requester'):
|
||||
raise ValueError('Create a separate provider to change the ChatGPT authentication type')
|
||||
merged = {**current, **provider_data}
|
||||
validate_config(merged)
|
||||
provider_data['base_url'] = merged['base_url']
|
||||
provider_data['api_keys'] = []
|
||||
if 'api_keys' in provider_data:
|
||||
submitted_keys = provider_data.get('api_keys')
|
||||
if contains_secret_placeholder(submitted_keys, sensitive=True):
|
||||
@@ -163,60 +195,107 @@ class ModelProviderService:
|
||||
raise WorkspaceNotFoundError('Provider not found')
|
||||
await self.ap.model_mgr.reload_provider(context, provider_uuid)
|
||||
|
||||
async def delete_provider(self, context: TenantContext, provider_uuid: str) -> None:
|
||||
"""Delete a provider (only if no models reference it)"""
|
||||
await self._assert_provider_mutable(context, provider_uuid)
|
||||
async def delete_provider(self, context: TenantContext, provider_uuid: str, cascade: bool = False) -> None:
|
||||
"""Delete a provider, optionally deleting all its Workspace-scoped models."""
|
||||
workspace_uuid = require_workspace_uuid(context)
|
||||
# Check if any models use this provider
|
||||
llm_result = await self.ap.persistence_mgr.execute_async(
|
||||
scope_statement(
|
||||
sqlalchemy.select(persistence_model.LLMModel).where(
|
||||
persistence_model.LLMModel.provider_uuid == provider_uuid
|
||||
),
|
||||
persistence_model.LLMModel,
|
||||
workspace_uuid,
|
||||
)
|
||||
persistence = self.ap.persistence_mgr
|
||||
model_types = (
|
||||
(persistence_model.LLMModel, 'LLM', 'remove_llm_model'),
|
||||
(persistence_model.EmbeddingModel, 'Embedding', 'remove_embedding_model'),
|
||||
(persistence_model.RerankModel, 'Rerank', 'remove_rerank_model'),
|
||||
)
|
||||
if llm_result.first() is not None:
|
||||
raise ValueError('Cannot delete provider: LLM models still reference it')
|
||||
|
||||
embedding_result = await self.ap.persistence_mgr.execute_async(
|
||||
scope_statement(
|
||||
sqlalchemy.select(persistence_model.EmbeddingModel).where(
|
||||
persistence_model.EmbeddingModel.provider_uuid == provider_uuid
|
||||
),
|
||||
persistence_model.EmbeddingModel,
|
||||
workspace_uuid,
|
||||
deleted_models: list[tuple[str, list[str]]] = []
|
||||
async with persistence.tenant_uow(workspace_uuid):
|
||||
# Check ownership before touching children. Lock the provider on PostgreSQL
|
||||
# so concurrent model inserts cannot race the reference check/deletion.
|
||||
provider_result = await persistence.execute_async(
|
||||
scope_statement(
|
||||
sqlalchemy.select(persistence_model.ModelProvider.requester)
|
||||
.where(persistence_model.ModelProvider.uuid == provider_uuid)
|
||||
.with_for_update(),
|
||||
persistence_model.ModelProvider,
|
||||
workspace_uuid,
|
||||
)
|
||||
)
|
||||
)
|
||||
if embedding_result.first() is not None:
|
||||
raise ValueError('Cannot delete provider: Embedding models still reference it')
|
||||
provider = provider_result.first()
|
||||
if provider is None:
|
||||
raise WorkspaceNotFoundError('Provider not found')
|
||||
if self._system_requester_is_reserved(provider.requester):
|
||||
raise ValueError('LangBot Models is managed by Cloud and cannot be modified')
|
||||
|
||||
rerank_result = await self.ap.persistence_mgr.execute_async(
|
||||
scope_statement(
|
||||
sqlalchemy.select(persistence_model.RerankModel).where(
|
||||
persistence_model.RerankModel.provider_uuid == provider_uuid
|
||||
),
|
||||
persistence_model.RerankModel,
|
||||
workspace_uuid,
|
||||
for model_type, label, remover in model_types:
|
||||
result = await persistence.execute_async(
|
||||
scope_statement(
|
||||
sqlalchemy.select(model_type.uuid).where(model_type.provider_uuid == provider_uuid),
|
||||
model_type,
|
||||
workspace_uuid,
|
||||
)
|
||||
)
|
||||
model_uuids = list(result.scalars())
|
||||
if model_uuids and not cascade:
|
||||
raise ValueError(f'Cannot delete provider: {label} models still reference it')
|
||||
if model_uuids:
|
||||
# Model services have no pipeline/KB deletion side effects: they
|
||||
# delete the scoped row and evict its runtime cache. Defer eviction
|
||||
# here rather than calling those services before our commit.
|
||||
await persistence.execute_async(
|
||||
scope_statement(
|
||||
sqlalchemy.delete(model_type).where(model_type.provider_uuid == provider_uuid),
|
||||
model_type,
|
||||
workspace_uuid,
|
||||
)
|
||||
)
|
||||
deleted_models.append((remover, model_uuids))
|
||||
|
||||
# Explicit cleanup also works on legacy SQLite connections without FK
|
||||
# enforcement; never load or serialize the private credential payload.
|
||||
await persistence.execute_async(
|
||||
scope_statement(
|
||||
sqlalchemy.delete(persistence_model.CodexCredential).where(
|
||||
persistence_model.CodexCredential.provider_uuid == provider_uuid
|
||||
),
|
||||
persistence_model.CodexCredential,
|
||||
workspace_uuid,
|
||||
)
|
||||
)
|
||||
)
|
||||
if rerank_result.first() is not None:
|
||||
raise ValueError('Cannot delete provider: Rerank models still reference it')
|
||||
|
||||
result = await self.ap.persistence_mgr.execute_async(
|
||||
scope_statement(
|
||||
sqlalchemy.delete(persistence_model.ModelProvider).where(
|
||||
persistence_model.ModelProvider.uuid == provider_uuid
|
||||
),
|
||||
persistence_model.ModelProvider,
|
||||
workspace_uuid,
|
||||
result = await persistence.execute_async(
|
||||
scope_statement(
|
||||
sqlalchemy.delete(persistence_model.ModelProvider).where(
|
||||
persistence_model.ModelProvider.uuid == provider_uuid
|
||||
),
|
||||
persistence_model.ModelProvider,
|
||||
workspace_uuid,
|
||||
)
|
||||
)
|
||||
)
|
||||
if getattr(result, 'rowcount', None) == 0:
|
||||
raise WorkspaceNotFoundError('Provider not found')
|
||||
if result.rowcount == 0:
|
||||
raise WorkspaceNotFoundError('Provider not found')
|
||||
|
||||
await self.ap.model_mgr.remove_provider(context, provider_uuid)
|
||||
async def remove_runtime() -> None:
|
||||
async with persistence.tenant_scope(workspace_uuid):
|
||||
for remover, model_uuids in deleted_models:
|
||||
for model_uuid in model_uuids:
|
||||
await getattr(self.ap.model_mgr, remover)(context, model_uuid)
|
||||
# This also closes the requester's HTTP client; models go first.
|
||||
await self.ap.model_mgr.remove_provider(context, provider_uuid)
|
||||
|
||||
if persistence.current_session() is None:
|
||||
await remove_runtime()
|
||||
else:
|
||||
# A nested UoW has not committed yet. Reuse the rollback-cancelled gate
|
||||
# and detached context boundary instead of evicting uncommitted data.
|
||||
task = create_detached_task(
|
||||
remove_runtime(),
|
||||
after_commit_manager=persistence,
|
||||
workspace_uuid=workspace_uuid,
|
||||
)
|
||||
self._deletion_tasks.add(task)
|
||||
|
||||
def completed(task: asyncio.Task[None]) -> None:
|
||||
self._deletion_tasks.discard(task)
|
||||
if not task.cancelled() and task.exception() is not None:
|
||||
self.ap.logger.error('Failed to remove deleted provider runtime', exc_info=task.exception())
|
||||
|
||||
task.add_done_callback(completed)
|
||||
|
||||
async def get_provider_model_counts(self, context: TenantContext, provider_uuid: str) -> dict:
|
||||
"""Get count of models using this provider"""
|
||||
|
||||
@@ -4,6 +4,7 @@ import sqlalchemy
|
||||
import argon2
|
||||
import jwt
|
||||
import datetime
|
||||
import json
|
||||
import typing
|
||||
import asyncio
|
||||
import dataclasses
|
||||
@@ -12,10 +13,19 @@ import hashlib
|
||||
import secrets
|
||||
import time
|
||||
import uuid
|
||||
import webauthn
|
||||
from webauthn.helpers import bytes_to_base64url, base64url_to_bytes
|
||||
from webauthn.helpers.structs import (
|
||||
AuthenticatorSelectionCriteria,
|
||||
PublicKeyCredentialDescriptor,
|
||||
ResidentKeyRequirement,
|
||||
UserVerificationRequirement,
|
||||
)
|
||||
|
||||
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
|
||||
|
||||
from ....entity.persistence import user
|
||||
from ....entity.persistence import passkey
|
||||
from ....entity.persistence.workspace import MembershipRole, MembershipStatus, WorkspaceMembership
|
||||
from ....utils import constants
|
||||
from ....entity.errors import account as account_errors
|
||||
@@ -29,6 +39,9 @@ if typing.TYPE_CHECKING:
|
||||
_SPACE_OAUTH_STATE_MAX_ENTRIES = 4096
|
||||
_SPACE_OAUTH_STATE_HEAP_COMPACT_FLOOR = 64
|
||||
_SPACE_OAUTH_STATE_HEAP_MAX_MULTIPLIER = 4
|
||||
_PASSKEY_CHALLENGE_MAX_ENTRIES = 4096
|
||||
_PASSKEY_CHALLENGE_HEAP_COMPACT_FLOOR = 64
|
||||
_PASSKEY_CHALLENGE_HEAP_MAX_MULTIPLIER = 4
|
||||
|
||||
|
||||
class AccountExistsLoginRequiredError(ValueError):
|
||||
@@ -54,6 +67,17 @@ class SpaceOAuthStateConsumption:
|
||||
launch_workspace_uuid: str | None = None
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True, slots=True)
|
||||
class PasskeyChallengeData:
|
||||
challenge: bytes
|
||||
purpose: typing.Literal['register', 'auth']
|
||||
rp_id: str
|
||||
origin: str
|
||||
expires_at: float
|
||||
account_uuid: str | None = None
|
||||
user_email: str | None = None
|
||||
|
||||
|
||||
class UserService:
|
||||
ap: Application
|
||||
_create_user_lock: asyncio.Lock
|
||||
@@ -65,6 +89,9 @@ class UserService:
|
||||
self._space_oauth_state_lock = asyncio.Lock()
|
||||
self._space_oauth_states: dict[str, tuple[str, str | None, float, str | None]] = {}
|
||||
self._space_oauth_state_expiry_heap: list[tuple[float, str]] = []
|
||||
self._passkey_challenge_lock = asyncio.Lock()
|
||||
self._passkey_challenges: dict[str, PasskeyChallengeData] = {}
|
||||
self._passkey_challenge_expiry_heap: list[tuple[float, str]] = []
|
||||
|
||||
@staticmethod
|
||||
def _space_oauth_state_digest(state: str) -> str:
|
||||
@@ -850,3 +877,309 @@ class UserService:
|
||||
await self._update_space_provider_for_account(local_account, api_key)
|
||||
|
||||
return await self.get_user_by_email(space_email)
|
||||
|
||||
def _prune_passkey_challenges(self, now: float) -> None:
|
||||
while self._passkey_challenge_expiry_heap:
|
||||
expires_at, token = self._passkey_challenge_expiry_heap[0]
|
||||
entry = self._passkey_challenges.get(token)
|
||||
if entry is None or entry.expires_at != expires_at:
|
||||
heapq.heappop(self._passkey_challenge_expiry_heap)
|
||||
continue
|
||||
if expires_at > now:
|
||||
break
|
||||
heapq.heappop(self._passkey_challenge_expiry_heap)
|
||||
self._passkey_challenges.pop(token, None)
|
||||
|
||||
max_heap_entries = max(
|
||||
_PASSKEY_CHALLENGE_HEAP_COMPACT_FLOOR,
|
||||
len(self._passkey_challenges) * _PASSKEY_CHALLENGE_HEAP_MAX_MULTIPLIER,
|
||||
)
|
||||
if len(self._passkey_challenge_expiry_heap) > max_heap_entries:
|
||||
self._passkey_challenge_expiry_heap[:] = [
|
||||
(entry.expires_at, token) for token, entry in self._passkey_challenges.items()
|
||||
]
|
||||
heapq.heapify(self._passkey_challenge_expiry_heap)
|
||||
|
||||
async def issue_passkey_challenge(
|
||||
self,
|
||||
purpose: typing.Literal['register', 'auth'],
|
||||
rp_id: str,
|
||||
origin: str,
|
||||
*,
|
||||
account_uuid: str | None = None,
|
||||
user_email: str | None = None,
|
||||
ttl_seconds: int = 300,
|
||||
) -> tuple[str, bytes]:
|
||||
now = time.monotonic()
|
||||
challenge_bytes = secrets.token_bytes(32)
|
||||
challenge_token = secrets.token_urlsafe(32)
|
||||
expires_at = now + ttl_seconds
|
||||
|
||||
async with self._passkey_challenge_lock:
|
||||
self._prune_passkey_challenges(now)
|
||||
while len(self._passkey_challenges) >= _PASSKEY_CHALLENGE_MAX_ENTRIES:
|
||||
if not self._passkey_challenge_expiry_heap:
|
||||
break
|
||||
_, oldest_token = heapq.heappop(self._passkey_challenge_expiry_heap)
|
||||
self._passkey_challenges.pop(oldest_token, None)
|
||||
|
||||
self._passkey_challenges[challenge_token] = PasskeyChallengeData(
|
||||
challenge=challenge_bytes,
|
||||
purpose=purpose,
|
||||
rp_id=rp_id,
|
||||
origin=origin,
|
||||
expires_at=expires_at,
|
||||
account_uuid=account_uuid,
|
||||
user_email=user_email,
|
||||
)
|
||||
heapq.heappush(self._passkey_challenge_expiry_heap, (expires_at, challenge_token))
|
||||
|
||||
return challenge_token, challenge_bytes
|
||||
|
||||
async def consume_passkey_challenge(
|
||||
self,
|
||||
challenge_token: str,
|
||||
purpose: typing.Literal['register', 'auth'],
|
||||
) -> PasskeyChallengeData:
|
||||
now = time.monotonic()
|
||||
async with self._passkey_challenge_lock:
|
||||
self._prune_passkey_challenges(now)
|
||||
data = self._passkey_challenges.pop(challenge_token, None)
|
||||
|
||||
if data is None or data.expires_at < now:
|
||||
raise ValueError('Invalid or expired passkey challenge')
|
||||
if data.purpose != purpose:
|
||||
raise ValueError('Passkey challenge purpose mismatch')
|
||||
return data
|
||||
|
||||
async def get_user_passkeys(self, account_uuid: str) -> list[passkey.PasskeyCredential]:
|
||||
statement = (
|
||||
sqlalchemy.select(passkey.PasskeyCredential)
|
||||
.where(passkey.PasskeyCredential.account_uuid == account_uuid)
|
||||
.order_by(passkey.PasskeyCredential.created_at.desc())
|
||||
)
|
||||
async with self._session_factory()() as session:
|
||||
result = await session.scalars(statement)
|
||||
return list(result.all())
|
||||
|
||||
async def get_passkey_by_credential_id(self, credential_id: str) -> passkey.PasskeyCredential | None:
|
||||
statement = sqlalchemy.select(passkey.PasskeyCredential).where(
|
||||
passkey.PasskeyCredential.credential_id == credential_id
|
||||
)
|
||||
async with self._session_factory()() as session:
|
||||
return await session.scalar(statement)
|
||||
|
||||
async def get_passkey_by_uuid(self, passkey_uuid: str) -> passkey.PasskeyCredential | None:
|
||||
statement = sqlalchemy.select(passkey.PasskeyCredential).where(passkey.PasskeyCredential.uuid == passkey_uuid)
|
||||
async with self._session_factory()() as session:
|
||||
return await session.scalar(statement)
|
||||
|
||||
async def generate_passkey_registration_options(
|
||||
self,
|
||||
account_uuid: str,
|
||||
rp_id: str,
|
||||
origin: str,
|
||||
rp_name: str = 'LangBot',
|
||||
) -> tuple[dict[str, typing.Any], str]:
|
||||
account = await self.get_user_by_uuid(account_uuid)
|
||||
if account is None:
|
||||
raise ValueError('User not found')
|
||||
self._require_active_account(account)
|
||||
|
||||
challenge_token, challenge_bytes = await self.issue_passkey_challenge(
|
||||
purpose='register',
|
||||
rp_id=rp_id,
|
||||
origin=origin,
|
||||
account_uuid=account_uuid,
|
||||
user_email=account.user,
|
||||
)
|
||||
|
||||
existing_passkeys = await self.get_user_passkeys(account_uuid)
|
||||
exclude_credentials = [
|
||||
PublicKeyCredentialDescriptor(id=base64url_to_bytes(pk.credential_id)) for pk in existing_passkeys
|
||||
]
|
||||
|
||||
options = webauthn.generate_registration_options(
|
||||
rp_id=rp_id,
|
||||
rp_name=rp_name,
|
||||
user_name=account.user,
|
||||
user_id=account.uuid.encode('utf-8'),
|
||||
user_display_name=account.user,
|
||||
challenge=challenge_bytes,
|
||||
exclude_credentials=exclude_credentials or None,
|
||||
authenticator_selection=AuthenticatorSelectionCriteria(
|
||||
resident_key=ResidentKeyRequirement.PREFERRED,
|
||||
),
|
||||
)
|
||||
|
||||
options_dict = json.loads(webauthn.options_to_json(options))
|
||||
return options_dict, challenge_token
|
||||
|
||||
async def verify_and_save_passkey_registration(
|
||||
self,
|
||||
challenge_token: str,
|
||||
credential_data: dict[str, typing.Any] | str,
|
||||
name: str | None = None,
|
||||
) -> passkey.PasskeyCredential:
|
||||
challenge_data = await self.consume_passkey_challenge(challenge_token, 'register')
|
||||
if not challenge_data.account_uuid:
|
||||
raise ValueError('Registration challenge must be bound to an account')
|
||||
|
||||
verification = webauthn.verify_registration_response(
|
||||
credential=credential_data,
|
||||
expected_challenge=challenge_data.challenge,
|
||||
expected_rp_id=challenge_data.rp_id,
|
||||
expected_origin=challenge_data.origin,
|
||||
require_user_verification=False,
|
||||
)
|
||||
|
||||
cred_id_str = bytes_to_base64url(verification.credential_id)
|
||||
pub_key_str = bytes_to_base64url(verification.credential_public_key)
|
||||
|
||||
transports = None
|
||||
if isinstance(credential_data, dict):
|
||||
resp = credential_data.get('response', {})
|
||||
if isinstance(resp, dict) and 'transports' in resp:
|
||||
t_list = resp.get('transports')
|
||||
if isinstance(t_list, list):
|
||||
transports = ','.join(str(x) for x in t_list)
|
||||
|
||||
credential_name = (name or '').strip()
|
||||
if not credential_name:
|
||||
credential_name = f'Passkey ({datetime.datetime.now().strftime("%Y-%m-%d %H:%M")})'
|
||||
|
||||
record = passkey.PasskeyCredential(
|
||||
uuid=str(uuid.uuid4()),
|
||||
account_uuid=challenge_data.account_uuid,
|
||||
name=credential_name,
|
||||
credential_id=cred_id_str,
|
||||
public_key=pub_key_str,
|
||||
sign_count=verification.sign_count,
|
||||
aaguid=verification.aaguid,
|
||||
transports=transports,
|
||||
backed_up=verification.credential_backed_up,
|
||||
)
|
||||
|
||||
async with self._session_factory()() as session:
|
||||
async with session.begin():
|
||||
session.add(record)
|
||||
await session.flush()
|
||||
await session.refresh(record)
|
||||
return record
|
||||
|
||||
async def generate_passkey_authentication_options(
|
||||
self,
|
||||
rp_id: str,
|
||||
origin: str,
|
||||
email: str | None = None,
|
||||
) -> tuple[dict[str, typing.Any], str]:
|
||||
challenge_token, challenge_bytes = await self.issue_passkey_challenge(
|
||||
purpose='auth',
|
||||
rp_id=rp_id,
|
||||
origin=origin,
|
||||
user_email=email,
|
||||
)
|
||||
|
||||
allow_credentials: list[PublicKeyCredentialDescriptor] | None = None
|
||||
if email:
|
||||
user_obj = await self.get_user_by_email(email)
|
||||
if user_obj:
|
||||
user_passkeys = await self.get_user_passkeys(user_obj.uuid)
|
||||
if user_passkeys:
|
||||
allow_credentials = [
|
||||
PublicKeyCredentialDescriptor(id=base64url_to_bytes(pk.credential_id)) for pk in user_passkeys
|
||||
]
|
||||
|
||||
options = webauthn.generate_authentication_options(
|
||||
rp_id=rp_id,
|
||||
challenge=challenge_bytes,
|
||||
allow_credentials=allow_credentials or None,
|
||||
user_verification=UserVerificationRequirement.PREFERRED,
|
||||
)
|
||||
|
||||
options_dict = json.loads(webauthn.options_to_json(options))
|
||||
return options_dict, challenge_token
|
||||
|
||||
async def verify_passkey_authentication(
|
||||
self,
|
||||
challenge_token: str,
|
||||
credential_data: dict[str, typing.Any] | str,
|
||||
) -> tuple[str, user.User]:
|
||||
challenge_data = await self.consume_passkey_challenge(challenge_token, 'auth')
|
||||
|
||||
raw_id = credential_data.get('id') if isinstance(credential_data, dict) else None
|
||||
if not raw_id:
|
||||
raise ValueError('Missing credential id')
|
||||
|
||||
stored_credential = await self.get_passkey_by_credential_id(raw_id)
|
||||
if stored_credential is None:
|
||||
raise ValueError('Passkey credential not recognized')
|
||||
|
||||
user_obj = await self.get_user_by_uuid(stored_credential.account_uuid)
|
||||
if user_obj is None:
|
||||
raise ValueError('Associated user not found')
|
||||
self._require_active_account(user_obj)
|
||||
|
||||
verification = webauthn.verify_authentication_response(
|
||||
credential=credential_data,
|
||||
expected_challenge=challenge_data.challenge,
|
||||
expected_rp_id=challenge_data.rp_id,
|
||||
expected_origin=challenge_data.origin,
|
||||
credential_public_key=base64url_to_bytes(stored_credential.public_key),
|
||||
credential_current_sign_count=stored_credential.sign_count,
|
||||
require_user_verification=False,
|
||||
)
|
||||
|
||||
async with self._session_factory()() as session:
|
||||
async with session.begin():
|
||||
record = await session.scalar(
|
||||
sqlalchemy.select(passkey.PasskeyCredential).where(
|
||||
passkey.PasskeyCredential.id == stored_credential.id
|
||||
)
|
||||
)
|
||||
if record:
|
||||
record.sign_count = verification.new_sign_count
|
||||
record.last_used_at = datetime.datetime.now()
|
||||
record.backed_up = verification.credential_backed_up
|
||||
|
||||
token = await self.generate_jwt_token(user_obj)
|
||||
return token, user_obj
|
||||
|
||||
async def rename_user_passkey(
|
||||
self,
|
||||
account_uuid: str,
|
||||
passkey_uuid: str,
|
||||
new_name: str,
|
||||
) -> passkey.PasskeyCredential | None:
|
||||
async with self._session_factory()() as session:
|
||||
async with session.begin():
|
||||
record = await session.scalar(
|
||||
sqlalchemy.select(passkey.PasskeyCredential).where(
|
||||
passkey.PasskeyCredential.uuid == passkey_uuid,
|
||||
passkey.PasskeyCredential.account_uuid == account_uuid,
|
||||
)
|
||||
)
|
||||
if record is None:
|
||||
return None
|
||||
record.name = new_name
|
||||
await session.flush()
|
||||
await session.refresh(record)
|
||||
return record
|
||||
|
||||
async def delete_user_passkey(
|
||||
self,
|
||||
account_uuid: str,
|
||||
passkey_uuid: str,
|
||||
) -> bool:
|
||||
async with self._session_factory()() as session:
|
||||
async with session.begin():
|
||||
record = await session.scalar(
|
||||
sqlalchemy.select(passkey.PasskeyCredential).where(
|
||||
passkey.PasskeyCredential.uuid == passkey_uuid,
|
||||
passkey.PasskeyCredential.account_uuid == account_uuid,
|
||||
)
|
||||
)
|
||||
if record is None:
|
||||
return False
|
||||
await session.delete(record)
|
||||
return True
|
||||
|
||||
@@ -1,9 +1,18 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import secrets
|
||||
|
||||
from .. import stage, app
|
||||
|
||||
# This stage runs before SetupLoggerStage, so ap.logger is still None here;
|
||||
# the module logger falls back to the stderr lastResort handler.
|
||||
_logger = logging.getLogger(__name__)
|
||||
|
||||
# 32 symbols without 0/O or 1/I; eight independent draws provide 40 random bits.
|
||||
_RECOVERY_KEY_ALPHABET = '23456789ABCDEFGHJKLMNPQRSTUVWXYZ'
|
||||
_RECOVERY_KEY_LENGTH = 8
|
||||
|
||||
|
||||
@stage.stage_class('GenKeysStage')
|
||||
class GenKeysStage(stage.BootingStage):
|
||||
@@ -20,5 +29,15 @@ class GenKeysStage(stage.BootingStage):
|
||||
ap.instance_config.data['system']['recovery_key'] = ''
|
||||
|
||||
if not ap.instance_config.data['system']['recovery_key']:
|
||||
ap.instance_config.data['system']['recovery_key'] = secrets.token_hex(3).upper()
|
||||
# Keep recovery practical to type. Security also requires the reset
|
||||
# endpoint's concurrency-safe quota (five admissions per 15 minutes).
|
||||
ap.instance_config.data['system']['recovery_key'] = ''.join(
|
||||
secrets.choice(_RECOVERY_KEY_ALPHABET) for _ in range(_RECOVERY_KEY_LENGTH)
|
||||
)
|
||||
await ap.instance_config.dump_config()
|
||||
elif len(ap.instance_config.data['system']['recovery_key']) < _RECOVERY_KEY_LENGTH:
|
||||
_logger.warning(
|
||||
'Low-entropy legacy recovery key detected (length < 8); '
|
||||
'regenerate system.recovery_key in the configuration file '
|
||||
'with a strong random value (#2392)'
|
||||
)
|
||||
|
||||
@@ -33,6 +33,28 @@ class ModelProvider(Base):
|
||||
)
|
||||
|
||||
|
||||
class CodexCredential(Base):
|
||||
"""Server-only OAuth state. Never joined into provider/model serialization."""
|
||||
|
||||
__tablename__ = 'codex_credentials'
|
||||
|
||||
provider_uuid = sqlalchemy.Column(sqlalchemy.String(255), primary_key=True)
|
||||
workspace_uuid = sqlalchemy.Column(sqlalchemy.String(36), nullable=False)
|
||||
payload = sqlalchemy.Column(sqlalchemy.JSON, nullable=False, default=dict)
|
||||
version = sqlalchemy.Column(sqlalchemy.Integer, nullable=False, default=0)
|
||||
lease_owner = sqlalchemy.Column(sqlalchemy.String(64), nullable=True)
|
||||
lease_until = sqlalchemy.Column(sqlalchemy.Float, nullable=False, default=0)
|
||||
__table_args__ = (
|
||||
sqlalchemy.ForeignKeyConstraint(
|
||||
['workspace_uuid', 'provider_uuid'],
|
||||
['model_providers.workspace_uuid', 'model_providers.uuid'],
|
||||
name='fk_codex_credentials_workspace_provider',
|
||||
ondelete='CASCADE',
|
||||
),
|
||||
sqlalchemy.Index('ix_codex_credentials_workspace', 'workspace_uuid'),
|
||||
)
|
||||
|
||||
|
||||
class LLMModel(Base):
|
||||
"""LLM model"""
|
||||
|
||||
|
||||
@@ -111,8 +111,8 @@ class MonitoringSession(Base):
|
||||
sqlalchemy.ForeignKey('workspaces.uuid', ondelete='CASCADE'),
|
||||
primary_key=True,
|
||||
)
|
||||
bot_id = sqlalchemy.Column(sqlalchemy.String(255), primary_key=True, index=True)
|
||||
session_id = sqlalchemy.Column(sqlalchemy.String(255), primary_key=True)
|
||||
bot_id = sqlalchemy.Column(sqlalchemy.String(255), nullable=False, index=True)
|
||||
bot_name = sqlalchemy.Column(sqlalchemy.String(255), nullable=False)
|
||||
pipeline_id = sqlalchemy.Column(sqlalchemy.String(255), nullable=False, index=True)
|
||||
pipeline_name = sqlalchemy.Column(sqlalchemy.String(255), nullable=False)
|
||||
|
||||
@@ -0,0 +1,38 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import uuid as uuid_lib
|
||||
|
||||
import sqlalchemy
|
||||
|
||||
from .base import Base
|
||||
|
||||
|
||||
class PasskeyCredential(Base):
|
||||
__tablename__ = 'passkey_credentials'
|
||||
|
||||
id = sqlalchemy.Column(sqlalchemy.Integer, primary_key=True, autoincrement=True)
|
||||
uuid = sqlalchemy.Column(
|
||||
sqlalchemy.String(36),
|
||||
nullable=False,
|
||||
default=lambda: str(uuid_lib.uuid4()),
|
||||
)
|
||||
account_uuid = sqlalchemy.Column(
|
||||
sqlalchemy.String(36),
|
||||
sqlalchemy.ForeignKey('users.uuid', ondelete='CASCADE'),
|
||||
nullable=False,
|
||||
)
|
||||
name = sqlalchemy.Column(sqlalchemy.String(255), nullable=False)
|
||||
credential_id = sqlalchemy.Column(sqlalchemy.String(255), nullable=False)
|
||||
public_key = sqlalchemy.Column(sqlalchemy.Text, nullable=False)
|
||||
sign_count = sqlalchemy.Column(sqlalchemy.Integer, nullable=False, default=0)
|
||||
aaguid = sqlalchemy.Column(sqlalchemy.String(64), nullable=True)
|
||||
transports = sqlalchemy.Column(sqlalchemy.String(255), nullable=True)
|
||||
backed_up = sqlalchemy.Column(sqlalchemy.Boolean, nullable=False, default=False)
|
||||
created_at = sqlalchemy.Column(sqlalchemy.DateTime, nullable=False, server_default=sqlalchemy.func.now())
|
||||
last_used_at = sqlalchemy.Column(sqlalchemy.DateTime, nullable=True)
|
||||
|
||||
__table_args__ = (
|
||||
sqlalchemy.Index('uq_passkey_credentials_uuid', 'uuid', unique=True),
|
||||
sqlalchemy.Index('uq_passkey_credentials_cred_id', 'credential_id', unique=True),
|
||||
sqlalchemy.Index('ix_passkey_credentials_account', 'account_uuid'),
|
||||
)
|
||||
@@ -0,0 +1,48 @@
|
||||
"""Add isolated server-only Codex credentials and tenant RLS.
|
||||
|
||||
Revision ID: 0022_codex_credentials
|
||||
Revises: 0021_merge_reasoning_config
|
||||
"""
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
revision = '0022_codex_credentials'
|
||||
down_revision = '0021_merge_reasoning_config'
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
conn = op.get_bind()
|
||||
# Fresh startup creates ORM metadata before running Alembic.
|
||||
if 'codex_credentials' not in sa.inspect(conn).get_table_names():
|
||||
op.create_table(
|
||||
'codex_credentials',
|
||||
sa.Column('provider_uuid', sa.String(255), primary_key=True),
|
||||
sa.Column('workspace_uuid', sa.String(36), nullable=False),
|
||||
sa.Column('payload', sa.JSON(), nullable=False),
|
||||
sa.Column('version', sa.Integer(), nullable=False),
|
||||
sa.Column('lease_owner', sa.String(64), nullable=True),
|
||||
sa.Column('lease_until', sa.Float(), nullable=False),
|
||||
sa.ForeignKeyConstraint(
|
||||
['workspace_uuid', 'provider_uuid'],
|
||||
['model_providers.workspace_uuid', 'model_providers.uuid'],
|
||||
name='fk_codex_credentials_workspace_provider',
|
||||
ondelete='CASCADE',
|
||||
),
|
||||
)
|
||||
op.create_index('ix_codex_credentials_workspace', 'codex_credentials', ['workspace_uuid'])
|
||||
if conn.dialect.name == 'postgresql':
|
||||
op.execute('ALTER TABLE codex_credentials ENABLE ROW LEVEL SECURITY')
|
||||
op.execute('ALTER TABLE codex_credentials FORCE ROW LEVEL SECURITY')
|
||||
op.execute('DROP POLICY IF EXISTS langbot_workspace_isolation ON codex_credentials')
|
||||
expression = "workspace_uuid::text = NULLIF(current_setting('langbot.workspace_uuid', true), '')"
|
||||
op.execute(
|
||||
f'CREATE POLICY langbot_workspace_isolation ON codex_credentials '
|
||||
f'FOR ALL USING ({expression}) WITH CHECK ({expression})'
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_table('codex_credentials')
|
||||
@@ -0,0 +1,104 @@
|
||||
"""Scope monitoring sessions by bot without changing runtime session IDs.
|
||||
|
||||
Revision ID: 0023_bot_scoped_sessions
|
||||
Revises: 0022_codex_credentials
|
||||
"""
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
from sqlalchemy.dialects import postgresql, sqlite
|
||||
|
||||
revision = '0023_bot_scoped_sessions'
|
||||
down_revision = '0022_codex_credentials'
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
_TABLE = 'monitoring_sessions'
|
||||
_KEY = ['workspace_uuid', 'bot_id', 'session_id']
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
conn = op.get_bind()
|
||||
inspector = sa.inspect(conn)
|
||||
if _TABLE not in inspector.get_table_names():
|
||||
return
|
||||
pk = inspector.get_pk_constraint(_TABLE)
|
||||
if pk['constrained_columns'] == _KEY:
|
||||
return
|
||||
# PostgreSQL alters in place, retaining indexes, grants, policies and RLS.
|
||||
# SQLite batch reflection retains all existing indexes and foreign keys.
|
||||
with op.batch_alter_table(_TABLE, naming_convention={'pk': 'pk_%(table_name)s'}) as batch:
|
||||
batch.drop_constraint(pk['name'] or f'pk_{_TABLE}', type_='primary')
|
||||
batch.create_primary_key(f'pk_{_TABLE}', _KEY)
|
||||
|
||||
metadata = sa.MetaData()
|
||||
sessions = sa.Table(_TABLE, metadata, autoload_with=conn)
|
||||
messages = sa.Table('monitoring_messages', metadata, autoload_with=conn)
|
||||
m = messages.c
|
||||
collisions = (
|
||||
sa.select(m.workspace_uuid, m.session_id)
|
||||
.group_by(m.workspace_uuid, m.session_id)
|
||||
.having(sa.func.count(sa.distinct(m.bot_id)) > 1)
|
||||
.subquery()
|
||||
)
|
||||
partition = [m.workspace_uuid, m.bot_id, m.session_id]
|
||||
# Repair only demonstrable collisions. Retention may have removed earlier
|
||||
# evidence; these summaries describe surviving messages, never invented text.
|
||||
ranked = (
|
||||
sa.select(
|
||||
*[m[name] for name in _KEY],
|
||||
m.bot_name,
|
||||
m.pipeline_id,
|
||||
m.pipeline_name,
|
||||
m.platform,
|
||||
m.user_id,
|
||||
m.user_name,
|
||||
sa.func.sum(sa.case((sa.or_(m.role == 'user', m.role.is_(None)), 1), else_=0))
|
||||
.over(partition_by=partition)
|
||||
.label('message_count'),
|
||||
sa.func.min(m.timestamp).over(partition_by=partition).label('start_time'),
|
||||
sa.func.max(m.timestamp).over(partition_by=partition).label('last_activity'),
|
||||
sa.func.row_number().over(partition_by=partition, order_by=[m.timestamp.desc(), m.id.desc()]).label('rank'),
|
||||
)
|
||||
.join(
|
||||
collisions,
|
||||
sa.and_(m.workspace_uuid == collisions.c.workspace_uuid, m.session_id == collisions.c.session_id),
|
||||
)
|
||||
.subquery()
|
||||
)
|
||||
columns = _KEY + [
|
||||
'bot_name',
|
||||
'pipeline_id',
|
||||
'pipeline_name',
|
||||
'platform',
|
||||
'user_id',
|
||||
'user_name',
|
||||
'message_count',
|
||||
'start_time',
|
||||
'last_activity',
|
||||
'is_active',
|
||||
]
|
||||
select = sa.select(*[ranked.c[name] for name in columns[:-1]], sa.literal(True)).where(ranked.c.rank == 1)
|
||||
insert = postgresql.insert if conn.dialect.name == 'postgresql' else sqlite.insert
|
||||
statement = insert(sessions).from_select(columns, select)
|
||||
conn.execute(
|
||||
statement.on_conflict_do_update(
|
||||
index_elements=_KEY,
|
||||
set_={name: statement.excluded[name] for name in columns if name not in _KEY and name != 'is_active'},
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
conn = op.get_bind()
|
||||
if _TABLE not in sa.inspect(conn).get_table_names():
|
||||
return
|
||||
collisions = conn.execute(
|
||||
sa.text('SELECT 1 FROM monitoring_sessions GROUP BY workspace_uuid, session_id HAVING COUNT(*) > 1 LIMIT 1')
|
||||
).first()
|
||||
if collisions:
|
||||
raise RuntimeError('Cannot downgrade bot-scoped sessions without losing colliding bot records')
|
||||
pk = sa.inspect(conn).get_pk_constraint(_TABLE)
|
||||
with op.batch_alter_table(_TABLE, naming_convention={'pk': 'pk_%(table_name)s'}) as batch:
|
||||
batch.drop_constraint(pk['name'] or f'pk_{_TABLE}', type_='primary')
|
||||
batch.create_primary_key(f'pk_{_TABLE}', ['workspace_uuid', 'session_id'])
|
||||
@@ -0,0 +1,54 @@
|
||||
"""add passkey credentials table
|
||||
|
||||
Revision ID: 0024_passkey_credentials
|
||||
Revises: 0023_bot_scoped_sessions
|
||||
Create Date: 2026-09-12
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
revision = '0024_passkey_credentials'
|
||||
down_revision = '0023_bot_scoped_sessions'
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
_TABLE_NAME = 'passkey_credentials'
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
conn = op.get_bind()
|
||||
existing_tables = set(sa.inspect(conn).get_table_names())
|
||||
if _TABLE_NAME not in existing_tables:
|
||||
op.create_table(
|
||||
_TABLE_NAME,
|
||||
sa.Column('id', sa.Integer(), primary_key=True, autoincrement=True),
|
||||
sa.Column('uuid', sa.String(36), nullable=False),
|
||||
sa.Column(
|
||||
'account_uuid',
|
||||
sa.String(36),
|
||||
sa.ForeignKey('users.uuid', ondelete='CASCADE'),
|
||||
nullable=False,
|
||||
),
|
||||
sa.Column('name', sa.String(255), nullable=False),
|
||||
sa.Column('credential_id', sa.String(255), nullable=False),
|
||||
sa.Column('public_key', sa.Text(), nullable=False),
|
||||
sa.Column('sign_count', sa.Integer(), nullable=False, server_default='0'),
|
||||
sa.Column('aaguid', sa.String(64), nullable=True),
|
||||
sa.Column('transports', sa.String(255), nullable=True),
|
||||
sa.Column('backed_up', sa.Boolean(), nullable=False, server_default='0'),
|
||||
sa.Column('created_at', sa.DateTime(), nullable=False, server_default=sa.func.now()),
|
||||
sa.Column('last_used_at', sa.DateTime(), nullable=True),
|
||||
)
|
||||
op.create_index('uq_passkey_credentials_uuid', _TABLE_NAME, ['uuid'], unique=True)
|
||||
op.create_index('uq_passkey_credentials_cred_id', _TABLE_NAME, ['credential_id'], unique=True)
|
||||
op.create_index('ix_passkey_credentials_account', _TABLE_NAME, ['account_uuid'], unique=False)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_index('ix_passkey_credentials_account', table_name=_TABLE_NAME)
|
||||
op.drop_index('uq_passkey_credentials_cred_id', table_name=_TABLE_NAME)
|
||||
op.drop_index('uq_passkey_credentials_uuid', table_name=_TABLE_NAME)
|
||||
op.drop_table(_TABLE_NAME)
|
||||
@@ -62,6 +62,8 @@ _ALEMBIC_TENANT_TABLES = {
|
||||
'binary_storages',
|
||||
'mcp_servers',
|
||||
'model_providers',
|
||||
'codex_credentials',
|
||||
'passkey_credentials',
|
||||
'llm_models',
|
||||
'embedding_models',
|
||||
'rerank_models',
|
||||
|
||||
@@ -51,6 +51,7 @@ TENANT_TABLE_COLUMNS: dict[str, str] = {
|
||||
'binary_storages': 'workspace_uuid',
|
||||
'mcp_servers': 'workspace_uuid',
|
||||
'model_providers': 'workspace_uuid',
|
||||
'codex_credentials': 'workspace_uuid',
|
||||
'llm_models': 'workspace_uuid',
|
||||
'embedding_models': 'workspace_uuid',
|
||||
'rerank_models': 'workspace_uuid',
|
||||
@@ -206,6 +207,8 @@ _SYNC_PROXY_CAPABILITY: contextvars.ContextVar[_ScopedSessionGuardState | None]
|
||||
_ALLOWED_SCOPED_BUILTIN_FUNCTION_TYPES = {
|
||||
'coalesce': sqlalchemy.sql.functions.coalesce,
|
||||
'count': sqlalchemy.sql.functions.count,
|
||||
'min': sqlalchemy.sql.functions.min,
|
||||
'max': sqlalchemy.sql.functions.max,
|
||||
'now': sqlalchemy.sql.functions.now,
|
||||
'sum': sqlalchemy.sql.functions.sum,
|
||||
}
|
||||
@@ -852,7 +855,30 @@ class TenantScopedAsyncSession(sqlalchemy_asyncio.AsyncSession):
|
||||
self._require_owner_task()
|
||||
self._enter_internal_access()
|
||||
try:
|
||||
await transaction.commit()
|
||||
# Retain the actual connection before COMMIT: after a failed SQLite
|
||||
# COMMIT the logical transaction is inactive, but the DBAPI writer
|
||||
# can still hold PENDING/RESERVED locks. Session.close()/rollback()
|
||||
# alone can then return that poisoned connection to the pool.
|
||||
connection = await super().connection()
|
||||
try:
|
||||
await transaction.commit()
|
||||
except BaseException as exc:
|
||||
cleanup = asyncio.create_task(connection.invalidate())
|
||||
# Invalidation does not access the task-owned Session. Shield
|
||||
# physical cleanup, including against repeated cancellation,
|
||||
# before the owner closes the Session and releases its scope.
|
||||
while not cleanup.done():
|
||||
try:
|
||||
await asyncio.shield(cleanup)
|
||||
except asyncio.CancelledError:
|
||||
continue
|
||||
except BaseException:
|
||||
break
|
||||
try:
|
||||
cleanup.result()
|
||||
except BaseException as cleanup_error:
|
||||
exc.add_note(f'Failed to invalidate transaction connection: {cleanup_error!r}')
|
||||
raise
|
||||
finally:
|
||||
self._exit_internal_access()
|
||||
|
||||
@@ -1369,6 +1395,7 @@ class TenantUnitOfWork:
|
||||
state.mark_rollback_only(exc_value)
|
||||
rollback_only = state.rollback_only
|
||||
committed = False
|
||||
transaction_error: BaseException | None = None
|
||||
try:
|
||||
if exc_type is None and not rollback_only:
|
||||
await typing.cast(TenantScopedAsyncSession, session)._commit_owned_transaction(
|
||||
@@ -1381,6 +1408,9 @@ class TenantUnitOfWork:
|
||||
_UOW_SESSION_CONTROL_CAPABILITY,
|
||||
transaction,
|
||||
)
|
||||
except BaseException as exc:
|
||||
transaction_error = exc
|
||||
raise
|
||||
finally:
|
||||
try:
|
||||
if self._active_transaction is not None and self._context_token is not None:
|
||||
@@ -1388,6 +1418,10 @@ class TenantUnitOfWork:
|
||||
await typing.cast(TenantScopedAsyncSession, session)._close_owned_session(
|
||||
_UOW_SESSION_CONTROL_CAPABILITY
|
||||
)
|
||||
except BaseException as cleanup_error:
|
||||
if transaction_error is None:
|
||||
raise
|
||||
transaction_error.add_note(f'Failed to close transaction Session: {cleanup_error!r}')
|
||||
finally:
|
||||
if self._database_operation_token is not None:
|
||||
_DATABASE_OPERATION_TRANSACTION.reset(self._database_operation_token)
|
||||
|
||||
@@ -79,6 +79,7 @@ class MonitoringHelper:
|
||||
session_updated = await ap.monitoring_service.update_session_activity(
|
||||
get_query_execution_context(query),
|
||||
session_id,
|
||||
bot_id=bot_id,
|
||||
pipeline_id=pipeline_id,
|
||||
pipeline_name=pipeline_name,
|
||||
)
|
||||
|
||||
@@ -682,8 +682,8 @@ class MatrixAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
|
||||
lines.append(f'[{bridge.user_id}] 跳过(未配置登录命令或无DM房间)')
|
||||
continue
|
||||
|
||||
# Use configured logout command, fallback to deriving from login command
|
||||
logout_cmd = bridge.logout_command or bridge.login_command.replace('login', 'logout')
|
||||
# Use configured logout command, fallback to deriving from login command
|
||||
logout_cmd = bridge.logout_command or bridge.login_command.replace('login', 'logout')
|
||||
lines.append(f'[{bridge.user_id}] 发送 "{logout_cmd}"...')
|
||||
|
||||
# Cancel existing tasks
|
||||
|
||||
@@ -0,0 +1,375 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import re
|
||||
import typing
|
||||
from urllib.parse import urlsplit, urlunsplit
|
||||
|
||||
import aiohttp
|
||||
|
||||
import langbot_plugin.api.definition.abstract.platform.adapter as abstract_platform_adapter
|
||||
import langbot_plugin.api.definition.abstract.platform.event_logger as abstract_platform_logger
|
||||
import langbot_plugin.api.entities.builtin.platform.entities as platform_entities
|
||||
import langbot_plugin.api.entities.builtin.platform.events as platform_events
|
||||
import langbot_plugin.api.entities.builtin.platform.message as platform_message
|
||||
|
||||
|
||||
_MATTERMOST_MAX_POST_LENGTH = 16_383
|
||||
_MENTION_BOUNDARY = r'(?<![\w.-])@{username}(?![\w.-])'
|
||||
|
||||
|
||||
def _normalize_server_url(server_url: str) -> str:
|
||||
"""Return a validated Mattermost server URL without a trailing slash."""
|
||||
|
||||
url = server_url.strip().rstrip('/')
|
||||
parsed = urlsplit(url)
|
||||
if parsed.scheme not in {'http', 'https'} or not parsed.netloc:
|
||||
raise ValueError('Mattermost server_url must be an absolute HTTP(S) URL')
|
||||
return url
|
||||
|
||||
|
||||
def _websocket_url(server_url: str) -> str:
|
||||
parsed = urlsplit(server_url)
|
||||
scheme = 'wss' if parsed.scheme == 'https' else 'ws'
|
||||
return urlunsplit((scheme, parsed.netloc, f'{parsed.path}/api/v4/websocket', '', ''))
|
||||
|
||||
|
||||
class MattermostMessageConverter(abstract_platform_adapter.AbstractMessageConverter):
|
||||
"""Translate Mattermost post text to and from LangBot message chains."""
|
||||
|
||||
@staticmethod
|
||||
async def yiri2target(message_chain: platform_message.MessageChain) -> str:
|
||||
parts: list[str] = []
|
||||
for component in message_chain:
|
||||
if isinstance(component, platform_message.Plain):
|
||||
parts.append(component.text)
|
||||
elif isinstance(component, platform_message.Image) and component.url:
|
||||
# Mattermost renders image URLs in Markdown messages.
|
||||
parts.append(component.url)
|
||||
elif isinstance(component, platform_message.File) and component.url:
|
||||
parts.append(component.url)
|
||||
return ''.join(parts)
|
||||
|
||||
@staticmethod
|
||||
async def target2yiri(post: dict, bot_username: str) -> platform_message.MessageChain:
|
||||
text = str(post.get('message') or '')
|
||||
components: list[typing.Any] = [
|
||||
platform_message.Source(
|
||||
id=str(post.get('id') or ''),
|
||||
time=float(post.get('create_at') or 0) / 1000,
|
||||
)
|
||||
]
|
||||
if bot_username:
|
||||
mention_pattern = re.compile(_MENTION_BOUNDARY.format(username=re.escape(bot_username)), re.IGNORECASE)
|
||||
if mention_pattern.search(text):
|
||||
components.append(platform_message.At(target=bot_username))
|
||||
text = mention_pattern.sub('', text).strip()
|
||||
if text:
|
||||
components.append(platform_message.Plain(text=text))
|
||||
return platform_message.MessageChain(components)
|
||||
|
||||
|
||||
class MattermostEventConverter(abstract_platform_adapter.AbstractEventConverter):
|
||||
@staticmethod
|
||||
async def yiri2target(event: platform_events.MessageEvent) -> dict:
|
||||
return event.source_platform_object
|
||||
|
||||
@staticmethod
|
||||
async def target2yiri(
|
||||
post: dict,
|
||||
channel: dict,
|
||||
sender_name: str,
|
||||
bot_username: str,
|
||||
) -> platform_events.MessageEvent:
|
||||
message_chain = await MattermostMessageConverter.target2yiri(post, bot_username)
|
||||
timestamp = float(post.get('create_at') or 0) / 1000
|
||||
sender_id = str(post.get('user_id') or '')
|
||||
channel_type = channel.get('type')
|
||||
|
||||
if channel_type == 'D':
|
||||
return platform_events.FriendMessage(
|
||||
sender=platform_entities.Friend(id=sender_id, nickname=sender_name or sender_id, remark=''),
|
||||
message_chain=message_chain,
|
||||
time=timestamp,
|
||||
source_platform_object={'post': post, 'channel': channel},
|
||||
)
|
||||
|
||||
return platform_events.GroupMessage(
|
||||
sender=platform_entities.GroupMember(
|
||||
id=sender_id,
|
||||
member_name=sender_name or sender_id,
|
||||
permission=platform_entities.Permission.Member,
|
||||
group=platform_entities.Group(
|
||||
id=str(post.get('channel_id') or ''),
|
||||
name=str(channel.get('display_name') or channel.get('name') or post.get('channel_id') or ''),
|
||||
permission=platform_entities.Permission.Member,
|
||||
),
|
||||
special_title='',
|
||||
),
|
||||
message_chain=message_chain,
|
||||
time=timestamp,
|
||||
source_platform_object={'post': post, 'channel': channel},
|
||||
)
|
||||
|
||||
|
||||
class MattermostAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
|
||||
"""Mattermost Bot Account adapter using the v4 REST and WebSocket APIs."""
|
||||
|
||||
server_url: str = ''
|
||||
access_token: str = ''
|
||||
session: aiohttp.ClientSession | None = None
|
||||
listeners: dict[typing.Type[platform_events.Event], typing.Callable] = {}
|
||||
channel_cache: dict[str, dict] = {}
|
||||
stream_post_ids: dict[str, str] = {}
|
||||
bot_username: str = ''
|
||||
_running: bool = False
|
||||
|
||||
message_converter: MattermostMessageConverter = MattermostMessageConverter()
|
||||
event_converter: MattermostEventConverter = MattermostEventConverter()
|
||||
|
||||
def __init__(self, config: dict, logger: abstract_platform_logger.AbstractEventLogger):
|
||||
server_url = _normalize_server_url(str(config.get('server_url') or ''))
|
||||
access_token = str(config.get('access_token') or '').strip()
|
||||
if not access_token:
|
||||
raise ValueError('Mattermost adapter requires an access_token')
|
||||
|
||||
super().__init__(
|
||||
config=config,
|
||||
logger=logger,
|
||||
server_url=server_url,
|
||||
access_token=access_token,
|
||||
bot_account_id='',
|
||||
session=None,
|
||||
listeners={},
|
||||
channel_cache={},
|
||||
stream_post_ids={},
|
||||
bot_username='',
|
||||
_running=False,
|
||||
)
|
||||
|
||||
async def _get_session(self) -> aiohttp.ClientSession:
|
||||
if self.session is None or self.session.closed:
|
||||
self.session = aiohttp.ClientSession(
|
||||
headers={'Authorization': f'Bearer {self.access_token}'},
|
||||
raise_for_status=False,
|
||||
)
|
||||
return self.session
|
||||
|
||||
async def _api_request(
|
||||
self,
|
||||
method: str,
|
||||
path: str,
|
||||
*,
|
||||
payload: dict | None = None,
|
||||
) -> dict:
|
||||
session = await self._get_session()
|
||||
async with session.request(method, f'{self.server_url}/api/v4{path}', json=payload) as response:
|
||||
raw_body = await response.text()
|
||||
if response.status >= 400:
|
||||
# Mattermost returns a useful JSON error, but never include request headers/tokens in errors.
|
||||
try:
|
||||
error = json.loads(raw_body).get('message', raw_body)
|
||||
except json.JSONDecodeError:
|
||||
error = raw_body
|
||||
raise RuntimeError(f'Mattermost API {method} {path} failed ({response.status}): {error}')
|
||||
if not raw_body:
|
||||
return {}
|
||||
return json.loads(raw_body)
|
||||
|
||||
async def _load_bot_identity(self) -> None:
|
||||
user = await self._api_request('GET', '/users/me')
|
||||
self.bot_account_id = str(user.get('id') or '')
|
||||
self.bot_username = str(user.get('username') or '')
|
||||
if not self.bot_account_id:
|
||||
raise RuntimeError('Mattermost API did not return a bot user ID')
|
||||
|
||||
async def _get_channel(self, channel_id: str) -> dict:
|
||||
if channel_id not in self.channel_cache:
|
||||
self.channel_cache[channel_id] = await self._api_request('GET', f'/channels/{channel_id}')
|
||||
return self.channel_cache[channel_id]
|
||||
|
||||
async def _post_message(self, channel_id: str, text: str, root_id: str = '') -> dict:
|
||||
if not text:
|
||||
return {}
|
||||
if len(text) > _MATTERMOST_MAX_POST_LENGTH:
|
||||
raise ValueError(f'Mattermost messages cannot exceed {_MATTERMOST_MAX_POST_LENGTH} characters')
|
||||
payload = {'channel_id': channel_id, 'message': text}
|
||||
if root_id:
|
||||
payload['root_id'] = root_id
|
||||
return await self._api_request('POST', '/posts', payload=payload)
|
||||
|
||||
async def _get_direct_channel_id(self, user_id: str) -> str:
|
||||
if not self.bot_account_id:
|
||||
await self._load_bot_identity()
|
||||
channel = await self._api_request(
|
||||
'POST',
|
||||
'/channels/direct',
|
||||
payload={'user_ids': [self.bot_account_id, user_id]},
|
||||
)
|
||||
channel_id = str(channel.get('id') or '')
|
||||
if not channel_id:
|
||||
raise RuntimeError('Mattermost did not return a direct-message channel ID')
|
||||
self.channel_cache[channel_id] = channel
|
||||
return channel_id
|
||||
|
||||
async def send_message(self, target_type: str, target_id: str, message: platform_message.MessageChain):
|
||||
if target_type not in {'person', 'group'}:
|
||||
raise ValueError("Mattermost target_type must be 'person' or 'group'")
|
||||
text = await self.message_converter.yiri2target(message)
|
||||
channel_id = str(target_id)
|
||||
if target_type == 'person':
|
||||
channel_id = await self._get_direct_channel_id(channel_id)
|
||||
await self._post_message(channel_id, text)
|
||||
|
||||
async def reply_message(
|
||||
self,
|
||||
message_source: platform_events.MessageEvent,
|
||||
message: platform_message.MessageChain,
|
||||
quote_origin: bool = False,
|
||||
):
|
||||
source = await self.event_converter.yiri2target(message_source)
|
||||
post = source['post']
|
||||
text = await self.message_converter.yiri2target(message)
|
||||
# A message received inside a Mattermost thread must remain in that thread. When
|
||||
# quote_origin is requested, make the response a reply to the source root post.
|
||||
root_id = str(post.get('root_id') or '')
|
||||
if quote_origin and not root_id:
|
||||
root_id = str(post.get('id') or '')
|
||||
await self._post_message(str(post['channel_id']), text, root_id)
|
||||
|
||||
async def create_message_card(self, message_id: str, event: platform_events.MessageEvent) -> bool:
|
||||
source = await self.event_converter.yiri2target(event)
|
||||
post = source['post']
|
||||
root_id = str(post.get('root_id') or post.get('id') or '')
|
||||
created = await self._post_message(str(post['channel_id']), 'Thinking…', root_id)
|
||||
if created.get('id'):
|
||||
self.stream_post_ids[str(message_id)] = str(created['id'])
|
||||
return True
|
||||
return False
|
||||
|
||||
async def reply_message_chunk(
|
||||
self,
|
||||
message_source: platform_events.MessageEvent,
|
||||
bot_message,
|
||||
message: platform_message.MessageChain,
|
||||
quote_origin: bool = False,
|
||||
is_final: bool = False,
|
||||
):
|
||||
response_id = str(bot_message.resp_message_id)
|
||||
text = await self.message_converter.yiri2target(message)
|
||||
if not text:
|
||||
return
|
||||
|
||||
post_id = self.stream_post_ids.get(response_id)
|
||||
if post_id:
|
||||
await self._api_request('PUT', f'/posts/{post_id}', payload={'id': post_id, 'message': text})
|
||||
else:
|
||||
source = await self.event_converter.yiri2target(message_source)
|
||||
post = source['post']
|
||||
root_id = str(post.get('root_id') or '')
|
||||
if quote_origin and not root_id:
|
||||
root_id = str(post.get('id') or '')
|
||||
created = await self._post_message(str(post['channel_id']), text, root_id)
|
||||
post_id = str(created.get('id') or '')
|
||||
if post_id:
|
||||
self.stream_post_ids[response_id] = post_id
|
||||
|
||||
if is_final and getattr(bot_message, 'tool_calls', None) is None:
|
||||
self.stream_post_ids.pop(response_id, None)
|
||||
|
||||
async def is_stream_output_supported(self) -> bool:
|
||||
return bool(self.config.get('enable_stream_reply', True))
|
||||
|
||||
def register_listener(
|
||||
self,
|
||||
event_type: typing.Type[platform_events.Event],
|
||||
callback: typing.Callable[
|
||||
[platform_events.Event, abstract_platform_adapter.AbstractMessagePlatformAdapter], typing.Awaitable[None]
|
||||
],
|
||||
):
|
||||
self.listeners[event_type] = callback
|
||||
|
||||
def unregister_listener(
|
||||
self,
|
||||
event_type: typing.Type[platform_events.Event],
|
||||
callback: typing.Callable[
|
||||
[platform_events.Event, abstract_platform_adapter.AbstractMessagePlatformAdapter], typing.Awaitable[None]
|
||||
],
|
||||
):
|
||||
self.listeners.pop(event_type, None)
|
||||
|
||||
async def _dispatch_post(self, payload: dict) -> None:
|
||||
data = payload.get('data') or {}
|
||||
try:
|
||||
post = json.loads(data.get('post') or '{}')
|
||||
except (TypeError, json.JSONDecodeError):
|
||||
await self.logger.error('Mattermost received a posted event with an invalid post payload')
|
||||
return
|
||||
|
||||
if not post or str(post.get('user_id') or '') == self.bot_account_id:
|
||||
return
|
||||
channel_id = str(post.get('channel_id') or '')
|
||||
if not channel_id:
|
||||
return
|
||||
|
||||
try:
|
||||
channel = await self._get_channel(channel_id)
|
||||
event = await self.event_converter.target2yiri(
|
||||
post,
|
||||
channel,
|
||||
str(data.get('sender_name') or post.get('user_id') or ''),
|
||||
self.bot_username,
|
||||
)
|
||||
callback = self.listeners.get(type(event))
|
||||
if callback:
|
||||
result = callback(event, self)
|
||||
if asyncio.iscoroutine(result):
|
||||
await result
|
||||
except Exception as exc:
|
||||
await self.logger.error(f'Error handling Mattermost post: {exc}')
|
||||
|
||||
async def _run_websocket_once(self) -> None:
|
||||
session = await self._get_session()
|
||||
async with session.ws_connect(_websocket_url(self.server_url), heartbeat=30) as websocket:
|
||||
await websocket.send_json(
|
||||
{
|
||||
'seq': 1,
|
||||
'action': 'authentication_challenge',
|
||||
'data': {'token': self.access_token},
|
||||
}
|
||||
)
|
||||
async for message in websocket:
|
||||
if message.type == aiohttp.WSMsgType.TEXT:
|
||||
try:
|
||||
payload = json.loads(message.data)
|
||||
except json.JSONDecodeError:
|
||||
continue
|
||||
if payload.get('event') == 'posted':
|
||||
await self._dispatch_post(payload)
|
||||
elif message.type in {aiohttp.WSMsgType.CLOSED, aiohttp.WSMsgType.CLOSE, aiohttp.WSMsgType.ERROR}:
|
||||
break
|
||||
|
||||
async def run_async(self):
|
||||
self._running = True
|
||||
await self._load_bot_identity()
|
||||
await self.logger.info(f'Mattermost bot connected: @{self.bot_username} ({self.bot_account_id})')
|
||||
|
||||
retry_delay = 1
|
||||
while self._running:
|
||||
try:
|
||||
await self._run_websocket_once()
|
||||
retry_delay = 1
|
||||
except asyncio.CancelledError:
|
||||
raise
|
||||
except Exception as exc:
|
||||
if self._running:
|
||||
await self.logger.error(f'Mattermost WebSocket disconnected: {exc}')
|
||||
await asyncio.sleep(retry_delay)
|
||||
retry_delay = min(retry_delay * 2, 30)
|
||||
|
||||
async def kill(self) -> bool:
|
||||
self._running = False
|
||||
if self.session and not self.session.closed:
|
||||
await self.session.close()
|
||||
return True
|
||||
@@ -0,0 +1 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?><svg id="Artwork" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 140 140"><defs><style>.cls-1{fill:#1e325c;fill-rule:evenodd;}</style></defs><path class="cls-1" d="M111.11,13.36l.74,14.86c12.04,13.3,16.8,32.15,10.81,49.86-8.95,26.44-38.46,40.33-65.92,31.04-27.46-9.29-42.45-38.26-33.5-64.7,6.01-17.77,21.32-29.87,39.05-33.07L71.87.03C41.99-.77,13.8,17.77,3.72,47.55c-12.4,36.6,7.24,76.33,43.85,88.73,36.6,12.4,76.33-7.24,88.73-43.85,10.07-29.74-1-61.55-25.14-79.07h-.03Z"/><path class="cls-1" d="M93.95,57.21l-.51-20.77-.41-11.95-.28-10.35s.07-4.99-.11-6.16c-.03-.25-.11-.44-.21-.62,0-.03-.02-.05-.03-.07,0-.02-.03-.05-.03-.07-.2-.33-.49-.59-.89-.72s-.8-.1-1.17.05h-.02s-.08.03-.13.07c-.16.08-.34.2-.51.36-.85.82-3.84,4.83-3.84,4.83l-6.5,8.06-7.59,9.25-13.02,16.19s-5.98,7.46-4.65,16.64c1.31,9.18,8.15,13.65,13.43,15.44,5.29,1.79,13.43,2.38,20.05-4.11,6.62-6.49,6.4-16.04,6.4-16.04l.02-.02Z"/></svg>
|
||||
|
After Width: | Height: | Size: 938 B |
@@ -0,0 +1,75 @@
|
||||
apiVersion: v1
|
||||
kind: MessagePlatformAdapter
|
||||
metadata:
|
||||
name: mattermost
|
||||
label:
|
||||
en_US: Mattermost
|
||||
zh_Hans: Mattermost
|
||||
zh_Hant: Mattermost
|
||||
ja_JP: Mattermost
|
||||
th_TH: Mattermost
|
||||
vi_VN: Mattermost
|
||||
es_ES: Mattermost
|
||||
icon: mattermost.svg
|
||||
description:
|
||||
en_US: Mattermost Bot Account adapter using the v4 REST and WebSocket APIs. Add me to the teams and channels where you want me to interact. Please use a browser or desktop application to do this.
|
||||
zh_Hans: 使用 Mattermost v4 REST API 与 WebSocket 的 Bot Account 适配器。请将我添加到您想要我互动的团队与频道。请使用浏览器或桌面应用进行操作。
|
||||
zh_Hant: 使用 Mattermost v4 REST API 與 WebSocket 的 Bot Account 介面卡。請將我加入您希望我互動的團隊與頻道。請使用瀏覽器或桌面應用程式操作。
|
||||
ja_JP: Mattermost v4 REST API と WebSocket を使用する Bot Account アダプター。利用させたいチームとチャンネルに私を追加してください。ブラウザまたはデスクトップアプリで操作してください。
|
||||
th_TH: อะแดปเตอร์ Bot Account ของ Mattermost ผ่าน v4 REST API และ WebSocket โปรดเพิ่มฉันไปยังทีมและช่องที่คุณต้องการให้ฉันโต้ตอบ โปรดดำเนินการผ่านเบราว์เซอร์หรือแอปเดสก์ท็อป
|
||||
vi_VN: Bộ điều hợp Bot Account Mattermost sử dụng REST API v4 và WebSocket. Hãy thêm tôi vào các nhóm và kênh mà bạn muốn tôi tương tác. Vui lòng thao tác bằng trình duyệt hoặc ứng dụng máy tính để bàn.
|
||||
es_ES: Adaptador de Bot Account de Mattermost mediante REST API v4 y WebSocket. Añádeme a los equipos y canales en los que quieras que interactúe. Hazlo desde un navegador o la aplicación de escritorio.
|
||||
spec:
|
||||
categories:
|
||||
- global
|
||||
- popular
|
||||
config:
|
||||
- name: server_url
|
||||
label:
|
||||
en_US: Mattermost Server URL
|
||||
zh_Hans: Mattermost 服务器地址
|
||||
zh_Hant: 位址伺服器 Mattermost
|
||||
ja_JP: Mattermost サーバー URL
|
||||
th_TH: URL เซิร์ฟเวอร์ Mattermost
|
||||
vi_VN: URL máy chủ Mattermost
|
||||
es_ES: URL del servidor Mattermost
|
||||
description:
|
||||
en_US: The base URL of the Mattermost server, for example https://mattermost.example.com
|
||||
zh_Hans: Mattermost 服务器基础地址,例如 https://mattermost.example.com
|
||||
type: string
|
||||
required: true
|
||||
default: ""
|
||||
- name: access_token
|
||||
label:
|
||||
en_US: Bot Access Token
|
||||
zh_Hans: Bot 访问令牌
|
||||
zh_Hant: Bot 存取權杖
|
||||
ja_JP: Bot アクセストークン
|
||||
th_TH: โทเค็นการเข้าถึงของบอต
|
||||
vi_VN: Mã truy cập Bot
|
||||
es_ES: Token de acceso del bot
|
||||
description:
|
||||
en_US: The personal access token generated for the Mattermost Bot Account
|
||||
zh_Hans: 为 Mattermost Bot Account 生成的个人访问令牌
|
||||
type: string
|
||||
required: true
|
||||
default: ""
|
||||
- name: enable_stream_reply
|
||||
label:
|
||||
en_US: Enable Stream Reply
|
||||
zh_Hans: 启用流式回复
|
||||
zh_Hant: 啟用串流回覆
|
||||
ja_JP: ストリーミング返信を有効化
|
||||
th_TH: เปิดใช้งานการตอบกลับแบบสตรีม
|
||||
vi_VN: Bật phản hồi luồng
|
||||
es_ES: Activar respuesta en streaming
|
||||
description:
|
||||
en_US: Update a Mattermost post while LangBot generates a response
|
||||
zh_Hans: 在 LangBot 生成回复时持续更新同一条 Mattermost 消息
|
||||
type: boolean
|
||||
required: false
|
||||
default: true
|
||||
execution:
|
||||
python:
|
||||
path: ./mattermost.py
|
||||
attr: MattermostAdapter
|
||||
@@ -48,6 +48,7 @@ from ..utils import constants
|
||||
|
||||
_DEFAULT_BINARY_STORAGE_VALUE_BYTES = 10 * 1024 * 1024
|
||||
_HARD_MAX_BINARY_STORAGE_VALUE_BYTES = 64 * 1024 * 1024
|
||||
_UNSET_INSTALLATION_SCOPE = object()
|
||||
|
||||
|
||||
def _binary_storage_value_limit(ap: Any) -> int:
|
||||
@@ -479,7 +480,6 @@ class RuntimeConnectionHandler(handler.Handler):
|
||||
self._outbound_installation_context: contextvars.ContextVar[InstallationBinding | None] = (
|
||||
contextvars.ContextVar(
|
||||
f'{self.__class__.__name__}_{id(self)}_outbound_installation',
|
||||
default=None,
|
||||
)
|
||||
)
|
||||
self._installation_bindings: dict[
|
||||
@@ -1631,13 +1631,15 @@ class RuntimeConnectionHandler(handler.Handler):
|
||||
) -> InstallationBinding | ActionContext | None:
|
||||
if action_context is not None:
|
||||
return super().resolve_outbound_action_context(action_context)
|
||||
inbound_context = self.current_action_context
|
||||
if inbound_context is not None:
|
||||
return inbound_context
|
||||
return self._outbound_installation_context.get()
|
||||
# An explicit scope targets the nested call, not its inbound caller.
|
||||
# None deliberately clears the context for runtime-scoped actions.
|
||||
scoped_context = self._outbound_installation_context.get(_UNSET_INSTALLATION_SCOPE)
|
||||
if scoped_context is not _UNSET_INSTALLATION_SCOPE:
|
||||
return typing.cast(InstallationBinding | None, scoped_context)
|
||||
return self.current_action_context
|
||||
|
||||
def require_outbound_installation_context(self) -> InstallationBinding:
|
||||
binding = self._outbound_installation_context.get()
|
||||
binding = self._outbound_installation_context.get(None)
|
||||
if not isinstance(binding, InstallationBinding):
|
||||
raise ValueError('Host plugin action requires an InstallationBinding scope')
|
||||
return binding
|
||||
|
||||
@@ -0,0 +1,420 @@
|
||||
"""ChatGPT device auth with server-only credentials and cross-process refresh leases.
|
||||
|
||||
Network I/O never holds a DB transaction. A persisted CAS lease serializes refresh
|
||||
and poll; cancel fences device exchanges but waits for existing-token refreshes.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import base64
|
||||
import json
|
||||
import math
|
||||
import secrets
|
||||
import time
|
||||
from contextlib import asynccontextmanager
|
||||
from datetime import datetime, timezone
|
||||
|
||||
import httpx
|
||||
import sqlalchemy as sa
|
||||
|
||||
from ...entity.persistence.model import CodexCredential, ModelProvider
|
||||
from ...api.http.context import PrincipalType, RequestContext
|
||||
from ...api.http.authz import Permission, has_permission
|
||||
from ...api.http.service.tenant import require_workspace_uuid
|
||||
from ...workspace.errors import WorkspaceNotFoundError
|
||||
|
||||
REQUESTER = 'openai-codex'
|
||||
BASE_URL = 'https://chatgpt.com/backend-api/codex'
|
||||
ISSUER = 'https://auth.openai.com'
|
||||
CLIENT_ID = 'app_EMoamEEZ73f0CkXaXp7hrann'
|
||||
LOGIN_REQUIRED = 'ChatGPT sign-in required. Open this provider and sign in again.'
|
||||
LEASE_SECONDS = 90
|
||||
|
||||
|
||||
def validate_config(data: dict) -> None:
|
||||
if data.get('requester') != REQUESTER:
|
||||
return
|
||||
if data.get('base_url') not in (None, '', BASE_URL):
|
||||
raise ValueError('Codex uses the fixed ChatGPT endpoint; custom base URLs are not supported')
|
||||
if data.get('api_keys') not in (None, [], ''):
|
||||
raise ValueError('Codex uses ChatGPT sign-in, not API keys')
|
||||
data['base_url'] = BASE_URL
|
||||
data['api_keys'] = []
|
||||
|
||||
|
||||
def _claims(token: str) -> dict:
|
||||
"""Read routing metadata, NOT trusted LangBot identity, from issuer tokens."""
|
||||
try:
|
||||
part = token.split('.')[1]
|
||||
value = json.loads(base64.urlsafe_b64decode(part + '=' * (-len(part) % 4)))
|
||||
return value if isinstance(value, dict) else {}
|
||||
except (ValueError, IndexError, TypeError):
|
||||
return {}
|
||||
|
||||
|
||||
def _tokens(data: dict, previous: dict | None = None) -> dict:
|
||||
previous = previous or {}
|
||||
access = data.get('access_token')
|
||||
refresh = data.get('refresh_token') or previous.get('refresh_token')
|
||||
account = None
|
||||
for token in (access, data.get('id_token')):
|
||||
namespace = _claims(token or '').get('https://api.openai.com/auth', {})
|
||||
if isinstance(namespace, dict) and isinstance(namespace.get('chatgpt_account_id'), str):
|
||||
account = namespace['chatgpt_account_id']
|
||||
break
|
||||
account = account or previous.get('account_id')
|
||||
try:
|
||||
expires_at = (
|
||||
time.time() + float(data['expires_in'])
|
||||
if data.get('expires_in') is not None
|
||||
else float(_claims(access or '').get('exp', 0))
|
||||
)
|
||||
except (TypeError, ValueError):
|
||||
expires_at = 0
|
||||
if (
|
||||
not all(isinstance(v, str) and v for v in (access, refresh, account))
|
||||
or not math.isfinite(expires_at)
|
||||
or expires_at <= time.time()
|
||||
):
|
||||
raise ValueError('ChatGPT returned an incomplete authorization. Please sign in again.')
|
||||
return {
|
||||
'access_token': access,
|
||||
'refresh_token': refresh,
|
||||
'account_id': account,
|
||||
'expires_at': expires_at,
|
||||
'connection_id': previous.get('connection_id') or secrets.token_urlsafe(24),
|
||||
}
|
||||
|
||||
|
||||
class CodexAuth:
|
||||
def __init__(self, ap):
|
||||
self.ap = ap
|
||||
|
||||
def _where(self, workspace: str, provider: str):
|
||||
return (CodexCredential.workspace_uuid == workspace, CodexCredential.provider_uuid == provider)
|
||||
|
||||
async def _execute(self, statement):
|
||||
# SQLAlchemy/driver/serialization errors may embed the entire secret payload.
|
||||
try:
|
||||
return await self.ap.persistence_mgr.execute_async(statement)
|
||||
except Exception:
|
||||
raise ValueError('ChatGPT credential storage failed. Please retry.') from None
|
||||
|
||||
async def _read(self, workspace: str, provider: str) -> dict | None:
|
||||
result = await self._execute(sa.select(CodexCredential).where(*self._where(workspace, provider)))
|
||||
try:
|
||||
row = result.first()
|
||||
return dict(row._mapping) if row is not None else None
|
||||
except Exception:
|
||||
raise ValueError('ChatGPT credential storage failed. Please retry.') from None
|
||||
|
||||
async def _provider(self, context, provider: str, *, user: bool = False) -> str:
|
||||
workspace = require_workspace_uuid(context)
|
||||
if user and (
|
||||
not isinstance(context, RequestContext)
|
||||
or context.principal.principal_type != PrincipalType.ACCOUNT
|
||||
or not context.account_uuid
|
||||
or not has_permission(context, Permission.PROVIDER_SECRET_MANAGE)
|
||||
):
|
||||
raise ValueError('ChatGPT authorization requires an authorized workspace user')
|
||||
result = await self._execute(
|
||||
sa.select(ModelProvider.requester).where(
|
||||
ModelProvider.workspace_uuid == workspace, ModelProvider.uuid == provider
|
||||
)
|
||||
)
|
||||
kind = result.scalar()
|
||||
if kind is None:
|
||||
raise WorkspaceNotFoundError('Provider not found')
|
||||
if kind != REQUESTER:
|
||||
raise ValueError('This provider does not use ChatGPT sign-in')
|
||||
return workspace
|
||||
|
||||
@asynccontextmanager
|
||||
async def _lease(self, workspace: str, provider: str, *, refresh: bool = False):
|
||||
owner = ('refresh:' if refresh else 'device:') + secrets.token_urlsafe(32)
|
||||
deadline = time.monotonic() + 65
|
||||
while True:
|
||||
now = time.time()
|
||||
result = await self._execute(
|
||||
sa.update(CodexCredential)
|
||||
.where(
|
||||
*self._where(workspace, provider),
|
||||
sa.or_(CodexCredential.lease_owner.is_(None), CodexCredential.lease_until < now),
|
||||
)
|
||||
.values(lease_owner=owner, lease_until=now + LEASE_SECONDS)
|
||||
)
|
||||
if result.rowcount == 1:
|
||||
break
|
||||
if await self._read(workspace, provider) is None:
|
||||
raise ValueError(LOGIN_REQUIRED)
|
||||
if time.monotonic() >= deadline:
|
||||
raise ValueError('ChatGPT authorization is busy. Please retry shortly.')
|
||||
await asyncio.sleep(0.1)
|
||||
try:
|
||||
yield owner
|
||||
finally:
|
||||
await self._execute(
|
||||
sa.update(CodexCredential)
|
||||
.where(*self._where(workspace, provider), CodexCredential.lease_owner == owner)
|
||||
.values(lease_owner=None, lease_until=0)
|
||||
)
|
||||
|
||||
async def _save(self, workspace: str, provider: str, owner: str, payload: dict) -> None:
|
||||
result = await self._execute(
|
||||
sa.update(CodexCredential)
|
||||
.where(
|
||||
*self._where(workspace, provider),
|
||||
CodexCredential.lease_owner == owner,
|
||||
CodexCredential.lease_until > time.time(),
|
||||
)
|
||||
.values(payload=payload, version=CodexCredential.version + 1)
|
||||
)
|
||||
if result.rowcount != 1:
|
||||
raise ValueError('ChatGPT authorization was cancelled or replaced. Please retry.')
|
||||
|
||||
async def _post(self, path: str, *, data=None, json_body=None) -> httpx.Response:
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=20, follow_redirects=False) as client:
|
||||
return await asyncio.wait_for(
|
||||
client.post(
|
||||
ISSUER + path,
|
||||
data=data,
|
||||
json=json_body,
|
||||
headers={'Accept': 'application/json', 'User-Agent': 'LangBot'},
|
||||
),
|
||||
25,
|
||||
)
|
||||
except (httpx.HTTPError, TimeoutError):
|
||||
raise ValueError('ChatGPT authorization network error. Please retry.') from None
|
||||
|
||||
@staticmethod
|
||||
def _json(response: httpx.Response) -> dict:
|
||||
try:
|
||||
value = response.json()
|
||||
if not isinstance(value, dict):
|
||||
raise ValueError
|
||||
return value
|
||||
except ValueError:
|
||||
raise ValueError('ChatGPT returned an invalid authorization response') from None
|
||||
|
||||
async def status(self, context, provider: str) -> dict:
|
||||
workspace = await self._provider(context, provider, user=True)
|
||||
row = await self._read(workspace, provider)
|
||||
payload = row['payload'] if row else {}
|
||||
tokens = payload.get('tokens')
|
||||
connected = bool(tokens and not payload.get('invalid'))
|
||||
return {
|
||||
'status': 'connected' if connected else 'expired' if payload.get('invalid') else 'disconnected',
|
||||
'connected': connected,
|
||||
'expires_at': tokens.get('expires_at') if tokens else None,
|
||||
}
|
||||
|
||||
async def start(self, context, provider: str) -> dict:
|
||||
workspace = await self._provider(context, provider, user=True)
|
||||
async with self._lease(workspace, provider) as owner:
|
||||
response = await self._post('/api/accounts/deviceauth/usercode', json_body={'client_id': CLIENT_ID})
|
||||
if response.status_code != 200:
|
||||
raise ValueError('Unable to start ChatGPT device login. Enable device code login in ChatGPT settings.')
|
||||
data = self._json(response)
|
||||
try:
|
||||
code = data.get('user_code') or data['usercode']
|
||||
device = data['device_auth_id']
|
||||
interval = max(5, min(60, int(data.get('interval') or 5)))
|
||||
if not isinstance(code, str) or not isinstance(device, str) or not code or not device:
|
||||
raise ValueError
|
||||
except (KeyError, ValueError, TypeError):
|
||||
raise ValueError('ChatGPT returned an invalid device code') from None
|
||||
now = time.time()
|
||||
try:
|
||||
expiry = data.get('expires_at')
|
||||
if expiry is None:
|
||||
expiry = now + float(data.get('expires_in', 900))
|
||||
try:
|
||||
expires_at = float(expiry)
|
||||
except ValueError:
|
||||
parsed = datetime.fromisoformat(expiry.replace('Z', '+00:00'))
|
||||
if parsed.tzinfo is None:
|
||||
parsed = parsed.replace(tzinfo=timezone.utc)
|
||||
expires_at = parsed.timestamp()
|
||||
if not math.isfinite(expires_at) or expires_at <= now:
|
||||
raise ValueError
|
||||
expires_at = min(now + 900, expires_at)
|
||||
except (ValueError, TypeError):
|
||||
raise ValueError('ChatGPT returned an invalid device code expiry') from None
|
||||
pending = {
|
||||
'authorization_id': secrets.token_urlsafe(32),
|
||||
'user_code': code,
|
||||
'device_auth_id': device,
|
||||
'account_uuid': context.account_uuid,
|
||||
'interval': interval,
|
||||
'expires_at': expires_at,
|
||||
'next_poll_at': now + interval,
|
||||
}
|
||||
row = await self._read(workspace, provider)
|
||||
payload = dict(row['payload'])
|
||||
payload['pending'] = pending
|
||||
await self._save(workspace, provider, owner, payload)
|
||||
return {k: pending[k] for k in ('authorization_id', 'user_code', 'interval', 'expires_at')} | {
|
||||
'verification_uri': ISSUER + '/codex/device'
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
def _attempt(payload: dict, context, authorization_id: str) -> dict | None:
|
||||
pending = payload.get('pending')
|
||||
if not pending or pending.get('authorization_id') != authorization_id:
|
||||
return None
|
||||
if pending.get('account_uuid') != context.account_uuid:
|
||||
raise WorkspaceNotFoundError('Authorization not found')
|
||||
return pending
|
||||
|
||||
async def poll(self, context, provider: str, authorization_id: str) -> dict:
|
||||
workspace = await self._provider(context, provider, user=True)
|
||||
if not isinstance(authorization_id, str) or not authorization_id:
|
||||
raise ValueError('authorization_id is required')
|
||||
async with self._lease(workspace, provider) as owner:
|
||||
row = await self._read(workspace, provider)
|
||||
payload = dict(row['payload'])
|
||||
pending = self._attempt(payload, context, authorization_id)
|
||||
if pending is None:
|
||||
completed = payload.get('completed', {})
|
||||
if (
|
||||
completed.get('authorization_id') == authorization_id
|
||||
and completed.get('account_uuid') == context.account_uuid
|
||||
):
|
||||
return {'status': 'connected'}
|
||||
return {'status': 'expired'}
|
||||
now = time.time()
|
||||
if pending['expires_at'] <= now or pending.get('consumed'):
|
||||
payload.pop('pending', None)
|
||||
await self._save(workspace, provider, owner, payload)
|
||||
return {'status': 'expired'}
|
||||
if pending['next_poll_at'] > now:
|
||||
return {'status': 'pending', 'interval': pending['interval']}
|
||||
pending['next_poll_at'] = now + pending['interval']
|
||||
await self._save(workspace, provider, owner, payload)
|
||||
response = await self._post(
|
||||
'/api/accounts/deviceauth/token',
|
||||
json_body={'device_auth_id': pending['device_auth_id'], 'user_code': pending['user_code']},
|
||||
)
|
||||
if response.status_code in (403, 404, 429):
|
||||
if response.status_code == 429:
|
||||
pending['interval'] = min(60, pending['interval'] + 5)
|
||||
pending['next_poll_at'] = time.time() + pending['interval']
|
||||
await self._save(workspace, provider, owner, payload)
|
||||
return {'status': 'pending', 'interval': pending['interval']}
|
||||
if response.status_code != 200:
|
||||
payload.pop('pending', None)
|
||||
await self._save(workspace, provider, owner, payload)
|
||||
raise ValueError('ChatGPT device authorization failed. Please start again.')
|
||||
data = self._json(response)
|
||||
if not data.get('authorization_code') or not data.get('code_verifier'):
|
||||
payload.pop('pending', None)
|
||||
await self._save(workspace, provider, owner, payload)
|
||||
raise ValueError('ChatGPT returned an incomplete device authorization')
|
||||
# Keep an attempt tombstone so cancel can preempt exchange, but never replay a code.
|
||||
pending['consumed'] = True
|
||||
await self._save(workspace, provider, owner, payload)
|
||||
response = await self._post(
|
||||
'/oauth/token',
|
||||
data={
|
||||
'grant_type': 'authorization_code',
|
||||
'client_id': CLIENT_ID,
|
||||
'code': data['authorization_code'],
|
||||
'code_verifier': data['code_verifier'],
|
||||
'redirect_uri': ISSUER + '/deviceauth/callback',
|
||||
},
|
||||
)
|
||||
if response.status_code != 200:
|
||||
raise ValueError('ChatGPT token exchange failed. Please start sign-in again.')
|
||||
tokens = _tokens(self._json(response))
|
||||
await self._save(
|
||||
workspace,
|
||||
provider,
|
||||
owner,
|
||||
{
|
||||
'tokens': tokens,
|
||||
'completed': {'authorization_id': authorization_id, 'account_uuid': context.account_uuid},
|
||||
},
|
||||
)
|
||||
return {'status': 'connected'}
|
||||
|
||||
async def disconnect(self, context, provider: str) -> None:
|
||||
workspace = await self._provider(context, provider, user=True)
|
||||
await self._execute(
|
||||
sa.update(CodexCredential)
|
||||
.where(*self._where(workspace, provider))
|
||||
.values(payload={}, lease_owner=None, lease_until=0, version=CodexCredential.version + 1)
|
||||
)
|
||||
|
||||
async def cancel(self, context, provider: str, authorization_id: str) -> None:
|
||||
workspace = await self._provider(context, provider, user=True)
|
||||
deadline = time.monotonic() + 65
|
||||
while time.monotonic() < deadline:
|
||||
row = await self._read(workspace, provider)
|
||||
if row is None:
|
||||
return
|
||||
old = row['payload']
|
||||
if self._attempt(old, context, authorization_id) is None:
|
||||
return
|
||||
lease_owner = row['lease_owner']
|
||||
if lease_owner and lease_owner.startswith('refresh:') and row['lease_until'] > time.time():
|
||||
# A rotated refresh token must be committed before removing the attempt.
|
||||
await asyncio.sleep(0.1)
|
||||
continue
|
||||
payload = dict(old)
|
||||
payload.pop('pending', None)
|
||||
result = await self._execute(
|
||||
sa.update(CodexCredential)
|
||||
.where(
|
||||
*self._where(workspace, provider),
|
||||
CodexCredential.version == row['version'],
|
||||
# Lease acquisition does not change version; fence that race too.
|
||||
CodexCredential.lease_owner == lease_owner,
|
||||
)
|
||||
.values(payload=payload, lease_owner=None, lease_until=0, version=CodexCredential.version + 1)
|
||||
)
|
||||
if result.rowcount == 1:
|
||||
return
|
||||
raise ValueError('Authorization changed concurrently. Please retry cancellation.')
|
||||
|
||||
async def access(self, context, provider: str, *, rejected_token: str | None = None) -> dict:
|
||||
workspace = await self._provider(context, provider)
|
||||
row = await self._read(workspace, provider)
|
||||
payload = row['payload'] if row else {}
|
||||
tokens = payload.get('tokens')
|
||||
if not tokens or payload.get('invalid'):
|
||||
raise ValueError(LOGIN_REQUIRED)
|
||||
if tokens['expires_at'] > time.time() + 120 and tokens['access_token'] != rejected_token:
|
||||
return tokens
|
||||
async with self._lease(workspace, provider, refresh=True) as owner:
|
||||
row = await self._read(workspace, provider)
|
||||
payload = dict(row['payload'])
|
||||
tokens = payload.get('tokens')
|
||||
if not tokens or payload.get('invalid'):
|
||||
raise ValueError(LOGIN_REQUIRED)
|
||||
if tokens['expires_at'] > time.time() + 120 and tokens['access_token'] != rejected_token:
|
||||
return tokens
|
||||
response = await self._post(
|
||||
'/oauth/token',
|
||||
data={'grant_type': 'refresh_token', 'client_id': CLIENT_ID, 'refresh_token': tokens['refresh_token']},
|
||||
)
|
||||
error = self._json(response).get('error') if response.status_code in (400, 401, 403) else None
|
||||
error_code = error.get('code') if isinstance(error, dict) else error
|
||||
if error_code in (
|
||||
'invalid_grant',
|
||||
'refresh_token_reused',
|
||||
'refresh_token_expired',
|
||||
'refresh_token_revoked',
|
||||
):
|
||||
payload['invalid'] = True
|
||||
payload.pop('tokens', None)
|
||||
payload.pop('completed', None)
|
||||
await self._save(workspace, provider, owner, payload)
|
||||
raise ValueError(LOGIN_REQUIRED)
|
||||
if response.status_code != 200:
|
||||
raise ValueError('ChatGPT token refresh temporarily failed. Please retry.')
|
||||
refreshed = _tokens(self._json(response), tokens)
|
||||
payload['tokens'] = refreshed
|
||||
await self._save(workspace, provider, owner, payload)
|
||||
return refreshed
|
||||
@@ -0,0 +1,10 @@
|
||||
"""Explicitly safe Codex failures; never construct messages from upstream bodies."""
|
||||
|
||||
|
||||
class CodexProviderError(ValueError):
|
||||
"""A known provider failure safe to expose at the HTTP boundary."""
|
||||
|
||||
def __init__(self, message: str, status_code: int = 502, error_code: str = 'codex_upstream_failure'):
|
||||
super().__init__(message)
|
||||
self.status_code = status_code
|
||||
self.error_code = error_code
|
||||
@@ -18,7 +18,7 @@ from ...discover import engine
|
||||
from ...entity.errors import provider as provider_errors
|
||||
from ...entity.persistence import model as persistence_model
|
||||
from ...workspace.entities import WorkspaceExecutionBinding
|
||||
from ...workspace.errors import WorkspaceError, WorkspaceInvariantError
|
||||
from ...workspace.errors import WorkspaceError, WorkspaceInvariantError, WorkspaceNotFoundError
|
||||
from . import requester, token
|
||||
|
||||
|
||||
@@ -638,10 +638,32 @@ class ModelManager:
|
||||
) -> requester.RuntimeLLMModel:
|
||||
execution_context = await self.resolve_execution_context(context)
|
||||
provider_info = {**model_info.get('provider', {}), 'workspace_uuid': execution_context.workspace_uuid}
|
||||
runtime_provider = await self._build_provider(
|
||||
execution_context,
|
||||
persistence_model.ModelProvider(**provider_info),
|
||||
)
|
||||
provider_uuid = model_info.get('provider_uuid') or provider_info.get('uuid')
|
||||
inline_codex = provider_info.get('requester') == 'openai-codex'
|
||||
provider_entity = persistence_model.ModelProvider(**provider_info)
|
||||
if provider_uuid:
|
||||
if provider_info.get('uuid') and provider_info['uuid'] != provider_uuid:
|
||||
raise ValueError('Conflicting provider identities')
|
||||
result = await self.ap.persistence_mgr.execute_async(
|
||||
sqlalchemy.select(persistence_model.ModelProvider).where(
|
||||
persistence_model.ModelProvider.workspace_uuid == execution_context.workspace_uuid,
|
||||
persistence_model.ModelProvider.uuid == provider_uuid,
|
||||
)
|
||||
)
|
||||
saved_provider = result.first()
|
||||
if saved_provider is None:
|
||||
if inline_codex or model_info.get('provider_uuid'):
|
||||
raise WorkspaceNotFoundError('Provider not found')
|
||||
else:
|
||||
saved_provider = self._coerce_provider(saved_provider, execution_context)
|
||||
if saved_provider.requester == 'openai-codex':
|
||||
# OAuth identity and transport configuration are server-owned.
|
||||
provider_entity = saved_provider
|
||||
elif inline_codex:
|
||||
raise ValueError('This provider does not use ChatGPT sign-in')
|
||||
elif inline_codex:
|
||||
raise WorkspaceNotFoundError('Provider not found')
|
||||
runtime_provider = await self._build_provider(execution_context, provider_entity)
|
||||
model_entity = persistence_model.LLMModel(
|
||||
workspace_uuid=execution_context.workspace_uuid,
|
||||
uuid=model_info.get('uuid', ''),
|
||||
@@ -723,6 +745,10 @@ class ModelManager:
|
||||
'requester_name': provider_entity.requester,
|
||||
}
|
||||
|
||||
if provider_entity.requester == 'openai-codex':
|
||||
config['provider_uuid'] = provider_entity.uuid
|
||||
config['workspace_uuid'] = context.workspace_uuid
|
||||
|
||||
if litellm_provider:
|
||||
from .requesters import litellmchat
|
||||
|
||||
|
||||
@@ -0,0 +1,423 @@
|
||||
"""Native ChatGPT Codex Responses/SSE requester (never Chat Completions)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import secrets
|
||||
import time
|
||||
from collections import OrderedDict
|
||||
|
||||
import httpx
|
||||
import langbot
|
||||
import langbot_plugin.api.entities.builtin.provider.message as pm
|
||||
|
||||
from .. import requester, reasoning
|
||||
from ..codex_auth import BASE_URL, CodexAuth, LOGIN_REQUIRED
|
||||
from ..codex_errors import CodexProviderError
|
||||
|
||||
|
||||
async def sse_events(response):
|
||||
"""Decode SSE records, including CRLF, comments, and multiline data."""
|
||||
data = []
|
||||
size = 0
|
||||
async for line in response.aiter_lines():
|
||||
if not line:
|
||||
if data:
|
||||
text = '\n'.join(data)
|
||||
if text == '[DONE]':
|
||||
return
|
||||
try:
|
||||
event = json.loads(text)
|
||||
if not isinstance(event, dict):
|
||||
raise ValueError
|
||||
except ValueError:
|
||||
raise ValueError('Codex returned an invalid stream event') from None
|
||||
yield event
|
||||
data, size = [], 0
|
||||
elif line.startswith('data:'):
|
||||
value = line[5:]
|
||||
if value.startswith(' '):
|
||||
value = value[1:]
|
||||
size += len(value)
|
||||
if size > 4 * 1024 * 1024:
|
||||
raise ValueError('Codex stream event exceeds the size limit')
|
||||
data.append(value)
|
||||
# SSE requires the blank separator; unterminated records cannot prove completion.
|
||||
|
||||
|
||||
def _content(message):
|
||||
content = message.content
|
||||
if isinstance(content, str):
|
||||
return [{'type': 'output_text' if message.role == 'assistant' else 'input_text', 'text': content}]
|
||||
result = []
|
||||
for part in content or []:
|
||||
if part.type == 'text':
|
||||
result.append(
|
||||
{'type': 'output_text' if message.role == 'assistant' else 'input_text', 'text': part.text or ''}
|
||||
)
|
||||
elif part.type == 'image_url' and part.image_url is not None:
|
||||
result.append({'type': 'input_image', 'image_url': part.image_url.url})
|
||||
elif part.type == 'image_base64' and part.image_base64:
|
||||
value = part.image_base64
|
||||
result.append(
|
||||
{
|
||||
'type': 'input_image',
|
||||
'image_url': value if value.startswith('data:') else 'data:image/png;base64,' + value,
|
||||
}
|
||||
)
|
||||
else:
|
||||
raise ValueError('Codex supports text and images only; this message contains unsupported content')
|
||||
return result
|
||||
|
||||
|
||||
def _tool(item):
|
||||
try:
|
||||
return pm.ToolCall(
|
||||
id=item['call_id'],
|
||||
type='function',
|
||||
function=pm.FunctionCall(name=item['name'], arguments=item.get('arguments') or ''),
|
||||
)
|
||||
except (KeyError, ValueError, TypeError):
|
||||
raise ValueError('Codex returned an invalid function call') from None
|
||||
|
||||
|
||||
def _usage(response):
|
||||
usage = response.get('usage') or {}
|
||||
return {
|
||||
'prompt_tokens': usage.get('input_tokens', 0),
|
||||
'completion_tokens': usage.get('output_tokens', 0),
|
||||
'total_tokens': usage.get('total_tokens', usage.get('input_tokens', 0) + usage.get('output_tokens', 0)),
|
||||
'prompt_tokens_details': usage.get('input_tokens_details', {}),
|
||||
'completion_tokens_details': usage.get('output_tokens_details', {}),
|
||||
}
|
||||
|
||||
|
||||
class CodexRequester(requester.ProviderAPIRequester):
|
||||
async def initialize(self):
|
||||
self.auth = CodexAuth(self.ap)
|
||||
self.workspace = self.requester_cfg['workspace_uuid']
|
||||
self.provider = self.requester_cfg['provider_uuid']
|
||||
# Opaque replay data stays server-side; handles are scoped to the same query,
|
||||
# model and OAuth connection. No token or encrypted reasoning enters messages.
|
||||
self._replay = OrderedDict()
|
||||
|
||||
async def aclose(self):
|
||||
self._replay.clear()
|
||||
|
||||
def get_reasoning_capabilities(self, model):
|
||||
return {
|
||||
'supported': True,
|
||||
'levels': ['provider_default', 'low', 'medium', 'high', 'xhigh'],
|
||||
'source': 'provider',
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
def _headers(tokens, *, stream=False):
|
||||
return {
|
||||
'Authorization': 'Bearer ' + tokens['access_token'],
|
||||
'ChatGPT-Account-ID': tokens['account_id'],
|
||||
'User-Agent': 'LangBot/' + langbot.__version__,
|
||||
'originator': 'langbot',
|
||||
'OpenAI-Beta': 'responses=experimental',
|
||||
'Accept': 'text/event-stream' if stream else 'application/json',
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
def _http_error(status):
|
||||
if status == 401:
|
||||
# Upstream authentication is not LangBot authentication: HTTP401 would
|
||||
# make the browser discard its own valid user session.
|
||||
return CodexProviderError(LOGIN_REQUIRED, 400, 'codex_reauthentication_required')
|
||||
if status == 429:
|
||||
return CodexProviderError(
|
||||
'ChatGPT request was limited (rate limit or usage restriction). Please retry later or check your plan.',
|
||||
429,
|
||||
'codex_rate_limited',
|
||||
)
|
||||
if status == 403:
|
||||
return CodexProviderError(
|
||||
'ChatGPT denied this request. Check subscription and workspace permissions.',
|
||||
403,
|
||||
'codex_access_denied',
|
||||
)
|
||||
if status == 400:
|
||||
return CodexProviderError(
|
||||
'ChatGPT rejected the model or request. Check the selected model and request settings.',
|
||||
400,
|
||||
'codex_invalid_request',
|
||||
)
|
||||
return CodexProviderError('ChatGPT Codex upstream request failed. Please retry later.')
|
||||
|
||||
async def _response_error(self, response):
|
||||
# Inspect only a bounded 429 error record and an allowlisted machine code.
|
||||
# Never expose upstream prose, reset metadata, headers or credentials.
|
||||
if response.status_code == 429:
|
||||
payload = bytearray()
|
||||
async for chunk in response.aiter_bytes():
|
||||
if len(payload) + len(chunk) > 8192:
|
||||
return self._http_error(429)
|
||||
payload.extend(chunk)
|
||||
try:
|
||||
data = json.loads(payload)
|
||||
error = data.get('error') if isinstance(data, dict) else None
|
||||
if isinstance(error, dict) and (
|
||||
error.get('type') == 'usage_limit_reached' or error.get('code') == 'usage_limit_reached'
|
||||
):
|
||||
return CodexProviderError(
|
||||
'ChatGPT subscription usage limit reached. Please retry later or check your plan.',
|
||||
429,
|
||||
'codex_usage_limit_reached',
|
||||
)
|
||||
except (ValueError, UnicodeError):
|
||||
pass
|
||||
return self._http_error(response.status_code)
|
||||
|
||||
def _scope(self, query, model, tokens):
|
||||
return (
|
||||
id(query),
|
||||
getattr(query, 'query_id', None),
|
||||
model.model_entity.name,
|
||||
tokens.get('connection_id'),
|
||||
tokens['account_id'],
|
||||
)
|
||||
|
||||
def _body(self, query, model, messages, funcs, extra_args, tokens):
|
||||
args = {**(model.model_entity.extra_args or {}), **(extra_args or {})}
|
||||
# Never permit credentials, transport overrides, store/history or arbitrary
|
||||
# SDK kwargs to be smuggled through model advanced parameters.
|
||||
allowed = {'reasoning', 'text', 'parallel_tool_calls', 'tool_choice'}
|
||||
unknown = set(args) - allowed
|
||||
if unknown:
|
||||
raise ValueError('Unsupported Codex advanced parameters: ' + ', '.join(sorted(unknown)))
|
||||
instructions = []
|
||||
items = []
|
||||
scope = self._scope(query, model, tokens)
|
||||
for message in messages:
|
||||
if message.role in ('system', 'developer'):
|
||||
instructions.append('\n'.join(p['text'] for p in _content(message) if 'text' in p))
|
||||
continue
|
||||
if message.role == 'tool':
|
||||
if not message.tool_call_id:
|
||||
raise ValueError('Codex tool results require a tool_call_id')
|
||||
output = (
|
||||
message.content
|
||||
if isinstance(message.content, str)
|
||||
else json.dumps([p.model_dump(exclude_none=True) for p in message.content or []])
|
||||
)
|
||||
items.append({'type': 'function_call_output', 'call_id': message.tool_call_id, 'output': output or ''})
|
||||
continue
|
||||
if message.role not in ('assistant', 'user'):
|
||||
raise ValueError('Unsupported Codex message role')
|
||||
handle = (message.provider_specific_fields or {}).get('codex_replay_id')
|
||||
cached = self._replay.get(handle) if isinstance(handle, str) else None
|
||||
if query is not None and cached and cached[0] == scope and cached[1] > time.time():
|
||||
items.extend(cached[2])
|
||||
continue
|
||||
content = _content(message)
|
||||
if content:
|
||||
items.append({'type': 'message', 'role': message.role, 'content': content})
|
||||
for call in message.tool_calls or []:
|
||||
items.append(
|
||||
{
|
||||
'type': 'function_call',
|
||||
'call_id': call.id,
|
||||
'name': call.function.name,
|
||||
'arguments': call.function.arguments,
|
||||
}
|
||||
)
|
||||
body = {
|
||||
**args,
|
||||
'model': model.model_entity.name,
|
||||
'instructions': '\n\n'.join(instructions),
|
||||
'input': items,
|
||||
'store': False,
|
||||
'stream': True,
|
||||
'include': ['reasoning.encrypted_content'],
|
||||
}
|
||||
level = reasoning.normalize_reasoning_config(getattr(model.model_entity, 'reasoning_config', None))['level']
|
||||
if level != 'provider_default':
|
||||
reasoning.validate_reasoning_capabilities(
|
||||
{'level': level}, self.get_reasoning_capabilities(model), model.model_entity.name
|
||||
)
|
||||
body['reasoning'] = {'effort': level, 'summary': 'auto'}
|
||||
if funcs:
|
||||
body['tools'] = [
|
||||
{
|
||||
'type': 'function',
|
||||
'name': f.name,
|
||||
'description': f.description,
|
||||
'parameters': f.parameters,
|
||||
'strict': False,
|
||||
}
|
||||
for f in funcs
|
||||
]
|
||||
return body
|
||||
|
||||
async def _events(self, query, model, messages, funcs, extra_args):
|
||||
tokens = await self.auth.access(self.workspace, self.provider)
|
||||
try:
|
||||
async with asyncio.timeout(300), httpx.AsyncClient(timeout=120, follow_redirects=False) as client:
|
||||
for attempt in range(2):
|
||||
body = self._body(query, model, messages, funcs, extra_args, tokens)
|
||||
async with client.stream(
|
||||
'POST', BASE_URL + '/responses', json=body, headers=self._headers(tokens, stream=True)
|
||||
) as response:
|
||||
if response.status_code == 401 and attempt == 0:
|
||||
tokens = await self.auth.access(
|
||||
self.workspace, self.provider, rejected_token=tokens['access_token']
|
||||
)
|
||||
continue
|
||||
if response.status_code != 200:
|
||||
raise await self._response_error(response)
|
||||
async for event in sse_events(response):
|
||||
yield event, tokens
|
||||
return
|
||||
except (httpx.HTTPError, TimeoutError):
|
||||
raise ValueError('ChatGPT Codex network error or timeout. Please retry.') from None
|
||||
|
||||
async def _chunks(self, query, model, messages, funcs, extra_args, remove_think, usage_out):
|
||||
text = ''
|
||||
seen_calls = set()
|
||||
output_items = {}
|
||||
response_id = None
|
||||
async for event, tokens in self._events(query, model, messages, funcs, extra_args):
|
||||
kind = event.get('type')
|
||||
response = event.get('response') or {}
|
||||
response_id = response.get('id') or response_id
|
||||
if kind in ('error', 'response.failed', 'response.incomplete'):
|
||||
raise CodexProviderError('ChatGPT Codex response failed or was incomplete. Please retry.')
|
||||
if kind == 'response.output_text.delta':
|
||||
delta = event.get('delta', '')
|
||||
text += delta
|
||||
yield pm.MessageChunk(role='assistant', content=delta, resp_message_id=response_id)
|
||||
elif kind in ('response.reasoning_summary_text.delta', 'response.reasoning_text.delta'):
|
||||
if not remove_think:
|
||||
yield pm.MessageChunk(
|
||||
role='assistant',
|
||||
content='',
|
||||
provider_specific_fields={'reasoning_content': event.get('delta', '')},
|
||||
)
|
||||
elif kind == 'response.output_item.done':
|
||||
item = event.get('item') or {}
|
||||
output_items[event.get('output_index', len(output_items))] = item
|
||||
if item.get('type') == 'function_call' and item.get('call_id') not in seen_calls:
|
||||
seen_calls.add(item.get('call_id'))
|
||||
yield pm.MessageChunk(role='assistant', content='', tool_calls=[_tool(item)])
|
||||
elif kind in ('response.completed', 'response.done'):
|
||||
if response.get('status') not in (None, 'completed'):
|
||||
raise CodexProviderError('ChatGPT Codex response was not completed')
|
||||
output = response.get('output') or [output_items[k] for k in sorted(output_items)]
|
||||
for item in output:
|
||||
if item.get('type') == 'function_call' and item.get('call_id') not in seen_calls:
|
||||
seen_calls.add(item.get('call_id'))
|
||||
yield pm.MessageChunk(role='assistant', content='', tool_calls=[_tool(item)])
|
||||
# Some servers send only the terminal output, without text deltas.
|
||||
final_text = ''.join(
|
||||
p.get('text', '')
|
||||
for item in output
|
||||
if item.get('type') == 'message'
|
||||
for p in item.get('content', [])
|
||||
if p.get('type') == 'output_text'
|
||||
)
|
||||
if not text and final_text:
|
||||
text = final_text
|
||||
yield pm.MessageChunk(role='assistant', content=text, resp_message_id=response_id)
|
||||
usage_out.update(_usage(response))
|
||||
if query is not None:
|
||||
if query.variables is None:
|
||||
query.variables = {}
|
||||
query.variables[requester.STREAM_USAGE_QUERY_VARIABLE] = dict(usage_out)
|
||||
fields = None
|
||||
if query is not None and output:
|
||||
handle = secrets.token_urlsafe(24)
|
||||
self._replay[handle] = (self._scope(query, model, tokens), time.time() + 3600, output)
|
||||
while len(self._replay) > 64:
|
||||
self._replay.popitem(last=False)
|
||||
fields = {'codex_replay_id': handle}
|
||||
yield pm.MessageChunk(
|
||||
role='assistant',
|
||||
content='',
|
||||
all_content=text,
|
||||
is_final=True,
|
||||
resp_message_id=response_id,
|
||||
provider_specific_fields=fields,
|
||||
)
|
||||
return
|
||||
raise CodexProviderError('ChatGPT Codex stream ended before completion. Please retry.')
|
||||
|
||||
async def invoke_llm_stream(self, query, model, messages, funcs=None, extra_args=None, remove_think=False):
|
||||
async for chunk in self._chunks(query, model, messages, funcs, extra_args, remove_think, {}):
|
||||
yield chunk
|
||||
|
||||
async def invoke_llm(self, query, model, messages, funcs=None, extra_args=None, remove_think=False):
|
||||
usage = {}
|
||||
text = ''
|
||||
calls = []
|
||||
fields = {}
|
||||
response_id = None
|
||||
async for chunk in self._chunks(query, model, messages, funcs, extra_args, remove_think, usage):
|
||||
text += chunk.content or ''
|
||||
calls.extend(chunk.tool_calls or [])
|
||||
response_id = chunk.resp_message_id or response_id
|
||||
for key, value in (chunk.provider_specific_fields or {}).items():
|
||||
fields[key] = fields.get(key, '') + value if key == 'reasoning_content' else value
|
||||
return pm.Message(
|
||||
role='assistant',
|
||||
content=text,
|
||||
tool_calls=calls or None,
|
||||
resp_message_id=response_id,
|
||||
provider_specific_fields=fields or None,
|
||||
), usage
|
||||
|
||||
async def scan_models(self, api_key=None):
|
||||
tokens = await self.auth.access(self.workspace, self.provider)
|
||||
try:
|
||||
async with asyncio.timeout(90), httpx.AsyncClient(timeout=30, follow_redirects=False) as client:
|
||||
for attempt in range(2):
|
||||
response = await client.get(
|
||||
BASE_URL + '/models',
|
||||
params={'client_version': langbot.__version__},
|
||||
headers=self._headers(tokens),
|
||||
)
|
||||
if response.status_code == 401 and attempt == 0:
|
||||
tokens = await self.auth.access(
|
||||
self.workspace, self.provider, rejected_token=tokens['access_token']
|
||||
)
|
||||
continue
|
||||
if response.status_code != 200:
|
||||
raise await self._response_error(response)
|
||||
data = response.json()
|
||||
if not isinstance(data, dict) or not isinstance(data.get('models'), list):
|
||||
raise ValueError('ChatGPT returned an invalid model catalog')
|
||||
result = {}
|
||||
for item in data['models']:
|
||||
name = item.get('slug') or item.get('id')
|
||||
if not isinstance(name, str) or not name or item.get('visibility') == 'hide':
|
||||
continue
|
||||
modalities = item.get('input_modalities') or ['text']
|
||||
abilities = ['func_call']
|
||||
if 'image' in modalities:
|
||||
abilities.append('vision')
|
||||
if item.get('supported_reasoning_levels'):
|
||||
abilities.append('reasoning')
|
||||
result[name] = {
|
||||
'id': name,
|
||||
'name': name,
|
||||
'type': 'llm',
|
||||
'abilities': abilities,
|
||||
'display_name': item.get('display_name'),
|
||||
'description': item.get('description'),
|
||||
'context_length': item.get('context_window'),
|
||||
'input_modalities': modalities,
|
||||
'output_modalities': ['text'],
|
||||
'owned_by': 'openai',
|
||||
}
|
||||
return {'models': list(result.values()), 'debug': None}
|
||||
except (httpx.HTTPError, TimeoutError):
|
||||
raise ValueError('ChatGPT model discovery network error. Please retry.') from None
|
||||
except (ValueError, TypeError, KeyError, AttributeError) as exc:
|
||||
# Never echo upstream response bodies (which may contain credentials).
|
||||
if isinstance(exc, ValueError) and str(exc).startswith(('ChatGPT', 'Codex')):
|
||||
raise
|
||||
raise ValueError('ChatGPT returned an invalid model catalog') from None
|
||||
@@ -0,0 +1,27 @@
|
||||
apiVersion: v1
|
||||
kind: LLMAPIRequester
|
||||
metadata:
|
||||
name: openai-codex
|
||||
label:
|
||||
en_US: OpenAI Codex
|
||||
zh_Hans: OpenAI Codex
|
||||
ja_JP: OpenAI Codex
|
||||
icon: openai.svg
|
||||
spec:
|
||||
config:
|
||||
- name: base_url
|
||||
label:
|
||||
en_US: ChatGPT endpoint
|
||||
zh_Hans: ChatGPT 服务地址
|
||||
ja_JP: ChatGPT エンドポイント
|
||||
type: string
|
||||
required: false
|
||||
default: https://chatgpt.com/backend-api/codex
|
||||
alias: "openai codex ChatGPT subscription OAuth 订阅"
|
||||
support_type:
|
||||
- llm
|
||||
provider_category: manufacturer
|
||||
execution:
|
||||
python:
|
||||
path: ./codex.py
|
||||
attr: CodexRequester
|
||||
@@ -1,5 +1,8 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 64 64" role="img" aria-label="TokenLab">
|
||||
<rect width="64" height="64" rx="14" fill="#111827"/>
|
||||
<path fill="#38bdf8" d="M17 14h30v8H36v28h-8V22H17z"/>
|
||||
<path fill="#a7f3d0" d="M40 30h8v20H28v-8h12z"/>
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 48 48" role="img" aria-labelledby="tokenlab-symbol-title tokenlab-symbol-desc">
|
||||
<title id="tokenlab-symbol-title">TokenLab</title>
|
||||
<desc id="tokenlab-symbol-desc">Specimen Split symbol, positive master for sizes from 32 to 96 pixels.</desc>
|
||||
<g fill="#151714">
|
||||
<path d="M5 13h14.25L35 43H5V13Z"/>
|
||||
<path d="M20.7 5H43v30H35.6L20.7 5Z"/>
|
||||
</g>
|
||||
</svg>
|
||||
|
||||
|
Before Width: | Height: | Size: 265 B After Width: | Height: | Size: 416 B |
@@ -90,7 +90,7 @@ class LangflowAPIRunner(runner.RequestRunner):
|
||||
}
|
||||
|
||||
# 如果配置中有tweaks,则添加到负载中
|
||||
tweaks = json.loads(self.pipeline_config['ai']['langflow-api'].get('tweaks'))
|
||||
tweaks = json.loads(self.pipeline_config['ai']['langflow-api'].get('tweaks') or '{}')
|
||||
if tweaks:
|
||||
payload['tweaks'] = tweaks
|
||||
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import dataclasses
|
||||
import enum
|
||||
import json
|
||||
import math
|
||||
@@ -206,6 +207,13 @@ class MCPSessionStatus(enum.Enum):
|
||||
ERROR = 'error'
|
||||
|
||||
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class MCPOAuthChallenge:
|
||||
"""Bearer challenge metadata returned by an OAuth-protected MCP server."""
|
||||
|
||||
resource_metadata_url: str | None
|
||||
|
||||
|
||||
class _TransportReconnect(Exception):
|
||||
"""Internal signal: the Box stdio WS transport dropped but the managed
|
||||
process is still alive. Triggers a lightweight transport reconnect that
|
||||
@@ -265,6 +273,7 @@ class RuntimeMCPSession:
|
||||
_ready_event: asyncio.Event
|
||||
|
||||
error_message: str | None = None
|
||||
_public_error_code: str = 'runtime_error'
|
||||
|
||||
error_phase: MCPSessionErrorPhase | None = None
|
||||
|
||||
@@ -510,6 +519,13 @@ class RuntimeMCPSession:
|
||||
await self._init_streamable_http_server()
|
||||
return
|
||||
except Exception as e:
|
||||
if self._extract_oauth_challenge(e) is not None:
|
||||
self.error_phase = MCPSessionErrorPhase.OAUTH_REQUIRED
|
||||
self.ap.logger.info(
|
||||
f'MCP server {self.server_name}: remote server requires OAuth authorization; '
|
||||
'not falling back to SSE'
|
||||
)
|
||||
raise
|
||||
if not self._should_fallback_to_sse(e):
|
||||
self.ap.logger.info(
|
||||
f'MCP server {self.server_name}: Streamable HTTP transport failed '
|
||||
@@ -630,6 +646,7 @@ class RuntimeMCPSession:
|
||||
except Exception as e:
|
||||
self.status = MCPSessionStatus.ERROR
|
||||
self.error_message = str(e)
|
||||
self._public_error_code = self._classify_public_error(e)
|
||||
self.ap.logger.error(f'Error in MCP session lifecycle {self.server_name}: {e}\n{traceback.format_exc()}')
|
||||
# Do NOT set _ready_event here — let _lifecycle_loop_with_retry
|
||||
# handle retries first. It will set the event when all retries
|
||||
@@ -752,6 +769,11 @@ class RuntimeMCPSession:
|
||||
except Exception as e:
|
||||
if self._shutdown_event.is_set():
|
||||
return # Shutdown requested, don't retry
|
||||
if self.error_phase == MCPSessionErrorPhase.OAUTH_REQUIRED:
|
||||
self.retry_count = attempt + 1
|
||||
self.status = MCPSessionStatus.ERROR
|
||||
self._ready_event.set()
|
||||
return
|
||||
if self.error_phase == MCPSessionErrorPhase.BOX_UNAVAILABLE:
|
||||
box_service = getattr(self.ap, 'box_service', None)
|
||||
if box_service is not None and getattr(box_service, 'enabled', True):
|
||||
@@ -832,6 +854,39 @@ class RuntimeMCPSession:
|
||||
else:
|
||||
yield exc
|
||||
|
||||
@staticmethod
|
||||
def _classify_public_error(exc: BaseException) -> str:
|
||||
"""Expose a safe category without transport URLs, headers, or arguments."""
|
||||
for leaf in RuntimeMCPSession._iter_exception_leaves(exc):
|
||||
if isinstance(leaf, httpx.HTTPStatusError):
|
||||
return f'http_{leaf.response.status_code}'
|
||||
if isinstance(leaf, (httpx.TimeoutException, TimeoutError)):
|
||||
return 'connection_timeout'
|
||||
if isinstance(leaf, httpx.ConnectError):
|
||||
return 'connection_unreachable'
|
||||
return 'runtime_error'
|
||||
|
||||
@staticmethod
|
||||
def _extract_oauth_challenge(exc: BaseException) -> MCPOAuthChallenge | None:
|
||||
"""Extract an OAuth Bearer challenge from a remote MCP connection failure."""
|
||||
for leaf in RuntimeMCPSession._iter_exception_leaves(exc):
|
||||
if not isinstance(leaf, httpx.HTTPStatusError) or leaf.response.status_code != 401:
|
||||
continue
|
||||
for header in leaf.response.headers.get_list('www-authenticate'):
|
||||
bearer_match = re.search(r'(?:^|,)\s*Bearer(?:\s|,|$)', header, flags=re.IGNORECASE)
|
||||
if bearer_match is None:
|
||||
continue
|
||||
metadata_match = re.search(
|
||||
r'(?:^|,)\s*resource_metadata\s*=\s*(?:"([^"]+)"|([^,\s]+))',
|
||||
header[bearer_match.end() :],
|
||||
flags=re.IGNORECASE,
|
||||
)
|
||||
if metadata_match is None:
|
||||
continue
|
||||
resource_metadata_url = metadata_match.group(1) or metadata_match.group(2)
|
||||
return MCPOAuthChallenge(resource_metadata_url=resource_metadata_url)
|
||||
return None
|
||||
|
||||
@staticmethod
|
||||
def _should_fallback_to_sse(exc: BaseException) -> bool:
|
||||
"""Whether a Streamable HTTP failure matches legacy-SSE fallback.
|
||||
@@ -1374,7 +1429,7 @@ class RuntimeMCPSession:
|
||||
# environment values. Detailed diagnostics belong in AUDIT_VIEW
|
||||
# logs; resource-list responses expose only a stable status.
|
||||
'error_message': 'MCP runtime failed' if self.error_message else None,
|
||||
'error_code': 'runtime_error' if self.error_message else None,
|
||||
'error_code': self._public_error_code if self.error_message else None,
|
||||
'error_phase': self.error_phase.value if self.error_phase else None,
|
||||
'retry_count': self.retry_count,
|
||||
'tool_count': len(self.get_tools()),
|
||||
|
||||
@@ -52,6 +52,7 @@ class MCPSessionErrorPhase(enum.Enum):
|
||||
MCP_INIT = 'mcp_init'
|
||||
RUNTIME = 'runtime'
|
||||
TOOL_CALL = 'tool_call'
|
||||
OAUTH_REQUIRED = 'oauth_required'
|
||||
# Stdio MCP refused because Box is disabled in config or currently
|
||||
# unavailable. Not transient — retries would be pointless. The frontend
|
||||
# uses this phase to render a localized actionable message instead of
|
||||
|
||||
@@ -101,18 +101,6 @@ class SeekDBVectorDatabase(VectorDatabase):
|
||||
self._collection_configs: Dict[str, HNSWConfiguration] = {}
|
||||
self._runtime_cache_limit = runtime_cache_limit(ap)
|
||||
|
||||
self._escape_table = str.maketrans(
|
||||
{
|
||||
'\x00': '',
|
||||
'\\': '\\\\',
|
||||
"'": "''", # Standard SQL escaping (OceanBase NO_BACKSLASH_ESCAPES)
|
||||
'"': '\\"',
|
||||
'\n': '\\n',
|
||||
'\r': '\\r',
|
||||
'\t': '\\t',
|
||||
}
|
||||
)
|
||||
|
||||
async def close(self) -> None:
|
||||
self._collections.clear()
|
||||
self._collection_configs.clear()
|
||||
@@ -192,16 +180,22 @@ class SeekDBVectorDatabase(VectorDatabase):
|
||||
return coll
|
||||
|
||||
def _clean_metadata(self, meta: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""SeekDB metadata doesn't support \\ and ", insert will error 3104"""
|
||||
return {
|
||||
k: v.translate(self._escape_table)
|
||||
if isinstance(v, str)
|
||||
else v
|
||||
if v is None or isinstance(v, (int, float, bool))
|
||||
else str(v)
|
||||
for k, v in meta.items()
|
||||
if v is not None
|
||||
}
|
||||
"""Keep supported scalar metadata values without altering strings."""
|
||||
return {k: v if isinstance(v, (str, int, float, bool)) else str(v) for k, v in meta.items() if v is not None}
|
||||
|
||||
@staticmethod
|
||||
def _relevance_scores_to_distances(results: Dict[str, Any]) -> None:
|
||||
"""Convert SeekDB hybrid relevance scores to lower-is-better distances."""
|
||||
distances = results.get('distances')
|
||||
if not isinstance(distances, list):
|
||||
return
|
||||
|
||||
results['distances'] = [
|
||||
[1.0 - float(score) if isinstance(score, (int, float)) else score for score in batch]
|
||||
if isinstance(batch, list)
|
||||
else batch
|
||||
for batch in distances
|
||||
]
|
||||
|
||||
async def get_or_create_collection(self, collection: str):
|
||||
"""Get or create collection (without vector size - will use default)."""
|
||||
@@ -236,10 +230,10 @@ class SeekDBVectorDatabase(VectorDatabase):
|
||||
|
||||
kwargs: Dict[str, Any] = dict(ids=ids, embeddings=embeddings_list, metadatas=cleaned_metadatas)
|
||||
if documents is not None:
|
||||
kwargs['documents'] = [doc.translate(self._escape_table) for doc in documents]
|
||||
await asyncio.to_thread(coll.add, **kwargs)
|
||||
kwargs['documents'] = documents
|
||||
await asyncio.to_thread(coll.upsert, **kwargs)
|
||||
|
||||
self.ap.logger.info(f"Added {len(ids)} embeddings to SeekDB collection '{collection}'")
|
||||
self.ap.logger.info(f"Upserted {len(ids)} embeddings into SeekDB collection '{collection}'")
|
||||
|
||||
async def search(
|
||||
self,
|
||||
@@ -287,7 +281,8 @@ class SeekDBVectorDatabase(VectorDatabase):
|
||||
# Route by search type.
|
||||
# pyseekdb's query() always requires embeddings, so full-text and
|
||||
# hybrid modes use hybrid_search() which supports text-only queries
|
||||
# and returns the same nested-list format with distances.
|
||||
# and returns relevance scores in the nested ``distances`` field.
|
||||
returns_relevance_scores = False
|
||||
if search_type == SearchType.FULL_TEXT:
|
||||
if not query_text:
|
||||
return {'ids': [[]], 'metadatas': [[]], 'distances': [[]]}
|
||||
@@ -309,6 +304,7 @@ class SeekDBVectorDatabase(VectorDatabase):
|
||||
n_results=k,
|
||||
include=['documents', 'metadatas'],
|
||||
)
|
||||
returns_relevance_scores = True
|
||||
|
||||
elif search_type == SearchType.HYBRID:
|
||||
if not query_text:
|
||||
@@ -352,6 +348,7 @@ class SeekDBVectorDatabase(VectorDatabase):
|
||||
n_results=k,
|
||||
include=['documents', 'metadatas'],
|
||||
)
|
||||
returns_relevance_scores = True
|
||||
self.ap.logger.info(
|
||||
f"SeekDB hybrid search in '{collection}' returned {len(results.get('ids', [[]])[0])} results."
|
||||
)
|
||||
@@ -363,6 +360,8 @@ class SeekDBVectorDatabase(VectorDatabase):
|
||||
results = await asyncio.to_thread(coll.query, **query_kwargs)
|
||||
|
||||
results = self._json_safe(results)
|
||||
if returns_relevance_scores:
|
||||
self._relevance_scores_to_distances(results)
|
||||
self.ap.logger.info(
|
||||
f"SeekDB {search_type} search in '{collection}' returned {len(results.get('ids', [[]])[0])} results"
|
||||
)
|
||||
|
||||
@@ -143,18 +143,41 @@ stages:
|
||||
operator: eq
|
||||
value: false
|
||||
disabled_tooltip:
|
||||
en_US: >-
|
||||
Sandbox scope can't be changed: either the Box sandbox is disabled
|
||||
or unavailable (enable it in config.yaml with box.enabled = true and
|
||||
ensure the runtime is reachable), or this deployment pins all
|
||||
pipelines to a fixed scope.
|
||||
zh_Hans: "无法修改沙箱作用域:Box 沙箱已禁用或不可用(请在配置中启用 box.enabled = true 并确认运行时连接正常),或本部署已将所有流水线固定为统一作用域。"
|
||||
zh_Hant: "無法修改沙箱作用域:Box 沙箱已停用或無法使用(請在設定中啟用 box.enabled = true 並確認執行時連線正常),或本部署已將所有流水線固定為統一作用域。"
|
||||
ja_JP: "サンドボックススコープを変更できません:Box サンドボックスが無効/利用不可(設定で box.enabled = true にしてランタイム接続を確認)、またはこのデプロイがすべてのパイプラインを固定スコープに制限しています。"
|
||||
vi_VN: "Không thể thay đổi phạm vi sandbox:Box sandbox bị tắt hoặc không khả dụng (bật box.enabled = true và đảm bảo runtime hoạt động), hoặc bản triển khai này cố định mọi pipeline về một phạm vi."
|
||||
th_TH: "ไม่สามารถเปลี่ยนขอบเขต Sandbox:Box sandbox ถูกปิดหรือไม่พร้อมใช้งาน (เปิด box.enabled = true และตรวจสอบรันไทม์) หรือการ deploy นี้ล็อกทุก pipeline ไว้ที่ขอบเขตเดียว"
|
||||
es_ES: "No se puede cambiar el alcance del sandbox: el sandbox de Box está desactivado o no disponible (actívelo con box.enabled = true y verifique el runtime), o este despliegue fija todas las pipelines a un alcance único."
|
||||
ru_RU: "Невозможно изменить область песочницы: песочница Box отключена или недоступна (включите box.enabled = true и проверьте среду выполнения), либо это развёртывание фиксирует единую область для всех конвейеров."
|
||||
en_US: "Sandbox is unavailable. Enable Box and check its connection before changing the scope."
|
||||
zh_Hans: "沙箱未启用,请启用 Box 并确认连接正常后再修改作用域。"
|
||||
zh_Hant: "沙箱未啟用,請啟用 Box 並確認連線正常後再修改作用域。"
|
||||
ja_JP: "サンドボックスは利用できません。Box を有効にし、接続を確認してからスコープを変更してください。"
|
||||
vi_VN: "Sandbox không khả dụng. Hãy bật Box và kiểm tra kết nối trước khi thay đổi phạm vi."
|
||||
th_TH: "Sandbox ไม่พร้อมใช้งาน โปรดเปิดใช้งาน Box และตรวจสอบการเชื่อมต่อก่อนเปลี่ยนขอบเขต"
|
||||
es_ES: "El sandbox no está disponible. Active Box y compruebe su conexión antes de cambiar el alcance."
|
||||
ru_RU: "Песочница недоступна. Включите Box и проверьте подключение, прежде чем менять область."
|
||||
disabled_tooltip_overrides:
|
||||
- when:
|
||||
field: __system.box_scope_forced_global
|
||||
operator: eq
|
||||
value: true
|
||||
tooltip:
|
||||
en_US: "A global sandbox is enforced; the scope cannot be changed."
|
||||
zh_Hans: "已强制使用全局沙箱,无法修改作用域。"
|
||||
zh_Hant: "已強制使用全域沙箱,無法修改作用域。"
|
||||
ja_JP: "グローバルサンドボックスの使用が強制されているため、スコープを変更できません。"
|
||||
vi_VN: "Bắt buộc sử dụng sandbox toàn cục; không thể thay đổi phạm vi."
|
||||
th_TH: "ระบบบังคับใช้ Sandbox ส่วนกลาง จึงไม่สามารถเปลี่ยนขอบเขตได้"
|
||||
es_ES: "Se impone un sandbox global; no se puede cambiar el alcance."
|
||||
ru_RU: "Принудительно используется глобальная песочница; изменить область нельзя."
|
||||
- when:
|
||||
field: __system.box_scope_forced
|
||||
operator: eq
|
||||
value: true
|
||||
tooltip:
|
||||
en_US: "A fixed sandbox scope is enforced; the scope cannot be changed."
|
||||
zh_Hans: "已强制使用固定沙箱作用域,无法修改作用域。"
|
||||
zh_Hant: "已強制使用固定沙箱作用域,無法修改作用域。"
|
||||
ja_JP: "固定のサンドボックススコープが強制されているため、スコープを変更できません。"
|
||||
vi_VN: "Phạm vi sandbox đã được cố định bắt buộc; không thể thay đổi phạm vi."
|
||||
th_TH: "ระบบบังคับใช้ขอบเขต Sandbox แบบตายตัว จึงไม่สามารถเปลี่ยนขอบเขตได้"
|
||||
es_ES: "Se impone un alcance fijo del sandbox; no se puede cambiar el alcance."
|
||||
ru_RU: "Принудительно задана фиксированная область песочницы; изменить её нельзя."
|
||||
type: select
|
||||
required: false
|
||||
default: "{launcher_type}_{launcher_id}"
|
||||
|
||||
@@ -0,0 +1,106 @@
|
||||
"""Exercise Codex provider wiring through a real LangBot process.
|
||||
|
||||
The default run does not contact OpenAI. Set LANGBOT_TEST_CODEX_DEVICE_AUTH=1
|
||||
to also exercise live device start/pending/cancel, without account sign-in.
|
||||
OAuth exchange and inference behavior are covered by deterministic tests.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import time
|
||||
|
||||
import pytest
|
||||
|
||||
pytestmark = pytest.mark.e2e
|
||||
|
||||
|
||||
def test_codex_provider_disconnected_journey(e2e_client):
|
||||
credentials = {'user': 'codex-e2e@example.com', 'password': 'codex-local-test-password'}
|
||||
initialized = e2e_client.post('/api/v1/user/init', json=credentials)
|
||||
assert initialized.status_code == 200, initialized.text
|
||||
authenticated = e2e_client.post('/api/v1/user/auth', json=credentials)
|
||||
assert authenticated.status_code == 200, authenticated.text
|
||||
headers = {'Authorization': f'Bearer {authenticated.json()["data"]["token"]}'}
|
||||
bootstrap = e2e_client.get('/api/v1/workspaces/bootstrap', headers=headers)
|
||||
assert bootstrap.status_code == 200, bootstrap.text
|
||||
headers['X-Workspace-Id'] = bootstrap.json()['data']['workspaces'][0]['workspace']['uuid']
|
||||
|
||||
requesters = e2e_client.get('/api/v1/provider/requesters?type=llm', headers=headers)
|
||||
assert requesters.status_code == 200, requesters.text
|
||||
codex = next(item for item in requesters.json()['data']['requesters'] if item['name'] == 'openai-codex')
|
||||
assert codex['spec']['support_type'] == ['llm']
|
||||
icon = e2e_client.get('/api/v1/provider/requesters/openai-codex/icon')
|
||||
assert icon.status_code == 200
|
||||
assert 'image/' in icon.headers['content-type']
|
||||
|
||||
base = '/api/v1/provider/providers'
|
||||
created = e2e_client.post(
|
||||
base,
|
||||
headers=headers,
|
||||
json={'name': 'Codex E2E', 'requester': 'openai-codex', 'base_url': '', 'api_keys': []},
|
||||
)
|
||||
assert created.status_code == 200, created.text
|
||||
provider_path = f'{base}/{created.json()["data"]["uuid"]}'
|
||||
try:
|
||||
provider = e2e_client.get(provider_path, headers=headers)
|
||||
assert provider.status_code == 200, provider.text
|
||||
data = provider.json()['data']['provider']
|
||||
assert data['requester'] == 'openai-codex'
|
||||
assert data['api_keys'] == []
|
||||
assert data['base_url'] == 'https://chatgpt.com/backend-api/codex'
|
||||
assert not {'access_token', 'refresh_token', 'id_token'} & data.keys()
|
||||
|
||||
status = e2e_client.get(f'{provider_path}/codex/status', headers=headers)
|
||||
assert status.status_code == 200, status.text
|
||||
assert status.json()['data']['connected'] is False
|
||||
assert status.json()['data']['status'] == 'disconnected'
|
||||
|
||||
anonymous = e2e_client.post(f'{provider_path}/codex/device', json={})
|
||||
assert anonymous.status_code == 401
|
||||
invalid = e2e_client.put(provider_path, headers=headers, json={'base_url': 'https://example.com'})
|
||||
assert invalid.status_code == 400, invalid.text
|
||||
invalid_key = e2e_client.put(provider_path, headers=headers, json={'api_keys': ['not-a-codex-key']})
|
||||
assert invalid_key.status_code == 400, invalid_key.text
|
||||
|
||||
scanned = e2e_client.get(f'{provider_path}/scan-models?type=llm', headers=headers)
|
||||
assert scanned.status_code == 400, scanned.text
|
||||
assert 'sign in' in scanned.json()['msg'].lower()
|
||||
|
||||
renamed = e2e_client.put(provider_path, headers=headers, json={'name': 'Codex renamed'})
|
||||
assert renamed.status_code == 200, renamed.text
|
||||
reread = e2e_client.get(provider_path, headers=headers)
|
||||
assert reread.json()['data']['provider']['name'] == 'Codex renamed'
|
||||
disconnected = e2e_client.delete(f'{provider_path}/codex/auth', headers=headers)
|
||||
assert disconnected.status_code == 200, disconnected.text
|
||||
|
||||
# Opt-in smoke contacts real OpenAI device endpoints, but never completes
|
||||
# account sign-in or prints the one-time code/device credentials.
|
||||
if os.environ.get('LANGBOT_TEST_CODEX_DEVICE_AUTH') == '1':
|
||||
started = e2e_client.post(f'{provider_path}/codex/device', headers=headers, json={})
|
||||
assert started.status_code == 200, started.json().get('msg', 'Device start failed')
|
||||
attempt = started.json()['data']
|
||||
assert attempt['verification_uri'] == 'https://auth.openai.com/codex/device'
|
||||
assert isinstance(attempt['user_code'], str) and attempt['user_code']
|
||||
assert 0 < attempt['expires_at'] - time.time() <= 900
|
||||
assert not {'access_token', 'refresh_token', 'device_auth_id'} & attempt.keys()
|
||||
time.sleep(attempt['interval'])
|
||||
pending = e2e_client.post(
|
||||
f'{provider_path}/codex/device/poll',
|
||||
headers=headers,
|
||||
json={'authorization_id': attempt['authorization_id']},
|
||||
)
|
||||
assert pending.status_code == 200
|
||||
assert pending.json()['data']['status'] == 'pending'
|
||||
canceled = e2e_client.delete(f'{provider_path}/codex/device/{attempt["authorization_id"]}', headers=headers)
|
||||
assert canceled.status_code == 200
|
||||
expired = e2e_client.post(
|
||||
f'{provider_path}/codex/device/poll',
|
||||
headers=headers,
|
||||
json={'authorization_id': attempt['authorization_id']},
|
||||
)
|
||||
assert expired.json()['data']['status'] == 'expired'
|
||||
finally:
|
||||
deleted = e2e_client.delete(provider_path, headers=headers)
|
||||
assert deleted.status_code == 200, deleted.text
|
||||
assert e2e_client.get(provider_path, headers=headers).status_code == 404
|
||||
@@ -69,7 +69,7 @@ class LangBotProcess:
|
||||
# Use coverage.py to collect coverage data
|
||||
# Set COVERAGE_PROCESS_START to enable coverage in subprocess
|
||||
self._coverage_file = self.work_dir / '.coverage.e2e'
|
||||
env['COVERAGE_PROCESS_START'] = str(self.project_root / '.coveragerc')
|
||||
env['COVERAGE_PROCESS_START'] = str(self.work_dir / '.coveragerc')
|
||||
env['COVERAGE_FILE'] = str(self._coverage_file)
|
||||
|
||||
# Create .coveragerc for subprocess
|
||||
|
||||
@@ -9,7 +9,7 @@ Run: uv run pytest tests/integration/api/test_monitoring.py -q
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
from unittest.mock import MagicMock, AsyncMock, Mock
|
||||
from unittest.mock import MagicMock, AsyncMock, Mock, patch
|
||||
from types import SimpleNamespace
|
||||
|
||||
from tests.factories import FakeApp
|
||||
@@ -280,13 +280,20 @@ class TestMonitoringAllDataEndpoint:
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_all_data_success(self, quart_test_client):
|
||||
"""GET /api/v1/monitoring/data returns all data."""
|
||||
response = await quart_test_client.get(
|
||||
'/api/v1/monitoring/data', headers={'Authorization': 'Bearer test_token'}
|
||||
)
|
||||
traffic = {'series': [], 'truncated': False}
|
||||
with patch(
|
||||
'langbot.pkg.api.http.controller.groups.monitoring.get_traffic_series',
|
||||
new=AsyncMock(return_value=traffic),
|
||||
) as get_traffic:
|
||||
response = await quart_test_client.get(
|
||||
'/api/v1/monitoring/data', headers={'Authorization': 'Bearer test_token'}
|
||||
)
|
||||
get_traffic.assert_awaited_once()
|
||||
|
||||
assert response.status_code == 200
|
||||
data = await response.get_json()
|
||||
assert 'overview' in data['data']
|
||||
assert data['data']['traffic'] == traffic
|
||||
|
||||
|
||||
@pytest.mark.usefixtures('mock_circular_import_chain')
|
||||
|
||||
@@ -0,0 +1,85 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import AsyncMock
|
||||
|
||||
import pytest
|
||||
from quart import Quart
|
||||
|
||||
from langbot.pkg.api.http.controller.groups import user as user_module
|
||||
from langbot.pkg.api.http.controller.groups.user import UserRouterGroup
|
||||
from langbot.pkg.api.http.service.user import UserService
|
||||
from langbot.pkg.core.stages.genkeys import GenKeysStage
|
||||
from langbot.pkg.persistence.mgr import PersistenceManager
|
||||
from langbot.pkg.utils import constants
|
||||
from langbot.pkg.workspace.collaboration import WorkspaceCollaborationService
|
||||
from langbot.pkg.workspace.service import WorkspaceService
|
||||
|
||||
pytestmark = [pytest.mark.integration, pytest.mark.asyncio]
|
||||
|
||||
|
||||
async def test_generated_recovery_code_resets_real_sqlite_account(tmp_path, monkeypatch):
|
||||
"""Exercise generation, reset, and old/new password login without mocked user services."""
|
||||
monkeypatch.setattr(constants, 'instance_id', 'recovery-journey')
|
||||
monkeypatch.setattr(user_module, '_reset_password_state', {'window_started_at': 0.0, 'attempts': 0})
|
||||
monkeypatch.setattr(user_module, 'asyncio', SimpleNamespace(sleep=AsyncMock()))
|
||||
application = SimpleNamespace(
|
||||
logger=logging.getLogger('recovery-password-journey'),
|
||||
instance_config=SimpleNamespace(
|
||||
data={
|
||||
'database': {'use': 'sqlite', 'sqlite': {'path': str(tmp_path / 'recovery.db')}},
|
||||
'system': {
|
||||
'jwt': {'secret': 'recovery-journey-test-secret-only', 'expire': 3600},
|
||||
'recovery_key': '',
|
||||
},
|
||||
},
|
||||
dump_config=AsyncMock(),
|
||||
),
|
||||
)
|
||||
await GenKeysStage().run(application)
|
||||
key = application.instance_config.data['system']['recovery_key']
|
||||
assert len(key) == 8
|
||||
assert set(key) <= set('23456789ABCDEFGHJKLMNPQRSTUVWXYZ')
|
||||
persistence = PersistenceManager(application)
|
||||
application.persistence_mgr = persistence
|
||||
try:
|
||||
await persistence.initialize()
|
||||
application.workspace_service = WorkspaceService(application, instance_uuid='recovery-journey')
|
||||
application.workspace_collaboration_service = WorkspaceCollaborationService(
|
||||
application, application.workspace_service
|
||||
)
|
||||
application.user_service = UserService(application)
|
||||
quart_app = Quart(__name__)
|
||||
await UserRouterGroup(application, quart_app).initialize()
|
||||
client = quart_app.test_client()
|
||||
|
||||
initial = await client.post(
|
||||
'/api/v1/user/init', json={'user': 'owner@example.com', 'password': 'OriginalPass1!'}
|
||||
)
|
||||
assert initial.status_code == 200
|
||||
assert (await initial.get_json())['code'] == 0
|
||||
|
||||
payload = {'user': 'owner@example.com', 'recovery_key': 'WRONG', 'new_password': 'RecoveredPass1!'}
|
||||
wrong = await client.post('/api/v1/user/reset-password', json=payload)
|
||||
assert wrong.status_code == 403
|
||||
unchanged = await client.post(
|
||||
'/api/v1/user/auth', json={'user': 'owner@example.com', 'password': 'OriginalPass1!'}
|
||||
)
|
||||
assert (await unchanged.get_json())['code'] == 0
|
||||
|
||||
reset = await client.post('/api/v1/user/reset-password', json={**payload, 'recovery_key': key})
|
||||
assert reset.status_code == 200
|
||||
assert (await reset.get_json())['code'] == 0
|
||||
old_login = await client.post(
|
||||
'/api/v1/user/auth', json={'user': 'owner@example.com', 'password': 'OriginalPass1!'}
|
||||
)
|
||||
assert (await old_login.get_json())['code'] != 0
|
||||
new_login = await client.post(
|
||||
'/api/v1/user/auth', json={'user': 'owner@example.com', 'password': 'RecoveredPass1!'}
|
||||
)
|
||||
new_data = await new_login.get_json()
|
||||
assert new_data['code'] == 0
|
||||
assert new_data['data']['token']
|
||||
finally:
|
||||
await persistence.get_db_engine().dispose()
|
||||
@@ -310,6 +310,8 @@ class TestUserInitEndpoint:
|
||||
'invitation_registration_enabled': True,
|
||||
'password_login_enabled': True,
|
||||
'space_login_enabled': False,
|
||||
'passkey_login_enabled': True,
|
||||
'passkey_supported': True,
|
||||
}
|
||||
fake_api_app.user_service.get_login_capabilities.assert_awaited_once_with()
|
||||
fake_api_app.user_service.get_first_user.assert_not_awaited()
|
||||
@@ -334,6 +336,8 @@ class TestUserInitEndpoint:
|
||||
'invitation_registration_enabled': False,
|
||||
'password_login_enabled': False,
|
||||
'space_login_enabled': True,
|
||||
'passkey_login_enabled': True,
|
||||
'passkey_supported': True,
|
||||
}
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@@ -355,6 +359,8 @@ class TestUserInitEndpoint:
|
||||
'invitation_registration_enabled': True,
|
||||
'password_login_enabled': False,
|
||||
'space_login_enabled': True,
|
||||
'passkey_login_enabled': True,
|
||||
'passkey_supported': True,
|
||||
}
|
||||
|
||||
@pytest.mark.asyncio
|
||||
|
||||
@@ -0,0 +1,190 @@
|
||||
"""
|
||||
Integration smoke tests for Passkey API endpoints.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from unittest.mock import AsyncMock, Mock
|
||||
|
||||
import pytest
|
||||
|
||||
from tests.factories import FakeApp
|
||||
from tests.utils.import_isolation import isolated_sys_modules, MockLifecycleControlScope
|
||||
|
||||
|
||||
pytestmark = [pytest.mark.integration, pytest.mark.usefixtures('mock_circular_import_chain')]
|
||||
|
||||
|
||||
@pytest.fixture(scope='module')
|
||||
def mock_circular_import_chain():
|
||||
class FakeMinimalApplication:
|
||||
pass
|
||||
|
||||
mock_app = Mock()
|
||||
mock_app.Application = FakeMinimalApplication
|
||||
|
||||
mock_entities = Mock()
|
||||
mock_entities.LifecycleControlScope = MockLifecycleControlScope
|
||||
|
||||
clear = [
|
||||
'langbot.pkg.api.http.controller.group',
|
||||
'langbot.pkg.api.http.controller.groups',
|
||||
'langbot.pkg.api.http.controller.groups.system',
|
||||
'langbot.pkg.api.http.controller.groups.user',
|
||||
'langbot.pkg.api.http.controller.main',
|
||||
]
|
||||
|
||||
with isolated_sys_modules(
|
||||
mocks={
|
||||
'langbot.pkg.core.app': mock_app,
|
||||
'langbot.pkg.core.entities': mock_entities,
|
||||
},
|
||||
clear=clear,
|
||||
):
|
||||
import langbot.pkg.api.http.controller.groups.user as _user_group # noqa: E402, F401
|
||||
|
||||
yield
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def fake_api_app():
|
||||
app = FakeApp()
|
||||
app.instance_config.data.update(
|
||||
{
|
||||
'api': {'port': 5300},
|
||||
'system': {'allow_modify_login_info': True},
|
||||
}
|
||||
)
|
||||
app.user_service = Mock()
|
||||
app.user_service.verify_jwt_token = AsyncMock(side_effect=ValueError('Invalid token'))
|
||||
app.user_service.get_user_by_email = AsyncMock(return_value=Mock())
|
||||
return app
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
async def quart_test_client(fake_api_app, http_controller_cls):
|
||||
controller = http_controller_cls(fake_api_app)
|
||||
await controller.initialize()
|
||||
|
||||
client = controller.quart_app.test_client()
|
||||
yield client
|
||||
|
||||
|
||||
class TestPasskeyPublicEndpoints:
|
||||
@pytest.mark.asyncio
|
||||
async def test_auth_options_endpoint(self, quart_test_client, fake_api_app):
|
||||
fake_api_app.user_service.generate_passkey_authentication_options = AsyncMock(
|
||||
return_value=({'challenge': 'test_chal', 'rpId': 'localhost'}, 'token_123')
|
||||
)
|
||||
|
||||
response = await quart_test_client.post(
|
||||
'/api/v1/user/passkey/auth/options',
|
||||
json={'origin': 'http://localhost:3000'},
|
||||
)
|
||||
|
||||
assert response.status_code == 200
|
||||
data = await response.get_json()
|
||||
assert data['code'] == 0
|
||||
assert data['data']['challenge_token'] == 'token_123'
|
||||
assert data['data']['options']['rpId'] == 'localhost'
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_auth_verify_missing_payload(self, quart_test_client, fake_api_app):
|
||||
response = await quart_test_client.post(
|
||||
'/api/v1/user/passkey/auth/verify',
|
||||
json={},
|
||||
)
|
||||
assert response.status_code == 200
|
||||
data = await response.get_json()
|
||||
assert data['code'] != 0
|
||||
assert 'Missing challenge_token or credential' in data['msg']
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_auth_verify_success(self, quart_test_client, fake_api_app):
|
||||
fake_api_app.user_service.verify_passkey_authentication = AsyncMock(
|
||||
return_value=('jwt_token_abc', Mock(user='user@example.com'))
|
||||
)
|
||||
|
||||
response = await quart_test_client.post(
|
||||
'/api/v1/user/passkey/auth/verify',
|
||||
json={'challenge_token': 'token_123', 'credential': {'id': 'cred_id'}},
|
||||
)
|
||||
|
||||
assert response.status_code == 200
|
||||
data = await response.get_json()
|
||||
assert data['code'] == 0
|
||||
assert data['data']['token'] == 'jwt_token_abc'
|
||||
assert data['data']['user'] == 'user@example.com'
|
||||
|
||||
|
||||
class TestPasskeyProtectedEndpoints:
|
||||
@pytest.mark.asyncio
|
||||
async def test_register_options_requires_auth(self, quart_test_client):
|
||||
response = await quart_test_client.post('/api/v1/user/passkey/register/options', json={})
|
||||
assert response.status_code == 401
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_passkeys_list_requires_auth(self, quart_test_client):
|
||||
response = await quart_test_client.get('/api/v1/user/passkeys')
|
||||
assert response.status_code == 401
|
||||
|
||||
|
||||
class TestPasskeyReverseProxyScenarios:
|
||||
@pytest.mark.asyncio
|
||||
async def test_auth_options_respects_custom_origin_body_behind_proxy(self, quart_test_client, fake_api_app):
|
||||
fake_api_app.user_service.generate_passkey_authentication_options = AsyncMock(
|
||||
return_value=({'challenge': 'test_chal', 'rpId': 'proxy.company.com'}, 'token_proxy')
|
||||
)
|
||||
|
||||
response = await quart_test_client.post(
|
||||
'/api/v1/user/passkey/auth/options',
|
||||
json={'origin': 'https://proxy.company.com:8443'},
|
||||
headers={'Host': '127.0.0.1:5300'},
|
||||
)
|
||||
|
||||
assert response.status_code == 200
|
||||
data = await response.get_json()
|
||||
assert data['code'] == 0
|
||||
fake_api_app.user_service.generate_passkey_authentication_options.assert_awaited_once_with(
|
||||
rp_id='proxy.company.com',
|
||||
origin='https://proxy.company.com:8443',
|
||||
email=None,
|
||||
)
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_auth_options_falls_back_to_origin_header(self, quart_test_client, fake_api_app):
|
||||
fake_api_app.user_service.generate_passkey_authentication_options = AsyncMock(
|
||||
return_value=({'challenge': 'test_chal', 'rpId': 'bot.example.com'}, 'token_header')
|
||||
)
|
||||
|
||||
response = await quart_test_client.post(
|
||||
'/api/v1/user/passkey/auth/options',
|
||||
json={},
|
||||
headers={'Origin': 'https://bot.example.com'},
|
||||
)
|
||||
|
||||
assert response.status_code == 200
|
||||
fake_api_app.user_service.generate_passkey_authentication_options.assert_awaited_once_with(
|
||||
rp_id='bot.example.com',
|
||||
origin='https://bot.example.com',
|
||||
email=None,
|
||||
)
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_auth_options_falls_back_to_referer_header(self, quart_test_client, fake_api_app):
|
||||
fake_api_app.user_service.generate_passkey_authentication_options = AsyncMock(
|
||||
return_value=({'challenge': 'test_chal', 'rpId': 'bot.example.com'}, 'token_referer')
|
||||
)
|
||||
|
||||
response = await quart_test_client.post(
|
||||
'/api/v1/user/passkey/auth/options',
|
||||
json={},
|
||||
headers={'Referer': 'https://bot.example.com:9000/login'},
|
||||
)
|
||||
|
||||
assert response.status_code == 200
|
||||
fake_api_app.user_service.generate_passkey_authentication_options.assert_awaited_once_with(
|
||||
rp_id='bot.example.com',
|
||||
origin='https://bot.example.com:9000',
|
||||
email=None,
|
||||
)
|
||||
@@ -193,6 +193,22 @@ async def create_legacy_resource_schema(engine, *, instance_uuid: str) -> None:
|
||||
sa.Column('message_id', sa.String(255), nullable=True),
|
||||
)
|
||||
|
||||
# Include historical monitoring columns consumed by later migrations.
|
||||
for table_name in ('monitoring_messages', 'monitoring_sessions'):
|
||||
table = monitoring_tables[table_name]
|
||||
for name, value in (('bot_name', 'bot'), ('pipeline_id', 'pipeline-1'), ('pipeline_name', 'pipeline')):
|
||||
table.append_column(sa.Column(name, sa.String(255), nullable=False, default=value))
|
||||
for name in ('platform', 'user_id', 'user_name'):
|
||||
table.append_column(sa.Column(name, sa.String(255)))
|
||||
if table_name == 'monitoring_messages':
|
||||
table.append_column(sa.Column('bot_id', sa.String(255), nullable=False, default='bot-1'))
|
||||
table.append_column(sa.Column('role', sa.String(50)))
|
||||
else:
|
||||
table.append_column(sa.Column('message_count', sa.Integer, nullable=False, default=1))
|
||||
table.append_column(
|
||||
sa.Column('start_time', sa.DateTime, nullable=False, default=datetime.datetime(2026, 1, 1))
|
||||
)
|
||||
|
||||
now = datetime.datetime(2026, 1, 1)
|
||||
async with engine.begin() as conn:
|
||||
await conn.run_sync(metadata.create_all)
|
||||
|
||||
@@ -0,0 +1,445 @@
|
||||
"""Deterministic OAuth tests using real SQLite CAS writes, never live credentials."""
|
||||
|
||||
import asyncio
|
||||
import base64
|
||||
import json
|
||||
import time
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import AsyncMock
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
import pytest_asyncio
|
||||
import sqlalchemy as sa
|
||||
from sqlalchemy.ext.asyncio import create_async_engine
|
||||
|
||||
from langbot.pkg.api.http.authz import Permission
|
||||
from langbot.pkg.api.http.context import PrincipalContext, PrincipalType, RequestContext, WorkspaceContext
|
||||
from langbot.pkg.entity.persistence.model import CodexCredential
|
||||
from langbot.pkg.persistence.alembic_runner import run_alembic_stamp, run_alembic_upgrade
|
||||
from langbot.pkg.provider.modelmgr.codex_auth import CodexAuth, _tokens, validate_config
|
||||
from langbot.pkg.workspace.errors import WorkspaceNotFoundError
|
||||
|
||||
|
||||
def context(workspace='w', user='u', principal=PrincipalType.ACCOUNT, permitted=True):
|
||||
return RequestContext(
|
||||
'i',
|
||||
0,
|
||||
'r',
|
||||
'user_token',
|
||||
PrincipalContext(principal, account_uuid=user),
|
||||
WorkspaceContext(
|
||||
workspace, 'm', 'owner', frozenset({Permission.PROVIDER_SECRET_MANAGE} if permitted else set())
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def jwt(**claims):
|
||||
return 'test.' + base64.urlsafe_b64encode(json.dumps(claims).encode()).decode().rstrip('=') + '.test'
|
||||
|
||||
|
||||
def token_response(**extra):
|
||||
return {
|
||||
'access_token': jwt(**{'https://api.openai.com/auth': {'chatgpt_account_id': 'account'}}),
|
||||
'refresh_token': 'refresh-secret',
|
||||
'expires_in': 3600,
|
||||
**extra,
|
||||
}
|
||||
|
||||
|
||||
@pytest_asyncio.fixture
|
||||
async def auth(tmp_path):
|
||||
engine = create_async_engine(f'sqlite+aiosqlite:///{tmp_path / "codex.db"}')
|
||||
|
||||
@sa.event.listens_for(engine.sync_engine, 'connect')
|
||||
def foreign_keys(connection, _):
|
||||
connection.execute('PRAGMA foreign_keys=ON')
|
||||
|
||||
async with engine.begin() as conn:
|
||||
await conn.execute(
|
||||
sa.text(
|
||||
'CREATE TABLE model_providers (uuid VARCHAR(255) PRIMARY KEY, workspace_uuid VARCHAR(36) NOT NULL, requester TEXT, UNIQUE(workspace_uuid, uuid))'
|
||||
)
|
||||
)
|
||||
await conn.execute(
|
||||
sa.text(
|
||||
"INSERT INTO model_providers VALUES ('p','w','openai-codex'), ('other','other','openai-codex'), ('api','w','openai-chat-completions')"
|
||||
)
|
||||
)
|
||||
await run_alembic_stamp(engine, '0021_merge_reasoning_config')
|
||||
await run_alembic_upgrade(engine, '0022_codex_credentials')
|
||||
|
||||
async def execute(statement):
|
||||
async with engine.begin() as conn:
|
||||
return await conn.execute(statement)
|
||||
|
||||
service = CodexAuth(SimpleNamespace(persistence_mgr=SimpleNamespace(execute_async=execute)))
|
||||
service.engine = engine
|
||||
await execute(
|
||||
sa.insert(CodexCredential).values(provider_uuid='p', workspace_uuid='w', payload={}, version=0, lease_until=0)
|
||||
)
|
||||
try:
|
||||
yield service
|
||||
finally:
|
||||
await engine.dispose()
|
||||
|
||||
|
||||
async def seed(auth, payload):
|
||||
await auth.ap.persistence_mgr.execute_async(sa.update(CodexCredential).values(payload=payload))
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_migration_upgrade_repeat_fk_cascade(auth):
|
||||
await run_alembic_upgrade(auth.engine, '0022_codex_credentials')
|
||||
await run_alembic_stamp(auth.engine, '0021_merge_reasoning_config')
|
||||
await run_alembic_upgrade(auth.engine, '0022_codex_credentials')
|
||||
with pytest.raises(sa.exc.IntegrityError):
|
||||
await auth.ap.persistence_mgr.execute_async(
|
||||
sa.insert(CodexCredential).values(
|
||||
provider_uuid='other', workspace_uuid='w', payload={}, version=0, lease_until=0
|
||||
)
|
||||
)
|
||||
await auth.ap.persistence_mgr.execute_async(sa.text("DELETE FROM model_providers WHERE uuid='p'"))
|
||||
assert await auth._read('w', 'p') is None
|
||||
from langbot.pkg.persistence.alembic_runner import run_alembic_downgrade
|
||||
|
||||
await run_alembic_downgrade(auth.engine, '0021_merge_reasoning_config')
|
||||
async with auth.engine.connect() as conn:
|
||||
assert 'codex_credentials' not in await conn.run_sync(lambda sync: sa.inspect(sync).get_table_names())
|
||||
await run_alembic_upgrade(auth.engine, '0022_codex_credentials')
|
||||
async with auth.engine.connect() as conn:
|
||||
assert 'codex_credentials' in await conn.run_sync(lambda sync: sa.inspect(sync).get_table_names())
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_device_pacing_exchange_secrecy_and_user_binding(auth):
|
||||
auth._post = AsyncMock(
|
||||
side_effect=[
|
||||
httpx.Response(200, json={'device_auth_id': 'device-secret', 'usercode': 'CODE', 'interval': '5'}),
|
||||
httpx.Response(200, json={'authorization_code': 'code-secret', 'code_verifier': 'verifier-secret'}),
|
||||
httpx.Response(200, json=token_response()),
|
||||
]
|
||||
)
|
||||
start = await auth.start(context(), 'p')
|
||||
assert set(start) == {'authorization_id', 'user_code', 'interval', 'expires_at', 'verification_uri'}
|
||||
assert 'device-secret' not in json.dumps(start)
|
||||
attempt = start['authorization_id']
|
||||
with pytest.raises(WorkspaceNotFoundError):
|
||||
await auth.poll(context(user='attacker'), 'p', attempt)
|
||||
assert (await auth.poll(context(), 'p', attempt))['status'] == 'pending'
|
||||
assert auth._post.await_count == 1
|
||||
row = await auth._read('w', 'p')
|
||||
row['payload']['pending']['next_poll_at'] = 0
|
||||
await seed(auth, row['payload'])
|
||||
assert await auth.poll(context(), 'p', attempt) == {'status': 'connected'}
|
||||
assert await auth.poll(context(), 'p', attempt) == {'status': 'connected'}
|
||||
exchange = auth._post.call_args.kwargs['data']
|
||||
assert exchange['grant_type'] == 'authorization_code'
|
||||
assert exchange['redirect_uri'] == 'https://auth.openai.com/deviceauth/callback'
|
||||
assert exchange['code_verifier'] == 'verifier-secret'
|
||||
status = await auth.status(context(), 'p')
|
||||
assert set(status) == {'status', 'connected', 'expires_at'}
|
||||
assert 'secret' not in json.dumps(status)
|
||||
await auth.disconnect(context(), 'p')
|
||||
assert (await auth._read('w', 'p'))['payload'] == {}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize(
|
||||
'ctx,provider,error',
|
||||
[
|
||||
(context('other'), 'p', WorkspaceNotFoundError),
|
||||
(context(principal=PrincipalType.API_KEY), 'p', ValueError),
|
||||
(context(permitted=False), 'p', ValueError),
|
||||
(context(), 'api', ValueError),
|
||||
],
|
||||
)
|
||||
async def test_auth_tenant_principal_permission_guards(auth, ctx, provider, error):
|
||||
auth._post = AsyncMock()
|
||||
with pytest.raises(error):
|
||||
await auth.start(ctx, provider)
|
||||
auth._post.assert_not_called()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_refresh_cross_instance_single_flight_and_rotation(auth):
|
||||
old = _tokens(token_response())
|
||||
old['expires_at'] = 0
|
||||
await seed(auth, {'tokens': old})
|
||||
entered, release = asyncio.Event(), asyncio.Event()
|
||||
|
||||
async def refresh(*args, **kwargs):
|
||||
entered.set()
|
||||
await release.wait()
|
||||
return httpx.Response(200, json=token_response(refresh_token='rotated-secret'))
|
||||
|
||||
auth._post = AsyncMock(side_effect=refresh)
|
||||
other = CodexAuth(auth.ap)
|
||||
other._post = auth._post
|
||||
first = asyncio.create_task(auth.access('w', 'p'))
|
||||
await entered.wait()
|
||||
second = asyncio.create_task(other.access('w', 'p'))
|
||||
release.set()
|
||||
a, b = await asyncio.gather(first, second)
|
||||
assert a == b
|
||||
assert a['refresh_token'] == 'rotated-secret'
|
||||
assert auth._post.await_count == 1
|
||||
assert (await auth._read('w', 'p'))['payload']['tokens'] == a
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize(
|
||||
'status,error,invalid',
|
||||
[
|
||||
(400, 'invalid_grant', True),
|
||||
(401, 'refresh_token_reused', True),
|
||||
(429, 'limited', False),
|
||||
(500, 'secret-upstream-body', False),
|
||||
(403, 'permission_denied', False),
|
||||
],
|
||||
)
|
||||
async def test_refresh_errors_are_safe_and_transient_preserves_tokens(auth, status, error, invalid):
|
||||
old = _tokens(token_response())
|
||||
old['expires_at'] = 0
|
||||
await seed(auth, {'tokens': old})
|
||||
auth._post = AsyncMock(
|
||||
return_value=httpx.Response(status, json={'error': error, 'access_token': 'secret-upstream-body'})
|
||||
)
|
||||
with pytest.raises(ValueError) as caught:
|
||||
await auth.access('w', 'p')
|
||||
assert 'secret' not in str(caught.value)
|
||||
payload = (await auth._read('w', 'p'))['payload']
|
||||
assert bool(payload.get('invalid')) == invalid
|
||||
assert ('tokens' not in payload) if invalid else payload['tokens'] == old
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize('cancel', [False, True])
|
||||
async def test_disconnect_or_cancel_fences_inflight_exchange(auth, cancel):
|
||||
old = _tokens(token_response())
|
||||
await seed(
|
||||
auth,
|
||||
{
|
||||
'tokens': old,
|
||||
'pending': {
|
||||
'authorization_id': 'attempt',
|
||||
'account_uuid': 'u',
|
||||
'expires_at': time.time() + 100,
|
||||
'next_poll_at': 0,
|
||||
'interval': 5,
|
||||
'device_auth_id': 'device',
|
||||
'user_code': 'code',
|
||||
},
|
||||
},
|
||||
)
|
||||
entered, release = asyncio.Event(), asyncio.Event()
|
||||
|
||||
async def post(path, **kwargs):
|
||||
if path.endswith('/token') and path != '/oauth/token':
|
||||
return httpx.Response(200, json={'authorization_code': 'code', 'code_verifier': 'verifier'})
|
||||
entered.set()
|
||||
await release.wait()
|
||||
return httpx.Response(200, json=token_response())
|
||||
|
||||
auth._post = post
|
||||
task = asyncio.create_task(auth.poll(context(), 'p', 'attempt'))
|
||||
await entered.wait()
|
||||
if cancel:
|
||||
await auth.cancel(context(), 'p', 'attempt')
|
||||
else:
|
||||
await auth.disconnect(context(), 'p')
|
||||
release.set()
|
||||
with pytest.raises(ValueError, match='cancelled or replaced'):
|
||||
await task
|
||||
payload = (await auth._read('w', 'p'))['payload']
|
||||
assert payload == ({'tokens': old} if cancel else {})
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize('status,interval', [(403, 5), (404, 5), (429, 10)])
|
||||
async def test_device_pending_and_backoff(auth, status, interval):
|
||||
await seed(
|
||||
auth,
|
||||
{
|
||||
'pending': {
|
||||
'authorization_id': 'attempt',
|
||||
'account_uuid': 'u',
|
||||
'expires_at': time.time() + 100,
|
||||
'next_poll_at': 0,
|
||||
'interval': 5,
|
||||
'device_auth_id': 'device',
|
||||
'user_code': 'code',
|
||||
}
|
||||
},
|
||||
)
|
||||
auth._post = AsyncMock(return_value=httpx.Response(status))
|
||||
assert await auth.poll(context(), 'p', 'attempt') == {'status': 'pending', 'interval': interval}
|
||||
assert await auth.poll(context(), 'p', 'attempt') == {'status': 'pending', 'interval': interval}
|
||||
assert auth._post.await_count == 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_device_replacement_expiry_and_idempotent_cancel(auth):
|
||||
auth._post = AsyncMock(return_value=httpx.Response(200, json={'device_auth_id': 'device', 'user_code': 'CODE'}))
|
||||
first = await auth.start(context(), 'p')
|
||||
second = await auth.start(context(), 'p')
|
||||
assert first['authorization_id'] != second['authorization_id']
|
||||
assert await auth.poll(context(), 'p', first['authorization_id']) == {'status': 'expired'}
|
||||
await auth.cancel(context(), 'p', first['authorization_id'])
|
||||
payload = (await auth._read('w', 'p'))['payload']
|
||||
assert payload['pending']['authorization_id'] == second['authorization_id']
|
||||
payload['pending']['expires_at'] = 0
|
||||
await seed(auth, payload)
|
||||
assert await auth.poll(context(), 'p', second['authorization_id']) == {'status': 'expired'}
|
||||
assert 'pending' not in (await auth._read('w', 'p'))['payload']
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_device_accepts_issuer_iso_expiry(auth):
|
||||
from datetime import datetime, timezone
|
||||
|
||||
expires = datetime.fromtimestamp(time.time() + 600, timezone.utc).isoformat().replace('+00:00', 'Z')
|
||||
auth._post = AsyncMock(
|
||||
return_value=httpx.Response(200, json={'device_auth_id': 'device', 'user_code': 'CODE', 'expires_at': expires})
|
||||
)
|
||||
result = await auth.start(context(), 'p')
|
||||
assert time.time() < result['expires_at'] < time.time() + 900
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_device_expires_in_fallback(auth):
|
||||
auth._post = AsyncMock(
|
||||
return_value=httpx.Response(200, json={'device_auth_id': 'device', 'user_code': 'CODE', 'expires_in': 60})
|
||||
)
|
||||
result = await auth.start(context(), 'p')
|
||||
assert time.time() < result['expires_at'] <= time.time() + 60
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize('cancel_reads_before_refresh', [False, True])
|
||||
async def test_cancel_pending_relogin_waits_for_existing_refresh(auth, cancel_reads_before_refresh):
|
||||
old = _tokens(token_response())
|
||||
old['expires_at'] = 0
|
||||
await seed(auth, {'tokens': old, 'pending': {'authorization_id': 'attempt', 'account_uuid': 'u'}})
|
||||
entered, release, cancel_read = asyncio.Event(), asyncio.Event(), asyncio.Event()
|
||||
|
||||
async def refresh(*args, **kwargs):
|
||||
entered.set()
|
||||
await release.wait()
|
||||
return httpx.Response(200, json=token_response(refresh_token='rotated-secret'))
|
||||
|
||||
other = CodexAuth(auth.ap)
|
||||
original_read = other._read
|
||||
|
||||
async def read(workspace, provider):
|
||||
row = await original_read(workspace, provider)
|
||||
cancel_read.set()
|
||||
if cancel_reads_before_refresh:
|
||||
await entered.wait()
|
||||
return row
|
||||
|
||||
other._read = read
|
||||
auth._post = refresh
|
||||
if cancel_reads_before_refresh:
|
||||
cancelling = asyncio.create_task(other.cancel(context(), 'p', 'attempt'))
|
||||
await cancel_read.wait()
|
||||
refreshing = asyncio.create_task(auth.access('w', 'p'))
|
||||
await entered.wait()
|
||||
if not cancel_reads_before_refresh:
|
||||
cancelling = asyncio.create_task(other.cancel(context(), 'p', 'attempt'))
|
||||
await cancel_read.wait()
|
||||
await asyncio.sleep(0.05)
|
||||
try:
|
||||
assert not cancelling.done(), 'Cancellation must not revoke the refresh lease'
|
||||
finally:
|
||||
release.set()
|
||||
results = await asyncio.gather(refreshing, cancelling, return_exceptions=True)
|
||||
assert not any(isinstance(result, Exception) for result in results)
|
||||
payload = (await auth._read('w', 'p'))['payload']
|
||||
assert payload['tokens']['refresh_token'] == 'rotated-secret'
|
||||
assert 'pending' not in payload
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize('operation', ['save', 'cancel', 'disconnect', 'acquire', 'release', 'read'])
|
||||
async def test_credential_database_errors_never_expose_secrets(auth, operation):
|
||||
import traceback
|
||||
|
||||
markers = ['ACCESS-MARKER', 'REFRESH-MARKER', 'DEVICE-MARKER', 'VERIFIER-MARKER']
|
||||
payload = {
|
||||
'tokens': {'access_token': markers[0], 'refresh_token': markers[1]},
|
||||
'pending': {
|
||||
'authorization_id': 'attempt',
|
||||
'account_uuid': 'u',
|
||||
'device_auth_id': markers[2],
|
||||
'code_verifier': markers[3],
|
||||
},
|
||||
}
|
||||
await seed(auth, payload)
|
||||
if operation == 'read':
|
||||
auth.ap.persistence_mgr.execute_async = AsyncMock(
|
||||
side_effect=sa.exc.StatementError(
|
||||
'failure', 'SELECT credentials', {'payload': payload}, RuntimeError(markers[0])
|
||||
)
|
||||
)
|
||||
else:
|
||||
column = 'payload' if operation in ('save', 'cancel', 'disconnect') else 'lease_owner'
|
||||
condition = ' WHEN NEW.lease_owner IS NULL' if operation == 'release' else ''
|
||||
# Trigger errors can themselves contain secrets, even for parameter-free writes.
|
||||
await auth.ap.persistence_mgr.execute_async(
|
||||
sa.text(
|
||||
f'CREATE TRIGGER reject_write BEFORE UPDATE OF {column} ON codex_credentials{condition} '
|
||||
f"BEGIN SELECT RAISE(ABORT, '{' '.join(markers)}'); END"
|
||||
)
|
||||
)
|
||||
with pytest.raises(ValueError, match='credential storage') as caught:
|
||||
if operation == 'save':
|
||||
async with auth._lease('w', 'p') as owner:
|
||||
await auth._save('w', 'p', owner, payload)
|
||||
elif operation == 'cancel':
|
||||
await auth.cancel(context(), 'p', 'attempt')
|
||||
elif operation == 'disconnect':
|
||||
await auth.disconnect(context(), 'p')
|
||||
elif operation == 'read':
|
||||
await auth._read('w', 'p')
|
||||
else:
|
||||
async with auth._lease('w', 'p'):
|
||||
pass
|
||||
rendered = ''.join(traceback.format_exception(caught.value))
|
||||
assert all(marker not in rendered for marker in markers)
|
||||
assert caught.value.__suppress_context__
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_credential_serialization_failure_is_sanitized(auth):
|
||||
import traceback
|
||||
|
||||
class Secret:
|
||||
def __repr__(self):
|
||||
return 'SERIALIZATION-SECRET'
|
||||
|
||||
with pytest.raises(ValueError, match='credential storage') as caught:
|
||||
async with auth._lease('w', 'p') as owner:
|
||||
await auth._save('w', 'p', owner, {'tokens': {'refresh_token': Secret()}})
|
||||
assert 'SERIALIZATION-SECRET' not in ''.join(traceback.format_exception(caught.value))
|
||||
assert caught.value.__suppress_context__
|
||||
|
||||
|
||||
def test_token_refresh_fallback_and_config_validation():
|
||||
old = _tokens(token_response())
|
||||
refreshed = _tokens({'access_token': 'opaque-access', 'expires_in': 3600}, old)
|
||||
assert refreshed['refresh_token'] == old['refresh_token']
|
||||
assert refreshed['connection_id'] == old['connection_id']
|
||||
for expiry in [float('nan'), float('inf'), -1, 'bad']:
|
||||
with pytest.raises(ValueError):
|
||||
_tokens(token_response(expires_in=expiry))
|
||||
data = {'requester': 'openai-codex'}
|
||||
validate_config(data)
|
||||
assert data['api_keys'] == []
|
||||
for update in [{'base_url': 'https://evil.invalid'}, {'api_keys': ['secret']}]:
|
||||
with pytest.raises(ValueError):
|
||||
validate_config({**data, **update})
|
||||
ordinary = {'requester': 'openai-chat-completions', 'api_keys': ['key'], 'base_url': 'https://custom.invalid'}
|
||||
before = dict(ordinary)
|
||||
validate_config(ordinary)
|
||||
assert ordinary == before
|
||||
@@ -17,6 +17,7 @@ from sqlalchemy import text
|
||||
from sqlalchemy.ext.asyncio import create_async_engine
|
||||
|
||||
from langbot.pkg.entity.persistence.base import Base
|
||||
from langbot.pkg.persistence import mgr as persistence_mgr # noqa: F401 -- register all ORM tables
|
||||
from langbot.pkg.persistence.alembic_runner import (
|
||||
run_alembic_downgrade,
|
||||
run_alembic_upgrade,
|
||||
@@ -108,7 +109,6 @@ class TestSQLiteMigrationUpgrade:
|
||||
await run_alembic_upgrade(sqlite_engine, 'head')
|
||||
|
||||
assert await get_alembic_current(sqlite_engine) == _get_script_head()
|
||||
assert _get_script_head() == '0021_merge_reasoning_config'
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_upgrade_from_reasoning_config_head_to_merged_head(self, sqlite_engine):
|
||||
@@ -119,7 +119,7 @@ class TestSQLiteMigrationUpgrade:
|
||||
await run_alembic_stamp(sqlite_engine, '0018_llm_reasoning_config')
|
||||
await run_alembic_upgrade(sqlite_engine, 'head')
|
||||
|
||||
assert await get_alembic_current(sqlite_engine) == '0021_merge_reasoning_config'
|
||||
assert await get_alembic_current(sqlite_engine) == _get_script_head()
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_upgrade_from_baseline_to_head(self, sqlite_engine):
|
||||
@@ -280,6 +280,15 @@ class TestSQLiteMigrationUpgrade:
|
||||
class TestSQLiteMigrationFreshDatabase:
|
||||
"""Tests for fresh database workflow."""
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_bot_scoped_sessions_skips_absent_table(self, sqlite_engine):
|
||||
"""A partial schema needs no session key migration in either direction."""
|
||||
await run_alembic_stamp(sqlite_engine, '0022_codex_credentials')
|
||||
await run_alembic_upgrade(sqlite_engine, '0023_bot_scoped_sessions')
|
||||
assert await get_alembic_current(sqlite_engine) == '0023_bot_scoped_sessions'
|
||||
await run_alembic_downgrade(sqlite_engine, '0022_codex_credentials')
|
||||
assert await get_alembic_current(sqlite_engine) == '0022_codex_credentials'
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_fresh_db_upgrade_from_scratch(self, tmp_path):
|
||||
"""
|
||||
|
||||
@@ -549,10 +549,14 @@ class TestPostgreSQLWorkspaceMigration:
|
||||
await conn.run_sync(lambda sync_conn: sa.inspect(sync_conn).get_table_names())
|
||||
)
|
||||
assert 'workspaces' not in tables_before_migration
|
||||
assert 'codex_credentials' not in tables_before_migration
|
||||
assert 'passkey_credentials' not in tables_before_migration
|
||||
|
||||
await manager._initialize_managed_schema()
|
||||
|
||||
async with postgres_engine.connect() as conn:
|
||||
assert 'codex_credentials' in await conn.run_sync(lambda sync: sa.inspect(sync).get_table_names())
|
||||
assert 'passkey_credentials' in await conn.run_sync(lambda sync: sa.inspect(sync).get_table_names())
|
||||
account = (await conn.execute(text('SELECT uuid, status, source FROM users'))).mappings().one()
|
||||
workspace = (
|
||||
(await conn.execute(text('SELECT * FROM workspaces WHERE source = :source'), {'source': 'local'}))
|
||||
|
||||
@@ -0,0 +1,354 @@
|
||||
"""Monitoring regressions through asyncpg, Cloud UoW guards, and migrated RLS.
|
||||
|
||||
TEST_POSTGRES_URL must identify a disposable PostgreSQL/pgvector test server
|
||||
with permission to create databases and roles. Each run owns a fresh database;
|
||||
no existing tables are dropped. Without that URL these tests are skipped.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import os
|
||||
import uuid
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
import pytest_asyncio
|
||||
import sqlalchemy as sa
|
||||
from sqlalchemy.ext.asyncio import create_async_engine
|
||||
|
||||
from langbot.pkg.api.http.context import ExecutionContext
|
||||
from langbot.pkg.api.http.service.monitoring import MonitoringService
|
||||
from langbot.pkg.entity.persistence import monitoring as models
|
||||
from langbot.pkg.entity.persistence.workspace import Workspace
|
||||
from langbot.pkg.persistence.mgr import PersistenceManager, PersistenceMode
|
||||
from langbot.pkg.persistence.tenant_uow import TenantScopeRequiredError
|
||||
from langbot.pkg.pipeline.monitoring_helper import MonitoringHelper
|
||||
|
||||
pytestmark = [pytest.mark.integration, pytest.mark.slow, pytest.mark.asyncio(loop_scope='module')]
|
||||
|
||||
WORKSPACE_A = '00000000-0000-0000-0000-00000000000a'
|
||||
WORKSPACE_B = '00000000-0000-0000-0000-00000000000b'
|
||||
RESOURCE = dict(bot_id='same-bot', bot_name='Bot', pipeline_id='same-pipeline', pipeline_name='Pipeline')
|
||||
MONITORING_TABLES = tuple(
|
||||
table for table in models.MonitoringMessage.metadata.sorted_tables if table.name.startswith('monitoring_')
|
||||
)
|
||||
|
||||
|
||||
def _context(workspace_uuid):
|
||||
return ExecutionContext(
|
||||
instance_uuid='monitoring-postgres-test',
|
||||
workspace_uuid=workspace_uuid,
|
||||
placement_generation=1,
|
||||
bot_uuid=RESOURCE['bot_id'],
|
||||
pipeline_uuid=RESOURCE['pipeline_id'],
|
||||
)
|
||||
|
||||
|
||||
def _application(url):
|
||||
return SimpleNamespace(
|
||||
instance_config=SimpleNamespace(
|
||||
data={
|
||||
'database': {
|
||||
'use': 'postgresql',
|
||||
'postgresql': {
|
||||
'host': url.host,
|
||||
'port': url.port,
|
||||
'user': url.username,
|
||||
'password': url.password,
|
||||
'database': url.database,
|
||||
},
|
||||
}
|
||||
}
|
||||
),
|
||||
logger=logging.getLogger('monitoring-postgres-test'),
|
||||
)
|
||||
|
||||
|
||||
@pytest_asyncio.fixture(scope='module', loop_scope='module')
|
||||
async def cloud_database():
|
||||
url = os.environ.get('TEST_POSTGRES_URL')
|
||||
if not url:
|
||||
pytest.skip('TEST_POSTGRES_URL not set')
|
||||
admin_url = sa.engine.make_url(url)
|
||||
admin = create_async_engine(admin_url, isolation_level='AUTOCOMMIT')
|
||||
suffix = uuid.uuid4().hex[:12]
|
||||
database_name = f'lb_monitoring_{suffix}'
|
||||
runtime_role = f'lb_monitoring_{suffix}'
|
||||
password = f'Test{uuid.uuid4().hex}'
|
||||
database_created = role_created = False
|
||||
release_manager = runtime_manager = None
|
||||
quote = admin.dialect.identifier_preparer.quote
|
||||
from langbot.pkg.persistence import mgr as mgr_module
|
||||
from langbot.pkg.persistence.databases.postgresql import PostgreSQLDatabaseManager
|
||||
from langbot.pkg.utils import constants
|
||||
|
||||
with pytest.MonkeyPatch.context() as patch:
|
||||
patch.setattr(mgr_module.database, 'preregistered_managers', [PostgreSQLDatabaseManager])
|
||||
patch.setattr(constants, 'instance_id', 'monitoring-postgres-test')
|
||||
try:
|
||||
async with admin.connect() as conn:
|
||||
await conn.execute(sa.text(f'CREATE DATABASE {quote(database_name)}'))
|
||||
database_created = True
|
||||
await conn.execute(
|
||||
sa.text(f"CREATE ROLE {quote(runtime_role)} LOGIN NOSUPERUSER NOBYPASSRLS PASSWORD '{password}'")
|
||||
)
|
||||
role_created = True
|
||||
release_app = _application(admin_url.set(database=database_name))
|
||||
release_manager = PersistenceManager(release_app, mode=PersistenceMode.RELEASE_MIGRATION)
|
||||
release_app.persistence_mgr = release_manager
|
||||
await release_manager.initialize()
|
||||
async with release_manager.get_db_engine().begin() as conn:
|
||||
for workspace in (WORKSPACE_A, WORKSPACE_B):
|
||||
await conn.execute(
|
||||
sa.insert(Workspace).values(
|
||||
uuid=workspace,
|
||||
instance_uuid='monitoring-postgres-test',
|
||||
name=workspace,
|
||||
slug=workspace,
|
||||
source='cloud_projection',
|
||||
)
|
||||
)
|
||||
tables = release_manager._runtime_business_table_names()
|
||||
quoted_tables = ', '.join(f'public.{quote(name)}' for name in tables)
|
||||
await conn.execute(
|
||||
sa.text(f'GRANT CONNECT ON DATABASE {quote(database_name)} TO {quote(runtime_role)}')
|
||||
)
|
||||
await conn.execute(sa.text(f'GRANT USAGE ON SCHEMA public TO {quote(runtime_role)}'))
|
||||
await conn.execute(
|
||||
sa.text(f'GRANT SELECT, INSERT, UPDATE, DELETE ON TABLE {quoted_tables} TO {quote(runtime_role)}')
|
||||
)
|
||||
await conn.execute(sa.text(f'GRANT SELECT ON public.alembic_version TO {quote(runtime_role)}'))
|
||||
sequences = await release_manager._runtime_business_sequence_names(conn, tables)
|
||||
if sequences:
|
||||
names = ', '.join(f'public.{quote(name)}' for name in sequences)
|
||||
await conn.execute(sa.text(f'GRANT USAGE, SELECT ON SEQUENCE {names} TO {quote(runtime_role)}'))
|
||||
runtime_app = _application(admin_url.set(database=database_name, username=runtime_role, password=password))
|
||||
runtime_manager = PersistenceManager(runtime_app, mode=PersistenceMode.CLOUD_RUNTIME)
|
||||
runtime_app.persistence_mgr = runtime_manager
|
||||
await runtime_manager.initialize()
|
||||
runtime_app.monitoring_service = MonitoringService(runtime_app)
|
||||
yield runtime_app, release_manager.get_db_engine()
|
||||
finally:
|
||||
if runtime_manager is not None:
|
||||
await runtime_manager.shutdown()
|
||||
if release_manager is not None:
|
||||
await release_manager.shutdown()
|
||||
async with admin.connect() as conn:
|
||||
if database_created:
|
||||
await conn.execute(sa.text(f'DROP DATABASE {quote(database_name)} WITH (FORCE)'))
|
||||
if role_created:
|
||||
await conn.execute(sa.text(f'DROP ROLE {quote(runtime_role)}'))
|
||||
await admin.dispose()
|
||||
|
||||
|
||||
@pytest_asyncio.fixture(loop_scope='module')
|
||||
async def service(cloud_database):
|
||||
application, admin = cloud_database
|
||||
async with admin.begin() as conn:
|
||||
for table in MONITORING_TABLES:
|
||||
await conn.execute(sa.delete(table))
|
||||
application.instance_config.data.pop('monitoring', None)
|
||||
return application.monitoring_service
|
||||
|
||||
|
||||
async def _read(service, method, context, *args, **kwargs):
|
||||
# HTTP auth binds a tenant scope; exercise that same guard for service reads.
|
||||
async with service.ap.persistence_mgr.tenant_scope(context.workspace_uuid):
|
||||
return await getattr(service, method)(context, *args, **kwargs)
|
||||
|
||||
|
||||
def _query(context, sender_id):
|
||||
return SimpleNamespace(
|
||||
_execution_context=context,
|
||||
launcher_type='person',
|
||||
launcher_id='same-user',
|
||||
sender_id=sender_id,
|
||||
message_chain=SimpleNamespace(model_dump=lambda: [{'type': 'Plain', 'text': 'hello'}]),
|
||||
resp_message_chain=[SimpleNamespace(model_dump=lambda: [{'type': 'Plain', 'text': 'reply'}])],
|
||||
message_event=SimpleNamespace(sender=SimpleNamespace(nickname='Alice')),
|
||||
variables={'public': 'value', '_private': 'hidden'},
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize('user_id', [123456789, -100123456789, 0, None, '', '00123', ' opaque用户 '])
|
||||
@pytest.mark.parametrize('record_type', ['message', 'session', 'feedback'])
|
||||
async def test_optional_user_ids_round_trip_through_asyncpg(service, user_id, record_type):
|
||||
context = _context(WORKSPACE_A)
|
||||
expected = str(user_id) if isinstance(user_id, int) else user_id
|
||||
if record_type == 'message':
|
||||
record_id = await service.record_message(
|
||||
context,
|
||||
**RESOURCE,
|
||||
message_content='hello',
|
||||
session_id='same-session',
|
||||
user_id=user_id,
|
||||
)
|
||||
details = await _read(service, 'get_message_details', context, record_id)
|
||||
assert details['message']['user_id'] == expected
|
||||
elif record_type == 'session':
|
||||
await service.record_session_start(context, **RESOURCE, session_id='same-session', user_id=user_id)
|
||||
rows, total = await _read(service, 'get_sessions', context)
|
||||
assert total == 1
|
||||
assert rows[0]['user_id'] == expected
|
||||
else:
|
||||
await service.record_feedback(context, feedback_id='same-feedback', feedback_type=1, user_id=user_id)
|
||||
rows, total = await _read(service, 'get_feedback_list', context)
|
||||
assert total == 1
|
||||
assert rows[0]['user_id'] == expected
|
||||
|
||||
|
||||
@pytest.mark.parametrize('user_id', [123456789, -100123456789])
|
||||
async def test_query_lifecycle_persists_messages_session_and_llm_link(service, user_id, caplog):
|
||||
context = _context(WORKSPACE_A)
|
||||
query = _query(context, user_id)
|
||||
message_id = await MonitoringHelper.record_query_start(service.ap, query, **RESOURCE)
|
||||
assert message_id, caplog.text
|
||||
await MonitoringHelper.record_llm_call(
|
||||
service.ap,
|
||||
query,
|
||||
**RESOURCE,
|
||||
model_name='model',
|
||||
input_tokens=3,
|
||||
output_tokens=5,
|
||||
duration_ms=25,
|
||||
message_id=message_id,
|
||||
)
|
||||
await MonitoringHelper.record_query_success(service.ap, message_id, query)
|
||||
await MonitoringHelper.record_query_response(service.ap, query, **RESOURCE)
|
||||
rows, total = await _read(service, 'get_messages', context)
|
||||
assert total == 2
|
||||
assert {row['role'] for row in rows} == {'user', 'assistant'}
|
||||
assert {row['user_id'] for row in rows} == {str(user_id)}
|
||||
details = await _read(service, 'get_message_details', context, message_id)
|
||||
assert details['message']['status'] == 'success'
|
||||
assert details['message']['variables'] == '{"public": "value"}'
|
||||
assert details['llm_calls'][0]['message_id'] == message_id
|
||||
assert details['llm_stats']['total_tokens'] == 8
|
||||
sessions, total = await _read(service, 'get_sessions', context)
|
||||
assert total == 1
|
||||
assert sessions[0]['session_id'] == 'person_same-user'
|
||||
assert sessions[0]['user_id'] == str(user_id)
|
||||
assert not [record for record in caplog.records if record.levelno >= logging.ERROR]
|
||||
|
||||
|
||||
@pytest.mark.parametrize('user_id', [123, -123])
|
||||
async def test_query_error_persists_error_message_and_linked_log(service, user_id, caplog):
|
||||
context = _context(WORKSPACE_A)
|
||||
message_id = await MonitoringHelper.record_query_error(
|
||||
service.ap,
|
||||
_query(context, user_id),
|
||||
**RESOURCE,
|
||||
error=ValueError('failed query'),
|
||||
)
|
||||
assert message_id, caplog.text
|
||||
details = await _read(service, 'get_message_details', context, message_id)
|
||||
assert details['message']['user_id'] == str(user_id)
|
||||
assert details['message']['status'] == 'error'
|
||||
assert details['errors'][0]['message_id'] == message_id
|
||||
assert details['errors'][0]['error_type'] == 'ValueError'
|
||||
|
||||
|
||||
@pytest.mark.parametrize('user_id', [True, 1.5, b'123', ['123']])
|
||||
@pytest.mark.parametrize('record_type', ['message', 'session', 'feedback'])
|
||||
async def test_unsupported_user_ids_fail_at_the_write_boundary(service, user_id, record_type):
|
||||
context = _context(WORKSPACE_A)
|
||||
with pytest.raises(TypeError, match='user_id must be a string, integer, or None'):
|
||||
if record_type == 'message':
|
||||
await service.record_message(
|
||||
context,
|
||||
**RESOURCE,
|
||||
message_content='hello',
|
||||
session_id='session',
|
||||
user_id=user_id,
|
||||
)
|
||||
elif record_type == 'session':
|
||||
await service.record_session_start(context, **RESOURCE, session_id='session', user_id=user_id)
|
||||
else:
|
||||
await service.record_feedback(context, feedback_id='feedback', feedback_type=1, user_id=user_id)
|
||||
async with service.ap.persistence_mgr.tenant_scope(WORKSPACE_A):
|
||||
for model in (models.MonitoringMessage, models.MonitoringSession, models.MonitoringFeedback):
|
||||
count = await service.ap.persistence_mgr.execute_async(sa.select(sa.func.count()).select_from(model))
|
||||
assert count.scalar_one() == 0
|
||||
|
||||
|
||||
async def test_session_analysis_aggregates_under_cloud_sql_guard(service):
|
||||
context = _context(WORKSPACE_A)
|
||||
await service.record_session_start(context, **RESOURCE, session_id='same-session')
|
||||
await service.record_message(context, **RESOURCE, session_id='same-session', message_content='hello')
|
||||
result = await _read(service, 'get_session_analysis', context, 'same-session')
|
||||
assert result['found'] is True
|
||||
assert result['message_stats'] == {'total': 1, 'success': 1, 'error': 0, 'pending': 0}
|
||||
assert result['llm_stats']['total_calls'] == 0
|
||||
assert result['tool_stats']['total_calls'] == 0
|
||||
assert result['session_duration_seconds'] == 0
|
||||
|
||||
|
||||
async def test_rls_is_enforced_without_application_workspace_predicates(service, cloud_database):
|
||||
_, admin = cloud_database
|
||||
for workspace in (WORKSPACE_A, WORKSPACE_B):
|
||||
await service.record_message(
|
||||
_context(workspace), **RESOURCE, session_id='same-session', message_content=workspace
|
||||
)
|
||||
async with admin.connect() as conn:
|
||||
states = (
|
||||
await conn.execute(
|
||||
sa.text(
|
||||
'SELECT relname, relrowsecurity, relforcerowsecurity FROM pg_class '
|
||||
"WHERE relname LIKE 'monitoring_%' AND relkind = 'r'"
|
||||
)
|
||||
)
|
||||
).all()
|
||||
assert len(states) == len(MONITORING_TABLES)
|
||||
assert all(enabled and forced for _, enabled, forced in states)
|
||||
engine = service.ap.persistence_mgr.get_db_engine()
|
||||
async with engine.connect() as conn:
|
||||
role = (
|
||||
await conn.execute(sa.text('SELECT rolsuper, rolbypassrls FROM pg_roles WHERE rolname = current_user'))
|
||||
).one()
|
||||
assert role == (False, False)
|
||||
assert (await conn.execute(sa.select(models.MonitoringMessage.id))).all() == []
|
||||
for workspace in (WORKSPACE_A, WORKSPACE_B):
|
||||
async with service.ap.persistence_mgr.tenant_uow(workspace):
|
||||
rows = (
|
||||
await service.ap.persistence_mgr.execute_async(sa.select(models.MonitoringMessage.workspace_uuid))
|
||||
).all()
|
||||
assert rows == [(workspace,)]
|
||||
with pytest.raises(TenantScopeRequiredError):
|
||||
await service.ap.persistence_mgr.execute_async(sa.select(models.MonitoringMessage.id))
|
||||
|
||||
|
||||
async def test_traffic_series_aggregates_all_rows_under_cloud_rls(service):
|
||||
import datetime
|
||||
from langbot.pkg.api.http.service.monitoring_traffic import get_traffic_series
|
||||
|
||||
context = _context(WORKSPACE_A)
|
||||
for workspace, count in ((WORKSPACE_A, 61), (WORKSPACE_B, 2)):
|
||||
async with service.ap.persistence_mgr.tenant_scope(workspace):
|
||||
await service.ap.persistence_mgr.execute_async(
|
||||
sa.insert(models.MonitoringMessage).values(
|
||||
[
|
||||
dict(
|
||||
workspace_uuid=workspace,
|
||||
id=f'{workspace}-m-{i}',
|
||||
**RESOURCE,
|
||||
session_id='shared',
|
||||
message_content='test',
|
||||
status='success',
|
||||
level='info',
|
||||
timestamp=datetime.datetime(2026, 9, 11, 1, 30),
|
||||
)
|
||||
for i in range(count)
|
||||
]
|
||||
)
|
||||
)
|
||||
async with service.ap.persistence_mgr.tenant_uow(WORKSPACE_A):
|
||||
result = await get_traffic_series(
|
||||
service.ap,
|
||||
context,
|
||||
bot_ids=[RESOURCE['bot_id']],
|
||||
start_time=datetime.datetime(2026, 9, 11),
|
||||
end_time=datetime.datetime(2026, 9, 12),
|
||||
)
|
||||
assert result['truncated'] is False
|
||||
assert sum(point['messages'] for point in result['points']) == 61
|
||||
@@ -142,7 +142,7 @@ async def test_legacy_sqlite_resources_are_backfilled_and_contracted(tmp_path):
|
||||
assert pk_columns == {
|
||||
'binary_storages': ('workspace_uuid', 'unique_key'),
|
||||
'plugin_settings': ('workspace_uuid', 'plugin_author', 'plugin_name'),
|
||||
'monitoring_sessions': ('workspace_uuid', 'session_id'),
|
||||
'monitoring_sessions': ('workspace_uuid', 'bot_id', 'session_id'),
|
||||
}
|
||||
|
||||
pipeline_run_foreign_keys = await _inspect(
|
||||
@@ -237,8 +237,10 @@ async def test_sqlite_scoped_keys_allow_cross_workspace_but_reject_same_workspac
|
||||
await conn.execute(
|
||||
sa.text(
|
||||
'INSERT INTO monitoring_sessions '
|
||||
'(workspace_uuid, session_id, bot_id, last_activity, is_active) '
|
||||
"VALUES (:workspace_uuid, 'session-1', 'bot-2', CURRENT_TIMESTAMP, 1)"
|
||||
'(workspace_uuid, session_id, bot_id, bot_name, pipeline_id, pipeline_name, '
|
||||
'start_time, last_activity, message_count, is_active) '
|
||||
"VALUES (:workspace_uuid, 'session-1', 'bot-2', 'bot', 'pipeline-2', 'pipeline', "
|
||||
'CURRENT_TIMESTAMP, CURRENT_TIMESTAMP, 1, 1)'
|
||||
),
|
||||
{'workspace_uuid': second_workspace_uuid},
|
||||
)
|
||||
|
||||
@@ -5,6 +5,7 @@ import logging
|
||||
import os
|
||||
import pathlib
|
||||
import sqlite3
|
||||
from contextlib import closing
|
||||
|
||||
import pytest
|
||||
import sqlalchemy as sa
|
||||
@@ -34,7 +35,7 @@ def _manifest_payloads(backup_directory) -> list[dict]:
|
||||
|
||||
def _assert_verified_backup(payload: dict) -> None:
|
||||
backup_path = pathlib.Path(payload['backup_path'])
|
||||
with sqlite3.connect(f'{backup_path.as_uri()}?mode=ro', uri=True) as connection:
|
||||
with closing(sqlite3.connect(f'{backup_path.as_uri()}?mode=ro', uri=True)) as connection:
|
||||
assert connection.execute('PRAGMA quick_check').fetchall() == [('ok',)]
|
||||
assert connection.execute('SELECT version_num FROM alembic_version').fetchone()[0] == payload['source_revision']
|
||||
|
||||
|
||||
@@ -403,10 +403,12 @@ async def test_persistence_startup_defers_workspace_tables_until_account_upgrade
|
||||
await conn.run_sync(lambda sync_conn: sa.inspect(sync_conn).get_table_names())
|
||||
)
|
||||
assert 'workspaces' not in tables_before_migration
|
||||
assert 'codex_credentials' not in tables_before_migration
|
||||
|
||||
await manager._run_alembic_migrations()
|
||||
|
||||
async with engine.connect() as conn:
|
||||
assert 'codex_credentials' in await conn.run_sync(lambda sync: sa.inspect(sync).get_table_names())
|
||||
workspace = (
|
||||
(await conn.execute(sa.text("SELECT * FROM workspaces WHERE source = 'local'"))).mappings().one()
|
||||
)
|
||||
|
||||
@@ -0,0 +1,307 @@
|
||||
"""Real Core/SDK protocol regression tests; no subprocesses or external services.
|
||||
|
||||
Run against the intended local SDK (``uv run --no-sync`` after local install).
|
||||
The in-memory transport carries JSON strings through Handler.run on both sides;
|
||||
send_file, envelope validation, base64 decoding and transfer storage are real.
|
||||
Only Core's database/object-storage services, parser dispatch/provider and host
|
||||
sandbox prerequisite probing are doubles. Worker launch/registration is
|
||||
represented by its already-registered state.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
from contextlib import asynccontextmanager
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import AsyncMock
|
||||
|
||||
import pytest
|
||||
|
||||
from langbot.pkg.plugin.handler import RuntimeConnectionHandler
|
||||
from langbot_plugin.entities.io.actions.enums import CommonAction, LangBotToRuntimeAction, PluginToRuntimeAction
|
||||
from langbot_plugin.entities.io.context import ActionContext, InstallationBinding, PluginWorkerPolicy, RuntimeIdentity
|
||||
from langbot_plugin.runtime.context import RuntimeContext
|
||||
from langbot_plugin.runtime.io.connection import Connection
|
||||
from langbot_plugin.entities.io.errors import ActionCallError, ConnectionClosedError
|
||||
from langbot_plugin.runtime.io.handler import FILE_CHUNK_LENGTH, Handler
|
||||
from langbot_plugin.runtime.io.handlers.control import ControlConnectionHandler
|
||||
from langbot_plugin.runtime.io.handlers.plugin import PluginConnectionHandler
|
||||
from langbot_plugin.runtime.plugin.mgr import PluginManager
|
||||
from langbot_plugin.runtime.security import PLUGIN_FILE_STORAGE_DIR_ENV
|
||||
|
||||
|
||||
pytestmark = pytest.mark.asyncio
|
||||
PAYLOAD = bytes(range(256)) * 161 + b'\x00original RAG file\xff'
|
||||
BINDING = InstallationBinding(
|
||||
instance_uuid='instance-a',
|
||||
workspace_uuid='workspace-a',
|
||||
placement_generation=7,
|
||||
installation_uuid='00000000-0000-4000-8000-000000000001',
|
||||
runtime_revision=3,
|
||||
artifact_digest='a' * 64,
|
||||
)
|
||||
LEGACY = ActionContext(**BINDING.model_dump(exclude={'runtime_revision', 'artifact_digest'}))
|
||||
|
||||
|
||||
class QueueConnection(Connection):
|
||||
"""Only the byte transport is replaced, not the request/response machinery."""
|
||||
|
||||
def __init__(self):
|
||||
self.incoming = asyncio.Queue()
|
||||
self.sent = []
|
||||
self.peer = None
|
||||
|
||||
async def send(self, message: str) -> None:
|
||||
assert isinstance(message, str)
|
||||
self.sent.append(json.loads(message))
|
||||
await self.peer.incoming.put(message)
|
||||
|
||||
async def receive(self) -> str:
|
||||
message = await self.incoming.get()
|
||||
if message is None:
|
||||
raise ConnectionClosedError('test transport closed')
|
||||
return message
|
||||
|
||||
async def close(self) -> None:
|
||||
await self.incoming.put(None)
|
||||
await self.peer.incoming.put(None)
|
||||
|
||||
|
||||
def connection_pair():
|
||||
left, right = QueueConnection(), QueueConnection()
|
||||
left.peer, right.peer = right, left
|
||||
return left, right
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def protocol_stack(tmp_path, monkeypatch, profile='oss_dev', binding=LEGACY):
|
||||
monkeypatch.chdir(tmp_path)
|
||||
stored = tmp_path / 'original.bin'
|
||||
stored.write_bytes(PAYLOAD)
|
||||
storage_calls = []
|
||||
|
||||
async def get_file_stream(execution_context, storage_path):
|
||||
storage_calls.append((execution_context, storage_path))
|
||||
assert execution_context.workspace_uuid == BINDING.workspace_uuid
|
||||
assert storage_path == 'knowledge/original.bin'
|
||||
return stored.read_bytes()
|
||||
|
||||
async def get_execution_binding(workspace_uuid, expected_generation):
|
||||
assert workspace_uuid == BINDING.workspace_uuid
|
||||
assert expected_generation == BINDING.placement_generation
|
||||
return BINDING
|
||||
|
||||
setting = SimpleNamespace(
|
||||
plugin_author='tester',
|
||||
plugin_name='engine',
|
||||
installation_uuid=BINDING.installation_uuid,
|
||||
runtime_revision=BINDING.runtime_revision,
|
||||
artifact_digest=BINDING.artifact_digest,
|
||||
)
|
||||
app = SimpleNamespace(
|
||||
deployment=SimpleNamespace(mode='oss' if profile == 'oss_dev' else 'cloud'),
|
||||
logger=logging.getLogger(__name__),
|
||||
persistence_mgr=SimpleNamespace(execute_async=AsyncMock(return_value=SimpleNamespace(first=lambda: setting))),
|
||||
workspace_service=SimpleNamespace(get_execution_binding=get_execution_binding),
|
||||
rag_runtime_service=SimpleNamespace(get_file_stream=get_file_stream),
|
||||
)
|
||||
core_conn, control_conn = connection_pair()
|
||||
monkeypatch.setenv(PLUGIN_FILE_STORAGE_DIR_ENV, str(tmp_path / 'core-transfer'))
|
||||
core = RuntimeConnectionHandler(core_conn, AsyncMock(return_value=False), app)
|
||||
core.register_installation_binding(BINDING, plugin_author='tester', plugin_name='engine')
|
||||
runtime = RuntimeContext()
|
||||
runtime.plugin_mgr = PluginManager(runtime)
|
||||
# No worker is launched: omit only host nsjail/cgroup prerequisite probing.
|
||||
monkeypatch.setattr(runtime.plugin_mgr.worker_launcher, 'configure', lambda policy, profile: None)
|
||||
monkeypatch.setenv(PLUGIN_FILE_STORAGE_DIR_ENV, str(tmp_path / 'runtime-transfer'))
|
||||
control = ControlConnectionHandler(control_conn, runtime)
|
||||
runtime.activate_control_handler(control)
|
||||
bridge_conn, plugin_conn = connection_pair()
|
||||
bridge = PluginConnectionHandler(bridge_conn, runtime, file_storage_dir=str(tmp_path / 'bridge-transfer'))
|
||||
plugin = Handler(plugin_conn, file_storage_dir=str(tmp_path / 'plugin-transfer'))
|
||||
# Trusted state left by registration, not plugin-supplied action data.
|
||||
bridge.bind_action_context(binding)
|
||||
runtime.plugin_mgr.plugin_handlers.append(bridge)
|
||||
runtime.plugin_mgr.plugins.append(SimpleNamespace(_runtime_plugin_handler=bridge))
|
||||
handlers = [core, control, bridge, plugin]
|
||||
tasks = [asyncio.create_task(handler.run()) for handler in handlers]
|
||||
try:
|
||||
await asyncio.wait_for(
|
||||
core.set_runtime_config(
|
||||
runtime_identity=RuntimeIdentity(instance_uuid='instance-a', runtime_id='test-runtime'),
|
||||
worker_policy=PluginWorkerPolicy(
|
||||
max_cpus=1,
|
||||
max_memory_mb=128,
|
||||
max_pids=32,
|
||||
max_open_files=64,
|
||||
max_file_size_mb=8,
|
||||
require_hard_limits=False,
|
||||
),
|
||||
runtime_profile=profile,
|
||||
cloud_service_url=None,
|
||||
),
|
||||
5,
|
||||
)
|
||||
if isinstance(binding, InstallationBinding):
|
||||
runtime.activate_installation_binding(binding)
|
||||
else:
|
||||
runtime.bind_workspace(binding)
|
||||
yield SimpleNamespace(
|
||||
core=core,
|
||||
control=control,
|
||||
runtime=runtime,
|
||||
bridge=bridge,
|
||||
plugin=plugin,
|
||||
core_conn=core_conn,
|
||||
control_conn=control_conn,
|
||||
bridge_conn=bridge_conn,
|
||||
plugin_conn=plugin_conn,
|
||||
app=app,
|
||||
storage_calls=storage_calls,
|
||||
)
|
||||
finally:
|
||||
for handler in handlers:
|
||||
await handler.close()
|
||||
await asyncio.wait_for(asyncio.gather(*tasks, return_exceptions=True), 5)
|
||||
|
||||
|
||||
def assert_chunks(connection, binding, payload=PAYLOAD):
|
||||
chunks = [message for message in connection.sent if message.get('action') == CommonAction.FILE_CHUNK.value]
|
||||
expected = (len(payload) + FILE_CHUNK_LENGTH - 1) // FILE_CHUNK_LENGTH
|
||||
assert expected > 1
|
||||
assert len(chunks) == expected
|
||||
assert [chunk['data']['chunk_index'] for chunk in chunks] == list(range(expected))
|
||||
assert {chunk['data']['chunk_amount'] for chunk in chunks} == {expected}
|
||||
assert all(chunk['context'] == binding.model_dump() for chunk in chunks)
|
||||
assert len({chunk['data']['file_key'] for chunk in chunks}) == 1
|
||||
return chunks[0]['data']['file_key']
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
'profile,binding',
|
||||
[('oss_dev', LEGACY), ('oss_dev', BINDING), ('shared', BINDING)],
|
||||
ids=['legacy-oss', 'managed-oss', 'managed-shared'],
|
||||
)
|
||||
async def test_knowledge_file_roundtrip_reaches_plugin_original_bytes(tmp_path, monkeypatch, profile, binding):
|
||||
async with protocol_stack(tmp_path, monkeypatch, profile, binding) as stack:
|
||||
# Legacy plugin API sends no authority; Runtime supplies its trusted binding.
|
||||
result = await asyncio.wait_for(
|
||||
stack.plugin.call_action(
|
||||
PluginToRuntimeAction.GET_KNOWLEDEGE_FILE_STREAM,
|
||||
{'storage_path': 'knowledge/original.bin'},
|
||||
),
|
||||
5,
|
||||
)
|
||||
assert await stack.plugin.read_local_file(result['file_key']) == PAYLOAD
|
||||
assert len(stack.storage_calls) == 1
|
||||
core_key = assert_chunks(stack.core_conn, binding)
|
||||
plugin_key = assert_chunks(stack.bridge_conn, binding)
|
||||
assert result['file_key'] == plugin_key != core_key
|
||||
assert not (Path(stack.control.file_storage_dir) / core_key).exists()
|
||||
assert not stack.control._owned_transfer_files
|
||||
callbacks = [
|
||||
message
|
||||
for message in stack.control_conn.sent
|
||||
if message.get('action') == PluginToRuntimeAction.GET_KNOWLEDEGE_FILE_STREAM.value
|
||||
]
|
||||
assert len(callbacks) == 1
|
||||
assert callbacks[0]['context'] == binding.model_dump()
|
||||
assert callbacks[0]['data'] == {'storage_path': 'knowledge/original.bin'}
|
||||
|
||||
|
||||
async def test_shared_control_rejects_legacy_chunks_before_storage(tmp_path, monkeypatch):
|
||||
async with protocol_stack(tmp_path, monkeypatch, 'shared', BINDING) as stack:
|
||||
with stack.core.installation_scope(LEGACY):
|
||||
with pytest.raises(ActionCallError, match='InstallationBinding|Legacy FILE_CHUNK'):
|
||||
await asyncio.wait_for(stack.core.send_file(PAYLOAD, ''), 5)
|
||||
assert not list(Path(stack.control.file_storage_dir).iterdir())
|
||||
assert not stack.control._owned_transfer_files
|
||||
|
||||
|
||||
async def test_candidate_artifact_pretransfer_does_not_require_active_installation(tmp_path, monkeypatch):
|
||||
async with protocol_stack(tmp_path, monkeypatch, 'shared', BINDING) as stack:
|
||||
candidate = BINDING.model_copy(
|
||||
update={'installation_uuid': 'candidate-installation', 'runtime_revision': 1, 'artifact_digest': 'c' * 64}
|
||||
)
|
||||
assert not stack.runtime.is_current_installation_binding(candidate)
|
||||
with stack.core.installation_scope(candidate):
|
||||
key = await asyncio.wait_for(stack.core.send_file(PAYLOAD, 'lbp'), 5)
|
||||
assert_chunks(stack.core_conn, candidate)
|
||||
assert await stack.control.read_local_file(key) == PAYLOAD
|
||||
assert not stack.runtime.is_current_installation_binding(candidate)
|
||||
|
||||
|
||||
async def test_nested_parser_target_owns_file_and_action_envelopes(tmp_path, monkeypatch):
|
||||
async with protocol_stack(tmp_path, monkeypatch, 'shared', BINDING) as stack:
|
||||
target = BINDING.model_copy(
|
||||
update={
|
||||
'installation_uuid': 'parser-installation',
|
||||
'runtime_revision': 2,
|
||||
'artifact_digest': 'b' * 64,
|
||||
}
|
||||
)
|
||||
stack.runtime.activate_installation_binding(target)
|
||||
parser_calls = []
|
||||
restored = []
|
||||
|
||||
async def parse_document(author, name, context_data, file_bytes):
|
||||
parser_calls.append((stack.control.current_action_context, author, name, context_data, file_bytes))
|
||||
return {'documents': [{'text': 'parsed'}]}
|
||||
|
||||
stack.runtime.plugin_mgr.parse_document = parse_document
|
||||
|
||||
class ParserConnector:
|
||||
async def require_workspace_context(self, context):
|
||||
assert context.workspace_uuid == BINDING.workspace_uuid
|
||||
|
||||
async def call_parser(self, plugin_name, context_data, file_bytes):
|
||||
assert plugin_name == 'tester/parser'
|
||||
assert stack.core.current_action_context == BINDING
|
||||
with stack.core.installation_scope(target):
|
||||
result = await stack.core.parse_document('tester', 'parser', context_data, file_bytes)
|
||||
restored.append(stack.core.resolve_outbound_action_context(None))
|
||||
return result
|
||||
|
||||
stack.app.plugin_connector = ParserConnector()
|
||||
result = await asyncio.wait_for(
|
||||
stack.plugin.call_action(
|
||||
PluginToRuntimeAction.INVOKE_PARSER,
|
||||
{
|
||||
'plugin_author': 'tester',
|
||||
'plugin_name': 'parser',
|
||||
'storage_path': 'knowledge/original.bin',
|
||||
'filename': 'original.bin',
|
||||
},
|
||||
),
|
||||
5,
|
||||
)
|
||||
assert result == {'documents': [{'text': 'parsed'}]}
|
||||
key = assert_chunks(stack.core_conn, target)
|
||||
parse_requests = [
|
||||
message
|
||||
for message in stack.core_conn.sent
|
||||
if message.get('action') == LangBotToRuntimeAction.PARSE_DOCUMENT.value
|
||||
]
|
||||
assert len(parse_requests) == 1
|
||||
assert parse_requests[0]['context'] == target.model_dump()
|
||||
assert parse_requests[0]['data']['context']['file_key'] == key
|
||||
assert parser_calls == [
|
||||
(
|
||||
target,
|
||||
'tester',
|
||||
'parser',
|
||||
{
|
||||
'mime_type': 'application/octet-stream',
|
||||
'filename': 'original.bin',
|
||||
'metadata': {},
|
||||
},
|
||||
PAYLOAD,
|
||||
)
|
||||
]
|
||||
assert restored == [BINDING]
|
||||
assert stack.core.current_action_context is None
|
||||
assert stack.core.resolve_outbound_action_context(None) is None
|
||||
assert not (Path(stack.control.file_storage_dir) / key).exists()
|
||||
@@ -0,0 +1,123 @@
|
||||
"""Real embedded SeekDB regression tests.
|
||||
|
||||
Install the optional dependency before running these slow tests::
|
||||
|
||||
uv sync --dev --extra seekdb
|
||||
uv run pytest tests/integration/vector/test_seekdb.py -m slow -q
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
from types import SimpleNamespace
|
||||
import uuid
|
||||
|
||||
import pytest
|
||||
|
||||
pytest.importorskip('pyseekdb')
|
||||
|
||||
from langbot.pkg.vector.vdbs.seekdb import SeekDBVectorDatabase
|
||||
|
||||
|
||||
pytestmark = [pytest.mark.integration, pytest.mark.slow]
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
async def backend(tmp_path):
|
||||
app = SimpleNamespace(
|
||||
instance_config=SimpleNamespace(
|
||||
data={
|
||||
'vdb': {
|
||||
'runtime_cache_limit': 16,
|
||||
'seekdb': {
|
||||
'mode': 'embedded',
|
||||
'path': str(tmp_path),
|
||||
'database': 'langbot_test',
|
||||
},
|
||||
}
|
||||
}
|
||||
),
|
||||
logger=SimpleNamespace(
|
||||
info=lambda *args, **kwargs: None,
|
||||
warning=lambda *args, **kwargs: None,
|
||||
),
|
||||
)
|
||||
database = SeekDBVectorDatabase(app)
|
||||
collection = f'test_{uuid.uuid4().hex}'
|
||||
yield database, collection
|
||||
|
||||
await database.delete_collection(collection)
|
||||
await database.close()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_upsert_and_text_round_trip(backend) -> None:
|
||||
database, collection = backend
|
||||
original = 'He said "hello".\nC:\\notes\\file.txt isn\'t empty. 中文'
|
||||
updated = f'Updated: {original}'
|
||||
|
||||
await database.add_embeddings(
|
||||
collection,
|
||||
['document-a'],
|
||||
[[1.0, 0.0, 0.0]],
|
||||
[{'file_id': 'file-a', 'text': original}],
|
||||
[original],
|
||||
)
|
||||
await database.add_embeddings(
|
||||
collection,
|
||||
['document-a'],
|
||||
[[0.0, 1.0, 0.0]],
|
||||
[{'file_id': 'file-a', 'text': updated}],
|
||||
[updated],
|
||||
)
|
||||
|
||||
items, _ = await database.list_by_filter(collection, {'file_id': 'file-a'})
|
||||
|
||||
assert len(items) == 1
|
||||
assert items[0]['id'] == 'document-a'
|
||||
assert items[0]['document'] == updated
|
||||
assert items[0]['metadata']['text'] == updated
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_full_text_and_hybrid_results_keep_relevance_order(backend) -> None:
|
||||
database, collection = backend
|
||||
documents = [
|
||||
'orchid orchid orchid flower',
|
||||
'orchid grows in a garden with many other beautiful plants',
|
||||
'a completely unrelated topic',
|
||||
]
|
||||
|
||||
await database.add_embeddings(
|
||||
collection,
|
||||
['best', 'weak', 'noise'],
|
||||
[[1.0, 0.0, 0.0], [0.9, 0.1, 0.0], [0.0, 0.0, 1.0]],
|
||||
[
|
||||
{'file_id': item_id, 'document_id': item_id, 'text': document}
|
||||
for item_id, document in zip(['best', 'weak', 'noise'], documents, strict=True)
|
||||
],
|
||||
documents,
|
||||
)
|
||||
seekdb_collection = await database.get_or_create_collection(collection)
|
||||
await asyncio.to_thread(seekdb_collection.refresh_index)
|
||||
|
||||
full_text = await database.search(
|
||||
collection,
|
||||
[1.0, 0.0, 0.0],
|
||||
k=3,
|
||||
search_type='full_text',
|
||||
query_text='orchid',
|
||||
)
|
||||
hybrid = await database.search(
|
||||
collection,
|
||||
[1.0, 0.0, 0.0],
|
||||
k=3,
|
||||
search_type='hybrid',
|
||||
query_text='orchid',
|
||||
vector_weight=0.65,
|
||||
)
|
||||
|
||||
assert full_text['ids'][0][:2] == ['best', 'weak']
|
||||
assert full_text['distances'][0] == sorted(full_text['distances'][0])
|
||||
assert hybrid['ids'][0] == ['best', 'weak', 'noise']
|
||||
assert hybrid['distances'][0] == sorted(hybrid['distances'][0])
|
||||
@@ -20,6 +20,7 @@ from langbot.pkg.entity.persistence.model import LLMModel, ModelProvider
|
||||
from langbot.pkg.entity.persistence.pipeline import LegacyPipeline
|
||||
from langbot.pkg.entity.persistence.workspace import Workspace
|
||||
from langbot.pkg.workspace.errors import WorkspaceNotFoundError
|
||||
from langbot.pkg.persistence.mgr import PersistenceManager
|
||||
|
||||
|
||||
pytestmark = pytest.mark.asyncio
|
||||
@@ -28,15 +29,10 @@ WORKSPACE_A = '00000000-0000-0000-0000-00000000000a'
|
||||
WORKSPACE_B = '00000000-0000-0000-0000-00000000000b'
|
||||
|
||||
|
||||
class _PersistenceManager:
|
||||
class _PersistenceManager(PersistenceManager):
|
||||
def __init__(self, engine):
|
||||
self.engine = engine
|
||||
|
||||
async def execute_async(self, *args, **kwargs):
|
||||
async with self.engine.connect() as connection:
|
||||
result = await connection.execute(*args, **kwargs)
|
||||
await connection.commit()
|
||||
return result
|
||||
super().__init__(SimpleNamespace())
|
||||
self.db = SimpleNamespace(get_engine=lambda: engine)
|
||||
|
||||
@staticmethod
|
||||
def serialize_model(model, data, masked_columns=None):
|
||||
|
||||
@@ -1009,6 +1009,37 @@ class TestMCPServiceTestMCPServer:
|
||||
# Verify - returns task ID
|
||||
assert task_id == 123
|
||||
|
||||
@pytest.mark.parametrize('refresh_first', [False, True])
|
||||
async def test_persisted_test_preserves_failure_details(self, refresh_first):
|
||||
from langbot.pkg.provider.tools.loaders.mcp import MCPSessionStatus
|
||||
|
||||
runtime_info = {'status': 'error', 'error_message': 'HTTP 403: access denied'}
|
||||
session = SimpleNamespace(
|
||||
status=MCPSessionStatus.CONNECTED if refresh_first else MCPSessionStatus.ERROR,
|
||||
session=object(),
|
||||
refresh=AsyncMock(side_effect=RuntimeError('refresh failed')),
|
||||
start=AsyncMock(side_effect=RuntimeError('Connection failed, please check URL')),
|
||||
get_runtime_info_dict=Mock(return_value=runtime_info),
|
||||
)
|
||||
captured = {}
|
||||
|
||||
def create_user_task(coroutine, **kwargs):
|
||||
captured.update(coroutine=coroutine, context=kwargs['context'])
|
||||
return SimpleNamespace(id=123)
|
||||
|
||||
ap = SimpleNamespace(
|
||||
tool_mgr=SimpleNamespace(mcp_tool_loader=SimpleNamespace(get_session=Mock(return_value=session))),
|
||||
task_mgr=SimpleNamespace(create_user_task=Mock(side_effect=create_user_task)),
|
||||
)
|
||||
service = _service(ap)
|
||||
service._require_server = AsyncMock(return_value=(_CONTEXT, {'name': 'existing-server'}))
|
||||
await service.test_mcp_server(_CONTEXT, 'existing-server', {})
|
||||
with pytest.raises(RuntimeError, match='Connection failed'):
|
||||
await captured['coroutine']
|
||||
assert captured['context'].metadata['runtime_info'] == runtime_info
|
||||
session.start.assert_awaited_once()
|
||||
assert session.refresh.await_count == int(refresh_first)
|
||||
|
||||
async def test_test_mcp_server_not_found_raises(self):
|
||||
"""Raises ValueError when server not found."""
|
||||
# Setup
|
||||
@@ -1052,6 +1083,45 @@ class TestMCPServiceTestMCPServer:
|
||||
ap.tool_mgr.mcp_tool_loader.load_mcp_server.assert_called_once()
|
||||
assert task_id == 456
|
||||
|
||||
async def test_transient_test_preserves_runtime_info_after_connection_failure(self):
|
||||
runtime_info = {
|
||||
'status': 'error',
|
||||
'error_phase': 'oauth_required',
|
||||
'retry_count': 1,
|
||||
}
|
||||
mock_session = SimpleNamespace(
|
||||
server_name='oauth-server',
|
||||
start=AsyncMock(side_effect=RuntimeError('connection failed')),
|
||||
get_runtime_info_dict=Mock(return_value=runtime_info),
|
||||
shutdown=AsyncMock(),
|
||||
)
|
||||
ap = SimpleNamespace(
|
||||
tool_mgr=SimpleNamespace(
|
||||
mcp_tool_loader=SimpleNamespace(load_mcp_server=AsyncMock(return_value=mock_session))
|
||||
)
|
||||
)
|
||||
captured: dict = {}
|
||||
|
||||
def create_user_task(coroutine, **kwargs):
|
||||
captured['coroutine'] = coroutine
|
||||
captured['context'] = kwargs['context']
|
||||
return SimpleNamespace(id=457)
|
||||
|
||||
ap.task_mgr = SimpleNamespace(create_user_task=Mock(side_effect=create_user_task))
|
||||
service = _service(ap)
|
||||
|
||||
task_id = await service.test_mcp_server(
|
||||
_CONTEXT,
|
||||
'_',
|
||||
{'name': 'OAuth server', 'mode': 'remote', 'enable': True, 'extra_args': {}},
|
||||
)
|
||||
|
||||
assert task_id == 457
|
||||
with pytest.raises(RuntimeError, match='connection failed'):
|
||||
await captured['coroutine']
|
||||
assert captured['context'].metadata['runtime_info'] == runtime_info
|
||||
mock_session.shutdown.assert_awaited_once_with()
|
||||
|
||||
async def test_rejected_transient_test_session_is_shut_down(self):
|
||||
ap = SimpleNamespace()
|
||||
mock_session = MagicMock()
|
||||
|
||||
@@ -0,0 +1,19 @@
|
||||
"""Identifier normalization must not rely on SQLite's permissive codecs."""
|
||||
|
||||
import pytest
|
||||
|
||||
from langbot.pkg.api.http.service import monitoring
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
('value', 'expected'),
|
||||
[(None, None), ('', ''), ('00123', '00123'), (' 用户 ', ' 用户 '), (123, '123'), (-123, '-123'), (0, '0')],
|
||||
)
|
||||
def test_normalize_user_id_preserves_opaque_strings(value, expected):
|
||||
assert monitoring._normalize_user_id(value) == expected
|
||||
|
||||
|
||||
@pytest.mark.parametrize('value', [True, False, 1.5, b'123', ['123'], {'id': 123}])
|
||||
def test_normalize_user_id_rejects_unsupported_types(value):
|
||||
with pytest.raises(TypeError, match='user_id must be a string, integer, or None'):
|
||||
monitoring._normalize_user_id(value)
|
||||
@@ -0,0 +1,220 @@
|
||||
"""Bot-scoped session regressions exercised against real SQL databases."""
|
||||
|
||||
import datetime as dt
|
||||
import logging
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
import sqlalchemy as sa
|
||||
from sqlalchemy.ext.asyncio import create_async_engine
|
||||
|
||||
from langbot.pkg.api.http.context import ExecutionContext
|
||||
from langbot.pkg.api.http.service.monitoring import MonitoringService
|
||||
from langbot.pkg.entity.persistence.base import Base
|
||||
from langbot.pkg.entity.persistence import monitoring as models
|
||||
from langbot.pkg.persistence.mgr import PersistenceManager
|
||||
from langbot.pkg.pipeline.monitoring_helper import MonitoringHelper
|
||||
|
||||
from tests.integration.persistence.test_monitoring_postgres import cloud_database # noqa: F401
|
||||
|
||||
pytestmark = pytest.mark.asyncio
|
||||
|
||||
|
||||
@pytest.mark.asyncio(loop_scope='module')
|
||||
async def test_postgres_upgrade_rls_and_concurrent_bot_counts(cloud_database): # noqa: F811
|
||||
import asyncio
|
||||
import importlib
|
||||
from alembic.migration import MigrationContext
|
||||
from alembic.operations import Operations
|
||||
from tests.integration.persistence.test_monitoring_postgres import WORKSPACE_A, _context, _read
|
||||
|
||||
ap, admin = cloud_database
|
||||
service = ap.monitoring_service
|
||||
ctx = _context(WORKSPACE_A)
|
||||
await service.record_session_start(ctx, session_id='person_42', **resource('a'))
|
||||
for bot in ['a', 'b']:
|
||||
await service.record_message(ctx, session_id='person_42', message_content=bot, **resource(bot))
|
||||
async with admin.begin() as conn:
|
||||
|
||||
def migrate(connection):
|
||||
migration = importlib.import_module('langbot.pkg.persistence.alembic.versions.0023_bot_scoped_sessions')
|
||||
with Operations.context(MigrationContext.configure(connection)):
|
||||
migration.downgrade()
|
||||
migration.upgrade()
|
||||
rls = connection.execute(
|
||||
sa.text("SELECT relrowsecurity, relforcerowsecurity FROM pg_class WHERE relname='monitoring_sessions'")
|
||||
).one()
|
||||
assert tuple(rls) == (True, True)
|
||||
assert (
|
||||
connection.execute(
|
||||
sa.text("SELECT count(*) FROM pg_policies WHERE tablename='monitoring_sessions'")
|
||||
).scalar_one()
|
||||
== 1
|
||||
)
|
||||
|
||||
await conn.run_sync(migrate)
|
||||
rows, total = await _read(service, 'get_sessions', ctx)
|
||||
assert total == 2
|
||||
assert {r['bot_id']: r['message_count'] for r in rows} == {'a': 1, 'b': 1}
|
||||
await asyncio.gather(*[service.record_session_start(ctx, session_id='race', **resource('a')) for _ in range(10)])
|
||||
result = await _read(service, 'get_session_analysis', ctx, 'race', bot_id='a')
|
||||
assert result['session']['message_count'] == 10
|
||||
assert not (await _read(service, 'get_session_analysis', ctx, 'person_42'))['found']
|
||||
assert (await _read(service, 'get_session_analysis', ctx, 'person_42', bot_id='b'))['message_stats']['total'] == 1
|
||||
|
||||
|
||||
async def test_migration_reconstructs_collisions_and_preserves_indexes(service):
|
||||
import importlib
|
||||
from alembic.migration import MigrationContext
|
||||
from alembic.operations import Operations
|
||||
|
||||
engine = service.ap.persistence_mgr.get_db_engine()
|
||||
async with engine.begin() as conn:
|
||||
|
||||
def upgrade(connection):
|
||||
table = models.MonitoringSession.__table__
|
||||
table.drop(connection)
|
||||
metadata = sa.MetaData()
|
||||
legacy = table.to_metadata(metadata)
|
||||
legacy.primary_key._columns.remove(legacy.c.bot_id)
|
||||
legacy.c.bot_id.primary_key = False
|
||||
# Resolve the unchanged Workspace FK in copied metadata.
|
||||
Base.metadata.tables['workspaces'].to_metadata(metadata)
|
||||
legacy.create(connection)
|
||||
now = dt.datetime(2026, 1, 1)
|
||||
connection.execute(
|
||||
sa.insert(legacy).values(
|
||||
workspace_uuid='workspace',
|
||||
session_id='person_42',
|
||||
**resource('a'),
|
||||
message_count=99,
|
||||
start_time=now,
|
||||
last_activity=now,
|
||||
is_active=True,
|
||||
)
|
||||
)
|
||||
for bot in ['a', 'b']:
|
||||
connection.execute(
|
||||
sa.insert(models.MonitoringMessage).values(
|
||||
id=bot,
|
||||
workspace_uuid='workspace',
|
||||
timestamp=now,
|
||||
**resource(bot),
|
||||
session_id='person_42',
|
||||
message_content=bot,
|
||||
role='user',
|
||||
status='success',
|
||||
level='info',
|
||||
)
|
||||
)
|
||||
indexes = {i['name'] for i in sa.inspect(connection).get_indexes('monitoring_sessions')}
|
||||
migration = importlib.import_module('langbot.pkg.persistence.alembic.versions.0023_bot_scoped_sessions')
|
||||
with Operations.context(MigrationContext.configure(connection)):
|
||||
migration.upgrade()
|
||||
migration.upgrade() # Fresh/already-upgraded schema is safe.
|
||||
assert sa.inspect(connection).get_pk_constraint('monitoring_sessions')['constrained_columns'] == [
|
||||
'workspace_uuid',
|
||||
'bot_id',
|
||||
'session_id',
|
||||
]
|
||||
assert indexes <= {i['name'] for i in sa.inspect(connection).get_indexes('monitoring_sessions')}
|
||||
|
||||
await conn.run_sync(upgrade)
|
||||
rows, total = await service.get_sessions(context())
|
||||
assert total == 2
|
||||
assert {r['bot_id']: r['message_count'] for r in rows} == {'a': 1, 'b': 1}
|
||||
assert {r['pipeline_id'] for r in rows} == {'a', 'b'}
|
||||
|
||||
|
||||
def context(bot=None):
|
||||
return ExecutionContext(instance_uuid='test', workspace_uuid='workspace', placement_generation=1, bot_uuid=bot)
|
||||
|
||||
|
||||
def resource(bot):
|
||||
return dict(bot_id=bot, bot_name=bot, pipeline_id=bot, pipeline_name=bot)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
async def service():
|
||||
engine = create_async_engine('sqlite+aiosqlite:///:memory:')
|
||||
async with engine.begin() as conn:
|
||||
await conn.run_sync(Base.metadata.create_all)
|
||||
|
||||
class Persistence:
|
||||
serialize_model = PersistenceManager.serialize_model
|
||||
|
||||
def get_db_engine(self):
|
||||
return engine
|
||||
|
||||
async def execute_async(self, stmt):
|
||||
async with engine.begin() as conn:
|
||||
return await conn.execute(stmt)
|
||||
|
||||
ap = SimpleNamespace(persistence_mgr=Persistence(), logger=logging.getLogger(__name__))
|
||||
ap.monitoring_service = MonitoringService(ap)
|
||||
yield ap.monitoring_service
|
||||
await engine.dispose()
|
||||
|
||||
|
||||
async def test_helper_first_message_count_and_two_bot_isolation(service):
|
||||
for bot in ['a', 'b', 'a']:
|
||||
query = SimpleNamespace(
|
||||
_execution_context=context(bot),
|
||||
launcher_type='person',
|
||||
launcher_id=42,
|
||||
sender_id=42,
|
||||
message_chain=SimpleNamespace(model_dump=lambda: []),
|
||||
)
|
||||
assert await MonitoringHelper.record_query_start(service.ap, query, **resource(bot))
|
||||
rows, total = await service.get_sessions(context())
|
||||
assert total == 2
|
||||
assert {r['bot_id']: r['message_count'] for r in rows} == {'a': 2, 'b': 1}
|
||||
assert {r['pipeline_id'] for r in rows} == {'a', 'b'}
|
||||
assert {r['session_id'] for r in rows} == {'person_42'}
|
||||
|
||||
|
||||
async def test_analysis_fails_closed_and_scopes_statistics(service):
|
||||
for bot in ['a', 'b']:
|
||||
await service.record_session_start(context(bot), session_id='person_42', **resource(bot))
|
||||
await service.record_message(context(bot), session_id='person_42', message_content=bot, **resource(bot))
|
||||
assert (await service.get_session_analysis(context(), 'person_42'))['found'] is False
|
||||
result = await service.get_session_analysis(context(), 'person_42', bot_id='b')
|
||||
assert result['message_stats']['total'] == 1
|
||||
assert result['session']['bot_id'] == 'b'
|
||||
|
||||
|
||||
async def test_activity_requires_bot_and_upsert_counts_racing_first_queries(service):
|
||||
for _ in range(2):
|
||||
await service.record_session_start(context('a'), session_id='person_42', **resource('a'))
|
||||
with pytest.raises(ValueError, match='bot'):
|
||||
await service.update_session_activity(context(), 'person_42')
|
||||
assert await service.update_session_activity(context('a'), 'person_42')
|
||||
assert not await service.update_session_activity(context('b'), 'person_42')
|
||||
rows, _ = await service.get_sessions(context())
|
||||
assert rows[0]['message_count'] == 3
|
||||
|
||||
|
||||
async def test_old_active_sessions_are_listed_exported_and_not_cleaned(service):
|
||||
for bot in ['a', 'b']:
|
||||
await service.record_session_start(context(bot), session_id='person_42', **resource(bot))
|
||||
old = dt.datetime(2000, 1, 1)
|
||||
await service.ap.persistence_mgr.execute_async(sa.update(models.MonitoringSession).values(start_time=old))
|
||||
await service.ap.persistence_mgr.execute_async(
|
||||
sa.update(models.MonitoringSession).where(models.MonitoringSession.bot_id == 'a').values(last_activity=old)
|
||||
)
|
||||
since = dt.datetime.now(dt.timezone.utc).replace(tzinfo=None) - dt.timedelta(days=1)
|
||||
rows, total = await service.get_sessions(context(), start_time=since)
|
||||
assert total == 1 and rows[0]['bot_id'] == 'b'
|
||||
assert len(await service.export_sessions(context(), start_time=since)) == 1
|
||||
count = await service._delete_expired_in_batches(
|
||||
context(),
|
||||
models.MonitoringSession,
|
||||
models.MonitoringSession.last_activity,
|
||||
models.MonitoringSession.session_id,
|
||||
since,
|
||||
1,
|
||||
2,
|
||||
)
|
||||
assert count == 1
|
||||
rows, total = await service.get_sessions(context())
|
||||
assert total == 1 and rows[0]['bot_id'] == 'b'
|
||||
@@ -0,0 +1,125 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import datetime
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
import sqlalchemy
|
||||
from sqlalchemy.ext.asyncio import create_async_engine
|
||||
|
||||
from langbot.pkg.api.http.context import ExecutionContext
|
||||
from langbot.pkg.entity.persistence.base import Base
|
||||
from langbot.pkg.entity.persistence.monitoring import MonitoringLLMCall, MonitoringMessage
|
||||
from langbot.pkg.entity.persistence.workspace import Workspace
|
||||
|
||||
pytestmark = pytest.mark.asyncio
|
||||
|
||||
A = '00000000-0000-0000-0000-00000000000a'
|
||||
B = '00000000-0000-0000-0000-00000000000b'
|
||||
START = datetime.datetime(2026, 1, 1)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
async def traffic_app():
|
||||
engine = create_async_engine('sqlite+aiosqlite:///:memory:')
|
||||
async with engine.begin() as connection:
|
||||
await connection.run_sync(Base.metadata.create_all)
|
||||
await connection.execute(
|
||||
sqlalchemy.insert(Workspace),
|
||||
[
|
||||
{'uuid': wid, 'instance_uuid': 'instance', 'name': wid, 'slug': wid, 'source': 'cloud_projection'}
|
||||
for wid in (A, B)
|
||||
],
|
||||
)
|
||||
for wid, bot, count in [(A, 'bot-a', 60), (A, 'bot-b', 7), (B, 'bot-a', 9)]:
|
||||
common = {
|
||||
'workspace_uuid': wid,
|
||||
'timestamp': START,
|
||||
'bot_id': bot,
|
||||
'bot_name': bot,
|
||||
'pipeline_id': 'pipeline',
|
||||
'pipeline_name': 'Pipeline',
|
||||
'session_id': 'person_42',
|
||||
'status': 'success',
|
||||
}
|
||||
await connection.execute(
|
||||
sqlalchemy.insert(MonitoringMessage),
|
||||
[
|
||||
dict(common, id=f'{wid}-{bot}-{i}', message_content='test fixture', level='info', role='user')
|
||||
for i in range(count)
|
||||
],
|
||||
)
|
||||
await connection.execute(
|
||||
sqlalchemy.insert(MonitoringLLMCall),
|
||||
[
|
||||
dict(
|
||||
common,
|
||||
id=f'{wid}-{bot}-{i}',
|
||||
model_name='fixture-model',
|
||||
input_tokens=1,
|
||||
output_tokens=1,
|
||||
total_tokens=2,
|
||||
duration=1,
|
||||
)
|
||||
for i in range(count)
|
||||
],
|
||||
)
|
||||
|
||||
class Persistence:
|
||||
def get_db_engine(self):
|
||||
return engine
|
||||
|
||||
async def execute_async(self, statement):
|
||||
async with engine.connect() as connection:
|
||||
return await connection.execute(statement)
|
||||
|
||||
yield SimpleNamespace(persistence_mgr=Persistence())
|
||||
await engine.dispose()
|
||||
|
||||
|
||||
async def test_traffic_counts_all_rows_not_just_latest_page(traffic_app):
|
||||
from langbot.pkg.api.http.service.monitoring_traffic import get_traffic_series
|
||||
|
||||
context = ExecutionContext(instance_uuid='instance', workspace_uuid=A, placement_generation=1)
|
||||
result = await get_traffic_series(
|
||||
traffic_app, context, bot_ids=['bot-a'], start_time=START, end_time=START + datetime.timedelta(hours=2)
|
||||
)
|
||||
assert result['bucket'] == 'hour'
|
||||
assert result['truncated'] is False
|
||||
assert sum(point['messages'] for point in result['points']) == 60
|
||||
assert sum(point['llm_calls'] for point in result['points']) == 60
|
||||
assert len(result['points']) == 3
|
||||
assert result['points'][1]['messages'] == result['points'][1]['llm_calls'] == 0
|
||||
assert result['points'][0]['timestamp'] == '2026-01-01T00:00:00Z'
|
||||
|
||||
|
||||
async def test_traffic_workspace_pipeline_and_empty_filters(traffic_app):
|
||||
from langbot.pkg.api.http.service.monitoring_traffic import get_traffic_series
|
||||
|
||||
context = ExecutionContext(instance_uuid='instance', workspace_uuid=B, placement_generation=1)
|
||||
kwargs = dict(start_time=START, end_time=START + datetime.timedelta(hours=2))
|
||||
result = await get_traffic_series(traffic_app, context, **kwargs)
|
||||
assert sum(point['messages'] for point in result['points']) == 9
|
||||
empty = await get_traffic_series(traffic_app, context, pipeline_ids=['missing'], **kwargs)
|
||||
assert sum(point['messages'] for point in empty['points']) == 0
|
||||
assert sum(point['llm_calls'] for point in empty['points']) == 0
|
||||
|
||||
|
||||
async def test_traffic_bounds_large_ranges_and_marks_truncation(traffic_app):
|
||||
from langbot.pkg.api.http.service.monitoring_traffic import get_traffic_series
|
||||
|
||||
context = ExecutionContext(instance_uuid='instance', workspace_uuid=A, placement_generation=1)
|
||||
result = await get_traffic_series(
|
||||
traffic_app, context, start_time=START, end_time=START + datetime.timedelta(days=5000)
|
||||
)
|
||||
assert result['bucket'] == 'day'
|
||||
assert result['truncated'] is True
|
||||
assert len(result['points']) == 1000
|
||||
|
||||
|
||||
async def test_traffic_fails_closed_without_workspace(traffic_app):
|
||||
from langbot.pkg.api.http.authz import WorkspaceRequiredError
|
||||
from langbot.pkg.api.http.service.monitoring_traffic import get_traffic_series
|
||||
|
||||
with pytest.raises(WorkspaceRequiredError):
|
||||
await get_traffic_series(traffic_app, None)
|
||||
@@ -0,0 +1,401 @@
|
||||
"""Provider deletion uses real SQLite transactions and real runtime cache cleanup."""
|
||||
|
||||
import asyncio
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import AsyncMock, Mock
|
||||
|
||||
import pytest
|
||||
import pytest_asyncio
|
||||
import quart
|
||||
import sqlalchemy as sa
|
||||
from sqlalchemy.ext.asyncio import create_async_engine
|
||||
|
||||
from langbot.pkg.api.http.context import ExecutionContext
|
||||
from langbot.pkg.api.http.controller.groups.provider.providers import ModelProvidersRouterGroup
|
||||
from langbot.pkg.api.http.service.provider import ModelProviderService
|
||||
from langbot.pkg.entity.persistence.model import CodexCredential, EmbeddingModel, LLMModel, ModelProvider, RerankModel
|
||||
from langbot.pkg.entity.persistence.user import User
|
||||
from langbot.pkg.entity.persistence.workspace import Workspace
|
||||
from langbot.pkg.persistence.mgr import PersistenceManager, PersistenceMode
|
||||
from langbot.pkg.provider.modelmgr.modelmgr import ModelManager
|
||||
from langbot.pkg.workspace.errors import WorkspaceNotFoundError
|
||||
|
||||
pytestmark = pytest.mark.asyncio
|
||||
MODEL_TYPES = (LLMModel, EmbeddingModel, RerankModel)
|
||||
TABLES = (*MODEL_TYPES, CodexCredential, ModelProvider)
|
||||
|
||||
|
||||
@pytest_asyncio.fixture
|
||||
async def deletion(tmp_path):
|
||||
engine = create_async_engine(f'sqlite+aiosqlite:///{tmp_path / "cascade.db"}')
|
||||
|
||||
@sa.event.listens_for(engine.sync_engine, 'connect')
|
||||
def enable_foreign_keys(connection, _record):
|
||||
connection.execute('PRAGMA foreign_keys=ON')
|
||||
|
||||
ap = SimpleNamespace(logger=Mock())
|
||||
pm = ap.persistence_mgr = PersistenceManager(ap)
|
||||
pm.db = SimpleNamespace(get_engine=lambda: engine)
|
||||
manager = ap.model_mgr = ModelManager(ap)
|
||||
contexts = {workspace: ExecutionContext('instance', workspace, 1) for workspace in ('a', 'b')}
|
||||
# Only execution binding discovery is stubbed; cache indexing/removal/close is real.
|
||||
manager.resolve_execution_context = AsyncMock(side_effect=lambda context: contexts[context])
|
||||
service = ModelProviderService(ap)
|
||||
closed = []
|
||||
|
||||
async def snapshot():
|
||||
async with engine.connect() as conn:
|
||||
return {
|
||||
table.__tablename__: [dict(row) for row in (await conn.execute(sa.select(table))).mappings()]
|
||||
for table in TABLES
|
||||
}
|
||||
|
||||
async with engine.begin() as conn:
|
||||
for table in (User, Workspace, ModelProvider, CodexCredential, *MODEL_TYPES):
|
||||
await conn.run_sync(table.__table__.create)
|
||||
for workspace in contexts:
|
||||
await conn.execute(
|
||||
sa.insert(Workspace).values(
|
||||
uuid=workspace,
|
||||
instance_uuid='instance',
|
||||
name=workspace,
|
||||
slug=workspace,
|
||||
source='cloud_projection',
|
||||
)
|
||||
)
|
||||
for provider, workspace in (('target', 'a'), ('neighbor', 'a'), ('foreign', 'b'), ('empty', 'a')):
|
||||
await conn.execute(
|
||||
sa.insert(ModelProvider).values(
|
||||
uuid=provider,
|
||||
workspace_uuid=workspace,
|
||||
name=provider,
|
||||
requester='openai-codex',
|
||||
base_url='https://chatgpt.com/backend-api/codex',
|
||||
api_keys=[],
|
||||
)
|
||||
)
|
||||
await conn.execute(
|
||||
sa.insert(CodexCredential).values(
|
||||
provider_uuid=provider,
|
||||
workspace_uuid=workspace,
|
||||
payload={'synthetic': provider},
|
||||
)
|
||||
)
|
||||
|
||||
async def close(provider=provider):
|
||||
# A separate connection must observe the durable deletion before close runs.
|
||||
state = await snapshot()
|
||||
assert all(row['uuid'] != provider for row in state['model_providers'])
|
||||
assert pm.current_session() is None
|
||||
closed.append(provider)
|
||||
|
||||
runtime = SimpleNamespace(requester=SimpleNamespace(aclose=AsyncMock(side_effect=close)))
|
||||
manager._cache_set(manager.provider_dict, manager._cache_key(contexts[workspace], provider), runtime)
|
||||
if provider == 'empty':
|
||||
continue
|
||||
for model_type, cache in zip(
|
||||
MODEL_TYPES,
|
||||
(
|
||||
manager.llm_model_dict,
|
||||
manager.embedding_model_dict,
|
||||
manager.rerank_model_dict,
|
||||
),
|
||||
):
|
||||
for index in range(2):
|
||||
uuid = f'{provider}-{model_type.__tablename__}-{index}'
|
||||
await conn.execute(
|
||||
sa.insert(model_type).values(
|
||||
uuid=uuid,
|
||||
workspace_uuid=workspace,
|
||||
provider_uuid=provider,
|
||||
name=uuid,
|
||||
)
|
||||
)
|
||||
manager._cache_set(cache, manager._cache_key(contexts[workspace], uuid), object())
|
||||
initial = await snapshot()
|
||||
initial_caches = [
|
||||
dict(cache)
|
||||
for cache in (
|
||||
manager.provider_dict,
|
||||
manager.llm_model_dict,
|
||||
manager.embedding_model_dict,
|
||||
manager.rerank_model_dict,
|
||||
)
|
||||
]
|
||||
try:
|
||||
yield SimpleNamespace(
|
||||
ap=ap,
|
||||
pm=pm,
|
||||
engine=engine,
|
||||
service=service,
|
||||
manager=manager,
|
||||
snapshot=snapshot,
|
||||
initial=initial,
|
||||
initial_caches=initial_caches,
|
||||
closed=closed,
|
||||
)
|
||||
finally:
|
||||
await engine.dispose()
|
||||
|
||||
|
||||
def assert_caches_unchanged(deletion):
|
||||
assert deletion.closed == []
|
||||
assert deletion.initial_caches == [
|
||||
dict(cache)
|
||||
for cache in (
|
||||
deletion.manager.provider_dict,
|
||||
deletion.manager.llm_model_dict,
|
||||
deletion.manager.embedding_model_dict,
|
||||
deletion.manager.rerank_model_dict,
|
||||
)
|
||||
]
|
||||
|
||||
|
||||
@pytest.mark.parametrize('mode', [PersistenceMode.OSS_COMPAT, PersistenceMode.CLOUD_RUNTIME])
|
||||
async def test_cascade_deletes_all_model_types_and_credentials_after_commit(deletion, mode):
|
||||
deletion.pm.mode = mode
|
||||
await deletion.service.delete_provider('a', 'target', cascade=True)
|
||||
state = await deletion.snapshot()
|
||||
for table, rows in deletion.initial.items():
|
||||
identity = 'uuid' if table == 'model_providers' else 'provider_uuid'
|
||||
assert state[table] == [row for row in rows if row[identity] != 'target']
|
||||
assert deletion.closed == ['target']
|
||||
deletion.ap.logger.warning.assert_not_called()
|
||||
for cache in (
|
||||
deletion.manager.provider_dict,
|
||||
deletion.manager.llm_model_dict,
|
||||
deletion.manager.embedding_model_dict,
|
||||
deletion.manager.rerank_model_dict,
|
||||
):
|
||||
assert all(not key[-1].startswith('target') for key in cache)
|
||||
assert any(key[1] == 'b' for key in cache)
|
||||
assert any(key[-1].startswith('neighbor') for key in cache)
|
||||
|
||||
|
||||
@pytest.mark.parametrize('model_type', MODEL_TYPES)
|
||||
@pytest.mark.parametrize('kwargs', [{}, {'cascade': False}])
|
||||
async def test_default_guard_preserves_each_model_type(deletion, model_type, kwargs):
|
||||
async with deletion.engine.begin() as conn:
|
||||
for other in MODEL_TYPES:
|
||||
if other is not model_type:
|
||||
await conn.execute(sa.delete(other).where(other.provider_uuid == 'target'))
|
||||
before = await deletion.snapshot()
|
||||
with pytest.raises(ValueError, match='models still reference it'):
|
||||
await deletion.service.delete_provider('a', 'target', **kwargs)
|
||||
assert await deletion.snapshot() == before
|
||||
assert_caches_unchanged(deletion)
|
||||
|
||||
|
||||
@pytest.mark.parametrize('kwargs', [{}, {'cascade': False}, {'cascade': True}])
|
||||
async def test_empty_provider_deletes_credentials_with_or_without_cascade(deletion, kwargs):
|
||||
await deletion.service.delete_provider('a', 'empty', **kwargs)
|
||||
state = await deletion.snapshot()
|
||||
assert all(row['uuid'] != 'empty' for row in state['model_providers'])
|
||||
assert all(row['provider_uuid'] != 'empty' for row in state['codex_credentials'])
|
||||
assert deletion.closed == ['empty']
|
||||
|
||||
|
||||
@pytest.mark.parametrize('provider', ['foreign', 'missing'])
|
||||
@pytest.mark.parametrize('cascade', [False, True])
|
||||
async def test_foreign_and_missing_provider_are_non_enumerating(deletion, provider, cascade):
|
||||
with pytest.raises(WorkspaceNotFoundError, match='Provider not found'):
|
||||
await deletion.service.delete_provider('a', provider, cascade=cascade)
|
||||
assert await deletion.snapshot() == deletion.initial
|
||||
assert_caches_unchanged(deletion)
|
||||
|
||||
|
||||
@pytest.mark.parametrize('cascade', [False, True])
|
||||
async def test_cloud_managed_provider_cannot_be_deleted(deletion, cascade):
|
||||
async with deletion.engine.begin() as conn:
|
||||
await conn.execute(
|
||||
sa.update(ModelProvider)
|
||||
.where(ModelProvider.uuid == 'target')
|
||||
.values(
|
||||
requester='space-chat-completions',
|
||||
)
|
||||
)
|
||||
before = await deletion.snapshot()
|
||||
deletion.pm.mode = PersistenceMode.CLOUD_RUNTIME
|
||||
with pytest.raises(ValueError, match='managed by Cloud'):
|
||||
await deletion.service.delete_provider('a', 'target', cascade=cascade)
|
||||
assert await deletion.snapshot() == before
|
||||
assert_caches_unchanged(deletion)
|
||||
|
||||
|
||||
@pytest.mark.parametrize('failure_table', ['embedding_models', 'codex_credentials', 'model_providers'])
|
||||
async def test_database_failure_rolls_back_all_rows_without_runtime_cleanup(deletion, failure_table):
|
||||
async with deletion.engine.begin() as conn:
|
||||
await conn.exec_driver_sql(
|
||||
f'CREATE TRIGGER fail_delete BEFORE DELETE ON {failure_table} '
|
||||
"BEGIN SELECT RAISE(ABORT, 'injected delete failure'); END"
|
||||
)
|
||||
with pytest.raises(sa.exc.IntegrityError, match='injected delete failure'):
|
||||
await deletion.service.delete_provider('a', 'target', cascade=True)
|
||||
assert await deletion.snapshot() == deletion.initial
|
||||
assert_caches_unchanged(deletion)
|
||||
|
||||
|
||||
@pytest.mark.parametrize('rollback', [False, True])
|
||||
async def test_nested_transaction_defers_cleanup_until_outer_commit(deletion, rollback):
|
||||
class Abort(Exception):
|
||||
pass
|
||||
|
||||
try:
|
||||
async with deletion.pm.tenant_uow('a'):
|
||||
await deletion.service.delete_provider('a', 'target', cascade=True)
|
||||
assert_caches_unchanged(deletion)
|
||||
if rollback:
|
||||
raise Abort
|
||||
except Abort:
|
||||
pass
|
||||
tasks = tuple(deletion.service._deletion_tasks)
|
||||
await asyncio.gather(*tasks, return_exceptions=True)
|
||||
if rollback:
|
||||
assert await deletion.snapshot() == deletion.initial
|
||||
assert_caches_unchanged(deletion)
|
||||
else:
|
||||
assert deletion.closed == ['target']
|
||||
deletion.ap.logger.warning.assert_not_called()
|
||||
|
||||
|
||||
async def test_cascade_ignores_foreign_workspace_references_even_without_foreign_keys(deletion):
|
||||
async with deletion.engine.connect() as conn:
|
||||
await conn.exec_driver_sql('PRAGMA foreign_keys=OFF')
|
||||
for model_type in MODEL_TYPES:
|
||||
await conn.execute(
|
||||
sa.update(model_type)
|
||||
.where(model_type.workspace_uuid == 'b')
|
||||
.values(
|
||||
provider_uuid='target',
|
||||
)
|
||||
)
|
||||
await conn.commit()
|
||||
await deletion.service.delete_provider('a', 'target', cascade=True)
|
||||
state = await deletion.snapshot()
|
||||
for model_type in MODEL_TYPES:
|
||||
assert len([row for row in state[model_type.__tablename__] if row['workspace_uuid'] == 'b']) == 2
|
||||
assert all(row['provider_uuid'] != 'target' for row in state['codex_credentials'])
|
||||
assert deletion.closed == ['target']
|
||||
|
||||
|
||||
@pytest_asyncio.fixture
|
||||
async def route_app(deletion):
|
||||
ap = deletion.ap
|
||||
ap.user_service = SimpleNamespace(
|
||||
get_authenticated_account=AsyncMock(
|
||||
return_value=SimpleNamespace(uuid='account', user='owner@example.invalid'),
|
||||
)
|
||||
)
|
||||
membership = SimpleNamespace(uuid='membership', role='owner', projection_revision=0)
|
||||
ap.workspace_collaboration_service = SimpleNamespace(
|
||||
resolve_account_workspace=AsyncMock(
|
||||
return_value=SimpleNamespace(
|
||||
workspace=SimpleNamespace(uuid='a'),
|
||||
membership=membership,
|
||||
execution=SimpleNamespace(instance_uuid='instance', placement_generation=1),
|
||||
),
|
||||
)
|
||||
)
|
||||
ap.provider_service = SimpleNamespace(delete_provider=AsyncMock())
|
||||
app = quart.Quart(__name__)
|
||||
await ModelProvidersRouterGroup(ap, app).initialize()
|
||||
return app.test_client(), ap.provider_service.delete_provider, membership
|
||||
|
||||
|
||||
@pytest.mark.parametrize('query, expected', [('', None), ('?cascade=true', True), ('?cascade=false', False)])
|
||||
async def test_route_passes_explicit_cascade_and_trusted_workspace(route_app, query, expected):
|
||||
client, delete, _ = route_app
|
||||
response = await client.delete(
|
||||
'/api/v1/provider/providers/target' + query,
|
||||
headers={
|
||||
'Authorization': 'Bearer token',
|
||||
'X-Workspace-Id': 'a',
|
||||
},
|
||||
)
|
||||
assert response.status_code == 200
|
||||
assert delete.await_count == 1
|
||||
assert delete.await_args.args[0].workspace_uuid == 'a'
|
||||
assert delete.await_args.args[1] == 'target'
|
||||
assert delete.await_args.kwargs == ({} if expected is None else {'cascade': expected})
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
'query',
|
||||
[
|
||||
'?cascade=',
|
||||
'?cascade',
|
||||
'?cascade=TRUE',
|
||||
'?cascade=1',
|
||||
'?cascade=yes',
|
||||
'?cascade=null',
|
||||
'?cascade=%20true',
|
||||
'?cascade=true&cascade=false',
|
||||
'?cascade=true&cascade=true',
|
||||
],
|
||||
)
|
||||
async def test_route_rejects_invalid_or_duplicate_cascade_before_deletion(route_app, query):
|
||||
client, delete, _ = route_app
|
||||
response = await client.delete(
|
||||
'/api/v1/provider/providers/target' + query,
|
||||
headers={
|
||||
'Authorization': 'Bearer token',
|
||||
'X-Workspace-Id': 'a',
|
||||
},
|
||||
)
|
||||
assert response.status_code == 400
|
||||
delete.assert_not_awaited()
|
||||
|
||||
|
||||
@pytest.mark.parametrize('role', ['viewer', 'operator'])
|
||||
async def test_cascade_requires_workspace_resource_manage_permission(route_app, role):
|
||||
client, delete, membership = route_app
|
||||
membership.role = role
|
||||
response = await client.delete(
|
||||
'/api/v1/provider/providers/target?cascade=true',
|
||||
headers={
|
||||
'Authorization': 'Bearer token',
|
||||
'X-Workspace-Id': 'a',
|
||||
},
|
||||
)
|
||||
assert response.status_code == 403
|
||||
delete.assert_not_awaited()
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
'provider, query, status',
|
||||
[
|
||||
('target', '', 400),
|
||||
('target', '?cascade=false', 400),
|
||||
('target', '?cascade=true', 200),
|
||||
('foreign', '?cascade=true', 404),
|
||||
('missing', '?cascade=true', 404),
|
||||
],
|
||||
)
|
||||
async def test_route_to_real_sqlite_service(deletion, route_app, provider, query, status):
|
||||
client, _, _ = route_app
|
||||
deletion.ap.provider_service = deletion.service
|
||||
# The route forwards RequestContext, unlike the string-context service tests.
|
||||
deletion.manager.resolve_execution_context = AsyncMock(
|
||||
side_effect=lambda context: ExecutionContext(
|
||||
context.instance_uuid,
|
||||
context.workspace_uuid,
|
||||
context.placement_generation,
|
||||
)
|
||||
)
|
||||
response = await client.delete(
|
||||
'/api/v1/provider/providers/' + provider + query,
|
||||
headers={
|
||||
'Authorization': 'Bearer token',
|
||||
'X-Workspace-Id': 'a',
|
||||
},
|
||||
)
|
||||
assert response.status_code == status
|
||||
if status == 200:
|
||||
assert deletion.closed == ['target']
|
||||
for model_type in MODEL_TYPES:
|
||||
assert all(
|
||||
row['provider_uuid'] != 'target' for row in (await deletion.snapshot())[model_type.__tablename__]
|
||||
)
|
||||
else:
|
||||
assert await deletion.snapshot() == deletion.initial
|
||||
assert_caches_unchanged(deletion)
|
||||
@@ -14,11 +14,12 @@ Source: src/langbot/pkg/api/http/service/provider.py
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
from contextlib import nullcontext
|
||||
from unittest.mock import AsyncMock, Mock
|
||||
from types import SimpleNamespace
|
||||
|
||||
from langbot.pkg.api.http.service.provider import ModelProviderService
|
||||
from langbot.pkg.entity.persistence.model import ModelProvider, LLMModel, EmbeddingModel, RerankModel
|
||||
from langbot.pkg.entity.persistence.model import ModelProvider, LLMModel
|
||||
from langbot.pkg.workspace.errors import WorkspaceNotFoundError
|
||||
|
||||
|
||||
@@ -383,112 +384,35 @@ class TestModelProviderServiceUpdateProvider:
|
||||
|
||||
|
||||
class TestModelProviderServiceDeleteProvider:
|
||||
"""Tests for delete_provider method."""
|
||||
|
||||
async def test_delete_provider_with_llm_models_raises_error(self):
|
||||
"""Raises ValueError when LLM models reference provider."""
|
||||
# Setup
|
||||
ap = SimpleNamespace()
|
||||
ap.persistence_mgr = SimpleNamespace()
|
||||
|
||||
# Mock LLM model exists - only return LLM result since that's first check
|
||||
llm_result = _create_mock_result([], first_item=_create_mock_llm_model())
|
||||
|
||||
ap.persistence_mgr.execute_async = AsyncMock(return_value=llm_result)
|
||||
"""Fast guard coverage; real transaction/cache behavior is in test_provider_cascade."""
|
||||
|
||||
@pytest.mark.parametrize('label', ['LLM', 'Embedding', 'Rerank', None])
|
||||
async def test_delete_provider_requires_no_references(self, label):
|
||||
provider_result = Mock()
|
||||
provider_result.first.return_value = SimpleNamespace(requester='openai')
|
||||
results = [provider_result]
|
||||
for model_label in ('LLM', 'Embedding', 'Rerank'):
|
||||
result = Mock()
|
||||
result.scalars.return_value = ['model'] if label == model_label else []
|
||||
results.append(result)
|
||||
results.extend([Mock(rowcount=1), Mock(rowcount=1)])
|
||||
ap = SimpleNamespace(
|
||||
persistence_mgr=SimpleNamespace(
|
||||
execute_async=AsyncMock(side_effect=results),
|
||||
tenant_uow=lambda _: nullcontext(),
|
||||
tenant_scope=lambda _: nullcontext(),
|
||||
current_session=lambda: None,
|
||||
),
|
||||
model_mgr=SimpleNamespace(remove_provider=AsyncMock()),
|
||||
)
|
||||
service = ModelProviderService(ap)
|
||||
|
||||
# Execute & Verify
|
||||
with pytest.raises(ValueError, match='Cannot delete provider: LLM models'):
|
||||
await service.delete_provider(WORKSPACE_UUID, 'provider-with-llm')
|
||||
|
||||
async def test_delete_provider_with_embedding_models_raises_error(self):
|
||||
"""Raises ValueError when Embedding models reference provider."""
|
||||
# Setup
|
||||
ap = SimpleNamespace()
|
||||
ap.persistence_mgr = SimpleNamespace()
|
||||
|
||||
# Create results for each check type
|
||||
llm_result = Mock()
|
||||
llm_result.first = Mock(return_value=None) # No LLM models
|
||||
embedding_result = Mock()
|
||||
embedding_result.first = Mock(return_value=Mock(spec=EmbeddingModel)) # Has embedding model
|
||||
rerank_result = Mock()
|
||||
rerank_result.first = Mock(return_value=None)
|
||||
|
||||
call_count = 0
|
||||
|
||||
async def mock_execute(query):
|
||||
nonlocal call_count
|
||||
call_count += 1
|
||||
if call_count == 1:
|
||||
return llm_result
|
||||
elif call_count == 2:
|
||||
return embedding_result
|
||||
return rerank_result
|
||||
|
||||
ap.persistence_mgr.execute_async = AsyncMock(side_effect=mock_execute)
|
||||
|
||||
service = ModelProviderService(ap)
|
||||
|
||||
# Execute & Verify - should raise embedding error (LLM check passes, embedding check fails)
|
||||
with pytest.raises(ValueError, match='Cannot delete provider: Embedding models'):
|
||||
await service.delete_provider(WORKSPACE_UUID, 'provider-with-embedding')
|
||||
|
||||
async def test_delete_provider_with_rerank_models_raises_error(self):
|
||||
"""Raises ValueError when Rerank models reference provider."""
|
||||
# Setup
|
||||
ap = SimpleNamespace()
|
||||
ap.persistence_mgr = SimpleNamespace()
|
||||
|
||||
# Create results for each check type
|
||||
llm_result = Mock()
|
||||
llm_result.first = Mock(return_value=None) # No LLM models
|
||||
embedding_result = Mock()
|
||||
embedding_result.first = Mock(return_value=None) # No embedding models
|
||||
rerank_result = Mock()
|
||||
rerank_result.first = Mock(return_value=Mock(spec=RerankModel)) # Has rerank model
|
||||
|
||||
call_count = 0
|
||||
|
||||
async def mock_execute(query):
|
||||
nonlocal call_count
|
||||
call_count += 1
|
||||
if call_count == 1:
|
||||
return llm_result
|
||||
elif call_count == 2:
|
||||
return embedding_result
|
||||
return rerank_result
|
||||
|
||||
ap.persistence_mgr.execute_async = AsyncMock(side_effect=mock_execute)
|
||||
|
||||
service = ModelProviderService(ap)
|
||||
|
||||
# Execute & Verify - should raise rerank error (LLM and embedding checks pass, rerank check fails)
|
||||
with pytest.raises(ValueError, match='Cannot delete provider: Rerank models'):
|
||||
await service.delete_provider(WORKSPACE_UUID, 'provider-with-rerank')
|
||||
|
||||
async def test_delete_provider_no_models_success(self):
|
||||
"""Deletes provider when no models reference it."""
|
||||
# Setup
|
||||
ap = SimpleNamespace()
|
||||
ap.persistence_mgr = SimpleNamespace()
|
||||
ap.model_mgr = SimpleNamespace()
|
||||
ap.model_mgr.remove_provider = AsyncMock()
|
||||
|
||||
# Mock no models reference provider
|
||||
empty_result = Mock()
|
||||
empty_result.first = Mock(return_value=None)
|
||||
|
||||
ap.persistence_mgr.execute_async = AsyncMock(return_value=empty_result)
|
||||
|
||||
service = ModelProviderService(ap)
|
||||
|
||||
# Execute
|
||||
await service.delete_provider(WORKSPACE_UUID, 'provider-no-models')
|
||||
|
||||
# Verify - delete and remove called
|
||||
ap.model_mgr.remove_provider.assert_called_once_with(WORKSPACE_UUID, 'provider-no-models')
|
||||
if label is not None:
|
||||
with pytest.raises(ValueError, match=f'Cannot delete provider: {label} models'):
|
||||
await service.delete_provider(WORKSPACE_UUID, 'provider')
|
||||
ap.model_mgr.remove_provider.assert_not_awaited()
|
||||
else:
|
||||
await service.delete_provider(WORKSPACE_UUID, 'provider')
|
||||
ap.model_mgr.remove_provider.assert_awaited_once_with(WORKSPACE_UUID, 'provider')
|
||||
|
||||
|
||||
class TestModelProviderServiceGetProviderModelCounts:
|
||||
@@ -1045,15 +969,18 @@ class TestCloudManagedProviderProtection:
|
||||
|
||||
async def test_cloud_rejects_update_and_delete_of_managed_provider(self):
|
||||
service = self._service()
|
||||
service.get_provider = AsyncMock(
|
||||
return_value={'uuid': 'system-provider', 'requester': SYSTEM_REQUESTER}
|
||||
)
|
||||
service.get_provider = AsyncMock(return_value={'uuid': 'system-provider', 'requester': SYSTEM_REQUESTER})
|
||||
|
||||
with pytest.raises(ValueError, match='managed by Cloud'):
|
||||
await service.update_provider(WORKSPACE_UUID, 'system-provider', {'name': 'Renamed'})
|
||||
service.ap.persistence_mgr.execute_async.assert_not_awaited()
|
||||
service.ap.persistence_mgr.tenant_uow = lambda _: nullcontext()
|
||||
result = Mock()
|
||||
result.first.return_value = SimpleNamespace(requester=SYSTEM_REQUESTER)
|
||||
service.ap.persistence_mgr.execute_async.return_value = result
|
||||
with pytest.raises(ValueError, match='managed by Cloud'):
|
||||
await service.delete_provider(WORKSPACE_UUID, 'system-provider')
|
||||
service.ap.persistence_mgr.execute_async.assert_not_awaited()
|
||||
assert service.ap.persistence_mgr.execute_async.await_count == 1
|
||||
|
||||
async def test_oss_does_not_reserve_space_requester(self):
|
||||
ap = SimpleNamespace(persistence_mgr=SimpleNamespace(mode=SimpleNamespace(value='oss_compat')))
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user