mirror of
https://github.com/langbot-app/LangBot.git
synced 2026-09-13 05:17:14 +00:00
Compare commits
3 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 9b130680ca | |||
| d26d0635c5 | |||
| 58cde8c022 |
@@ -0,0 +1,111 @@
|
||||
# Discord release announcements
|
||||
|
||||
This independent workflow announces new stable LangBot releases in the channel
|
||||
selected by a dedicated Discord incoming webhook. It does not change the existing
|
||||
release/build workflows, edit releases, run a persistent service, poll, or backfill.
|
||||
Announcements run on publication, independently of artifact builds finishing.
|
||||
|
||||
## Setup and read-only validation
|
||||
|
||||
1. In the intended community **announcement channel**, create a dedicated incoming
|
||||
webhook (Channel Settings → Integrations → Webhooks). Copy its URL; do not reuse
|
||||
a webhook belonging to another automation.
|
||||
2. In `langbot-app/LangBot` → Settings → Secrets and variables → Actions, create the
|
||||
**repository secret** `DISCORD_RELEASE_WEBHOOK_URL`. Its value must be exactly
|
||||
`https://discord.com/api/webhooks/<id>/<token>` — no query, trailing slash,
|
||||
API-version segment, or alternate domain. Treat the entire URL as a password.
|
||||
3. Once this workflow is on `master`, open Actions → **Discord Release Announcement**
|
||||
→ Run workflow, choosing `master`. Alternatively:
|
||||
|
||||
```sh
|
||||
gh workflow run discord-release.yml --repo langbot-app/LangBot --ref master
|
||||
```
|
||||
|
||||
4. Inspect **Validate webhook (GET only, no message)**. It checks webhook type `1`
|
||||
and reports `guild_id` and `channel_id`; compare both with the intended server
|
||||
and channel using Discord Developer Mode → Copy ID. The secret determines the
|
||||
destination; no channel ID is guessed or overridden. The URL/token is never
|
||||
logged. Dispatch cannot send a test message or announce an old release, even
|
||||
when run again. Missing/invalid secrets fail validation clearly; offline tests
|
||||
do not need secrets.
|
||||
|
||||
GET validation confirms the webhook's identity, not delivery or notification
|
||||
permissions. Verify those on the first genuine release. `mention_everyone=true`
|
||||
confirms Discord parsed the mention; it cannot prove every member received a push
|
||||
notification (member/server notification settings still apply).
|
||||
|
||||
## Activation and message
|
||||
|
||||
The workflow and `.github/discord-release/` helper **must be in the commit targeted
|
||||
by each new release tag**. Merging to `master` does not enable announcements for
|
||||
old tags whose commits lack these files. Manual dispatch becomes available when
|
||||
the workflow is on the default branch. Only publish release tags from trusted,
|
||||
reviewed commits: release workflows execute that tag's code with the secret.
|
||||
|
||||
Only `release` events with action `published`, `draft=false`, and
|
||||
`prerelease=false` can send. Drafts and prereleases are skipped; release edits do
|
||||
not trigger announcements. The helper requires the repository to be exactly
|
||||
`langbot-app/LangBot`, a stable `vX.Y.Z` tag (ASCII digits, at most 64 characters),
|
||||
and its exact canonical GitHub release URL. Other naming schemes fail closed.
|
||||
|
||||
Example message (the version and URL come from the validated event file):
|
||||
|
||||
```text
|
||||
@everyone LangBot v4.10.11 is now available!
|
||||
Release notes: https://github.com/langbot-app/LangBot/releases/tag/v4.10.11
|
||||
```
|
||||
|
||||
The release title/body is never copied. There is one literal `@everyone`, explicit
|
||||
`allowed_mentions.parse=["everyone"]`, empty user/role allowlists, and no reply
|
||||
mention. TTS and notification-suppressing flags are disabled. Requests use HTTPS
|
||||
only to `discord.com`, an explicit User-Agent, and no redirects or automatic
|
||||
retries. After a webhook identity GET, one `POST ?wait=true` obtains a message ID;
|
||||
an exact `/messages/<id>` GET verifies its ID, webhook/channel, content,
|
||||
`mention_everyone=true`, and empty user/role mention arrays before success.
|
||||
|
||||
## Repeat guard and manual recovery
|
||||
|
||||
Production sending requires **`GITHUB_RUN_ATTEMPT == "1"`**. Any Actions rerun
|
||||
(including “Re-run failed jobs”) refuses to POST and requires manual reconciliation,
|
||||
even if the first attempt failed before sending. Read-only dispatch may be rerun.
|
||||
|
||||
This is a practical repeat guard, **not durable exactly-once delivery**. It cannot
|
||||
prevent duplicates from a separate new run/event (for example deleting/recreating
|
||||
a release), separate automation, or manual posting. It stores no durable dedupe
|
||||
state and never modifies the release to mark delivery.
|
||||
|
||||
If a POST times out, returns an error, or readback fails, the message may already
|
||||
exist. The workflow fails rather than blindly sending again. A returned message ID
|
||||
is included in the safe error when available. A runner termination can also leave
|
||||
an ambiguous send without that log line.
|
||||
|
||||
1. Inspect the announcement channel and the failed run logs. Locate the canonical
|
||||
release link and, if available, the returned message ID. A failed verification
|
||||
does **not** mean the message was absent.
|
||||
2. If present, reconcile the existing message/mention problem manually; do not
|
||||
rerun, create another release event, or send a duplicate ping.
|
||||
3. If an operator has positively confirmed no message exists, fix the secret or
|
||||
permission issue and use read-only dispatch to validate configuration. A
|
||||
maintainer may then post the announcement manually once in Discord and record
|
||||
the message link in the incident/run notes. Do not override the attempt guard
|
||||
or delete/recreate a release to force recovery.
|
||||
4. If absence cannot be established, pause and reconcile rather than resending.
|
||||
|
||||
To stop future sends, disable **Discord Release Announcement** in Actions. Rotate
|
||||
or delete the dedicated Discord webhook if the URL is exposed, and update the
|
||||
secret before validation. No rollback of release artifacts is involved.
|
||||
|
||||
## Local checks
|
||||
|
||||
Requires Python 3.11+ and the standard library only:
|
||||
|
||||
```sh
|
||||
python3 -m unittest discover -s .github/discord-release -p 'test_*.py' -v
|
||||
python3 -m py_compile .github/discord-release/announce.py .github/discord-release/test_announce.py
|
||||
```
|
||||
|
||||
Tests exercise policy, CLI/event-file handling, mention payloads, hostile inputs,
|
||||
HTTP failures, exact message readback, and refusal to retry. Only the HTTPS
|
||||
transport is mocked for Discord tests; no live Discord requests or messages are
|
||||
made. Changes to this directory or its workflow run the offline tests on push and
|
||||
pull request; tests also gate release sending and read-only dispatch validation.
|
||||
@@ -0,0 +1,162 @@
|
||||
"""Announce only first-attempt stable releases; dispatch is read-only validation."""
|
||||
|
||||
import http.client
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import sys
|
||||
|
||||
REPOSITORY = 'langbot-app/LangBot'
|
||||
RELEASE_PREFIX = f'https://github.com/{REPOSITORY}/releases/tag/'
|
||||
RECONCILE = (
|
||||
'Do not resend or bypass the run-attempt guard; manual reconciliation is required. '
|
||||
'Inspect the announcement channel and workflow logs before any manual recovery '
|
||||
'(see .github/discord-release/README.md).'
|
||||
)
|
||||
|
||||
|
||||
class AnnouncementError(Exception):
|
||||
"""A safe, operator-facing error containing no webhook URL or response body."""
|
||||
|
||||
|
||||
def release_payload(event, attempt):
|
||||
"""Return a bounded, mention-safe payload, or None for draft/preview releases."""
|
||||
if not isinstance(event, dict) or event.get('action') != 'published':
|
||||
raise AnnouncementError('Only release.published events are accepted.')
|
||||
repository = event.get('repository')
|
||||
if not isinstance(repository, dict) or repository.get('full_name') != REPOSITORY:
|
||||
raise AnnouncementError('Unexpected release repository.')
|
||||
release = event.get('release')
|
||||
if not isinstance(release, dict) or any(type(release.get(key)) is not bool for key in ('draft', 'prerelease')):
|
||||
raise AnnouncementError('Invalid release flags.')
|
||||
if release['draft'] or release['prerelease']:
|
||||
return None
|
||||
if attempt != '1':
|
||||
raise AnnouncementError(f'Release reruns or missing run attempts are refused. {RECONCILE}')
|
||||
tag = release.get('tag_name')
|
||||
if not isinstance(tag, str) or len(tag) > 64 or not re.fullmatch(r'v[0-9]+\.[0-9]+\.[0-9]+', tag):
|
||||
raise AnnouncementError('Expected a stable release tag in vX.Y.Z format (at most 64 characters).')
|
||||
url = RELEASE_PREFIX + tag
|
||||
if release.get('html_url') != url:
|
||||
raise AnnouncementError('Release URL must be the canonical LangBot release URL matching its tag.')
|
||||
return {
|
||||
'content': f'@everyone LangBot {tag} is now available!\nRelease notes: {url}',
|
||||
'allowed_mentions': {'parse': ['everyone'], 'users': [], 'roles': [], 'replied_user': False},
|
||||
'tts': False,
|
||||
'flags': 0,
|
||||
}
|
||||
|
||||
|
||||
def is_snowflake(value):
|
||||
return isinstance(value, str) and re.fullmatch(r'[0-9]{1,20}', value) is not None
|
||||
|
||||
|
||||
class DiscordWebhook:
|
||||
def __init__(self, url):
|
||||
if not url:
|
||||
raise AnnouncementError('DISCORD_RELEASE_WEBHOOK_URL is missing. Set the repository Actions secret.')
|
||||
match = re.fullmatch(r'https://discord\.com(/api/webhooks/([0-9]{1,20})/[A-Za-z0-9_-]+)', url)
|
||||
if not match:
|
||||
raise AnnouncementError('Invalid webhook URL; expected https://discord.com/api/webhooks/<id>/<token>.')
|
||||
self.path, self.id = match.groups()
|
||||
|
||||
def _request(self, method, suffix='', payload=None):
|
||||
# Direct HTTPS, default certificate verification, no proxies or redirect/retry machinery.
|
||||
connection = http.client.HTTPSConnection('discord.com', timeout=20)
|
||||
try:
|
||||
body = json.dumps(payload).encode('utf-8') if payload is not None else None
|
||||
connection.request(
|
||||
method,
|
||||
self.path + suffix,
|
||||
body=body,
|
||||
headers={'Content-Type': 'application/json', 'User-Agent': 'LangBot-Release-Announcements/1.0'},
|
||||
)
|
||||
response = connection.getresponse()
|
||||
if response.status != 200:
|
||||
raise AnnouncementError(f'Discord {method} returned HTTP {response.status}; no retry was attempted.')
|
||||
raw = response.read(1_048_577)
|
||||
if len(raw) > 1_048_576:
|
||||
raise AnnouncementError('Discord response exceeded the size limit.')
|
||||
return json.loads(raw)
|
||||
except (OSError, http.client.HTTPException, ValueError, UnicodeError):
|
||||
# Exceptions and bodies can contain the token; never print them or chain them.
|
||||
raise AnnouncementError(
|
||||
f'Discord {method} failed or returned invalid JSON; no retry was attempted.'
|
||||
) from None
|
||||
finally:
|
||||
connection.close()
|
||||
|
||||
def validate(self):
|
||||
"""GET only: verify an incoming webhook and return safe identifying fields."""
|
||||
webhook = self._request('GET')
|
||||
if (
|
||||
not isinstance(webhook, dict)
|
||||
or type(webhook.get('type')) is not int
|
||||
or webhook['type'] != 1
|
||||
or webhook.get('id') != self.id
|
||||
or not is_snowflake(webhook.get('guild_id'))
|
||||
or not is_snowflake(webhook.get('channel_id'))
|
||||
):
|
||||
raise AnnouncementError('Expected an incoming (type 1) webhook with matching ID and guild/channel IDs.')
|
||||
return {key: webhook[key] for key in ('id', 'type', 'guild_id', 'channel_id')}
|
||||
|
||||
def send(self, payload):
|
||||
"""One POST, followed by exact message GET; never automatically retry a send."""
|
||||
webhook = self.validate()
|
||||
message_id = None
|
||||
try:
|
||||
sent = self._request('POST', '?wait=true', payload)
|
||||
if not isinstance(sent, dict) or not is_snowflake(sent.get('id')):
|
||||
raise AnnouncementError('Discord did not return a valid message ID.')
|
||||
message_id = sent['id']
|
||||
saved = self._request('GET', f'/messages/{message_id}')
|
||||
if (
|
||||
not isinstance(saved, dict)
|
||||
or saved.get('id') != message_id
|
||||
or saved.get('webhook_id') != self.id
|
||||
or saved.get('channel_id') != webhook['channel_id']
|
||||
or saved.get('content') != payload['content']
|
||||
or saved.get('mention_everyone') is not True
|
||||
or saved.get('mentions') != []
|
||||
or saved.get('mention_roles') != []
|
||||
):
|
||||
raise AnnouncementError('Discord message readback did not match content, identity, or mentions.')
|
||||
except AnnouncementError as error:
|
||||
reference = f' Returned message ID: {message_id}.' if message_id else ''
|
||||
raise AnnouncementError(f'Delivery not confirmed. {error}{reference} {RECONCILE}') from None
|
||||
return message_id
|
||||
|
||||
|
||||
def main(env=None):
|
||||
env = os.environ if env is None else env
|
||||
try:
|
||||
if env.get('GITHUB_REPOSITORY') != REPOSITORY:
|
||||
raise AnnouncementError('This workflow is restricted to langbot-app/LangBot.')
|
||||
name = env.get('GITHUB_EVENT_NAME')
|
||||
if name == 'workflow_dispatch':
|
||||
webhook = DiscordWebhook(env.get('DISCORD_RELEASE_WEBHOOK_URL')).validate()
|
||||
print(
|
||||
f'Validated incoming webhook: guild_id={webhook["guild_id"]} channel_id={webhook["channel_id"]}. No message sent.'
|
||||
)
|
||||
return 0
|
||||
if name != 'release':
|
||||
raise AnnouncementError('Only release and workflow_dispatch events are accepted by this helper.')
|
||||
try:
|
||||
event = json.loads(Path(env.get('GITHUB_EVENT_PATH', '')).read_text(encoding='utf-8'))
|
||||
except (OSError, ValueError, UnicodeError):
|
||||
raise AnnouncementError('Cannot read a valid JSON release event from GITHUB_EVENT_PATH.') from None
|
||||
payload = release_payload(event, env.get('GITHUB_RUN_ATTEMPT'))
|
||||
if payload is None:
|
||||
print('Skipped draft or prerelease; no message sent.')
|
||||
return 0
|
||||
message_id = DiscordWebhook(env.get('DISCORD_RELEASE_WEBHOOK_URL')).send(payload)
|
||||
print(f'Announcement verified by exact message readback: message_id={message_id}.')
|
||||
return 0
|
||||
except AnnouncementError as error:
|
||||
print(f'Error: {error}', file=sys.stderr)
|
||||
return 1
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,427 @@
|
||||
"""Offline contract tests; no Discord credentials or network required."""
|
||||
|
||||
import contextlib
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
try:
|
||||
import announce
|
||||
except ModuleNotFoundError:
|
||||
announce = None
|
||||
|
||||
WEBHOOK = 'https://discord.com/api/webhooks/123456789012345678/fixture_token-ONLY'
|
||||
WEBHOOK_ID = '123456789012345678'
|
||||
GUILD_ID = '234567890123456789'
|
||||
CHANNEL_ID = '345678901234567890'
|
||||
MESSAGE_ID = '456789012345678901'
|
||||
REPO = 'langbot-app/LangBot'
|
||||
URL = f'https://github.com/{REPO}/releases/tag/v4.10.11'
|
||||
CONTENT = f'@everyone LangBot v4.10.11 is now available!\nRelease notes: {URL}'
|
||||
|
||||
|
||||
def event():
|
||||
return {
|
||||
'action': 'published',
|
||||
'repository': {'full_name': REPO},
|
||||
'release': {
|
||||
'draft': False,
|
||||
'prerelease': False,
|
||||
'tag_name': 'v4.10.11',
|
||||
'html_url': URL,
|
||||
'name': 'Hostile @everyone <@123> $(touch /tmp/unsafe)',
|
||||
'body': '@everyone @here <@123> <@&456> `hostile`',
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def metadata():
|
||||
return {'id': WEBHOOK_ID, 'type': 1, 'guild_id': GUILD_ID, 'channel_id': CHANNEL_ID}
|
||||
|
||||
|
||||
def message():
|
||||
return {
|
||||
'id': MESSAGE_ID,
|
||||
'webhook_id': WEBHOOK_ID,
|
||||
'channel_id': CHANNEL_ID,
|
||||
'content': CONTENT,
|
||||
'mention_everyone': True,
|
||||
'mentions': [],
|
||||
'mention_roles': [],
|
||||
}
|
||||
|
||||
|
||||
class BaseTest(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.assertIsNotNone(announce, 'The release announcement helper must exist')
|
||||
|
||||
|
||||
class PolicyTests(BaseTest):
|
||||
def test_payload_has_one_literal_everyone_and_no_untrusted_body(self):
|
||||
payload = announce.release_payload(event(), '1')
|
||||
self.assertEqual(payload['content'], CONTENT)
|
||||
self.assertEqual(json.dumps(payload).count('@everyone'), 1)
|
||||
self.assertEqual(
|
||||
payload['allowed_mentions'],
|
||||
{
|
||||
'parse': ['everyone'],
|
||||
'users': [],
|
||||
'roles': [],
|
||||
'replied_user': False,
|
||||
},
|
||||
)
|
||||
self.assertIs(payload['tts'], False)
|
||||
self.assertEqual(payload['flags'], 0)
|
||||
|
||||
def test_drafts_and_prereleases_are_skipped(self):
|
||||
for flag in ('draft', 'prerelease'):
|
||||
with self.subTest(flag=flag):
|
||||
value = event()
|
||||
value['release'][flag] = True
|
||||
self.assertIsNone(announce.release_payload(value, '1'))
|
||||
|
||||
def test_only_published_action_is_accepted(self):
|
||||
for action in ('edited', 'created', 'released', 'deleted', '', None):
|
||||
with self.subTest(action=action):
|
||||
value = event()
|
||||
value['action'] = action
|
||||
with self.assertRaises(announce.AnnouncementError):
|
||||
announce.release_payload(value, '1')
|
||||
|
||||
def test_reruns_and_missing_attempt_refuse_manual_reconciliation(self):
|
||||
for attempt in ('2', '3', '', None, '01', '0', '1\n'):
|
||||
with self.subTest(attempt=attempt):
|
||||
with self.assertRaisesRegex(announce.AnnouncementError, 'manual reconciliation'):
|
||||
announce.release_payload(event(), attempt)
|
||||
|
||||
def test_repository_must_match_exactly(self):
|
||||
for repo in ('evil/LangBot', 'langbot-app/langbot', None):
|
||||
value = event()
|
||||
value['repository']['full_name'] = repo
|
||||
with self.assertRaises(announce.AnnouncementError):
|
||||
announce.release_payload(value, '1')
|
||||
|
||||
def test_hostile_and_noncanonical_tags_are_rejected(self):
|
||||
for tag in (
|
||||
'v1.2.3 @everyone',
|
||||
'v1.2.3\n',
|
||||
'v1.2.3/../../x',
|
||||
'v1.2.3?x=y',
|
||||
'$(id)',
|
||||
'v1.2.3-rc.1',
|
||||
'v1.2.3',
|
||||
'v1.2.3%0a',
|
||||
'<@123>',
|
||||
'v1.2.' + '3' * 100,
|
||||
'',
|
||||
None,
|
||||
123,
|
||||
):
|
||||
with self.subTest(tag=tag):
|
||||
value = event()
|
||||
value['release']['tag_name'] = tag
|
||||
value['release']['html_url'] = f'https://github.com/{REPO}/releases/tag/{tag}'
|
||||
with self.assertRaises(announce.AnnouncementError):
|
||||
announce.release_payload(value, '1')
|
||||
|
||||
def test_release_url_must_be_canonical_and_match_tag(self):
|
||||
for url in (
|
||||
'https://evil.example/tag/v4.10.11',
|
||||
URL + '?x=y',
|
||||
URL + '#anchor',
|
||||
URL + '/',
|
||||
URL.replace('v4.10.11', 'v4.10.12'),
|
||||
URL.replace('github.com', 'github.com@evil.example'),
|
||||
URL.replace('https:', 'http:'),
|
||||
URL + '\n',
|
||||
None,
|
||||
):
|
||||
with self.subTest(url=url):
|
||||
value = event()
|
||||
value['release']['html_url'] = url
|
||||
with self.assertRaises(announce.AnnouncementError):
|
||||
announce.release_payload(value, '1')
|
||||
|
||||
def test_malformed_events_fail_closed(self):
|
||||
for value in (None, [], {}, {'release': []}, {'repository': None}):
|
||||
with self.subTest(value=value):
|
||||
with self.assertRaises(announce.AnnouncementError):
|
||||
announce.release_payload(value, '1')
|
||||
for flag in ('draft', 'prerelease'):
|
||||
for bad in (None, 'false', 0, 1):
|
||||
value = event()
|
||||
value['release'][flag] = bad
|
||||
with self.assertRaises(announce.AnnouncementError):
|
||||
announce.release_payload(value, '1')
|
||||
|
||||
|
||||
class DiscordTests(BaseTest):
|
||||
def setUp(self):
|
||||
super().setUp()
|
||||
self.patch = patch('announce.http.client.HTTPSConnection')
|
||||
self.connection_class = self.patch.start()
|
||||
self.addCleanup(self.patch.stop)
|
||||
self.connection = self.connection_class.return_value
|
||||
|
||||
def respond(self, *values):
|
||||
responses = []
|
||||
for value in values:
|
||||
response = MagicMock()
|
||||
response.status = 200
|
||||
response.read.return_value = json.dumps(value).encode()
|
||||
responses.append(response)
|
||||
self.connection.getresponse.side_effect = responses
|
||||
|
||||
def methods(self):
|
||||
return [call.args[0] for call in self.connection.request.call_args_list]
|
||||
|
||||
def test_webhook_validation_is_get_only_and_reports_ids(self):
|
||||
self.respond(metadata())
|
||||
result = announce.DiscordWebhook(WEBHOOK).validate()
|
||||
self.assertEqual(result, metadata())
|
||||
self.assertEqual(self.methods(), ['GET'])
|
||||
self.assertEqual(
|
||||
self.connection.request.call_args.args[:2], ('GET', f'/api/webhooks/{WEBHOOK_ID}/fixture_token-ONLY')
|
||||
)
|
||||
self.connection_class.assert_called_with('discord.com', timeout=20)
|
||||
self.connection.close.assert_called_once()
|
||||
|
||||
def test_invalid_webhook_urls_are_rejected_before_network(self):
|
||||
for url in (
|
||||
'',
|
||||
None,
|
||||
WEBHOOK + '/',
|
||||
WEBHOOK + '?wait=true',
|
||||
WEBHOOK + '#x',
|
||||
WEBHOOK + '\n',
|
||||
' ' + WEBHOOK,
|
||||
WEBHOOK.replace('https:', 'http:'),
|
||||
WEBHOOK.replace('discord.com', 'discord.com.evil.example'),
|
||||
WEBHOOK.replace('discord.com', 'discord.com@evil.example'),
|
||||
WEBHOOK.replace('discord.com', 'discord.com:443'),
|
||||
WEBHOOK.replace('/api/', '/api/v10/'),
|
||||
WEBHOOK.replace(WEBHOOK_ID, 'abc'),
|
||||
WEBHOOK + '/../../x',
|
||||
WEBHOOK.replace('fixture_token-ONLY', 'a%2Fb'),
|
||||
):
|
||||
with self.subTest(url=url):
|
||||
with self.assertRaises(announce.AnnouncementError):
|
||||
announce.DiscordWebhook(url)
|
||||
self.connection_class.assert_not_called()
|
||||
|
||||
def test_webhook_metadata_requires_incoming_type_and_ids(self):
|
||||
invalid = [
|
||||
None,
|
||||
[],
|
||||
{},
|
||||
dict(metadata(), type=2),
|
||||
dict(metadata(), type=True),
|
||||
dict(metadata(), id='999'),
|
||||
dict(metadata(), channel_id=None),
|
||||
dict(metadata(), guild_id='::error::hostile'),
|
||||
]
|
||||
for value in invalid:
|
||||
with self.subTest(value=value):
|
||||
self.respond(value)
|
||||
with self.assertRaises(announce.AnnouncementError):
|
||||
announce.DiscordWebhook(WEBHOOK).validate()
|
||||
self.assertNotIn('POST', self.methods())
|
||||
|
||||
def test_send_waits_and_reads_back_exact_returned_message(self):
|
||||
self.respond(metadata(), message(), message())
|
||||
result = announce.DiscordWebhook(WEBHOOK).send(announce.release_payload(event(), '1'))
|
||||
self.assertEqual(result, MESSAGE_ID)
|
||||
self.assertEqual(self.methods(), ['GET', 'POST', 'GET'])
|
||||
calls = self.connection.request.call_args_list
|
||||
self.assertEqual(calls[1].args[:2], ('POST', f'/api/webhooks/{WEBHOOK_ID}/fixture_token-ONLY?wait=true'))
|
||||
self.assertEqual(json.loads(calls[1].kwargs['body']), announce.release_payload(event(), '1'))
|
||||
self.assertEqual(
|
||||
calls[2].args[:2], ('GET', f'/api/webhooks/{WEBHOOK_ID}/fixture_token-ONLY/messages/{MESSAGE_ID}')
|
||||
)
|
||||
|
||||
def test_readback_must_match_content_mentions_and_identity(self):
|
||||
for field, bad in (
|
||||
('content', 'wrong'),
|
||||
('mention_everyone', False),
|
||||
('mention_everyone', 1),
|
||||
('mentions', [{'id': '123'}]),
|
||||
('mention_roles', ['123']),
|
||||
('id', '999'),
|
||||
('channel_id', '999'),
|
||||
('webhook_id', '999'),
|
||||
):
|
||||
with self.subTest(field=field, bad=bad):
|
||||
self.connection.reset_mock()
|
||||
self.respond(metadata(), message(), dict(message(), **{field: bad}))
|
||||
with self.assertRaisesRegex(announce.AnnouncementError, 'manual reconciliation'):
|
||||
announce.DiscordWebhook(WEBHOOK).send(announce.release_payload(event(), '1'))
|
||||
self.assertEqual(self.methods().count('POST'), 1)
|
||||
|
||||
def test_missing_readback_fields_fail_closed(self):
|
||||
for field in message():
|
||||
value = message()
|
||||
del value[field]
|
||||
self.respond(metadata(), message(), value)
|
||||
with self.assertRaises(announce.AnnouncementError):
|
||||
announce.DiscordWebhook(WEBHOOK).send(announce.release_payload(event(), '1'))
|
||||
|
||||
def test_unsafe_post_message_id_never_becomes_get_path(self):
|
||||
for value in (None, {}, dict(message(), id='../evil'), dict(message(), id='123?x=y')):
|
||||
self.connection.reset_mock()
|
||||
self.respond(metadata(), value)
|
||||
with self.assertRaisesRegex(announce.AnnouncementError, 'manual reconciliation'):
|
||||
announce.DiscordWebhook(WEBHOOK).send(announce.release_payload(event(), '1'))
|
||||
self.assertEqual(self.methods(), ['GET', 'POST'])
|
||||
|
||||
def test_post_failure_never_retries_and_never_logs_secret(self):
|
||||
for status in (301, 302, 307, 308, 400, 401, 403, 429, 500, 204):
|
||||
with self.subTest(status=status):
|
||||
self.connection.reset_mock()
|
||||
self.respond(metadata(), message())
|
||||
responses = list(self.connection.getresponse.side_effect)
|
||||
responses[1].status = status
|
||||
self.connection.getresponse.side_effect = responses
|
||||
with self.assertRaisesRegex(announce.AnnouncementError, 'manual reconciliation') as caught:
|
||||
announce.DiscordWebhook(WEBHOOK).send(announce.release_payload(event(), '1'))
|
||||
self.assertNotIn('fixture_token', str(caught.exception))
|
||||
self.assertEqual(self.methods(), ['GET', 'POST'])
|
||||
|
||||
def test_ambiguous_timeout_never_retries_or_echoes_exception(self):
|
||||
self.respond(metadata())
|
||||
first = next(self.connection.getresponse.side_effect)
|
||||
self.connection.getresponse.side_effect = [first, TimeoutError(WEBHOOK)]
|
||||
with self.assertRaisesRegex(announce.AnnouncementError, 'manual reconciliation') as caught:
|
||||
announce.DiscordWebhook(WEBHOOK).send(announce.release_payload(event(), '1'))
|
||||
self.assertNotIn('fixture_token', str(caught.exception))
|
||||
self.assertEqual(self.methods(), ['GET', 'POST'])
|
||||
|
||||
def test_malformed_json_response_is_sanitized(self):
|
||||
self.respond(metadata())
|
||||
response = next(self.connection.getresponse.side_effect)
|
||||
response.read.return_value = WEBHOOK.encode()
|
||||
self.connection.getresponse.side_effect = [response]
|
||||
with self.assertRaises(announce.AnnouncementError) as caught:
|
||||
announce.DiscordWebhook(WEBHOOK).validate()
|
||||
self.assertNotIn('fixture_token', str(caught.exception))
|
||||
|
||||
def test_get_redirect_is_not_followed(self):
|
||||
self.respond(metadata())
|
||||
response = next(self.connection.getresponse.side_effect)
|
||||
response.status = 302
|
||||
response.getheader.return_value = 'https://evil.example/'
|
||||
self.connection.getresponse.side_effect = [response]
|
||||
with self.assertRaises(announce.AnnouncementError):
|
||||
announce.DiscordWebhook(WEBHOOK).validate()
|
||||
self.assertEqual(self.methods(), ['GET'])
|
||||
|
||||
|
||||
class EntrypointTests(BaseTest):
|
||||
def run_main(self, data=None, **overrides):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
path = Path(directory) / 'event.json'
|
||||
path.write_text(json.dumps(event() if data is None else data))
|
||||
env = {
|
||||
'GITHUB_EVENT_NAME': 'release',
|
||||
'GITHUB_EVENT_PATH': str(path),
|
||||
'GITHUB_REPOSITORY': REPO,
|
||||
'GITHUB_RUN_ATTEMPT': '1',
|
||||
'DISCORD_RELEASE_WEBHOOK_URL': WEBHOOK,
|
||||
}
|
||||
env.update(overrides)
|
||||
output = io.StringIO()
|
||||
with contextlib.redirect_stdout(output), contextlib.redirect_stderr(output):
|
||||
result = announce.main(env)
|
||||
return result, output.getvalue()
|
||||
|
||||
def test_dispatch_only_validates_even_if_event_contains_release(self):
|
||||
with patch('announce.DiscordWebhook') as client:
|
||||
client.return_value.validate.return_value = metadata()
|
||||
result, output = self.run_main(GITHUB_EVENT_NAME='workflow_dispatch')
|
||||
self.assertEqual(result, 0)
|
||||
client.return_value.validate.assert_called_once()
|
||||
client.return_value.send.assert_not_called()
|
||||
self.assertIn(GUILD_ID, output)
|
||||
self.assertIn(CHANNEL_ID, output)
|
||||
self.assertNotIn('fixture_token', output)
|
||||
|
||||
def test_production_release_sends_once(self):
|
||||
with patch('announce.DiscordWebhook') as client:
|
||||
client.return_value.send.return_value = MESSAGE_ID
|
||||
result, output = self.run_main()
|
||||
self.assertEqual(result, 0)
|
||||
client.return_value.send.assert_called_once_with(announce.release_payload(event(), '1'))
|
||||
self.assertIn(MESSAGE_ID, output)
|
||||
|
||||
def test_skipped_releases_need_no_secret_or_network(self):
|
||||
for flag in ('draft', 'prerelease'):
|
||||
value = event()
|
||||
value['release'][flag] = True
|
||||
with patch('announce.DiscordWebhook') as client:
|
||||
result, _ = self.run_main(value, DISCORD_RELEASE_WEBHOOK_URL='')
|
||||
self.assertEqual(result, 0)
|
||||
client.assert_not_called()
|
||||
|
||||
def test_rerun_never_constructs_client(self):
|
||||
with patch('announce.DiscordWebhook') as client:
|
||||
result, output = self.run_main(GITHUB_RUN_ATTEMPT='2')
|
||||
self.assertEqual(result, 1)
|
||||
self.assertIn('manual reconciliation', output)
|
||||
client.assert_not_called()
|
||||
|
||||
def test_unexpected_event_or_repository_cannot_send(self):
|
||||
for overrides in (
|
||||
{'GITHUB_EVENT_NAME': 'push'},
|
||||
{'GITHUB_EVENT_NAME': 'pull_request'},
|
||||
{'GITHUB_REPOSITORY': 'evil/LangBot'},
|
||||
):
|
||||
with patch('announce.DiscordWebhook') as client:
|
||||
result, _ = self.run_main(**overrides)
|
||||
self.assertEqual(result, 1)
|
||||
client.assert_not_called()
|
||||
|
||||
def test_missing_secret_fails_clearly_for_send_and_validation(self):
|
||||
for name in ('release', 'workflow_dispatch'):
|
||||
result, output = self.run_main(GITHUB_EVENT_NAME=name, DISCORD_RELEASE_WEBHOOK_URL='')
|
||||
self.assertEqual(result, 1)
|
||||
self.assertIn('DISCORD_RELEASE_WEBHOOK_URL is missing', output)
|
||||
|
||||
def test_cli_reads_event_file_and_redacts_invalid_input(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
path = Path(directory) / 'event.json'
|
||||
value = event()
|
||||
value['release']['tag_name'] = '::error::hostile @everyone'
|
||||
path.write_text(json.dumps(value))
|
||||
env = dict(
|
||||
os.environ,
|
||||
GITHUB_EVENT_NAME='release',
|
||||
GITHUB_EVENT_PATH=str(path),
|
||||
GITHUB_REPOSITORY=REPO,
|
||||
GITHUB_RUN_ATTEMPT='1',
|
||||
DISCORD_RELEASE_WEBHOOK_URL=WEBHOOK,
|
||||
)
|
||||
result = subprocess.run(
|
||||
[sys.executable, str(Path(__file__).with_name('announce.py'))],
|
||||
env=env,
|
||||
text=True,
|
||||
capture_output=True,
|
||||
check=False,
|
||||
)
|
||||
self.assertEqual(result.returncode, 1)
|
||||
self.assertNotIn('hostile', result.stderr)
|
||||
self.assertNotIn('fixture_token', result.stderr)
|
||||
self.assertNotIn('Traceback', result.stderr)
|
||||
|
||||
def test_unreadable_event_fails_safely(self):
|
||||
result, output = self.run_main(GITHUB_EVENT_PATH='/nonexistent/event.json')
|
||||
self.assertEqual(result, 1)
|
||||
self.assertNotIn('Traceback', output)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
@@ -0,0 +1,64 @@
|
||||
name: Discord Release Announcement
|
||||
|
||||
on:
|
||||
release:
|
||||
types: [published]
|
||||
workflow_dispatch:
|
||||
push:
|
||||
paths:
|
||||
- '.github/workflows/discord-release.yml'
|
||||
- '.github/discord-release/**'
|
||||
pull_request:
|
||||
paths:
|
||||
- '.github/workflows/discord-release.yml'
|
||||
- '.github/discord-release/**'
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
tests:
|
||||
name: Offline announcement tests
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 5
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
|
||||
with:
|
||||
persist-credentials: false
|
||||
- name: Test helper without secrets or network
|
||||
run: python3 -m unittest discover -s .github/discord-release -p 'test_*.py' -v
|
||||
|
||||
validate:
|
||||
name: Validate webhook (GET only, no message)
|
||||
if: github.repository == 'langbot-app/LangBot' && github.event_name == 'workflow_dispatch'
|
||||
needs: tests
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 5
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
|
||||
with:
|
||||
persist-credentials: false
|
||||
- name: Validate incoming webhook and report guild/channel IDs
|
||||
env:
|
||||
DISCORD_RELEASE_WEBHOOK_URL: ${{ secrets.DISCORD_RELEASE_WEBHOOK_URL }}
|
||||
run: python3 .github/discord-release/announce.py
|
||||
|
||||
announce:
|
||||
name: Announce published stable release
|
||||
if: >-
|
||||
github.repository == 'langbot-app/LangBot' &&
|
||||
github.event_name == 'release' && github.event.action == 'published' &&
|
||||
github.event.release.draft == false && github.event.release.prerelease == false
|
||||
needs: tests
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 5
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
|
||||
with:
|
||||
persist-credentials: false
|
||||
# The helper refuses GITHUB_RUN_ATTEMPT != 1 with recovery guidance.
|
||||
# Never interpolate release data into a shell command.
|
||||
- name: Send once and verify the exact Discord message
|
||||
env:
|
||||
DISCORD_RELEASE_WEBHOOK_URL: ${{ secrets.DISCORD_RELEASE_WEBHOOK_URL }}
|
||||
run: python3 .github/discord-release/announce.py
|
||||
+2
-1
@@ -110,7 +110,8 @@ classifiers = [
|
||||
|
||||
[project.optional-dependencies]
|
||||
seekdb = [
|
||||
"pyseekdb==1.1.0.post3",
|
||||
"pyseekdb==1.4.0.post1",
|
||||
"pylibseekdb==1.4.0; sys_platform == 'linux' or (sys_platform == 'darwin' and platform_machine == 'arm64')",
|
||||
]
|
||||
|
||||
[project.urls]
|
||||
|
||||
@@ -101,18 +101,6 @@ class SeekDBVectorDatabase(VectorDatabase):
|
||||
self._collection_configs: Dict[str, HNSWConfiguration] = {}
|
||||
self._runtime_cache_limit = runtime_cache_limit(ap)
|
||||
|
||||
self._escape_table = str.maketrans(
|
||||
{
|
||||
'\x00': '',
|
||||
'\\': '\\\\',
|
||||
"'": "''", # Standard SQL escaping (OceanBase NO_BACKSLASH_ESCAPES)
|
||||
'"': '\\"',
|
||||
'\n': '\\n',
|
||||
'\r': '\\r',
|
||||
'\t': '\\t',
|
||||
}
|
||||
)
|
||||
|
||||
async def close(self) -> None:
|
||||
self._collections.clear()
|
||||
self._collection_configs.clear()
|
||||
@@ -192,16 +180,22 @@ class SeekDBVectorDatabase(VectorDatabase):
|
||||
return coll
|
||||
|
||||
def _clean_metadata(self, meta: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""SeekDB metadata doesn't support \\ and ", insert will error 3104"""
|
||||
return {
|
||||
k: v.translate(self._escape_table)
|
||||
if isinstance(v, str)
|
||||
else v
|
||||
if v is None or isinstance(v, (int, float, bool))
|
||||
else str(v)
|
||||
for k, v in meta.items()
|
||||
if v is not None
|
||||
}
|
||||
"""Keep supported scalar metadata values without altering strings."""
|
||||
return {k: v if isinstance(v, (str, int, float, bool)) else str(v) for k, v in meta.items() if v is not None}
|
||||
|
||||
@staticmethod
|
||||
def _relevance_scores_to_distances(results: Dict[str, Any]) -> None:
|
||||
"""Convert SeekDB hybrid relevance scores to lower-is-better distances."""
|
||||
distances = results.get('distances')
|
||||
if not isinstance(distances, list):
|
||||
return
|
||||
|
||||
results['distances'] = [
|
||||
[1.0 - float(score) if isinstance(score, (int, float)) else score for score in batch]
|
||||
if isinstance(batch, list)
|
||||
else batch
|
||||
for batch in distances
|
||||
]
|
||||
|
||||
async def get_or_create_collection(self, collection: str):
|
||||
"""Get or create collection (without vector size - will use default)."""
|
||||
@@ -236,10 +230,10 @@ class SeekDBVectorDatabase(VectorDatabase):
|
||||
|
||||
kwargs: Dict[str, Any] = dict(ids=ids, embeddings=embeddings_list, metadatas=cleaned_metadatas)
|
||||
if documents is not None:
|
||||
kwargs['documents'] = [doc.translate(self._escape_table) for doc in documents]
|
||||
await asyncio.to_thread(coll.add, **kwargs)
|
||||
kwargs['documents'] = documents
|
||||
await asyncio.to_thread(coll.upsert, **kwargs)
|
||||
|
||||
self.ap.logger.info(f"Added {len(ids)} embeddings to SeekDB collection '{collection}'")
|
||||
self.ap.logger.info(f"Upserted {len(ids)} embeddings into SeekDB collection '{collection}'")
|
||||
|
||||
async def search(
|
||||
self,
|
||||
@@ -287,7 +281,8 @@ class SeekDBVectorDatabase(VectorDatabase):
|
||||
# Route by search type.
|
||||
# pyseekdb's query() always requires embeddings, so full-text and
|
||||
# hybrid modes use hybrid_search() which supports text-only queries
|
||||
# and returns the same nested-list format with distances.
|
||||
# and returns relevance scores in the nested ``distances`` field.
|
||||
returns_relevance_scores = False
|
||||
if search_type == SearchType.FULL_TEXT:
|
||||
if not query_text:
|
||||
return {'ids': [[]], 'metadatas': [[]], 'distances': [[]]}
|
||||
@@ -309,6 +304,7 @@ class SeekDBVectorDatabase(VectorDatabase):
|
||||
n_results=k,
|
||||
include=['documents', 'metadatas'],
|
||||
)
|
||||
returns_relevance_scores = True
|
||||
|
||||
elif search_type == SearchType.HYBRID:
|
||||
if not query_text:
|
||||
@@ -352,6 +348,7 @@ class SeekDBVectorDatabase(VectorDatabase):
|
||||
n_results=k,
|
||||
include=['documents', 'metadatas'],
|
||||
)
|
||||
returns_relevance_scores = True
|
||||
self.ap.logger.info(
|
||||
f"SeekDB hybrid search in '{collection}' returned {len(results.get('ids', [[]])[0])} results."
|
||||
)
|
||||
@@ -363,6 +360,8 @@ class SeekDBVectorDatabase(VectorDatabase):
|
||||
results = await asyncio.to_thread(coll.query, **query_kwargs)
|
||||
|
||||
results = self._json_safe(results)
|
||||
if returns_relevance_scores:
|
||||
self._relevance_scores_to_distances(results)
|
||||
self.ap.logger.info(
|
||||
f"SeekDB {search_type} search in '{collection}' returned {len(results.get('ids', [[]])[0])} results"
|
||||
)
|
||||
|
||||
@@ -0,0 +1,123 @@
|
||||
"""Real embedded SeekDB regression tests.
|
||||
|
||||
Install the optional dependency before running these slow tests::
|
||||
|
||||
uv sync --dev --extra seekdb
|
||||
uv run pytest tests/integration/vector/test_seekdb.py -m slow -q
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
from types import SimpleNamespace
|
||||
import uuid
|
||||
|
||||
import pytest
|
||||
|
||||
pytest.importorskip('pyseekdb')
|
||||
|
||||
from langbot.pkg.vector.vdbs.seekdb import SeekDBVectorDatabase
|
||||
|
||||
|
||||
pytestmark = [pytest.mark.integration, pytest.mark.slow]
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
async def backend(tmp_path):
|
||||
app = SimpleNamespace(
|
||||
instance_config=SimpleNamespace(
|
||||
data={
|
||||
'vdb': {
|
||||
'runtime_cache_limit': 16,
|
||||
'seekdb': {
|
||||
'mode': 'embedded',
|
||||
'path': str(tmp_path),
|
||||
'database': 'langbot_test',
|
||||
},
|
||||
}
|
||||
}
|
||||
),
|
||||
logger=SimpleNamespace(
|
||||
info=lambda *args, **kwargs: None,
|
||||
warning=lambda *args, **kwargs: None,
|
||||
),
|
||||
)
|
||||
database = SeekDBVectorDatabase(app)
|
||||
collection = f'test_{uuid.uuid4().hex}'
|
||||
yield database, collection
|
||||
|
||||
await database.delete_collection(collection)
|
||||
await database.close()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_upsert_and_text_round_trip(backend) -> None:
|
||||
database, collection = backend
|
||||
original = 'He said "hello".\nC:\\notes\\file.txt isn\'t empty. 中文'
|
||||
updated = f'Updated: {original}'
|
||||
|
||||
await database.add_embeddings(
|
||||
collection,
|
||||
['document-a'],
|
||||
[[1.0, 0.0, 0.0]],
|
||||
[{'file_id': 'file-a', 'text': original}],
|
||||
[original],
|
||||
)
|
||||
await database.add_embeddings(
|
||||
collection,
|
||||
['document-a'],
|
||||
[[0.0, 1.0, 0.0]],
|
||||
[{'file_id': 'file-a', 'text': updated}],
|
||||
[updated],
|
||||
)
|
||||
|
||||
items, _ = await database.list_by_filter(collection, {'file_id': 'file-a'})
|
||||
|
||||
assert len(items) == 1
|
||||
assert items[0]['id'] == 'document-a'
|
||||
assert items[0]['document'] == updated
|
||||
assert items[0]['metadata']['text'] == updated
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_full_text_and_hybrid_results_keep_relevance_order(backend) -> None:
|
||||
database, collection = backend
|
||||
documents = [
|
||||
'orchid orchid orchid flower',
|
||||
'orchid grows in a garden with many other beautiful plants',
|
||||
'a completely unrelated topic',
|
||||
]
|
||||
|
||||
await database.add_embeddings(
|
||||
collection,
|
||||
['best', 'weak', 'noise'],
|
||||
[[1.0, 0.0, 0.0], [0.9, 0.1, 0.0], [0.0, 0.0, 1.0]],
|
||||
[
|
||||
{'file_id': item_id, 'document_id': item_id, 'text': document}
|
||||
for item_id, document in zip(['best', 'weak', 'noise'], documents, strict=True)
|
||||
],
|
||||
documents,
|
||||
)
|
||||
seekdb_collection = await database.get_or_create_collection(collection)
|
||||
await asyncio.to_thread(seekdb_collection.refresh_index)
|
||||
|
||||
full_text = await database.search(
|
||||
collection,
|
||||
[1.0, 0.0, 0.0],
|
||||
k=3,
|
||||
search_type='full_text',
|
||||
query_text='orchid',
|
||||
)
|
||||
hybrid = await database.search(
|
||||
collection,
|
||||
[1.0, 0.0, 0.0],
|
||||
k=3,
|
||||
search_type='hybrid',
|
||||
query_text='orchid',
|
||||
vector_weight=0.65,
|
||||
)
|
||||
|
||||
assert full_text['ids'][0][:2] == ['best', 'weak']
|
||||
assert full_text['distances'][0] == sorted(full_text['distances'][0])
|
||||
assert hybrid['ids'][0] == ['best', 'weak', 'noise']
|
||||
assert hybrid['distances'][0] == sorted(hybrid['distances'][0])
|
||||
@@ -12,4 +12,7 @@ def test_seekdb_is_only_declared_as_an_optional_dependency() -> None:
|
||||
project = pyproject['project']
|
||||
base_dependencies = project['dependencies']
|
||||
assert not any(dependency.lower().startswith('pyseekdb') for dependency in base_dependencies)
|
||||
assert project['optional-dependencies']['seekdb'] == ['pyseekdb==1.1.0.post3']
|
||||
assert project['optional-dependencies']['seekdb'] == [
|
||||
'pyseekdb==1.4.0.post1',
|
||||
"pylibseekdb==1.4.0; sys_platform == 'linux' or (sys_platform == 'darwin' and platform_machine == 'arm64')",
|
||||
]
|
||||
|
||||
@@ -0,0 +1,96 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import AsyncMock, MagicMock
|
||||
|
||||
import pytest
|
||||
|
||||
from langbot.pkg.vector.vdbs.seekdb import SeekDBVectorDatabase
|
||||
|
||||
|
||||
def _adapter_with_collection(collection: MagicMock) -> SeekDBVectorDatabase:
|
||||
adapter = SeekDBVectorDatabase.__new__(SeekDBVectorDatabase)
|
||||
adapter.ap = SimpleNamespace(logger=MagicMock())
|
||||
adapter.client = MagicMock()
|
||||
adapter.client.has_collection.return_value = True
|
||||
adapter._collections = {'knowledge_base': collection}
|
||||
adapter._runtime_cache_limit = 16
|
||||
return adapter
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_add_embeddings_upserts_and_preserves_text() -> None:
|
||||
collection = MagicMock()
|
||||
adapter = _adapter_with_collection(collection)
|
||||
adapter._get_or_create_collection_internal = AsyncMock(return_value=collection)
|
||||
original = 'He said "hello".\nC:\\notes\\file.txt isn\'t empty. 中文'
|
||||
|
||||
await adapter.add_embeddings(
|
||||
collection='knowledge_base',
|
||||
ids=['document-a'],
|
||||
embeddings_list=[[1.0, 0.0, 0.0]],
|
||||
metadatas=[{'text': original}],
|
||||
documents=[original],
|
||||
)
|
||||
|
||||
collection.upsert.assert_called_once_with(
|
||||
ids=['document-a'],
|
||||
embeddings=[[1.0, 0.0, 0.0]],
|
||||
metadatas=[{'text': original}],
|
||||
documents=[original],
|
||||
)
|
||||
collection.add.assert_not_called()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize(
|
||||
('search_type', 'scores', 'expected_distances'),
|
||||
[
|
||||
('full_text', [0.4508196721, 0.25], [0.5491803279, 0.75]),
|
||||
('hybrid', [0.0328, 0.0323, 0.0159], [0.9672, 0.9677, 0.9841]),
|
||||
],
|
||||
)
|
||||
async def test_search_converts_relevance_scores_to_distances(
|
||||
search_type: str,
|
||||
scores: list[float],
|
||||
expected_distances: list[float],
|
||||
) -> None:
|
||||
collection = MagicMock()
|
||||
collection.hybrid_search.return_value = {
|
||||
'ids': [['best', 'weak', 'noise'][: len(scores)]],
|
||||
'metadatas': [[{} for _ in scores]],
|
||||
'distances': [scores],
|
||||
}
|
||||
adapter = _adapter_with_collection(collection)
|
||||
|
||||
results = await adapter.search(
|
||||
collection='knowledge_base',
|
||||
query_embedding=[1.0, 0.0, 0.0],
|
||||
k=len(scores),
|
||||
search_type=search_type,
|
||||
query_text='orchid',
|
||||
vector_weight=0.65,
|
||||
)
|
||||
|
||||
assert results['distances'][0] == pytest.approx(expected_distances)
|
||||
assert results['distances'][0] == sorted(results['distances'][0])
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_vector_search_keeps_seekdb_cosine_distances() -> None:
|
||||
collection = MagicMock()
|
||||
collection.query.return_value = {
|
||||
'ids': [['best', 'weak']],
|
||||
'metadatas': [[{}, {}]],
|
||||
'distances': [[0.1, 0.25]],
|
||||
}
|
||||
adapter = _adapter_with_collection(collection)
|
||||
|
||||
results = await adapter.search(
|
||||
collection='knowledge_base',
|
||||
query_embedding=[1.0, 0.0, 0.0],
|
||||
k=2,
|
||||
search_type='vector',
|
||||
)
|
||||
|
||||
assert results['distances'] == [[0.1, 0.25]]
|
||||
@@ -1066,7 +1066,7 @@ name = "cuda-bindings"
|
||||
version = "13.3.1"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "cuda-pathfinder" },
|
||||
{ name = "cuda-pathfinder", marker = "python_full_version >= '3.14' and sys_platform != 'emscripten' and sys_platform != 'win32'" },
|
||||
]
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/51/6b/457ca12dad3ee9bfcc9a545cfd6b64b359ba49de40f776f6e028e678f262/cuda_bindings-13.3.1-cp311-cp311-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:c5879712accf6e14bb01aa5e67440eb84998b8d104b509cc7a6dc0b8f656a474", size = 6053539, upload-time = "2026-05-29T23:11:43.19Z" },
|
||||
@@ -1099,34 +1099,34 @@ wheels = [
|
||||
|
||||
[package.optional-dependencies]
|
||||
cudart = [
|
||||
{ name = "nvidia-cuda-runtime" },
|
||||
{ name = "nvidia-cuda-runtime", marker = "python_full_version >= '3.14' and sys_platform == 'linux'" },
|
||||
]
|
||||
cufft = [
|
||||
{ name = "nvidia-cufft" },
|
||||
{ name = "nvidia-cufft", marker = "python_full_version >= '3.14' and sys_platform == 'linux'" },
|
||||
]
|
||||
cufile = [
|
||||
{ name = "nvidia-cufile" },
|
||||
{ name = "nvidia-cufile", marker = "python_full_version >= '3.14' and sys_platform == 'linux'" },
|
||||
]
|
||||
cupti = [
|
||||
{ name = "nvidia-cuda-cupti" },
|
||||
{ name = "nvidia-cuda-cupti", marker = "python_full_version >= '3.14' and sys_platform == 'linux'" },
|
||||
]
|
||||
curand = [
|
||||
{ name = "nvidia-curand" },
|
||||
{ name = "nvidia-curand", marker = "python_full_version >= '3.14' and sys_platform == 'linux'" },
|
||||
]
|
||||
cusolver = [
|
||||
{ name = "nvidia-cusolver" },
|
||||
{ name = "nvidia-cusolver", marker = "python_full_version >= '3.14' and sys_platform == 'linux'" },
|
||||
]
|
||||
cusparse = [
|
||||
{ name = "nvidia-cusparse" },
|
||||
{ name = "nvidia-cusparse", marker = "python_full_version >= '3.14' and sys_platform == 'linux'" },
|
||||
]
|
||||
nvjitlink = [
|
||||
{ name = "nvidia-nvjitlink" },
|
||||
{ name = "nvidia-nvjitlink", marker = "python_full_version >= '3.14' and sys_platform == 'linux'" },
|
||||
]
|
||||
nvrtc = [
|
||||
{ name = "nvidia-cuda-nvrtc" },
|
||||
{ name = "nvidia-cuda-nvrtc", marker = "python_full_version >= '3.14' and sys_platform == 'linux'" },
|
||||
]
|
||||
nvtx = [
|
||||
{ name = "nvidia-nvtx" },
|
||||
{ name = "nvidia-nvtx", marker = "python_full_version >= '3.14' and sys_platform == 'linux'" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -2139,6 +2139,7 @@ dependencies = [
|
||||
|
||||
[package.optional-dependencies]
|
||||
seekdb = [
|
||||
{ name = "pylibseekdb", marker = "(platform_machine == 'arm64' and sys_platform == 'darwin') or sys_platform == 'linux'" },
|
||||
{ name = "pyseekdb" },
|
||||
]
|
||||
|
||||
@@ -2203,10 +2204,11 @@ requires-dist = [
|
||||
{ name = "pycryptodome", specifier = ">=3.22.0" },
|
||||
{ name = "pydantic", specifier = ">2.0" },
|
||||
{ name = "pyjwt", specifier = ">=2.12.0" },
|
||||
{ name = "pylibseekdb", marker = "(platform_machine == 'arm64' and sys_platform == 'darwin' and extra == 'seekdb') or (sys_platform == 'linux' and extra == 'seekdb')", specifier = "==1.4.0" },
|
||||
{ name = "pymilvus", specifier = ">=2.6.4" },
|
||||
{ name = "pynacl", specifier = ">=1.5.0" },
|
||||
{ name = "pypdf2", specifier = ">=3.0.1" },
|
||||
{ name = "pyseekdb", marker = "extra == 'seekdb'", specifier = "==1.1.0.post3" },
|
||||
{ name = "pyseekdb", marker = "extra == 'seekdb'", specifier = "==1.4.0.post1" },
|
||||
{ name = "python-docx", specifier = ">=1.1.0" },
|
||||
{ name = "python-multipart", specifier = ">=0.0.27" },
|
||||
{ name = "python-socks", specifier = ">=2.7.1" },
|
||||
@@ -3297,7 +3299,7 @@ name = "nvidia-cublas"
|
||||
version = "13.1.1.3"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "nvidia-cuda-nvrtc" },
|
||||
{ name = "nvidia-cuda-nvrtc", marker = "python_full_version >= '3.14' and sys_platform != 'emscripten' and sys_platform != 'win32'" },
|
||||
]
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/a7/a1/0bd24ee8c8d03adac032fd2909426a00c88f8c57961b1277ded97f91119f/nvidia_cublas-13.1.1.3-py3-none-manylinux_2_27_aarch64.whl", hash = "sha256:b7a210458267ac818974c53038fbec2e969d5c99f305ab15c72522fa9f001dd5", size = 542848918, upload-time = "2026-04-08T18:46:22.985Z" },
|
||||
@@ -3336,7 +3338,7 @@ name = "nvidia-cudnn-cu13"
|
||||
version = "9.20.0.48"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "nvidia-cublas" },
|
||||
{ name = "nvidia-cublas", marker = "python_full_version >= '3.14' and sys_platform != 'emscripten' and sys_platform != 'win32'" },
|
||||
]
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/56/c5/83384d846b2fd17c44bd499b36c75a45ed4f095fbbb2252294e89cea5c5c/nvidia_cudnn_cu13-9.20.0.48-py3-none-manylinux_2_27_aarch64.whl", hash = "sha256:e31454ae00094b0c55319d9d15b6fa2fc50a9e1c0f5c8c80fb75258234e731e1", size = 444574296, upload-time = "2026-03-09T19:28:27.751Z" },
|
||||
@@ -3348,7 +3350,7 @@ name = "nvidia-cufft"
|
||||
version = "12.0.0.61"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "nvidia-nvjitlink" },
|
||||
{ name = "nvidia-nvjitlink", marker = "python_full_version >= '3.14' and sys_platform != 'emscripten' and sys_platform != 'win32'" },
|
||||
]
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/8b/ae/f417a75c0259e85c1d2f83ca4e960289a5f814ed0cea74d18c353d3e989d/nvidia_cufft-12.0.0.61-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:2708c852ef8cd89d1d2068bdbece0aa188813a0c934db3779b9b1faa8442e5f5", size = 214053554, upload-time = "2025-09-04T08:31:38.196Z" },
|
||||
@@ -3378,9 +3380,9 @@ name = "nvidia-cusolver"
|
||||
version = "12.0.4.66"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "nvidia-cublas" },
|
||||
{ name = "nvidia-cusparse" },
|
||||
{ name = "nvidia-nvjitlink" },
|
||||
{ name = "nvidia-cublas", marker = "python_full_version >= '3.14' and sys_platform != 'emscripten' and sys_platform != 'win32'" },
|
||||
{ name = "nvidia-cusparse", marker = "python_full_version >= '3.14' and sys_platform != 'emscripten' and sys_platform != 'win32'" },
|
||||
{ name = "nvidia-nvjitlink", marker = "python_full_version >= '3.14' and sys_platform != 'emscripten' and sys_platform != 'win32'" },
|
||||
]
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/c8/c3/b30c9e935fc01e3da443ec0116ed1b2a009bb867f5324d3f2d7e533e776b/nvidia_cusolver-12.0.4.66-py3-none-manylinux_2_27_aarch64.whl", hash = "sha256:02c2457eaa9e39de20f880f4bd8820e6a1cfb9f9a34f820eb12a155aa5bc92d2", size = 223467760, upload-time = "2025-09-04T08:33:04.222Z" },
|
||||
@@ -3392,7 +3394,7 @@ name = "nvidia-cusparse"
|
||||
version = "12.6.3.3"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "nvidia-nvjitlink" },
|
||||
{ name = "nvidia-nvjitlink", marker = "python_full_version >= '3.14' and sys_platform != 'emscripten' and sys_platform != 'win32'" },
|
||||
]
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/f8/94/5c26f33738ae35276672f12615a64bd008ed5be6d1ebcb23579285d960a9/nvidia_cusparse-12.6.3.3-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:80bcc4662f23f1054ee334a15c72b8940402975e0eab63178fc7e670aa59472c", size = 162155568, upload-time = "2025-09-04T08:33:42.864Z" },
|
||||
@@ -4482,21 +4484,18 @@ crypto = [
|
||||
|
||||
[[package]]
|
||||
name = "pylibseekdb"
|
||||
version = "1.3.0"
|
||||
version = "1.4.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "pymysql", marker = "sys_platform != 'emscripten' and sys_platform != 'win32'" },
|
||||
]
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/23/1e/5d971387d4bcdcf0f6f3c85d681a207c49f20715cf566a88d2222e5cd4c0/pylibseekdb-1.3.0-cp311-cp311-macosx_15_0_arm64.whl", hash = "sha256:1d33cf82f34339bc58ac160688fc7d15ac2f7cbb226338d3887fe8350f65b762", size = 142749176, upload-time = "2026-05-25T08:59:18.118Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/4d/9e/47f4a1ebad7e95169cfff1b87433b38623cc68426b3dfaac244c2492e5d4/pylibseekdb-1.3.0-cp311-cp311-manylinux_2_28_aarch64.whl", hash = "sha256:77ba6786908cd8ab320ed4e5d5ef352759ef8990d72aff913467db5fe32542c4", size = 140878003, upload-time = "2026-05-25T06:11:51.929Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/a7/b1/c772c15444ddec07365c5728624824b7b2137c319398c3cfc44d2e6b09a3/pylibseekdb-1.3.0-cp311-cp311-manylinux_2_28_x86_64.whl", hash = "sha256:4b127c21ac1178ab903735041b6afe25295731d7bcee9813e5e1576c9d384937", size = 160132660, upload-time = "2026-05-25T06:12:02.817Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/60/e8/d53bb80f6ed27f19dfb5b2f996cf9bef0e054442d473493e4f2425265762/pylibseekdb-1.3.0-cp312-cp312-macosx_15_0_arm64.whl", hash = "sha256:23cd6ad60a80543dfccb4dc9500401347b82fddb8cef10f5503e5eb816adb39f", size = 142736028, upload-time = "2026-05-25T08:59:41.571Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/2b/e6/3811303e0740e45dd475e6cf8ccea2abb706f047e50455ec1834bdeb6068/pylibseekdb-1.3.0-cp312-cp312-manylinux_2_28_aarch64.whl", hash = "sha256:ec2465e206574f5dee7870bde2434a5ab9a03c2001786b1765fcb5dd790d6f98", size = 140881851, upload-time = "2026-05-25T06:12:11.973Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/5d/29/856ea807cbe997c9fe2df6257106b2b2924ef9458bf87db7e4bd0b8dec03/pylibseekdb-1.3.0-cp312-cp312-manylinux_2_28_x86_64.whl", hash = "sha256:1b78f26dfbb80157169b81f22ebb80957e3c6ee7b33e5ff35beaa4d628c33915", size = 160133328, upload-time = "2026-05-25T06:12:22.051Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/3d/f1/5ec7782810746e9c065a419e8105a5925b3b04f495296b507706da9dc3b3/pylibseekdb-1.3.0-cp313-cp313-macosx_15_0_arm64.whl", hash = "sha256:f6f739454aff786beeccfe71b66a0d89d01b5a8a260e0b8c5c30f8e9184bd88a", size = 142743219, upload-time = "2026-05-25T09:00:08.798Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/13/8a/4d8150f6ad5f11dca40a6d42df9e2a41ed47125735a49afc7d2528460cd3/pylibseekdb-1.3.0-cp313-cp313-manylinux_2_28_aarch64.whl", hash = "sha256:89069e1aeeb51f61aeaa0cf5d94bedb918f46c3476d7b30183dde7b2101e5954", size = 140884366, upload-time = "2026-05-25T06:12:31.689Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/46/29/0583f2e00dbad80efffd7cb7df6431bd086b01a94d8b69688bae15a52e84/pylibseekdb-1.3.0-cp313-cp313-manylinux_2_28_x86_64.whl", hash = "sha256:2515ea14bbac59e6f9f90a43bbaf179050ad7f8ab683d1cb9fd7fe225ccdca4e", size = 160137143, upload-time = "2026-05-25T06:12:43.005Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/ad/5d/8c9afc77d32adbb1f7af85c3131419bcc9860677c5d6efb2d8d0ae9a7a66/pylibseekdb-1.3.0-cp314-cp314-macosx_15_0_arm64.whl", hash = "sha256:a4177a3a6369699c9791cef3a7bfe7b472af301352237ed6e4cea42034fc0047", size = 142739982, upload-time = "2026-05-25T09:00:26.672Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/56/91/bd3f9dea464cc22b454bbe384df3423e36e9fcbe7b1779c861f7ca9721e3/pylibseekdb-1.3.0-cp314-cp314-manylinux_2_28_aarch64.whl", hash = "sha256:8651b8e0324fa78a5ed93b9952f4140c968655c344ef11fdb20d754077efeb05", size = 140896377, upload-time = "2026-05-25T06:12:53.468Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/1e/f4/fcf930ed8c6d40154f41edfb2054794c786dd66deced3a8cc3fef5898af7/pylibseekdb-1.3.0-cp314-cp314-manylinux_2_28_x86_64.whl", hash = "sha256:e6e58bce51e709c46aae3891e723b786132da925b9b6362db4486c07044d99e8", size = 160135373, upload-time = "2026-05-25T06:13:03.535Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/ae/a8/7413d33218aff55a14ec9d20532b49243ffd0579e7a92244922c1885444e/pylibseekdb-1.4.0-cp311-cp311-macosx_15_0_arm64.whl", hash = "sha256:5cb2efab9f1321cdb4b034d3a2bd92e41a402fc95e7dc9579c7473a426f96e24", size = 52173499, upload-time = "2026-08-27T13:05:09.347Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/64/93/e9a13b996b5561f89c9a4f1b62796f8a6230a5dce215869e3cfef8adc4f1/pylibseekdb-1.4.0-cp311-cp311-manylinux_2_28_aarch64.whl", hash = "sha256:e37b931417b7fc7fc88d15fd8b9b0dad499cd05e693cc483aa3743840e85f0c0", size = 49442143, upload-time = "2026-08-27T13:03:41.823Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/71/cd/e54bb304512042cac0514fd607175f5425bd0924330e3eb74937c2afe827/pylibseekdb-1.4.0-cp311-cp311-manylinux_2_28_x86_64.whl", hash = "sha256:6aaa3c9e4865d32f533af04eb8eab06d2c10fc581ac38097d618efb44c05dc5b", size = 53975703, upload-time = "2026-08-27T13:04:41.008Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/ca/03/4380094699cbd4539971c0b943701f776408c94c28cc3ecdaed7c217bb29/pylibseekdb-1.4.0-cp312-abi3-macosx_15_0_arm64.whl", hash = "sha256:2fee55af299f2992dd5d61c9e239ef8855629f4117ee8b4c21dc877160707004", size = 52171602, upload-time = "2026-08-27T13:05:15.263Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/df/b5/ea71acbee58925a51cb1a7137afd2d1c4e3fbb5bf2a0144cd53d299a20df/pylibseekdb-1.4.0-cp312-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:f606579904a19bcd7ec96bc117251db9d4202485fc7355f2a289804d1b3b2c1b", size = 49438937, upload-time = "2026-08-27T13:03:48.418Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/5c/f3/452485e45676a7d738720a7a3f7abbf4d24a3d272cbacabef41ae8b8e52c/pylibseekdb-1.4.0-cp312-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:2d32d6f0d3b92b0c719b6c4230a24748ce10666f67052c696359e69138d8fbe7", size = 53972507, upload-time = "2026-08-27T13:04:46.946Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -4621,7 +4620,7 @@ wheels = [
|
||||
|
||||
[[package]]
|
||||
name = "pyseekdb"
|
||||
version = "1.1.0.post3"
|
||||
version = "1.4.0.post1"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "httpx", marker = "python_full_version < '3.14'" },
|
||||
@@ -4635,7 +4634,7 @@ dependencies = [
|
||||
{ name = "tqdm", marker = "python_full_version < '3.14'" },
|
||||
]
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/58/6e/2373239ab80c35a17aa14e8219727f06567e91d3b7f1b8c36d28ce94d04b/pyseekdb-1.1.0.post3-py3-none-any.whl", hash = "sha256:0437c9a4de72be44eb24b070b2b8099086467c08af10a57191498a61257a4bfb", size = 110985, upload-time = "2026-02-12T14:19:05.402Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/22/87/d5dd862faa3d4adf3847c1ce19c3ea5ecd0dcfda9c2584a95bfd2b0fac0f/pyseekdb-1.4.0.post1-py3-none-any.whl", hash = "sha256:a3379f6962a0c01aa029d3e5a8f0c0f5a59b27a689b8aae1931d9ce5563f252c", size = 158375, upload-time = "2026-08-03T08:56:59.501Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -5256,10 +5255,10 @@ name = "scikit-learn"
|
||||
version = "1.8.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "joblib" },
|
||||
{ name = "numpy" },
|
||||
{ name = "scipy" },
|
||||
{ name = "threadpoolctl" },
|
||||
{ name = "joblib", marker = "python_full_version >= '3.14'" },
|
||||
{ name = "numpy", marker = "python_full_version >= '3.14'" },
|
||||
{ name = "scipy", marker = "python_full_version >= '3.14'" },
|
||||
{ name = "threadpoolctl", marker = "python_full_version >= '3.14'" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/0e/d4/40988bf3b8e34feec1d0e6a051446b1f66225f8529b9309becaeef62b6c4/scikit_learn-1.8.0.tar.gz", hash = "sha256:9bccbb3b40e3de10351f8f5068e105d0f4083b1a65fa07b6634fbc401a6287fd", size = 7335585, upload-time = "2025-12-10T07:08:53.618Z" }
|
||||
wheels = [
|
||||
@@ -5306,7 +5305,7 @@ name = "scipy"
|
||||
version = "1.17.1"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "numpy" },
|
||||
{ name = "numpy", marker = "python_full_version >= '3.14'" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/7a/97/5a3609c4f8d58b039179648e62dd220f89864f56f7357f5d4f45c29eb2cc/scipy-1.17.1.tar.gz", hash = "sha256:95d8e012d8cb8816c226aef832200b1d45109ed4464303e997c5b13122b297c0", size = 30573822, upload-time = "2026-02-23T00:26:24.851Z" }
|
||||
wheels = [
|
||||
@@ -5377,14 +5376,14 @@ name = "sentence-transformers"
|
||||
version = "5.2.3"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "huggingface-hub" },
|
||||
{ name = "numpy" },
|
||||
{ name = "scikit-learn" },
|
||||
{ name = "scipy" },
|
||||
{ name = "torch" },
|
||||
{ name = "tqdm" },
|
||||
{ name = "transformers" },
|
||||
{ name = "typing-extensions" },
|
||||
{ name = "huggingface-hub", marker = "python_full_version >= '3.14'" },
|
||||
{ name = "numpy", marker = "python_full_version >= '3.14'" },
|
||||
{ name = "scikit-learn", marker = "python_full_version >= '3.14'" },
|
||||
{ name = "scipy", marker = "python_full_version >= '3.14'" },
|
||||
{ name = "torch", marker = "python_full_version >= '3.14'" },
|
||||
{ name = "tqdm", marker = "python_full_version >= '3.14'" },
|
||||
{ name = "transformers", marker = "python_full_version >= '3.14'" },
|
||||
{ name = "typing-extensions", marker = "python_full_version >= '3.14'" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/5b/30/21664028fc0776eb1ca024879480bbbab36f02923a8ff9e4cae5a150fa35/sentence_transformers-5.2.3.tar.gz", hash = "sha256:3cd3044e1f3fe859b6a1b66336aac502eaae5d3dd7d5c8fc237f37fbf58137c7", size = 381623, upload-time = "2026-02-17T14:05:20.238Z" }
|
||||
wheels = [
|
||||
@@ -5757,21 +5756,21 @@ name = "torch"
|
||||
version = "2.12.1"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "cuda-bindings", marker = "sys_platform == 'linux'" },
|
||||
{ name = "cuda-toolkit", extra = ["cudart", "cufft", "cufile", "cupti", "curand", "cusolver", "cusparse", "nvjitlink", "nvrtc", "nvtx"], marker = "sys_platform == 'linux'" },
|
||||
{ name = "filelock" },
|
||||
{ name = "fsspec" },
|
||||
{ name = "jinja2" },
|
||||
{ name = "networkx" },
|
||||
{ name = "nvidia-cublas", marker = "sys_platform == 'linux'" },
|
||||
{ name = "nvidia-cudnn-cu13", marker = "sys_platform == 'linux'" },
|
||||
{ name = "nvidia-cusparselt-cu13", marker = "sys_platform == 'linux'" },
|
||||
{ name = "nvidia-nccl-cu13", marker = "sys_platform == 'linux'" },
|
||||
{ name = "nvidia-nvshmem-cu13", marker = "sys_platform == 'linux'" },
|
||||
{ name = "setuptools" },
|
||||
{ name = "sympy" },
|
||||
{ name = "triton", marker = "sys_platform == 'linux'" },
|
||||
{ name = "typing-extensions" },
|
||||
{ name = "cuda-bindings", marker = "python_full_version >= '3.14' and sys_platform == 'linux'" },
|
||||
{ name = "cuda-toolkit", extra = ["cudart", "cufft", "cufile", "cupti", "curand", "cusolver", "cusparse", "nvjitlink", "nvrtc", "nvtx"], marker = "python_full_version >= '3.14' and sys_platform == 'linux'" },
|
||||
{ name = "filelock", marker = "python_full_version >= '3.14'" },
|
||||
{ name = "fsspec", marker = "python_full_version >= '3.14'" },
|
||||
{ name = "jinja2", marker = "python_full_version >= '3.14'" },
|
||||
{ name = "networkx", marker = "python_full_version >= '3.14'" },
|
||||
{ name = "nvidia-cublas", marker = "python_full_version >= '3.14' and sys_platform == 'linux'" },
|
||||
{ name = "nvidia-cudnn-cu13", marker = "python_full_version >= '3.14' and sys_platform == 'linux'" },
|
||||
{ name = "nvidia-cusparselt-cu13", marker = "python_full_version >= '3.14' and sys_platform == 'linux'" },
|
||||
{ name = "nvidia-nccl-cu13", marker = "python_full_version >= '3.14' and sys_platform == 'linux'" },
|
||||
{ name = "nvidia-nvshmem-cu13", marker = "python_full_version >= '3.14' and sys_platform == 'linux'" },
|
||||
{ name = "setuptools", marker = "python_full_version >= '3.14'" },
|
||||
{ name = "sympy", marker = "python_full_version >= '3.14'" },
|
||||
{ name = "triton", marker = "python_full_version >= '3.14' and sys_platform == 'linux'" },
|
||||
{ name = "typing-extensions", marker = "python_full_version >= '3.14'" },
|
||||
]
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/59/38/7028d3be540f1dcdf41660a2b01d0c51d2cb73915fe370d84e4d277a6d47/torch-2.12.1-cp311-cp311-macosx_14_0_arm64.whl", hash = "sha256:ef81f503912effea2ce3d9b12a2e3a6ed488943e91271c90c7a829f60baf6aa2", size = 87975425, upload-time = "2026-06-17T21:08:34.094Z" },
|
||||
@@ -5813,15 +5812,15 @@ name = "transformers"
|
||||
version = "5.3.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "huggingface-hub" },
|
||||
{ name = "numpy" },
|
||||
{ name = "packaging" },
|
||||
{ name = "pyyaml" },
|
||||
{ name = "regex" },
|
||||
{ name = "safetensors" },
|
||||
{ name = "tokenizers" },
|
||||
{ name = "tqdm" },
|
||||
{ name = "typer" },
|
||||
{ name = "huggingface-hub", marker = "python_full_version >= '3.14'" },
|
||||
{ name = "numpy", marker = "python_full_version >= '3.14'" },
|
||||
{ name = "packaging", marker = "python_full_version >= '3.14'" },
|
||||
{ name = "pyyaml", marker = "python_full_version >= '3.14'" },
|
||||
{ name = "regex", marker = "python_full_version >= '3.14'" },
|
||||
{ name = "safetensors", marker = "python_full_version >= '3.14'" },
|
||||
{ name = "tokenizers", marker = "python_full_version >= '3.14'" },
|
||||
{ name = "tqdm", marker = "python_full_version >= '3.14'" },
|
||||
{ name = "typer", marker = "python_full_version >= '3.14'" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/fc/1a/70e830d53ecc96ce69cfa8de38f163712d2b43ac52fbd743f39f56025c31/transformers-5.3.0.tar.gz", hash = "sha256:009555b364029da9e2946d41f1c5de9f15e6b1df46b189b7293f33a161b9c557", size = 8830831, upload-time = "2026-03-04T17:41:46.119Z" }
|
||||
wheels = [
|
||||
@@ -6082,9 +6081,9 @@ name = "valkey-glide"
|
||||
version = "2.4.1"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "anyio" },
|
||||
{ name = "protobuf" },
|
||||
{ name = "sniffio" },
|
||||
{ name = "anyio", marker = "sys_platform != 'win32'" },
|
||||
{ name = "protobuf", marker = "sys_platform != 'win32'" },
|
||||
{ name = "sniffio", marker = "sys_platform != 'win32'" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/72/a2/582b34c6acc8dc857c537f6007459cba48dfa0dc404789a657e5c1a998c0/valkey_glide-2.4.1.tar.gz", hash = "sha256:f1155d84156d11b90488aa67e90102f0bf98a45314f5b99308ac9074c05f7241", size = 898030, upload-time = "2026-05-28T21:41:55.881Z" }
|
||||
wheels = [
|
||||
|
||||
Reference in New Issue
Block a user