Compare commits

...

126 Commits

Author SHA1 Message Date
RockChinQ
98ccbf0f99 refactor: extract RoutingRulesEditor component, revert log levels to debug
- Extract ~250 lines of inline routing rules UI from BotForm into
  a dedicated RoutingRulesEditor component
- Revert stage interrupt and event prevented-default log levels
  from warning back to debug (these are normal flow, not errors)
- Remove message content from log lines to avoid leaking user data
2026-04-02 22:19:28 +08:00
Typer_Body
eb633f8849 fix: format BotForm.tsx with prettier 2026-04-02 01:38:21 +08:00
Typer_Body
ac337b31df feat: pipeline routing fix - add routed_by_rule bypass and diagnostic logging
- Skip GroupRespondRuleCheckStage when message is routed by rule
- Add WARNING logs when queries are silently dropped
- Add pipeline routing rules support (bot entity, migration, web UI)
- Pass routed_by_rule flag through aggregator -> pool -> query variables
2026-04-02 01:33:17 +08:00
Typer_Body
c3e2d5e055 Merge remote-tracking branch 'origin/master' into temp-update
# Conflicts:
#	web/pnpm-lock.yaml
2026-04-02 01:18:38 +08:00
Junyan Qin
f8aedd02b3 fix: update version to 4.9.5 and langbot-plugin to 0.3.6 in project files 2026-03-31 09:30:09 +08:00
Junyan Qin
ea638cab80 feat: add help links for message platform adapters in YAML and update documentation retrieval logic 2026-03-31 00:29:24 +08:00
Junyan Qin
7129dd536e style(web): change adapter doc button to link style with external link icon 2026-03-31 00:08:37 +08:00
Junyan Qin
1b1cc7769b style(web): move adapter doc link to icon button beside selector with tooltip 2026-03-31 00:06:15 +08:00
Junyan Qin
44b8354dfd fix(deps): update langbot-plugin version to 0.3.6 2026-03-30 23:59:55 +08:00
Junyan Qin
55ec9d11ae fix(web): add missing feedback i18n translations for zh-Hant, ja-JP, th-TH, vi-VN, es-ES 2026-03-30 23:56:40 +08:00
Junyan Qin
5b3d3801b5 refactor: clean up Dockerfile and .gitignore by removing unused entries 2026-03-30 23:46:12 +08:00
Typer_Body
9f1ea75d09 Update API base URL to localhost 2026-03-30 23:34:34 +08:00
6mvp6
6e37aae636 feat(wecom): add user feedback support for WeChat Work AI Bot (#2078)
* feat(wecom): add user feedback support for WeChat Work AI Bot

This commit implements user feedback functionality (like/dislike) for
WeChat Work AI Bot conversations, including:

Backend changes:
- Add feedback_id and stream_id fields to WecomBotEvent
- Implement feedback event handling in WecomBotClient (api.py)
- Add StreamSessionManager._feedback_index for feedback_id lookup
- Add on_feedback decorator for custom feedback handlers
- Create MonitoringFeedback entity for database persistence
- Add dbm025 migration for monitoring_feedback table
- Implement FeedbackMonitor helper class
- Update all platform adapters with ap parameter support
- Update botmgr to pass bot_info for monitoring context

Frontend changes:
- Add FeedbackCard and FeedbackList components
- Add useFeedbackData hook for feedback data fetching
- Add feedback tab to monitoring page
- Add feedback types and interfaces
- Add i18n translations (zh-Hans, en-US)

Other changes:
- Update Dockerfile with Chinese mirror for faster builds
- Update docker-compose.yaml with network configuration
- Update .gitignore for docker data and backup files

Note: Known issues that need future improvement:
- feedback_type=3 (cancel) is recorded but not properly handled
- Duplicate feedback records are not deduplicated

* chore: remove unnecessary migration for new table will be created automatically

* chore: ruff format

* chore: prettier

* feat: add feedback handling support across multiple platform adapters

* fix(web): remove unused imports and variables in monitoring module

---------

Co-authored-by: 6mvp6 <13727783693@163.com>
Co-authored-by: Junyan Qin <rockchinq@gmail.com>
2026-03-30 20:23:52 +08:00
RockChinQ
921d12f596 feat: add adapter documentation link button
Add 'View Docs' button that links to the corresponding adapter's
documentation page via link.langbot.app short links.

Appears in:
- Wizard adapter selection cards (Step 0)
- Wizard bot config card header (Step 1)
- Bot create/edit form (adapter config section)

Supports all 7 languages (en/zh-Hans/zh-Hant/ja/th/vi/es).
Doc links auto-resolve to the correct language based on UI locale.
2026-03-30 16:06:54 +08:00
RockChinQ
6bf6deaefd style: fix prettier formatting in i18n locale files 2026-03-30 10:55:20 +08:00
RockChinQ
1201949f2c refactor: replace docs.langbot.app URLs with link.langbot.app short links
All documentation URLs now go through Cloudflare Bulk Redirects
(link.langbot.app) so future doc path changes won't break
already-released versions.

Short link format: link.langbot.app/{lang}/docs/{topic}
Supported languages: zh, en, ja
2026-03-30 10:53:21 +08:00
Typer_Body
1c419e3591 Optimize the plugin system (#2090)
* Optimize the plugin system

* feat: enhance plugin installation process and improve task management

* fix: linter err

---------

Co-authored-by: Junyan Qin <rockchinq@gmail.com>
2026-03-29 23:58:34 +08:00
Junyan Qin
723c57d751 fix: linter err 2026-03-29 23:57:48 +08:00
Junyan Qin
0a69875c09 feat: enhance plugin installation process and improve task management 2026-03-29 23:55:36 +08:00
Typer_Body
f41d69324c Optimize the plugin system 2026-03-29 16:45:54 +08:00
Junyan Qin
b0a9be77b0 feat(web): move Quick Start to account menu and update i18n references 2026-03-29 00:49:02 +08:00
Junyan Qin
e02ade5a30 feat: add preset selection options and update translations for select preset 2026-03-29 00:32:26 +08:00
Junyan Qin
1a51ba8e7e fix(market): add request plugin CTA to empty search results 2026-03-28 22:16:23 +08:00
Junyan Qin
e7b22d6ebf fix: i18n issues 2026-03-28 20:55:43 +08:00
Junyan Qin
dddfa8ac79 chore: add more language supports 2026-03-28 20:48:36 +08:00
Junyan Qin
99e2976826 feat(i18n): add zh_Hant and ja_JP translations to all adapter YAML files
- Add zh_Hant (Traditional Chinese) to all 17 adapter YAML metadata and config fields
- Add ja_JP translations to global adapters (Telegram, Discord, Slack, Lark, LINE)
- Fix buggy zh_Hant in line.yaml and slack.yaml (contained simplified Chinese)
- Add zh_Hant field to backend I18nString model
- Add adapter category grouping with locale-aware ordering
- Add webhook Cloud CTA for community edition users
- Fix wizard progress not clearing on skip/complete
2026-03-28 19:41:27 +08:00
Junyan Chin
71e44f0e54 Feat/space cta optimization (#2089)
* feat(wizard): persist wizard progress to backend for session resumption

Store wizard step, selected adapter, created bot UUID, and runner
selection in the metadata table. On revisit, the wizard restores
progress and verifies the bot still exists. Progress is cleared
automatically when the wizard is completed or skipped.

* feat(dynamic-form): optimize LLM model selection with space login CTA and improve localization strings

* feat(web): add LangBot Cloud CTA for webhook URL fields in community edition

Show a subtle hint below webhook URL fields prompting users about
LangBot Cloud's public endpoint, only visible in community edition.
Covers all 8 webhook-based adapters with i18n support (4 locales).
2026-03-28 17:24:39 +08:00
Junyan Chin
4c904c2375 Fix/frontend optimizations (#2088)
* fix(web): auto-redirect to wizard on first visit and change sidebar icons to blue

* refactor(wizard): use backend metadata table instead of localStorage for wizard completion state

- Add wizard_completed field to system info API (read from metadata table)
- Add POST /api/v1/system/wizard/completed endpoint to mark wizard done
- Frontend home layout checks systemInfo.wizard_completed for auto-redirect
- Wizard calls markWizardCompleted API on skip/finish
- Ensures consistent behavior across all browsers on the same instance

* fix(wizard): update systemInfo in memory before navigation to prevent redirect loop

* fix(monitoring): prevent horizontal overflow and unify empty state styles

* fix(wizard): use Object.assign for systemInfo and await wizard completion API

- Replace systemInfo reassignment with Object.assign in all 3 locations
  to preserve object identity across module imports
- Await markWizardCompleted() POST in wizard skip/finish handlers
  instead of fire-and-forget to ensure backend persistence
- Always re-fetch systemInfo in home layout to get latest
  wizard_completed state from backend

* fix(wizard): prevent redirect loop by blocking navigation on failed status save

- Refactor wizard_completed (boolean) to wizard_status (string: none/skipped/completed)
- Remove ALL localStorage usage from wizard page (form state persistence)
- Replace AlertDialogAction with Button so skip dialog stays open during POST
- Add loading spinners for skip and complete actions
- If POST fails, show error toast and keep dialog/button active for retry
- If POST succeeds, update in-memory state and navigate

* fix(wizard): fix row[0].value bug causing GET /info to always return wizard_status=none

conn.execute(select(Entity)) returns Row with raw column values, not ORM
entities. row[0] is the key column (a string), so row[0].value raises
AttributeError which was silently swallowed by except-pass, making the
GET endpoint always return wizard_status=none regardless of DB state.

* fix(wizard): replace AlertDialog with Dialog for skip confirmation to remove slide animation

* chore: optimize toast in wizard

* fix(wizard): set default token value for Telegram adapter and initialize adapter config in wizard

* feat(web): move webhook URL to dynamic form system, add market category filter, fix layout overflow

- Add 'webhook-url' dynamic form field type rendered as read-only input
  with copy button, defined in adapter YAML specs instead of hardcoded
  in BotForm. Supports show_if conditions for optional-webhook adapters.
- Remove hardcoded webhook display logic from BotForm.tsx, pass webhook
  URLs via systemContext to DynamicFormComponent.
- Fetch webhook URLs after bot creation in wizard and pass to Step 1.
- Support ?category= query param on /home/market page for filtering by
  component type (mirrors langbot-space behavior).
- Link 'install knowledge engine' hint to /home/market?category=KnowledgeEngine.
- Fix SidebarInset missing min-w-0 causing content overflow when sidebar
  is expanded.
- Add vertical divider between plugin detail config and readme panels.
- Fix infinite re-render loop in DynamicFormComponent by memoizing
  editableItems array.

* fix: lint

* fix(web): change systemInfo to const to satisfy prefer-const lint rule

* fix: update adapter descriptions for clarity and usage requirements
2026-03-28 15:50:32 +08:00
fdc310
498d030da9 Fix/weconbot image and file (#2085)
* fix:wecombot file and image

* fix: add enable-stream-reply config
2026-03-28 01:24:54 +08:00
Junyan Chin
c111bf1714 Feat/onboarding wizard (#2086)
* feat(web): add onboarding wizard for guided bot creation

Implement a full-screen 4-step wizard at /wizard that guides users
through selecting a platform, configuring a bot, choosing an AI engine,
and completing setup. The wizard uses DynamicFormComponent for adapter
and pipeline configuration, embeds BotLogListComponent for real-time
debugging, persists state to localStorage, and integrates with Space
OAuth flow. Also fixes a prompt-editor crash in DynamicFormComponent
when value is undefined.

* feat(wizard): redesign step 0/1 flow, add skip dialog, auto-expand log images

- Step 0: Remove bot name/description fields; auto-derive name from adapter
  label; create disabled bot on confirm; advance to Step 1 automatically
- Step 1: Replace 'Create Bot' with 'Save & Enable Bot'; update adapter
  config and enable bot; disable form fields after saving
- Add skip confirmation AlertDialog with i18n message
- Add LanguageSelector to wizard header
- Move wizard sidebar entry to last position to prevent fallback redirect loop
- Add defaultExpanded prop to BotLogCard; auto-expand entries with images
  in wizard via autoExpandImages prop on BotLogListComponent
- Remove automatic default pipeline creation (write_default_pipeline) from
  backend persistence manager since the wizard now handles pipeline creation
- Update all 4 locale files (en-US, zh-Hans, zh-Hant, ja-JP)

* fix(wizard): hide detailed logs link in wizard, allow re-editing bot config after save

- Add hideDetailedLogsLink prop to BotLogListComponent; pass it in wizard
- Remove isEditing on DynamicFormComponent so form stays editable after save
- Always show save button; label changes to 'Re-save' after first save
- Add resaveBot i18n key to all 4 locale files

* style(wizard): move save button into config card header

* fix(wizard): initialize userInfo/systemInfo so model selector works

The wizard runs outside /home layout, so userInfo was null. This caused
the model-fallback-selector to filter out all Space models, showing an
empty dropdown. Fix by calling initializeUserInfo() and
initializeSystemInfo() before fetching wizard data.

Also:
- Hide log toolbar in wizard via hideToolbar prop on BotLogListComponent
- Add empty state message for bot logs (noLogs i18n key, all 4 locales)

* feat(wizard): redesign AI Engine step with left-right split layout

Before selecting a runner: centered grid of runner cards.
After selecting: left panel shows compact runner list for switching,
right panel shows runner config form with slide-in animations.

Also fix prompt field default: add default value to prompt-editor field
in ai.yaml metadata so the prompt is pre-populated with
'You are a helpful assistant.' instead of being empty.

* feat(pipeline): add default values to ai.yaml runner configs and show_if for n8n auth fields

- Sync default values from default-pipeline-config.json to all runner
  config fields in ai.yaml so wizard forms are pre-populated
- Add show_if conditions to n8n-service-api auth fields so only the
  relevant credentials appear based on selected auth-type
- Fix prompt-editor crash in DynamicFormItemComponent when field.value
  is undefined (Array.isArray guard + fallback)
- Improve wizard Step 2 split layout with fixed column widths,
  independent scroll, ring clipping fix, and mobile responsiveness
- Use key={selected} on DynamicFormComponent to force remount on
  runner switch
- Improve pipeline creation flow: create → fetch defaults → merge AI
  section → update (preserves trigger/safety/output defaults)

* feat(dynamic-form): add systemContext prop with __system.* namespace for show_if conditions

- Add systemContext prop to DynamicFormComponent for injecting external
  variables accessible via __system.* prefix in show_if conditions
- Extract resolveShowIfValue() helper for cleaner field resolution
- Pass { is_wizard: true } from wizard to hide knowledge-bases field
- Remove bot config save toast in wizard (keep inline indicator)

* feat(sidebar): render wizard as standalone item before Home group with fallback redirect fix

* fix(wizard): remove unused setBotDescription to fix lint error
2026-03-28 00:46:22 +08:00
Junyan Qin
6570f276d2 feat(web): add plugin install dropdown to sidebar with context-based action dispatch
Add '+' dropdown menu to plugins sidebar category with three install
options: marketplace, upload local, and install from GitHub. Use shared
React context (pendingPluginInstallAction) instead of URL params to
reliably trigger install actions across components. Add e.stopPropagation
on all DropdownMenuItem handlers to prevent React portal event bubbling
from triggering parent SidebarMenuButton navigation.
2026-03-27 20:39:26 +08:00
Junyan Qin
42e1e038bd feat(web): add test functionality to MCPForm and integrate with MCPDetailContent 2026-03-27 20:09:15 +08:00
Junyan Qin
d0e54a45c7 fix(web): show correct MCP server runtime status in sidebar dots
Use runtime_info.status from the API instead of only checking the enable
flag. Dots now show: green (connected), yellow (connecting), red (error),
gray (disabled or no status).
2026-03-27 20:02:16 +08:00
Junyan Qin
23fa47b07e feat(web): refactor MCP servers as sidebar entities and improve sidebar footer
- Refactor MCP servers to be managed as collapsible sidebar sub-items with
  ?id= detail routing and inline form (matching bots/pipelines pattern)
- Add MCPDetailContent with create/edit modes, enable toggle, and danger zone
- Extract MCPForm as standalone inline form from MCPFormDialog
- Move API Integration to standalone sidebar footer button
- Add GitHub star CTA with live star count badge in user dropdown menu
- Add MCP server status dot indicators in sidebar (green/gray for enabled/disabled)
- Add i18n keys for MCP detail page and GitHub star CTA in all 4 locales
2026-03-27 19:59:34 +08:00
Junyan Qin
4902c1d3b2 fix(web): only show ws connection status on active debug tab 2026-03-27 19:16:27 +08:00
Junyan Qin
a6f96e5209 fix(web): improve mobile responsiveness for marketplace, plugin detail, session monitor, and pipeline form 2026-03-27 19:02:24 +08:00
Junyan Qin
37c41bcfe4 feat(web): add popover flyout for collapsed sidebar entity categories 2026-03-27 18:53:17 +08:00
Junyan Qin
9e223949a7 fix(web): refresh sidebar and navigate away after pipeline deletion
The onDeletePipeline callback was a no-op, causing the sidebar to
remain stale and the content area to stay on the deleted pipeline.
Now calls refreshPipelines() and navigates to /home/pipelines,
consistent with bot and knowledge base deletion behavior.
2026-03-27 18:28:34 +08:00
Junyan Qin
267bd72c63 fix(web): resolve zodResolver type mismatch for optional description fields
Remove .default('') from zod schemas to align input/output types,
preventing type conflict between zodResolver and useForm in
@hookform/resolvers v5. Use nullish coalescing at entity assignment
sites to ensure string type safety.
2026-03-27 18:10:30 +08:00
Junyan Qin
af0d00e5e9 refactor(web): make description optional and remove default values for bot, pipeline, and knowledge base
- Remove .min(1) validation on description field, replace with .optional().default('')
- Remove pre-filled default description text from all three create forms
- Remove required asterisk (*) marker from description labels
- No backend changes needed: Bot/Pipeline DB accepts empty string, KB DB allows null
2026-03-27 18:00:48 +08:00
Junyan Qin
244e16c491 perf: ui 2026-03-27 17:22:24 +08:00
Junyan Qin
cad259fe39 refactor(web): simplify sidebar visual design
- Remove vertical guide lines from collapsible sub-items (border-l)
- Move create button from list bottom to category header row as a hover-revealed + icon
- Remove active background highlight from category headers; only child entities show active state
- Remove unused CREATE_I18N_KEYS constant
2026-03-27 15:00:17 +08:00
Junyan Qin
bc3199bf29 feat(web): add icons/emoji to selectors, sync bot enable status and plugin list in sidebar
- Bot adapter selector: show adapter icon in trigger and dropdown items
- Knowledge engine selector: show plugin icon derived from plugin_id
- Pipeline binding selector: show pipeline emoji in trigger and dropdown items
- Knowledge base selectors (single/multi): show KB emoji in all views
- Sidebar bot entries: show green/gray status dot on adapter icon for enable/disable state
- Sidebar plugin list: sync after install/uninstall from all entry points (PluginInstalledComponent, plugins page, marketplace page)
- Pipeline form: add cursor-pointer to left-side tab list buttons
- Clean up unused onBotDeleted prop from BotForm
2026-03-27 14:51:15 +08:00
Junyan Qin
127dc455c3 refactor(web): redesign bot config page with card-based layout and dirty-aware save button
- Restructure bot edit page from flat form to card-based layout (Basic Info, Pipeline Binding, Adapter Config, Danger Zone)
- Move enable switch and save button to sticky header for quick access
- Move webhook URL display into adapter config card (contextually related)
- Remove redundant adapter icon card; show description as FormDescription
- Add dedicated Danger Zone card with red border for delete action
- Remove duplicate delete dialog from BotForm (single source in BotDetailContent)
- Implement form dirty tracking: save button is disabled until user modifies content
- Add i18n keys for new card titles/descriptions across all 4 locales
2026-03-27 12:29:18 +08:00
Junyan Qin
e8dc6fde53 feat: autoclean monitoring events 2026-03-27 11:57:24 +08:00
Junyan Chin
4a97895dea Feat/shadcn sidebar and page views (#2084)
* feat(web): migrate sidebar to shadcn and convert entity editors to page views

* feat(web): enhance sidebar with sections, collapsible persistence, sub-item sorting/limiting, and UI polish

- Reorganize sidebar into Home and Extensions sections with collapsible groups
- Split plugins page into plugins, market, and mcp as separate routes
- Add sidebar sub-items sorted by updatedAt with max 5 visible and expand/collapse toggle
- Persist collapsible section state and sidebar open state in localStorage
- Fix page refresh stripping query params by splitting handleChildClick/selectChild
- Swap plugin detail layout (config left, readme right)
- Add fixed headers with internal scroll for all detail and list pages
- Remove entity form borders and sidebar rail
- Improve dark mode sidebar/content contrast
- Rename monitoring to Dashboard, move to first position
- Update breadcrumb to show Home or Extensions based on current route
- Add i18n translations for more/less toggle in all 4 locales

* fix(web): fix scroll behavior - constrain layout to viewport, fix fixed headers and independent scroll areas

- Change SidebarProvider wrapper from min-h-svh to h-svh overflow-hidden to constrain layout to viewport height (root cause of all scroll issues)
- Fix create mode pages (bot, pipeline, knowledge): extract title bar out of scroll container so only form content scrolls
- Fix plugin detail: add overflow-x-hidden on both config and readme panels to prevent horizontal overflow
- Add min-h-0 to all TabsContent in edit mode for cross-browser flex shrink safety
- Change nested <main> to <div> in layout to avoid invalid nested <main> tags (SidebarInset already renders as <main>)

* style(web): polish UI - dashboard i18n, sidebar create text, cursor-pointer tabs, remove cancel buttons

* feat(web): add plugin context menu to sidebar sub-items

- Add hover-reveal dropdown menu (Ellipsis icon) on plugin sidebar items
- Menu items: Update (marketplace only), View Source (marketplace/github), Delete
- Add confirmation dialog with async task polling for delete/update operations
- Extend SidebarEntityItem with installSource and installInfo fields
- Fix PipelineFormComponent optional onCancel invocation

* fix(web): prevent plugin sidebar text from overlapping menu button

Add right padding on plugin sub-items and explicit truncate on text
span so long plugin names never overlap the hover menu button.

* feat(web): show update indicator on sidebar plugin menu

- Fetch marketplace plugin versions in SidebarDataContext.refreshPlugins
- Compare with installed version using isNewerVersion to set hasUpdate
- Show red dot on menu trigger when update available (always visible)
- Show 'New' badge on Update menu item when update available
- Marketplace fetch failure is silently caught to avoid blocking sidebar

* refactor(web): remove entity list pages, back buttons, and make sidebar toggle collapse

- Remove card grid list views from bots, pipelines, knowledge pages
- Show empty state placeholder when no entity is selected
- Preserve KB migration dialog at page level
- Remove back (ArrowLeft) buttons from all detail pages (bots, pipelines, knowledge, plugins)
- Sidebar parent click for bots/pipelines/knowledge now toggles collapse instead of navigating
- Breadcrumb second level is now non-clickable (always BreadcrumbPage)
- Add selectFromSidebar i18n keys in all 4 locales

* feat(web): enhance bot session monitor with refresh functionality and improve log card UI

* refactor(web): optimize pipeline detail page with vertical config nav and debug chat polish

- Convert pipeline config tab's horizontal sub-tabs to vertical left-side navigation with icons
- Replace hardcoded colors in PipelineFormComponent and DebugDialog with theme-aware Tailwind classes
- Replace custom SVG icons with lucide-react (User, Users, ImageIcon, Send, Reply, etc.)
- Replace hardcoded Chinese strings with i18n keys (allMembers, file, voice, uploadImage, uploading)
- Modernize chat bubbles to use bg-primary/10 and bg-muted instead of hardcoded blue/gray
- Translate all Chinese comments to English in both components
- Delete unused pipelineFormStyle.module.css
- Remove max-w-2xl constraint from config tab container

* fix(web): improve dark mode contrast and relocate WebSocket status indicator

Bump dark mode --muted, --accent, --secondary from oklch(0.18) to oklch(0.24)
to fix invisible TabsList, message bubbles, and selected items against the
oklch(0.17) background. Move WebSocket connection dot from pipeline title
into the Debug Chat tab trigger so it is always visible. Replace hardcoded
Quote border colors with theme-aware border-muted-foreground/50.

* fix(web): increase dark mode contrast for muted/accent/secondary to oklch(0.27)

Previous value of oklch(0.24) was still not distinguishable enough against
the oklch(0.17) background. Bump to oklch(0.27) for a 0.10 lightness gap,
matching the contrast ratio of the default shadcn zinc dark theme.

* style(web): replace hardcoded colors with theme tokens in monitoring dashboard

Convert all monitoring page components from hardcoded gray/white colors
to theme-aware CSS variable tokens (bg-card, text-foreground,
text-muted-foreground, bg-muted, bg-background, bg-accent, border).
Semantic colors (red/green/blue/purple for status badges and error
styling) are intentionally preserved.

* feat(web): show debug indicator for debugging plugins in sidebar

Add orange Bug icon next to plugin name in sidebar sub-items when the
plugin is connected via WebSocket debug mode. Hide context menu for
debug plugins since delete/update operations are not supported.

* feat(web): show install source and debug badge on plugin detail page

Display a badge next to the plugin title indicating the install source
(GitHub blue, Local green, Marketplace purple) or debugging status
(orange with Bug icon), matching the existing plugin card convention.

* fix(web): resolve eslint errors for CI - remove unused imports and variables

* fix(web): remove stale setSubtitle call and fix prettier formatting

* Refactor code formatting and improve readability

- Updated HomeSidebar.tsx to enhance clarity in conditional assignment.
- Adjusted CSS formatting in github-markdown.css for better alignment.
- Cleaned up tsconfig.json by consolidating array formatting for consistency.

* fix(ci): use local prettier instead of mirrors-prettier to avoid version mismatch (3.1.0 vs 3.8.1)
2026-03-27 01:51:13 +08:00
xiaolou
3c0495fc51 fix: 修复钉钉文件消息解析失效问题(优化 downloadCode 提取逻辑) (#2080)
* fix: resolve dingtalk file parsing issue by extracting downloadCode from content

* style: fix ruff format trailing whitespace

---------

Co-authored-by: RockChinQ <rockchinq@gmail.com>
2026-03-27 00:17:26 +08:00
Junyan Qin
dfd25deb68 feat(web): hide deprecated KnowledgeRetriever plugins from marketplace
KnowledgeRetriever has been superseded by KnowledgeEngine. Filter out
plugins that only contain KnowledgeRetriever components from both the
main plugin list and recommendation lists, and remove the now-unused
deprecated badge UI.
2026-03-26 00:56:24 +08:00
Junyan Qin
f4db53b759 chore: bump version to 4.9.4 in pyproject.toml and __init__.py 2026-03-26 00:16:21 +08:00
Junyan Qin
9f90341dcb fix(web): correct UTC timestamp parsing in monitoring panel
Backend serializes monitoring timestamps as naive ISO strings without
timezone designator. JavaScript's new Date() treats such strings as
local time, causing displayed times to be off by the user's UTC offset.
Add parseUTCTimestamp() utility that appends 'Z' to ensure correct UTC
interpretation.
2026-03-26 00:05:44 +08:00
Junyan Qin
67b726afb2 chore: uv.lock 2026-03-25 23:44:34 +08:00
fdc310
01852b81d4 Feat/openclaw weixin adapter (#2074)
* feat: add wexin openclaw adapter

* feat: The new feature will store the token and other configurations after login.

* fix: wexin qc to base64 and in log image print

* feat: add image to base64

* feat: add update file and image and voice
2026-03-25 23:34:35 +08:00
RockChinQ
4d6f109788 chore: bump langbot-plugin SDK to 0.3.5 2026-03-25 21:10:59 +08:00
Junyan Chin
e1e5e7aedf fix: get_llm_models handler returns UUID strings instead of full model dicts (#2081)
The plugin SDK declares get_llm_models() -> list[str] (UUID strings),
but the host handler returned the full model dict list from
llm_model_service.get_llm_models(). This caused TypeError when
invoke_llm passed a dict to get_model_by_uuid (which is decorated
with @async_lru and requires hashable arguments).

Extract only the 'uuid' field to match the SDK contract.
2026-03-25 21:06:49 +08:00
RockChinQ
cd53abc440 fix(web): prevent plugin market search trigger during IME composition 2026-03-24 21:39:49 +08:00
Junyan Qin
16a15a122a fix: update langbot-plugin dependency to version 0.3.4 2026-03-24 12:00:12 +08:00
zpf2000
6fa653f232 feat: 支持可配置的混合检索融合权重 (#2071)
* feat: 支持可配置的混合检索融合权重

* style: 修复 ruff format 检查
2026-03-24 09:50:08 +08:00
Junyan Chin
c13971d7d6 feat(web): merge plugin readme and config into single detail dialog (#2076)
* feat(web): merge plugin readme and config into single detail dialog

- Click plugin card directly opens combined dialog (left: readme, right: config)
- Remove hover overlay with separate readme/config buttons
- Dropdown menu (⋯) still available for update/delete/view source

* fix: prettier format for lucide import
2026-03-23 22:22:31 +08:00
Junyan Qin
9c659ce8fa fix: update langbot-plugin dependency to version 0.3.4 2026-03-23 22:14:41 +08:00
Junyan Qin
c9fc64360f feat(plugin): add unrestricted knowledge base query handlers
Add handlers for LIST_KNOWLEDGE_BASES and RETRIEVE_KNOWLEDGE actions
that allow plugins to list and retrieve from any knowledge base without
pipeline scope restrictions, complementing the existing pipeline-scoped handlers.
2026-03-23 21:06:23 +08:00
Guanchao Wang
88a04fdbe8 Merge pull request #2055 from langbot-app/copilot/fix-sender-name-parameter 2026-03-23 14:14:36 +08:00
WangCham
bbe019f0c6 fix: wrong agentid 2026-03-23 14:02:10 +08:00
RockChinQ
865f6ee81b style: format telegram.py for ruff 2026-03-21 22:10:23 +08:00
fdc310
bd5ec59b7c fix:The fix is in place — content = '' is now reset at the start of each loop iteration , which prevents stale text from being duplicated across tool call and end-turn chunks. (#2060) 2026-03-21 22:08:35 +08:00
fdc310
9c0cc1003d Fixed the issue where the at bot did not remove the at symbol, result… (#2062)
* Fixed the issue where the at bot did not remove the at symbol, resulting in some commands not being activated in group chats. Also, adjusted the logic in the on_message section.

* fix:reply_message  del bot_name
2026-03-21 22:07:31 +08:00
Bijin
ea07d8ad00 fix(telegram): add document message support (docx/pdf/etc) (#2069)
The Telegram adapter only handles TEXT, COMMAND, PHOTO, and VOICE
messages. Document files (docx, pdf, etc.) sent by users are silently
dropped because:

1. MessageHandler filters lack filters.Document.ALL
2. target2yiri() has no message.document branch
3. yiri2target() has no platform_message.File branch
4. send_message() has no 'document' component handler

Changes:
- Add filters.Document.ALL to the MessageHandler filter set
- Add message.document parsing in target2yiri() → platform_message.File
- Add platform_message.File handling in yiri2target() → document component
- Add 'document' type handling in send_message() via bot.send_document()

This allows Telegram document messages to flow through the existing
PreProcessor and Dify file upload pipeline, consistent with how other
adapters (Lark, KOOK, Discord, WeCom) already handle files.

Closes #2065
2026-03-21 22:06:54 +08:00
youhuanghe
3ac3fad4bc chore: upgrade plugin sdk to 0.3.3 2026-03-19 12:48:29 +00:00
youhuanghe
254a13bba3 fix: 4355f0fa78 ruff lint 2026-03-16 06:39:29 +00:00
youhuanghe
4355f0fa78 feat(rag): expose vector listing API with backend filter support 2026-03-16 06:26:05 +00:00
Junyan Qin
031737f05d chore: remove all preset sensitive words 2026-03-16 13:42:19 +08:00
Nody the lobster
9e366fc536 fix: allow env overrides to create missing config keys (#2064)
Previously, environment variable overrides (e.g. SYSTEM__INSTANCE_ID)
were silently skipped if the target key didn't already exist in
data/config.yaml. This caused SaaS pods running older LangBot images
(whose config template lacked system.instance_id) to ignore the
SYSTEM__INSTANCE_ID env var, falling back to a random UUID that
didn't match the pod UUID — breaking idle timeout tracking.

Now env overrides create missing keys (as strings) and missing
intermediate dicts, so they work regardless of template version.

Co-authored-by: rocksclawbot <rocksclawbot@users.noreply.github.com>
2026-03-15 23:03:40 +08:00
youhuanghe
8bd6442965 chore: upgrade plugin sdk to 0.3.2 2026-03-14 12:56:54 +00:00
Junyan Qin
1a1eadb282 chore: bump version 4.9.3 2026-03-14 20:20:48 +08:00
Nody the lobster
eed72b1c12 fix: show error message on login page when backend is unreachable (#2063) 2026-03-14 19:20:01 +08:00
RockChinQ
351350ea03 fix: instance_id priority: config.yaml > file > generate new
- If system.instance_id set in config (via env var), use it
- If not set but file exists, read from file (don't generate new)
- If neither, generate new and save to file
2026-03-13 11:33:32 -04:00
RockChinQ
bc3d6ba92f feat: support instance_id in system config
Add instance_id field to system section in config.yaml.
Can be set via SYSTEM__INSTANCE_ID env var (auto-mapped).
Falls back to data/labels/instance_id.json if not set.
2026-03-13 11:31:51 -04:00
RockChinQ
345e4baf2a Revert "feat: support pre-setting instance_id via LANGBOT__INSTANCE_ID env var"
This reverts commit 6c64dc057f.
2026-03-13 11:30:36 -04:00
RockChinQ
6c64dc057f feat: support pre-setting instance_id via LANGBOT__INSTANCE_ID env var
In SaaS (cloud edition), the instance_id can now be injected via
environment variable to match the pod UUID. This enables zero-lookup
telemetry routing in Space - no need to reverse-lookup instance_id
to find the pod.
2026-03-13 11:26:16 -04:00
youhuanghe
eec0a9c9d9 feat(plugin): expose KB UUIDs in query variables and pass session context to retrieve API
Extract knowledge base UUID list into query.variables['_knowledge_base_uuids']
in PreProcessor so plugins can modify it during PromptPreProcessing. Runner now
reads from variables instead of pipeline_config. Also pass session_name,
bot_uuid, and sender_id to kb.retrieve() in the RETRIEVE_KNOWLEDGE_BASE handler
so knowledge engines receive proper session context.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 14:23:19 +00:00
Junyan Qin
6896a55485 fix: bot form error 2026-03-13 12:26:45 +08:00
Junyan Qin
4b0fad233e chore: bump version 4.9.2 2026-03-13 12:15:21 +08:00
Junyan Qin
52eb991a70 feat: add extra webhook prefix config 2026-03-13 12:06:22 +08:00
Junyan Qin
10c716be0c fix: bad model field ref 2026-03-13 11:47:31 +08:00
youhuanghe
6e77351eda refactor: up rag ingest timeout 2026-03-13 02:37:32 +00:00
Junyan Qin
20f5ebd9b8 chore: bump version 4.9.1 2026-03-12 23:24:33 +08:00
Junyan Qin
d2c75329cf fix: kbform react error 2026-03-12 23:20:51 +08:00
Junyan Qin
7e2fe082f0 chore: bump langbot-plugin to 0.3.1 2026-03-12 23:16:09 +08:00
fdc310
d451b059fd feat: Implement WebSocket long connection client for WeChat Work AI Bot (#2054)
* feat: Implement WebSocket long connection client for WeChat Work AI Bot

- Added WecomBotWsClient to handle WebSocket connections for receiving messages and sending replies.
- Introduced a new migration (dbm022) to add 'enable-webhook' field to existing wecombot adapter configs, ensuring backward compatibility.
- Updated WecomBotAdapter to support both WebSocket and webhook modes based on the new configuration.
- Enhanced YAML configuration for WecomBot to include 'enable-webhook' and 'Secret' fields, adjusting requirements accordingly.
- Incremented database version to 22 to reflect schema changes.

* fix:db enable-webhook is false

* fix:add logic

* fix:Removed an unnecessary configuration check

* fix: migration

* fix: update migration

* fix:migration
2026-03-12 22:31:14 +08:00
marun
93c52fcd4c Enhance Lark Bot Ability to Reply to Quoted Messages (#2043)
* fix(database): Update database version requirement to 20

- Increase required_database_version from 19 to 20
- Add documentation on database schema version check

* feat(lark): Added support for message references and topic message grouping

- Implemented the function to extract reference message IDs from messages, supporting parent message identification

- Added a method to construct event messages from SDK message items

- Implemented the function to asynchronously obtain reference messages and convert them into message chains

- Integrated reference message injection logic into the message processing flow

- Added a mechanism to filter source components while retaining reference content

- Implemented a method to obtain the starter ID with topic awareness

- Provided session isolation support for topic range in group thread messages

- Supported stable maintenance of conversation context in group thread discussions

- Handled cases where topic messages cannot reliably detect reference targets

* feat(lark): Implement a duplicate prevention mechanism for Feishu topic message references

- Add class-level cache to store processed topic IDs and timestamps

- Implement a timed cleanup mechanism to remove expired topic records

- Add cache size limit to prevent memory from growing indefinitely

- Return the parent message ID and mark it as processed when the first reply is made to a topic

- Return None in subsequent replies to the same topic to avoid duplicate references

- Implement automatic cache trimming to ensure stable performance
2026-03-12 21:48:30 +08:00
huanghuoguoguo
f1608682e6 Feat/agentic rag and parser invoke api (#2052)
* feat: add pipeline api

* feat: add list parser

* ruff lint

* fix: add filter but agentic rag not to use

* feat: add bot uuid for memory..
2026-03-12 21:47:27 +08:00
youhuanghe
077e631c13 fix(rag): normalize vector search to distance semantics 2026-03-12 12:33:09 +00:00
Junyan Chin
d7df1f05d1 fix: resolve security vulnerabilities in dependencies (#2059)
Python (uv.lock):
- langchain-core 1.2.7 → 1.2.18 (SSRF via image_url token counting)
- langgraph 1.0.7 → 1.1.1 (unsafe msgpack deserialization)
- flask 3.1.2 → 3.1.3 (missing Vary: Cookie header)
- werkzeug 3.1.5 → 3.1.6 (Windows special device name in safe_join)

npm (web/pnpm-lock.yaml):
- minimatch updated to fix ReDoS vulnerabilities
2026-03-12 20:09:19 +08:00
Junyan Chin
8b8cfb76de fix(market): sync plugin market UI improvements from Space (#2056)
* fix(market): sync plugin market UI from space - page size 12, full list display, fix double separator, adaptive tag display

* fix: lint and prettier formatting

* fix: prettier formatting for remaining files
2026-03-12 15:06:11 +08:00
Junyan Chin
79311ccde3 feat: model fallback chain (#2017) (#2018) 2026-03-12 03:33:05 +08:00
copilot-swe-agent[bot]
def798bf1f fix: WeCom sender_name shows user ID instead of actual username
- Add get_user_info() to WecomClient to fetch user name via /user/get API
- Update WecomEventConverter.target2yiri to accept bot param and fetch real user name
- Update register_listener call to pass self.bot for user name lookup
- URL-encode userid parameter for safety

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>
2026-03-11 17:52:43 +00:00
copilot-swe-agent[bot]
5290834b8b Initial plan 2026-03-11 17:48:12 +00:00
Guanchao Wang
89064a9d5b feat: add support for username (#2047)
* feat: add support for username

* fix: lint

* fix: migerations

* fix: change to version 21

* fix: remove duplicate dbm021 migration and rename dbm022

* feat: add user_id and user_name display with copy functionality in BotSessionMonitor

---------

Co-authored-by: wangcham <wangcham@gmail.com>
Co-authored-by: Junyan Qin <rockchinq@gmail.com>
2026-03-12 01:27:22 +08:00
RockChinQ
8c2aef3734 fix: prettier formatting for long URL strings 2026-03-11 07:05:45 -04:00
RockChinQ
3fb9e542b6 fix(web): use locale-aware data collection policy URL 2026-03-11 07:03:52 -04:00
RockChinQ
01844d8687 feat(web): add privacy & data collection policy consent to login/register pages 2026-03-11 06:50:54 -04:00
Copilot
2655425fbe fix: deduplicate final chunk yield in Dify chatflow streaming (#2049)
* Initial plan

* fix: prevent duplicate messages when Dify chatflow sends both workflow_finished and message_end events

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>

* style: apply ruff formatting to difysvapi.py

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>
2026-03-11 14:45:55 +08:00
youhuanghe
bd15b630b0 fix: chroma ruff lint 2026-03-11 04:07:21 +00:00
youhuanghe
fe5ce68436 feat(vector): add full-text and hybrid search support for Chroma backend
- Implement full-text search via Chroma's $contains filter
  - Implement hybrid search with RRF (Reciprocal Rank Fusion) combining
    vector and full-text results, with min-max normalized distances
  - Fix add_embeddings to use col.upsert instead of col.add for idempotency
  - Bump chromadb dependency to >=1.0.0,<2.0.0
  - Re-lock uv.lock with official PyPI source
2026-03-11 03:59:14 +00:00
Typer_Body
0541b05966 refactor: optimized error handling (#2020)
* Update output.yaml

* Update default-pipeline-config.json

* Update chat.py

* Add files via upload

* Update chat.py

* Update default-pipeline-config.json

* Update output.yaml

* Update constants.py

* feat: update logic

* fix: update required database version to 21

---------

Co-authored-by: Junyan Qin <rockchinq@gmail.com>
2026-03-10 22:01:23 +08:00
youhuanghe
13cb0aa9be bugfix: rollback filter, add to retrive settings 2026-03-10 12:49:24 +00:00
youhuanghe
a048369b38 feat: Pass session context (session_name) to knowledge engine retrieval filters.
Allow KnowledgeEngine plugins to filter retrieval results by session,enabling per-session memory isolation in plugin-based knowledge bases
2026-03-10 12:27:50 +00:00
Junyan Qin
9ae0c263dc fix: update documentation links and translations for knowledge engine 2026-03-09 20:31:50 +08:00
Junyan Qin
a4e66f6459 feat: update version to 4.9.0 in pyproject.toml, __init__.py, and uv.lock 2026-03-09 20:10:01 +08:00
huanghuoguoguo
2a74a8d6ae Feat/dbm20 rag (#2037)
* feat(rag): add knowledge base migration from v4.9.0 to plugin architecture

Rewrite dbm020 to backup old knowledge_bases data and preserve
external_knowledge_bases table. Add migration API endpoints and
frontend dialog so users can opt-in to auto-install LangRAG plugin
and restore their knowledge bases with original UUIDs preserved.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(rag): query marketplace for actual plugin version instead of 'latest'

The marketplace API does not support 'latest' as a version string.
Fetch the plugin info first to get latest_version, then use that
concrete version for installation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(rag): add data-only migration option and fix dialog width

Add option to migrate knowledge base data without auto-installing
the LangRAG plugin (for offline/intranet environments). Also
narrow the migration dialog to match other confirmation dialogs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: to red and no more

* fix lint

* fix ruff lint

* feat: add external migration

* fix: show

* feat: add external plugin auto download

* feat: update migration messages for knowledge base in multiple languages

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Junyan Qin <rockchinq@gmail.com>
2026-03-09 20:05:38 +08:00
Guanchao Wang
d31f25c8df Merge pull request #2041 from langbot-app/fix/websocket-chat-bug
Fix/websocket chat bug
2026-03-09 16:11:17 +08:00
WangCham
11c05ea8db style(format): fix ruff formatting issues 2026-03-09 16:04:38 +08:00
WangCham
2b8bd1cc71 fix: invoke_llm failed when use plugin 2026-03-09 16:01:45 +08:00
doujianghub
9148e02679 fix: centralized pipeline config type coercion to prevent string-type crashes (#2031)
* fix: coerce pipeline config types at load time using metadata definitions

Pipeline configs stored in SQLAlchemy JSON columns can have values turned
into strings after UI edits (e.g. "120" instead of 120), causing runtime
arithmetic/logic errors. Add centralized type coercion in load_pipeline()
that leverages existing metadata YAML type definitions (integer, number,
float, boolean) to convert values before they reach downstream stages.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address review - defensive getattr + add unit tests for config_coercion

- Use getattr with defaults for pipeline_config_meta_* attributes to
  avoid AttributeError when MockApplication lacks these fields
- Add 18 unit tests for config_coercion module covering all code paths

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add dynamic form stage tracking and snapshot management

* fix: standardize string formatting in config coercion and improve logging messages

---------

Co-authored-by: KPC <kpc@kpc.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Junyan Qin <rockchinq@gmail.com>
2026-03-09 14:30:07 +08:00
fdc310
fd15284d91 fix(platform): websocket send_message not delivering to webchat frontend (#2039)
- Include websocket_proxy_bot in get_bot_by_uuid lookup so plugins can
  find it by uuid
- Rewrite send_message to broadcast directly via ws_connection_manager
  using the correct pipeline_uuid instead of misusing target_id
- Save messages to session history with unique IDs so they persist
  across page reloads and don't overwrite each other

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 13:22:03 +08:00
Junyan Qin
8c7a0ec027 fix: update langbot-plugin version to 0.3.0 2026-03-08 21:08:08 +08:00
youhuanghe
a1cef5c9bf bugfix: update uv.lock 2026-03-08 11:10:03 +00:00
youhuanghe
90438cec36 lint: update web knowledge pnpm lint 2026-03-08 11:05:00 +00:00
youhuanghe
95dd19f4d7 bugfix: now knowledge toast right msg 2026-03-08 11:01:13 +00:00
youhuanghe
c64eb58cf8 feat: update pyseekdb version to 1.1.0.post3 2026-03-08 10:42:20 +00:00
Junyan Qin
fbd3d7ae3a feat: enhance RecommendationLists component with responsive pagination and auto-advance functionality
- Added dynamic column measurement to adjust the number of visible plugins based on the grid layout.
- Implemented auto-advance feature for pagination every 5 seconds when there are more plugins than the visible count.
- Updated pagination controls to reflect the current page accurately.
- Refactored code to improve readability and maintainability.
2026-03-08 17:35:30 +08:00
youhuanghe
40c7b0f731 fix(web): display document_name instead of file_id in retrieval results
The getTitle fallback order was reversed, always showing the UUID
(file_id) since it's always truthy. Swap priority to document_name
first.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 04:24:41 +00:00
huanghuoguoguo
cadcf10047 Feat/rag plugin (#1995)
* [issue:1933] RAG engine plugin architecture (#1967)

* refactor: migrate RAG knowledge services to a plugin-oriented host service architecture.

* feat(rag): phase 2 core refactor with RPC Action handlers

* feat: 为 RAG 插件添加知识库创建和删除事件通知,并优化了 RAG 动作的参数传递和枚举使用。

* feat: 统一知识库管理为RAG引擎,支持动态配置并移除旧的外部知识库组件。

* refactor(rag): remove plugin_adapter, inline logic into RuntimeKnowledgeBase

BREAKING CHANGE: RAGPluginAdapter has been removed. All plugin
communication is now handled directly by RuntimeKnowledgeBase.

Architecture change:
- Before: RuntimeKnowledgeBase → RAGPluginAdapter → plugin_connector
- After:  RuntimeKnowledgeBase → plugin_connector (direct)

Changes to kbmgr.py (RuntimeKnowledgeBase):
- Remove RAGPluginAdapter import and usage
- Inline plugin communication methods:
  - _on_kb_create(): Notify plugin when KB is created
  - _on_kb_delete(): Notify plugin when KB is deleted
  - _ingest_document(): Call plugin for document ingestion
  - _retrieve(): Call plugin for retrieval
  - _delete_document(): Call plugin to delete document
- Simplify dispose(): Only notify plugin, no built-in VDB assumption

Changes to base.py (KnowledgeBaseInterface):
- Remove get_type() abstract method (outdated internal/external concept)
- Add get_rag_engine_plugin_id() abstract method

Changes to localagent.py:
- Remove get_type() call
- Simplify top_k retrieval from KB entity

Deleted files:
- pkg/rag/knowledge/plugin_adapter.py

Benefits:
- Reduced abstraction layer, simpler code
- Plugin communication logic centralized in RuntimeKnowledgeBase
- Easier to understand and maintain

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor(api): remove ExternalKnowledgeBase infrastructure

BREAKING CHANGE: ExternalKnowledgeBase has been completely removed.
All knowledge bases are now unified under the single KnowledgeBase model,
differentiated by their rag_engine_plugin_id.

Deleted files:
- pkg/api/http/controller/groups/knowledge/external.py
  (ExternalKBController with /external-bases routes)
- pkg/api/http/service/external_kb.py
  (ExternalKnowledgeBaseService)
- pkg/rag/knowledge/external.py
  (ExternalKnowledgeBase implementation)

Modified files:
- pkg/entity/persistence/rag.py:
  Remove ExternalKnowledgeBase SQLAlchemy table definition
- pkg/core/app.py:
  Remove external_kb_service attribute from LangBotApplication
- pkg/core/stages/build_app.py:
  Remove external_kb_service initialization

Migration notes:
- Existing external knowledge base data should be migrated manually
- API consumers should use /api/v1/knowledge/bases for all KB operations
- Use /api/v1/knowledge/engines to discover available RAG engines

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor(plugin): remove list_knowledge_retrievers from connector

Remove deprecated list_knowledge_retrievers functionality from the
plugin communication layer. This aligns with the SDK change that
removed the LIST_KNOWLEDGE_RETRIEVERS action.

Changes:
- connector.py: Remove list_knowledge_retrievers() method
- handler.py: Remove list_knowledge_retrievers() handler

The functionality is replaced by the new /api/v1/knowledge/engines
endpoint which lists available RAGEngine components with their
capabilities and configuration schemas.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor(service): update knowledge service with capability-based checks

Replace type-based checks with capability-based checks for file
operations, aligning with the unified knowledge base architecture.

Changes to knowledge.py:
- store_file(): Replace get_type() check with doc_ingestion capability check
- delete_file(): Replace get_type() check with doc_ingestion capability check
- list_rag_engines(): Remove list_knowledge_retrievers call, simplify to
  only list RAGEngine components (KnowledgeRetriever type removed)

Changes to pipelines.py:
- Minor cleanup related to knowledge base references

The capability-based approach allows RAG engines to declare their
supported features (doc_ingestion, chunking_config, rerank, hybrid_search)
and the system responds accordingly, rather than hardcoding behavior
based on internal/external type distinction.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(web): unify knowledge base UI, remove external KB components

BREAKING CHANGE: The internal/external knowledge base distinction
has been removed from the frontend. All knowledge bases are now
displayed in a unified list, differentiated by their RAG engine.

Changes to page.tsx:
- Remove Tab component (内置/外置 tabs)
- Remove selectedKbType state
- Unified knowledge base list display
- Single "Create Knowledge Base" button for all types

Changes to KBDetailDialog.tsx:
- Remove kbType prop
- Simplify dialog logic for unified KB handling
- Documents menu item conditionally shown based on doc_ingestion capability

Changes to KBForm.tsx:
- Remove retriever type handling code
- Simplify form for unified KB creation
- Dynamic form rendering based on RAG engine's creation_schema

Changes to KBCardVO.ts:
- Remove 'type' field from KBCardVO interface

Changes to BackendClient.ts:
- Remove all external KB related methods:
  - getExternalKnowledgeBases()
  - getExternalKnowledgeBase()
  - createExternalKnowledgeBase()
  - updateExternalKnowledgeBase()
  - deleteExternalKnowledgeBase()
  - retrieveFromExternalKnowledgeBase()

Changes to api/index.ts:
- Remove ExternalKnowledgeBase interface definition

UI/UX improvements:
- Users no longer need to understand internal vs external distinction
- RAG engine selection is now the primary differentiator
- Documents panel visibility is capability-driven (doc_ingestion)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor(plugin): code review improvements for RAG handlers

- Unify embed_model field naming to embedding_model_uuid only
- Add structured error responses with error_type for RAG actions
- Fix file_size and mime_type detection in _store_file_task
- Improve error handling with detailed error context (error_type, original_error)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor(rag): refactor KB dynamic form and vector manager

- Frontend: Refactor Knowledge Base form using DynamicForm components.
- Frontend: Remove obsolete jsonSchemaConverter utility.
- Backend: Update VectorManager and PluginHandler to support new RAG architecture.
- Chore: Update dependencies in pyproject.toml.

* fix: code review fixes for RAG refactor

- Remove DEBUG stderr outputs in handler.py
- Move repeated `import json` to file top
- Add warning log for unimplemented delete_by_filter

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor(rag): consolidate valid_fields into entity constants

Define MUTABLE_FIELDS, CREATE_FIELDS, ALL_DB_FIELDS as class
constants in KnowledgeBase entity to eliminate duplication.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: 将知识库获取和RAG引擎信息丰富逻辑移至知识库管理器。

* refactor(rag): introduce RAGRuntimeService and clean up plugin handler

- Create RAGRuntimeService to encapsulate RAG capability implementation (Embedding, VectorOps).
- Refactor PluginHandler to delegate RAG actions to RAGRuntimeService.
- Move KnowledgeService enrichment and creation logic to RAGManager.
- Register RAGRuntimeService in Application and BuildAppStage.
- Clean up legacy code in KnowledgeService.

* refactor(rag): standardize logger and fix type hints

- Use self.ap.logger consistently in kbmgr.py and runtime.py, removing module-level loggers.
- Fix type hints for retrieve_knowledge in handler.py and connector.py to match implementation returning dict.

* refactor: 将引擎徽章的样式从 Tailwind CSS 类迁移到 CSS 模块。

* fix(web): resolve React rendering errors in plugins page

- Fix missing key prop in PluginComponentList by using ternary instead of Fragment
- Fix RAGEngine.name type to I18nObject and use extractI18nObject() for rendering
- Preserves multi-language support

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(rag): update runtime service and web components

* refactor: 优化知识库设置结构并增强前端距离显示健壮性。

* fix: 处理前端距离显示中的空值。

* fix(rag): document retrieve ui and kbmgr top_k validation

* 更新 uv.lock 中的 PyPI 镜像源为官方地址。

* fix: address code review issues for RAG engine plugin architecture

P0 fixes:
- Fix ALL_DB_FIELDS missing collection_id and emoji fields
- Move rag_engine_plugin_id to CREATE_FIELDS (immutable after creation)
- Fix creation_settings mutable default value (dict -> None)
- Rename vector delete method to delete_by_file_id for correct semantics
- Fix delete_by_filter to raise NotImplementedError instead of silent no-op
- Add database migration script (dbm019) for new columns and table cleanup

P1 fixes:
- Clean up design-hesitation comments in connector.py
- Add _parse_plugin_id() with format validation for all RAG methods
- Make _retrieve() raise exceptions instead of silently returning empty results
- Extract _make_rag_error_response() helper for clean error formatting
- Remove unused imports from handler.py

P2 fixes:
- Fix runtime.py indentation inconsistencies
- Simplify get_file_stream to use storage abstraction uniformly
- Reduce redundant DB queries in knowledge service (extract _check_doc_capability)
- Fix engines.py URL encoding: use <path:plugin_id> instead of __ replacement
- Add read-only mode for engine settings in KBForm edit mode
- Simplify page.tsx handleKBCardClick to pass only kbId string

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix: address code review findings for RAG plugin architecture

- Frontend: add retrieval_settings param to retrieveKnowledgeBase API call
- Backend: return {uuid} from PUT knowledge base to match frontend expectation
- Backend: validate query is non-empty in retrieve endpoint (400 on empty)
- Backend: rename vector_delete ids→file_ids for semantic clarity, keep
  backward compat by accepting both 'file_ids' and 'ids' in RPC handler
- Backend: ensure rag_engine.name fallback is always I18nObject-compatible
  dict, preventing frontend extractI18nObject from receiving plain strings
- Migration: fix misleading docstring about external_kb data migration

Co-authored-by: Cursor <cursoragent@cursor.com>

* Update langbot-plugin version to 0.2.6

* chore: update required database version from 18 to 19

* refactor: remove unused polymorphic component framework

* chore: fix lint and format issues for python and frontend

* fix(plugin): remove legacy `ids` fallback in rag_vector_delete handler

SDK now sends `file_ids` directly, the `ids` backward-compat fallback
is no longer needed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(rag): deep review fixes for critical bugs, security and quality

Critical:
- Fix StorageMgr.load() -> storage_provider.load() (C1, AttributeError)
- Update required_database_version 18 -> 19 (C2, migration never runs)

Security:
- Add path traversal validation in get_file_stream (C11)
- Add vectors/ids/metadata length validation in rag_vector_upsert (C12)

Logic fixes:
- Legacy KBs: set capabilities to [] instead of ['doc_ingestion'] (C4)
- Fix store_file return type int -> str (C5)
- Fix retrieve_knowledge return [] -> {'results': []} when disabled (C6)
- Re-raise exception in _on_kb_create instead of silently swallowing (C7)
- Log warning when KB not found in memory during delete (C8)

API fixes:
- Catch ValueError as 400 in create_knowledge_base endpoint (C15)
- Validate plugin_id format in engines endpoints (C16)

Quality:
- Remove dead if/else in migration with identical branches (C17)
- Fix variable shadowing: rag_context -> rag_context_text (C18)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: remove unused os import to fix ruff lint

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(plugin): remove PolymorphicComponent sync from LangBot side

Remove sync_polymorphic_component_instances() from connector and handler,
and the post-connection sync call in initialize(). This dead code synced
an always-empty list of polymorphic instances that were never created.

Companion change to langbot-plugin-sdk PolymorphicComponent removal.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(rag): fix vector_delete count bug and remove vestigial instance_id parameter

1. vector_delete: assign return value from delete_by_filter to count
   instead of silently returning 0 for filter-based deletion.

2. Remove instance_id parameter from the entire retrieve_knowledge
   call chain (kbmgr → connector → handler → runtime). This parameter
   was a remnant of the PolymorphicComponent mechanism and is no longer
   used — RAGEngine operates as a stateless singleton.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(web): 支持 creation_schema 字段级别的 editable 属性控制编辑模式可修改性

- IDynamicFormItemSchema 添加 editable 可选属性
- DynamicFormItemConfig 透传 editable 属性
- DynamicFormComponent 接收 isEditing prop,按字段 editable 值控制禁用
- KBForm 解析 editable 并传递 isEditing 给动态表单组件
- editable 未指定时默认可编辑,editable: false 时编辑模式下禁用该字段

* feat(storage): 添加 size() 抽象方法及 LocalStorage/S3 实现

支持获取存储对象大小,S3 使用 head_object 避免下载整个文件

* fix(migration): 删除 external_knowledge_bases 表前记录日志警告

- 迁移时如果表中存在数据,先 warning 日志记录避免无感数据丢失
- 添加 chunk 清理注释说明:仅对旧版非插件架构 KB 有效

* fix(web): 修复检索结果长文本撑大容器导致查询按钮不可见

KBDetailDialog 的 main 容器添加 min-w-0 overflow-x-hidden,
限制 flex-1 子容器宽度,防止 Dify RAG 长文本撑出 Dialog 边界

* fix(rag): address code review issues for plugin architecture PR

- Fix SQL injection in migration helpers by using bind parameters
- Move numpy import to module level in vector/mgr.py
- Improve path traversal validation using posixpath.normpath
- Add call_rag_retrieve to connector, eliminating duplicate plugin_id
  parsing in kbmgr.py _retrieve
- Normalize typing style to modern dict/list/None syntax

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* style(web): fix prettier formatting errors

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(rag): update embedding handling in RuntimeConnectionHandler

- Renamed RAG_EMBED_DOCUMENTS and RAG_EMBED_QUERY actions to INVOKE_EMBEDDING for clarity.
- Removed embed_documents and embed_query methods from RuntimeEmbeddingModel and RAGRuntimeService.
- Integrated embedding model retrieval directly in the invoke_embedding method, improving error handling for missing models.
- Updated the embedding invocation logic to streamline the process and enhance error reporting.

* refactor(web): replace KnowledgeRetriever with RAGEngine across frontend and tests

KnowledgeRetriever component type has been removed in favor of the new
RAGEngine architecture. Update all remaining references in i18n locales,
plugin component icon mappings, marketplace filter, and unit tests.

Addresses reviewer notes from RockChinQ on PR #1967.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(rag): address critical bugs found in deep review

- Fix path traversal bypass in runtime.py (check all path components for '..')
- Use normalized path for file loading instead of raw user input
- Change knowledge_bases from list to dict for O(1) lookup and race safety
- Add rollback on KB creation failure (clean up DB + runtime on plugin error)
- Add null check after KB update in knowledge service
- Fix file extension parsing to use os.path.splitext instead of split('.')
  (handles multi-dot filenames like 'report.v2.pdf' correctly)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(rag): address remaining review issues across frontend and backend

Frontend:
- Fix KB delete: use async/await with error handling instead of fire-and-forget
- Fix capabilities null check: add optional chaining to prevent crash
- Add toast.error on KB info load failure instead of silent console.error
- Replace hard-coded Chinese validation message with i18n key
- Replace hard-coded English error messages in DynamicFormItemComponent with i18n
- Optimize document polling: stop when all documents reach terminal state
- Add i18n keys (fieldRequired, loadKnowledgeBaseFailed,
  deleteKnowledgeBaseFailed, getKnowledgeBaseListError) to all 4 locales

Backend:
- Fix KB delete atomicity: delete from DB first, then notify plugin
- Add RAG engine plugin existence validation before creating KB

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* style(rag): fix ruff formatting in kbmgr.py

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Junyan Qin <rockchinq@gmail.com>

* chore: bump langbot-plugin to 0.3.0 (#1992)

* chore: correct sdk version to 0.3.0a1

* feat: normalize rag related actions' names

* refactor(rag): align IngestionContext fields with SDK changes

Remove redundant `chunking_strategy` field and rename `custom_settings`
to `creation_settings` to match the updated SDK entity definitions
(langbot-plugin-sdk#36).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* style: fix ruff formatting

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(rag): enforce immutability of embedding_model_uuid and non-editable creation_settings fields

Remove embedding_model_uuid from MUTABLE_FIELDS to prevent post-creation
modification via API. Add backend validation for creation_settings to
preserve fields marked editable:false in the plugin's creation schema.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* style(rag): fix ruff formatting in knowledge service

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(rag): split settings into immutable creation_settings and mutable retrieval_settings

- Remove standalone embedding_model_uuid and top_k columns from KB entity
- Add retrieval_settings column; update MUTABLE_FIELDS/CREATE_FIELDS accordingly
- Merge migration logic into dbm019 (add retrieval_settings, migrate top_k
  and embedding_model_uuid into JSON settings, drop old columns on PostgreSQL)
- Remove _filter_creation_settings and per-field editable concept
- Frontend: creation_settings fields are all disabled when editing,
  retrieval_settings fields are always editable via a second DynamicFormComponent
- Remove editable from IDynamicFormItemSchema, DynamicFormItemConfig
- Clean up KBCardVO, KnowledgeBase API type, and localagent runner

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* bugfix: if ingest_document failed,not raise exep

* fix: ruff lint

* refactor(rag): remove unused _get_kb_entity method from RAGRuntimeService

* feat(vector): implement metadata filters for vector_search and vector_delete (#1997)

Add functional metadata filter support across all 5 VDB backends using
Chroma-style where syntax as the canonical format. Previously the filters
parameter existed throughout the stack but was entirely ignored.

- Add filter_utils.py with normalize_filter() and strip_unsupported_fields()
- Implement filter in search() and add delete_by_filter() for all backends:
  Chroma/SeekDB (native passthrough), Qdrant (translated to models.Filter),
  Milvus (translated to expr string), pgvector (translated to SQLAlchemy conditions)
- Milvus/pgvector limited to {text, file_id, chunk_uuid}; other fields logged and ignored
- Replace delete_by_filter() NotImplementedError with backend delegation in mgr.py
- Populate retrieval_context['filters'] from settings in kbmgr._retrieve()
- Pass search_type/query_text/documents through handler and runtime service

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* style(vector): fix ruff formatting

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(vector): remove numpy dependency and fix SeekDB search modes

- Remove numpy array conversion for query vectors; all VDB backends
  accept list[float] directly
- Remove redundant get_or_create_collection call from upsert; backends
  handle collection creation internally in add_embeddings
- Fix SeekDB to raise ValueError when vector dimension is unknown
  instead of defaulting to 384
- Use hybrid_search() for full-text and hybrid search modes in SeekDB,
  since pyseekdb's query() always requires embeddings

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(vector): escape single quotes in SeekDB documents and metadata

Document text containing apostrophes (e.g. "don't", "it's") causes
SQL syntax errors in OceanBase because single quotes were not in the
escape table. Add single-quote escaping and apply the escape table to
the documents parameter in add_embeddings(), not just metadata.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(vector): use standard SQL escaping for single quotes in SeekDB

Change single quote escaping from MySQL-style \' to standard SQL ''
(doubled quote). The backslash escape is not recognized by OceanBase
in NO_BACKSLASH_ESCAPES mode, causing SQL syntax errors when metadata
text contains apostrophes (e.g. O'Shea in academic citations).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(rag): persist retrieval_settings on knowledge base creation

retrieval_settings was not being passed from the service layer to
RAGManager.create_knowledge_base(), causing retrieval schema fields
(e.g. query_rewrite) to be lost on initial KB creation. They only
took effect after a subsequent edit/update.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(web): add show_if conditional rendering for dynamic forms

Support conditional field visibility in plugin-defined forms via
show_if rules (eq, neq, in operators). Fields can depend on values
from the same form or cross-reference between creation and retrieval
settings via externalDependentValues.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(rag): replace base64 with chunked file transfer for get_rag_file_stream

Use send_file() instead of base64 encoding for returning file content
in the GET_RAG_FILE_STREAM handler, avoiding memory issues with large files.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(parser): add parser plugin integration and capability-aware upload UI (#2000)

* feat(parser): add parser plugin integration and capability-aware upload UI

Backend: add parser plugin API endpoints (list/invoke), connector and
handler support for parser actions, and KB manager passthrough.

Frontend: thread ragEngineCapabilities prop to FileUploadZone and use
doc_parsing capability to conditionally show the RAG engine option in
the parser selector. When no parser is available, show a warning
prompting users to install a parser plugin.

Update i18n: rename builtInParser to "Provided by RAG engine" and add
noParserAvailable warning message in all 4 locales.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(parser): replace base64 with chunked file transfer and remove stale cache

- Remove @alru_cache from list_parsers() and list_rag_engines()
- Replace inline base64 file content with send_file/read_local_file
  chunked transfer pattern in parse_document and invoke_parser flows
- Remove unused base64 import from kbmgr.py

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* feat(web): add Parser component kind to plugin market UI and i18n

Add Parser to kindIconMap, market filter toggle, and all 4 locale files
so parser plugins are properly displayed and filterable in the plugin
market, matching the existing RAGEngine treatment.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* style(web): fix prettier formatting from merge

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: rename RAGEngine to KnowledgeEngine across frontend and backend

* fix(web): fix I18nObject import path in FileUploadZone and KBDoc

* chore: format files involved in RAGEngine to KnowledgeEngine refactor

* refactor: change rag engine to knowledge engine

* fix: update langbot-plugin version to 0.3.0rc1

* chore: disable migration 20 for now

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Junyan Qin <rockchinq@gmail.com>
2026-03-06 21:54:38 +08:00
fdc310
3e8f47fd97 feat: judge and send runner category (local or cloud) for telemetry
* feat(chat): add runner_url to payload for telemetry tracking

* feat(telemetry): add runner_url to sanitized fields in telemetry payload

* feat(telemetry): replace runner_url with runner_category in telemetry payload and add runner utility functions

* fix:ruff
2026-03-06 00:44:09 +08:00
youhuanghe
b11ae55c6e fix: update web/lint src 2026-03-05 15:02:03 +00:00
marun
2d63d528c6 refactor(dify): Optimize the Dify API output parsing and workflow processing logic (#2027)
- Add the _extract_dify_text_output method to uniformly handle the parsing of Dify output content

- Modify the content extraction method for the answer node in workflow mode

- Add workflow mode detection logic to support the workflow_started event

- Handle error state checks upon completion of the workflow

- Improve the message chunking logic for both basic and workflow modes

- Add a mechanism to capture answer content upon completion of a workflow node
2026-03-05 15:15:40 +08:00
fdc310
10f253015d Fix/tg send msg chunk (#2021)
* feat(telegram): enhance message handling with markdown support and draft messages

* fix(telegram): update draft message ID generation to use current timestamp
2026-03-04 20:42:33 +08:00
249 changed files with 29838 additions and 8763 deletions

View File

@@ -1,5 +1,5 @@
name: 漏洞反馈
description: 【供中文用户】报错或漏洞请使用这个模板创建不使用此模板创建的异常、漏洞相关issue将被直接关闭。由于自己操作不当/不甚了解所用技术栈引起的网络连接问题恕无法解决,请勿提 issue。容器间网络连接问题参考文档 https://docs.langbot.app/zh/workshop/network-details.html
description: 【供中文用户】报错或漏洞请使用这个模板创建不使用此模板创建的异常、漏洞相关issue将被直接关闭。由于自己操作不当/不甚了解所用技术栈引起的网络连接问题恕无法解决,请勿提 issue。容器间网络连接问题参考文档 https://link.langbot.app/zh/docs/network
title: "[Bug]: "
labels: ["bug?"]
body:

View File

@@ -1,5 +1,5 @@
name: Bug report
description: Report bugs or vulnerabilities using this template. For container network connection issues, refer to the documentation https://docs.langbot.app/en/workshop/network-details.html
description: Report bugs or vulnerabilities using this template. For container network connection issues, refer to the documentation https://link.langbot.app/en/docs/network
title: "[Bug]: "
labels: ["bug?"]
body:

View File

@@ -9,16 +9,14 @@ repos:
# Run the formatter of backend.
- id: ruff-format
- repo: https://github.com/pre-commit/mirrors-prettier
rev: v3.1.0
hooks:
- id: prettier
types_or: [javascript, jsx, ts, tsx, css, scss]
additional_dependencies:
- prettier@3.1.0
- repo: local
hooks:
- id: prettier
name: prettier
entry: npx --prefix web prettier --write --ignore-unknown
language: system
types_or: [javascript, jsx, ts, tsx, css, scss]
- id: lint-staged
name: lint-staged
entry: cd web && pnpm lint-staged

View File

@@ -19,9 +19,9 @@ English / [简体中文](README_CN.md) / [繁體中文](README_TW.md) / [日本
[![GitHub stars](https://img.shields.io/github/stars/langbot-app/LangBot?style=social)](https://github.com/langbot-app/LangBot/stargazers)
<a href="https://langbot.app">Website</a>
<a href="https://docs.langbot.app/en/insight/features">Features</a>
<a href="https://docs.langbot.app/en/insight/guide">Docs</a>
<a href="https://docs.langbot.app/en/tags/readme">API</a>
<a href="https://link.langbot.app/en/docs/features">Features</a>
<a href="https://link.langbot.app/en/docs/guide">Docs</a>
<a href="https://link.langbot.app/en/docs/api">API</a>
<a href="https://space.langbot.app/cloud">Cloud</a>
<a href="https://space.langbot.app">Plugin Market</a>
<a href="https://langbot.featurebase.app/roadmap">Roadmap</a>
@@ -45,7 +45,7 @@ LangBot is an **open-source, production-grade platform** for building AI-powered
- **Web Management Panel** — Configure, manage, and monitor your bots through an intuitive browser interface. No YAML editing required.
- **Multi-Pipeline Architecture** — Different bots for different scenarios, with comprehensive monitoring and exception handling.
[→ Learn more about all features](https://docs.langbot.app/en/insight/features)
[→ Learn more about all features](https://link.langbot.app/en/docs/features)
---
@@ -76,7 +76,7 @@ docker compose up -d
[![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/en-US/templates/ZKTBDH)
[![Deploy on Railway](https://railway.com/button.svg)](https://railway.app/template/yRrAyL?referralCode=vogKPF)
**More options:** [Docker](https://docs.langbot.app/en/deploy/langbot/docker) · [Manual](https://docs.langbot.app/en/deploy/langbot/manual) · [BTPanel](https://docs.langbot.app/en/deploy/langbot/one-click/bt) · [Kubernetes](./docker/README_K8S.md)
**More options:** [Docker](https://link.langbot.app/en/docs/docker) · [Manual](https://link.langbot.app/en/docs/manual-deploy) · [BTPanel](https://link.langbot.app/en/docs/bt-panel) · [Kubernetes](./docker/README_K8S.md)
---
@@ -124,7 +124,7 @@ docker compose up -d
| [接口 AI](https://jiekou.ai/) | Gateway | ✅ |
| [302.AI](https://share.302.ai/SuTG99) | Gateway | ✅ |
[→ View all integrations](https://docs.langbot.app/en/insight/features)
[→ View all integrations](https://link.langbot.app/en/docs/features)
---

View File

@@ -21,9 +21,9 @@
[![star](https://gitcode.com/RockChinQ/LangBot/star/badge.svg)](https://gitcode.com/RockChinQ/LangBot)
<a href="https://langbot.app">官网</a>
<a href="https://docs.langbot.app/zh/insight/features.html">特性</a>
<a href="https://docs.langbot.app/zh/insight/guide.html">文档</a>
<a href="https://docs.langbot.app/zh/tags/readme.html">API</a>
<a href="https://link.langbot.app/zh/docs/features">特性</a>
<a href="https://link.langbot.app/zh/docs/guide">文档</a>
<a href="https://link.langbot.app/zh/docs/api">API</a>
<a href="https://space.langbot.app/cloud">Cloud</a>
<a href="https://space.langbot.app">插件市场</a>
<a href="https://langbot.featurebase.app/roadmap">路线图</a>
@@ -34,8 +34,6 @@
---
## 什么是 LangBot
LangBot 是一个**开源的生产级平台**,用于构建 AI 驱动的即时通信机器人。它将大语言模型LLM连接到各种聊天平台帮助你创建能够对话、执行任务、并集成到现有工作流程中的智能 Agent。
### 核心能力
@@ -43,11 +41,11 @@ LangBot 是一个**开源的生产级平台**,用于构建 AI 驱动的即时
- **AI 对话与 Agent** — 多轮对话、工具调用、多模态、流式输出。自带 RAG知识库深度集成 [Dify](https://dify.ai)、[Coze](https://coze.com)、[n8n](https://n8n.io)、[Langflow](https://langflow.org) 等 LLMOps 平台。
- **全平台支持** — 一套代码,覆盖 QQ、微信、企业微信、飞书、钉钉、Discord、Telegram、Slack、LINE、KOOK 等平台。
- **生产就绪** — 访问控制、限速、敏感词过滤、全面监控与异常处理,已被多家企业采用。
- **插件生态** — 数百个插件,事件驱动架构,组件扩展,适配 [MCP 协议](https://modelcontextprotocol.io/)。
- **插件生态** — 数百个插件,跨进程的事件驱动架构,组件扩展,适配 [MCP 协议](https://modelcontextprotocol.io/)。
- **Web 管理面板** — 通过浏览器直观地配置、管理和监控机器人,无需手动编辑配置文件。
- **多流水线架构** — 不同机器人用于不同场景,具备全面的监控和异常处理能力。
[→ 了解更多功能特性](https://docs.langbot.app/zh/insight/features.html)
[→ 了解更多功能特性](https://link.langbot.app/zh/docs/features)
---
@@ -78,7 +76,7 @@ docker compose up -d
[![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/zh-CN/templates/ZKTBDH)
[![Deploy on Railway](https://railway.com/button.svg)](https://railway.app/template/yRrAyL?referralCode=vogKPF)
**更多方式:** [Docker](https://docs.langbot.app/zh/deploy/langbot/docker.html) · [手动部署](https://docs.langbot.app/zh/deploy/langbot/manual.html) · [宝塔面板](https://docs.langbot.app/zh/deploy/langbot/one-click/bt.html) · [Kubernetes](./docker/README_K8S.md)
**更多方式:** [Docker](https://link.langbot.app/zh/docs/docker) · [手动部署](https://link.langbot.app/zh/docs/manual-deploy) · [宝塔面板](https://link.langbot.app/zh/docs/bt-panel) · [Kubernetes](./docker/README_K8S.md)
---
@@ -127,7 +125,7 @@ docker compose up -d
| [小马算力](https://www.tokenpony.cn/453z1) | 聚合平台 | ✅ |
| [百宝箱Tbox](https://www.tbox.cn/open) | 智能体平台 | ✅ |
[→ 查看完整集成列表](https://docs.langbot.app/zh/insight/features.html)
[→ 查看完整集成列表](https://link.langbot.app/zh/docs/features)
### TTS语音合成

View File

@@ -19,9 +19,9 @@
[![GitHub stars](https://img.shields.io/github/stars/langbot-app/LangBot?style=social)](https://github.com/langbot-app/LangBot/stargazers)
<a href="https://langbot.app">Inicio</a>
<a href="https://docs.langbot.app/en/insight/features.html">Características</a>
<a href="https://docs.langbot.app/en/insight/guide.html">Documentación</a>
<a href="https://docs.langbot.app/en/tags/readme.html">API</a>
<a href="https://link.langbot.app/en/docs/features">Características</a>
<a href="https://link.langbot.app/en/docs/guide">Documentación</a>
<a href="https://link.langbot.app/en/docs/api">API</a>
<a href="https://space.langbot.app">Mercado de Plugins</a>
<a href="https://langbot.featurebase.app/roadmap">Hoja de Ruta</a>
@@ -44,7 +44,7 @@ LangBot es una **plataforma de código abierto y grado de producción** para con
- **Panel de Gestión Web** — Configure, gestione y monitoree sus bots a través de una interfaz de navegador intuitiva. Sin necesidad de editar YAML.
- **Arquitectura Multi-Pipeline** — Diferentes bots para diferentes escenarios, con monitoreo completo y manejo de excepciones.
[→ Conocer más sobre todas las funcionalidades](https://docs.langbot.app/en/insight/features.html)
[→ Conocer más sobre todas las funcionalidades](https://link.langbot.app/en/docs/features)
---
@@ -75,7 +75,7 @@ docker compose up -d
[![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/en-US/templates/ZKTBDH)
[![Deploy on Railway](https://railway.com/button.svg)](https://railway.app/template/yRrAyL?referralCode=vogKPF)
**Más opciones:** [Docker](https://docs.langbot.app/en/deploy/langbot/docker.html) · [Manual](https://docs.langbot.app/en/deploy/langbot/manual.html) · [BTPanel](https://docs.langbot.app/en/deploy/langbot/one-click/bt.html) · [Kubernetes](./docker/README_K8S.md)
**Más opciones:** [Docker](https://link.langbot.app/en/docs/docker) · [Manual](https://link.langbot.app/en/docs/manual-deploy) · [BTPanel](https://link.langbot.app/en/docs/bt-panel) · [Kubernetes](./docker/README_K8S.md)
---
@@ -123,7 +123,7 @@ docker compose up -d
| [接口 AI](https://jiekou.ai/) | Pasarela | ✅ |
| [302.AI](https://share.302.ai/SuTG99) | Pasarela | ✅ |
[→ Ver todas las integraciones](https://docs.langbot.app/en/insight/features.html)
[→ Ver todas las integraciones](https://link.langbot.app/en/docs/features)
---

View File

@@ -19,9 +19,9 @@
[![GitHub stars](https://img.shields.io/github/stars/langbot-app/LangBot?style=social)](https://github.com/langbot-app/LangBot/stargazers)
<a href="https://langbot.app">Accueil</a>
<a href="https://docs.langbot.app/en/insight/features.html">Fonctionnalités</a>
<a href="https://docs.langbot.app/en/insight/guide.html">Documentation</a>
<a href="https://docs.langbot.app/en/tags/readme.html">API</a>
<a href="https://link.langbot.app/en/docs/features">Fonctionnalités</a>
<a href="https://link.langbot.app/en/docs/guide">Documentation</a>
<a href="https://link.langbot.app/en/docs/api">API</a>
<a href="https://space.langbot.app">Marché des Plugins</a>
<a href="https://langbot.featurebase.app/roadmap">Feuille de Route</a>
@@ -44,7 +44,7 @@ LangBot est une **plateforme open-source de niveau production** pour créer des
- **Panneau de Gestion Web** — Configurez, gérez et surveillez vos bots via une interface navigateur intuitive. Aucune édition de YAML requise.
- **Architecture Multi-Pipeline** — Différents bots pour différents scénarios, avec surveillance complète et gestion des exceptions.
[→ En savoir plus sur toutes les fonctionnalités](https://docs.langbot.app/en/insight/features.html)
[→ En savoir plus sur toutes les fonctionnalités](https://link.langbot.app/en/docs/features)
---
@@ -75,7 +75,7 @@ docker compose up -d
[![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/en-US/templates/ZKTBDH)
[![Deploy on Railway](https://railway.com/button.svg)](https://railway.app/template/yRrAyL?referralCode=vogKPF)
**Plus d'options :** [Docker](https://docs.langbot.app/en/deploy/langbot/docker.html) · [Manuel](https://docs.langbot.app/en/deploy/langbot/manual.html) · [BTPanel](https://docs.langbot.app/en/deploy/langbot/one-click/bt.html) · [Kubernetes](./docker/README_K8S.md)
**Plus d'options :** [Docker](https://link.langbot.app/en/docs/docker) · [Manuel](https://link.langbot.app/en/docs/manual-deploy) · [BTPanel](https://link.langbot.app/en/docs/bt-panel) · [Kubernetes](./docker/README_K8S.md)
---
@@ -123,7 +123,7 @@ docker compose up -d
| [PPIO](https://ppinfra.com/user/register?invited_by=QJKFYD&utm_source=github_langbot) | Plateforme GPU | ✅ |
| [ShengSuanYun](https://www.shengsuanyun.com/?from=CH_KYIPP758) | Plateforme GPU | ✅ |
[→ Voir toutes les intégrations](https://docs.langbot.app/en/insight/features.html)
[→ Voir toutes les intégrations](https://link.langbot.app/en/docs/features)
---

View File

@@ -19,9 +19,9 @@
[![GitHub stars](https://img.shields.io/github/stars/langbot-app/LangBot?style=social)](https://github.com/langbot-app/LangBot/stargazers)
<a href="https://langbot.app">ホーム</a>
<a href="https://docs.langbot.app/ja/insight/features.html">機能</a>
<a href="https://docs.langbot.app/ja/insight/guide.html">ドキュメント</a>
<a href="https://docs.langbot.app/ja/tags/readme.html">API</a>
<a href="https://link.langbot.app/ja/docs/features">機能</a>
<a href="https://link.langbot.app/ja/docs/guide">ドキュメント</a>
<a href="https://link.langbot.app/ja/docs/api">API</a>
<a href="https://space.langbot.app">プラグインマーケット</a>
<a href="https://langbot.featurebase.app/roadmap">ロードマップ</a>
@@ -44,7 +44,7 @@ LangBot は、AI搭載のインスタントメッセージングボットを構
- **Web管理パネル** — 直感的なブラウザインターフェースからボットの設定、管理、監視が可能。YAML編集は不要。
- **マルチパイプラインアーキテクチャ** — 異なるシナリオに異なるボットを配置し、包括的な監視と例外処理を実現。
[→ すべての機能について詳しく見る](https://docs.langbot.app/ja/insight/features.html)
[→ すべての機能について詳しく見る](https://link.langbot.app/ja/docs/features)
---
@@ -75,7 +75,7 @@ docker compose up -d
[![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/en-US/templates/ZKTBDH)
[![Deploy on Railway](https://railway.com/button.svg)](https://railway.app/template/yRrAyL?referralCode=vogKPF)
**その他:** [Docker](https://docs.langbot.app/en/deploy/langbot/docker.html) · [手動デプロイ](https://docs.langbot.app/en/deploy/langbot/manual.html) · [BTPanel](https://docs.langbot.app/en/deploy/langbot/one-click/bt.html) · [Kubernetes](./docker/README_K8S.md)
**その他:** [Docker](https://link.langbot.app/en/docs/docker) · [手動デプロイ](https://link.langbot.app/en/docs/manual-deploy) · [BTPanel](https://link.langbot.app/en/docs/bt-panel) · [Kubernetes](./docker/README_K8S.md)
---
@@ -123,7 +123,7 @@ docker compose up -d
| [接口 AI](https://jiekou.ai/) | ゲートウェイ | ✅ |
| [302.AI](https://share.302.ai/SuTG99) | ゲートウェイ | ✅ |
[→ すべての統合を表示](https://docs.langbot.app/en/insight/features.html)
[→ すべての統合を表示](https://link.langbot.app/en/docs/features)
---

View File

@@ -19,9 +19,9 @@
[![GitHub stars](https://img.shields.io/github/stars/langbot-app/LangBot?style=social)](https://github.com/langbot-app/LangBot/stargazers)
<a href="https://langbot.app">홈</a>
<a href="https://docs.langbot.app/en/insight/features.html">기능</a>
<a href="https://docs.langbot.app/en/insight/guide.html">문서</a>
<a href="https://docs.langbot.app/en/tags/readme.html">API</a>
<a href="https://link.langbot.app/en/docs/features">기능</a>
<a href="https://link.langbot.app/en/docs/guide">문서</a>
<a href="https://link.langbot.app/en/docs/api">API</a>
<a href="https://space.langbot.app">플러그인 마켓</a>
<a href="https://langbot.featurebase.app/roadmap">로드맵</a>
@@ -44,7 +44,7 @@ LangBot은 AI 기반 인스턴트 메시징 봇을 구축하기 위한 **오픈
- **웹 관리 패널** — 직관적인 브라우저 인터페이스로 봇을 구성, 관리 및 모니터링. YAML 편집 불필요.
- **멀티 파이프라인 아키텍처** — 다양한 시나리오에 맞는 다양한 봇 구성, 종합 모니터링 및 예외 처리.
[→ 모든 기능 자세히 보기](https://docs.langbot.app/en/insight/features.html)
[→ 모든 기능 자세히 보기](https://link.langbot.app/en/docs/features)
---
@@ -75,7 +75,7 @@ docker compose up -d
[![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/en-US/templates/ZKTBDH)
[![Deploy on Railway](https://railway.com/button.svg)](https://railway.app/template/yRrAyL?referralCode=vogKPF)
**더 많은 옵션:** [Docker](https://docs.langbot.app/en/deploy/langbot/docker.html) · [수동 배포](https://docs.langbot.app/en/deploy/langbot/manual.html) · [BTPanel](https://docs.langbot.app/en/deploy/langbot/one-click/bt.html) · [Kubernetes](./docker/README_K8S.md)
**더 많은 옵션:** [Docker](https://link.langbot.app/en/docs/docker) · [수동 배포](https://link.langbot.app/en/docs/manual-deploy) · [BTPanel](https://link.langbot.app/en/docs/bt-panel) · [Kubernetes](./docker/README_K8S.md)
---
@@ -123,7 +123,7 @@ docker compose up -d
| [接口 AI](https://jiekou.ai/) | 게이트웨이 | ✅ |
| [302.AI](https://share.302.ai/SuTG99) | 게이트웨이 | ✅ |
[→ 모든 통합 보기](https://docs.langbot.app/en/insight/features.html)
[→ 모든 통합 보기](https://link.langbot.app/en/docs/features)
---

View File

@@ -19,9 +19,9 @@
[![GitHub stars](https://img.shields.io/github/stars/langbot-app/LangBot?style=social)](https://github.com/langbot-app/LangBot/stargazers)
<a href="https://langbot.app">Главная</a>
<a href="https://docs.langbot.app/en/insight/features.html">Возможности</a>
<a href="https://docs.langbot.app/en/insight/guide.html">Документация</a>
<a href="https://docs.langbot.app/en/tags/readme.html">API</a>
<a href="https://link.langbot.app/en/docs/features">Возможности</a>
<a href="https://link.langbot.app/en/docs/guide">Документация</a>
<a href="https://link.langbot.app/en/docs/api">API</a>
<a href="https://space.langbot.app">Магазин плагинов</a>
<a href="https://langbot.featurebase.app/roadmap">Дорожная карта</a>
@@ -44,7 +44,7 @@ LangBot — это **платформа с открытым исходным к
- **Веб-панель управления** — Настраивайте, управляйте и мониторьте ваших ботов через интуитивный браузерный интерфейс. Ручное редактирование YAML не требуется.
- **Мультиконвейерная архитектура** — Разные боты для разных сценариев с комплексным мониторингом и обработкой исключений.
[→ Подробнее обо всех возможностях](https://docs.langbot.app/en/insight/features.html)
[→ Подробнее обо всех возможностях](https://link.langbot.app/en/docs/features)
---
@@ -75,7 +75,7 @@ docker compose up -d
[![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/en-US/templates/ZKTBDH)
[![Deploy on Railway](https://railway.com/button.svg)](https://railway.app/template/yRrAyL?referralCode=vogKPF)
**Другие варианты:** [Docker](https://docs.langbot.app/en/deploy/langbot/docker.html) · [Ручная установка](https://docs.langbot.app/en/deploy/langbot/manual.html) · [BTPanel](https://docs.langbot.app/en/deploy/langbot/one-click/bt.html) · [Kubernetes](./docker/README_K8S.md)
**Другие варианты:** [Docker](https://link.langbot.app/en/docs/docker) · [Ручная установка](https://link.langbot.app/en/docs/manual-deploy) · [BTPanel](https://link.langbot.app/en/docs/bt-panel) · [Kubernetes](./docker/README_K8S.md)
---
@@ -123,7 +123,7 @@ docker compose up -d
| [PPIO](https://ppinfra.com/user/register?invited_by=QJKFYD&utm_source=github_langbot) | Платформа GPU | ✅ |
| [ShengSuanYun](https://www.shengsuanyun.com/?from=CH_KYIPP758) | Платформа GPU | ✅ |
[→ Смотреть все интеграции](https://docs.langbot.app/en/insight/features.html)
[→ Смотреть все интеграции](https://link.langbot.app/en/docs/features)
---

View File

@@ -21,9 +21,9 @@
[![star](https://gitcode.com/RockChinQ/LangBot/star/badge.svg)](https://gitcode.com/RockChinQ/LangBot)
<a href="https://langbot.app">官網</a>
<a href="https://docs.langbot.app/zh/insight/features.html">特性</a>
<a href="https://docs.langbot.app/zh/insight/guide.html">文件</a>
<a href="https://docs.langbot.app/zh/tags/readme.html">API</a>
<a href="https://link.langbot.app/zh/docs/features">特性</a>
<a href="https://link.langbot.app/zh/docs/guide">文件</a>
<a href="https://link.langbot.app/zh/docs/api">API</a>
<a href="https://space.langbot.app">外掛市場</a>
<a href="https://langbot.featurebase.app/roadmap">路線圖</a>
@@ -46,7 +46,7 @@ LangBot 是一個**開源的生產級平台**,用於建構 AI 驅動的即時
- **Web 管理面板** — 透過瀏覽器直觀地配置、管理和監控機器人,無需手動編輯設定檔。
- **多流水線架構** — 不同機器人用於不同場景,具備全面的監控和異常處理能力。
[→ 了解更多功能特性](https://docs.langbot.app/zh/insight/features.html)
[→ 了解更多功能特性](https://link.langbot.app/zh/docs/features)
---
@@ -77,7 +77,7 @@ docker compose up -d
[![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/zh-CN/templates/ZKTBDH)
[![Deploy on Railway](https://railway.com/button.svg)](https://railway.app/template/yRrAyL?referralCode=vogKPF)
**更多方式:** [Docker](https://docs.langbot.app/zh/deploy/langbot/docker.html) · [手動部署](https://docs.langbot.app/zh/deploy/langbot/manual.html) · [寶塔面板](https://docs.langbot.app/zh/deploy/langbot/one-click/bt.html) · [Kubernetes](./docker/README_K8S.md)
**更多方式:** [Docker](https://link.langbot.app/zh/docs/docker) · [手動部署](https://link.langbot.app/zh/docs/manual-deploy) · [寶塔面板](https://link.langbot.app/zh/docs/bt-panel) · [Kubernetes](./docker/README_K8S.md)
---
@@ -139,7 +139,7 @@ docker compose up -d
|-----------|------|
| 阿里雲百煉 | [外掛](https://github.com/Thetail001/LangBot_BailianTextToImagePlugin) |
[→ 查看完整整合列表](https://docs.langbot.app/zh/insight/features.html)
[→ 查看完整整合列表](https://link.langbot.app/zh/docs/features)
---

View File

@@ -19,9 +19,9 @@
[![GitHub stars](https://img.shields.io/github/stars/langbot-app/LangBot?style=social)](https://github.com/langbot-app/LangBot/stargazers)
<a href="https://langbot.app">Trang chủ</a>
<a href="https://docs.langbot.app/en/insight/features.html">Tính năng</a>
<a href="https://docs.langbot.app/en/insight/guide.html">Tài liệu</a>
<a href="https://docs.langbot.app/en/tags/readme.html">API</a>
<a href="https://link.langbot.app/en/docs/features">Tính năng</a>
<a href="https://link.langbot.app/en/docs/guide">Tài liệu</a>
<a href="https://link.langbot.app/en/docs/api">API</a>
<a href="https://space.langbot.app">Chợ Plugin</a>
<a href="https://langbot.featurebase.app/roadmap">Lộ trình</a>
@@ -44,7 +44,7 @@ LangBot là một **nền tảng mã nguồn mở, cấp sản xuất** để x
- **Bảng quản lý Web** — Cấu hình, quản lý và giám sát bot thông qua giao diện trình duyệt trực quan. Không cần chỉnh sửa YAML.
- **Kiến trúc đa Pipeline** — Các bot khác nhau cho các kịch bản khác nhau, với giám sát toàn diện và xử lý ngoại lệ.
[→ Tìm hiểu thêm về tất cả tính năng](https://docs.langbot.app/en/insight/features.html)
[→ Tìm hiểu thêm về tất cả tính năng](https://link.langbot.app/en/docs/features)
---
@@ -75,7 +75,7 @@ docker compose up -d
[![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/en-US/templates/ZKTBDH)
[![Deploy on Railway](https://railway.com/button.svg)](https://railway.app/template/yRrAyL?referralCode=vogKPF)
**Thêm tùy chọn:** [Docker](https://docs.langbot.app/en/deploy/langbot/docker.html) · [Thủ công](https://docs.langbot.app/en/deploy/langbot/manual.html) · [BTPanel](https://docs.langbot.app/en/deploy/langbot/one-click/bt.html) · [Kubernetes](./docker/README_K8S.md)
**Thêm tùy chọn:** [Docker](https://link.langbot.app/en/docs/docker) · [Thủ công](https://link.langbot.app/en/docs/manual-deploy) · [BTPanel](https://link.langbot.app/en/docs/bt-panel) · [Kubernetes](./docker/README_K8S.md)
---
@@ -123,7 +123,7 @@ docker compose up -d
| [接口 AI](https://jiekou.ai/) | Cổng | ✅ |
| [302.AI](https://share.302.ai/SuTG99) | Cổng | ✅ |
[→ Xem tất cả tích hợp](https://docs.langbot.app/en/insight/features.html)
[→ Xem tất cả tích hợp](https://link.langbot.app/en/docs/features)
---

View File

@@ -312,7 +312,7 @@ spec:
### 参考资源
- [LangBot 官方文档](https://docs.langbot.app)
- [Docker 部署文档](https://docs.langbot.app/zh/deploy/langbot/docker.html)
- [Docker 部署文档](https://link.langbot.app/zh/docs/docker)
- [Kubernetes 官方文档](https://kubernetes.io/docs/)
---
@@ -625,5 +625,5 @@ spec:
### References
- [LangBot Official Documentation](https://docs.langbot.app)
- [Docker Deployment Guide](https://docs.langbot.app/zh/deploy/langbot/docker.html)
- [Docker Deployment Guide](https://link.langbot.app/zh/docs/docker)
- [Kubernetes Official Documentation](https://kubernetes.io/docs/)

View File

@@ -34,4 +34,4 @@ services:
networks:
langbot_network:
driver: bridge
driver: bridge

View File

@@ -1,6 +1,6 @@
[project]
name = "langbot"
version = "4.8.7"
version = "4.9.5"
description = "Production-grade platform for building agentic IM bots"
readme = "README.md"
license-files = ["LICENSE"]
@@ -61,10 +61,10 @@ dependencies = [
"html2text>=2024.2.26",
"langchain>=0.2.0",
"langchain-text-splitters>=0.0.1",
"chromadb>=0.4.24",
"chromadb>=1.0.0,<2.0.0",
"qdrant-client (>=1.15.1,<2.0.0)",
"pyseekdb==1.0.0b7",
"langbot-plugin==0.2.7",
"pyseekdb==1.1.0.post3",
"langbot-plugin==0.3.6",
"asyncpg>=0.30.0",
"line-bot-sdk>=3.19.0",
"tboxsdk>=0.0.10",

View File

@@ -1,3 +1,3 @@
"""LangBot - Production-grade platform for building agentic IM bots"""
__version__ = '4.8.7'
__version__ = '4.9.5'

View File

@@ -272,15 +272,30 @@ class DingTalkClient:
message_data['Type'] = 'audio'
elif incoming_message.message_type == 'file':
down_list = incoming_message.get_down_list()
if len(down_list) >= 2:
message_data['File'] = await self.get_file_url(down_list[0])
message_data['Name'] = down_list[1]
# 获取原始数据字典并提取嵌套的文件信息
raw_data = incoming_message.to_dict()
file_info = raw_data.get('content', {})
# 兼容处理:如果 content 仍为 JSON 字符串则进行解析
if isinstance(file_info, str):
try:
file_info = json.loads(file_info)
except (json.JSONDecodeError, TypeError):
file_info = {}
download_code = file_info.get('downloadCode')
file_name = file_info.get('fileName')
if download_code and file_name:
# 转换 downloadCode 为可下载的真实 URL
message_data['File'] = await self.get_file_url(download_code)
message_data['Name'] = file_name
else:
if self.logger:
await self.logger.error(f'get_down_list() returned fewer than 2 elements: {down_list}')
await self.logger.error(f'Failed to extract file info from message content: {file_info}')
message_data['File'] = None
message_data['Name'] = None
message_data['Type'] = 'file'
copy_message_data = message_data.copy()

View File

@@ -0,0 +1,3 @@
from .client import OpenClawWeixinClient as OpenClawWeixinClient
from .types import ApiError as ApiError
from .types import LoginResult as LoginResult

View File

@@ -0,0 +1,807 @@
"""Async HTTP client for the OpenClaw WeChat API.
Implements the iLink Bot API protocol.
Reference: https://github.com/epiral/weixin-bot
Endpoints: getUpdates (long-poll), sendMessage, getUploadUrl, getConfig, sendTyping.
"""
from __future__ import annotations
import asyncio
import base64
import io
import logging
import os
import struct
import typing
import uuid
from typing import Optional
from urllib.parse import quote
import aiohttp
from .types import (
ApiError,
CDNMedia,
FileItem,
GetConfigResponse,
GetUpdatesResponse,
GetUploadUrlResponse,
ImageItem,
LoginResult,
MessageItem,
QRCodeResponse,
QRStatusResponse,
RefMessage,
TextItem,
VideoItem,
VoiceItem,
WeixinMessage,
)
logger = logging.getLogger('openclaw-weixin-sdk')
DEFAULT_BASE_URL = 'https://ilinkai.weixin.qq.com'
CDN_BASE_URL = 'https://novac2c.cdn.weixin.qq.com/c2c'
CHANNEL_VERSION = '1.0.0'
DEFAULT_API_TIMEOUT = 15
DEFAULT_LONG_POLL_TIMEOUT = 40
DEFAULT_CONFIG_TIMEOUT = 10
DEFAULT_QR_POLL_TIMEOUT = 35
SESSION_EXPIRED_ERRCODE = -14
DEFAULT_BOT_TYPE = '3'
# Maximum text length per message chunk (WeChat limit)
MAX_TEXT_CHUNK_SIZE = 2000
def _random_wechat_uin() -> str:
"""Generate the X-WECHAT-UIN header: random uint32 -> decimal string -> base64."""
rand_bytes = os.urandom(4)
uint32_val = struct.unpack('>I', rand_bytes)[0]
return base64.b64encode(str(uint32_val).encode('utf-8')).decode('utf-8')
def _build_base_info() -> dict:
"""Build the base_info payload included in every API request."""
return {'channel_version': CHANNEL_VERSION}
def _chunk_text(text: str, max_size: int = MAX_TEXT_CHUNK_SIZE) -> list[str]:
"""Split long text into chunks that fit within WeChat's message size limit."""
if len(text) <= max_size:
return [text]
chunks = []
while text:
chunks.append(text[:max_size])
text = text[max_size:]
return chunks
class OpenClawWeixinClient:
"""Async client for the OpenClaw WeChat HTTP JSON API."""
def __init__(self, base_url: str, token: str):
self.base_url = base_url.rstrip('/')
self.token = token
self._session: Optional[aiohttp.ClientSession] = None
async def _get_session(self) -> aiohttp.ClientSession:
if self._session is None or self._session.closed:
self._session = aiohttp.ClientSession()
return self._session
async def close(self):
if self._session and not self._session.closed:
await self._session.close()
def _build_headers(self) -> dict[str, str]:
headers = {
'Content-Type': 'application/json',
'AuthorizationType': 'ilink_bot_token',
'X-WECHAT-UIN': _random_wechat_uin(),
}
if self.token:
headers['Authorization'] = f'Bearer {self.token}'
return headers
async def _post(self, endpoint: str, payload: dict, timeout: float = DEFAULT_API_TIMEOUT) -> dict:
"""Make a POST request and return the JSON response.
Raises ApiError on HTTP errors or when the response contains a non-zero errcode.
"""
payload['base_info'] = _build_base_info()
session = await self._get_session()
url = f'{self.base_url}/{endpoint}'
headers = self._build_headers()
async with session.post(
url, json=payload, headers=headers, timeout=aiohttp.ClientTimeout(total=timeout)
) as resp:
if resp.status != 200:
text = await resp.text()
raise ApiError(
f'OpenClaw API error {resp.status}: {text}',
status=resp.status,
)
data = await resp.json(content_type=None)
# Check for application-level errors in the response body
errcode = data.get('errcode') or data.get('ret')
if errcode and errcode != 0:
raise ApiError(
data.get('errmsg') or f'API errcode {errcode}',
status=200,
code=errcode,
payload=data,
)
return data
async def get_updates(
self, get_updates_buf: str = '', timeout: float = DEFAULT_LONG_POLL_TIMEOUT
) -> GetUpdatesResponse:
"""Long-poll for new messages.
Note: This method does NOT raise ApiError for errcode responses —
it returns them in the GetUpdatesResponse so the caller can handle
session expiry and other errors with full context.
"""
try:
# Bypass the errcode check in _post since get_updates needs
# to return error info (e.g. session expired) to the caller.
payload: dict = {'get_updates_buf': get_updates_buf}
payload['base_info'] = _build_base_info()
session = await self._get_session()
url = f'{self.base_url}/ilink/bot/getupdates'
headers = self._build_headers()
async with session.post(
url,
json=payload,
headers=headers,
timeout=aiohttp.ClientTimeout(total=timeout),
) as resp:
if resp.status != 200:
text = await resp.text()
raise ApiError(
f'OpenClaw API error {resp.status}: {text}',
status=resp.status,
)
data = await resp.json(content_type=None)
except (asyncio.TimeoutError, aiohttp.ServerTimeoutError):
return GetUpdatesResponse(ret=0, msgs=[], get_updates_buf=get_updates_buf)
except ApiError:
raise
except Exception as e:
if 'timeout' in str(e).lower():
return GetUpdatesResponse(ret=0, msgs=[], get_updates_buf=get_updates_buf)
raise
return _parse_get_updates_response(data)
async def send_message(
self,
to_user_id: str,
item_list: list[MessageItem],
context_token: str = '',
) -> None:
"""Send a message to a user."""
items_payload = [_message_item_to_dict(item) for item in item_list]
payload = {
'msg': {
'from_user_id': '',
'to_user_id': to_user_id,
'client_id': f'langbot-{uuid.uuid4().hex[:16]}',
'message_type': WeixinMessage.TYPE_BOT,
'message_state': WeixinMessage.STATE_FINISH,
'item_list': items_payload,
'context_token': context_token or None,
}
}
await self._post('ilink/bot/sendmessage', payload)
async def send_text(self, to_user_id: str, text: str, context_token: str = '') -> None:
"""Send a plain text message, automatically chunking if too long."""
chunks = _chunk_text(text)
for chunk in chunks:
item = MessageItem(type=MessageItem.TEXT, text_item=TextItem(text=chunk))
await self.send_message(to_user_id, [item], context_token)
async def get_config(self, ilink_user_id: str, context_token: str = '') -> GetConfigResponse:
"""Get bot config including typing_ticket."""
data = await self._post(
'ilink/bot/getconfig',
{'ilink_user_id': ilink_user_id, 'context_token': context_token or None},
timeout=DEFAULT_CONFIG_TIMEOUT,
)
return GetConfigResponse(
ret=data.get('ret'),
errmsg=data.get('errmsg'),
typing_ticket=data.get('typing_ticket'),
)
async def send_typing(self, ilink_user_id: str, typing_ticket: str, status: int = 1) -> None:
"""Send typing indicator. status: 1=typing, 2=cancel."""
await self._post(
'ilink/bot/sendtyping',
{
'ilink_user_id': ilink_user_id,
'typing_ticket': typing_ticket,
'status': status,
},
timeout=DEFAULT_CONFIG_TIMEOUT,
)
async def stop_typing(self, ilink_user_id: str, typing_ticket: str) -> None:
"""Cancel the typing indicator for a user."""
await self.send_typing(ilink_user_id, typing_ticket, status=2)
async def download_media(
self,
media: CDNMedia,
) -> bytes:
"""Download and decrypt a file from the WeChat CDN.
Args:
media: CDNMedia object with encrypt_query_param and aes_key.
Returns:
Decrypted file bytes.
"""
from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes
from cryptography.hazmat.primitives.padding import PKCS7
if not media.encrypt_query_param:
raise ApiError('CDN media has no encrypt_query_param', status=0)
if not media.aes_key:
raise ApiError('CDN media has no aes_key', status=0)
# Derive 16-byte AES key
# aes_key is base64-encoded; the decoded content may be:
# - raw 16 bytes (direct AES key)
# - 32-char hex string (decode hex to get 16 bytes)
raw = base64.b64decode(media.aes_key)
if len(raw) == 16:
aes_key = raw
elif len(raw) == 32:
# Hex-encoded 16-byte key
aes_key = bytes.fromhex(raw.decode('utf-8'))
else:
raise ApiError(f'Invalid AES key length: {len(raw)} (expected 16 or 32)', status=0)
# Download encrypted bytes from CDN
session = await self._get_session()
cdn_url = f'{CDN_BASE_URL}/download?encrypted_query_param={quote(media.encrypt_query_param, safe="")}'
async with session.get(cdn_url, timeout=aiohttp.ClientTimeout(total=120)) as resp:
if resp.status != 200:
text = await resp.text()
raise ApiError(f'CDN download failed: {resp.status} {text}', status=resp.status)
encrypted = await resp.read()
# Decrypt AES-128-ECB with PKCS7 padding
cipher = Cipher(algorithms.AES(aes_key), modes.ECB())
decryptor = cipher.decryptor()
padded = decryptor.update(encrypted) + decryptor.finalize()
unpadder = PKCS7(128).unpadder()
return unpadder.update(padded) + unpadder.finalize()
async def upload_media(
self,
file_bytes: bytes,
to_user_id: str,
media_type: int,
) -> CDNMedia:
"""Encrypt and upload media to WeChat CDN.
Args:
file_bytes: Raw file bytes to upload.
to_user_id: Recipient user ID.
media_type: 1=IMAGE, 2=VIDEO, 3=FILE, 4=VOICE.
Returns:
CDNMedia with encrypt_query_param and aes_key for use in sendMessage.
"""
import hashlib
from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes
from cryptography.hazmat.primitives.padding import PKCS7
# 1. Generate random 16-byte AES key
raw_key = os.urandom(16)
aes_key_hex = raw_key.hex() # 32-char hex string
# 2. Encode key for CDNMedia: base64(hex_string) — same for all media types
# Matches official SDK: Buffer.from(aeskey_hex).toString("base64")
encoded_key = base64.b64encode(aes_key_hex.encode('utf-8')).decode('utf-8')
# 3. Encrypt file with AES-128-ECB + PKCS7
padder = PKCS7(128).padder()
padded = padder.update(file_bytes) + padder.finalize()
cipher = Cipher(algorithms.AES(raw_key), modes.ECB())
encryptor = cipher.encryptor()
encrypted = encryptor.update(padded) + encryptor.finalize()
# 4. Get upload URL
raw_md5 = hashlib.md5(file_bytes).hexdigest()
filekey = os.urandom(16).hex() # 32-char hex, matches official SDK
upload_resp = await self.get_upload_url(
filekey=filekey,
media_type=media_type,
to_user_id=to_user_id,
rawsize=len(file_bytes),
rawfilemd5=raw_md5,
filesize=len(encrypted),
aeskey=aes_key_hex, # hex string, as expected by the API
)
if not upload_resp.upload_param:
raise ApiError('Failed to get upload URL', status=0)
# 5. Upload to CDN
# upload_param is an opaque token from the server — pass it as-is
session = await self._get_session()
cdn_url = f'{CDN_BASE_URL}/upload?encrypted_query_param={quote(upload_resp.upload_param, safe="")}&filekey={quote(filekey, safe="")}'
logger.debug(
'CDN upload: url=%s raw_size=%d encrypted_size=%d md5=%s aeskey=%s',
cdn_url,
len(file_bytes),
len(encrypted),
raw_md5,
encoded_key,
)
async with session.post(
cdn_url,
data=encrypted,
headers={'Content-Type': 'application/octet-stream'},
timeout=aiohttp.ClientTimeout(total=120),
) as resp:
if resp.status != 200:
text = await resp.text()
logger.error('CDN upload failed: status=%d url=%s body=%s', resp.status, cdn_url, text[:500])
raise ApiError(f'CDN upload failed: {resp.status} {text}', status=resp.status)
download_param = resp.headers.get('x-encrypted-param', '')
if not download_param:
raise ApiError('CDN upload succeeded but no x-encrypted-param returned', status=0)
return CDNMedia(
encrypt_query_param=download_param,
aes_key=encoded_key,
encrypt_type=1,
)
async def send_image(
self,
to_user_id: str,
image_bytes: bytes,
context_token: str = '',
) -> None:
"""Upload an image to CDN and send it."""
media = await self.upload_media(image_bytes, to_user_id, media_type=1)
item = MessageItem(
type=MessageItem.IMAGE,
image_item=ImageItem(
media=media,
aeskey=media.aes_key,
),
)
await self.send_message(to_user_id, [item], context_token)
async def send_file(
self,
to_user_id: str,
file_bytes: bytes,
file_name: str,
context_token: str = '',
) -> None:
"""Upload a file to CDN and send it."""
import hashlib
media = await self.upload_media(file_bytes, to_user_id, media_type=3)
item = MessageItem(
type=MessageItem.FILE,
file_item=FileItem(
media=media,
file_name=file_name,
md5=hashlib.md5(file_bytes).hexdigest(),
len=str(len(file_bytes)),
),
)
await self.send_message(to_user_id, [item], context_token)
async def send_voice(
self,
to_user_id: str,
voice_bytes: bytes,
playtime: int = 0,
context_token: str = '',
) -> None:
"""Upload a voice message to CDN and send it."""
media = await self.upload_media(voice_bytes, to_user_id, media_type=4)
item = MessageItem(
type=MessageItem.VOICE,
voice_item=VoiceItem(
media=media,
playtime=playtime,
),
)
await self.send_message(to_user_id, [item], context_token)
async def get_upload_url(
self,
filekey: str,
media_type: int,
to_user_id: str,
rawsize: int,
rawfilemd5: str,
filesize: int,
thumb_rawsize: Optional[int] = None,
thumb_rawfilemd5: Optional[str] = None,
thumb_filesize: Optional[int] = None,
aeskey: Optional[str] = None,
) -> GetUploadUrlResponse:
"""Get a pre-signed CDN upload URL."""
payload: dict = {
'filekey': filekey,
'media_type': media_type,
'to_user_id': to_user_id,
'rawsize': rawsize,
'rawfilemd5': rawfilemd5,
'filesize': filesize,
'no_need_thumb': True,
}
if thumb_rawsize is not None:
payload['thumb_rawsize'] = thumb_rawsize
if thumb_rawfilemd5 is not None:
payload['thumb_rawfilemd5'] = thumb_rawfilemd5
if thumb_filesize is not None:
payload['thumb_filesize'] = thumb_filesize
if aeskey is not None:
payload['aeskey'] = aeskey
data = await self._post('ilink/bot/getuploadurl', payload)
logger.debug('get_upload_url response: %s', data)
return GetUploadUrlResponse(
upload_param=data.get('upload_param'),
thumb_upload_param=data.get('thumb_upload_param'),
)
# -----------------------------------------------------------------------
# QR Code Login
# -----------------------------------------------------------------------
async def fetch_qrcode(self, bot_type: str = DEFAULT_BOT_TYPE) -> QRCodeResponse:
"""Fetch a QR code for WeChat login authorization (GET, no auth needed)."""
session = await self._get_session()
url = f'{self.base_url}/ilink/bot/get_bot_qrcode?bot_type={bot_type}'
async with session.get(url, timeout=aiohttp.ClientTimeout(total=DEFAULT_API_TIMEOUT)) as resp:
if resp.status != 200:
text = await resp.text()
raise ApiError(
f'Failed to fetch QR code: {resp.status} {text}',
status=resp.status,
)
data = await resp.json(content_type=None)
logger.debug(
'fetch_qrcode response: qrcode=%s, img=%s', data.get('qrcode'), bool(data.get('qrcode_img_content'))
)
return QRCodeResponse(
qrcode=data.get('qrcode'),
qrcode_img_content=data.get('qrcode_img_content'),
)
async def _fetch_qr_image_base64(self, url: str) -> str:
"""Generate a QR code image from the URL and return a data URI string.
The qrcode_img_content URL points to an HTML page (not a raw image),
so we generate the QR code locally using the qrcode library.
"""
import qrcode
qr = qrcode.QRCode(error_correction=qrcode.constants.ERROR_CORRECT_L)
qr.add_data(url)
qr.make(fit=True)
img = qr.make_image(fill_color='black', back_color='white')
buf = io.BytesIO()
img.save(buf, format='PNG')
b64 = base64.b64encode(buf.getvalue()).decode('utf-8')
return f'data:image/png;base64,{b64}'
async def poll_qrcode_status(self, qrcode: str) -> QRStatusResponse:
"""Long-poll the QR code scan status (GET with iLink-App-ClientVersion header)."""
session = await self._get_session()
url = f'{self.base_url}/ilink/bot/get_qrcode_status?qrcode={quote(qrcode, safe="")}'
headers = {'iLink-App-ClientVersion': '1'}
try:
async with session.get(
url, headers=headers, timeout=aiohttp.ClientTimeout(total=DEFAULT_QR_POLL_TIMEOUT)
) as resp:
if resp.status != 200:
text = await resp.text()
raise ApiError(
f'Failed to poll QR status: {resp.status} {text}',
status=resp.status,
)
data = await resp.json(content_type=None)
logger.debug('QR status poll response: %s', data)
except (asyncio.TimeoutError, aiohttp.ServerTimeoutError):
return QRStatusResponse(status='wait')
return QRStatusResponse(
status=data.get('status'),
bot_token=data.get('bot_token'),
ilink_bot_id=data.get('ilink_bot_id'),
baseurl=data.get('baseurl'),
ilink_user_id=data.get('ilink_user_id'),
)
async def login(
self,
max_retries: int = 5,
poll_timeout_ms: int = 480_000,
on_qrcode: Optional[typing.Callable[[str, str], typing.Any]] = None,
on_status: Optional[typing.Callable[[str], typing.Any]] = None,
) -> LoginResult:
"""Complete QR code login flow with auto-retry on expiry.
Args:
max_retries: Max number of QR code refreshes on expiry.
poll_timeout_ms: Timeout per QR code in milliseconds.
on_qrcode: Callback(qr_image_base64, qr_url) called each time a
new QR code is fetched. Use this to display the QR code.
on_status: Callback(status_str) called on each status poll change.
Returns:
LoginResult with token, base_url, and account_id.
Raises:
ApiError: On unrecoverable API errors.
Exception: If all retries are exhausted.
"""
last_qr_base64: Optional[str] = None
for attempt in range(max_retries):
qr_resp = await self.fetch_qrcode()
if not qr_resp.qrcode or not qr_resp.qrcode_img_content:
raise ApiError('Failed to get QR code from server', status=0)
# Convert QR image to base64 and notify caller
last_qr_base64 = await self._fetch_qr_image_base64(qr_resp.qrcode_img_content)
if on_qrcode:
try:
result = on_qrcode(last_qr_base64, qr_resp.qrcode_img_content)
if asyncio.iscoroutine(result) or asyncio.isfuture(result):
await result
except Exception as e:
logger.warning('on_qrcode callback error: %s', e)
# Poll until confirmed / expired / timeout
loop = asyncio.get_running_loop()
deadline = loop.time() + poll_timeout_ms / 1000.0
while loop.time() < deadline:
try:
status_resp = await self.poll_qrcode_status(qr_resp.qrcode)
except Exception as e:
logger.error('Error polling QR status: %s', e)
await asyncio.sleep(2)
continue
if on_status:
try:
cb_result = on_status(status_resp.status or 'unknown')
if asyncio.iscoroutine(cb_result) or asyncio.isfuture(cb_result):
await cb_result
except Exception as e:
logger.warning('on_status callback error: %s', e)
if status_resp.status == 'confirmed' and status_resp.bot_token:
new_base_url = status_resp.baseurl or self.base_url
# Update this client instance as well
self.token = status_resp.bot_token
self.base_url = new_base_url.rstrip('/')
return LoginResult(
token=status_resp.bot_token,
base_url=new_base_url,
account_id=status_resp.ilink_bot_id or '',
qr_image_base64=last_qr_base64,
)
if status_resp.status == 'expired':
break # retry with a new QR code
await asyncio.sleep(1)
else:
# While-loop ended without break → poll timeout, treat as expired
pass
remaining = max_retries - attempt - 1
if remaining > 0:
logger.info('QR code expired, refreshing... (%d retries left)', remaining)
else:
raise ApiError('QR code login failed: max retries exceeded', status=0)
# Should not reach here, but just in case
raise ApiError('QR code login failed', status=0)
# ---------------------------------------------------------------------------
# Parsing helpers
# ---------------------------------------------------------------------------
def _parse_cdn_media(data: Optional[dict]) -> Optional[CDNMedia]:
if not data:
return None
return CDNMedia(
encrypt_query_param=data.get('encrypt_query_param'),
aes_key=data.get('aes_key'),
encrypt_type=data.get('encrypt_type'),
)
def _parse_message_item(data: dict) -> MessageItem:
item = MessageItem(
type=data.get('type'),
create_time_ms=data.get('create_time_ms'),
update_time_ms=data.get('update_time_ms'),
is_completed=data.get('is_completed'),
msg_id=data.get('msg_id'),
)
if data.get('text_item'):
item.text_item = TextItem(text=data['text_item'].get('text'))
if data.get('image_item'):
img = data['image_item']
item.image_item = ImageItem(
media=_parse_cdn_media(img.get('media')),
thumb_media=_parse_cdn_media(img.get('thumb_media')),
aeskey=img.get('aeskey'),
url=img.get('url'),
mid_size=img.get('mid_size'),
)
if data.get('voice_item'):
v = data['voice_item']
item.voice_item = VoiceItem(
media=_parse_cdn_media(v.get('media')),
encode_type=v.get('encode_type'),
playtime=v.get('playtime'),
text=v.get('text'),
)
if data.get('file_item'):
f = data['file_item']
item.file_item = FileItem(
media=_parse_cdn_media(f.get('media')),
file_name=f.get('file_name'),
md5=f.get('md5'),
len=f.get('len'),
)
if data.get('video_item'):
vid = data['video_item']
item.video_item = VideoItem(
media=_parse_cdn_media(vid.get('media')),
video_size=vid.get('video_size'),
play_length=vid.get('play_length'),
video_md5=vid.get('video_md5'),
thumb_media=_parse_cdn_media(vid.get('thumb_media')),
)
if data.get('ref_msg'):
ref = data['ref_msg']
item.ref_msg = RefMessage(
title=ref.get('title'),
message_item=_parse_message_item(ref['message_item']) if ref.get('message_item') else None,
)
return item
def _parse_weixin_message(data: dict) -> WeixinMessage:
msg = WeixinMessage(
seq=data.get('seq'),
message_id=data.get('message_id'),
from_user_id=data.get('from_user_id'),
to_user_id=data.get('to_user_id'),
client_id=data.get('client_id'),
create_time_ms=data.get('create_time_ms'),
session_id=data.get('session_id'),
group_id=data.get('group_id'),
message_type=data.get('message_type'),
message_state=data.get('message_state'),
context_token=data.get('context_token'),
)
if data.get('item_list'):
msg.item_list = [_parse_message_item(item) for item in data['item_list']]
return msg
def _parse_get_updates_response(data: dict) -> GetUpdatesResponse:
resp = GetUpdatesResponse(
ret=data.get('ret'),
errcode=data.get('errcode'),
errmsg=data.get('errmsg'),
get_updates_buf=data.get('get_updates_buf'),
longpolling_timeout_ms=data.get('longpolling_timeout_ms'),
)
if data.get('msgs'):
resp.msgs = [_parse_weixin_message(m) for m in data['msgs']]
return resp
def _cdn_media_to_dict(media: Optional[CDNMedia]) -> Optional[dict]:
if not media:
return None
d: dict = {}
if media.encrypt_query_param is not None:
d['encrypt_query_param'] = media.encrypt_query_param
if media.aes_key is not None:
d['aes_key'] = media.aes_key
if media.encrypt_type is not None:
d['encrypt_type'] = media.encrypt_type
return d or None
def _message_item_to_dict(item: MessageItem) -> dict:
d: dict = {'type': item.type}
if item.text_item:
d['text_item'] = {'text': item.text_item.text}
if item.image_item:
img_d: dict = {}
if item.image_item.media:
img_d['media'] = _cdn_media_to_dict(item.image_item.media)
if item.image_item.mid_size is not None:
img_d['mid_size'] = item.image_item.mid_size
d['image_item'] = img_d
if item.voice_item:
voice_d: dict = {}
if item.voice_item.media:
voice_d['media'] = _cdn_media_to_dict(item.voice_item.media)
if item.voice_item.playtime is not None:
voice_d['playtime'] = item.voice_item.playtime
d['voice_item'] = voice_d
if item.file_item:
file_d: dict = {}
if item.file_item.media:
file_d['media'] = _cdn_media_to_dict(item.file_item.media)
if item.file_item.file_name:
file_d['file_name'] = item.file_item.file_name
if item.file_item.len:
file_d['len'] = item.file_item.len
d['file_item'] = file_d
if item.video_item:
vid_d: dict = {}
if item.video_item.media:
vid_d['media'] = _cdn_media_to_dict(item.video_item.media)
if item.video_item.video_size is not None:
vid_d['video_size'] = item.video_item.video_size
d['video_item'] = vid_d
return d

View File

@@ -0,0 +1,200 @@
"""Type definitions for the OpenClaw WeChat API, mirroring the upstream protocol."""
from __future__ import annotations
from dataclasses import dataclass, field
from typing import Any, Optional
SESSION_EXPIRED_ERRCODE = -14
class ApiError(Exception):
"""Structured error raised by the OpenClaw WeChat API."""
def __init__(
self,
message: str,
*,
status: int = 0,
code: int | None = None,
payload: Any = None,
):
super().__init__(message)
self.status = status
self.code = code
self.payload = payload
@property
def is_session_expired(self) -> bool:
return self.code == SESSION_EXPIRED_ERRCODE
@dataclass
class CDNMedia:
encrypt_query_param: Optional[str] = None
aes_key: Optional[str] = None
encrypt_type: Optional[int] = None
@dataclass
class TextItem:
text: Optional[str] = None
@dataclass
class ImageItem:
media: Optional[CDNMedia] = None
thumb_media: Optional[CDNMedia] = None
aeskey: Optional[str] = None
url: Optional[str] = None
mid_size: Optional[int] = None
thumb_size: Optional[int] = None
thumb_height: Optional[int] = None
thumb_width: Optional[int] = None
hd_size: Optional[int] = None
_downloaded_bytes: Optional[bytes] = field(default=None, repr=False)
@dataclass
class VoiceItem:
media: Optional[CDNMedia] = None
encode_type: Optional[int] = None
bits_per_sample: Optional[int] = None
sample_rate: Optional[int] = None
playtime: Optional[int] = None
text: Optional[str] = None
_downloaded_bytes: Optional[bytes] = field(default=None, repr=False)
@dataclass
class FileItem:
media: Optional[CDNMedia] = None
file_name: Optional[str] = None
md5: Optional[str] = None
len: Optional[str] = None
_downloaded_bytes: Optional[bytes] = field(default=None, repr=False)
@dataclass
class VideoItem:
media: Optional[CDNMedia] = None
video_size: Optional[int] = None
play_length: Optional[int] = None
video_md5: Optional[str] = None
thumb_media: Optional[CDNMedia] = None
thumb_size: Optional[int] = None
thumb_height: Optional[int] = None
thumb_width: Optional[int] = None
_downloaded_bytes: Optional[bytes] = field(default=None, repr=False)
@dataclass
class RefMessage:
message_item: Optional[MessageItem] = None
title: Optional[str] = None
@dataclass
class MessageItem:
"""A single content item inside a WeixinMessage."""
# Item types
NONE = 0
TEXT = 1
IMAGE = 2
VOICE = 3
FILE = 4
VIDEO = 5
type: Optional[int] = None
create_time_ms: Optional[int] = None
update_time_ms: Optional[int] = None
is_completed: Optional[bool] = None
msg_id: Optional[str] = None
ref_msg: Optional[RefMessage] = None
text_item: Optional[TextItem] = None
image_item: Optional[ImageItem] = None
voice_item: Optional[VoiceItem] = None
file_item: Optional[FileItem] = None
video_item: Optional[VideoItem] = None
@dataclass
class WeixinMessage:
"""Unified message from getUpdates or for sendMessage."""
# Message types
TYPE_USER = 1
TYPE_BOT = 2
# Message states
STATE_NEW = 0
STATE_GENERATING = 1
STATE_FINISH = 2
seq: Optional[int] = None
message_id: Optional[int] = None
from_user_id: Optional[str] = None
to_user_id: Optional[str] = None
client_id: Optional[str] = None
create_time_ms: Optional[int] = None
update_time_ms: Optional[int] = None
delete_time_ms: Optional[int] = None
session_id: Optional[str] = None
group_id: Optional[str] = None
message_type: Optional[int] = None
message_state: Optional[int] = None
item_list: Optional[list[MessageItem]] = None
context_token: Optional[str] = None
@dataclass
class GetUpdatesResponse:
ret: Optional[int] = None
errcode: Optional[int] = None
errmsg: Optional[str] = None
msgs: list[WeixinMessage] = field(default_factory=list)
get_updates_buf: Optional[str] = None
longpolling_timeout_ms: Optional[int] = None
@dataclass
class GetConfigResponse:
ret: Optional[int] = None
errmsg: Optional[str] = None
typing_ticket: Optional[str] = None
@dataclass
class GetUploadUrlResponse:
upload_param: Optional[str] = None
thumb_upload_param: Optional[str] = None
@dataclass
class QRCodeResponse:
"""Response from get_bot_qrcode endpoint."""
qrcode: Optional[str] = None
qrcode_img_content: Optional[str] = None
@dataclass
class QRStatusResponse:
"""Response from get_qrcode_status endpoint."""
status: Optional[str] = None # "wait" | "scaned" | "confirmed" | "expired"
bot_token: Optional[str] = None
ilink_bot_id: Optional[str] = None
baseurl: Optional[str] = None
ilink_user_id: Optional[str] = None
@dataclass
class LoginResult:
"""Result returned by the login flow."""
token: str
base_url: str
account_id: str
qr_image_base64: Optional[str] = None # data URI of the last QR code shown

View File

@@ -6,7 +6,8 @@ import traceback
import uuid
import xml.etree.ElementTree as ET
from dataclasses import dataclass, field
from typing import Any, Callable, Optional
import re
from typing import Any, Callable, Optional, Tuple
from urllib.parse import unquote
import httpx
@@ -63,6 +64,9 @@ class StreamSession:
# 缓存最近一次片段,处理重试或超时兜底
last_chunk: Optional[StreamChunk] = None
# 反馈 ID用于接收用户点赞/点踩反馈
feedback_id: Optional[str] = None
class StreamSessionManager:
"""管理 stream 会话的生命周期,并负责队列的生产消费。"""
@@ -73,6 +77,7 @@ class StreamSessionManager:
self.ttl = ttl # 超时时间(秒),超过该时间未被访问的会话会被清理由 cleanup
self._sessions: dict[str, StreamSession] = {} # stream_id -> StreamSession 映射
self._msg_index: dict[str, str] = {} # msgid -> stream_id 映射,便于流水线根据消息 ID 找到会话
self._feedback_index: dict[str, str] = {} # feedback_id -> stream_id 映射
def get_stream_id_by_msg(self, msg_id: str) -> Optional[str]:
if not msg_id:
@@ -82,6 +87,32 @@ class StreamSessionManager:
def get_session(self, stream_id: str) -> Optional[StreamSession]:
return self._sessions.get(stream_id)
def get_session_by_feedback_id(self, feedback_id: str) -> Optional[StreamSession]:
"""根据 feedback_id 查找会话。
Args:
feedback_id: 企业微信反馈事件中的反馈 ID。
Returns:
Optional[StreamSession]: 找到的会话实例,未找到返回 None。
"""
if not feedback_id:
return None
stream_id = self._feedback_index.get(feedback_id)
if stream_id:
return self._sessions.get(stream_id)
return None
def register_feedback_id(self, stream_id: str, feedback_id: str) -> None:
"""注册 feedback_id 与 stream_id 的映射。
Args:
stream_id: 企业微信流式会话 ID。
feedback_id: 反馈 ID。
"""
if feedback_id and stream_id:
self._feedback_index[feedback_id] = stream_id
def create_or_get(self, msg_json: dict[str, Any]) -> tuple[StreamSession, bool]:
"""根据企业微信回调创建或获取会话。
@@ -199,6 +230,366 @@ class StreamSessionManager:
self._msg_index.pop(msg_id, None)
def _decrypt_file(encrypted_data: bytes, aes_key_str: str) -> bytes:
"""Decrypt AES-256-CBC encrypted file data.
Aligned with the official WeCom AI Bot Python SDK (crypto_utils.py).
Args:
encrypted_data: The raw encrypted bytes.
aes_key_str: Base64-encoded AES key (may lack padding).
Returns:
Decrypted bytes with PKCS#7 padding removed.
"""
if not encrypted_data:
raise ValueError('encrypted_data is empty')
if not aes_key_str:
raise ValueError('aes_key is empty')
# Python's base64.b64decode requires proper padding (length % 4 == 0).
# Node.js Buffer.from tolerates missing '=', so we must pad manually.
remainder = len(aes_key_str) % 4
if remainder != 0:
aes_key_str = aes_key_str + '=' * (4 - remainder)
key = base64.b64decode(aes_key_str)
iv = key[:16]
cipher = AES.new(key, AES.MODE_CBC, iv)
# Ensure encrypted data is aligned to AES block size (16 bytes).
# Node.js setAutoPadding(false) silently handles unaligned data,
# but PyCryptodome will raise an error.
block_size = 16
data_remainder = len(encrypted_data) % block_size
if data_remainder != 0:
encrypted_data = encrypted_data + b'\x00' * (block_size - data_remainder)
decrypted = cipher.decrypt(encrypted_data)
# Remove PKCS#7 padding with validation
if len(decrypted) == 0:
raise ValueError('Decrypted data is empty')
pad_len = decrypted[-1]
if pad_len < 1 or pad_len > 32 or pad_len > len(decrypted):
raise ValueError(f'Invalid PKCS#7 padding value: {pad_len}')
# Verify all padding bytes are consistent
for i in range(len(decrypted) - pad_len, len(decrypted)):
if decrypted[i] != pad_len:
raise ValueError('Invalid PKCS#7 padding: padding bytes mismatch')
return decrypted[: len(decrypted) - pad_len]
def _extract_filename(content_disposition: str) -> Optional[str]:
"""Extract filename from a Content-Disposition header value."""
if not content_disposition:
return None
# RFC 5987: filename*=UTF-8''xxx
utf8_match = re.search(r"filename\*=UTF-8''([^;\s]+)", content_disposition, re.IGNORECASE)
if utf8_match:
return unquote(utf8_match.group(1))
# Standard: filename="xxx" or filename=xxx
match = re.search(r'filename="?([^";\s]+)"?', content_disposition, re.IGNORECASE)
if match:
return unquote(match.group(1))
return None
def _bytes_to_data_uri(data: bytes) -> str:
"""Convert raw bytes to a data URI with auto-detected MIME type."""
if data.startswith(b'\xff\xd8'):
mime_type = 'image/jpeg'
elif data.startswith(b'\x89PNG'):
mime_type = 'image/png'
elif data.startswith((b'GIF87a', b'GIF89a')):
mime_type = 'image/gif'
elif data.startswith(b'BM'):
mime_type = 'image/bmp'
elif data.startswith(b'II*\x00') or data.startswith(b'MM\x00*'):
mime_type = 'image/tiff'
elif data[:4] == b'%PDF':
mime_type = 'application/pdf'
elif data[:4] == b'PK\x03\x04':
mime_type = 'application/zip'
else:
mime_type = 'application/octet-stream'
base64_str = base64.b64encode(data).decode('utf-8')
return f'data:{mime_type};base64,{base64_str}'
async def download_encrypted_file(
download_url: str, aes_key: str, logger: EventLogger
) -> Tuple[Optional[bytes], Optional[str]]:
"""Download an AES-encrypted file from WeChat Work and decrypt it.
Args:
download_url: The encrypted file download URL.
aes_key: The AES key for decryption (base64-encoded, per-message aeskey
or platform EncodingAESKey).
logger: Logger instance.
Returns:
A tuple of (decrypted_bytes, filename) or (None, None) on failure.
"""
if not download_url:
return None, None
if not aes_key:
await logger.error('download_encrypted_file: aes_key is empty, cannot decrypt')
return None, None
filename: Optional[str] = None
try:
async with httpx.AsyncClient(timeout=30.0) as client:
response = await client.get(download_url)
if response.status_code != 200:
await logger.error(f'Failed to download file (HTTP {response.status_code}): {response.text[:200]}')
return None, None
encrypted_bytes = response.content
filename = _extract_filename(response.headers.get('content-disposition', ''))
except Exception:
await logger.error(f'Failed to download file: {traceback.format_exc()}')
return None, None
try:
decrypted = _decrypt_file(encrypted_bytes, aes_key)
return decrypted, filename
except Exception:
await logger.error(f'Failed to decrypt file: {traceback.format_exc()}')
return None, None
async def parse_wecom_bot_message(
msg_json: dict[str, Any], encoding_aes_key: str, logger: EventLogger
) -> dict[str, Any]:
"""Parse a decrypted WeChat Work AI Bot message JSON into a unified message dict.
This is the shared message parsing logic used by both webhook and WebSocket modes.
Args:
msg_json: The decrypted message JSON from WeChat Work.
encoding_aes_key: AES key for file decryption.
logger: Logger instance.
Returns:
A dict suitable for constructing a WecomBotEvent.
"""
message_data: dict[str, Any] = {}
msg_type = msg_json.get('msgtype', '')
if msg_type:
message_data['msgtype'] = msg_type
if msg_json.get('chattype', '') == 'single':
message_data['type'] = 'single'
elif msg_json.get('chattype', '') == 'group':
message_data['type'] = 'group'
max_inline_file_size = 5 * 1024 * 1024
async def _safe_download(url: str, per_msg_aeskey: str = '') -> Tuple[Optional[bytes], Optional[str]]:
"""Download and decrypt a file, preferring per-message aeskey over platform key."""
if not url:
return None, None
key = per_msg_aeskey or encoding_aes_key
if not key:
await logger.warning('No AES key available for file decryption, skipping download')
return None, None
return await download_encrypted_file(url, key, logger)
async def _safe_download_as_data_uri(url: str, per_msg_aeskey: str = '') -> Optional[str]:
"""Download, decrypt, and convert to data URI for backward compatibility."""
data, _filename = await _safe_download(url, per_msg_aeskey)
if data:
return _bytes_to_data_uri(data)
return None
if msg_type == 'text':
message_data['content'] = msg_json.get('text', {}).get('content')
elif msg_type == 'markdown':
message_data['content'] = msg_json.get('markdown', {}).get('content') or msg_json.get('text', {}).get(
'content', ''
)
elif msg_type == 'image':
image_info = msg_json.get('image', {})
picurl = image_info.get('url', '')
per_msg_aeskey = image_info.get('aeskey', '')
base64_data = await _safe_download_as_data_uri(picurl, per_msg_aeskey)
if base64_data:
message_data['picurl'] = base64_data
message_data['images'] = [base64_data]
elif msg_type == 'voice':
voice_info = msg_json.get('voice', {}) or {}
download_url = voice_info.get('url')
per_msg_aeskey = voice_info.get('aeskey', '')
message_data['voice'] = {
'url': download_url,
'md5sum': voice_info.get('md5sum') or voice_info.get('md5'),
'filesize': voice_info.get('filesize') or voice_info.get('size'),
'sdkfileid': voice_info.get('sdkfileid') or voice_info.get('fileid'),
}
if voice_info.get('content'):
message_data['content'] = voice_info.get('content')
if (message_data['voice'].get('filesize') or 0) <= max_inline_file_size:
voice_base64 = await _safe_download_as_data_uri(download_url, per_msg_aeskey)
if voice_base64:
message_data['voice']['base64'] = voice_base64
elif msg_type == 'video':
video_info = msg_json.get('video', {}) or {}
download_url = video_info.get('url')
per_msg_aeskey = video_info.get('aeskey', '')
video_data = {
'url': download_url,
'filesize': video_info.get('filesize') or video_info.get('size'),
'sdkfileid': video_info.get('sdkfileid') or video_info.get('fileid'),
'md5sum': video_info.get('md5sum') or video_info.get('md5'),
'filename': video_info.get('filename') or video_info.get('name'),
}
if (video_data.get('filesize') or 0) <= max_inline_file_size:
video_base64 = await _safe_download_as_data_uri(download_url, per_msg_aeskey)
if video_base64:
video_data['base64'] = video_base64
message_data['video'] = video_data
elif msg_type == 'file':
file_info = msg_json.get('file', {}) or {}
download_url = file_info.get('url') or file_info.get('fileurl')
per_msg_aeskey = file_info.get('aeskey', '')
file_data = {
'filename': file_info.get('filename') or file_info.get('name'),
'filesize': file_info.get('filesize') or file_info.get('size'),
'md5sum': file_info.get('md5sum') or file_info.get('md5'),
'sdkfileid': file_info.get('sdkfileid') or file_info.get('fileid'),
'download_url': download_url,
'extra': file_info,
}
if (file_data.get('filesize') or 0) <= max_inline_file_size:
file_bytes, dl_filename = await _safe_download(download_url, per_msg_aeskey)
if file_bytes:
file_data['base64'] = _bytes_to_data_uri(file_bytes)
if dl_filename and not file_data.get('filename'):
file_data['filename'] = dl_filename
message_data['file'] = file_data
elif msg_type == 'link':
message_data['link'] = msg_json.get('link', {})
if not message_data.get('content'):
title = message_data['link'].get('title', '')
desc = message_data['link'].get('description') or message_data['link'].get('digest', '')
message_data['content'] = '\n'.join(filter(None, [title, desc]))
elif msg_type == 'mixed':
items = msg_json.get('mixed', {}).get('msg_item', [])
texts = []
images = []
files = []
voices = []
videos = []
links = []
for item in items:
item_type = item.get('msgtype')
if item_type == 'text':
texts.append(item.get('text', {}).get('content', ''))
elif item_type == 'image':
img_info = item.get('image', {})
img_url = img_info.get('url')
img_aeskey = img_info.get('aeskey', '')
base64_data = await _safe_download_as_data_uri(img_url, img_aeskey)
if base64_data:
images.append(base64_data)
elif item_type == 'file':
file_info = item.get('file', {}) or {}
download_url = file_info.get('url') or file_info.get('fileurl')
item_aeskey = file_info.get('aeskey', '')
file_data = {
'filename': file_info.get('filename') or file_info.get('name'),
'filesize': file_info.get('filesize') or file_info.get('size'),
'md5sum': file_info.get('md5sum') or file_info.get('md5'),
'sdkfileid': file_info.get('sdkfileid') or file_info.get('fileid'),
'download_url': download_url,
'extra': file_info,
}
if (file_data.get('filesize') or 0) <= max_inline_file_size:
file_bytes, dl_filename = await _safe_download(download_url, item_aeskey)
if file_bytes:
file_data['base64'] = _bytes_to_data_uri(file_bytes)
if dl_filename and not file_data.get('filename'):
file_data['filename'] = dl_filename
files.append(file_data)
elif item_type == 'voice':
voice_info = item.get('voice', {}) or {}
download_url = voice_info.get('url')
item_aeskey = voice_info.get('aeskey', '')
voice_data = {
'url': download_url,
'md5sum': voice_info.get('md5sum') or voice_info.get('md5'),
'filesize': voice_info.get('filesize') or voice_info.get('size'),
'sdkfileid': voice_info.get('sdkfileid') or voice_info.get('fileid'),
}
if voice_info.get('content'):
texts.append(voice_info.get('content'))
if (voice_data.get('filesize') or 0) <= max_inline_file_size:
voice_base64 = await _safe_download_as_data_uri(download_url, item_aeskey)
if voice_base64:
voice_data['base64'] = voice_base64
voices.append(voice_data)
elif item_type == 'video':
video_info = item.get('video', {}) or {}
download_url = video_info.get('url')
item_aeskey = video_info.get('aeskey', '')
video_data = {
'url': download_url,
'filesize': video_info.get('filesize') or video_info.get('size'),
'sdkfileid': video_info.get('sdkfileid') or video_info.get('fileid'),
'md5sum': video_info.get('md5sum') or video_info.get('md5'),
'filename': video_info.get('filename') or video_info.get('name'),
}
if (video_data.get('filesize') or 0) <= max_inline_file_size:
video_base64 = await _safe_download_as_data_uri(download_url, item_aeskey)
if video_base64:
video_data['base64'] = video_base64
videos.append(video_data)
elif item_type == 'link':
links.append(item.get('link', {}))
if texts:
message_data['content'] = ' '.join(texts)
if images:
message_data['images'] = images
message_data['picurl'] = images[0]
if files:
message_data['files'] = files
message_data['file'] = files[0]
if voices:
message_data['voices'] = voices
message_data['voice'] = voices[0]
if videos:
message_data['videos'] = videos
message_data['video'] = videos[0]
if links:
message_data['link'] = links[0]
if items:
message_data['attachments'] = items
else:
message_data['raw_msg'] = msg_json
from_info = msg_json.get('from', {})
message_data['userid'] = from_info.get('userid', '')
message_data['username'] = from_info.get('alias', '') or from_info.get('name', '') or from_info.get('userid', '')
if msg_json.get('chattype', '') == 'group':
message_data['chatid'] = msg_json.get('chatid', '')
message_data['chatname'] = msg_json.get('chatname', '') or msg_json.get('chatid', '')
message_data['msgid'] = msg_json.get('msgid', '')
if msg_json.get('aibotid'):
message_data['aibotid'] = msg_json.get('aibotid', '')
return message_data
class WecomBotClient:
def __init__(self, Token: str, EnCodingAESKey: str, Corpid: str, logger: EventLogger, unified_mode: bool = False):
"""企业微信智能机器人客户端。
@@ -236,14 +627,27 @@ class WecomBotClient:
self.stream_sessions = StreamSessionManager(logger=logger)
self.stream_poll_timeout = 0.5
self._feedback_callback: Optional[Callable] = None
def set_feedback_callback(self, callback: Callable) -> None:
"""设置反馈回调函数。
Args:
callback: 反馈回调函数,签名: async def callback(feedback_id, feedback_type, feedback_content, inaccurate_reasons, session)
"""
self._feedback_callback = callback
@staticmethod
def _build_stream_payload(stream_id: str, content: str, finish: bool) -> dict[str, Any]:
def _build_stream_payload(
stream_id: str, content: str, finish: bool, feedback_id: Optional[str] = None
) -> dict[str, Any]:
"""按照企业微信协议拼装返回报文。
Args:
stream_id: 企业微信会话 ID。
content: 推送的文本内容。
finish: 是否为最终片段。
feedback_id: 反馈 ID用于接收用户点赞/点踩反馈。
Returns:
dict[str, Any]: 可直接加密返回的 payload。
@@ -251,13 +655,16 @@ class WecomBotClient:
Example:
组装 `{'msgtype': 'stream', 'stream': {'id': 'sid', ...}}` 结构。
"""
stream_payload = {
'id': stream_id,
'finish': finish,
'content': content,
}
if feedback_id:
stream_payload['feedback'] = {'id': feedback_id}
return {
'msgtype': 'stream',
'stream': {
'id': stream_id,
'finish': finish,
'content': content,
},
'stream': stream_payload,
}
async def _encrypt_and_reply(self, payload: dict[str, Any], nonce: str) -> tuple[Response, int]:
@@ -313,9 +720,14 @@ class WecomBotClient:
"""
session, is_new = self.stream_sessions.create_or_get(msg_json)
feedback_id = str(uuid.uuid4())
session.feedback_id = feedback_id
self.stream_sessions.register_feedback_id(session.stream_id, feedback_id)
message_data = await self.get_message(msg_json)
if message_data:
message_data['stream_id'] = session.stream_id
message_data['feedback_id'] = feedback_id
try:
event = wecombotevent.WecomBotEvent(message_data)
except Exception:
@@ -324,7 +736,7 @@ class WecomBotClient:
if is_new:
asyncio.create_task(self._dispatch_event(event))
payload = self._build_stream_payload(session.stream_id, '', False)
payload = self._build_stream_payload(session.stream_id, '', False, feedback_id)
return await self._encrypt_and_reply(payload, nonce)
async def _handle_post_followup_response(self, msg_json: dict[str, Any], nonce: str) -> tuple[Response, int]:
@@ -449,202 +861,80 @@ class WecomBotClient:
msg_json = json.loads(decrypted_xml)
event = msg_json.get('event', {})
event_type = event.get('eventtype', '')
if event_type == 'feedback_event':
return await self._handle_feedback_event(msg_json, nonce)
if msg_json.get('msgtype') == 'stream':
return await self._handle_post_followup_response(msg_json, nonce)
return await self._handle_post_initial_response(msg_json, nonce)
async def get_message(self, msg_json):
message_data = {}
async def _handle_feedback_event(self, msg_json: dict[str, Any], nonce: str) -> tuple[Response, int]:
"""处理企业微信用户反馈事件(点赞/点踩)。
msg_type = msg_json.get('msgtype', '')
if msg_type:
message_data['msgtype'] = msg_type
Args:
msg_json: 解密后的企业微信反馈事件 JSON。
nonce: 企业微信回调参数 nonce。
if msg_json.get('chattype', '') == 'single':
message_data['type'] = 'single'
elif msg_json.get('chattype', '') == 'group':
message_data['type'] = 'group'
Returns:
Tuple[Response, int]: Quart Response 及状态码。
max_inline_file_size = 5 * 1024 * 1024 # avoid decoding very large payloads by default
Note:
企业微信协议要求:反馈事件目前仅支持回复空包。
"""
try:
feedback_event = msg_json.get('event', {}).get('feedback_event', {})
feedback_id = feedback_event.get('id', '')
feedback_type = feedback_event.get('type', 0)
feedback_content = feedback_event.get('content', '')
inaccurate_reasons = feedback_event.get('inaccurate_reason_list', [])
async def _safe_download(url: str):
if not url:
return None
return await self.download_url_to_base64(url, self.EnCodingAESKey)
if msg_type == 'text':
message_data['content'] = msg_json.get('text', {}).get('content')
elif msg_type == 'markdown':
message_data['content'] = msg_json.get('markdown', {}).get('content') or msg_json.get('text', {}).get(
'content', ''
await self.logger.info(
f'收到用户反馈事件: feedback_id={feedback_id}, type={feedback_type}, '
f'content={feedback_content}, reasons={inaccurate_reasons}'
)
elif msg_type == 'image':
picurl = msg_json.get('image', {}).get('url', '')
base64_data = await _safe_download(picurl)
if base64_data:
message_data['picurl'] = base64_data
message_data['images'] = [base64_data]
elif msg_type == 'voice':
voice_info = msg_json.get('voice', {}) or {}
download_url = voice_info.get('url')
message_data['voice'] = {
'url': download_url,
'md5sum': voice_info.get('md5sum') or voice_info.get('md5'),
'filesize': voice_info.get('filesize') or voice_info.get('size'),
'sdkfileid': voice_info.get('sdkfileid') or voice_info.get('fileid'),
}
# 企业微信智能转写文本(如果已有)直接复用,避免重复转写
if voice_info.get('content'):
message_data['content'] = voice_info.get('content')
if (message_data['voice'].get('filesize') or 0) <= max_inline_file_size:
voice_base64 = await _safe_download(download_url)
if voice_base64:
message_data['voice']['base64'] = voice_base64
elif msg_type == 'video':
video_info = msg_json.get('video', {}) or {}
download_url = video_info.get('url')
video_data = {
'url': download_url,
'filesize': video_info.get('filesize') or video_info.get('size'),
'sdkfileid': video_info.get('sdkfileid') or video_info.get('fileid'),
'md5sum': video_info.get('md5sum') or video_info.get('md5'),
'filename': video_info.get('filename') or video_info.get('name'),
}
if (video_data.get('filesize') or 0) <= max_inline_file_size:
video_base64 = await _safe_download(download_url)
if video_base64:
video_data['base64'] = video_base64
message_data['video'] = video_data
elif msg_type == 'file':
file_info = msg_json.get('file', {}) or {}
download_url = file_info.get('url') or file_info.get('fileurl')
file_data = {
'filename': file_info.get('filename') or file_info.get('name'),
'filesize': file_info.get('filesize') or file_info.get('size'),
'md5sum': file_info.get('md5sum') or file_info.get('md5'),
'sdkfileid': file_info.get('sdkfileid') or file_info.get('fileid'),
'download_url': download_url,
'extra': file_info,
}
if (file_data.get('filesize') or 0) <= max_inline_file_size:
file_base64 = await _safe_download(download_url)
if file_base64:
file_data['base64'] = file_base64
message_data['file'] = file_data
elif msg_type == 'link':
message_data['link'] = msg_json.get('link', {})
if not message_data.get('content'):
title = message_data['link'].get('title', '')
desc = message_data['link'].get('description') or message_data['link'].get('digest', '')
message_data['content'] = '\n'.join(filter(None, [title, desc]))
elif msg_type == 'mixed':
items = msg_json.get('mixed', {}).get('msg_item', [])
texts = []
images = []
files = []
voices = []
videos = []
links = []
for item in items:
item_type = item.get('msgtype')
if item_type == 'text':
texts.append(item.get('text', {}).get('content', ''))
elif item_type == 'image':
img_url = item.get('image', {}).get('url')
base64_data = await _safe_download(img_url)
if base64_data:
images.append(base64_data)
elif item_type == 'file':
file_info = item.get('file', {}) or {}
download_url = file_info.get('url') or file_info.get('fileurl')
file_data = {
'filename': file_info.get('filename') or file_info.get('name'),
'filesize': file_info.get('filesize') or file_info.get('size'),
'md5sum': file_info.get('md5sum') or file_info.get('md5'),
'sdkfileid': file_info.get('sdkfileid') or file_info.get('fileid'),
'download_url': download_url,
'extra': file_info,
}
if (file_data.get('filesize') or 0) <= max_inline_file_size:
file_base64 = await _safe_download(download_url)
if file_base64:
file_data['base64'] = file_base64
files.append(file_data)
elif item_type == 'voice':
voice_info = item.get('voice', {}) or {}
download_url = voice_info.get('url')
voice_data = {
'url': download_url,
'md5sum': voice_info.get('md5sum') or voice_info.get('md5'),
'filesize': voice_info.get('filesize') or voice_info.get('size'),
'sdkfileid': voice_info.get('sdkfileid') or voice_info.get('fileid'),
}
if voice_info.get('content'):
texts.append(voice_info.get('content'))
if (voice_data.get('filesize') or 0) <= max_inline_file_size:
voice_base64 = await _safe_download(download_url)
if voice_base64:
voice_data['base64'] = voice_base64
voices.append(voice_data)
elif item_type == 'video':
video_info = item.get('video', {}) or {}
download_url = video_info.get('url')
video_data = {
'url': download_url,
'filesize': video_info.get('filesize') or video_info.get('size'),
'sdkfileid': video_info.get('sdkfileid') or video_info.get('fileid'),
'md5sum': video_info.get('md5sum') or video_info.get('md5'),
'filename': video_info.get('filename') or video_info.get('name'),
}
if (video_data.get('filesize') or 0) <= max_inline_file_size:
video_base64 = await _safe_download(download_url)
if video_base64:
video_data['base64'] = video_base64
videos.append(video_data)
elif item_type == 'link':
links.append(item.get('link', {}))
if texts:
message_data['content'] = ' '.join(texts) # 拼接所有 text
if images:
message_data['images'] = images
message_data['picurl'] = images[0] # 只保留第一个 image
if files:
message_data['files'] = files
message_data['file'] = files[0]
if voices:
message_data['voices'] = voices
message_data['voice'] = voices[0]
if videos:
message_data['videos'] = videos
message_data['video'] = videos[0]
if links:
message_data['link'] = links[0]
if items:
message_data['attachments'] = items
else:
message_data['raw_msg'] = msg_json
session = self.stream_sessions.get_session_by_feedback_id(feedback_id)
if session:
await self.logger.info(
f'反馈关联到会话: stream_id={session.stream_id}, msg_id={session.msg_id}, user_id={session.user_id}'
)
for handler in self._message_handlers.get('feedback', []):
try:
await handler(
feedback_id=feedback_id,
feedback_type=feedback_type,
feedback_content=feedback_content,
inaccurate_reasons=inaccurate_reasons,
session=session,
)
except Exception:
await self.logger.error(traceback.format_exc())
# Extract user information
from_info = msg_json.get('from', {})
message_data['userid'] = from_info.get('userid', '')
message_data['username'] = (
from_info.get('alias', '') or from_info.get('name', '') or from_info.get('userid', '')
)
if self._feedback_callback:
try:
await self._feedback_callback(
feedback_id=feedback_id,
feedback_type=feedback_type,
feedback_content=feedback_content,
inaccurate_reasons=inaccurate_reasons,
session=session,
)
except Exception:
await self.logger.error(traceback.format_exc())
else:
await self.logger.warning(f'未找到 feedback_id={feedback_id} 对应的会话')
# Extract chat/group information
if msg_json.get('chattype', '') == 'group':
message_data['chatid'] = msg_json.get('chatid', '')
# Try to get group name if available
message_data['chatname'] = msg_json.get('chatname', '') or msg_json.get('chatid', '')
except Exception:
await self.logger.error(traceback.format_exc())
message_data['msgid'] = msg_json.get('msgid', '')
return await self._encrypt_and_reply({}, nonce)
if msg_json.get('aibotid'):
message_data['aibotid'] = msg_json.get('aibotid', '')
return message_data
async def get_message(self, msg_json):
return await parse_wecom_bot_message(msg_json, self.EnCodingAESKey, self.logger)
async def _handle_message(self, event: wecombotevent.WecomBotEvent):
"""
@@ -711,40 +1001,20 @@ class WecomBotClient:
return decorator
def on_feedback(self):
def decorator(func: Callable):
if 'feedback' not in self._message_handlers:
self._message_handlers['feedback'] = []
self._message_handlers['feedback'].append(func)
return func
return decorator
async def download_url_to_base64(self, download_url, encoding_aes_key):
async with httpx.AsyncClient() as client:
response = await client.get(download_url)
if response.status_code != 200:
await self.logger.error(f'failed to get file: {response.text}')
return None
encrypted_bytes = response.content
aes_key = base64.b64decode(encoding_aes_key + '=') # base64 补齐
iv = aes_key[:16]
cipher = AES.new(aes_key, AES.MODE_CBC, iv)
decrypted = cipher.decrypt(encrypted_bytes)
pad_len = decrypted[-1]
decrypted = decrypted[:-pad_len]
if decrypted.startswith(b'\xff\xd8'): # JPEG
mime_type = 'image/jpeg'
elif decrypted.startswith(b'\x89PNG'): # PNG
mime_type = 'image/png'
elif decrypted.startswith((b'GIF87a', b'GIF89a')): # GIF
mime_type = 'image/gif'
elif decrypted.startswith(b'BM'): # BMP
mime_type = 'image/bmp'
elif decrypted.startswith(b'II*\x00') or decrypted.startswith(b'MM\x00*'): # TIFF
mime_type = 'image/tiff'
else:
mime_type = 'application/octet-stream'
# 转 base64
base64_str = base64.b64encode(decrypted).decode('utf-8')
return f'data:{mime_type};base64,{base64_str}'
data, _filename = await download_encrypted_file(download_url, encoding_aes_key, self.logger)
if data:
return _bytes_to_data_uri(data)
return None
async def run_task(self, host: str, port: int, *args, **kwargs):
"""

View File

@@ -133,3 +133,17 @@ class WecomBotEvent(dict):
AI Bot ID
"""
return self.get('aibotid', '')
@property
def feedback_id(self) -> str:
"""
反馈 ID用于关联用户点赞/点踩反馈
"""
return self.get('feedback_id', '')
@property
def stream_id(self) -> str:
"""
流式消息 ID
"""
return self.get('stream_id', '')

View File

@@ -0,0 +1,596 @@
"""WeChat Work AI Bot WebSocket long connection client.
Implements the WebSocket protocol for receiving messages and sending replies
via a persistent connection to wss://openws.work.weixin.qq.com, as an
alternative to the HTTP callback (webhook) mode.
Protocol reference: https://developer.work.weixin.qq.com/document/path/101463
Official Node.js SDK: https://github.com/WecomTeam/aibot-node-sdk
"""
from __future__ import annotations
import asyncio
import json
import secrets
import time
import traceback
from typing import Any, Callable, Optional
import aiohttp
from langbot.libs.wecom_ai_bot_api import wecombotevent
from langbot.libs.wecom_ai_bot_api.api import parse_wecom_bot_message
from langbot.pkg.platform.logger import EventLogger
DEFAULT_WS_URL = 'wss://openws.work.weixin.qq.com'
# WebSocket frame command constants
CMD_SUBSCRIBE = 'aibot_subscribe'
CMD_HEARTBEAT = 'ping'
CMD_MSG_CALLBACK = 'aibot_msg_callback'
CMD_EVENT_CALLBACK = 'aibot_event_callback'
CMD_RESPOND_MSG = 'aibot_respond_msg'
CMD_RESPOND_WELCOME = 'aibot_respond_welcome_msg'
CMD_RESPOND_UPDATE = 'aibot_respond_update_msg'
CMD_SEND_MSG = 'aibot_send_msg'
def _generate_req_id(prefix: str) -> str:
"""Generate a unique request ID in the format: {prefix}_{timestamp}_{random}."""
ts = int(time.time() * 1000)
rand = secrets.token_hex(4)
return f'{prefix}_{ts}_{rand}'
class WecomBotWsClient:
"""WeChat Work AI Bot WebSocket long connection client.
Provides message receiving, streaming reply, proactive message sending,
and event callback handling over a persistent WebSocket connection.
"""
def __init__(
self,
bot_id: str,
secret: str,
logger: EventLogger,
encoding_aes_key: str = '',
ws_url: str = DEFAULT_WS_URL,
heartbeat_interval: float = 30.0,
max_reconnect_attempts: int = -1,
reconnect_base_delay: float = 1.0,
reconnect_max_delay: float = 30.0,
):
self.bot_id = bot_id
self.secret = secret
self.logger = logger
self.encoding_aes_key = encoding_aes_key
self.ws_url = ws_url
self.heartbeat_interval = heartbeat_interval
self.max_reconnect_attempts = max_reconnect_attempts
self.reconnect_base_delay = reconnect_base_delay
self.reconnect_max_delay = reconnect_max_delay
self._ws: Optional[aiohttp.ClientWebSocketResponse] = None
self._session: Optional[aiohttp.ClientSession] = None
self._running = False
self._heartbeat_task: Optional[asyncio.Task] = None
self._missed_pong_count = 0
self._max_missed_pong = 2
self._reconnect_attempts = 0
# Message handler registry (same pattern as WecomBotClient)
self._message_handlers: dict[str, list[Callable]] = {}
# Message deduplication
self._msg_id_map: dict[str, int] = {}
# Pending ACK futures: req_id -> Future[dict]
self._pending_acks: dict[str, asyncio.Future] = {}
# Per-req_id serial reply queues
self._reply_queues: dict[str, asyncio.Queue] = {}
self._reply_workers: dict[str, asyncio.Task] = {}
self._reply_ack_timeout = 5.0
# Stream ID tracking for WebSocket mode
self._stream_ids: dict[str, str] = {} # msg_id -> req_id|stream_id
# Dedup: skip sending when content hasn't changed
self._stream_last_content: dict[str, str] = {} # msg_id -> last content sent
# ── Public API ──────────────────────────────────────────────────
async def connect(self):
"""Connect to WebSocket server with automatic reconnection.
This method blocks until disconnect() is called or max reconnect
attempts are exhausted.
"""
self._running = True
self._reconnect_attempts = 0
while self._running:
try:
await self._connect_once()
except Exception:
if not self._running:
break
await self.logger.error(f'WebSocket connection error: {traceback.format_exc()}')
if not self._running:
break
# Reconnect with exponential backoff
if self.max_reconnect_attempts != -1 and self._reconnect_attempts >= self.max_reconnect_attempts:
await self.logger.error(f'Max reconnect attempts reached ({self.max_reconnect_attempts}), giving up')
break
self._reconnect_attempts += 1
delay = min(
self.reconnect_base_delay * (2 ** (self._reconnect_attempts - 1)),
self.reconnect_max_delay,
)
await self.logger.info(f'Reconnecting in {delay:.1f}s (attempt {self._reconnect_attempts})...')
await asyncio.sleep(delay)
async def disconnect(self):
"""Gracefully disconnect from the WebSocket server."""
self._running = False
if self._heartbeat_task and not self._heartbeat_task.done():
self._heartbeat_task.cancel()
for task in self._reply_workers.values():
if not task.done():
task.cancel()
if self._ws and not self._ws.closed:
await self._ws.close()
self._ws = None
if self._session and not self._session.closed:
await self._session.close()
self._session = None
def on_message(self, msg_type: str) -> Callable:
"""Decorator to register a message handler.
Same interface as WecomBotClient.on_message for compatibility.
Args:
msg_type: 'single', 'group', or specific message type.
"""
def decorator(func: Callable[[wecombotevent.WecomBotEvent], Any]):
if msg_type not in self._message_handlers:
self._message_handlers[msg_type] = []
self._message_handlers[msg_type].append(func)
return func
return decorator
async def reply_stream(
self,
req_id: str,
stream_id: str,
content: str,
finish: bool = False,
) -> Optional[dict]:
"""Send a streaming reply frame.
Args:
req_id: The req_id from the original message frame (must be passed through).
stream_id: The stream ID for this streaming session.
content: The content to send (supports Markdown).
finish: Whether this is the final chunk.
Returns:
The ACK frame dict, or None on failure.
"""
body = {
'msgtype': 'stream',
'stream': {
'id': stream_id,
'finish': finish,
'content': content,
},
}
return await self._send_reply(req_id, body)
async def reply_text(self, req_id: str, content: str) -> Optional[dict]:
"""Send a non-streaming text reply.
Args:
req_id: The req_id from the original message frame.
content: The text content to reply.
Returns:
The ACK frame dict, or None on failure.
"""
body = {
'msgtype': 'markdown',
'markdown': {
'content': content,
},
}
return await self._send_reply(req_id, body)
async def send_message(self, chat_id: str, content: str, msgtype: str = 'markdown') -> Optional[dict]:
"""Proactively send a message to a specified chat.
Args:
chat_id: The chat ID (userid for single chat, chatid for group chat).
content: The message content.
msgtype: Message type, 'markdown' by default.
Returns:
The ACK frame dict, or None on failure.
"""
req_id = _generate_req_id(CMD_SEND_MSG)
body: dict[str, Any] = {
'chatid': chat_id,
'msgtype': msgtype,
}
if msgtype == 'markdown':
body['markdown'] = {'content': content}
elif msgtype == 'text':
body['text'] = {'content': content}
return await self._send_reply(req_id, body, cmd=CMD_SEND_MSG)
async def push_stream_chunk(self, msg_id: str, content: str, is_final: bool = False) -> bool:
"""Push a streaming chunk for a given message ID.
Compatible interface with WecomBotClient.push_stream_chunk.
Args:
msg_id: The original message ID.
content: The cumulative content from the pipeline.
is_final: Whether this is the final chunk.
Returns:
True if the stream session exists and chunk was sent.
"""
key = self._stream_ids.get(msg_id)
if not key:
return False
req_id, stream_id = key.split('|', 1)
try:
# Skip sending if content hasn't changed (e.g. during tool call argument streaming)
if not is_final and content == self._stream_last_content.get(msg_id):
return True
await self.reply_stream(req_id, stream_id, content, finish=is_final)
self._stream_last_content[msg_id] = content
if is_final:
self._stream_ids.pop(msg_id, None)
self._stream_last_content.pop(msg_id, None)
return True
except Exception:
await self.logger.error(f'Failed to push stream chunk: {traceback.format_exc()}')
return False
async def set_message(self, msg_id: str, content: str):
"""Fallback: send content as a final stream chunk or direct reply.
Compatible interface with WecomBotClient.set_message.
"""
handled = await self.push_stream_chunk(msg_id, content, is_final=True)
if not handled:
await self.logger.warning(f'No active stream for msg_id={msg_id}, message dropped')
# ── Connection lifecycle ────────────────────────────────────────
async def _connect_once(self):
"""Establish a single WebSocket connection, authenticate, and listen."""
await self.logger.info(f'Connecting to {self.ws_url}...')
self._session = aiohttp.ClientSession()
try:
self._ws = await self._session.ws_connect(self.ws_url)
self._missed_pong_count = 0
self._reconnect_attempts = 0
await self.logger.info('WebSocket connected, sending auth...')
await self._send_auth()
# Wait for auth response
auth_ok = await self._wait_for_auth()
if not auth_ok:
await self.logger.error('Authentication failed')
return
await self.logger.info('Authenticated successfully')
# Start heartbeat
self._heartbeat_task = asyncio.create_task(self._heartbeat_loop())
try:
await self._listen_loop()
finally:
if self._heartbeat_task and not self._heartbeat_task.done():
self._heartbeat_task.cancel()
self._clear_pending_acks('Connection closed')
finally:
if self._ws and not self._ws.closed:
await self._ws.close()
self._ws = None
if self._session and not self._session.closed:
await self._session.close()
self._session = None
async def _send_auth(self):
"""Send the authentication frame."""
frame = {
'cmd': CMD_SUBSCRIBE,
'headers': {'req_id': _generate_req_id(CMD_SUBSCRIBE)},
'body': {
'bot_id': self.bot_id,
'secret': self.secret,
},
}
await self._send_frame(frame)
async def _wait_for_auth(self) -> bool:
"""Wait for and validate the authentication response."""
try:
msg = await asyncio.wait_for(self._ws.receive(), timeout=10.0)
if msg.type in (aiohttp.WSMsgType.TEXT,):
frame = json.loads(msg.data)
req_id = frame.get('headers', {}).get('req_id', '')
if req_id.startswith(CMD_SUBSCRIBE) and frame.get('errcode') == 0:
return True
await self.logger.error(f'Auth response: errcode={frame.get("errcode")}, errmsg={frame.get("errmsg")}')
return False
elif msg.type in (aiohttp.WSMsgType.ERROR, aiohttp.WSMsgType.CLOSED, aiohttp.WSMsgType.CLOSING):
await self.logger.error(f'WebSocket closed during auth: {msg.type}')
return False
await self.logger.error(f'Unexpected message type during auth: {msg.type}')
return False
except asyncio.TimeoutError:
await self.logger.error('Auth response timeout')
return False
async def _heartbeat_loop(self):
"""Periodically send heartbeat pings."""
try:
while self._running and self._ws and not self._ws.closed:
await asyncio.sleep(self.heartbeat_interval)
if not self._running or not self._ws or self._ws.closed:
break
if self._missed_pong_count >= self._max_missed_pong:
await self.logger.warning(
f'No heartbeat ack for {self._missed_pong_count} consecutive pings, connection considered dead'
)
await self._ws.close()
break
self._missed_pong_count += 1
frame = {
'cmd': CMD_HEARTBEAT,
'headers': {'req_id': _generate_req_id(CMD_HEARTBEAT)},
}
try:
await self._send_frame(frame)
except Exception:
break
except asyncio.CancelledError:
pass
async def _listen_loop(self):
"""Listen for incoming WebSocket frames and dispatch them."""
async for msg in self._ws:
if not self._running:
break
if msg.type == aiohttp.WSMsgType.TEXT:
try:
frame = json.loads(msg.data)
await self._handle_frame(frame)
except json.JSONDecodeError:
await self.logger.error(f'Failed to parse WebSocket message: {str(msg.data)[:200]}')
except Exception:
await self.logger.error(f'Error handling frame: {traceback.format_exc()}')
elif msg.type == aiohttp.WSMsgType.BINARY:
try:
frame = json.loads(msg.data)
await self._handle_frame(frame)
except Exception:
await self.logger.error(f'Error handling binary frame: {traceback.format_exc()}')
elif msg.type in (aiohttp.WSMsgType.ERROR, aiohttp.WSMsgType.CLOSED, aiohttp.WSMsgType.CLOSING):
await self.logger.warning(f'WebSocket connection closed: {msg.type}')
break
# ── Frame handling ──────────────────────────────────────────────
async def _handle_frame(self, frame: dict):
"""Route an incoming frame to the appropriate handler."""
cmd = frame.get('cmd', '')
# Message push
if cmd == CMD_MSG_CALLBACK:
asyncio.create_task(self._handle_message_callback(frame))
return
# Event push
if cmd == CMD_EVENT_CALLBACK:
asyncio.create_task(self._handle_event_callback(frame))
return
# No cmd → response/ACK frame, dispatch by req_id prefix
req_id = frame.get('headers', {}).get('req_id', '')
# Check pending ACKs first
if req_id in self._pending_acks:
future = self._pending_acks.pop(req_id)
if not future.done():
future.set_result(frame)
return
# Heartbeat response
if req_id.startswith(CMD_HEARTBEAT):
if frame.get('errcode') == 0:
self._missed_pong_count = 0
return
# Unknown frame
await self.logger.warning(f'Unknown frame: {json.dumps(frame, ensure_ascii=False)[:200]}')
async def _handle_message_callback(self, frame: dict):
"""Handle an incoming message callback frame."""
try:
body = frame.get('body', {})
req_id = frame.get('headers', {}).get('req_id', '')
# Parse message using shared logic
message_data = await parse_wecom_bot_message(body, self.encoding_aes_key, self.logger)
if not message_data:
return
# Generate stream_id for this message and store the mapping
stream_id = _generate_req_id('stream')
msg_id = message_data.get('msgid', '')
if msg_id:
self._stream_ids[msg_id] = f'{req_id}|{stream_id}'
message_data['stream_id'] = stream_id
message_data['req_id'] = req_id
event = wecombotevent.WecomBotEvent(message_data)
await self._dispatch_event(event)
except Exception:
await self.logger.error(f'Error in message callback: {traceback.format_exc()}')
async def _handle_event_callback(self, frame: dict):
"""Handle an incoming event callback frame (enter_chat, template_card_event, etc.)."""
try:
body = frame.get('body', {})
req_id = frame.get('headers', {}).get('req_id', '')
event_info = body.get('event', {})
event_type = event_info.get('eventtype', '')
message_data = {
'msgtype': 'event',
'type': body.get('chattype', 'single'),
'event': event_info,
'eventtype': event_type,
'msgid': body.get('msgid', ''),
'aibotid': body.get('aibotid', ''),
'req_id': req_id,
}
from_info = body.get('from', {})
message_data['userid'] = from_info.get('userid', '')
message_data['username'] = from_info.get('alias', '') or from_info.get('userid', '')
if body.get('chatid'):
message_data['chatid'] = body.get('chatid', '')
event = wecombotevent.WecomBotEvent(message_data)
# Dispatch to event-specific handlers
if event_type in self._message_handlers:
for handler in self._message_handlers[event_type]:
await handler(event)
# Also dispatch to generic 'event' handlers
if 'event' in self._message_handlers:
for handler in self._message_handlers['event']:
await handler(event)
except Exception:
await self.logger.error(f'Error in event callback: {traceback.format_exc()}')
async def _dispatch_event(self, event: wecombotevent.WecomBotEvent):
"""Dispatch a message event to registered handlers with deduplication."""
try:
message_id = event.message_id
if message_id in self._msg_id_map:
self._msg_id_map[message_id] += 1
return
self._msg_id_map[message_id] = 1
msg_type = event.type
if msg_type in self._message_handlers:
for handler in self._message_handlers[msg_type]:
await handler(event)
except Exception:
await self.logger.error(f'Error dispatching event: {traceback.format_exc()}')
# ── Reply sending with serial queue ─────────────────────────────
async def _send_reply(
self,
req_id: str,
body: dict,
cmd: str = CMD_RESPOND_MSG,
) -> Optional[dict]:
"""Send a reply frame and wait for ACK.
Replies with the same req_id are serialized to maintain ordering.
"""
if not self._ws or self._ws.closed:
return None
frame = {
'cmd': cmd,
'headers': {'req_id': req_id},
'body': body,
}
# Ensure serial delivery per req_id
if req_id not in self._reply_queues:
self._reply_queues[req_id] = asyncio.Queue()
self._reply_workers[req_id] = asyncio.create_task(self._reply_queue_worker(req_id))
future: asyncio.Future = asyncio.get_event_loop().create_future()
await self._reply_queues[req_id].put((frame, future))
return await future
async def _reply_queue_worker(self, req_id: str):
"""Process reply queue items serially for a given req_id."""
queue = self._reply_queues[req_id]
try:
while self._running:
try:
frame, future = await asyncio.wait_for(queue.get(), timeout=60.0)
except asyncio.TimeoutError:
# Queue idle, clean up worker
break
try:
ack = await self._send_and_wait_ack(frame)
if not future.done():
future.set_result(ack)
except Exception as e:
if not future.done():
future.set_exception(e)
except asyncio.CancelledError:
pass
finally:
self._reply_queues.pop(req_id, None)
self._reply_workers.pop(req_id, None)
async def _send_and_wait_ack(self, frame: dict) -> Optional[dict]:
"""Send a frame and wait for the corresponding ACK."""
req_id = frame['headers']['req_id']
ack_future: asyncio.Future = asyncio.get_event_loop().create_future()
self._pending_acks[req_id] = ack_future
try:
await self._send_frame(frame)
result = await asyncio.wait_for(ack_future, timeout=self._reply_ack_timeout)
if result.get('errcode', 0) != 0:
await self.logger.warning(
f'Reply ACK error: errcode={result.get("errcode")}, errmsg={result.get("errmsg")}'
)
return result
except asyncio.TimeoutError:
self._pending_acks.pop(req_id, None)
await self.logger.warning(f'Reply ACK timeout ({self._reply_ack_timeout}s) for req_id={req_id}')
return None
async def _send_frame(self, frame: dict):
"""Send a JSON frame over the WebSocket connection."""
if self._ws and not self._ws.closed:
await self._ws.send_str(json.dumps(frame, ensure_ascii=False))
def _clear_pending_acks(self, reason: str):
"""Reject all pending ACK futures on disconnection."""
for req_id, future in self._pending_acks.items():
if not future.done():
future.set_exception(ConnectionError(reason))
self._pending_acks.clear()

View File

@@ -4,6 +4,7 @@ import base64
import binascii
import httpx
import traceback
from urllib.parse import quote
from quart import Quart
import xml.etree.ElementTree as ET
from typing import Callable, Dict, Any
@@ -67,6 +68,31 @@ class WecomClient:
await self.logger.error(f'获取accesstoken失败:{response.json()}')
raise Exception(f'未获取access token: {data}')
async def get_user_info(self, userid: str) -> dict:
"""
Get user information by user ID using the application secret.
Args:
userid: The user ID to look up.
Returns:
dict: User information including 'name' field.
"""
if not await self.check_access_token():
self.access_token = await self.get_access_token(self.secret)
url = self.base_url + '/user/get?access_token=' + self.access_token + '&userid=' + quote(userid)
async with httpx.AsyncClient() as client:
response = await client.get(url)
data = response.json()
if data.get('errcode') == 40014 or data.get('errcode') == 42001:
self.access_token = await self.get_access_token(self.secret)
return await self.get_user_info(userid)
if data.get('errcode', 0) != 0:
await self.logger.error(f'获取用户信息失败:{data}')
return {}
return data
async def get_users(self):
if not self.check_access_token_for_contacts():
self.access_token_for_contacts = await self.get_access_token(self.secret_for_contacts)

View File

@@ -10,6 +10,7 @@ from typing import Callable
from .wecomcsevent import WecomCSEvent
import langbot_plugin.api.entities.builtin.platform.message as platform_message
import aiofiles
import time
class WecomCSClient:
@@ -34,6 +35,10 @@ class WecomCSClient:
self.unified_mode = unified_mode
self.app = Quart(__name__)
# Customer info cache: {external_userid: (info_dict, timestamp)}
self._customer_cache: dict[str, tuple[dict, float]] = {}
self._cache_ttl = 60 # Cache TTL in seconds (1 minute)
# 只有在非统一模式下才注册独立路由
if not self.unified_mode:
self.app.add_url_rule(
@@ -378,3 +383,53 @@ class WecomCSClient:
async def get_media_id(self, image: platform_message.Image):
media_id = await self.upload_to_work(image=image)
return media_id
async def get_customer_info(self, external_userid: str) -> dict | None:
"""
Get customer information by external_userid with caching.
Uses a 1-minute cache to avoid repeated API calls for the same user.
Args:
external_userid: The external user ID of the customer.
Returns:
Customer info dict with 'nickname', 'avatar', etc., or None if not found.
"""
# Check cache first
current_time = time.time()
if external_userid in self._customer_cache:
cached_info, cached_time = self._customer_cache[external_userid]
if current_time - cached_time < self._cache_ttl:
return cached_info
# Cache miss or expired, fetch from API
if not await self.check_access_token():
self.access_token = await self.get_access_token(self.secret)
url = f'{self.base_url}/kf/customer/batchget?access_token={self.access_token}'
payload = {
'external_userid_list': [external_userid],
}
async with httpx.AsyncClient() as client:
response = await client.post(url, json=payload)
data = response.json()
if data.get('errcode') in [40014, 42001]:
self.access_token = await self.get_access_token(self.secret)
return await self.get_customer_info(external_userid)
if data.get('errcode', 0) != 0:
if self.logger:
await self.logger.warning(f'Failed to get customer info: {data}')
return None
customer_list = data.get('customer_list', [])
if customer_list:
customer_info = customer_list[0]
# Store in cache
self._customer_cache[external_userid] = (customer_info, current_time)
return customer_info
return None

View File

@@ -13,9 +13,9 @@ from .. import group
@group.group_class('files', '/api/v1/files')
class FilesRouterGroup(group.RouterGroup):
async def initialize(self) -> None:
@self.route('/image/<image_key>', methods=['GET'], auth_type=group.AuthType.NONE)
@self.route('/image/<path:image_key>', methods=['GET'], auth_type=group.AuthType.NONE)
async def _(image_key: str) -> quart.Response:
if '/' in image_key or '\\' in image_key:
if '..' in image_key or '\\' in image_key:
return quart.Response(status=404)
if not await self.ap.storage_mgr.storage_provider.exists(image_key):

View File

@@ -13,7 +13,10 @@ class KnowledgeBaseRouterGroup(group.RouterGroup):
elif quart.request.method == 'POST':
json_data = await quart.request.json
knowledge_base_uuid = await self.ap.knowledge_service.create_knowledge_base(json_data)
try:
knowledge_base_uuid = await self.ap.knowledge_service.create_knowledge_base(json_data)
except ValueError as e:
return self.http_status(400, -1, str(e))
return self.success(data={'uuid': knowledge_base_uuid})
return self.http_status(405, -1, 'Method not allowed')
@@ -39,7 +42,7 @@ class KnowledgeBaseRouterGroup(group.RouterGroup):
elif quart.request.method == 'PUT':
json_data = await quart.request.json
await self.ap.knowledge_service.update_knowledge_base(knowledge_base_uuid, json_data)
return self.success({})
return self.success(data={'uuid': knowledge_base_uuid})
elif quart.request.method == 'DELETE':
await self.ap.knowledge_service.delete_knowledge_base(knowledge_base_uuid)
@@ -65,8 +68,12 @@ class KnowledgeBaseRouterGroup(group.RouterGroup):
if not file_id:
return self.http_status(400, -1, 'File ID is required')
parser_plugin_id = json_data.get('parser_plugin_id')
# 调用服务层方法将文件与知识库关联
task_id = await self.ap.knowledge_service.store_file(knowledge_base_uuid, file_id)
task_id = await self.ap.knowledge_service.store_file(
knowledge_base_uuid, file_id, parser_plugin_id=parser_plugin_id
)
return self.success(
{
'task_id': task_id,
@@ -90,5 +97,13 @@ class KnowledgeBaseRouterGroup(group.RouterGroup):
async def retrieve_knowledge_base(knowledge_base_uuid: str) -> str:
json_data = await quart.request.json
query = json_data.get('query')
results = await self.ap.knowledge_service.retrieve_knowledge_base(knowledge_base_uuid, query)
if not query or not query.strip():
return self.http_status(400, -1, 'Query is required and cannot be empty')
# Extract retrieval_settings to allow dynamic control over Knowledge Engine behavior (e.g. top_k, filters)
retrieval_settings = json_data.get('retrieval_settings', {})
results = await self.ap.knowledge_service.retrieve_knowledge_base(
knowledge_base_uuid, query, retrieval_settings
)
return self.success(data={'results': results})

View File

@@ -0,0 +1,45 @@
import quart
from urllib.parse import unquote
from ... import group
@group.group_class('knowledge_engines', '/api/v1/knowledge/engines')
class KnowledgeEnginesRouterGroup(group.RouterGroup):
async def initialize(self) -> None:
@self.route('', methods=['GET'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def list_knowledge_engines() -> quart.Response:
"""List all available Knowledge Engines from plugins.
Returns a list of Knowledge Engines with their capabilities and configuration schemas.
This is used by the frontend to render the knowledge base creation wizard.
"""
engines = await self.ap.knowledge_service.list_knowledge_engines()
return self.success(data={'engines': engines})
@self.route(
'/<path:plugin_id>/creation-schema', methods=['GET'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY
)
async def get_engine_creation_schema(plugin_id: str) -> quart.Response:
"""Get creation settings schema for a specific Knowledge Engine.
plugin_id is in 'author/name' format, captured via <path:> converter.
"""
plugin_id = unquote(plugin_id)
if '/' not in plugin_id:
return self.http_status(400, -1, 'Invalid plugin_id format. Expected author/name.')
schema = await self.ap.knowledge_service.get_engine_creation_schema(plugin_id)
return self.success(data={'schema': schema})
@self.route(
'/<path:plugin_id>/retrieval-schema', methods=['GET'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY
)
async def get_engine_retrieval_schema(plugin_id: str) -> quart.Response:
"""Get retrieval settings schema for a specific Knowledge Engine.
plugin_id is in 'author/name' format, captured via <path:> converter.
"""
plugin_id = unquote(plugin_id)
if '/' not in plugin_id:
return self.http_status(400, -1, 'Invalid plugin_id format. Expected author/name.')
schema = await self.ap.knowledge_service.get_engine_retrieval_schema(plugin_id)
return self.success(data={'schema': schema})

View File

@@ -1,61 +0,0 @@
import quart
from ... import group
@group.group_class('external_knowledge_base', '/api/v1/knowledge/external-bases')
class ExternalKnowledgeBaseRouterGroup(group.RouterGroup):
async def initialize(self) -> None:
@self.route('/retrievers', methods=['GET'])
async def list_knowledge_retrievers() -> quart.Response:
"""List all available knowledge retrievers from plugins."""
retrievers = await self.ap.plugin_connector.list_knowledge_retrievers()
return self.success(data={'retrievers': retrievers})
@self.route('', methods=['POST', 'GET'])
async def handle_external_knowledge_bases() -> quart.Response:
if quart.request.method == 'GET':
external_kbs = await self.ap.external_kb_service.get_external_knowledge_bases()
return self.success(data={'bases': external_kbs})
elif quart.request.method == 'POST':
json_data = await quart.request.json
kb_uuid = await self.ap.external_kb_service.create_external_knowledge_base(json_data)
return self.success(data={'uuid': kb_uuid})
return self.http_status(405, -1, 'Method not allowed')
@self.route(
'/<kb_uuid>',
methods=['GET', 'DELETE', 'PUT'],
)
async def handle_specific_external_knowledge_base(kb_uuid: str) -> quart.Response:
if quart.request.method == 'GET':
external_kb = await self.ap.external_kb_service.get_external_knowledge_base(kb_uuid)
if external_kb is None:
return self.http_status(404, -1, 'external knowledge base not found')
return self.success(
data={
'base': external_kb,
}
)
elif quart.request.method == 'PUT':
json_data = await quart.request.json
await self.ap.external_kb_service.update_external_knowledge_base(kb_uuid, json_data)
return self.success({})
elif quart.request.method == 'DELETE':
await self.ap.external_kb_service.delete_external_knowledge_base(kb_uuid)
return self.success({})
@self.route(
'/<kb_uuid>/retrieve',
methods=['POST'],
)
async def retrieve_external_knowledge_base(kb_uuid: str) -> str:
json_data = await quart.request.json
query = json_data.get('query')
results = await self.ap.external_kb_service.retrieve_external_knowledge_base(kb_uuid, query)
return self.success(data={'results': results})

View File

@@ -0,0 +1,372 @@
import asyncio
import json
import httpx
import quart
import sqlalchemy
from ... import group
from ......core import taskmgr
from ......entity.persistence import metadata as persistence_metadata
from langbot_plugin.runtime.plugin.mgr import PluginInstallSource
LANGRAG_PLUGIN_AUTHOR = 'langbot-team'
LANGRAG_PLUGIN_NAME = 'LangRAG'
LANGRAG_PLUGIN_ID = f'{LANGRAG_PLUGIN_AUTHOR}/{LANGRAG_PLUGIN_NAME}'
DEFAULT_SPACE_URL = 'https://space.langbot.app'
# Old Retriever plugin_name -> New Connector plugin_name
EXTERNAL_PLUGIN_NAME_MAPPING = {
'DifyDatasetsRetriever': 'DifyDatasetsConnector',
'RAGFlowRetriever': 'RAGFlowConnector',
'FastGPTRetriever': 'FastGPTConnector',
}
# Per-plugin: which old retriever_config fields belong to creation_settings.
# Remaining fields go to retrieval_settings.
# None means ALL fields go to creation_settings (no retrieval_schema).
EXTERNAL_PLUGIN_CREATION_FIELDS: dict[str, set[str] | None] = {
'langbot-team/DifyDatasetsConnector': {'api_base_url', 'dify_apikey', 'dataset_id'},
'langbot-team/RAGFlowConnector': {'api_base_url', 'api_key', 'dataset_ids'},
'langbot-team/FastGPTConnector': None, # all fields -> creation_settings
}
@group.group_class('knowledge/migration', '/api/v1/knowledge/migration')
class KnowledgeMigrationRouterGroup(group.RouterGroup):
async def _get_migration_flag(self) -> bool:
"""Check if rag_plugin_migration_needed flag is set."""
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_metadata.Metadata).where(
persistence_metadata.Metadata.key == 'rag_plugin_migration_needed'
)
)
row = result.first()
return row is not None and row.value == 'true'
async def _set_migration_flag(self, value: str):
"""Set rag_plugin_migration_needed flag."""
await self.ap.persistence_mgr.execute_async(
sqlalchemy.update(persistence_metadata.Metadata)
.where(persistence_metadata.Metadata.key == 'rag_plugin_migration_needed')
.values(value=value)
)
async def _table_exists(self, table_name: str) -> bool:
"""Check if a table exists."""
if self.ap.persistence_mgr.db.name == 'postgresql':
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'SELECT EXISTS (SELECT FROM information_schema.tables WHERE table_name = :table_name);'
).bindparams(table_name=table_name)
)
return result.scalar()
else:
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("SELECT name FROM sqlite_master WHERE type='table' AND name=:table_name;").bindparams(
table_name=table_name
)
)
return result.first() is not None
async def _install_plugin_from_marketplace(
self, plugin_id: str, task_context: taskmgr.TaskContext, space_url: str
) -> None:
"""Install a single plugin from the marketplace."""
p_author, p_name = plugin_id.split('/', 1)
self.ap.logger.info(f'RAG migration: installing plugin {plugin_id} from marketplace...')
task_context.trace(f'Installing plugin {plugin_id} from marketplace...')
async with httpx.AsyncClient(trust_env=True, timeout=15) as client:
resp = await client.get(f'{space_url}/api/v1/marketplace/plugins/{p_author}/{p_name}')
resp.raise_for_status()
p_data = resp.json().get('data', {}).get('plugin', {})
p_version = p_data.get('latest_version')
if not p_version:
raise Exception(f'Could not determine latest version for {plugin_id}')
await self.ap.plugin_connector.install_plugin(
PluginInstallSource.MARKETPLACE,
{
'plugin_author': p_author,
'plugin_name': p_name,
'plugin_version': p_version,
},
task_context=task_context,
)
self.ap.logger.info(f'RAG migration: plugin {plugin_id} install request sent.')
async def _execute_rag_migration(self, task_context: taskmgr.TaskContext, install_plugin: bool = True):
"""Execute RAG migration: install required plugins and restore backup data."""
warnings = []
# Collect all plugins we need: LangRAG (always) + connector plugins (from external KBs)
needed_plugins: dict[str, str] = {
LANGRAG_PLUGIN_ID: LANGRAG_PLUGIN_NAME,
}
has_external = await self._table_exists('external_knowledge_bases')
if has_external:
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT DISTINCT plugin_author, plugin_name FROM external_knowledge_bases;')
)
for row in result.fetchall():
plugin_author = row[0] or ''
plugin_name = row[1] or ''
mapped_name = EXTERNAL_PLUGIN_NAME_MAPPING.get(plugin_name, plugin_name)
plugin_id = f'{plugin_author}/{mapped_name}'
if plugin_id not in needed_plugins:
needed_plugins[plugin_id] = mapped_name
self.ap.logger.info(f'RAG migration: plugins needed: {list(needed_plugins.keys())}')
if install_plugin:
# Step 1: Install all required plugins from marketplace
task_context.trace('Installing required plugins...', action='install-plugin')
space_url = self.ap.instance_config.data.get('space', {}).get('url', DEFAULT_SPACE_URL).rstrip('/')
for plugin_id in needed_plugins:
try:
await self._install_plugin_from_marketplace(plugin_id, task_context, space_url)
except Exception as e:
self.ap.logger.warning(f'RAG migration: plugin {plugin_id} install returned: {e}')
task_context.trace(f'Plugin install note ({plugin_id}): {e}')
# Step 2: Wait for all plugins to become available as knowledge engines
task_context.trace(
f'Waiting for plugins to become available: {list(needed_plugins.keys())}...',
action='wait-plugin',
)
max_retries = 30
engine_id_set: set[str] = set()
for i in range(max_retries):
try:
engines = await self.ap.plugin_connector.list_knowledge_engines()
engine_id_set = {e.get('plugin_id') for e in engines}
except Exception:
pass
if all(pid in engine_id_set for pid in needed_plugins):
self.ap.logger.info(f'RAG migration: all plugins ready: {engine_id_set}')
task_context.trace('All required plugins are ready.')
break
if i == max_retries - 1:
still_missing = [pid for pid in needed_plugins if pid not in engine_id_set]
warning = f'Plugin(s) {still_missing} did not become available after {max_retries} retries'
self.ap.logger.warning(f'RAG migration: {warning}')
warnings.append(warning)
task_context.trace(warning)
await asyncio.sleep(2)
else:
try:
engines = await self.ap.plugin_connector.list_knowledge_engines()
engine_id_set = {e.get('plugin_id') for e in engines}
except Exception:
engine_id_set = set()
# Step 3: Restore internal knowledge bases from backup
task_context.trace('Restoring internal knowledge bases...', action='restore-internal')
if await self._table_exists('knowledge_bases_backup'):
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT * FROM knowledge_bases_backup;')
)
rows = result.fetchall()
columns = result.keys()
for row in rows:
row_dict = dict(zip(columns, row))
kb_uuid = row_dict.get('uuid')
name = row_dict.get('name', 'Untitled')
description = row_dict.get('description', '')
emoji = row_dict.get('emoji', '\U0001f4da')
embedding_model_uuid = row_dict.get('embedding_model_uuid', '')
top_k = row_dict.get('top_k', 5)
created_at = row_dict.get('created_at')
updated_at = row_dict.get('updated_at')
creation_settings = json.dumps({'embedding_model_uuid': embedding_model_uuid})
retrieval_settings = json.dumps({'top_k': top_k})
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'INSERT INTO knowledge_bases '
'(uuid, name, description, emoji, created_at, updated_at, '
'knowledge_engine_plugin_id, collection_id, creation_settings, retrieval_settings) '
'VALUES (:uuid, :name, :description, :emoji, :created_at, :updated_at, '
':plugin_id, :collection_id, :creation_settings, :retrieval_settings);'
).bindparams(
uuid=kb_uuid,
name=name,
description=description,
emoji=emoji,
created_at=created_at,
updated_at=updated_at,
plugin_id=LANGRAG_PLUGIN_ID,
collection_id=kb_uuid,
creation_settings=creation_settings,
retrieval_settings=retrieval_settings,
)
)
try:
config = {'embedding_model_uuid': embedding_model_uuid}
await self.ap.plugin_connector.rag_on_kb_create(LANGRAG_PLUGIN_ID, kb_uuid, config)
task_context.trace(f'Restored internal KB: {name} ({kb_uuid})')
except Exception as e:
warning = f'Failed to notify plugin for KB {name} ({kb_uuid}): {e}'
warnings.append(warning)
task_context.trace(warning)
await self.ap.rag_mgr.load_knowledge_bases_from_db()
# Step 4: Restore external knowledge bases
task_context.trace('Restoring external knowledge bases...', action='restore-external')
if has_external:
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT * FROM external_knowledge_bases;')
)
rows = result.fetchall()
columns = result.keys()
self.ap.logger.info(
f'RAG migration: {len(rows)} external KB(s) to restore. Available engines: {engine_id_set}'
)
task_context.trace(f'Found {len(rows)} external KB(s). Available engines: {engine_id_set}')
for row in rows:
row_dict = dict(zip(columns, row))
kb_uuid = row_dict.get('uuid')
name = row_dict.get('name', 'Untitled')
description = row_dict.get('description', '')
emoji = row_dict.get('emoji', '\U0001f517')
plugin_author = row_dict.get('plugin_author', '')
plugin_name = row_dict.get('plugin_name', '')
retriever_config = row_dict.get('retriever_config', {})
created_at = row_dict.get('created_at')
mapped_plugin_name = EXTERNAL_PLUGIN_NAME_MAPPING.get(plugin_name, plugin_name)
external_plugin_id = f'{plugin_author}/{mapped_plugin_name}'
self.ap.logger.info(
f'RAG migration: processing external KB "{name}" ({kb_uuid}), '
f'plugin: {plugin_author}/{plugin_name} -> {external_plugin_id}'
)
if isinstance(retriever_config, str):
try:
retriever_config = json.loads(retriever_config)
except (json.JSONDecodeError, TypeError):
retriever_config = {}
creation_fields = EXTERNAL_PLUGIN_CREATION_FIELDS.get(external_plugin_id)
if creation_fields is None:
creation_settings_dict = retriever_config
retrieval_settings_dict = {}
else:
creation_settings_dict = {k: v for k, v in retriever_config.items() if k in creation_fields}
retrieval_settings_dict = {k: v for k, v in retriever_config.items() if k not in creation_fields}
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'INSERT INTO knowledge_bases '
'(uuid, name, description, emoji, created_at, updated_at, '
'knowledge_engine_plugin_id, collection_id, creation_settings, retrieval_settings) '
'VALUES (:uuid, :name, :description, :emoji, :created_at, :updated_at, '
':plugin_id, :collection_id, :creation_settings, :retrieval_settings);'
).bindparams(
uuid=kb_uuid,
name=name,
description=description,
emoji=emoji,
created_at=created_at,
updated_at=created_at,
plugin_id=external_plugin_id,
collection_id=kb_uuid,
creation_settings=json.dumps(creation_settings_dict),
retrieval_settings=json.dumps(retrieval_settings_dict),
)
)
if external_plugin_id not in engine_id_set:
warning = (
f'External KB "{name}" ({kb_uuid}) record saved, but plugin {external_plugin_id} '
f'is not installed yet. Install the connector plugin to use it.'
)
warnings.append(warning)
task_context.trace(warning)
else:
try:
await self.ap.plugin_connector.rag_on_kb_create(
external_plugin_id, kb_uuid, creation_settings_dict
)
task_context.trace(f'Restored external KB: {name} ({kb_uuid})')
except Exception as e:
warning = f'Failed to notify plugin for external KB {name} ({kb_uuid}): {e}'
warnings.append(warning)
task_context.trace(warning)
await self.ap.rag_mgr.load_knowledge_bases_from_db()
# Step 5: Clear migration flag
await self._set_migration_flag('false')
task_context.trace('RAG migration completed.', action='done')
if warnings:
task_context.trace(f'Completed with {len(warnings)} warning(s).')
async def initialize(self) -> None:
@self.route('/status', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
async def _() -> str:
needed = await self._get_migration_flag()
internal_kb_count = 0
external_kb_count = 0
if needed:
if await self._table_exists('knowledge_bases_backup'):
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT COUNT(*) FROM knowledge_bases_backup;')
)
internal_kb_count = result.scalar() or 0
if await self._table_exists('external_knowledge_bases'):
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT COUNT(*) FROM external_knowledge_bases;')
)
external_kb_count = result.scalar() or 0
return self.success(
data={
'needed': needed,
'internal_kb_count': internal_kb_count,
'external_kb_count': external_kb_count,
}
)
@self.route('/execute', methods=['POST'], auth_type=group.AuthType.USER_TOKEN)
async def _() -> str:
needed = await self._get_migration_flag()
if not needed:
return self.http_status(400, -1, 'RAG migration is not needed')
data = await quart.request.get_json(silent=True) or {}
install_plugin = data.get('install_plugin', True)
ctx = taskmgr.TaskContext.new()
wrapper = self.ap.task_mgr.create_user_task(
self._execute_rag_migration(task_context=ctx, install_plugin=install_plugin),
kind='rag-migration',
name='rag-migration-execute',
label='Migrating knowledge bases to plugin architecture',
context=ctx,
)
return self.success(data={'task_id': wrapper.id})
@self.route('/dismiss', methods=['POST'], auth_type=group.AuthType.USER_TOKEN)
async def _() -> str:
needed = await self._get_migration_flag()
if not needed:
return self.http_status(400, -1, 'RAG migration is not needed')
await self._set_migration_flag('false')
return self.success()

View File

@@ -0,0 +1,16 @@
import quart
from ... import group
@group.group_class('parsers', '/api/v1/knowledge/parsers')
class ParsersRouterGroup(group.RouterGroup):
async def initialize(self) -> None:
@self.route('', methods=['GET'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def list_parsers() -> quart.Response:
"""List all available parsers from plugins.
Optional query parameter `mime_type` to filter parsers by supported MIME type.
"""
mime_type = quart.request.args.get('mime_type')
parsers = await self.ap.knowledge_service.list_parsers(mime_type)
return self.success(data={'parsers': parsers})

View File

@@ -456,6 +456,31 @@ class MonitoringRouterGroup(group.RouterGroup):
'platform',
'user_id',
]
elif export_type == 'feedback':
data = await self.ap.monitoring_service.export_feedback(
bot_ids=bot_ids if bot_ids else None,
pipeline_ids=pipeline_ids if pipeline_ids else None,
start_time=start_time,
end_time=end_time,
limit=limit,
)
headers = [
'id',
'timestamp',
'feedback_id',
'feedback_type',
'feedback_content',
'inaccurate_reasons',
'bot_id',
'bot_name',
'pipeline_id',
'pipeline_name',
'session_id',
'message_id',
'stream_id',
'user_id',
'platform',
]
else:
return self.error(message=f'Invalid export type: {export_type}', code=400)
@@ -486,3 +511,63 @@ class MonitoringRouterGroup(group.RouterGroup):
)
return response, 200
@self.route('/feedback/stats', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
async def get_feedback_stats() -> str:
"""Get feedback statistics"""
# Parse query parameters
bot_ids = quart.request.args.getlist('botId')
pipeline_ids = quart.request.args.getlist('pipelineId')
start_time_str = quart.request.args.get('startTime')
end_time_str = quart.request.args.get('endTime')
# Parse datetime
start_time = parse_iso_datetime(start_time_str)
end_time = parse_iso_datetime(end_time_str)
stats = await self.ap.monitoring_service.get_feedback_stats(
bot_ids=bot_ids if bot_ids else None,
pipeline_ids=pipeline_ids if pipeline_ids else None,
start_time=start_time,
end_time=end_time,
)
return self.success(data=stats)
@self.route('/feedback', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
async def get_feedback() -> str:
"""Get feedback list"""
# Parse query parameters
bot_ids = quart.request.args.getlist('botId')
pipeline_ids = quart.request.args.getlist('pipelineId')
feedback_type_str = quart.request.args.get('feedbackType')
start_time_str = quart.request.args.get('startTime')
end_time_str = quart.request.args.get('endTime')
limit = int(quart.request.args.get('limit', 100))
offset = int(quart.request.args.get('offset', 0))
# Parse datetime
start_time = parse_iso_datetime(start_time_str)
end_time = parse_iso_datetime(end_time_str)
# Parse feedback type
feedback_type = int(feedback_type_str) if feedback_type_str else None
feedback_list, total = await self.ap.monitoring_service.get_feedback_list(
bot_ids=bot_ids if bot_ids else None,
pipeline_ids=pipeline_ids if pipeline_ids else None,
feedback_type=feedback_type,
start_time=start_time,
end_time=end_time,
limit=limit,
offset=offset,
)
return self.success(
data={
'feedback': feedback_list,
'total': total,
'limit': limit,
'offset': offset,
}
)

View File

@@ -68,7 +68,7 @@ class PipelinesRouterGroup(group.RouterGroup):
return self.http_status(404, -1, 'pipeline not found')
# Only include plugins with pipeline-related components (Command, EventListener, Tool)
# Plugins that only have KnowledgeRetriever components are not suitable for pipeline extensions
# Plugins that only have KnowledgeEngine components are not suitable for pipeline extensions
pipeline_component_kinds = ['Command', 'EventListener', 'Tool']
plugins = await self.ap.plugin_connector.list_plugins(component_kinds=pipeline_component_kinds)
mcp_servers = await self.ap.mcp_service.get_mcp_servers(contain_runtime_info=True)

View File

@@ -265,6 +265,8 @@ class PluginsRouterGroup(group.RouterGroup):
return self.http_status(400, -1, 'Missing asset_url parameter')
ctx = taskmgr.TaskContext.new()
ctx.metadata['plugin_name'] = f'{owner}/{repo}'
ctx.metadata['install_source'] = 'github'
install_info = {
'asset_url': asset_url,
'owner': owner,
@@ -295,12 +297,17 @@ class PluginsRouterGroup(group.RouterGroup):
data = await quart.request.json
plugin_author = data.get('plugin_author', '')
plugin_name = data.get('plugin_name', '')
ctx = taskmgr.TaskContext.new()
ctx.metadata['plugin_name'] = f'{plugin_author}/{plugin_name}'
ctx.metadata['install_source'] = 'marketplace'
wrapper = self.ap.task_mgr.create_user_task(
self.ap.plugin_connector.install_plugin(PluginInstallSource.MARKETPLACE, data, task_context=ctx),
kind='plugin-operation',
name='plugin-install-marketplace',
label=f'Installing plugin from marketplace ...{data}',
label=f'Installing plugin from marketplace {plugin_author}/{plugin_name}',
context=ctx,
)
@@ -323,11 +330,13 @@ class PluginsRouterGroup(group.RouterGroup):
}
ctx = taskmgr.TaskContext.new()
ctx.metadata['plugin_name'] = file.filename or 'local plugin'
ctx.metadata['install_source'] = 'local'
wrapper = self.ap.task_mgr.create_user_task(
self.ap.plugin_connector.install_plugin(PluginInstallSource.LOCAL, data, task_context=ctx),
kind='plugin-operation',
name='plugin-install-local',
label=f'Installing plugin from local ...{file.filename}',
label=f'Installing plugin from local {file.filename}',
context=ctx,
)

View File

@@ -1,7 +1,11 @@
import json
import quart
import sqlalchemy
from .. import group
from .....utils import constants
from .....entity.persistence.metadata import Metadata
@group.group_class('system', '/api/v1/system')
@@ -9,6 +13,24 @@ class SystemRouterGroup(group.RouterGroup):
async def initialize(self) -> None:
@self.route('/info', methods=['GET'], auth_type=group.AuthType.NONE)
async def _() -> str:
# Read wizard_status and wizard_progress from metadata table
wizard_status = 'none'
wizard_progress = None
try:
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(Metadata).where(Metadata.key.in_(['wizard_status', 'wizard_progress']))
)
for row in result:
if row.key == 'wizard_status':
wizard_status = row.value
elif row.key == 'wizard_progress':
try:
wizard_progress = json.loads(row.value)
except (json.JSONDecodeError, TypeError):
wizard_progress = None
except Exception:
pass
return self.success(
data={
'version': constants.semantic_version,
@@ -27,17 +49,83 @@ class SystemRouterGroup(group.RouterGroup):
'disable_models_service', False
),
'limitation': self.ap.instance_config.data.get('system', {}).get('limitation', {}),
'wizard_status': wizard_status,
'wizard_progress': wizard_progress,
}
)
@self.route('/wizard/completed', methods=['POST'], auth_type=group.AuthType.USER_TOKEN)
async def _() -> str:
"""Mark wizard status in metadata table and clear progress.
Accepts JSON body: { "status": "skipped" | "completed" }
"""
data = await quart.request.get_json(silent=True) or {}
status = data.get('status', 'completed')
if status not in ('skipped', 'completed'):
return self.http_status(400, 400, f'Invalid wizard status: {status}')
try:
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(Metadata).where(Metadata.key == 'wizard_status')
)
if result.first():
await self.ap.persistence_mgr.execute_async(
sqlalchemy.update(Metadata).where(Metadata.key == 'wizard_status').values(value=status)
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.insert(Metadata).values(key='wizard_status', value=status)
)
# Clear wizard progress when wizard is completed/skipped
await self.ap.persistence_mgr.execute_async(
sqlalchemy.delete(Metadata).where(Metadata.key == 'wizard_progress')
)
except Exception as e:
return self.http_status(500, 500, f'Failed to update wizard status: {e}')
return self.success(data={})
@self.route('/wizard/progress', methods=['PUT'], auth_type=group.AuthType.USER_TOKEN)
async def _() -> str:
"""Save wizard progress to metadata table.
Accepts JSON body with wizard state fields:
{ "step": int, "selected_adapter": str|null, "created_bot_uuid": str|null,
"bot_saved": bool, "selected_runner": str|null }
"""
data = await quart.request.get_json(silent=True) or {}
progress_json = json.dumps(data, ensure_ascii=False)
try:
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(Metadata).where(Metadata.key == 'wizard_progress')
)
if result.first():
await self.ap.persistence_mgr.execute_async(
sqlalchemy.update(Metadata).where(Metadata.key == 'wizard_progress').values(value=progress_json)
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.insert(Metadata).values(key='wizard_progress', value=progress_json)
)
except Exception as e:
return self.http_status(500, 500, f'Failed to save wizard progress: {e}')
return self.success(data={})
@self.route('/tasks', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
async def _() -> str:
task_type = quart.request.args.get('type')
task_kind = quart.request.args.get('kind')
if task_type == '':
task_type = None
if task_kind == '':
task_kind = None
return self.success(data=self.ap.task_mgr.get_tasks_dict(task_type))
return self.success(data=self.ap.task_mgr.get_tasks_dict(task_type, task_kind))
@self.route('/tasks/<task_id>', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
async def _(task_id: str) -> str:

View File

@@ -105,6 +105,28 @@ class HTTPController:
):
if os.path.exists(os.path.join(frontend_path, path + '.html')):
path += '.html'
elif path.startswith('home/'):
# SPA fallback for /home/* sub-routes.
# Entity detail views use query params (e.g. /home/bots?id=uuid),
# so the pre-rendered list page is served directly via path + '.html'.
# This fallback handles any remaining unmatched sub-paths.
segments = path.rstrip('/').split('/')
# Walk up parent segments looking for matching .html files
for i in range(len(segments) - 1, 0, -1):
parent_path = '/'.join(segments[:i]) + '.html'
if os.path.exists(os.path.join(frontend_path, parent_path)):
response = await quart.send_from_directory(frontend_path, parent_path, mimetype='text/html')
response.headers['Cache-Control'] = 'no-cache, no-store, must-revalidate'
response.headers['Pragma'] = 'no-cache'
response.headers['Expires'] = '0'
return response
# Final fallback to index.html for /home/* routes
response = await quart.send_from_directory(frontend_path, 'index.html', mimetype='text/html')
response.headers['Cache-Control'] = 'no-cache, no-store, must-revalidate'
response.headers['Pragma'] = 'no-cache'
response.headers['Expires'] = '0'
return response
else:
return await quart.send_from_directory(frontend_path, '404.html')

View File

@@ -70,12 +70,17 @@ class BotService:
'lark',
]:
webhook_prefix = self.ap.instance_config.data['api'].get('webhook_prefix', 'http://127.0.0.1:5300')
extra_webhook_prefix = self.ap.instance_config.data['api'].get('extra_webhook_prefix', '')
webhook_url = f'/bots/{bot_uuid}'
adapter_runtime_values['webhook_url'] = webhook_url
adapter_runtime_values['webhook_full_url'] = f'{webhook_prefix}{webhook_url}'
adapter_runtime_values['extra_webhook_full_url'] = (
f'{extra_webhook_prefix}{webhook_url}' if extra_webhook_prefix else ''
)
else:
adapter_runtime_values['webhook_url'] = None
adapter_runtime_values['webhook_full_url'] = None
adapter_runtime_values['extra_webhook_full_url'] = None
persistence_bot['adapter_runtime_values'] = adapter_runtime_values

View File

@@ -1,80 +0,0 @@
from __future__ import annotations
from ....core import app
import sqlalchemy
from langbot.pkg.entity.persistence import rag as persistence_rag
import uuid
class ExternalKBService:
"""External KB service"""
ap: app.Application
def __init__(self, ap: app.Application) -> None:
self.ap = ap
# External Knowledge Base methods
async def get_external_knowledge_bases(self) -> list[dict]:
result = await self.ap.persistence_mgr.execute_async(sqlalchemy.select(persistence_rag.ExternalKnowledgeBase))
external_kbs = result.all()
return [
self.ap.persistence_mgr.serialize_model(persistence_rag.ExternalKnowledgeBase, external_kb)
for external_kb in external_kbs
]
async def get_external_knowledge_base(self, kb_uuid: str) -> dict | None:
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_rag.ExternalKnowledgeBase).where(
persistence_rag.ExternalKnowledgeBase.uuid == kb_uuid
)
)
external_kb = result.first()
if external_kb is None:
return None
return self.ap.persistence_mgr.serialize_model(persistence_rag.ExternalKnowledgeBase, external_kb)
async def create_external_knowledge_base(self, kb_data: dict) -> str:
kb_data['uuid'] = str(uuid.uuid4())
await self.ap.persistence_mgr.execute_async(
sqlalchemy.insert(persistence_rag.ExternalKnowledgeBase).values(kb_data)
)
kb = await self.get_external_knowledge_base(kb_data['uuid'])
await self.ap.rag_mgr.load_external_knowledge_base(kb)
return kb_data['uuid']
async def retrieve_external_knowledge_base(self, kb_uuid: str, query: str) -> list[dict]:
"""Retrieve external knowledge base"""
runtime_kb = await self.ap.rag_mgr.get_knowledge_base_by_uuid(kb_uuid)
if runtime_kb is None:
raise Exception('Knowledge base not found')
return [
result.model_dump() for result in await runtime_kb.retrieve(query, 5)
] # top_k is just a placeholder for external knowledge base
async def update_external_knowledge_base(self, kb_uuid: str, kb_data: dict) -> None:
if 'uuid' in kb_data:
del kb_data['uuid']
await self.ap.persistence_mgr.execute_async(
sqlalchemy.update(persistence_rag.ExternalKnowledgeBase)
.values(kb_data)
.where(persistence_rag.ExternalKnowledgeBase.uuid == kb_uuid)
)
await self.ap.rag_mgr.remove_knowledge_base_from_runtime(kb_uuid)
kb = await self.get_external_knowledge_base(kb_uuid)
await self.ap.rag_mgr.load_external_knowledge_base(kb)
async def delete_external_knowledge_base(self, kb_uuid: str) -> None:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.delete(persistence_rag.ExternalKnowledgeBase).where(
persistence_rag.ExternalKnowledgeBase.uuid == kb_uuid
)
)
await self.ap.rag_mgr.delete_knowledge_base(kb_uuid)

View File

@@ -1,6 +1,5 @@
from __future__ import annotations
import uuid
import sqlalchemy
from ....core import app
@@ -17,64 +16,77 @@ class KnowledgeService:
async def get_knowledge_bases(self) -> list[dict]:
"""获取所有知识库"""
result = await self.ap.persistence_mgr.execute_async(sqlalchemy.select(persistence_rag.KnowledgeBase))
knowledge_bases = result.all()
return [
self.ap.persistence_mgr.serialize_model(persistence_rag.KnowledgeBase, knowledge_base)
for knowledge_base in knowledge_bases
]
return await self.ap.rag_mgr.get_all_knowledge_base_details()
async def get_knowledge_base(self, kb_uuid: str) -> dict | None:
"""获取知识库"""
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_rag.KnowledgeBase).where(persistence_rag.KnowledgeBase.uuid == kb_uuid)
)
knowledge_base = result.first()
if knowledge_base is None:
return None
return self.ap.persistence_mgr.serialize_model(persistence_rag.KnowledgeBase, knowledge_base)
return await self.ap.rag_mgr.get_knowledge_base_details(kb_uuid)
async def create_knowledge_base(self, kb_data: dict) -> str:
"""创建知识库"""
kb_data['uuid'] = str(uuid.uuid4())
await self.ap.persistence_mgr.execute_async(sqlalchemy.insert(persistence_rag.KnowledgeBase).values(kb_data))
# In new architecture, we delegate entirely to RAGManager which uses plugins.
# Legacy internal KB creation is removed.
kb = await self.get_knowledge_base(kb_data['uuid'])
knowledge_engine_plugin_id = kb_data.get('knowledge_engine_plugin_id')
if not knowledge_engine_plugin_id:
raise ValueError('knowledge_engine_plugin_id is required')
await self.ap.rag_mgr.load_knowledge_base(kb)
return kb_data['uuid']
kb = await self.ap.rag_mgr.create_knowledge_base(
name=kb_data.get('name', 'Untitled'),
knowledge_engine_plugin_id=knowledge_engine_plugin_id,
creation_settings=kb_data.get('creation_settings', {}),
retrieval_settings=kb_data.get('retrieval_settings', {}),
description=kb_data.get('description', ''),
)
return kb.uuid
async def update_knowledge_base(self, kb_uuid: str, kb_data: dict) -> None:
"""更新知识库"""
if 'uuid' in kb_data:
del kb_data['uuid']
# Filter to only mutable fields
filtered_data = {k: v for k, v in kb_data.items() if k in persistence_rag.KnowledgeBase.MUTABLE_FIELDS}
if 'embedding_model_uuid' in kb_data:
del kb_data['embedding_model_uuid']
if not filtered_data:
return
await self.ap.persistence_mgr.execute_async(
sqlalchemy.update(persistence_rag.KnowledgeBase)
.values(kb_data)
.values(filtered_data)
.where(persistence_rag.KnowledgeBase.uuid == kb_uuid)
)
await self.ap.rag_mgr.remove_knowledge_base_from_runtime(kb_uuid)
kb = await self.get_knowledge_base(kb_uuid)
if kb is None:
raise Exception('Knowledge base not found after update')
await self.ap.rag_mgr.load_knowledge_base(kb)
async def store_file(self, kb_uuid: str, file_id: str) -> int:
async def _check_doc_capability(self, kb_uuid: str, operation: str) -> None:
"""Check if the KB's Knowledge Engine supports document operations.
Args:
kb_uuid: Knowledge base UUID.
operation: Human-readable operation name for error messages.
Raises:
Exception: If the KB does not support doc_ingestion.
"""
kb_info = await self.ap.rag_mgr.get_knowledge_base_details(kb_uuid)
if not kb_info:
raise Exception('Knowledge base not found')
capabilities = kb_info.get('knowledge_engine', {}).get('capabilities', [])
if 'doc_ingestion' not in capabilities:
raise Exception(f'This knowledge base does not support {operation}')
async def store_file(self, kb_uuid: str, file_id: str, parser_plugin_id: str | None = None) -> str:
"""存储文件"""
# await self.ap.persistence_mgr.execute_async(sqlalchemy.insert(persistence_rag.File).values(kb_id=kb_uuid, file_id=file_id))
# await self.ap.rag_mgr.store_file(file_id)
runtime_kb = await self.ap.rag_mgr.get_knowledge_base_by_uuid(kb_uuid)
if runtime_kb is None:
raise Exception('Knowledge base not found')
# Only internal KBs support file storage
if runtime_kb.get_type() != 'internal':
raise Exception('Only internal knowledge bases support file storage')
result = await runtime_kb.store_file(file_id)
await self._check_doc_capability(kb_uuid, 'document upload')
result = await runtime_kb.store_file(file_id, parser_plugin_id=parser_plugin_id)
# Update the KB's updated_at timestamp
await self.ap.persistence_mgr.execute_async(
@@ -85,14 +97,18 @@ class KnowledgeService:
return result
async def retrieve_knowledge_base(self, kb_uuid: str, query: str) -> list[dict]:
async def retrieve_knowledge_base(
self, kb_uuid: str, query: str, retrieval_settings: dict | None = None
) -> list[dict]:
"""检索知识库"""
runtime_kb = await self.ap.rag_mgr.get_knowledge_base_by_uuid(kb_uuid)
if runtime_kb is None:
raise Exception('Knowledge base not found')
return [
result.model_dump() for result in await runtime_kb.retrieve(query, runtime_kb.knowledge_base_entity.top_k)
]
# Pass retrieval_settings
results = await runtime_kb.retrieve(query, settings=retrieval_settings)
return [result.model_dump() for result in results]
async def get_files_by_knowledge_base(self, kb_uuid: str) -> list[dict]:
"""获取知识库文件"""
@@ -107,9 +123,9 @@ class KnowledgeService:
runtime_kb = await self.ap.rag_mgr.get_knowledge_base_by_uuid(kb_uuid)
if runtime_kb is None:
raise Exception('Knowledge base not found')
# Only internal KBs support file deletion
if runtime_kb.get_type() != 'internal':
raise Exception('Only internal knowledge bases support file deletion')
await self._check_doc_capability(kb_uuid, 'document deletion')
await runtime_kb.delete_file(file_id)
# Update the KB's updated_at timestamp
@@ -121,13 +137,14 @@ class KnowledgeService:
async def delete_knowledge_base(self, kb_uuid: str) -> None:
"""删除知识库"""
await self.ap.rag_mgr.delete_knowledge_base(kb_uuid)
# Delete from DB first to commit the deletion, then clean up runtime/plugin (best-effort)
await self.ap.persistence_mgr.execute_async(
sqlalchemy.delete(persistence_rag.KnowledgeBase).where(persistence_rag.KnowledgeBase.uuid == kb_uuid)
)
# delete files
# NOTE: Chunk cleanup is for legacy (pre-plugin) KBs that stored chunks locally.
# For plugin-based Knowledge Engines, the Chunk table is not populated, so this is a no-op.
files = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_rag.File).where(persistence_rag.File.kb_id == kb_uuid)
)
@@ -140,3 +157,53 @@ class KnowledgeService:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.delete(persistence_rag.File).where(persistence_rag.File.uuid == file.uuid)
)
# Remove from runtime and notify plugin (best-effort, DB is already cleaned up)
await self.ap.rag_mgr.delete_knowledge_base(kb_uuid)
# ================= Knowledge Engine Discovery =================
async def list_knowledge_engines(self) -> list[dict]:
"""List all available Knowledge Engines from plugins."""
engines = []
if not self.ap.plugin_connector.is_enable_plugin:
return engines
# Get KnowledgeEngine plugins
try:
knowledge_engines = await self.ap.plugin_connector.list_knowledge_engines()
engines.extend(knowledge_engines)
except Exception as e:
self.ap.logger.warning(f'Failed to list Knowledge Engines from plugins: {e}')
return engines
async def list_parsers(self, mime_type: str | None = None) -> list[dict]:
"""List available parsers, optionally filtered by MIME type."""
if not self.ap.plugin_connector.is_enable_plugin:
return []
try:
parsers = await self.ap.plugin_connector.list_parsers()
if mime_type:
parsers = [p for p in parsers if mime_type in p.get('supported_mime_types', [])]
return parsers
except Exception as e:
self.ap.logger.warning(f'Failed to list parsers: {e}')
return []
async def get_engine_creation_schema(self, plugin_id: str) -> dict:
"""Get creation settings schema for a specific Knowledge Engine."""
try:
return await self.ap.plugin_connector.get_rag_creation_schema(plugin_id)
except Exception as e:
self.ap.logger.warning(f'Failed to get creation schema for {plugin_id}: {e}')
return {}
async def get_engine_retrieval_schema(self, plugin_id: str) -> dict:
"""Get retrieval settings schema for a specific Knowledge Engine."""
try:
return await self.ap.plugin_connector.get_rag_retrieval_schema(plugin_id)
except Exception as e:
self.ap.logger.warning(f'Failed to get retrieval schema for {plugin_id}: {e}')
return {}

View File

@@ -105,11 +105,16 @@ class LLMModelsService:
)
)
pipeline = result.first()
if pipeline is not None and pipeline.config['ai']['local-agent']['model'] == '':
pipeline_config = pipeline.config
pipeline_config['ai']['local-agent']['model'] = model_data['uuid']
pipeline_data = {'config': pipeline_config}
await self.ap.pipeline_service.update_pipeline(pipeline.uuid, pipeline_data)
if pipeline is not None:
model_config = pipeline.config.get('ai', {}).get('local-agent', {}).get('model', {})
if not model_config.get('primary', ''):
pipeline_config = pipeline.config
pipeline_config['ai']['local-agent']['model'] = {
'primary': model_data['uuid'],
'fallbacks': [],
}
pipeline_data = {'config': pipeline_config}
await self.ap.pipeline_service.update_pipeline(pipeline.uuid, pipeline_data)
return model_data['uuid']

View File

@@ -16,6 +16,57 @@ class MonitoringService:
def __init__(self, ap: app.Application) -> None:
self.ap = ap
# ========== Cleanup Methods ==========
async def cleanup_expired_records(self, retention_days: int) -> dict[str, int]:
"""Delete monitoring records older than the specified retention period.
Args:
retention_days: Number of days to retain records.
Returns:
A dict mapping table name to the number of deleted rows.
"""
cutoff = datetime.datetime.now(datetime.timezone.utc).replace(tzinfo=None) - datetime.timedelta(
days=retention_days
)
tables_and_columns: list[tuple[str, type, sqlalchemy.Column]] = [
(
'monitoring_messages',
persistence_monitoring.MonitoringMessage,
persistence_monitoring.MonitoringMessage.timestamp,
),
(
'monitoring_llm_calls',
persistence_monitoring.MonitoringLLMCall,
persistence_monitoring.MonitoringLLMCall.timestamp,
),
(
'monitoring_embedding_calls',
persistence_monitoring.MonitoringEmbeddingCall,
persistence_monitoring.MonitoringEmbeddingCall.timestamp,
),
(
'monitoring_errors',
persistence_monitoring.MonitoringError,
persistence_monitoring.MonitoringError.timestamp,
),
(
'monitoring_sessions',
persistence_monitoring.MonitoringSession,
persistence_monitoring.MonitoringSession.last_activity,
),
]
deleted_counts: dict[str, int] = {}
for table_name, model_cls, ts_column in tables_and_columns:
result = await self.ap.persistence_mgr.execute_async(sqlalchemy.delete(model_cls).where(ts_column < cutoff))
deleted_counts[table_name] = result.rowcount
return deleted_counts
# ========== Recording Methods ==========
async def record_message(
@@ -30,6 +81,7 @@ class MonitoringService:
level: str = 'info',
platform: str | None = None,
user_id: str | None = None,
user_name: str | None = None,
runner_name: str | None = None,
variables: str | None = None,
role: str = 'user',
@@ -49,6 +101,7 @@ class MonitoringService:
'level': level,
'platform': platform,
'user_id': user_id,
'user_name': user_name,
'runner_name': runner_name,
'variables': variables,
'role': role,
@@ -152,6 +205,7 @@ class MonitoringService:
pipeline_name: str,
platform: str | None = None,
user_id: str | None = None,
user_name: str | None = None,
) -> None:
"""Record a new session"""
session_data = {
@@ -166,6 +220,7 @@ class MonitoringService:
'is_active': True,
'platform': platform,
'user_id': user_id,
'user_name': user_name,
}
await self.ap.persistence_mgr.execute_async(
@@ -1128,3 +1183,261 @@ class MonitoringService:
}
for row in rows
]
# ========== Feedback Methods ==========
async def record_feedback(
self,
feedback_id: str,
feedback_type: int,
feedback_content: str | None = None,
inaccurate_reasons: list[str] | None = None,
bot_id: str | None = None,
bot_name: str | None = None,
pipeline_id: str | None = None,
pipeline_name: str | None = None,
session_id: str | None = None,
message_id: str | None = None,
stream_id: str | None = None,
user_id: str | None = None,
platform: str | None = None,
) -> str:
"""Record user feedback (like/dislike) from AI Bot conversation.
Args:
feedback_id: Unique feedback identifier from platform (e.g., WeChat Work)
feedback_type: 1 = like (thumbs up), 2 = dislike (thumbs down)
feedback_content: Optional user feedback text
inaccurate_reasons: List of reasons for inaccurate response (for dislike)
bot_id: Bot ID
bot_name: Bot name
pipeline_id: Pipeline ID
pipeline_name: Pipeline name
session_id: Session ID
message_id: Message ID
stream_id: Stream ID (for WeChat Work streaming messages)
user_id: User ID
platform: Platform name (e.g., 'wecom')
Returns:
The record ID
"""
import json
record_id = str(uuid.uuid4())
record_data = {
'id': record_id,
'timestamp': datetime.datetime.now(datetime.timezone.utc).replace(tzinfo=None),
'feedback_id': feedback_id,
'feedback_type': feedback_type,
'feedback_content': feedback_content,
'inaccurate_reasons': json.dumps(inaccurate_reasons, ensure_ascii=False) if inaccurate_reasons else None,
'bot_id': bot_id,
'bot_name': bot_name,
'pipeline_id': pipeline_id,
'pipeline_name': pipeline_name,
'session_id': session_id,
'message_id': message_id,
'stream_id': stream_id,
'user_id': user_id,
'platform': platform,
}
await self.ap.persistence_mgr.execute_async(
sqlalchemy.insert(persistence_monitoring.MonitoringFeedback).values(record_data)
)
return record_id
async def get_feedback_stats(
self,
bot_ids: list[str] | None = None,
pipeline_ids: list[str] | None = None,
start_time: datetime.datetime | None = None,
end_time: datetime.datetime | None = None,
) -> dict:
"""Get feedback statistics.
Returns:
Dictionary with total likes, dislikes, and breakdown by bot/pipeline
"""
conditions = []
if bot_ids:
conditions.append(persistence_monitoring.MonitoringFeedback.bot_id.in_(bot_ids))
if pipeline_ids:
conditions.append(persistence_monitoring.MonitoringFeedback.pipeline_id.in_(pipeline_ids))
if start_time:
conditions.append(persistence_monitoring.MonitoringFeedback.timestamp >= start_time)
if end_time:
conditions.append(persistence_monitoring.MonitoringFeedback.timestamp <= end_time)
# Get total likes (feedback_type = 1)
likes_query = sqlalchemy.select(sqlalchemy.func.count(persistence_monitoring.MonitoringFeedback.id)).where(
persistence_monitoring.MonitoringFeedback.feedback_type == 1
)
if conditions:
likes_query = likes_query.where(sqlalchemy.and_(*conditions))
likes_result = await self.ap.persistence_mgr.execute_async(likes_query)
total_likes = likes_result.scalar() or 0
# Get total dislikes (feedback_type = 2)
dislikes_query = sqlalchemy.select(sqlalchemy.func.count(persistence_monitoring.MonitoringFeedback.id)).where(
persistence_monitoring.MonitoringFeedback.feedback_type == 2
)
if conditions:
dislikes_query = dislikes_query.where(sqlalchemy.and_(*conditions))
dislikes_result = await self.ap.persistence_mgr.execute_async(dislikes_query)
total_dislikes = dislikes_result.scalar() or 0
# Get total feedback count
total_query = sqlalchemy.select(sqlalchemy.func.count(persistence_monitoring.MonitoringFeedback.id))
if conditions:
total_query = total_query.where(sqlalchemy.and_(*conditions))
total_result = await self.ap.persistence_mgr.execute_async(total_query)
total_feedback = total_result.scalar() or 0
# Calculate satisfaction rate
satisfaction_rate = (total_likes / total_feedback * 100) if total_feedback > 0 else 0
# Get feedback by bot
bot_stats_query = sqlalchemy.select(
persistence_monitoring.MonitoringFeedback.bot_id,
persistence_monitoring.MonitoringFeedback.bot_name,
sqlalchemy.func.count(persistence_monitoring.MonitoringFeedback.id).label('total'),
sqlalchemy.func.sum(
sqlalchemy.case((persistence_monitoring.MonitoringFeedback.feedback_type == 1, 1), else_=0)
).label('likes'),
sqlalchemy.func.sum(
sqlalchemy.case((persistence_monitoring.MonitoringFeedback.feedback_type == 2, 1), else_=0)
).label('dislikes'),
).group_by(
persistence_monitoring.MonitoringFeedback.bot_id,
persistence_monitoring.MonitoringFeedback.bot_name,
)
if conditions:
bot_stats_query = bot_stats_query.where(sqlalchemy.and_(*conditions))
bot_stats_result = await self.ap.persistence_mgr.execute_async(bot_stats_query)
bot_stats = [
{
'bot_id': row.bot_id,
'bot_name': row.bot_name,
'total': row.total,
'likes': row.likes or 0,
'dislikes': row.dislikes or 0,
}
for row in bot_stats_result.all()
]
return {
'total_feedback': total_feedback,
'total_likes': total_likes,
'total_dislikes': total_dislikes,
'satisfaction_rate': round(satisfaction_rate, 2),
'by_bot': bot_stats,
}
async def get_feedback_list(
self,
bot_ids: list[str] | None = None,
pipeline_ids: list[str] | None = None,
feedback_type: int | None = None,
start_time: datetime.datetime | None = None,
end_time: datetime.datetime | None = None,
limit: int = 100,
offset: int = 0,
) -> tuple[list[dict], int]:
"""Get feedback list with filters."""
conditions = []
if bot_ids:
conditions.append(persistence_monitoring.MonitoringFeedback.bot_id.in_(bot_ids))
if pipeline_ids:
conditions.append(persistence_monitoring.MonitoringFeedback.pipeline_id.in_(pipeline_ids))
if feedback_type is not None:
conditions.append(persistence_monitoring.MonitoringFeedback.feedback_type == feedback_type)
if start_time:
conditions.append(persistence_monitoring.MonitoringFeedback.timestamp >= start_time)
if end_time:
conditions.append(persistence_monitoring.MonitoringFeedback.timestamp <= end_time)
# Get total count
count_query = sqlalchemy.select(sqlalchemy.func.count(persistence_monitoring.MonitoringFeedback.id))
if conditions:
count_query = count_query.where(sqlalchemy.and_(*conditions))
count_result = await self.ap.persistence_mgr.execute_async(count_query)
total = count_result.scalar() or 0
# Get feedback list
query = sqlalchemy.select(persistence_monitoring.MonitoringFeedback).order_by(
persistence_monitoring.MonitoringFeedback.timestamp.desc()
)
if conditions:
query = query.where(sqlalchemy.and_(*conditions))
query = query.limit(limit).offset(offset)
result = await self.ap.persistence_mgr.execute_async(query)
rows = result.all()
return (
[
self.ap.persistence_mgr.serialize_model(
persistence_monitoring.MonitoringFeedback, row[0] if isinstance(row, tuple) else row
)
for row in rows
],
total,
)
async def export_feedback(
self,
bot_ids: list[str] | None = None,
pipeline_ids: list[str] | None = None,
start_time: datetime.datetime | None = None,
end_time: datetime.datetime | None = None,
limit: int = 100000,
) -> list[dict]:
"""Export feedback as list of dictionaries for CSV conversion."""
conditions = []
if bot_ids:
conditions.append(persistence_monitoring.MonitoringFeedback.bot_id.in_(bot_ids))
if pipeline_ids:
conditions.append(persistence_monitoring.MonitoringFeedback.pipeline_id.in_(pipeline_ids))
if start_time:
conditions.append(persistence_monitoring.MonitoringFeedback.timestamp >= start_time)
if end_time:
conditions.append(persistence_monitoring.MonitoringFeedback.timestamp <= end_time)
query = sqlalchemy.select(persistence_monitoring.MonitoringFeedback).order_by(
persistence_monitoring.MonitoringFeedback.timestamp.desc()
)
if conditions:
query = query.where(sqlalchemy.and_(*conditions))
query = query.limit(limit)
result = await self.ap.persistence_mgr.execute_async(query)
rows = result.all()
return [
{
'id': row[0].id if isinstance(row, tuple) else row.id,
'timestamp': self._format_timestamp(row[0].timestamp if isinstance(row, tuple) else row.timestamp),
'feedback_id': row[0].feedback_id if isinstance(row, tuple) else row.feedback_id,
'feedback_type': 'like'
if (row[0].feedback_type if isinstance(row, tuple) else row.feedback_type) == 1
else 'dislike',
'feedback_content': row[0].feedback_content if isinstance(row, tuple) else row.feedback_content,
'inaccurate_reasons': row[0].inaccurate_reasons if isinstance(row, tuple) else row.inaccurate_reasons,
'bot_id': row[0].bot_id if isinstance(row, tuple) else row.bot_id,
'bot_name': row[0].bot_name if isinstance(row, tuple) else row.bot_name,
'pipeline_id': row[0].pipeline_id if isinstance(row, tuple) else row.pipeline_id,
'pipeline_name': row[0].pipeline_name if isinstance(row, tuple) else row.pipeline_name,
'session_id': row[0].session_id if isinstance(row, tuple) else row.session_id,
'message_id': row[0].message_id if isinstance(row, tuple) else row.message_id,
'stream_id': row[0].stream_id if isinstance(row, tuple) else row.stream_id,
'user_id': row[0].user_id if isinstance(row, tuple) else row.user_id,
'platform': row[0].platform if isinstance(row, tuple) else row.platform,
}
for row in rows
]

View File

@@ -9,6 +9,7 @@ from ..platform import botmgr as im_mgr
from ..platform.webhook_pusher import WebhookPusher
from ..provider.session import sessionmgr as llm_session_mgr
from ..provider.modelmgr import modelmgr as llm_model_mgr
from langbot.pkg.provider.tools import toolmgr as llm_tool_mgr
from ..config import manager as config_mgr
from ..command import cmdmgr
@@ -29,14 +30,15 @@ from ..api.http.service import knowledge as knowledge_service
from ..api.http.service import mcp as mcp_service
from ..api.http.service import apikey as apikey_service
from ..api.http.service import webhook as webhook_service
from ..api.http.service import external_kb as external_kb_service
from ..api.http.service import monitoring as monitoring_service
from ..discover import engine as discover_engine
from ..storage import mgr as storagemgr
from ..utils import logcache
from . import taskmgr
from . import entities as core_entities
from ..rag.knowledge import kbmgr as rag_mgr
from ..rag.service import RAGRuntimeService
from ..vector import mgr as vectordb_mgr
from ..telemetry import telemetry as telemetry_module
from ..survey import manager as survey_module
@@ -63,6 +65,7 @@ class Application:
model_mgr: llm_model_mgr.ModelManager = None
rag_mgr: rag_mgr.RAGManager = None
rag_runtime_service: RAGRuntimeService = None
# TODO move to pipeline
tool_mgr: llm_tool_mgr.ToolManager = None
@@ -138,8 +141,6 @@ class Application:
knowledge_service: knowledge_service.KnowledgeService = None
external_kb_service: external_kb_service.ExternalKBService = None
mcp_service: mcp_service.MCPService = None
apikey_service: apikey_service.ApiKeyService = None
@@ -187,6 +188,34 @@ class Application:
scopes=[core_entities.LifecycleControlScope.APPLICATION],
)
# Start monitoring data cleanup task if enabled
monitoring_cfg = self.instance_config.data.get('monitoring', {})
auto_cleanup_cfg = monitoring_cfg.get('auto_cleanup', {})
if auto_cleanup_cfg.get('enabled', True):
retention_days = auto_cleanup_cfg.get('retention_days', 30)
check_interval_hours = auto_cleanup_cfg.get('check_interval_hours', 1)
async def monitoring_cleanup_loop():
check_interval_seconds = check_interval_hours * 3600
while True:
try:
deleted = await self.monitoring_service.cleanup_expired_records(retention_days)
total_deleted = sum(deleted.values())
if total_deleted > 0:
self.logger.info(
f'Monitoring auto-cleanup: deleted {total_deleted} expired records '
f'(retention={retention_days}d): {deleted}'
)
except Exception as e:
self.logger.warning(f'Monitoring auto-cleanup error: {e}')
await asyncio.sleep(check_interval_seconds)
self.task_mgr.create_task(
monitoring_cleanup_loop(),
name='monitoring-cleanup',
scopes=[core_entities.LifecycleControlScope.APPLICATION],
)
self.task_mgr.create_task(
never_ending(),
name='never-ending-task',

View File

@@ -12,6 +12,7 @@ from ...provider.session import sessionmgr as llm_session_mgr
from ...provider.modelmgr import modelmgr as llm_model_mgr
from ...provider.tools import toolmgr as llm_tool_mgr
from ...rag.knowledge import kbmgr as rag_mgr
from ...rag.service import RAGRuntimeService
from ...platform import botmgr as im_mgr
from ...platform.webhook_pusher import WebhookPusher
from ...persistence import mgr as persistencemgr
@@ -26,7 +27,6 @@ from ...api.http.service import knowledge as knowledge_service
from ...api.http.service import mcp as mcp_service
from ...api.http.service import apikey as apikey_service
from ...api.http.service import webhook as webhook_service
from ...api.http.service import external_kb as external_kb_service
from ...api.http.service import monitoring as monitoring_service
from ...discover import engine as discover_engine
from ...storage import mgr as storagemgr
@@ -73,9 +73,6 @@ class BuildAppStage(stage.BootingStage):
knowledge_service_inst = knowledge_service.KnowledgeService(ap)
ap.knowledge_service = knowledge_service_inst
external_kb_service_inst = external_kb_service.ExternalKBService(ap)
ap.external_kb_service = external_kb_service_inst
mcp_service_inst = mcp_service.MCPService(ap)
ap.mcp_service = mcp_service_inst
@@ -152,6 +149,9 @@ class BuildAppStage(stage.BootingStage):
await rag_mgr_inst.initialize()
ap.rag_mgr = rag_mgr_inst
# Initialize RAG Runtime Service for plugins
ap.rag_runtime_service = RAGRuntimeService(ap)
# 初始化向量数据库管理器
vectordb_mgr_inst = vectordb_mgr.VectorDBManager(ap)
await vectordb_mgr_inst.initialize()

View File

@@ -74,20 +74,26 @@ def _apply_env_overrides_to_config(cfg: dict) -> dict:
current = cfg
for i, key in enumerate(keys):
if not isinstance(current, dict) or key not in current:
if not isinstance(current, dict):
break
if i == len(keys) - 1:
# At the final key - check if it's a scalar value
if isinstance(current[key], (dict, list)):
# Skip dict and list types
pass
# At the final key
if key in current:
if isinstance(current[key], (dict, list)):
# Skip dict and list types
pass
else:
# Valid scalar value - convert and set it
converted_value = convert_value(env_value, current[key])
current[key] = converted_value
else:
# Valid scalar value - convert and set it
converted_value = convert_value(env_value, current[key])
current[key] = converted_value
# Key doesn't exist yet - create it as string
current[key] = env_value
else:
# Navigate deeper
# Navigate deeper - create intermediate dict if needed
if key not in current:
current[key] = {}
current = current[key]
return cfg
@@ -146,16 +152,50 @@ class LoadConfigStage(stage.BootingStage):
await ap.instance_config.dump_config()
# load or generate instance id
ap.instance_id = await config.load_json_config(
'data/labels/instance_id.json',
template_data={
'instance_id': f'instance_{str(uuid.uuid4())}',
'instance_create_ts': int(time.time()),
},
completion=False,
)
# Priority:
# 1. system.instance_id from config.yaml (can be set via SYSTEM__INSTANCE_ID env var)
# 2. data/labels/instance_id.json (if file exists)
# 3. Generate new and save to file
config_instance_id = ap.instance_config.data.get('system', {}).get('instance_id', '')
constants.instance_id = ap.instance_id.data['instance_id']
if config_instance_id:
# Use the instance_id from config.yaml
constants.instance_id = config_instance_id
# Still load/create the file for backward compat, but don't use its value
ap.instance_id = await config.load_json_config(
'data/labels/instance_id.json',
template_data={
'instance_id': f'instance_{str(uuid.uuid4())}',
'instance_create_ts': int(time.time()),
},
completion=False,
)
else:
# Try loading file-based instance id
instance_id_path = os.path.join('data', 'labels', 'instance_id.json')
if os.path.exists(instance_id_path):
# File exists, read it
ap.instance_id = await config.load_json_config(
'data/labels/instance_id.json',
template_data={
'instance_id': '',
'instance_create_ts': 0,
},
completion=False,
)
constants.instance_id = ap.instance_id.data['instance_id']
else:
# Neither config nor file, generate new and save to file
new_id = f'instance_{str(uuid.uuid4())}'
ap.instance_id = await config.load_json_config(
'data/labels/instance_id.json',
template_data={
'instance_id': new_id,
'instance_create_ts': int(time.time()),
},
completion=False,
)
constants.instance_id = new_id
constants.edition = ap.instance_config.data.get('system', {}).get('edition', 'community')
print(f'LangBot instance id: {constants.instance_id}')

View File

@@ -17,9 +17,13 @@ class TaskContext:
log: str
"""Log"""
metadata: dict
"""Structured metadata for progress reporting"""
def __init__(self):
self.current_action = 'default'
self.log = ''
self.metadata = {}
def _log(self, msg: str):
self.log += msg + '\n'
@@ -38,7 +42,7 @@ class TaskContext:
self._log(f'{datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S")} | {self.current_action} | {msg}')
def to_dict(self) -> dict:
return {'current_action': self.current_action, 'log': self.log}
return {'current_action': self.current_action, 'log': self.log, 'metadata': self.metadata}
@staticmethod
def new() -> TaskContext:
@@ -211,9 +215,14 @@ class AsyncTaskManager:
def get_tasks_dict(
self,
type: str = None,
kind: str = None,
) -> dict:
return {
'tasks': [t.to_dict() for t in self.tasks if type is None or t.task_type == type],
'tasks': [
t.to_dict()
for t in self.tasks
if (type is None or t.task_type == type) and (kind is None or t.kind == kind)
],
'id_index': TaskWrapper._id_index,
}

View File

@@ -17,11 +17,23 @@ class I18nString(pydantic.BaseModel):
"""英文"""
zh_Hans: typing.Optional[str] = None
"""中文"""
"""简体中文"""
zh_Hant: typing.Optional[str] = None
"""繁体中文"""
ja_JP: typing.Optional[str] = None
"""日文"""
th_TH: typing.Optional[str] = None
"""泰文"""
vi_VN: typing.Optional[str] = None
"""越南文"""
es_ES: typing.Optional[str] = None
"""西班牙文"""
def to_dict(self) -> dict:
"""转换为字典"""
dic = {}
@@ -29,8 +41,16 @@ class I18nString(pydantic.BaseModel):
dic['en_US'] = self.en_US
if self.zh_Hans is not None:
dic['zh_Hans'] = self.zh_Hans
if self.zh_Hant is not None:
dic['zh_Hant'] = self.zh_Hant
if self.ja_JP is not None:
dic['ja_JP'] = self.ja_JP
if self.th_TH is not None:
dic['th_TH'] = self.th_TH
if self.vi_VN is not None:
dic['vi_VN'] = self.vi_VN
if self.es_ES is not None:
dic['es_ES'] = self.es_ES
return dic

View File

@@ -16,6 +16,7 @@ class Bot(Base):
enable = sqlalchemy.Column(sqlalchemy.Boolean, nullable=False, default=False)
use_pipeline_name = sqlalchemy.Column(sqlalchemy.String(255), nullable=True)
use_pipeline_uuid = sqlalchemy.Column(sqlalchemy.String(255), nullable=True)
pipeline_routing_rules = sqlalchemy.Column(sqlalchemy.JSON, nullable=False, server_default='[]')
created_at = sqlalchemy.Column(sqlalchemy.DateTime, nullable=False, server_default=sqlalchemy.func.now())
updated_at = sqlalchemy.Column(
sqlalchemy.DateTime,

View File

@@ -20,6 +20,7 @@ class MonitoringMessage(Base):
level = sqlalchemy.Column(sqlalchemy.String(50), nullable=False) # info, warning, error, debug
platform = sqlalchemy.Column(sqlalchemy.String(255), nullable=True)
user_id = sqlalchemy.Column(sqlalchemy.String(255), nullable=True)
user_name = sqlalchemy.Column(sqlalchemy.String(255), nullable=True) # User display name
runner_name = sqlalchemy.Column(sqlalchemy.String(255), nullable=True) # Runner name for this query
variables = sqlalchemy.Column(sqlalchemy.Text, nullable=True) # Query variables as JSON string
role = sqlalchemy.Column(sqlalchemy.String(50), nullable=True, default='user') # user, assistant
@@ -64,6 +65,7 @@ class MonitoringSession(Base):
is_active = sqlalchemy.Column(sqlalchemy.Boolean, nullable=False, default=True, index=True)
platform = sqlalchemy.Column(sqlalchemy.String(255), nullable=True)
user_id = sqlalchemy.Column(sqlalchemy.String(255), nullable=True)
user_name = sqlalchemy.Column(sqlalchemy.String(255), nullable=True) # User display name
class MonitoringError(Base):
@@ -104,3 +106,26 @@ class MonitoringEmbeddingCall(Base):
session_id = sqlalchemy.Column(sqlalchemy.String(255), nullable=True, index=True)
message_id = sqlalchemy.Column(sqlalchemy.String(255), nullable=True, index=True)
call_type = sqlalchemy.Column(sqlalchemy.String(50), nullable=True) # embedding, retrieve
class MonitoringFeedback(Base):
"""User feedback records (like/dislike) from AI Bot conversations"""
__tablename__ = 'monitoring_feedback'
id = sqlalchemy.Column(sqlalchemy.String(255), primary_key=True)
timestamp = sqlalchemy.Column(sqlalchemy.DateTime, nullable=False, index=True)
feedback_id = sqlalchemy.Column(sqlalchemy.String(255), nullable=False, unique=True, index=True)
feedback_type = sqlalchemy.Column(sqlalchemy.Integer, nullable=False) # 1=like, 2=dislike
feedback_content = sqlalchemy.Column(sqlalchemy.Text, nullable=True) # User feedback text
inaccurate_reasons = sqlalchemy.Column(sqlalchemy.Text, nullable=True) # JSON list of inaccurate reasons
# Context fields
bot_id = sqlalchemy.Column(sqlalchemy.String(255), nullable=True, index=True)
bot_name = sqlalchemy.Column(sqlalchemy.String(255), nullable=True)
pipeline_id = sqlalchemy.Column(sqlalchemy.String(255), nullable=True, index=True)
pipeline_name = sqlalchemy.Column(sqlalchemy.String(255), nullable=True)
session_id = sqlalchemy.Column(sqlalchemy.String(255), nullable=True, index=True)
message_id = sqlalchemy.Column(sqlalchemy.String(255), nullable=True, index=True)
stream_id = sqlalchemy.Column(sqlalchemy.String(255), nullable=True, index=True)
user_id = sqlalchemy.Column(sqlalchemy.String(255), nullable=True)
platform = sqlalchemy.Column(sqlalchemy.String(255), nullable=True) # e.g., wecom

View File

@@ -10,8 +10,21 @@ class KnowledgeBase(Base):
emoji = sqlalchemy.Column(sqlalchemy.String(10), nullable=True, default='📚')
created_at = sqlalchemy.Column(sqlalchemy.DateTime, default=sqlalchemy.func.now())
updated_at = sqlalchemy.Column(sqlalchemy.DateTime, default=sqlalchemy.func.now(), onupdate=sqlalchemy.func.now())
embedding_model_uuid = sqlalchemy.Column(sqlalchemy.String, default='')
top_k = sqlalchemy.Column(sqlalchemy.Integer, default=5)
# New fields for plugin-based RAG
knowledge_engine_plugin_id = sqlalchemy.Column(sqlalchemy.String, nullable=True)
collection_id = sqlalchemy.Column(sqlalchemy.String, nullable=True)
creation_settings = sqlalchemy.Column(sqlalchemy.JSON, nullable=True, default=None)
retrieval_settings = sqlalchemy.Column(sqlalchemy.JSON, nullable=True, default=None)
# Field sets for different operations
MUTABLE_FIELDS = {'name', 'description', 'retrieval_settings'}
"""Fields that can be updated after creation."""
CREATE_FIELDS = MUTABLE_FIELDS | {'uuid', 'knowledge_engine_plugin_id', 'collection_id', 'creation_settings'}
"""Fields used when creating a new knowledge base."""
ALL_DB_FIELDS = CREATE_FIELDS | {'emoji', 'created_at', 'updated_at'}
"""All fields stored in database (for loading from DB row)."""
class File(Base):
@@ -29,16 +42,3 @@ class Chunk(Base):
uuid = sqlalchemy.Column(sqlalchemy.String(255), primary_key=True, unique=True)
file_id = sqlalchemy.Column(sqlalchemy.String(255), nullable=True)
text = sqlalchemy.Column(sqlalchemy.Text)
class ExternalKnowledgeBase(Base):
__tablename__ = 'external_knowledge_bases'
uuid = sqlalchemy.Column(sqlalchemy.String(255), primary_key=True, unique=True)
name = sqlalchemy.Column(sqlalchemy.String, index=True)
description = sqlalchemy.Column(sqlalchemy.Text)
emoji = sqlalchemy.Column(sqlalchemy.String(10), nullable=True, default='🔗')
plugin_author = sqlalchemy.Column(sqlalchemy.String, nullable=False)
plugin_name = sqlalchemy.Column(sqlalchemy.String, nullable=False)
retriever_name = sqlalchemy.Column(sqlalchemy.String, nullable=False)
retriever_config = sqlalchemy.Column(sqlalchemy.JSON, nullable=False, default={})
created_at = sqlalchemy.Column(sqlalchemy.DateTime, default=sqlalchemy.func.now())

View File

@@ -2,18 +2,16 @@ from __future__ import annotations
import datetime
import typing
import json
import uuid
import sqlalchemy.ext.asyncio as sqlalchemy_asyncio
import sqlalchemy
from . import database, migration
from ..entity.persistence import base, pipeline, metadata, model as persistence_model
from ..entity.persistence import base, metadata, model as persistence_model
from ..entity import persistence
from ..core import app
from ..utils import constants, importutil
from ..api.http.service import pipeline as pipeline_service
from . import databases, migrations
importutil.import_modules_in_pkg(databases)
@@ -78,7 +76,6 @@ class PersistenceManager:
self.ap.logger.info(f'Successfully upgraded database to version {last_migration_number}.')
await self.write_default_pipeline()
await self.write_space_model_providers()
async def create_tables(self):
@@ -101,29 +98,6 @@ class PersistenceManager:
if row is None:
await self.execute_async(sqlalchemy.insert(metadata.Metadata).values(item))
async def write_default_pipeline(self):
# write default pipeline
result = await self.execute_async(sqlalchemy.select(pipeline.LegacyPipeline))
default_pipeline_uuid = None
if result.first() is None:
self.ap.logger.info('Creating default pipeline...')
pipeline_config = json.loads(importutil.read_resource_file('templates/default-pipeline-config.json'))
default_pipeline_uuid = str(uuid.uuid4())
pipeline_data = {
'uuid': default_pipeline_uuid,
'for_version': self.ap.ver_mgr.get_current_version(),
'stages': pipeline_service.default_stage_order,
'is_default': True,
'name': 'ChatPipeline',
'description': 'Default pipeline, new bots will be bound to this pipeline | 默认提供的流水线,您配置的机器人将自动绑定到此流水线',
'config': pipeline_config,
'extensions_preferences': {},
}
await self.execute_async(sqlalchemy.insert(pipeline.LegacyPipeline).values(pipeline_data))
async def write_space_model_providers(self):
space_models_gateway_api_url = self.ap.instance_config.data.get('space', {}).get(
'models_gateway_api_url', 'https://api.langbot.cloud/v1'

View File

@@ -0,0 +1,161 @@
import sqlalchemy
from .. import migration
@migration.migration_class(20)
class DBMigrateKnowledgeEnginePluginArchitecture(migration.DBMigration):
"""Migrate to unified Knowledge Engine plugin architecture.
Changes:
- Backup existing knowledge_bases data to knowledge_bases_backup
- Clear knowledge_bases table and add new plugin architecture columns
- Drop old columns (PostgreSQL only; SQLite leaves them unmapped)
- Preserve external_knowledge_bases table as-is for future migration
- Set rag_plugin_migration_needed flag in metadata if old data exists
"""
async def upgrade(self):
"""Upgrade"""
has_internal_data = await self._backup_knowledge_bases()
has_external_data = await self._check_external_knowledge_bases()
await self._clear_knowledge_bases()
await self._add_columns_to_knowledge_bases()
await self._drop_old_columns()
if has_internal_data or has_external_data:
await self._set_migration_flag()
async def _get_table_columns(self, table_name: str) -> list[str]:
"""Get column names from a table (works for both SQLite and PostgreSQL)."""
if self.ap.persistence_mgr.db.name == 'postgresql':
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'SELECT column_name FROM information_schema.columns WHERE table_name = :table_name;'
).bindparams(table_name=table_name)
)
return [row[0] for row in result.fetchall()]
else:
# SQLite PRAGMA does not support bind parameters; validate identifier.
if not table_name.isidentifier():
raise ValueError(f'Invalid table name: {table_name}')
result = await self.ap.persistence_mgr.execute_async(sqlalchemy.text(f'PRAGMA table_info({table_name});'))
return [row[1] for row in result.fetchall()]
async def _table_exists(self, table_name: str) -> bool:
"""Check if a table exists."""
if self.ap.persistence_mgr.db.name == 'postgresql':
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'SELECT EXISTS (SELECT FROM information_schema.tables WHERE table_name = :table_name);'
).bindparams(table_name=table_name)
)
return result.scalar()
else:
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("SELECT name FROM sqlite_master WHERE type='table' AND name=:table_name;").bindparams(
table_name=table_name
)
)
return result.first() is not None
async def _backup_knowledge_bases(self) -> bool:
"""Backup knowledge_bases data. Returns True if data was backed up."""
result = await self.ap.persistence_mgr.execute_async(sqlalchemy.text('SELECT COUNT(*) FROM knowledge_bases;'))
count = result.scalar()
if count == 0:
return False
# Drop backup table if it already exists (from a previous failed migration)
if await self._table_exists('knowledge_bases_backup'):
await self.ap.persistence_mgr.execute_async(sqlalchemy.text('DROP TABLE knowledge_bases_backup;'))
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('CREATE TABLE knowledge_bases_backup AS SELECT * FROM knowledge_bases;')
)
self.ap.logger.info(
'Backed up %d knowledge base(s) to knowledge_bases_backup table.',
count,
)
return True
async def _check_external_knowledge_bases(self) -> bool:
"""Check if external_knowledge_bases table exists and has data.
The table is preserved as-is (not dropped) for future migration.
"""
if not await self._table_exists('external_knowledge_bases'):
return False
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT COUNT(*) FROM external_knowledge_bases;')
)
count = result.scalar()
if count > 0:
self.ap.logger.info(
'Found %d external knowledge base(s) in external_knowledge_bases table. '
'Table preserved for future migration.',
count,
)
return count > 0
async def _clear_knowledge_bases(self):
"""Clear all rows from knowledge_bases table (preserve table structure)."""
await self.ap.persistence_mgr.execute_async(sqlalchemy.text('DELETE FROM knowledge_bases;'))
async def _add_columns_to_knowledge_bases(self):
"""Add new RAG plugin architecture columns to knowledge_bases table."""
columns = await self._get_table_columns('knowledge_bases')
new_columns = {
'knowledge_engine_plugin_id': 'VARCHAR',
'collection_id': 'VARCHAR',
'creation_settings': 'TEXT', # JSON stored as TEXT for SQLite compatibility
'retrieval_settings': 'TEXT',
}
for col_name, col_type in new_columns.items():
if col_name not in columns:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(f'ALTER TABLE knowledge_bases ADD COLUMN {col_name} {col_type};')
)
async def _drop_old_columns(self):
"""Drop embedding_model_uuid and top_k columns (PostgreSQL only).
SQLite does not support DROP COLUMN in older versions, so we leave the
columns in place — the SQLAlchemy entity simply won't map them.
"""
if self.ap.persistence_mgr.db.name != 'postgresql':
return
columns = await self._get_table_columns('knowledge_bases')
if 'embedding_model_uuid' in columns:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('ALTER TABLE knowledge_bases DROP COLUMN embedding_model_uuid;')
)
if 'top_k' in columns:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('ALTER TABLE knowledge_bases DROP COLUMN top_k;')
)
async def _set_migration_flag(self):
"""Set rag_plugin_migration_needed flag in metadata table."""
# Check if the key already exists
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("SELECT value FROM metadata WHERE key = 'rag_plugin_migration_needed';")
)
row = result.first()
if row is not None:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("UPDATE metadata SET value = 'true' WHERE key = 'rag_plugin_migration_needed';")
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("INSERT INTO metadata (key, value) VALUES ('rag_plugin_migration_needed', 'true');")
)
self.ap.logger.info('Set rag_plugin_migration_needed=true in metadata.')
async def downgrade(self):
"""Downgrade"""
pass

View File

@@ -0,0 +1,74 @@
from .. import migration
import sqlalchemy
import json
@migration.migration_class(21)
class DBMigrateMergeExceptionHandling(migration.DBMigration):
"""Merge hide-exception and block-failed-request-output into a single exception-handling select option,
and add failure-hint field.
Conversion logic:
- block-failed-request-output=true -> exception-handling: hide
- hide-exception=true -> exception-handling: show-hint
- hide-exception=false -> exception-handling: show-error
"""
async def upgrade(self):
"""Upgrade"""
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT uuid, config FROM legacy_pipelines')
)
pipelines = result.fetchall()
current_version = self.ap.ver_mgr.get_current_version()
for pipeline_row in pipelines:
uuid = pipeline_row[0]
config = json.loads(pipeline_row[1]) if isinstance(pipeline_row[1], str) else pipeline_row[1]
if 'output' not in config:
config['output'] = {}
if 'misc' not in config['output']:
config['output']['misc'] = {}
misc = config['output']['misc']
# Determine new exception-handling value from legacy fields
hide_exception = misc.get('hide-exception', True)
block_failed = misc.get('block-failed-request-output', False)
if block_failed:
exception_handling = 'hide'
elif hide_exception:
exception_handling = 'show-hint'
else:
exception_handling = 'show-error'
misc['exception-handling'] = exception_handling
# Add failure-hint with default value
misc['failure-hint'] = 'Request failed.'
# Remove legacy fields
misc.pop('hide-exception', None)
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET config = :config::jsonb, for_version = :for_version WHERE uuid = :uuid'
),
{'config': json.dumps(config), 'for_version': current_version, 'uuid': uuid},
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET config = :config, for_version = :for_version WHERE uuid = :uuid'
),
{'config': json.dumps(config), 'for_version': current_version, 'uuid': uuid},
)
async def downgrade(self):
"""Downgrade"""
pass

View File

@@ -0,0 +1,73 @@
import sqlalchemy
from .. import migration
@migration.migration_class(22)
class DBMigrateMonitoringUserId(migration.DBMigration):
"""Add user_id and user_name columns to monitoring_sessions table
This migration adds the missing user_id column and also ensures user_name
column exists (in case migration 21 failed or was skipped).
"""
async def _table_exists(self, table_name: str) -> bool:
"""Check if a table exists (works for both SQLite and PostgreSQL)."""
if self.ap.persistence_mgr.db.name == 'postgresql':
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'SELECT EXISTS (SELECT FROM information_schema.tables WHERE table_name = :table_name);'
).bindparams(table_name=table_name)
)
return bool(result.scalar())
else:
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("SELECT name FROM sqlite_master WHERE type='table' AND name=:table_name;").bindparams(
table_name=table_name
)
)
return result.first() is not None
async def _get_table_columns(self, table_name: str) -> list[str]:
"""Get column names from a table (works for both SQLite and PostgreSQL)."""
if self.ap.persistence_mgr.db.name == 'postgresql':
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'SELECT column_name FROM information_schema.columns WHERE table_name = :table_name;'
).bindparams(table_name=table_name)
)
return [row[0] for row in result.fetchall()]
else:
if not table_name.isidentifier():
raise ValueError(f'Invalid table name: {table_name}')
result = await self.ap.persistence_mgr.execute_async(sqlalchemy.text(f'PRAGMA table_info({table_name});'))
return [row[1] for row in result.fetchall()]
async def _add_column_if_not_exists(self, table_name: str, column_name: str, column_type: str):
"""Add a column to a table if it does not already exist."""
columns = await self._get_table_columns(table_name)
if column_name in columns:
self.ap.logger.debug('%s column already exists in %s.', column_name, table_name)
return
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(f'ALTER TABLE {table_name} ADD COLUMN {column_name} {column_type};')
)
self.ap.logger.info('Added %s column to %s table.', column_name, table_name)
async def upgrade(self):
# Check if monitoring_sessions table exists
if not await self._table_exists('monitoring_sessions'):
self.ap.logger.warning('monitoring_sessions table does not exist, skipping migration.')
return
# Add user_id column to monitoring_sessions table
await self._add_column_if_not_exists('monitoring_sessions', 'user_id', 'VARCHAR(255)')
# Add user_name column to monitoring_sessions table (in case migration 21 failed)
await self._add_column_if_not_exists('monitoring_sessions', 'user_name', 'VARCHAR(255)')
# Add user_name column to monitoring_messages table (in case migration 21 failed)
if await self._table_exists('monitoring_messages'):
await self._add_column_if_not_exists('monitoring_messages', 'user_name', 'VARCHAR(255)')
async def downgrade(self):
pass

View File

@@ -0,0 +1,102 @@
from .. import migration
import sqlalchemy
import json
@migration.migration_class(23)
class DBMigrateModelFallbackConfig(migration.DBMigration):
"""Convert model field from plain UUID string to object with primary/fallbacks"""
async def upgrade(self):
"""Upgrade"""
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT uuid, config FROM legacy_pipelines')
)
pipelines = result.fetchall()
current_version = self.ap.ver_mgr.get_current_version()
for pipeline_row in pipelines:
uuid = pipeline_row[0]
config = json.loads(pipeline_row[1]) if isinstance(pipeline_row[1], str) else pipeline_row[1]
if 'ai' not in config or 'local-agent' not in config['ai']:
continue
local_agent = config['ai']['local-agent']
changed = False
# Convert model from string to object
model_value = local_agent.get('model', '')
if isinstance(model_value, str):
local_agent['model'] = {
'primary': model_value,
'fallbacks': [],
}
changed = True
# Remove leftover fallback-models field if present
if 'fallback-models' in local_agent:
del local_agent['fallback-models']
changed = True
if not changed:
continue
# Update using raw SQL with compatibility for both SQLite and PostgreSQL
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET config = :config::jsonb, for_version = :for_version WHERE uuid = :uuid'
),
{'config': json.dumps(config), 'for_version': current_version, 'uuid': uuid},
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET config = :config, for_version = :for_version WHERE uuid = :uuid'
),
{'config': json.dumps(config), 'for_version': current_version, 'uuid': uuid},
)
async def downgrade(self):
"""Downgrade"""
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT uuid, config FROM legacy_pipelines')
)
pipelines = result.fetchall()
current_version = self.ap.ver_mgr.get_current_version()
for pipeline_row in pipelines:
uuid = pipeline_row[0]
config = json.loads(pipeline_row[1]) if isinstance(pipeline_row[1], str) else pipeline_row[1]
if 'ai' not in config or 'local-agent' not in config['ai']:
continue
local_agent = config['ai']['local-agent']
# Convert model from object back to string
model_value = local_agent.get('model', '')
if isinstance(model_value, dict):
local_agent['model'] = model_value.get('primary', '')
else:
continue
# Update using raw SQL with compatibility for both SQLite and PostgreSQL
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET config = :config::jsonb, for_version = :for_version WHERE uuid = :uuid'
),
{'config': json.dumps(config), 'for_version': current_version, 'uuid': uuid},
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET config = :config, for_version = :for_version WHERE uuid = :uuid'
),
{'config': json.dumps(config), 'for_version': current_version, 'uuid': uuid},
)

View File

@@ -0,0 +1,49 @@
from .. import migration
import sqlalchemy
import json
@migration.migration_class(24)
class DBMigrateWecomBotWebSocketMode(migration.DBMigration):
"""Add enable-webhook field to existing wecombot adapter configs.
Existing wecombot bots were all using webhook mode, so we set
enable-webhook=true to preserve their behavior after the new
WebSocket long connection mode is introduced as default.
"""
async def upgrade(self):
"""Upgrade"""
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("SELECT uuid, adapter_config FROM bots WHERE adapter = 'wecombot'")
)
bots = result.fetchall()
for bot_row in bots:
bot_uuid = bot_row[0]
adapter_config = json.loads(bot_row[1]) if isinstance(bot_row[1], str) else bot_row[1]
if 'enable-webhook' in adapter_config:
continue
# Determine mode based on existing config: if webhook fields are present, keep webhook mode
has_webhook_config = bool(
adapter_config.get('Token') and adapter_config.get('EncodingAESKey') and adapter_config.get('Corpid')
)
adapter_config['enable-webhook'] = has_webhook_config
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('UPDATE bots SET adapter_config = :config::jsonb WHERE uuid = :uuid'),
{'config': json.dumps(adapter_config), 'uuid': bot_uuid},
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('UPDATE bots SET adapter_config = :config WHERE uuid = :uuid'),
{'config': json.dumps(adapter_config), 'uuid': bot_uuid},
)
async def downgrade(self):
"""Downgrade"""
pass

View File

@@ -0,0 +1,15 @@
import sqlalchemy
from .. import migration
@migration.migration_class(25)
class DBMigrateBotPipelineRoutingRules(migration.DBMigration):
"""Add pipeline_routing_rules column to bots table"""
async def upgrade(self):
sql_text = sqlalchemy.text("ALTER TABLE bots ADD COLUMN pipeline_routing_rules JSON NOT NULL DEFAULT '[]'")
await self.ap.persistence_mgr.execute_async(sql_text)
async def downgrade(self):
sql_text = sqlalchemy.text('ALTER TABLE bots DROP COLUMN pipeline_routing_rules')
await self.ap.persistence_mgr.execute_async(sql_text)

View File

@@ -37,6 +37,7 @@ class PendingMessage:
message_chain: platform_message.MessageChain
adapter: abstract_platform_adapter.AbstractMessagePlatformAdapter
pipeline_uuid: typing.Optional[str]
routed_by_rule: bool = False
timestamp: float = field(default_factory=time.time)
@@ -125,6 +126,7 @@ class MessageAggregator:
message_chain: platform_message.MessageChain,
adapter: abstract_platform_adapter.AbstractMessagePlatformAdapter,
pipeline_uuid: typing.Optional[str] = None,
routed_by_rule: bool = False,
) -> None:
"""Add a message to the aggregation buffer
@@ -145,6 +147,7 @@ class MessageAggregator:
message_chain=message_chain,
adapter=adapter,
pipeline_uuid=pipeline_uuid,
routed_by_rule=routed_by_rule,
)
return
@@ -159,6 +162,7 @@ class MessageAggregator:
message_chain=message_chain,
adapter=adapter,
pipeline_uuid=pipeline_uuid,
routed_by_rule=routed_by_rule,
)
force_flush = False
@@ -217,6 +221,7 @@ class MessageAggregator:
message_chain=msg.message_chain,
adapter=msg.adapter,
pipeline_uuid=msg.pipeline_uuid,
routed_by_rule=msg.routed_by_rule,
)
return
@@ -231,6 +236,7 @@ class MessageAggregator:
message_chain=merged_msg.message_chain,
adapter=merged_msg.adapter,
pipeline_uuid=merged_msg.pipeline_uuid,
routed_by_rule=merged_msg.routed_by_rule,
)
def _merge_messages(self, messages: list[PendingMessage]) -> PendingMessage:

View File

@@ -0,0 +1,105 @@
from __future__ import annotations
import logging
logger = logging.getLogger(__name__)
# metadata type -> coercion function
_COERCE_MAP = {
'integer': lambda v: int(v),
'number': lambda v: float(v),
'float': lambda v: float(v),
}
def _coerce_bool(v):
if isinstance(v, bool):
return v
if isinstance(v, str):
if v.lower() == 'true':
return True
if v.lower() == 'false':
return False
raise ValueError(f'Cannot convert string {v!r} to bool')
return bool(v)
def _coerce_value(value, expected_type: str):
"""Convert a single value to the expected type.
Returns the converted value, or the original value if no conversion needed.
"""
if value is None:
return value
if expected_type == 'boolean':
if isinstance(value, bool):
return value
return _coerce_bool(value)
coerce_fn = _COERCE_MAP.get(expected_type)
if coerce_fn is None:
return value
# Already the correct type
if expected_type == 'integer' and isinstance(value, int) and not isinstance(value, bool):
return value
if expected_type in ('number', 'float') and isinstance(value, (int, float)) and not isinstance(value, bool):
return float(value)
return coerce_fn(value)
def coerce_pipeline_config(
config: dict,
*metadata_list: dict,
) -> None:
"""Coerce pipeline config values according to metadata type definitions.
Walks each metadata dict (trigger, safety, ai, output) and converts
config values in-place so that strings coming from the JSON column are
cast to their declared types (integer, number/float, boolean).
Args:
config: The pipeline config dict to modify in-place.
*metadata_list: Metadata dicts loaded from the YAML templates.
"""
for meta in metadata_list:
section_name = meta.get('name')
if not section_name or section_name not in config:
continue
section = config[section_name]
if not isinstance(section, dict):
continue
for stage_def in meta.get('stages', []):
stage_name = stage_def.get('name')
if not stage_name or stage_name not in section:
continue
stage_config = section[stage_name]
if not isinstance(stage_config, dict):
continue
for field_def in stage_def.get('config', []):
field_name = field_def.get('name')
field_type = field_def.get('type')
if not field_name or not field_type or field_name not in stage_config:
continue
old_value = stage_config[field_name]
try:
new_value = _coerce_value(old_value, field_type)
if new_value is not old_value:
stage_config[field_name] = new_value
except (ValueError, TypeError) as e:
logger.warning(
'Failed to coerce config %s.%s.%s (%r) to %s: %s',
section_name,
stage_name,
field_name,
old_value,
field_type,
e,
)

View File

@@ -63,6 +63,14 @@ class Controller:
pipeline = await self.ap.pipeline_mgr.get_pipeline_by_uuid(pipeline_uuid)
if pipeline:
await pipeline.run(selected_query)
else:
self.ap.logger.warning(
f'Pipeline {pipeline_uuid} not found for query {selected_query.query_id}, query dropped'
)
else:
self.ap.logger.warning(
f'No pipeline_uuid for query {selected_query.query_id}, query dropped'
)
async with self.ap.query_pool:
(await self.ap.sess_mgr.get_session(selected_query))._semaphore.release()

View File

@@ -34,6 +34,15 @@ class MonitoringHelper:
# Check if session exists, if not, record session start
session_id = f'{query.launcher_type}_{query.launcher_id}'
# Get sender name from message event
sender_name = None
if hasattr(query, 'message_event'):
if hasattr(query.message_event, 'sender'):
if hasattr(query.message_event.sender, 'nickname'):
sender_name = query.message_event.sender.nickname
elif hasattr(query.message_event.sender, 'member_name'):
sender_name = query.message_event.sender.member_name
# Try to record message
# Use JSON serialization to preserve message chain structure (including image URLs, etc.)
if hasattr(query, 'message_chain') and hasattr(query.message_chain, 'model_dump'):
@@ -57,6 +66,7 @@ class MonitoringHelper:
if hasattr(query.launcher_type, 'value')
else str(query.launcher_type),
user_id=query.sender_id,
user_name=sender_name,
runner_name=runner_name,
variables=None, # Will be updated in record_query_success
)
@@ -80,6 +90,7 @@ class MonitoringHelper:
if hasattr(query.launcher_type, 'value')
else str(query.launcher_type),
user_id=query.sender_id,
user_name=sender_name,
)
return message_id
@@ -128,6 +139,15 @@ class MonitoringHelper:
try:
session_id = f'{query.launcher_type}_{query.launcher_id}'
# Get sender name from message event
sender_name = None
if hasattr(query, 'message_event'):
if hasattr(query.message_event, 'sender'):
if hasattr(query.message_event.sender, 'nickname'):
sender_name = query.message_event.sender.nickname
elif hasattr(query.message_event.sender, 'member_name'):
sender_name = query.message_event.sender.member_name
# Extract response content from resp_message_chain
if hasattr(query, 'resp_message_chain') and query.resp_message_chain:
# Serialize the last response message chain
@@ -162,6 +182,7 @@ class MonitoringHelper:
if hasattr(query.launcher_type, 'value')
else str(query.launcher_type),
user_id=query.sender_id,
user_name=sender_name,
runner_name=runner_name,
role='assistant',
)
@@ -183,6 +204,15 @@ class MonitoringHelper:
try:
session_id = f'{query.launcher_type}_{query.launcher_id}'
# Get sender name from message event
sender_name = None
if hasattr(query, 'message_event'):
if hasattr(query.message_event, 'sender'):
if hasattr(query.message_event.sender, 'nickname'):
sender_name = query.message_event.sender.nickname
elif hasattr(query.message_event.sender, 'member_name'):
sender_name = query.message_event.sender.member_name
# Record error message
message_id = await ap.monitoring_service.record_message(
bot_id=bot_id,
@@ -197,6 +227,7 @@ class MonitoringHelper:
if hasattr(query.launcher_type, 'value')
else str(query.launcher_type),
user_id=query.sender_id,
user_name=sender_name,
runner_name=runner_name,
)

View File

@@ -13,6 +13,7 @@ import langbot_plugin.api.entities.builtin.platform.message as platform_message
import langbot_plugin.api.entities.builtin.platform.events as platform_events
import langbot_plugin.api.entities.events as events
from ..utils import importutil
from .config_coercion import coerce_pipeline_config
import langbot_plugin.api.entities.builtin.provider.session as provider_session
import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
@@ -246,7 +247,9 @@ class RuntimePipeline:
await self._check_output(query, result)
if result.result_type == pipeline_entities.ResultType.INTERRUPT:
self.ap.logger.debug(f'Stage {stage_container.inst_name} interrupted query {query.query_id}')
self.ap.logger.debug(
f'Stage {stage_container.inst_name} interrupted query {query.query_id}'
)
break
elif result.result_type == pipeline_entities.ResultType.CONTINUE:
query = result.new_query
@@ -260,7 +263,9 @@ class RuntimePipeline:
await self._check_output(query, sub_result)
if sub_result.result_type == pipeline_entities.ResultType.INTERRUPT:
self.ap.logger.debug(f'Stage {stage_container.inst_name} interrupted query {query.query_id}')
self.ap.logger.debug(
f'Stage {stage_container.inst_name} interrupted query {query.query_id}'
)
break
elif sub_result.result_type == pipeline_entities.ResultType.CONTINUE:
query = sub_result.new_query
@@ -322,6 +327,9 @@ class RuntimePipeline:
event_ctx = await self.ap.plugin_connector.emit_event(event_obj, bound_plugins)
if event_ctx.is_prevented_default():
self.ap.logger.debug(
f'MessageReceived event prevented default for query {query.query_id}, pipeline={pipeline_name}'
)
return
self.ap.logger.debug(f'Processing query {query.query_id}')
@@ -420,6 +428,14 @@ class PipelineManager:
elif isinstance(pipeline_entity, dict):
pipeline_entity = persistence_pipeline.LegacyPipeline(**pipeline_entity)
coerce_pipeline_config(
pipeline_entity.config,
getattr(self.ap, 'pipeline_config_meta_trigger', {'name': 'trigger', 'stages': []}),
getattr(self.ap, 'pipeline_config_meta_safety', {'name': 'safety', 'stages': []}),
getattr(self.ap, 'pipeline_config_meta_ai', {'name': 'ai', 'stages': []}),
getattr(self.ap, 'pipeline_config_meta_output', {'name': 'output', 'stages': []}),
)
# initialize stage containers according to pipeline_entity.stages
stage_containers: list[StageInstContainer] = []
for stage_name in pipeline_entity.stages:

View File

@@ -41,6 +41,7 @@ class QueryPool:
message_chain: platform_message.MessageChain,
adapter: abstract_platform_adapter.AbstractMessagePlatformAdapter,
pipeline_uuid: typing.Optional[str] = None,
routed_by_rule: bool = False,
) -> pipeline_query.Query:
async with self.condition:
query_id = self.query_id_counter
@@ -52,7 +53,7 @@ class QueryPool:
sender_id=sender_id,
message_event=message_event,
message_chain=message_chain,
variables={},
variables={'_routed_by_rule': routed_by_rule},
resp_messages=[],
resp_message_chain=[],
adapter=adapter,

View File

@@ -36,17 +36,36 @@ class PreProcessor(stage.PipelineStage):
session = await self.ap.sess_mgr.get_session(query)
# When not local-agent, llm_model is None
try:
llm_model = (
await self.ap.model_mgr.get_model_by_uuid(query.pipeline_config['ai']['local-agent']['model'])
if selected_runner == 'local-agent'
else None
)
except ValueError:
self.ap.logger.warning(
f'LLM model {query.pipeline_config["ai"]["local-agent"]["model"] + " "}not found or not configured'
)
llm_model = None
llm_model = None
if selected_runner == 'local-agent':
# Read model config — new format is { primary: str, fallbacks: [str] },
# but handle legacy plain string for backward compatibility
model_config = query.pipeline_config['ai']['local-agent'].get('model', {})
if isinstance(model_config, str):
# Legacy format: plain UUID string
primary_uuid = model_config
fallback_uuids = []
else:
primary_uuid = model_config.get('primary', '')
fallback_uuids = model_config.get('fallbacks', [])
if primary_uuid:
try:
llm_model = await self.ap.model_mgr.get_model_by_uuid(primary_uuid)
except ValueError:
self.ap.logger.warning(f'LLM model {primary_uuid} not found or not configured')
# Resolve fallback model UUIDs
if fallback_uuids:
valid_fallbacks = []
for fb_uuid in fallback_uuids:
try:
await self.ap.model_mgr.get_model_by_uuid(fb_uuid)
valid_fallbacks.append(fb_uuid)
except ValueError:
self.ap.logger.warning(f'Fallback model {fb_uuid} not found, skipping')
if valid_fallbacks:
query.variables['_fallback_model_uuids'] = valid_fallbacks
conversation = await self.ap.sess_mgr.get_conversation(
query,
@@ -61,20 +80,28 @@ class PreProcessor(stage.PipelineStage):
query.prompt = conversation.prompt.copy()
query.messages = conversation.messages.copy()
if selected_runner == 'local-agent' and llm_model:
if selected_runner == 'local-agent':
query.use_funcs = []
query.use_llm_model_uuid = llm_model.model_entity.uuid
if llm_model:
query.use_llm_model_uuid = llm_model.model_entity.uuid
if llm_model.model_entity.abilities.__contains__('func_call'):
# Get bound plugins and MCP servers for filtering tools
if llm_model.model_entity.abilities.__contains__('func_call'):
# Get bound plugins and MCP servers for filtering tools
bound_plugins = query.variables.get('_pipeline_bound_plugins', None)
bound_mcp_servers = query.variables.get('_pipeline_bound_mcp_servers', None)
query.use_funcs = await self.ap.tool_mgr.get_all_tools(bound_plugins, bound_mcp_servers)
self.ap.logger.debug(f'Bound plugins: {bound_plugins}')
self.ap.logger.debug(f'Bound MCP servers: {bound_mcp_servers}')
self.ap.logger.debug(f'Use funcs: {query.use_funcs}')
# If primary model doesn't support func_call but fallback models exist,
# load tools anyway since fallback models may support them
if not query.use_funcs and query.variables.get('_fallback_model_uuids'):
bound_plugins = query.variables.get('_pipeline_bound_plugins', None)
bound_mcp_servers = query.variables.get('_pipeline_bound_mcp_servers', None)
query.use_funcs = await self.ap.tool_mgr.get_all_tools(bound_plugins, bound_mcp_servers)
self.ap.logger.debug(f'Bound plugins: {bound_plugins}')
self.ap.logger.debug(f'Bound MCP servers: {bound_mcp_servers}')
self.ap.logger.debug(f'Use funcs: {query.use_funcs}')
sender_name = ''
if isinstance(query.message_event, platform_events.GroupMessage):
@@ -149,6 +176,16 @@ class PreProcessor(stage.PipelineStage):
query.variables['user_message_text'] = plain_text
query.user_message = provider_message.Message(role='user', content=content_list)
# Extract knowledge base UUIDs into query variables so plugins can modify them
# during PromptPreProcessing before the runner performs retrieval.
kb_uuids = query.pipeline_config['ai']['local-agent'].get('knowledge-bases', [])
if not kb_uuids:
old_kb_uuid = query.pipeline_config['ai']['local-agent'].get('knowledge-base', '')
if old_kb_uuid and old_kb_uuid != '__none__':
kb_uuids = [old_kb_uuid]
query.variables['_knowledge_base_uuids'] = list(kb_uuids)
# =========== 触发事件 PromptPreProcessing
event = events.PromptPreProcessing(

View File

@@ -12,7 +12,7 @@ from ... import entities
from ....provider import runner as runner_module
import langbot_plugin.api.entities.events as events
from ....utils import importutil, constants
from ....utils import importutil, constants, runner as runner_utils
from ....provider import runners
import langbot_plugin.api.entities.builtin.provider.session as provider_session
import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
@@ -61,6 +61,9 @@ class ChatMessageHandler(handler.MessageHandler):
yield entities.StageProcessResult(result_type=entities.ResultType.CONTINUE, new_query=query)
else:
self.ap.logger.debug(
f'NormalMessageReceived event prevented default for query {query.query_id} without reply'
)
yield entities.StageProcessResult(result_type=entities.ResultType.INTERRUPT, new_query=query)
else:
if event_ctx.event.user_message_alter is not None:
@@ -149,12 +152,19 @@ class ChatMessageHandler(handler.MessageHandler):
self.ap.logger.error(f'Conversation({query.query_id}) Request Failed: {error_info}')
traceback.print_exc()
hide_exception_info = query.pipeline_config['output']['misc']['hide-exception']
exception_handling = query.pipeline_config['output']['misc'].get('exception-handling', 'show-hint')
if exception_handling == 'show-error':
user_notice = f'{e}'
elif exception_handling == 'show-hint':
user_notice = query.pipeline_config['output']['misc'].get('failure-hint', 'Request failed.')
else: # hide
user_notice = None
yield entities.StageProcessResult(
result_type=entities.ResultType.INTERRUPT,
new_query=query,
user_notice='请求失败' if hide_exception_info else f'{e}',
user_notice=user_notice,
error_notice=f'{e}',
debug_notice=traceback.format_exc(),
)
@@ -185,10 +195,15 @@ class ChatMessageHandler(handler.MessageHandler):
pipeline_plugins = query.variables.get('_pipeline_bound_plugins', None)
runner_category = runner_utils.get_runner_category_from_runner(
runner_name, runner, query.pipeline_config
)
payload = {
'query_id': query.query_id,
'adapter': adapter_name,
'runner': runner_name,
'runner_category': runner_category,
'duration_ms': duration_ms,
'model_name': model_name,
'version': constants.semantic_version,

View File

@@ -37,6 +37,10 @@ class GroupRespondRuleCheckStage(stage.PipelineStage):
if query.launcher_type.value != 'group': # 只处理群消息
return entities.StageProcessResult(result_type=entities.ResultType.CONTINUE, new_query=query)
# 通过路由规则明确指定的流水线,跳过群响应规则检查
if query.variables and query.variables.get('_routed_by_rule', False):
return entities.StageProcessResult(result_type=entities.ResultType.CONTINUE, new_query=query)
rules = query.pipeline_config['trigger']['group-respond-rules']
use_rule = rules

View File

@@ -1,6 +1,7 @@
from __future__ import annotations
import asyncio
import re
import traceback
import sqlalchemy
@@ -9,6 +10,7 @@ from ..core import app, entities as core_entities, taskmgr
from ..discover import engine
from ..entity.persistence import bot as persistence_bot
from ..entity.persistence import pipeline as persistence_pipeline
from ..entity.errors import platform as platform_errors
@@ -51,6 +53,69 @@ class RuntimeBot:
self.task_context = taskmgr.TaskContext()
self.logger = logger
@staticmethod
def _match_operator(actual: str, operator: str, expected: str) -> bool:
"""Evaluate a single operator condition."""
if operator == 'eq':
return actual == expected
elif operator == 'neq':
return actual != expected
elif operator == 'contains':
return expected in actual
elif operator == 'not_contains':
return expected not in actual
elif operator == 'starts_with':
return actual.startswith(expected)
elif operator == 'regex':
try:
return bool(re.search(expected, actual))
except re.error:
return False
return False
def resolve_pipeline_uuid(
self,
launcher_type: str,
launcher_id: str,
message_text: str,
) -> tuple[str | None, bool]:
"""Resolve pipeline UUID based on routing rules.
Rules are evaluated in order; first match wins.
Falls back to use_pipeline_uuid if no rule matches.
Rule types:
- launcher_type: session type ("person" / "group")
- launcher_id: session / group id
- message_content: message text content
Operators: eq, neq, contains, not_contains, starts_with, regex
Returns:
tuple: (pipeline_uuid, routed_by_rule) - routed_by_rule is True
when a routing rule matched, False when falling back to default.
"""
rules = self.bot_entity.pipeline_routing_rules or []
for rule in rules:
rule_type = rule.get('type')
operator = rule.get('operator', 'eq')
rule_value = rule.get('value', '')
target_uuid = rule.get('pipeline_uuid')
if not rule_type or not target_uuid:
continue
if rule_type == 'launcher_type':
if self._match_operator(launcher_type, operator, rule_value):
return target_uuid, True
elif rule_type == 'launcher_id':
if self._match_operator(str(launcher_id), operator, str(rule_value)):
return target_uuid, True
elif rule_type == 'message_content':
if self._match_operator(message_text, operator, rule_value):
return target_uuid, True
return self.bot_entity.use_pipeline_uuid, False
async def initialize(self):
async def on_friend_message(
event: platform_events.FriendMessage,
@@ -82,6 +147,9 @@ class RuntimeBot:
if custom_launcher_id:
launcher_id = custom_launcher_id
message_text = str(event.message_chain)
pipeline_uuid, routed_by_rule = self.resolve_pipeline_uuid('person', launcher_id, message_text)
await self.ap.msg_aggregator.add_message(
bot_uuid=self.bot_entity.uuid,
launcher_type=provider_session.LauncherTypes.PERSON,
@@ -90,7 +158,8 @@ class RuntimeBot:
message_event=event,
message_chain=event.message_chain,
adapter=adapter,
pipeline_uuid=self.bot_entity.use_pipeline_uuid,
pipeline_uuid=pipeline_uuid,
routed_by_rule=routed_by_rule,
)
else:
await self.logger.info('Pipeline skipped for person message due to webhook response')
@@ -125,6 +194,9 @@ class RuntimeBot:
if custom_launcher_id:
launcher_id = custom_launcher_id
message_text = str(event.message_chain)
pipeline_uuid, routed_by_rule = self.resolve_pipeline_uuid('group', launcher_id, message_text)
await self.ap.msg_aggregator.add_message(
bot_uuid=self.bot_entity.uuid,
launcher_type=provider_session.LauncherTypes.GROUP,
@@ -133,7 +205,8 @@ class RuntimeBot:
message_event=event,
message_chain=event.message_chain,
adapter=adapter,
pipeline_uuid=self.bot_entity.use_pipeline_uuid,
pipeline_uuid=pipeline_uuid,
routed_by_rule=routed_by_rule,
)
else:
await self.logger.info('Pipeline skipped for group message due to webhook response')
@@ -141,6 +214,50 @@ class RuntimeBot:
self.adapter.register_listener(platform_events.FriendMessage, on_friend_message)
self.adapter.register_listener(platform_events.GroupMessage, on_group_message)
# Register feedback listener (only effective on adapters that support it)
async def on_feedback(
event: platform_events.FeedbackEvent,
adapter: abstract_platform_adapter.AbstractMessagePlatformAdapter,
):
try:
# Resolve pipeline name
pipeline_name = ''
if self.bot_entity.use_pipeline_uuid:
try:
pipeline_result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_pipeline.LegacyPipeline.name).where(
persistence_pipeline.LegacyPipeline.uuid == self.bot_entity.use_pipeline_uuid
)
)
pipeline_row = pipeline_result.first()
if pipeline_row:
pipeline_name = pipeline_row[0]
except Exception:
pass
await self.ap.monitoring_service.record_feedback(
feedback_id=event.feedback_id,
feedback_type=event.feedback_type,
feedback_content=event.feedback_content,
inaccurate_reasons=event.inaccurate_reasons,
bot_id=self.bot_entity.uuid,
bot_name=self.bot_entity.name,
pipeline_id=self.bot_entity.use_pipeline_uuid or '',
pipeline_name=pipeline_name,
session_id=event.session_id,
message_id=event.message_id,
stream_id=event.stream_id,
user_id=event.user_id,
platform=adapter.__class__.__name__,
)
await self.logger.info(
f'Recorded feedback: feedback_id={event.feedback_id}, type={event.feedback_type}'
)
except Exception:
await self.logger.error(f'Failed to record feedback: {traceback.format_exc()}')
self.adapter.register_listener(platform_events.FeedbackEvent, on_feedback)
async def run(self):
async def exception_wrapper():
try:
@@ -282,6 +399,8 @@ class PlatformManager:
return runtime_bot
async def get_bot_by_uuid(self, bot_uuid: str) -> RuntimeBot | None:
if self.websocket_proxy_bot and self.websocket_proxy_bot.bot_entity.uuid == bot_uuid:
return self.websocket_proxy_bot
for bot in self.bots:
if bot.bot_entity.uuid == bot_uuid:
return bot

View File

@@ -5,19 +5,29 @@ metadata:
label:
en_US: OneBot v11
zh_Hans: OneBot v11
zh_Hant: OneBot v11
description:
en_US: OneBot v11 Adapter
zh_Hans: OneBot v11 适配器,请查看文档了解使用方式
en_US: OneBot v11 Adapter, used for QQ bots
zh_Hans: OneBot v11 适配器,用于接入 QQ 机器人协议端,请查看文档了解使用方式
zh_Hant: OneBot v11 適配器,用於接入 QQ 機器人協定端,請查看文件了解使用方式
icon: onebot.png
spec:
categories:
- protocol
help_links:
zh: https://link.langbot.app/zh/platforms/aiocqhttp
en: https://link.langbot.app/en/platforms/aiocqhttp
ja: https://link.langbot.app/ja/platforms/aiocqhttp
config:
- name: host
label:
en_US: Host
zh_Hans: 主机
zh_Hant: 主機
description:
en_US: The host that OneBot v11 listens on for reverse WebSocket connections. Unless you know what you're doing, use 0.0.0.0
zh_Hans: OneBot v11 监听的反向 WS 主机,除非你知道自己在做什么,否则请写 0.0.0.0
zh_Hant: OneBot v11 監聽的反向 WS 主機,除非你知道自己在做什麼,否則請填 0.0.0.0
type: string
required: true
default: 0.0.0.0
@@ -25,9 +35,11 @@ spec:
label:
en_US: Port
zh_Hans: 端口
zh_Hant: 連接埠
description:
en_US: Port
zh_Hans: 监听的端口
zh_Hant: 監聽的連接埠
type: integer
required: true
default: 2280
@@ -35,9 +47,11 @@ spec:
label:
en_US: Access Token
zh_Hans: 访问令牌
zh_Hant: 存取令牌
description:
en_US: Custom connection token for the protocol endpoint. If the protocol endpoint is not set, don't fill it
zh_Hans: 自定义的与协议端的连接令牌,若协议端未设置,则不填
zh_Hant: 自訂的與協定端的連線令牌,若協定端未設定,則不填
type: string
required: false
default: ""

View File

@@ -5,16 +5,25 @@ metadata:
label:
en_US: DingTalk
zh_Hans: 钉钉
zh_Hant: 釘釘
description:
en_US: DingTalk Adapter
zh_Hans: 钉钉适配器,请查看文档了解使用方式
zh_Hant: 釘釘適配器,請查看文件了解使用方式
icon: dingtalk.svg
spec:
categories:
- china
help_links:
zh: https://link.langbot.app/zh/platforms/dingtalk
en: https://link.langbot.app/en/platforms/dingtalk
ja: https://link.langbot.app/ja/platforms/dingtalk
config:
- name: client_id
label:
en_US: Client ID
zh_Hans: 客户端ID
zh_Hant: 用戶端ID
type: string
required: true
default: ""
@@ -22,6 +31,7 @@ spec:
label:
en_US: Client Secret
zh_Hans: 客户端密钥
zh_Hant: 用戶端密鑰
type: string
required: true
default: ""
@@ -29,6 +39,7 @@ spec:
label:
en_US: Robot Code
zh_Hans: 机器人代码
zh_Hant: 機器人代碼
type: string
required: true
default: ""
@@ -36,6 +47,7 @@ spec:
label:
en_US: Robot Name
zh_Hans: 机器人名称
zh_Hant: 機器人名稱
type: string
required: true
default: ""
@@ -43,6 +55,7 @@ spec:
label:
en_US: Markdown Card
zh_Hans: 是否使用 Markdown 卡片
zh_Hant: 是否使用 Markdown 卡片
type: boolean
required: false
default: true
@@ -50,9 +63,11 @@ spec:
label:
en_US: Enable Stream Reply Mode
zh_Hans: 启用钉钉卡片流式回复模式
zh_Hant: 啟用釘釘卡片串流回覆模式
description:
en_US: If enabled, the bot will use the stream of lark reply mode
zh_Hans: 如果启用,将使用钉钉卡片流式方式来回复内容
zh_Hant: 如果啟用,將使用釘釘卡片串流方式來回覆內容
type: boolean
required: true
default: false
@@ -60,6 +75,7 @@ spec:
label:
en_US: Card Auto Layout
zh_Hans: 卡片宽屏自动布局
zh_Hant: 卡片寬螢幕自動佈局
type: boolean
required: false
default: false
@@ -67,6 +83,7 @@ spec:
label:
en_US: card template id
zh_Hans: 卡片模板ID
zh_Hant: 卡片範本ID
type: string
required: true
default: "填写你的卡片template_id"

View File

@@ -5,16 +5,38 @@ metadata:
label:
en_US: Discord
zh_Hans: Discord
zh_Hant: Discord
ja_JP: Discord
th_TH: Discord
vi_VN: Discord
es_ES: Discord
description:
en_US: Discord Adapter
zh_Hans: Discord 适配器,请查看文档了解使用方式
zh_Hans: Discord 适配器,需要可连接 Discord 服务器的网络环境
zh_Hant: Discord 適配器,需要可連線 Discord 伺服器的網路環境
ja_JP: Discord アダプター、Discord サーバーに接続可能なネットワーク環境が必要です
th_TH: อะแดปเตอร์ Discord ต้องการสภาพแวดล้อมเครือข่ายที่สามารถเชื่อมต่อกับเซิร์ฟเวอร์ Discord ได้
vi_VN: Bộ điều hợp Discord, cần môi trường mạng có thể kết nối với máy chủ Discord
es_ES: Adaptador de Discord, requiere un entorno de red con acceso al servidor de Discord
icon: discord.svg
spec:
categories:
- popular
- global
help_links:
zh: https://link.langbot.app/zh/platforms/discord
en: https://link.langbot.app/en/platforms/discord
ja: https://link.langbot.app/ja/platforms/discord
config:
- name: client_id
label:
en_US: Client ID
zh_Hans: 客户端ID
zh_Hant: 用戶端ID
ja_JP: クライアント ID
th_TH: รหัสไคลเอนต์
vi_VN: ID khách hàng
es_ES: ID de cliente
type: string
required: true
default: ""
@@ -22,6 +44,11 @@ spec:
label:
en_US: Token
zh_Hans: 令牌
zh_Hant: 令牌
ja_JP: トークン
th_TH: โทเค็น
vi_VN: Mã thông báo
es_ES: Token
type: string
required: true
default: ""

View File

@@ -5,16 +5,25 @@ metadata:
label:
en_US: KOOK
zh_Hans: KOOK
zh_Hant: KOOK
description:
en_US: KOOK Adapter (formerly KaiHeiLa)
zh_Hans: KOOK 适配器(原开黑啦),支持频道消息和私聊消息
zh_Hant: KOOK 適配器(原開黑啦),支援頻道訊息和私聊訊息
icon: kook.png
spec:
categories:
- china
help_links:
zh: https://link.langbot.app/zh/platforms/kook
en: https://link.langbot.app/en/platforms/kook
ja: https://link.langbot.app/ja/platforms/kook
config:
- name: token
label:
en_US: Bot Token
zh_Hans: 机器人令牌
zh_Hant: 機器人令牌
type: string
required: true
default: ""

View File

@@ -575,6 +575,127 @@ class LarkMessageConverter(abstract_platform_adapter.AbstractMessageConverter):
class LarkEventConverter(abstract_platform_adapter.AbstractEventConverter):
_processed_thread_quote_cache: typing.ClassVar[dict[str, float]] = {}
_processed_thread_quote_cache_max_size: typing.ClassVar[int] = 4096
_processed_thread_quote_cache_ttl_seconds: typing.ClassVar[int] = 86400
@classmethod
def _prune_processed_thread_quote_cache(cls, now: typing.Optional[float] = None) -> None:
if now is None:
now = time.time()
expire_before = now - cls._processed_thread_quote_cache_ttl_seconds
while cls._processed_thread_quote_cache:
oldest_key, oldest_ts = next(iter(cls._processed_thread_quote_cache.items()))
if oldest_ts >= expire_before:
break
cls._processed_thread_quote_cache.pop(oldest_key, None)
while len(cls._processed_thread_quote_cache) > cls._processed_thread_quote_cache_max_size:
oldest_key = next(iter(cls._processed_thread_quote_cache))
cls._processed_thread_quote_cache.pop(oldest_key, None)
@classmethod
def _mark_thread_quote_processed(cls, thread_id: str) -> None:
now = time.time()
cls._prune_processed_thread_quote_cache(now)
cls._processed_thread_quote_cache[thread_id] = now
@classmethod
def _extract_quote_message_id(cls, message: EventMessage) -> typing.Optional[str]:
"""
Extract the message ID to quote from the given message.
Rules:
- First thread reply in a topic: return parent_id and mark topic as processed
- Follow-up thread replies in the same topic: return None
- Non-thread message: return parent_id if valid (non-empty, different from message_id)
Thread reply state is kept in a bounded TTL cache to avoid unbounded memory growth.
"""
parent_id = getattr(message, 'parent_id', None)
if not parent_id:
return None
message_id = getattr(message, 'message_id', None)
if parent_id == message_id:
return None
thread_id = getattr(message, 'thread_id', None)
if thread_id:
cls._prune_processed_thread_quote_cache()
if thread_id in cls._processed_thread_quote_cache:
return None
cls._mark_thread_quote_processed(thread_id)
return parent_id
@staticmethod
def _build_event_message_from_message_item(message_item: Message) -> typing.Optional[EventMessage]:
"""
Build EventMessage from SDK typed Message item.
Returns None if body or content is missing.
"""
body = getattr(message_item, 'body', None)
if not body:
return None
content = getattr(body, 'content', None)
if not content:
return None
event_data = {
'message_id': message_item.message_id,
'message_type': message_item.msg_type,
'content': content,
'create_time': message_item.create_time,
'mentions': getattr(message_item, 'mentions', []) or [],
}
# Preserve thread-related fields
if hasattr(message_item, 'parent_id') and message_item.parent_id:
event_data['parent_id'] = message_item.parent_id
if hasattr(message_item, 'root_id') and message_item.root_id:
event_data['root_id'] = message_item.root_id
if hasattr(message_item, 'thread_id') and message_item.thread_id:
event_data['thread_id'] = message_item.thread_id
if hasattr(message_item, 'chat_id') and message_item.chat_id:
event_data['chat_id'] = message_item.chat_id
return EventMessage(event_data)
@staticmethod
async def _fetch_quoted_message(
quote_message_id: str,
api_client: lark_oapi.Client,
) -> typing.Optional[platform_message.MessageChain]:
"""
Fetch the quoted message and convert to MessageChain.
Returns None if:
- API call fails
- Response items is empty
- Message item normalization fails
"""
request = GetMessageRequest.builder().message_id(quote_message_id).build()
response = await api_client.im.v1.message.aget(request)
if not response.success():
return None
items = getattr(response.data, 'items', None)
if not items:
return None
message_item = items[0]
event_message = LarkEventConverter._build_event_message_from_message_item(message_item)
if event_message is None:
return None
quote_chain = await LarkMessageConverter.target2yiri(event_message, api_client)
return quote_chain
@staticmethod
async def yiri2target(
event: platform_events.MessageEvent,
@@ -587,6 +708,23 @@ class LarkEventConverter(abstract_platform_adapter.AbstractEventConverter):
) -> platform_events.Event:
message_chain = await LarkMessageConverter.target2yiri(event.event.message, api_client)
# Check for quote/reply message
quote_message_id = LarkEventConverter._extract_quote_message_id(event.event.message)
if quote_message_id:
quote_chain = await LarkEventConverter._fetch_quoted_message(quote_message_id, api_client)
if quote_chain:
# Filter out Source component from quoted chain, keep only content
quote_origin = platform_message.MessageChain(
[comp for comp in quote_chain if not isinstance(comp, platform_message.Source)]
)
if quote_origin:
message_chain.append(
platform_message.Quote(
message_id=quote_message_id,
origin=quote_origin,
)
)
if event.event.message.chat_type == 'p2p':
return platform_events.FriendMessage(
sender=platform_entities.Friend(
@@ -770,6 +908,32 @@ class LarkAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
self.request_tenant_access_token(tenant_key)
return self.tenant_access_tokens.get(tenant_key)['token'] if self.tenant_access_tokens.get(tenant_key) else None
def get_launcher_id(self, event: platform_events.MessageEvent) -> str | None:
"""
Get topic-scoped launcher_id for thread-aware session isolation.
For group thread messages, returns "{group_id}_{thread_id}"
to ensure conversation context stays stable per topic.
Returns None for non-thread messages or P2P messages.
"""
source_event = getattr(event.source_platform_object, 'event', None)
if not source_event:
return None
message = getattr(source_event, 'message', None)
if not message:
return None
thread_id = getattr(message, 'thread_id', None)
if not thread_id:
return None
if isinstance(event, platform_events.GroupMessage):
return f'{event.group.id}_{thread_id}'
return None
def build_api_client(self, config):
app_id = config['app_id']
app_secret = config['app_secret']

View File

@@ -5,16 +5,30 @@ metadata:
label:
en_US: Lark
zh_Hans: 飞书
zh_Hant: 飛書
ja_JP: Lark
description:
en_US: Lark Adapter
zh_Hans: 飞书适配器,请查看文档了解使用方式
en_US: Lark Adapter, supports both long connection and Webhook modes. Please refer to the documentation for usage details.
zh_Hans: 飞书适配器,支持长连接和 Webhook 两种接入方式,请查看文档了解使用方式
zh_Hant: 飛書適配器,支援長連線和 Webhook 兩種接入方式,請查看文件了解使用方式
ja_JP: Lark アダプター、長期接続およびWebhookモードの両方をサポートしています。使用方法の詳細については、ドキュメントを参照してください。
icon: lark.svg
spec:
categories:
- popular
- china
- global
help_links:
zh: https://link.langbot.app/zh/platforms/lark
en: https://link.langbot.app/en/platforms/lark
ja: https://link.langbot.app/ja/platforms/lark
config:
- name: app_id
label:
en_US: App ID
zh_Hans: 应用ID
zh_Hant: 應用ID
ja_JP: アプリ ID
type: string
required: true
default: ""
@@ -22,6 +36,8 @@ spec:
label:
en_US: App Secret
zh_Hans: 应用密钥
zh_Hant: 應用密鑰
ja_JP: アプリシークレット
type: string
required: true
default: ""
@@ -29,9 +45,13 @@ spec:
label:
en_US: Bot Name
zh_Hans: 机器人名称
zh_Hant: 機器人名稱
ja_JP: ボット名
description:
en_US: Must be the same as the name of the bot in Lark, otherwise the bot will not be able to receive messages in the group
zh_Hans: 必须与飞书机器人名称一致,否则机器人将无法在群内正常接收消息
zh_Hant: 必須與飛書機器人名稱一致,否則機器人將無法在群組內正常接收訊息
ja_JP: Lark のボット名と一致する必要があります。一致しない場合、グループ内でメッセージを受信できません
type: string
required: true
default: ""
@@ -39,29 +59,63 @@ spec:
label:
en_US: Enable Webhook Mode
zh_Hans: 启用Webhook模式
zh_Hant: 啟用 Webhook 模式
ja_JP: Webhook モードを有効化
description:
en_US: If enabled, the bot will use webhook mode to receive messages. Otherwise, it will use WS long connection mode
zh_Hans: 如果启用,机器人将使用 Webhook 模式接收消息。否则,将使用 WS 长连接模式
zh_Hant: 如果啟用,機器人將使用 Webhook 模式接收訊息。否則,將使用 WS 長連線模式
ja_JP: 有効にすると、ボットは Webhook モードでメッセージを受信します。無効の場合は WS 長期接続モードを使用します
type: boolean
required: true
default: false
- name: webhook_url
label:
en_US: Webhook Callback URL
zh_Hans: Webhook 回调地址
zh_Hant: Webhook 回調地址
ja_JP: Webhook コールバック URL
description:
en_US: Copy this URL and paste it into your Lark app's webhook configuration
zh_Hans: 复制此地址并粘贴到飞书应用的 Webhook 配置中
zh_Hant: 複製此地址並貼到飛書應用的 Webhook 設定中
ja_JP: この URL をコピーして Lark アプリの Webhook 設定に貼り付けてください
type: webhook-url
required: false
default: ""
show_if:
field: enable-webhook
operator: eq
value: true
- name: encrypt-key
label:
en_US: Encrypt Key
zh_Hans: 加密密钥
zh_Hant: 加密密鑰
ja_JP: 暗号化キー
description:
en_US: Only valid when webhook mode is enabled, please fill in the encrypt key
zh_Hans: 仅在启用 Webhook 模式时有效,请填写加密密钥
zh_Hant: 僅在啟用 Webhook 模式時有效,請填寫加密密鑰
ja_JP: Webhook モードが有効な場合にのみ有効です。暗号化キーを入力してください
type: string
required: true
default: ""
show_if:
field: enable-webhook
operator: eq
value: true
- name: enable-stream-reply
label:
en_US: Enable Stream Reply Mode
zh_Hans: 启用飞书流式回复模式
zh_Hant: 啟用飛書串流回覆模式
ja_JP: ストリーミング返信モードを有効化
description:
en_US: If enabled, the bot will use the stream of lark reply mode
zh_Hans: 如果启用,将使用飞书流式方式来回复内容
zh_Hant: 如果啟用,將使用飛書串流方式來回覆內容
ja_JP: 有効にすると、ボットはストリーミングモードでメッセージに返信します
type: boolean
required: true
default: false
@@ -69,28 +123,40 @@ spec:
label:
en_US: App Type
zh_Hans: 应用类型
zh_Hant: 應用類型
ja_JP: アプリタイプ
description:
en_US: Default to self-built application, refer to https://open.feishu.cn/document/platform-overveiw/overview
zh_Hans: 默认为企业自建应用,参考 https://open.feishu.cn/document/platform-overveiw/overview
zh_Hant: 預設為企業自建應用,參考 https://open.feishu.cn/document/platform-overveiw/overview
ja_JP: デフォルトはカスタムアプリです。詳細は https://open.feishu.cn/document/platform-overveiw/overview を参照してください
type: select
options:
- name: self
label:
en_US: Self-built Application
zh_Hans: 自建应用
zh_Hant: 自建應用
ja_JP: カスタムアプリ
- name: isv
label:
en_US: Store Application
zh_Hans: 商店应用
zh_Hant: 商店應用
ja_JP: ストアアプリ
required: false
default: self
- name: bot_added_welcome
label:
en_US: Bot Welcome Message
zh_Hans: 机器人进群欢迎语
zh_Hant: 機器人進群歡迎語
ja_JP: ボット参加時のウェルカムメッセージ
description:
en_US: Welcome message when the bot is added to a group, supports Markdown format
zh_Hans: 机器人进群欢迎语,支持 Markdown 格式
zh_Hant: 機器人進群歡迎語,支援 Markdown 格式
ja_JP: ボットがグループに追加された際のウェルカムメッセージ。Markdown 形式に対応しています
type: text
required: false
default: ""

View File

@@ -5,20 +5,56 @@ metadata:
label:
en_US: LINE
zh_Hans: LINE
zh_Hant: LINE
th_TH: LINE
vi_VN: LINE
es_ES: LINE
description:
en_US: LINE Adapter
zh_Hans: LINE适配器请查看文档了解使用方式
ja_JP: LINEアダプター、ドキュメントを参照してください
zh_Hant: LINE適配器,請查看文檔了解使用方式
en_US: LINE Adapter, requires a public URL to receive LINE message pushes, please refer to the documentation for usage details
zh_Hans: LINE适配器需要公网地址以接收 LINE 消息推送,请查看文档了解使用方式
zh_Hant: LINE 適配器,需要公網地址以接收 LINE 訊息推送,請查看文件了解使用方式
ja_JP: LINEアダプター、LINEのメッセージプッシュを受信するためにパブリックURLが必要です。使用方法の詳細については、ドキュメントを参照してください。
th_TH: อะแดปเตอร์ LINE ต้องการ URL สาธารณะเพื่อรับการแจ้งเตือนข้อความจาก LINE โปรดดูเอกสารประกอบสำหรับรายละเอียดการใช้งาน
vi_VN: Bộ điều hợp LINE, cần URL công cộng để nhận thông báo tin nhắn LINE, vui lòng xem tài liệu để biết chi tiết cách sử dụng
es_ES: Adaptador de LINE, requiere una URL pública para recibir notificaciones de mensajes de LINE, consulte la documentación para obtener detalles de uso
icon: line.png
spec:
categories:
- global
help_links:
zh: https://link.langbot.app/zh/platforms/line
en: https://link.langbot.app/en/platforms/line
ja: https://link.langbot.app/ja/platforms/line
config:
- name: webhook_url
label:
en_US: Webhook Callback URL
zh_Hans: Webhook 回调地址
ja_JP: Webhook コールバック URL
zh_Hant: Webhook 回調地址
th_TH: URL การเรียกกลับ Webhook
vi_VN: URL gọi lại Webhook
es_ES: URL de devolución de llamada Webhook
description:
en_US: Copy this URL and paste it into your LINE channel's webhook configuration
zh_Hans: 复制此地址并粘贴到 LINE 频道的 Webhook 配置中
ja_JP: この URL をコピーして LINE チャンネルの Webhook 設定に貼り付けてください
zh_Hant: 複製此地址並貼到 LINE 頻道的 Webhook 設定中
th_TH: คัดลอก URL นี้แล้ววางในการตั้งค่า Webhook ของช่อง LINE ของคุณ
vi_VN: Sao chép URL này và dán vào cấu hình webhook của kênh LINE của bạn
es_ES: Copie esta URL y péguela en la configuración de webhook de su canal LINE
type: webhook-url
required: false
default: ""
- name: channel_access_token
label:
en_US: Channel access token
zh_Hans: 频道访问令牌
ja_JP: チャンネルアクセストークン
zh_Hant: 頻道訪問令牌
zh_Hant: 頻道存取令牌
th_TH: โทเค็นการเข้าถึงช่อง
vi_VN: Mã truy cập kênh
es_ES: Token de acceso del canal
type: string
required: true
default: ""
@@ -27,12 +63,18 @@ spec:
en_US: Channel secret
zh_Hans: 消息密钥
ja_JP: チャンネルシークレット
zh_Hant: 息密
zh_Hant: 息密
th_TH: รหัสลับช่อง
vi_VN: Khóa bí mật kênh
es_ES: Secreto del canal
description:
en_US: Only valid when webhook mode is enabled, please fill in the encrypt key
zh_Hans: 请填写加密密钥
ja_JP: Webhookモードが有効な場合にのみ、暗号化キーを入力してください
zh_Hant: 請填寫加密密
zh_Hant: 請填寫加密密
th_TH: กรุณากรอกคีย์เข้ารหัส
vi_VN: Vui lòng điền khóa mã hóa
es_ES: Por favor, introduzca la clave de cifrado
type: string
required: true
default: ""

View File

@@ -5,23 +5,44 @@ metadata:
label:
en_US: Official Account
zh_Hans: 微信公众号
zh_Hant: 微信公眾號
description:
en_US: Official Account Adapter
zh_Hans: 微信公众号适配器,请查看文档了解使用方式
zh_Hans: 微信公众号适配器,需要公网地址以接收消息推送,请查看文档了解使用方式
zh_Hant: 微信公眾號適配器,需要公網地址以接收訊息推送,請查看文件了解使用方式
icon: officialaccount.png
spec:
categories:
- china
help_links:
zh: https://link.langbot.app/zh/platforms/officialaccount
en: https://link.langbot.app/en/platforms/officialaccount
ja: https://link.langbot.app/ja/platforms/officialaccount
config:
- name: webhook_url
label:
en_US: Webhook Callback URL
zh_Hans: Webhook 回调地址
zh_Hant: Webhook 回調地址
description:
en_US: Copy this URL and paste it into your Official Account webhook configuration
zh_Hans: 复制此地址并粘贴到微信公众号的 Webhook 配置中
zh_Hant: 複製此地址並貼到微信公眾號的 Webhook 設定中
type: webhook-url
required: false
default: ""
- name: token
label:
en_US: Token
zh_Hans: 令牌
type: string
zh_Hant: 令牌
required: true
default: ""
- name: EncodingAESKey
label:
en_US: EncodingAESKey
zh_Hans: 消息加解密密钥
zh_Hant: 訊息加解密密鑰
type: string
required: true
default: ""
@@ -29,6 +50,7 @@ spec:
label:
en_US: App ID
zh_Hans: 应用ID
zh_Hant: 應用ID
type: string
required: true
default: ""
@@ -36,6 +58,7 @@ spec:
label:
en_US: App Secret
zh_Hans: 应用密钥
zh_Hant: 應用密鑰
type: string
required: true
default: ""
@@ -43,6 +66,7 @@ spec:
label:
en_US: Mode
zh_Hans: 接入模式
zh_Hant: 接入模式
type: string
required: true
default: "drop"
@@ -50,6 +74,7 @@ spec:
label:
en_US: Loading Message
zh_Hans: 加载消息
zh_Hant: 載入訊息
type: string
required: true
default: "AI正在思考中请发送任意内容获取回复。"
@@ -57,9 +82,11 @@ spec:
label:
en_US: API Base URL
zh_Hans: API 基础 URL
zh_Hant: API 基礎 URL
description:
en_US: API Base URL, used for accessing the Official Account API. If you are deploying in an internal network environment and accessing the Official Account API through a reverse proxy, please fill in this item according to the documentation.
zh_Hans: 可选,若您部署在内网环境并通过反向代理访问微信公众号 API可根据文档修改此项
zh_Hant: 可選,若您部署在內網環境並透過反向代理存取微信公眾號 API可根據文件修改此項
type: string
required: false
default: "https://api.weixin.qq.com"

View File

@@ -0,0 +1,577 @@
"""OpenClaw WeChat adapter for LangBot.
Uses the OpenClaw WeChat HTTP JSON API (long-poll getUpdates + sendMessage)
to integrate personal WeChat accounts with LangBot.
Reference: https://github.com/epiral/weixin-bot
"""
from __future__ import annotations
import asyncio
import base64
import traceback
import typing
import pydantic
import sqlalchemy
from langbot.libs.openclaw_weixin_api.client import (
DEFAULT_BASE_URL,
SESSION_EXPIRED_ERRCODE,
OpenClawWeixinClient,
)
from langbot.libs.openclaw_weixin_api.types import (
MessageItem,
WeixinMessage,
)
from langbot.pkg.entity.persistence import bot as persistence_bot
import langbot_plugin.api.definition.abstract.platform.adapter as abstract_platform_adapter
import langbot_plugin.api.definition.abstract.platform.event_logger as abstract_platform_logger
import langbot_plugin.api.entities.builtin.platform.entities as platform_entities
import langbot_plugin.api.entities.builtin.platform.events as platform_events
import langbot_plugin.api.entities.builtin.platform.message as platform_message
class OpenClawWeixinMessageConverter(abstract_platform_adapter.AbstractMessageConverter):
"""Converts between LangBot MessageChain and OpenClaw WeChat message items."""
@staticmethod
async def yiri2target(message_chain: platform_message.MessageChain) -> list[dict]:
"""Convert LangBot MessageChain to a list of OpenClaw message item dicts."""
items = []
for component in message_chain:
if isinstance(component, platform_message.Plain):
items.append({'type': MessageItem.TEXT, 'text_item': {'text': component.text}})
elif isinstance(component, platform_message.Image):
# OpenClaw WeChat only supports text messages without CDN upload.
# For images, we send a placeholder text with the URL if available.
if component.url:
items.append(
{
'type': MessageItem.TEXT,
'text_item': {'text': f'[Image: {component.url}]'},
}
)
elif component.base64:
items.append(
{
'type': MessageItem.TEXT,
'text_item': {'text': '[Image]'},
}
)
elif isinstance(component, platform_message.File):
if component.name:
items.append(
{
'type': MessageItem.TEXT,
'text_item': {'text': f'[File: {component.name}]'},
}
)
elif isinstance(component, platform_message.Forward):
for node in component.node_list:
if node.message_chain:
items.extend(await OpenClawWeixinMessageConverter.yiri2target(node.message_chain))
return items
@staticmethod
async def target2yiri(
msg: WeixinMessage,
) -> platform_message.MessageChain:
"""Convert an OpenClaw WeixinMessage to LangBot MessageChain."""
components: list[platform_message.MessageComponent] = []
if not msg.item_list:
return platform_message.MessageChain(components)
for item in msg.item_list:
if item.type == MessageItem.TEXT and item.text_item and item.text_item.text:
text = item.text_item.text
# Handle quoted messages
if item.ref_msg:
ref_parts = []
if item.ref_msg.title:
ref_parts.append(item.ref_msg.title)
if item.ref_msg.message_item:
ref_item = item.ref_msg.message_item
if ref_item.text_item and ref_item.text_item.text:
ref_parts.append(ref_item.text_item.text)
if ref_parts:
components.append(
platform_message.Quote(
sender_id='',
origin=platform_message.MessageChain(
[platform_message.Plain(text=' | '.join(ref_parts))]
),
)
)
components.append(platform_message.Plain(text=text))
elif item.type == MessageItem.IMAGE and item.image_item:
if hasattr(item.image_item, '_downloaded_bytes') and item.image_item._downloaded_bytes:
b64 = base64.b64encode(item.image_item._downloaded_bytes).decode('utf-8')
components.append(platform_message.Image(base64=f'data:image/jpeg;base64,{b64}'))
else:
components.append(platform_message.Unknown(text='[Image]'))
elif item.type == MessageItem.VOICE and item.voice_item:
# Voice with speech-to-text: use the transcribed text
if item.voice_item.text:
components.append(platform_message.Plain(text=item.voice_item.text))
else:
components.append(platform_message.Unknown(text='[Voice]'))
# TODO: enable after full testing
# elif item.type == MessageItem.VOICE and item.voice_item:
# if item.voice_item.text:
# components.append(platform_message.Plain(text=item.voice_item.text))
# elif hasattr(item.voice_item, '_downloaded_bytes') and item.voice_item._downloaded_bytes:
# b64 = base64.b64encode(item.voice_item._downloaded_bytes).decode('utf-8')
# components.append(
# platform_message.Voice(
# base64=b64,
# length=item.voice_item.playtime or 0,
# )
# )
# else:
# components.append(
# platform_message.Voice(
# length=item.voice_item.playtime or 0,
# )
# )
elif item.type == MessageItem.FILE and item.file_item:
components.append(platform_message.Unknown(text=f'[File: {item.file_item.file_name or ""}]'))
# TODO: enable after full testing
# elif item.type == MessageItem.FILE and item.file_item:
# file_name = item.file_item.file_name or ''
# file_size = int(item.file_item.len) if item.file_item.len else 0
# if hasattr(item.file_item, '_downloaded_bytes') and item.file_item._downloaded_bytes:
# b64 = base64.b64encode(item.file_item._downloaded_bytes).decode('utf-8')
# components.append(
# platform_message.File(
# name=file_name,
# size=file_size,
# base64=b64,
# )
# )
# else:
# components.append(
# platform_message.File(
# name=file_name,
# size=file_size,
# )
# )
elif item.type == MessageItem.VIDEO and item.video_item:
components.append(platform_message.Unknown(text='[Video]'))
# TODO: enable after full testing
# elif item.type == MessageItem.VIDEO and item.video_item:
# if hasattr(item.video_item, '_downloaded_bytes') and item.video_item._downloaded_bytes:
# b64 = base64.b64encode(item.video_item._downloaded_bytes).decode('utf-8')
# components.append(
# platform_message.File(
# name='video.mp4',
# size=item.video_item.video_size or 0,
# base64=b64,
# )
# )
# else:
# components.append(
# platform_message.File(
# name='video.mp4',
# size=item.video_item.video_size or 0,
# )
# )
else:
components.append(platform_message.Unknown(text='[Unknown message type]'))
return platform_message.MessageChain(components)
class OpenClawWeixinEventConverter(abstract_platform_adapter.AbstractEventConverter):
"""Converts OpenClaw WeChat messages to LangBot events."""
@staticmethod
async def yiri2target(event: platform_events.MessageEvent) -> dict:
return event.source_platform_object
@staticmethod
async def target2yiri(msg: WeixinMessage) -> typing.Optional[platform_events.MessageEvent]:
"""Convert an inbound WeixinMessage to a LangBot event."""
if msg.message_type != WeixinMessage.TYPE_USER:
return None
from_user_id = msg.from_user_id or ''
if not from_user_id:
return None
message_chain = await OpenClawWeixinMessageConverter.target2yiri(msg)
if not message_chain:
return None
timestamp = (msg.create_time_ms or 0) / 1000.0
return platform_events.FriendMessage(
sender=platform_entities.Friend(
id=from_user_id,
nickname=from_user_id,
remark='',
),
message_chain=message_chain,
time=timestamp,
source_platform_object=msg,
)
class OpenClawWeixinAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
"""LangBot adapter for OpenClaw WeChat (long-poll based)."""
name: str = 'openclaw-weixin'
client: OpenClawWeixinClient = pydantic.Field(exclude=True)
config: dict
message_converter: OpenClawWeixinMessageConverter = OpenClawWeixinMessageConverter()
event_converter: OpenClawWeixinEventConverter = OpenClawWeixinEventConverter()
# context_token cache: from_user_id -> context_token
_context_tokens: dict[str, str] = pydantic.PrivateAttr(default_factory=dict)
_polling: bool = pydantic.PrivateAttr(default=False)
_poll_task: typing.Optional[asyncio.Task] = pydantic.PrivateAttr(default=None)
_bot_uuid: typing.Optional[str] = pydantic.PrivateAttr(default=None)
listeners: typing.Dict[
typing.Type[platform_events.Event],
typing.Callable[[platform_events.Event, abstract_platform_adapter.AbstractMessagePlatformAdapter], None],
] = {}
def __init__(self, config: dict, logger: abstract_platform_logger.AbstractEventLogger):
client = OpenClawWeixinClient(
base_url=config.get('base_url', DEFAULT_BASE_URL),
token=config.get('token', ''),
)
super().__init__(
config=config,
logger=logger,
client=client,
bot_account_id='',
listeners={},
name='openclaw-weixin',
)
def set_bot_uuid(self, bot_uuid: str):
"""Called by BotManager to provide the bot's UUID for config persistence."""
self._bot_uuid = bot_uuid
async def _persist_config(self) -> None:
"""Persist current self.config to the database so token survives restart."""
if not self._bot_uuid:
return
try:
ap = self.logger.ap
await ap.persistence_mgr.execute_async(
sqlalchemy.update(persistence_bot.Bot)
.where(persistence_bot.Bot.uuid == self._bot_uuid)
.values(adapter_config=self.config)
)
except Exception as e:
await self.logger.warning(f'Failed to persist adapter config: {e}')
async def _do_login(self) -> None:
"""Run the QR code login flow via client.login() and update config."""
adapter_logger = self.logger
async def _on_qrcode(qr_base64: str, _qr_url: str):
await adapter_logger.info(
f'Please scan the QR code to login WeChat: {_qr_url}',
images=[platform_message.Image(base64=qr_base64)],
)
login_result = await self.client.login(
on_qrcode=_on_qrcode,
)
# client.login() already updates client.token and client.base_url
self.config['token'] = login_result.token
self.config['base_url'] = login_result.base_url
if login_result.account_id:
self.config['account_id'] = login_result.account_id
await self.logger.info(f'WeChat login successful! account_id={login_result.account_id}')
# Persist token to database so it survives restart
await self._persist_config()
async def send_message(
self,
target_type: str,
target_id: str,
message: platform_message.MessageChain,
):
"""Send a message to a user."""
context_token = self._context_tokens.get(target_id, '')
for component in message:
try:
if isinstance(component, platform_message.Plain):
if component.text:
await self.client.send_text(target_id, component.text, context_token)
elif isinstance(component, platform_message.Image):
img_bytes, _ = await component.get_bytes()
await self.client.send_image(target_id, img_bytes, context_token)
elif isinstance(component, platform_message.File):
file_bytes = await self._get_component_bytes(component)
if file_bytes:
await self.client.send_file(target_id, file_bytes, component.name or 'file', context_token)
elif isinstance(component, platform_message.Voice):
voice_bytes = await self._get_component_bytes(component)
if voice_bytes:
await self.client.send_voice(target_id, voice_bytes, component.length or 0, context_token)
elif isinstance(component, platform_message.Forward):
for node in component.node_list:
if node.message_chain:
await self.send_message(target_type, target_id, node.message_chain)
except Exception:
await self.logger.error(
f'Failed to send component {type(component).__name__}: {traceback.format_exc()}'
)
async def reply_message(
self,
message_source: platform_events.MessageEvent,
message: platform_message.MessageChain,
quote_origin: bool = False,
):
"""Reply to a received message."""
source_msg = message_source.source_platform_object
if isinstance(source_msg, WeixinMessage):
target_id = source_msg.from_user_id or ''
if target_id:
await self.send_message('friend', target_id, message)
async def is_muted(self, group_id: int) -> bool:
return False
@staticmethod
async def _get_component_bytes(component: platform_message.MessageComponent) -> typing.Optional[bytes]:
"""Extract raw bytes from a File or Voice component."""
b64_val = getattr(component, 'base64', None)
url_val = getattr(component, 'url', None)
path_val = getattr(component, 'path', None)
if b64_val:
return base64.b64decode(b64_val)
elif url_val and url_val.startswith(('http://', 'https://')):
import aiohttp
async with aiohttp.ClientSession() as session:
async with session.get(url_val) as resp:
if resp.status == 200:
return await resp.read()
elif path_val:
import asyncio
with open(path_val, 'rb') as f:
return await asyncio.to_thread(f.read)
return None
def register_listener(
self,
event_type: typing.Type[platform_events.Event],
callback: typing.Callable[
[platform_events.Event, abstract_platform_adapter.AbstractMessagePlatformAdapter],
None,
],
):
self.listeners[event_type] = callback
def unregister_listener(
self,
event_type: typing.Type[platform_events.Event],
callback: typing.Callable[
[platform_events.Event, abstract_platform_adapter.AbstractMessagePlatformAdapter],
None,
],
):
self.listeners.pop(event_type, None)
async def run_async(self):
"""Start the adapter. If no token is configured, trigger QR code login first."""
base_url = self.config.get('base_url', DEFAULT_BASE_URL)
token = self.config.get('token', '')
await self.logger.info('OpenClaw WeChat adapter starting...')
# QR code login flow when no token is provided
if not token:
await self.logger.info('No token configured, starting QR code login...')
try:
await self._do_login()
except Exception as e:
await self.logger.error(f'QR code login failed: {e}')
raise
# Rebuild client with the (possibly updated) config
self.client = OpenClawWeixinClient(
base_url=self.config.get('base_url', base_url),
token=self.config.get('token', token),
)
self.bot_account_id = self.config.get('account_id', 'openclaw-weixin')
self._polling = True
# Start the long-poll loop
self._poll_task = asyncio.create_task(self._poll_loop())
await self.logger.info('OpenClaw WeChat adapter running')
try:
await self._poll_task
except asyncio.CancelledError:
pass
async def _poll_loop(self):
"""Long-poll loop: call getUpdates continuously.
Error handling follows the weixin-bot SDK pattern:
- Exponential backoff (1s -> 10s max) on failures
- Session expired (errcode -14) triggers automatic re-login
"""
get_updates_buf = ''
poll_timeout = float(self.config.get('poll_timeout', 35))
backoff_delay = 1.0
max_backoff = 10.0
while self._polling:
try:
resp = await self.client.get_updates(
get_updates_buf=get_updates_buf,
timeout=poll_timeout + 5,
)
if resp.longpolling_timeout_ms and resp.longpolling_timeout_ms > 0:
poll_timeout = resp.longpolling_timeout_ms / 1000.0
is_api_error = (resp.ret is not None and resp.ret != 0) or (
resp.errcode is not None and resp.errcode != 0
)
if is_api_error:
is_session_expired = resp.errcode == SESSION_EXPIRED_ERRCODE or resp.ret == SESSION_EXPIRED_ERRCODE
if is_session_expired:
await self.logger.error('OpenClaw WeChat session expired, attempting re-login...')
try:
await self._do_login()
# Rebuild client with new credentials
self.client = OpenClawWeixinClient(
base_url=self.config.get('base_url', DEFAULT_BASE_URL),
token=self.config.get('token', ''),
)
self._context_tokens.clear()
get_updates_buf = ''
backoff_delay = 1.0
continue
except Exception:
await self.logger.error(f'Re-login failed: {traceback.format_exc()}')
break
await self.logger.error(
f'OpenClaw getUpdates failed: ret={resp.ret} errcode={resp.errcode} errmsg={resp.errmsg}'
)
await asyncio.sleep(backoff_delay)
backoff_delay = min(backoff_delay * 2, max_backoff)
continue
backoff_delay = 1.0
if resp.get_updates_buf:
get_updates_buf = resp.get_updates_buf
for msg in resp.msgs:
try:
await self._handle_inbound_message(msg)
except Exception:
await self.logger.error(f'Error handling message: {traceback.format_exc()}')
except asyncio.CancelledError:
break
except Exception:
await self.logger.error(f'OpenClaw poll error: {traceback.format_exc()}')
await asyncio.sleep(backoff_delay)
backoff_delay = min(backoff_delay * 2, max_backoff)
async def _handle_inbound_message(self, msg: WeixinMessage):
"""Process a single inbound message from getUpdates."""
if msg.context_token and msg.from_user_id:
self._context_tokens[msg.from_user_id] = msg.context_token
# Download CDN media (files, images) before converting to LangBot events
await self._download_media_items(msg)
event = await OpenClawWeixinEventConverter.target2yiri(msg)
if event is None:
return
if type(event) in self.listeners:
await self.listeners[type(event)](event, self)
async def _download_media_items(self, msg: WeixinMessage):
"""Download CDN media for image items in the message."""
if not msg.item_list:
return
for item in msg.item_list:
try:
if item.type == MessageItem.IMAGE and item.image_item:
if (
item.image_item.media
and item.image_item.media.encrypt_query_param
and item.image_item.media.aes_key
):
img_bytes = await self.client.download_media(item.image_item.media)
item.image_item._downloaded_bytes = img_bytes
# TODO: enable after full testing
# elif item.type == MessageItem.FILE and item.file_item and item.file_item.media:
# if item.file_item.media.encrypt_query_param and item.file_item.media.aes_key:
# file_bytes = await self.client.download_media(item.file_item.media)
# item.file_item._downloaded_bytes = file_bytes
#
# elif item.type == MessageItem.VOICE and item.voice_item and item.voice_item.media:
# if item.voice_item.media.encrypt_query_param and item.voice_item.media.aes_key:
# voice_bytes = await self.client.download_media(item.voice_item.media)
# item.voice_item._downloaded_bytes = voice_bytes
#
# elif item.type == MessageItem.VIDEO and item.video_item and item.video_item.media:
# if item.video_item.media.encrypt_query_param and item.video_item.media.aes_key:
# video_bytes = await self.client.download_media(item.video_item.media)
# item.video_item._downloaded_bytes = video_bytes
except Exception:
await self.logger.warning(f'Failed to download CDN media: {traceback.format_exc()}')
async def kill(self) -> bool:
"""Stop the adapter."""
self._polling = False
if self._poll_task and not self._poll_task.done():
self._poll_task.cancel()
try:
await self._poll_task
except asyncio.CancelledError:
pass
await self.client.close()
await self.logger.info('OpenClaw WeChat adapter stopped')
return True

View File

@@ -0,0 +1,74 @@
apiVersion: v1
kind: MessagePlatformAdapter
metadata:
name: openclaw-weixin
label:
en_US: OpenClaw WeChat
zh_Hans: 个人微信机器人
zh_Hant: 個人微信機器人
description:
en_US: OpenClaw WeChat adapter, supports personal WeChat via QR code login
zh_Hans: 微信官方个人助手,扫码即可登录使用
zh_Hant: 微信官方個人助手,掃碼即可登入使用
icon: wechat.png
spec:
categories:
- popular
- china
help_links:
zh: https://link.langbot.app/zh/platforms/openclaw_weixin
en: https://link.langbot.app/en/platforms/openclaw_weixin
ja: https://link.langbot.app/ja/platforms/openclaw_weixin
config:
- name: base_url
label:
en_US: API Base URL
zh_Hans: API 基础地址
zh_Hant: API 基礎地址
description:
en_US: The base URL of the OpenClaw WeChat backend API
zh_Hans: OpenClaw 微信后端 API 的基础地址
zh_Hant: OpenClaw 微信後端 API 的基礎地址
type: string
required: true
default: "https://ilinkai.weixin.qq.com"
- name: token
label:
en_US: Token
zh_Hans: 令牌
zh_Hant: 令牌
description:
en_US: Bearer token obtained after QR code login authorization. Leave empty to trigger QR code login on startup.
zh_Hans: 扫码登录授权后获取的 Bearer 令牌。请留空并保存,将在启动时输出二维码到日志,扫码后即可自动登录。
zh_Hant: 掃碼登入授權後取得的 Bearer 令牌。請留空並儲存,將在啟動時輸出 QR Code 到日誌,掃碼後即可自動登入。
type: string
required: false
default: ""
- name: account_id
label:
en_US: Account ID
zh_Hans: 账号标识
zh_Hant: 帳號標識
description:
en_US: A label for this WeChat account (used for display purposes)
zh_Hans: 此微信账号的标识(用于显示)
zh_Hant: 此微信帳號的標識(用於顯示)
type: string
required: false
default: "openclaw-weixin"
- name: poll_timeout
label:
en_US: Poll Timeout (seconds)
zh_Hans: 轮询超时(秒)
zh_Hant: 輪詢逾時(秒)
description:
en_US: Long-poll timeout for getUpdates, the server may hold the request up to this duration
zh_Hans: getUpdates 长轮询超时时间,服务端最多持有请求的时长
zh_Hant: getUpdates 長輪詢逾時時間,伺服端最多持有請求的時長
type: integer
required: false
default: 35
execution:
python:
path: ./openclaw_weixin.py
attr: OpenClawWeixinAdapter

View File

@@ -5,16 +5,37 @@ metadata:
label:
en_US: QQ Official API
zh_Hans: QQ 官方 API
zh_Hant: QQ 官方 API
description:
en_US: QQ Official API (Webhook)
zh_Hans: QQ 官方 API (Webhook),请查看文档了解使用方式
zh_Hans: QQ 官方 API (Webhook)需要公网地址以接收消息推送,请查看文档了解使用方式
zh_Hant: QQ 官方 API (Webhook),需要公網地址以接收訊息推送,請查看文件了解使用方式
icon: qqofficial.svg
spec:
categories:
- china
help_links:
zh: https://link.langbot.app/zh/platforms/qqofficial
en: https://link.langbot.app/en/platforms/qqofficial
ja: https://link.langbot.app/ja/platforms/qqofficial
config:
- name: webhook_url
label:
en_US: Webhook Callback URL
zh_Hans: Webhook 回调地址
zh_Hant: Webhook 回調地址
description:
en_US: Copy this URL and paste it into your QQ Official API webhook configuration
zh_Hans: 复制此地址并粘贴到 QQ 官方 API 的 Webhook 配置中
zh_Hant: 複製此地址並貼到 QQ 官方 API 的 Webhook 設定中
type: webhook-url
required: false
default: ""
- name: appid
label:
en_US: App ID
zh_Hans: 应用ID
zh_Hant: 應用ID
type: string
required: true
default: ""
@@ -22,6 +43,7 @@ spec:
label:
en_US: Secret
zh_Hans: 密钥
zh_Hant: 密鑰
type: string
required: true
default: ""
@@ -29,6 +51,7 @@ spec:
label:
en_US: Token
zh_Hans: 令牌
zh_Hant: 令牌
type: string
required: true
default: ""

View File

@@ -5,36 +5,70 @@ metadata:
label:
en_US: Satori
zh_Hans: Satori
zh_Hant: Satori
th_TH: Satori
vi_VN: Satori
es_ES: Satori
description:
en_US: SatoriAdapter
zh_Hans: 古明地觉协议适配器
zh_Hans: Satori 协议适配器,支持多种平台的接入,请查看文档了解使用方式
zh_Hant: Satori 協定適配器,支援多種平台的接入,請查看文件了解使用方式
th_TH: อะแดปเตอร์โปรโตคอล Satori รองรับการเชื่อมต่อหลายแพลตฟอร์ม โปรดดูเอกสารประกอบสำหรับวิธีการใช้งาน
vi_VN: Bộ điều hợp giao thức Satori, hỗ trợ kết nối nhiều nền tảng, vui lòng xem tài liệu để biết cách sử dụng
es_ES: Adaptador del protocolo Satori, soporta acceso a múltiples plataformas, consulte la documentación para obtener instrucciones de uso
icon: satori.png
spec:
categories:
- protocol
help_links:
zh: https://link.langbot.app/zh/platforms/satori
en: https://link.langbot.app/en/platforms/satori
ja: https://link.langbot.app/ja/platforms/satori
config:
- name: platform
label:
en_US: Platform
zh_Hans: 平台名称
zh_Hant: 平台名稱
th_TH: ชื่อแพลตฟอร์ม
vi_VN: Tên nền tảng
es_ES: Nombre de la plataforma
type: string
required: true
default: "llonebot"
description:
en_US: The platform name (e.g., llonebot, discord, telegram)
zh_Hans: 平台名称(如 llonebot, discord, telegram
zh_Hant: 平台名稱(如 llonebot、discord、telegram
th_TH: ชื่อแพลตฟอร์ม (เช่น llonebot, discord, telegram)
vi_VN: "Tên nền tảng (ví dụ: llonebot, discord, telegram)"
es_ES: El nombre de la plataforma (p. ej., llonebot, discord, telegram)
- name: host
label:
en_US: Host
zh_Hans: 主机地址
zh_Hant: 主機地址
th_TH: ที่อยู่โฮสต์
vi_VN: Địa chỉ máy chủ
es_ES: Dirección del host
type: string
required: true
default: "127.0.0.1"
description:
en_US: The host address of LLOneBot Satori server (e.g., 127.0.0.1, localhost, 192.168.1.100)
zh_Hans: LLOneBot Satori服务器的主机地址如 127.0.0.1, localhost, 192.168.1.100
zh_Hant: LLOneBot Satori 伺服器的主機地址(如 127.0.0.1、localhost、192.168.1.100
th_TH: ที่อยู่โฮสต์ของเซิร์ฟเวอร์ LLOneBot Satori (เช่น 127.0.0.1, localhost, 192.168.1.100)
vi_VN: "Địa chỉ máy chủ LLOneBot Satori (ví dụ: 127.0.0.1, localhost, 192.168.1.100)"
es_ES: La dirección del host del servidor LLOneBot Satori (p. ej., 127.0.0.1, localhost, 192.168.1.100)
- name: port
label:
en_US: Port
zh_Hans: 监听端口
zh_Hant: 監聽連接埠
th_TH: พอร์ต
vi_VN: Cổng
es_ES: Puerto
type: integer
required: true
default: 5600
@@ -42,6 +76,10 @@ spec:
label:
en_US: Satori API Endpoint
zh_Hans: Satori API 终结点
zh_Hant: Satori API 端點
th_TH: จุดปลาย Satori API
vi_VN: Điểm cuối Satori API
es_ES: Punto de acceso de la API Satori
type: string
required: true
default: "http://localhost:5600/v1"
@@ -49,6 +87,10 @@ spec:
label:
en_US: Satori WebSocket Endpoint
zh_Hans: Satori WebSocket 终结点
zh_Hant: Satori WebSocket 端點
th_TH: จุดปลาย Satori WebSocket
vi_VN: Điểm cuối Satori WebSocket
es_ES: Punto de acceso WebSocket de Satori
type: string
required: true
default: "ws://localhost:5600/v1/events"
@@ -56,6 +98,10 @@ spec:
label:
en_US: Token
zh_Hans: 令牌
zh_Hant: 令牌
th_TH: โทเค็น
vi_VN: Mã thông báo
es_ES: Token
type: string
required: true
default: ""

View File

@@ -5,16 +5,58 @@ metadata:
label:
en_US: Slack
zh_Hans: Slack
zh_Hant: Slack
ja_JP: Slack
th_TH: Slack
vi_VN: Slack
es_ES: Slack
description:
en_US: Slack Adapter
zh_Hans: Slack 适配器,请查看文档了解使用方式
zh_Hans: Slack 适配器,需要公网地址以接收 Slack 消息推送,请查看文档了解使用方式
zh_Hant: Slack 適配器,需要公網地址以接收 Slack 訊息推送,請查看文件了解使用方式
ja_JP: Slack アダプター、Slackのメッセージプッシュを受信するためにパブリックURLが必要です。使用方法の詳細については、ドキュメントを参照してください。
th_TH: อะแดปเตอร์ Slack ต้องการที่อยู่สาธารณะเพื่อรับการแจ้งเตือนข้อความจาก Slack โปรดดูเอกสารประกอบสำหรับวิธีการใช้งาน
vi_VN: Bộ điều hợp Slack, cần địa chỉ công cộng để nhận thông báo tin nhắn từ Slack, vui lòng xem tài liệu để biết cách sử dụng
es_ES: Adaptador de Slack, requiere una dirección pública para recibir notificaciones de mensajes de Slack, consulte la documentación para obtener instrucciones de uso
icon: slack.png
spec:
categories:
- popular
- global
help_links:
zh: https://link.langbot.app/zh/platforms/slack
en: https://link.langbot.app/en/platforms/slack
ja: https://link.langbot.app/ja/platforms/slack
config:
- name: webhook_url
label:
en_US: Webhook Callback URL
zh_Hans: Webhook 回调地址
zh_Hant: Webhook 回調地址
ja_JP: Webhook コールバック URL
th_TH: URL การเรียกกลับ Webhook
vi_VN: URL gọi lại Webhook
es_ES: URL de devolución de llamada Webhook
description:
en_US: Copy this URL and paste it into your Slack app's event subscription configuration
zh_Hans: 复制此地址并粘贴到 Slack 应用的事件订阅配置中
zh_Hant: 複製此地址並貼到 Slack 應用的事件訂閱設定中
ja_JP: この URL をコピーして Slack アプリのイベントサブスクリプション設定に貼り付けてください
th_TH: คัดลอก URL นี้แล้ววางในการตั้งค่าการสมัครรับเหตุการณ์ของแอป Slack ของคุณ
vi_VN: Sao chép URL này và dán vào cấu hình đăng ký sự kiện của ứng dụng Slack của bạn
es_ES: Copie esta URL y péguela en la configuración de suscripción de eventos de su aplicación Slack
type: webhook-url
required: false
default: ""
- name: bot_token
label:
en_US: Bot Token
zh_Hans: 机器人令牌
zh_Hant: 機器人令牌
ja_JP: ボットトークン
th_TH: โทเค็นบอท
vi_VN: Mã thông báo Bot
es_ES: Token del bot
type: string
required: true
default: ""
@@ -22,6 +64,11 @@ spec:
label:
en_US: signing_secret
zh_Hans: 密钥
zh_Hant: 密鑰
ja_JP: 署名シークレット
th_TH: คีย์ลายเซ็น
vi_VN: Khóa ký
es_ES: Secreto de firma
type: string
required: true
default: ""

View File

@@ -1,4 +1,5 @@
from __future__ import annotations
import time
import telegram
@@ -41,6 +42,25 @@ class TelegramMessageConverter(abstract_platform_adapter.AbstractMessageConverte
photo_bytes = f.read()
components.append({'type': 'photo', 'photo': photo_bytes})
elif isinstance(component, platform_message.File):
file_bytes = None
if component.base64:
# Strip data URI prefix if present (e.g. "data:application/pdf;base64,...")
b64_data = component.base64
if ';base64,' in b64_data:
b64_data = b64_data.split(';base64,', 1)[1]
file_bytes = base64.b64decode(b64_data)
elif component.url:
session = httpclient.get_session()
async with session.get(component.url) as response:
file_bytes = await response.read()
elif component.path:
with open(component.path, 'rb') as f:
file_bytes = f.read()
file_name = getattr(component, 'name', None) or 'file'
components.append({'type': 'document', 'document': file_bytes, 'filename': file_name})
elif isinstance(component, platform_message.Forward):
for node in component.node_list:
components.extend(await TelegramMessageConverter.yiri2target(node.message_chain, bot))
@@ -103,6 +123,27 @@ class TelegramMessageConverter(abstract_platform_adapter.AbstractMessageConverte
)
)
if message.document:
if message.caption:
message_components.extend(parse_message_text(message.caption))
file = await message.document.get_file()
file_name = message.document.file_name or 'document'
file_size = message.document.file_size or 0
file_format = message.document.mime_type or 'application/octet-stream'
file_bytes = None
async with httpclient.get_session(trust_env=True).get(file.file_path) as response:
file_bytes = await response.read()
message_components.append(
platform_message.File(
name=file_name,
size=file_size,
base64=f'data:{file_format};base64,{base64.b64encode(file_bytes).decode("utf-8")}',
)
)
return platform_message.MessageChain(message_components)
@@ -178,7 +219,10 @@ class TelegramAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
application = ApplicationBuilder().token(config['token']).build()
bot = application.bot
application.add_handler(
MessageHandler(filters.TEXT | (filters.COMMAND) | filters.PHOTO | filters.VOICE, telegram_callback)
MessageHandler(
filters.TEXT | (filters.COMMAND) | filters.PHOTO | filters.VOICE | filters.Document.ALL,
telegram_callback,
)
)
super().__init__(
config=config,
@@ -217,6 +261,13 @@ class TelegramAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
continue
args['photo'] = telegram.InputFile(photo)
await self.bot.send_photo(**args)
elif component_type == 'document':
doc = component.get('document')
if doc is None:
continue
filename = component.get('filename', 'file')
args['document'] = telegram.InputFile(doc, filename=filename)
await self.bot.send_document(**args)
async def reply_message(
self,
@@ -250,6 +301,39 @@ class TelegramAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
await self.bot.send_message(**args)
def _process_markdown(self, text: str) -> str:
if self.config.get('markdown_card', False):
return telegramify_markdown.markdownify(content=text)
return text
def _build_message_args(self, chat_id: int, text: str, message_thread_id: int = None, **extra_args) -> dict:
args = {'chat_id': chat_id, 'text': self._process_markdown(text), **extra_args}
if message_thread_id:
args['message_thread_id'] = message_thread_id
if self.config.get('markdown_card', False):
args['parse_mode'] = 'MarkdownV2'
return args
async def create_message_card(self, message_id, event):
assert isinstance(event.source_platform_object, Update)
update = event.source_platform_object
chat_id = update.effective_chat.id
chat_type = update.effective_chat.type
message_thread_id = update.message.message_thread_id
if chat_type == 'private':
draft_id = int(time.time() * 1000)
self.msg_stream_id[message_id] = ('private', draft_id)
args = self._build_message_args(chat_id, 'Thinking...', message_thread_id, draft_id=draft_id)
await self.bot.send_message_draft(**args)
else:
args = self._build_message_args(chat_id, 'Thinking...', message_thread_id)
send_msg = await self.bot.send_message(**args)
self.msg_stream_id[message_id] = ('group', send_msg.message_id)
return True
async def reply_message_chunk(
self,
message_source: platform_events.MessageEvent,
@@ -258,59 +342,47 @@ class TelegramAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
quote_origin: bool = False,
is_final: bool = False,
):
message_id = bot_message.resp_message_id
msg_seq = bot_message.msg_sequence
if (msg_seq - 1) % 8 == 0 or is_final:
assert isinstance(message_source.source_platform_object, Update)
components = await TelegramMessageConverter.yiri2target(message, self.bot)
args = {}
message_id = message_source.source_platform_object.message.id
assert isinstance(message_source.source_platform_object, Update)
update = message_source.source_platform_object
chat_id = update.effective_chat.id
message_thread_id = update.message.message_thread_id
component = components[0]
if message_id not in self.msg_stream_id: # 当消息回复第一次时,发送新消息
# time.sleep(0.6)
if component['type'] == 'text':
if self.config['markdown_card'] is True:
content = telegramify_markdown.markdownify(
content=component['text'],
)
else:
content = component['text']
args = {
'chat_id': message_source.source_platform_object.effective_chat.id,
'text': content,
}
if message_source.source_platform_object.message.message_thread_id:
args['message_thread_id'] = message_source.source_platform_object.message.message_thread_id
if message_id not in self.msg_stream_id:
return
if quote_origin:
args['reply_to_message_id'] = message_source.source_platform_object.message.id
chat_mode, draft_id = self.msg_stream_id[message_id]
components = await TelegramMessageConverter.yiri2target(message, self.bot)
if self.config['markdown_card'] is True:
args['parse_mode'] = 'MarkdownV2'
send_msg = await self.bot.send_message(**args)
send_msg_id = send_msg.message_id
self.msg_stream_id[message_id] = send_msg_id
else: # 存在消息的时候直接编辑消息1
if component['type'] == 'text':
if self.config['markdown_card'] is True:
content = telegramify_markdown.markdownify(
content=component['text'],
)
else:
content = component['text']
args = {
'message_id': self.msg_stream_id[message_id],
'chat_id': message_source.source_platform_object.effective_chat.id,
'text': content,
}
if self.config['markdown_card'] is True:
args['parse_mode'] = 'MarkdownV2'
await self.bot.edit_message_text(**args)
if not components or components[0]['type'] != 'text':
if is_final and bot_message.tool_calls is None:
# self.seq = 1 # 消息回复结束之后重置seq
self.msg_stream_id.pop(message_id) # 消息回复结束之后删除流式消息id
self.msg_stream_id.pop(message_id)
return
content = components[0]['text']
if chat_mode == 'private':
args = self._build_message_args(chat_id, content, message_thread_id, draft_id=draft_id)
await self.bot.send_message_draft(**args)
if is_final and bot_message.tool_calls is None:
del args['draft_id']
await self.bot.send_message(**args)
self.msg_stream_id.pop(message_id)
else:
stream_id = draft_id
if (msg_seq - 1) % 8 == 0 or is_final:
args = {
'message_id': stream_id,
'chat_id': chat_id,
'text': self._process_markdown(content),
}
if self.config.get('markdown_card', False):
args['parse_mode'] = 'MarkdownV2'
await self.bot.edit_message_text(**args)
if is_final and bot_message.tool_calls is None:
self.msg_stream_id.pop(message_id)
def get_launcher_id(self, event: platform_events.MessageEvent) -> str | None:
if not isinstance(event.source_platform_object, Update):

View File

@@ -5,23 +5,50 @@ metadata:
label:
en_US: Telegram
zh_Hans: 电报
zh_Hant: Telegram
ja_JP: Telegram
th_TH: Telegram
vi_VN: Telegram
es_ES: Telegram
description:
en_US: Telegram Adapter
zh_Hans: 电报适配器,请查看文档了解使用方式
zh_Hans: Telegram 适配器,请查看文档了解使用方式
zh_Hant: Telegram 適配器,請查看文件了解使用方式
ja_JP: Telegram アダプター。使用方法の詳細については、ドキュメントを参照してください。
th_TH: อะแดปเตอร์ Telegram โปรดดูเอกสารประกอบสำหรับวิธีการใช้งาน
vi_VN: Bộ điều hợp Telegram, vui lòng xem tài liệu để biết cách sử dụng
es_ES: Adaptador de Telegram, consulte la documentación para obtener instrucciones de uso
icon: telegram.svg
spec:
categories:
- popular
- global
help_links:
zh: https://link.langbot.app/zh/platforms/telegram
en: https://link.langbot.app/en/platforms/telegram
ja: https://link.langbot.app/ja/platforms/telegram
config:
- name: token
label:
en_US: Token
zh_Hans: 令牌
zh_Hant: 令牌
ja_JP: トークン
th_TH: โทเค็น
vi_VN: Mã thông báo
es_ES: Token
type: string
required: true
default: ""
default: "token_from_botfather"
- name: markdown_card
label:
en_US: Markdown Card
zh_Hans: 是否使用 Markdown 卡片
zh_Hant: 是否使用 Markdown 卡片
ja_JP: Markdown カードを使用
th_TH: การ์ด Markdown
vi_VN: Thẻ Markdown
es_ES: Tarjeta Markdown
type: boolean
required: false
default: true
@@ -29,9 +56,19 @@ spec:
label:
en_US: Enable Stream Reply Mode
zh_Hans: 启用电报流式回复模式
zh_Hant: 啟用 Telegram 串流回覆模式
ja_JP: ストリーミング返信モードを有効化
th_TH: เปิดใช้งานโหมดตอบกลับแบบสตรีม
vi_VN: Bật chế độ trả lời trực tuyến
es_ES: Habilitar modo de respuesta en streaming
description:
en_US: If enabled, the bot will use the stream of telegram reply mode
zh_Hans: 如果启用,将使用电报流式方式来回复内容
zh_Hant: 如果啟用,將使用 Telegram 串流方式來回覆內容
ja_JP: 有効にすると、ボットはストリーミングモードでメッセージに返信します
th_TH: หากเปิดใช้งาน บอทจะใช้โหมดสตรีมของ Telegram ในการตอบกลับ
vi_VN: Nếu bật, bot sẽ sử dụng chế độ trả lời trực tuyến của Telegram
es_ES: Si está habilitado, el bot usará el modo de respuesta en streaming de Telegram
type: boolean
required: true
default: false

View File

@@ -5,11 +5,21 @@ metadata:
label:
en_US: "WebSocket Chat"
zh_Hans: "WebSocket 聊天"
zh_Hant: "WebSocket 聊天"
th_TH: "แชท WebSocket"
vi_VN: "Trò chuyện WebSocket"
es_ES: "Chat WebSocket"
description:
en_US: "WebSocket adapter for bidirectional real-time communication"
zh_Hans: "用于双向实时通信的 WebSocket 适配器"
zh_Hant: "用於雙向即時通訊的 WebSocket 適配器"
th_TH: "อะแดปเตอร์ WebSocket สำหรับการสื่อสารแบบเรียลไทม์สองทิศทาง"
vi_VN: "Bộ điều hợp WebSocket cho giao tiếp thời gian thực hai chiều"
es_ES: "Adaptador WebSocket para comunicación bidireccional en tiempo real"
icon: ""
spec:
categories:
- protocol
config: []
execution:
python:

View File

@@ -37,16 +37,24 @@ class WebSocketSession:
id: str
message_lists: dict[str, list[WebSocketMessage]] = {}
"""消息列表 {pipeline_uuid: [messages]}"""
stream_message_indexes: dict[str, dict[str, int]] = {}
"""流式消息索引 {pipeline_uuid: {resp_message_id: message_index}}"""
def __init__(self, id: str):
self.id = id
self.message_lists = {}
self.stream_message_indexes = {}
def get_message_list(self, pipeline_uuid: str) -> list[WebSocketMessage]:
if pipeline_uuid not in self.message_lists:
self.message_lists[pipeline_uuid] = []
return self.message_lists[pipeline_uuid]
def get_stream_message_indexes(self, pipeline_uuid: str) -> dict[str, int]:
if pipeline_uuid not in self.stream_message_indexes:
self.stream_message_indexes[pipeline_uuid] = {}
return self.stream_message_indexes[pipeline_uuid]
class WebSocketAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
"""WebSocket适配器 - 支持双向实时通信"""
@@ -89,20 +97,46 @@ class WebSocketAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter)
target_id: str,
message: platform_message.MessageChain,
) -> dict:
"""发送消息 - 这里用于主动推送消息到前端"""
message_data = {
'type': 'bot_message',
'target_type': target_type,
'target_id': target_id,
'content': str(message),
'message_chain': [component.__dict__ for component in message],
'timestamp': datetime.now().isoformat(),
}
"""发送消息 - 这里用于主动推送消息到前端
# 推送到所有相关连接
await self.outbound_message_queue.put(message_data)
对于 WebSocket 适配器,我们需要将消息广播到正确的 pipeline 连接。
target_id 可能是 launcher_id如 websocket_xxx或 pipeline_uuid。
我们需要尝试两种方式来确保消息能够送达。
"""
# 获取当前的 pipeline_uuid
pipeline_uuid = self.ap.platform_mgr.websocket_proxy_bot.bot_entity.use_pipeline_uuid
session_type = 'group' if target_type == 'group' else 'person'
return message_data
# 选择会话
session = self.websocket_group_session if session_type == 'group' else self.websocket_person_session
# 生成唯一消息ID
msg_id = len(session.get_message_list(pipeline_uuid)) + 1
message_data = WebSocketMessage(
id=msg_id,
role='assistant',
content=str(message),
message_chain=[component.__dict__ for component in message],
timestamp=datetime.now().isoformat(),
is_final=True,
)
# 保存到历史记录
session.get_message_list(pipeline_uuid).append(message_data)
# 直接广播到当前pipeline的连接
await ws_connection_manager.broadcast_to_pipeline(
pipeline_uuid,
{
'type': 'response',
'session_type': session_type,
'data': message_data.model_dump(),
},
session_type=session_type,
)
return message_data.model_dump()
async def reply_message(
self,
@@ -169,10 +203,16 @@ class WebSocketAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter)
pipeline_uuid = self.ap.platform_mgr.websocket_proxy_bot.bot_entity.use_pipeline_uuid
session_type = 'group' if isinstance(message_source, platform_events.GroupMessage) else 'person'
message_list = session.get_message_list(pipeline_uuid)
stream_message_indexes = session.get_stream_message_indexes(pipeline_uuid)
# 检查是否是新的流式消息通过bot_message对象判断
# 如果列表为空或者最后一条消息已经is_final=True则创建新消息
if not message_list or message_list[-1].is_final:
# Streaming messages in LangBot have a stable resp_message_id during the same assistant reply.
# Use it as the primary key to avoid overwriting an old card from a previous reply.
resp_message_id = str(getattr(bot_message, 'resp_message_id', '') or '')
existing_index = stream_message_indexes.get(resp_message_id) if resp_message_id else None
message_is_final = is_final and bot_message.tool_calls is None
if existing_index is None or existing_index >= len(message_list):
# 创建新消息
msg_id = len(message_list) + 1
message_data = WebSocketMessage(
@@ -181,27 +221,31 @@ class WebSocketAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter)
content=str(message),
message_chain=[component.__dict__ for component in message],
timestamp=datetime.now().isoformat(),
is_final=is_final and bot_message.tool_calls is None,
is_final=message_is_final,
)
# 只有在is_final时才保存到历史记录
if is_final and bot_message.tool_calls is None:
message_list.append(message_data)
# 立即添加到历史记录即使is_final=False以便后续块可以更新它
message_list.append(message_data)
if resp_message_id:
stream_message_indexes[resp_message_id] = len(message_list) - 1
else:
# 更新最后一条消息
msg_id = message_list[-1].id
# 更新同一条流式消息
old_message = message_list[existing_index]
msg_id = old_message.id
message_data = WebSocketMessage(
id=msg_id,
role='assistant',
content=str(message),
message_chain=[component.__dict__ for component in message],
timestamp=message_list[-1].timestamp, # 保持原始时间戳
is_final=is_final and bot_message.tool_calls is None,
timestamp=old_message.timestamp, # 保持原始时间戳
is_final=message_is_final,
)
# 如果是final更新历史记录中的最后一条
if is_final and bot_message.tool_calls is None:
message_list[-1] = message_data
# 更新历史记录中的对应消息
message_list[existing_index] = message_data
if message_is_final and resp_message_id:
stream_message_indexes.pop(resp_message_id, None)
# 直接广播到所有该pipeline的连接包含session_type信息
await ws_connection_manager.broadcast_to_pipeline(
@@ -410,6 +454,10 @@ class WebSocketAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter)
if session_type == 'person':
if pipeline_uuid in self.websocket_person_session.message_lists:
self.websocket_person_session.message_lists[pipeline_uuid] = []
if pipeline_uuid in self.websocket_person_session.stream_message_indexes:
self.websocket_person_session.stream_message_indexes[pipeline_uuid] = {}
else:
if pipeline_uuid in self.websocket_group_session.message_lists:
self.websocket_group_session.message_lists[pipeline_uuid] = []
if pipeline_uuid in self.websocket_group_session.stream_message_indexes:
self.websocket_group_session.stream_message_indexes[pipeline_uuid] = {}

Binary file not shown.

After

Width:  |  Height:  |  Size: 466 KiB

View File

@@ -4,17 +4,26 @@ metadata:
name: wechatpad
label:
en_US: WeChatPad
zh_CN: WeChatPad个人微信ipad
zh_Hans: WeChatPad个人微信ipad
zh_Hant: WeChatPad個人微信iPad
description:
en_US: WeChatPad Adapter
zh_CN: WeChatPad 适配器
zh_Hans: WeChatPad 适配器基于WeChatPad的个人微信解决方案请查看文档了解使用方式
zh_Hant: WeChatPad 適配器,基於 WeChatPad 的個人微信解決方案,請查看文件了解使用方式
icon: wechatpad.png
spec:
categories:
- china
help_links:
zh: https://link.langbot.app/zh/platforms/wechatpad
en: https://link.langbot.app/en/platforms/wechatpad
ja: https://link.langbot.app/ja/platforms/wechatpad
config:
- name: wechatpad_url
label:
en_US: WeChatPad ERL
zh_CN: WeChatPad URL
zh_Hant: WeChatPad URL
type: string
required: true
default: ""
@@ -22,6 +31,7 @@ spec:
label:
en_US: WeChatPad_Ws
zh_CN: WeChatPad_Ws
zh_Hant: WeChatPad_Ws
type: string
required: true
default: ""
@@ -29,6 +39,7 @@ spec:
label:
en_US: Admin_Key
zh_CN: 管理员密匙
zh_Hant: 管理員密鑰
type: string
required: true
default: ""
@@ -36,6 +47,7 @@ spec:
label:
en_US: Token
zh_CN: 令牌
zh_Hant: 令牌
type: string
required: true
default: ""
@@ -43,6 +55,7 @@ spec:
label:
en_US: wxid
zh_CN: wxid
zh_Hant: wxid
type: string
required: true
default: ""

View File

@@ -148,51 +148,54 @@ class WecomEventConverter(abstract_platform_adapter.AbstractEventConverter):
pass
if type(event) is platform_events.FriendMessage:
payload = {
'MsgType': 'text',
'Content': '',
'FromUserName': event.sender.id,
'ToUserName': bot_account_id,
'CreateTime': int(datetime.datetime.now().timestamp()),
'AgentID': event.sender.nickname,
}
wecom_event = WecomEvent.from_payload(payload=payload)
if not wecom_event:
raise ValueError('无法从 message_data 构造 WecomEvent 对象')
return wecom_event
return event.source_platform_object
@staticmethod
async def target2yiri(event: WecomEvent):
async def target2yiri(event: WecomEvent, bot: WecomClient = None):
"""
将 WecomEvent 转换为平台的 FriendMessage 对象。
Args:
event (WecomEvent): 企业微信事件。
bot (WecomClient): 企业微信客户端,用于获取用户信息。
Returns:
platform_events.FriendMessage: 转换后的 FriendMessage 对象。
"""
# Try to get the user's real name from the WeCom API
nickname = str(event.user_id)
if bot and event.user_id:
try:
user_info = await bot.get_user_info(event.user_id)
if user_info and user_info.get('name'):
nickname = user_info.get('name')
except Exception:
pass # Fall back to user_id as nickname
# 转换消息链
if event.type == 'text':
yiri_chain = await WecomMessageConverter.target2yiri(event.message, event.message_id)
friend = platform_entities.Friend(
id=f'u{event.user_id}',
nickname=str(event.agent_id),
nickname=nickname,
remark='',
)
return platform_events.FriendMessage(sender=friend, message_chain=yiri_chain, time=event.timestamp)
return platform_events.FriendMessage(
sender=friend, message_chain=yiri_chain, time=event.timestamp, source_platform_object=event
)
elif event.type == 'image':
friend = platform_entities.Friend(
id=f'u{event.user_id}',
nickname=str(event.agent_id),
nickname=nickname,
remark='',
)
yiri_chain = await WecomMessageConverter.target2yiri_image(picurl=event.picurl, message_id=event.message_id)
return platform_events.FriendMessage(sender=friend, message_chain=yiri_chain, time=event.timestamp)
return platform_events.FriendMessage(
sender=friend, message_chain=yiri_chain, time=event.timestamp, source_platform_object=event
)
class WecomAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
@@ -210,7 +213,6 @@ class WecomAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
'secret',
'token',
'EncodingAESKey',
'contacts_secret',
]
missing_keys = [key for key in required_keys if key not in config]
@@ -223,7 +225,7 @@ class WecomAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
secret=config['secret'],
token=config['token'],
EncodingAESKey=config['EncodingAESKey'],
contacts_secret=config['contacts_secret'],
contacts_secret=config.get('contacts_secret', ''), # Optional, kept for backward compatibility
logger=logger,
unified_mode=True,
api_base_url=config.get('api_base_url', 'https://qyapi.weixin.qq.com/cgi-bin'),
@@ -248,18 +250,17 @@ class WecomAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
):
Wecom_event = await WecomEventConverter.yiri2target(message_source, self.bot_account_id, self.bot)
content_list = await WecomMessageConverter.yiri2target(message, self.bot)
fixed_user_id = Wecom_event.user_id
# 删掉开头的u
fixed_user_id = fixed_user_id[1:]
# user_id is the original FromUserName from WecomEvent
user_id = Wecom_event.user_id
for content in content_list:
if content['type'] == 'text':
await self.bot.send_private_msg(fixed_user_id, Wecom_event.agent_id, content['content'])
await self.bot.send_private_msg(user_id, Wecom_event.agent_id, content['content'])
elif content['type'] == 'image':
await self.bot.send_image(fixed_user_id, Wecom_event.agent_id, content['media_id'])
await self.bot.send_image(user_id, Wecom_event.agent_id, content['media_id'])
elif content['type'] == 'voice':
await self.bot.send_voice(fixed_user_id, Wecom_event.agent_id, content['media_id'])
await self.bot.send_voice(user_id, Wecom_event.agent_id, content['media_id'])
elif content['type'] == 'file':
await self.bot.send_file(fixed_user_id, Wecom_event.agent_id, content['media_id'])
await self.bot.send_file(user_id, Wecom_event.agent_id, content['media_id'])
async def send_message(self, target_type: str, target_id: str, message: platform_message.MessageChain):
content_list = await WecomMessageConverter.yiri2target(message, self.bot)
@@ -287,7 +288,7 @@ class WecomAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
async def on_message(event: WecomEvent):
self.bot_account_id = event.receiver_id
try:
return await callback(await self.event_converter.target2yiri(event), self)
return await callback(await self.event_converter.target2yiri(event, self.bot), self)
except Exception:
await self.logger.error(f'Error in wecom callback: {traceback.format_exc()}')

View File

@@ -5,16 +5,38 @@ metadata:
label:
en_US: WeCom
zh_Hans: 企业微信
zh_Hant: 企業微信
description:
en_US: WeCom Adapter
zh_Hans: 企业微信适配器,请查看文档了解使用方式
zh_Hans: 企业微信内部机器人,请查看文档了解使用方式
zh_Hant: 企業微信內部機器人,請查看文件了解使用方式
icon: wecom.png
spec:
categories:
- popular
- china
help_links:
zh: https://link.langbot.app/zh/platforms/wecom
en: https://link.langbot.app/en/platforms/wecom
ja: https://link.langbot.app/ja/platforms/wecom
config:
- name: webhook_url
label:
en_US: Webhook Callback URL
zh_Hans: Webhook 回调地址
zh_Hant: Webhook 回調地址
description:
en_US: Copy this URL and paste it into your WeCom app's webhook configuration
zh_Hans: 复制此地址并粘贴到企业微信应用的 Webhook 配置中
zh_Hant: 複製此地址並貼到企業微信應用的 Webhook 設定中
type: webhook-url
required: false
default: ""
- name: corpid
label:
en_US: Corpid
zh_Hans: 企业ID
zh_Hant: 企業ID
type: string
required: true
default: ""
@@ -22,6 +44,7 @@ spec:
label:
en_US: Secret
zh_Hans: 密钥 (Secret)
zh_Hant: 密鑰 (Secret)
type: string
required: true
default: ""
@@ -29,6 +52,7 @@ spec:
label:
en_US: Token
zh_Hans: 令牌 (Token)
zh_Hant: 令牌 (Token)
type: string
required: true
default: ""
@@ -36,13 +60,7 @@ spec:
label:
en_US: EncodingAESKey
zh_Hans: 消息加解密密钥 (EncodingAESKey)
type: string
required: true
default: ""
- name: contacts_secret
label:
en_US: Contacts Secret
zh_Hans: 通讯录密钥
zh_Hant: 訊息加解密密鑰 (EncodingAESKey)
type: string
required: true
default: ""
@@ -50,9 +68,11 @@ spec:
label:
en_US: API Base URL
zh_Hans: API 基础 URL
zh_Hant: API 基礎 URL
description:
en_US: API Base URL, used for accessing the WeCom API. If you are deploying in an internal network environment and accessing the WeCom Customer Service API through a reverse proxy, please fill in this item according to the documentation.
zh_Hans: 可选,若您部署在内网环境并通过反向代理访问企业微信 API可根据文档填写此项
zh_Hant: 可選,若您部署在內網環境並透過反向代理存取企業微信 API可根據文件填寫此項
type: string
required: false
default: "https://qyapi.weixin.qq.com/cgi-bin"

View File

@@ -11,6 +11,7 @@ import langbot_plugin.api.entities.builtin.platform.entities as platform_entitie
from ..logger import EventLogger
from langbot.libs.wecom_ai_bot_api.wecombotevent import WecomBotEvent
from langbot.libs.wecom_ai_bot_api.api import WecomBotClient
from langbot.libs.wecom_ai_bot_api.ws_client import WecomBotWsClient
class WecomBotMessageConverter(abstract_platform_adapter.AbstractMessageConverter):
@@ -23,14 +24,18 @@ class WecomBotMessageConverter(abstract_platform_adapter.AbstractMessageConverte
return content
@staticmethod
async def target2yiri(event: WecomBotEvent):
async def target2yiri(event: WecomBotEvent, bot_name: str = ''):
yiri_msg_list = []
if event.type == 'group':
yiri_msg_list.append(platform_message.At(target=event.ai_bot_id))
yiri_msg_list.append(platform_message.Source(id=event.message_id, time=datetime.datetime.now()))
if event.content:
yiri_msg_list.append(platform_message.Plain(text=event.content))
content = event.content
if bot_name:
content = content.replace(f'@{bot_name}', '').strip()
yiri_msg_list.append(platform_message.Plain(text=content))
images = []
if event.images:
@@ -133,13 +138,15 @@ class WecomBotMessageConverter(abstract_platform_adapter.AbstractMessageConverte
class WecomBotEventConverter(abstract_platform_adapter.AbstractEventConverter):
def __init__(self, bot_name: str = ''):
self.bot_name = bot_name
@staticmethod
async def yiri2target(event: platform_events.MessageEvent):
return event.source_platform_object
@staticmethod
async def target2yiri(event: WecomBotEvent):
message_chain = await WecomBotMessageConverter.target2yiri(event)
async def target2yiri(self, event: WecomBotEvent):
message_chain = await WecomBotMessageConverter.target2yiri(event, bot_name=self.bot_name)
if event.type == 'single':
return platform_events.FriendMessage(
sender=platform_entities.Friend(
@@ -176,34 +183,53 @@ class WecomBotEventConverter(abstract_platform_adapter.AbstractEventConverter):
class WecomBotAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
bot: WecomBotClient
bot: typing.Union[WecomBotClient, WecomBotWsClient]
bot_account_id: str
message_converter: WecomBotMessageConverter = WecomBotMessageConverter()
event_converter: WecomBotEventConverter = WecomBotEventConverter()
event_converter: WecomBotEventConverter
config: dict
bot_uuid: str = None
_ws_mode: bool = False
bot_name: str = ''
listeners: dict = {}
def __init__(self, config: dict, logger: EventLogger):
required_keys = ['Token', 'EncodingAESKey', 'Corpid', 'BotId']
missing_keys = [key for key in required_keys if key not in config]
if missing_keys:
raise Exception(f'WecomBot 缺少配置项: {missing_keys}')
enable_webhook = config.get('enable-webhook', False)
bot_name = config.get('robot_name', '')
bot = WecomBotClient(
Token=config['Token'],
EnCodingAESKey=config['EncodingAESKey'],
Corpid=config['Corpid'],
logger=logger,
unified_mode=True,
)
bot_account_id = config['BotId']
if not enable_webhook:
bot = WecomBotWsClient(
bot_id=config['BotId'],
secret=config['Secret'],
logger=logger,
encoding_aes_key=config.get('EncodingAESKey', ''),
)
else:
# Webhook callback mode
required_keys = ['Token', 'EncodingAESKey', 'Corpid']
missing_keys = [key for key in required_keys if key not in config or not config[key]]
if missing_keys:
raise Exception(f'WecomBot webhook mode missing config: {missing_keys}')
bot = WecomBotClient(
Token=config['Token'],
EnCodingAESKey=config['EncodingAESKey'],
Corpid=config['Corpid'],
logger=logger,
unified_mode=True,
)
bot_account_id = config.get('BotId', '')
event_converter = WecomBotEventConverter(bot_name=bot_name)
super().__init__(
config=config,
logger=logger,
bot=bot,
bot_account_id=bot_account_id,
bot_name=bot_name,
event_converter=event_converter,
)
self.listeners = {}
async def reply_message(
self,
@@ -212,7 +238,17 @@ class WecomBotAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
quote_origin: bool = False,
):
content = await self.message_converter.yiri2target(message)
await self.bot.set_message(message_source.source_platform_object.message_id, content)
_ws_mode = not self.config.get('enable-webhook', False)
if _ws_mode:
event = message_source.source_platform_object
req_id = event.get('req_id', '')
if req_id:
await self.bot.reply_text(req_id, content)
else:
await self.bot.set_message(event.message_id, content)
else:
await self.bot.set_message(message_source.source_platform_object.message_id, content)
async def reply_message_chunk(
self,
@@ -222,44 +258,44 @@ class WecomBotAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
quote_origin: bool = False,
is_final: bool = False,
):
"""将流水线增量输出写入企业微信 stream 会话。
Args:
message_source: 流水线提供的原始消息事件。
bot_message: 当前片段对应的模型元信息(未使用)。
message: 需要回复的消息链。
quote_origin: 是否引用原消息(企业微信暂不支持)。
is_final: 标记当前片段是否为最终回复。
Returns:
dict: 包含 `stream` 键,标识写入是否成功。
Example:
在流水线 `reply_message_chunk` 调用中自动触发,无需手动调用。
"""
# 转换为纯文本(智能机器人当前协议仅支持文本流)
content = await self.message_converter.yiri2target(message)
msg_id = message_source.source_platform_object.message_id
_ws_mode = not self.config.get('enable-webhook', False)
# 将片段推送到 WecomBotClient 中的队列,返回值用于判断是否走降级逻辑
success = await self.bot.push_stream_chunk(msg_id, content, is_final=is_final)
if not success and is_final:
# 未命中流式队列时使用旧有 set_message 兜底
await self.bot.set_message(msg_id, content)
return {'stream': success}
if _ws_mode:
success = await self.bot.push_stream_chunk(msg_id, content, is_final=is_final)
if not success and is_final:
event = message_source.source_platform_object
req_id = event.get('req_id', '')
if req_id:
await self.bot.reply_text(req_id, content)
return {'stream': success}
else:
success = await self.bot.push_stream_chunk(msg_id, content, is_final=is_final)
if not success and is_final:
await self.bot.set_message(msg_id, content)
return {'stream': success}
async def is_stream_output_supported(self) -> bool:
"""智能机器人侧默认开启流式能力。
Returns:
bool: 恒定返回 True。
Example:
流水线执行阶段会调用此方法以确认是否启用流式。"""
return True
"""Whether streaming output is enabled for this bot instance."""
return self.config.get('enable-stream-reply', True)
async def send_message(self, target_type, target_id, message):
pass
_ws_mode = not self.config.get('enable-webhook', False)
if _ws_mode:
content = await self.message_converter.yiri2target(message)
await self.bot.send_message(target_id, content)
else:
pass
async def on_message(self, event: WecomBotEvent):
try:
lb_event = await self.event_converter.target2yiri(event)
if lb_event:
await self.listeners[type(lb_event)](lb_event, self)
except Exception:
await self.logger.error(f'Error in wecombot callback: {traceback.format_exc()}')
print(traceback.format_exc())
def register_listener(
self,
@@ -268,18 +304,16 @@ class WecomBotAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
[platform_events.Event, abstract_platform_adapter.AbstractMessagePlatformAdapter], None
],
):
async def on_message(event: WecomBotEvent):
try:
return await callback(await self.event_converter.target2yiri(event), self)
except Exception:
await self.logger.error(f'Error in wecombot callback: {traceback.format_exc()}')
print(traceback.format_exc())
self.listeners[event_type] = callback
try:
if event_type == platform_events.FriendMessage:
self.bot.on_message('single')(on_message)
self.bot.on_message('single')(self.on_message)
elif event_type == platform_events.GroupMessage:
self.bot.on_message('group')(on_message)
self.bot.on_message('group')(self.on_message)
elif event_type == platform_events.FeedbackEvent:
if hasattr(self.bot, 'on_feedback'):
self.bot.on_feedback()(self._on_feedback)
except Exception:
print(traceback.format_exc())
@@ -287,30 +321,68 @@ class WecomBotAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
"""设置 bot UUID用于生成 webhook URL"""
self.bot_uuid = bot_uuid
async def _on_feedback(self, **kwargs):
"""Handle feedback event from WeChat Work AI Bot SDK and dispatch as FeedbackEvent."""
try:
feedback_id = kwargs.get('feedback_id', '')
feedback_type = kwargs.get('feedback_type', 0)
feedback_content = kwargs.get('feedback_content', '') or None
inaccurate_reasons = kwargs.get('inaccurate_reasons', []) or None
session = kwargs.get('session')
session_id = None
user_id = None
message_id = None
stream_id = None
if session:
if session.chat_id:
session_id = f'group_{session.chat_id}'
elif session.user_id:
session_id = f'person_{session.user_id}'
user_id = session.user_id
message_id = session.msg_id
stream_id = session.stream_id
event = platform_events.FeedbackEvent(
feedback_id=feedback_id,
feedback_type=feedback_type,
feedback_content=feedback_content,
inaccurate_reasons=inaccurate_reasons,
user_id=user_id,
session_id=session_id,
message_id=message_id,
stream_id=stream_id,
source_platform_object=session,
)
if platform_events.FeedbackEvent in self.listeners:
await self.listeners[platform_events.FeedbackEvent](event, self)
except Exception:
await self.logger.error(f'Error in wecombot feedback callback: {traceback.format_exc()}')
async def handle_unified_webhook(self, bot_uuid: str, path: str, request):
"""处理统一 webhook 请求。
Args:
bot_uuid: Bot 的 UUID
path: 子路径(如果有的话)
request: Quart Request 对象
Returns:
响应数据
"""
_ws_mode = not self.config.get('enable-webhook', False)
if _ws_mode:
return None
return await self.bot.handle_unified_webhook(request)
async def run_async(self):
# 统一 webhook 模式下,不启动独立的 Quart 应用
# 保持运行但不启动独立端口
_ws_mode = not self.config.get('enable-webhook', False)
if _ws_mode:
await self.bot.connect()
else:
async def keep_alive():
while True:
await asyncio.sleep(1)
async def keep_alive():
while True:
await asyncio.sleep(1)
await keep_alive()
await keep_alive()
async def kill(self) -> bool:
_ws_mode = not self.config.get('enable-webhook', False)
if _ws_mode:
await self.bot.disconnect()
return True
return False
async def unregister_listener(

View File

@@ -5,41 +5,125 @@ metadata:
label:
en_US: WeComBot
zh_Hans: 企业微信智能机器人
zh_Hant: 企業微信智慧機器人
description:
en_US: WeComBot Adapter
zh_Hans: 企业微信智能机器人适配器,请查看文档了解使用方式
zh_Hans: 企业微信智能机器人,支持长连接和 Webhook 两种接入方式,请查看文档了解使用方式
zh_Hant: 企業微信智慧機器人,支援長連線和 Webhook 兩種接入方式,請查看文件了解使用方式
icon: wecombot.png
spec:
categories:
- china
help_links:
zh: https://link.langbot.app/zh/platforms/wecombot
en: https://link.langbot.app/en/platforms/wecombot
ja: https://link.langbot.app/ja/platforms/wecombot
config:
- name: BotId
label:
en_US: BotId
zh_Hans: 机器人ID (BotId)
zh_Hant: 機器人ID (BotId)
type: string
required: true
default: ""
- name: robot_name
label:
en_US: Robot Name
zh_Hans: 机器人名称
zh_Hant: 機器人名稱
type: string
required: true
default: ""
- name: enable-webhook
label:
en_US: Enable Webhook Mode
zh_Hans: 启用Webhook模式
zh_Hant: 啟用 Webhook 模式
description:
en_US: If enabled, the bot will use webhook mode to receive messages. Otherwise, it will use WS long connection mode
zh_Hans: 如果启用,机器人将使用 Webhook 模式接收消息。否则,将使用 WS 长连接模式
zh_Hant: 如果啟用,機器人將使用 Webhook 模式接收訊息。否則,將使用 WS 長連線模式
type: boolean
required: true
default: false
- name: webhook_url
label:
en_US: Webhook Callback URL
zh_Hans: Webhook 回调地址
zh_Hant: Webhook 回調地址
description:
en_US: Copy this URL and paste it into your WeComBot webhook configuration
zh_Hans: 复制此地址并粘贴到企业微信智能机器人的 Webhook 配置中
zh_Hant: 複製此地址並貼到企業微信智慧機器人的 Webhook 設定中
type: webhook-url
required: false
default: ""
show_if:
field: enable-webhook
operator: eq
value: true
- name: Secret
label:
en_US: Secret
zh_Hans: 机器人密钥 (Secret)
zh_Hant: 機器人密鑰 (Secret)
description:
en_US: Required for WebSocket long connection mode
zh_Hans: 使用 WS 长连接模式时必填
zh_Hant: 使用 WS 長連線模式時必填
type: string
required: false
default: ""
- name: Corpid
label:
en_US: Corpid
zh_Hans: 企业ID
zh_Hant: 企業ID
description:
en_US: Required for Webhook mode
zh_Hans: 使用 Webhook 模式时必填
zh_Hant: 使用 Webhook 模式時必填
type: string
required: true
required: false
default: ""
- name: Token
label:
en_US: Token
zh_Hans: 令牌 (Token)
zh_Hant: 令牌 (Token)
description:
en_US: Required for Webhook mode
zh_Hans: 使用 Webhook 模式时必填
zh_Hant: 使用 Webhook 模式時必填
type: string
required: true
required: false
default: ""
- name: EncodingAESKey
label:
en_US: EncodingAESKey
zh_Hans: 消息加解密密钥 (EncodingAESKey)
type: string
required: true
default: ""
- name: BotId
label:
en_US: BotId
zh_Hans: 机器人ID
zh_Hant: 訊息加解密密鑰 (EncodingAESKey)
description:
en_US: Required for Webhook mode. Optional for WebSocket mode (used for file decryption)
zh_Hans: 使用 Webhook 模式时必填。WebSocket 模式下可选(用于文件解密)
zh_Hant: 使用 Webhook 模式時必填。WebSocket 模式下可選(用於檔案解密)
type: string
required: false
default: ""
- name: enable-stream-reply
label:
en_US: Enable Stream Reply
zh_Hans: 启用流式回复
zh_Hant: 啟用串流回覆
description:
en_US: If enabled, the bot will use streaming mode to reply messages
zh_Hans: 如果启用,机器人将使用流式模式回复消息
zh_Hant: 如果啟用,機器人將使用串流模式回覆訊息
type: boolean
required: false
default: true
execution:
python:
path: ./wecombot.py
attr: WecomBotAdapter
attr: WecomBotAdapter

View File

@@ -81,22 +81,33 @@ class WecomEventConverter(abstract_platform_adapter.AbstractEventConverter):
return event.source_platform_object
@staticmethod
async def target2yiri(event: WecomCSEvent):
async def target2yiri(event: WecomCSEvent, bot: WecomCSClient = None):
"""
将 WecomEvent 转换为平台的 FriendMessage 对象。
Args:
event (WecomEvent): 企业微信客服事件。
bot (WecomCSClient): 企业微信客服客户端,用于获取用户信息。
Returns:
platform_events.FriendMessage: 转换后的 FriendMessage 对象。
"""
# Try to get customer nickname from WeChat API
nickname = str(event.user_id)
if bot and event.user_id:
try:
customer_info = await bot.get_customer_info(event.user_id)
if customer_info and customer_info.get('nickname'):
nickname = customer_info.get('nickname')
except Exception:
pass # Fall back to user_id as nickname
# 转换消息链
if event.type == 'text':
yiri_chain = await WecomMessageConverter.target2yiri(event.message, event.message_id)
friend = platform_entities.Friend(
id=f'u{event.user_id}',
nickname=str(event.user_id),
nickname=nickname,
remark='',
)
@@ -106,7 +117,7 @@ class WecomEventConverter(abstract_platform_adapter.AbstractEventConverter):
elif event.type == 'image':
friend = platform_entities.Friend(
id=f'u{event.user_id}',
nickname=str(event.user_id),
nickname=nickname,
remark='',
)
@@ -187,7 +198,7 @@ class WecomCSAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
async def on_message(event: WecomCSEvent):
self.bot_account_id = event.receiver_id
try:
return await callback(await self.event_converter.target2yiri(event), self)
return await callback(await self.event_converter.target2yiri(event, self.bot), self)
except Exception:
await self.logger.error(f'Error in wecomcs callback: {traceback.format_exc()}')

View File

@@ -5,16 +5,37 @@ metadata:
label:
en_US: WeComCustomerService
zh_Hans: 企业微信客服
zh_Hant: 企業微信客服
description:
en_US: WeComCSAdapter
zh_Hans: 企业微信客服适配器
zh_Hans: 企业微信对外客服机器人,需要公网地址以接收消息推送,请查看文档了解使用方式
zh_Hant: 企業微信對外客服機器人,需要公網地址以接收訊息推送,請查看文件了解使用方式
icon: wecom.png
spec:
categories:
- china
help_links:
zh: https://link.langbot.app/zh/platforms/wecomcs
en: https://link.langbot.app/en/platforms/wecomcs
ja: https://link.langbot.app/ja/platforms/wecomcs
config:
- name: webhook_url
label:
en_US: Webhook Callback URL
zh_Hans: Webhook 回调地址
zh_Hant: Webhook 回調地址
description:
en_US: Copy this URL and paste it into your WeCom Customer Service webhook configuration
zh_Hans: 复制此地址并粘贴到企业微信客服的 Webhook 配置中
zh_Hant: 複製此地址並貼到企業微信客服的 Webhook 設定中
type: webhook-url
required: false
default: ""
- name: corpid
label:
en_US: Corpid
zh_Hans: 企业ID
zh_Hant: 企業ID
type: string
required: true
default: ""
@@ -22,6 +43,7 @@ spec:
label:
en_US: Secret
zh_Hans: 密钥
zh_Hant: 密鑰
type: string
required: true
default: ""
@@ -29,6 +51,7 @@ spec:
label:
en_US: Token
zh_Hans: 令牌
zh_Hant: 令牌
type: string
required: true
default: ""
@@ -36,6 +59,7 @@ spec:
label:
en_US: EncodingAESKey
zh_Hans: 消息加解密密钥
zh_Hant: 訊息加解密密鑰
type: string
required: true
default: ""
@@ -43,9 +67,11 @@ spec:
label:
en_US: API Base URL
zh_Hans: API 基础 URL
zh_Hant: API 基礎 URL
description:
en_US: API Base URL, used for accessing the WeCom API. If you are deploying in an internal network environment and accessing the WeCom Customer Service API through a reverse proxy, please fill in this item according to the documentation.
zh_Hans: 可选,若您部署在内网环境并通过反向代理访问企业微信 API可根据文档修改此项
zh_Hant: 可選,若您部署在內網環境並透過反向代理存取企業微信 API可根據文件修改此項
type: string
required: false
default: "https://qyapi.weixin.qq.com/cgi-bin"

View File

@@ -2,12 +2,14 @@
from __future__ import annotations
import asyncio
import io
import time
import zipfile
from typing import Any
import typing
import os
import sys
import httpx
import traceback
import sqlalchemy
from async_lru import alru_cache
from langbot_plugin.api.entities.builtin.pipeline.query import provider_session
@@ -102,12 +104,6 @@ class PluginRuntimeConnector:
self.handler_task = asyncio.create_task(self.handler.run())
_ = await self.handler.ping()
self.ap.logger.info('Connected to plugin runtime.')
# Sync polymorphic component instances after connection
try:
await self.sync_polymorphic_component_instances()
except Exception as e:
traceback.print_exc()
self.ap.logger.error(f'Failed to sync polymorphic component instances: {e}')
await self.handler_task
task: asyncio.Task | None = None
@@ -199,6 +195,30 @@ class PluginRuntimeConnector:
return await self.handler.ping()
def _extract_deps_metadata(
self,
file_bytes: bytes,
task_context: taskmgr.TaskContext | None,
):
"""Extract dependency count from requirements.txt inside plugin zip."""
if task_context is None:
return
try:
with zipfile.ZipFile(io.BytesIO(file_bytes)) as zf:
for name in zf.namelist():
if name.endswith('requirements.txt'):
content = zf.read(name).decode('utf-8', errors='ignore')
deps = [
line.strip()
for line in content.splitlines()
if line.strip() and not line.strip().startswith('#')
]
task_context.metadata['deps_total'] = len(deps)
task_context.metadata['deps_list'] = deps
break
except Exception:
pass
async def install_plugin(
self,
install_source: PluginInstallSource,
@@ -208,23 +228,44 @@ class PluginRuntimeConnector:
if install_source == PluginInstallSource.LOCAL:
# transfer file before install
file_bytes = install_info['plugin_file']
self._extract_deps_metadata(file_bytes, task_context)
file_key = await self.handler.send_file(file_bytes, 'lbpkg')
install_info['plugin_file_key'] = file_key
del install_info['plugin_file']
self.ap.logger.info(f'Transfered file {file_key} to plugin runtime')
elif install_source == PluginInstallSource.GITHUB:
# download and transfer file
# download and transfer file with streaming progress
try:
async with httpx.AsyncClient(
trust_env=True,
follow_redirects=True,
timeout=20,
timeout=60,
) as client:
response = await client.get(
install_info['asset_url'],
)
response.raise_for_status()
file_bytes = response.content
async with client.stream('GET', install_info['asset_url']) as response:
response.raise_for_status()
total = int(response.headers.get('content-length', 0))
downloaded = 0
chunks: list[bytes] = []
start_time = time.time()
if task_context is not None:
task_context.set_current_action('downloading plugin package')
task_context.metadata['download_total'] = total
task_context.metadata['download_current'] = 0
task_context.metadata['download_speed'] = 0
async for chunk in response.aiter_bytes(chunk_size=8192):
chunks.append(chunk)
downloaded += len(chunk)
if task_context is not None:
elapsed = time.time() - start_time
task_context.metadata['download_current'] = downloaded
task_context.metadata['download_total'] = total
task_context.metadata['download_speed'] = downloaded / elapsed if elapsed > 0 else 0
file_bytes = b''.join(chunks)
self._extract_deps_metadata(file_bytes, task_context)
file_key = await self.handler.send_file(file_bytes, 'lbpkg')
install_info['plugin_file_key'] = file_key
self.ap.logger.info(f'Transfered file {file_key} to plugin runtime')
@@ -243,6 +284,11 @@ class PluginRuntimeConnector:
if task_context is not None:
task_context.trace(trace)
# Forward structured metadata from runtime
metadata = ret.get('metadata', None)
if metadata is not None and task_context is not None:
task_context.metadata.update(metadata)
async def upgrade_plugin(
self,
plugin_author: str,
@@ -463,30 +509,18 @@ class PluginRuntimeConnector:
yield cmd_ret
# KnowledgeRetriever methods
async def list_knowledge_retrievers(self, bound_plugins: list[str] | None = None) -> list[dict[str, Any]]:
"""List all available KnowledgeRetriever components."""
if not self.is_enable_plugin:
return []
retrievers_data = await self.handler.list_knowledge_retrievers(include_plugins=bound_plugins)
return retrievers_data
async def retrieve_knowledge(
self,
plugin_author: str,
plugin_name: str,
retriever_name: str,
instance_id: str,
retrieval_context: dict[str, Any],
) -> list[dict[str, Any]]:
"""Retrieve knowledge using a KnowledgeRetriever instance."""
) -> dict[str, Any]:
"""Retrieve knowledge using a KnowledgeEngine instance."""
if not self.is_enable_plugin:
return []
return {'results': []}
return await self.handler.retrieve_knowledge(
plugin_author, plugin_name, retriever_name, instance_id, retrieval_context
)
return await self.handler.retrieve_knowledge(plugin_author, plugin_name, retriever_name, retrieval_context)
def dispose(self):
# No need to consider the shutdown on Windows
@@ -500,41 +534,84 @@ class PluginRuntimeConnector:
self.heartbeat_task.cancel()
self.heartbeat_task = None
async def sync_polymorphic_component_instances(self) -> dict[str, Any]:
"""Sync polymorphic component instances with runtime.
@staticmethod
def _parse_plugin_id(plugin_id: str) -> tuple[str, str]:
"""Parse a plugin ID string into (author, name).
This collects all external knowledge bases from database and sends to runtime
to ensure instance integrity across restarts.
Args:
plugin_id: Plugin ID in 'author/name' format.
Returns:
Tuple of (plugin_author, plugin_name).
Raises:
ValueError: If plugin_id is not in the expected 'author/name' format.
"""
if '/' not in plugin_id:
raise ValueError(
f"Invalid plugin_id format: '{plugin_id}'. Expected 'author/name' format (e.g. 'langbot/rag-engine')."
)
return plugin_id.split('/', 1)
async def call_rag_ingest(self, plugin_id: str, context_data: dict[str, Any]) -> dict[str, Any]:
"""Call plugin to ingest document.
Args:
plugin_id: Target plugin ID (author/name).
context_data: IngestionContext data.
"""
plugin_author, plugin_name = self._parse_plugin_id(plugin_id)
return await self.handler.rag_ingest_document(plugin_author, plugin_name, context_data)
async def call_rag_delete_document(self, plugin_id: str, document_id: str, kb_id: str) -> bool:
plugin_author, plugin_name = self._parse_plugin_id(plugin_id)
return await self.handler.rag_delete_document(plugin_author, plugin_name, document_id, kb_id)
async def get_rag_creation_schema(self, plugin_id: str) -> dict[str, Any]:
plugin_author, plugin_name = self._parse_plugin_id(plugin_id)
return await self.handler.get_rag_creation_schema(plugin_author, plugin_name)
async def get_rag_retrieval_schema(self, plugin_id: str) -> dict[str, Any]:
plugin_author, plugin_name = self._parse_plugin_id(plugin_id)
return await self.handler.get_rag_retrieval_schema(plugin_author, plugin_name)
async def rag_on_kb_create(self, plugin_id: str, kb_id: str, config: dict[str, Any]) -> dict[str, Any]:
"""Notify plugin about KB creation."""
plugin_author, plugin_name = self._parse_plugin_id(plugin_id)
return await self.handler.rag_on_kb_create(plugin_author, plugin_name, kb_id, config)
async def rag_on_kb_delete(self, plugin_id: str, kb_id: str) -> dict[str, Any]:
"""Notify plugin about KB deletion."""
plugin_author, plugin_name = self._parse_plugin_id(plugin_id)
return await self.handler.rag_on_kb_delete(plugin_author, plugin_name, kb_id)
async def call_rag_retrieve(self, plugin_id: str, retrieval_context: dict[str, Any]) -> dict[str, Any]:
"""Call plugin to retrieve knowledge.
Args:
plugin_id: Target plugin ID (author/name).
retrieval_context: RetrievalContext data.
"""
plugin_author, plugin_name = self._parse_plugin_id(plugin_id)
return await self.handler.retrieve_knowledge(plugin_author, plugin_name, '', retrieval_context)
async def list_knowledge_engines(self) -> list[dict[str, Any]]:
"""List all available Knowledge Engines from plugins.
Returns a list of Knowledge Engines with their capabilities and configuration schemas.
"""
if not self.is_enable_plugin:
return {}
return []
# ===== external knowledge bases =====
return await self.handler.list_knowledge_engines()
external_kbs = await self.ap.external_kb_service.get_external_knowledge_bases()
async def list_parsers(self) -> list[dict[str, Any]]:
"""List all available parsers from plugins."""
if not self.is_enable_plugin:
return []
return await self.handler.list_parsers()
# Build required_instances list
required_instances = []
for kb in external_kbs:
required_instances.append(
{
'instance_id': kb['uuid'],
'plugin_author': kb['plugin_author'],
'plugin_name': kb['plugin_name'],
'component_kind': 'KnowledgeRetriever',
'component_name': kb['retriever_name'],
'config': kb['retriever_config'],
}
)
self.ap.logger.info(f'Syncing {len(required_instances)} polymorphic component instances to runtime')
# Send to runtime
sync_result = await self.handler.sync_polymorphic_component_instances(required_instances)
self.ap.logger.info(
f'Sync complete: {len(sync_result.get("success_instances", []))} succeeded, '
f'{len(sync_result.get("failed_instances", []))} failed'
)
return sync_result
async def call_parser(self, plugin_id: str, context_data: dict[str, Any], file_bytes: bytes) -> dict[str, Any]:
"""Call plugin to parse a document."""
plugin_author, plugin_name = self._parse_plugin_id(plugin_id)
return await self.handler.parse_document(plugin_author, plugin_name, context_data, file_bytes)

View File

@@ -26,6 +26,20 @@ from ..core import app
from ..utils import constants
def _make_rag_error_response(error: Exception, error_type: str, **extra_context) -> handler.ActionResponse:
"""Create a clean error response for RAG operations.
Args:
error: The caught exception.
error_type: A category string like 'EmbeddingError', 'VectorStoreError'.
**extra_context: Additional context fields for the error message.
"""
context_parts = [f'{k}={v}' for k, v in extra_context.items()]
context_str = f' [{", ".join(context_parts)}]' if context_parts else ''
message = f'[{error_type}/{type(error).__name__}]{context_str} {str(error)}'
return handler.ActionResponse.error(message=message)
class RuntimeConnectionHandler(handler.Handler):
"""Runtime connection handler"""
@@ -300,11 +314,11 @@ class RuntimeConnectionHandler(handler.Handler):
@self.action(PluginToRuntimeAction.GET_LLM_MODELS)
async def get_llm_models(data: dict[str, Any]) -> handler.ActionResponse:
"""Get llm models"""
"""Get llm models, returns list of UUID strings"""
llm_models = await self.ap.llm_model_service.get_llm_models(include_secret=False)
return handler.ActionResponse.success(
data={
'llm_models': llm_models,
'llm_models': [m['uuid'] for m in llm_models],
},
)
@@ -323,7 +337,14 @@ class RuntimeConnectionHandler(handler.Handler):
)
messages_obj = [provider_message.Message.model_validate(message) for message in messages]
funcs_obj = [resource_tool.LLMTool.model_validate(func) for func in funcs]
# The func field is excluded during model_dump() in plugin side (marked as exclude=True),
# but it's a required field for LLMTool validation. We need to provide a placeholder
# function when reconstructing the LLMTool objects from serialized data.
async def _placeholder_func(**kwargs):
pass
funcs_obj = [resource_tool.LLMTool.model_validate({**func, 'func': _placeholder_func}) for func in funcs]
result = await llm_model.provider.invoke_llm(
query=None,
@@ -439,7 +460,7 @@ class RuntimeConnectionHandler(handler.Handler):
},
)
@self.action(RuntimeToLangBotAction.GET_CONFIG_FILE)
@self.action(PluginToRuntimeAction.GET_CONFIG_FILE)
async def get_config_file(data: dict[str, Any]) -> handler.ActionResponse:
"""Get a config file by file key"""
file_key = data['file_key']
@@ -458,6 +479,282 @@ class RuntimeConnectionHandler(handler.Handler):
message=f'Failed to load config file {file_key}: {e}',
)
# ================= RAG Capability Handlers =================
@self.action(PluginToRuntimeAction.INVOKE_EMBEDDING)
async def invoke_embedding(data: dict[str, Any]) -> handler.ActionResponse:
embedding_model_uuid = data['embedding_model_uuid']
texts = data['texts']
embedding_model = await self.ap.model_mgr.get_embedding_model_by_uuid(embedding_model_uuid)
if embedding_model is None:
return handler.ActionResponse.error(
message=f'Embedding model with embedding_model_uuid {embedding_model_uuid} not found',
)
try:
vectors = await embedding_model.provider.invoke_embedding(embedding_model, texts)
return handler.ActionResponse.success(data={'vectors': vectors})
except Exception as e:
return _make_rag_error_response(e, 'EmbeddingError', embedding_model_uuid=embedding_model_uuid)
@self.action(PluginToRuntimeAction.VECTOR_UPSERT)
async def vector_upsert(data: dict[str, Any]) -> handler.ActionResponse:
collection_id = data['collection_id']
vectors = data['vectors']
ids = data['ids']
metadata = data.get('metadata')
documents = data.get('documents')
if len(vectors) != len(ids):
return handler.ActionResponse.error(message='vectors and ids must have same length')
if metadata and len(metadata) != len(vectors):
return handler.ActionResponse.error(message='metadata must match vectors length')
if documents and len(documents) != len(vectors):
return handler.ActionResponse.error(message='documents must match vectors length')
try:
await self.ap.rag_runtime_service.vector_upsert(
collection_id,
vectors,
ids,
metadata,
documents,
)
return handler.ActionResponse.success(data={})
except Exception as e:
return _make_rag_error_response(e, 'VectorStoreError', collection_id=collection_id)
@self.action(PluginToRuntimeAction.VECTOR_SEARCH)
async def vector_search(data: dict[str, Any]) -> handler.ActionResponse:
collection_id = data['collection_id']
query_vector = data['query_vector']
top_k = data['top_k']
filters = data.get('filters')
search_type = data.get('search_type', 'vector')
query_text = data.get('query_text', '')
vector_weight = data.get('vector_weight')
try:
results = await self.ap.rag_runtime_service.vector_search(
collection_id,
query_vector,
top_k,
filters,
search_type,
query_text,
vector_weight=vector_weight,
)
return handler.ActionResponse.success(data={'results': results})
except Exception as e:
return _make_rag_error_response(e, 'VectorStoreError', collection_id=collection_id)
@self.action(PluginToRuntimeAction.VECTOR_DELETE)
async def vector_delete(data: dict[str, Any]) -> handler.ActionResponse:
collection_id = data['collection_id']
file_ids = data.get('file_ids')
filters = data.get('filters')
try:
count = await self.ap.rag_runtime_service.vector_delete(collection_id, file_ids, filters)
return handler.ActionResponse.success(data={'count': count})
except Exception as e:
return _make_rag_error_response(e, 'VectorStoreError', collection_id=collection_id)
@self.action(PluginToRuntimeAction.VECTOR_LIST)
async def vector_list(data: dict[str, Any]) -> handler.ActionResponse:
collection_id = data['collection_id']
filters = data.get('filters')
limit = data.get('limit', 20)
offset = data.get('offset', 0)
try:
items, total = await self.ap.rag_runtime_service.vector_list(collection_id, filters, limit, offset)
return handler.ActionResponse.success(data={'items': items, 'total': total})
except Exception as e:
return _make_rag_error_response(e, 'VectorStoreError', collection_id=collection_id)
@self.action(PluginToRuntimeAction.GET_KNOWLEDEGE_FILE_STREAM)
async def get_knowledge_file_stream(data: dict[str, Any]) -> handler.ActionResponse:
storage_path = data['storage_path']
try:
content_bytes = await self.ap.rag_runtime_service.get_file_stream(storage_path)
file_key = await self.send_file(content_bytes, '')
return handler.ActionResponse.success(data={'file_key': file_key})
except Exception as e:
return _make_rag_error_response(e, 'FileServiceError', storage_path=storage_path)
@self.action(PluginToRuntimeAction.LIST_PARSERS)
async def list_parsers(data: dict[str, Any]) -> handler.ActionResponse:
"""Plugin requests host to list available parser plugins."""
mime_type = data.get('mime_type')
try:
parsers = await self.ap.knowledge_service.list_parsers(mime_type)
return handler.ActionResponse.success(data={'parsers': parsers})
except Exception as e:
return _make_rag_error_response(e, 'ParserDiscoveryError', mime_type=mime_type)
@self.action(PluginToRuntimeAction.INVOKE_PARSER)
async def invoke_parser(data: dict[str, Any]) -> handler.ActionResponse:
"""Plugin requests host to invoke a parser plugin."""
plugin_author = data['plugin_author']
plugin_name = data['plugin_name']
storage_path = data['storage_path']
mime_type = data.get('mime_type', 'application/octet-stream')
filename = data.get('filename', '')
metadata = data.get('metadata', {})
try:
# Read file from storage
file_bytes = await self.ap.rag_runtime_service.get_file_stream(storage_path)
context_data = {
'mime_type': mime_type,
'filename': filename,
'metadata': metadata,
}
result = await self.ap.plugin_connector.call_parser(
f'{plugin_author}/{plugin_name}', context_data, file_bytes
)
return handler.ActionResponse.success(data=result)
except Exception as e:
return _make_rag_error_response(e, 'ParserError')
# ================= Knowledge Base Query APIs =================
@self.action(PluginToRuntimeAction.LIST_KNOWLEDGE_BASES)
async def list_knowledge_bases(data: dict[str, Any]) -> handler.ActionResponse:
"""List all knowledge bases available in the LangBot instance (unrestricted)."""
knowledge_bases = []
for kb_uuid, kb in self.ap.rag_mgr.knowledge_bases.items():
knowledge_bases.append(
{
'uuid': kb.get_uuid(),
'name': kb.get_name(),
'description': kb.knowledge_base_entity.description or '',
}
)
return handler.ActionResponse.success(data={'knowledge_bases': knowledge_bases})
@self.action(PluginToRuntimeAction.RETRIEVE_KNOWLEDGE)
async def retrieve_knowledge(data: dict[str, Any]) -> handler.ActionResponse:
"""Retrieve documents from any knowledge base (unrestricted)."""
kb_id = data['kb_id']
query_text = data['query_text']
top_k = data.get('top_k', 5)
filters = data.get('filters', {})
kb = await self.ap.rag_mgr.get_knowledge_base_by_uuid(kb_id)
if not kb:
return handler.ActionResponse.error(
message=f'Knowledge base {kb_id} not found',
)
try:
entries = await kb.retrieve(
query_text,
settings={
'top_k': top_k,
'filters': filters,
},
)
results = [entry.model_dump(mode='json') for entry in entries]
return handler.ActionResponse.success(data={'results': results})
except Exception as e:
return _make_rag_error_response(e, 'RetrievalError', kb_id=kb_id)
@self.action(PluginToRuntimeAction.LIST_PIPELINE_KNOWLEDGE_BASES)
async def list_pipeline_knowledge_bases(data: dict[str, Any]) -> handler.ActionResponse:
"""List knowledge bases configured for the current query's pipeline."""
query_id = data['query_id']
if query_id not in self.ap.query_pool.cached_queries:
return handler.ActionResponse.error(
message=f'Query with query_id {query_id} not found',
)
query = self.ap.query_pool.cached_queries[query_id]
kb_uuids = []
if query.pipeline_config:
local_agent_config = query.pipeline_config.get('ai', {}).get('local-agent', {})
kb_uuids = local_agent_config.get('knowledge-bases', [])
# Backward compatibility
if not kb_uuids:
old_kb_uuid = local_agent_config.get('knowledge-base', '')
if old_kb_uuid and old_kb_uuid != '__none__':
kb_uuids = [old_kb_uuid]
knowledge_bases = []
for kb_uuid in kb_uuids:
kb = await self.ap.rag_mgr.get_knowledge_base_by_uuid(kb_uuid)
if kb:
knowledge_bases.append(
{
'uuid': kb.get_uuid(),
'name': kb.get_name(),
'description': kb.knowledge_base_entity.description or '',
}
)
return handler.ActionResponse.success(data={'knowledge_bases': knowledge_bases})
@self.action(PluginToRuntimeAction.RETRIEVE_KNOWLEDGE_BASE)
async def retrieve_knowledge_base(data: dict[str, Any]) -> handler.ActionResponse:
"""Retrieve documents from a knowledge base within the pipeline's scope."""
query_id = data['query_id']
kb_id = data['kb_id']
query_text = data['query_text']
top_k = data.get('top_k', 5)
filters = data.get('filters', {})
if query_id not in self.ap.query_pool.cached_queries:
return handler.ActionResponse.error(
message=f'Query with query_id {query_id} not found',
)
query = self.ap.query_pool.cached_queries[query_id]
# Validate kb_id is in pipeline's allowed list
allowed_kb_uuids = []
if query.pipeline_config:
local_agent_config = query.pipeline_config.get('ai', {}).get('local-agent', {})
allowed_kb_uuids = local_agent_config.get('knowledge-bases', [])
if not allowed_kb_uuids:
old_kb_uuid = local_agent_config.get('knowledge-base', '')
if old_kb_uuid and old_kb_uuid != '__none__':
allowed_kb_uuids = [old_kb_uuid]
if kb_id not in allowed_kb_uuids:
return handler.ActionResponse.error(
message=f'Knowledge base {kb_id} is not configured for this pipeline',
)
kb = await self.ap.rag_mgr.get_knowledge_base_by_uuid(kb_id)
if not kb:
return handler.ActionResponse.error(
message=f'Knowledge base {kb_id} not found',
)
try:
session_name = f'{query.session.launcher_type.value}_{query.session.launcher_id}'
entries = await kb.retrieve(
query_text,
settings={
'top_k': top_k,
'filters': filters,
'session_name': session_name,
'bot_uuid': query.bot_uuid or '',
'sender_id': str(query.sender_id),
},
)
results = [entry.model_dump(mode='json') for entry in entries]
return handler.ActionResponse.success(data={'results': results})
except Exception as e:
return _make_rag_error_response(e, 'RetrievalError', kb_id=kb_id)
@self.action(CommonAction.PING)
async def ping(data: dict[str, Any]) -> handler.ActionResponse:
"""Ping"""
return handler.ActionResponse.success(
data={
'pong': 'pong',
},
)
async def ping(self) -> dict[str, Any]:
"""Ping the runtime"""
return await self.call_action(
@@ -717,26 +1014,13 @@ class RuntimeConnectionHandler(handler.Handler):
async for ret in gen:
yield ret
# KnowledgeRetriever methods
async def list_knowledge_retrievers(self, include_plugins: list[str] | None = None) -> list[dict[str, Any]]:
"""List knowledge retrievers"""
result = await self.call_action(
LangBotToRuntimeAction.LIST_KNOWLEDGE_RETRIEVERS,
{
'include_plugins': include_plugins,
},
timeout=10,
)
return result['retrievers']
async def retrieve_knowledge(
self,
plugin_author: str,
plugin_name: str,
retriever_name: str,
instance_id: str,
retrieval_context: dict[str, Any],
) -> list[dict[str, Any]]:
) -> dict[str, Any]:
"""Retrieve knowledge"""
result = await self.call_action(
LangBotToRuntimeAction.RETRIEVE_KNOWLEDGE,
@@ -744,22 +1028,10 @@ class RuntimeConnectionHandler(handler.Handler):
'plugin_author': plugin_author,
'plugin_name': plugin_name,
'retriever_name': retriever_name,
'instance_id': instance_id,
'retrieval_context': retrieval_context,
},
timeout=30,
)
return result['retrieval_results']
async def sync_polymorphic_component_instances(self, required_instances: list[dict[str, Any]]) -> dict[str, Any]:
"""Sync polymorphic component instances with runtime"""
result = await self.call_action(
LangBotToRuntimeAction.SYNC_POLYMORPHIC_COMPONENT_INSTANCES,
{
'required_instances': required_instances,
},
timeout=30,
)
return result
async def get_debug_info(self) -> dict[str, Any]:
@@ -770,3 +1042,91 @@ class RuntimeConnectionHandler(handler.Handler):
timeout=10,
)
return result
# ================= RAG Capability Callers (LangBot -> Runtime) =================
async def rag_ingest_document(
self, plugin_author: str, plugin_name: str, context_data: dict[str, Any]
) -> dict[str, Any]:
"""Send INGEST_DOCUMENT action to runtime."""
result = await self.call_action(
LangBotToRuntimeAction.RAG_INGEST_DOCUMENT,
{'plugin_author': plugin_author, 'plugin_name': plugin_name, 'context': context_data},
timeout=1200, # Ingestion can be slow for large documents
)
return result
async def rag_delete_document(self, plugin_author: str, plugin_name: str, document_id: str, kb_id: str) -> bool:
result = await self.call_action(
LangBotToRuntimeAction.RAG_DELETE_DOCUMENT,
{'plugin_author': plugin_author, 'plugin_name': plugin_name, 'document_id': document_id, 'kb_id': kb_id},
timeout=30,
)
return result.get('success', False)
async def rag_on_kb_create(
self, plugin_author: str, plugin_name: str, kb_id: str, config: dict[str, Any]
) -> dict[str, Any]:
"""Notify plugin about KB creation."""
result = await self.call_action(
LangBotToRuntimeAction.RAG_ON_KB_CREATE,
{'plugin_author': plugin_author, 'plugin_name': plugin_name, 'kb_id': kb_id, 'config': config},
timeout=30,
)
return result
async def rag_on_kb_delete(self, plugin_author: str, plugin_name: str, kb_id: str) -> dict[str, Any]:
"""Notify plugin about KB deletion."""
result = await self.call_action(
LangBotToRuntimeAction.RAG_ON_KB_DELETE,
{'plugin_author': plugin_author, 'plugin_name': plugin_name, 'kb_id': kb_id},
timeout=30,
)
return result
async def get_rag_creation_schema(self, plugin_author: str, plugin_name: str) -> dict[str, Any]:
return await self.call_action(
LangBotToRuntimeAction.GET_RAG_CREATION_SETTINGS_SCHEMA,
{'plugin_author': plugin_author, 'plugin_name': plugin_name},
timeout=10,
)
async def get_rag_retrieval_schema(self, plugin_author: str, plugin_name: str) -> dict[str, Any]:
return await self.call_action(
LangBotToRuntimeAction.GET_RAG_RETRIEVAL_SETTINGS_SCHEMA,
{'plugin_author': plugin_author, 'plugin_name': plugin_name},
timeout=10,
)
async def list_knowledge_engines(self) -> list[dict[str, Any]]:
"""List all available Knowledge Engines from plugins."""
result = await self.call_action(LangBotToRuntimeAction.LIST_KNOWLEDGE_ENGINES, {}, timeout=60)
return result.get('engines', [])
# ================= Parser Capability Callers (LangBot -> Runtime) =================
async def list_parsers(self) -> list[dict[str, Any]]:
"""List all available parsers from plugins."""
result = await self.call_action(LangBotToRuntimeAction.LIST_PARSERS, {}, timeout=60)
return result.get('parsers', [])
async def parse_document(
self, plugin_author: str, plugin_name: str, context_data: dict[str, Any], file_bytes: bytes
) -> dict[str, Any]:
"""Send PARSE_DOCUMENT action to runtime.
Sends file content via chunked FILE_CHUNK transfer, then invokes
the PARSE_DOCUMENT action with a file_key reference.
"""
# Send file to runtime via chunked transfer
file_key = await self.send_file(file_bytes, '')
# Include file_key in context_data for the runtime to read
context_data['file_key'] = file_key
result = await self.call_action(
LangBotToRuntimeAction.PARSE_DOCUMENT,
{'plugin_author': plugin_author, 'plugin_name': plugin_name, 'context': context_data},
timeout=300,
)
return result

View File

@@ -288,10 +288,10 @@ class AnthropicMessages(requester.ProviderAPIRequester):
think_started = False
think_ended = False
finish_reason = False
content = ''
tool_name = ''
tool_id = ''
async for chunk in await self.client.messages.create(**args):
content = ''
tool_call = {'id': None, 'function': {'name': None, 'arguments': None}, 'type': 'function'}
if isinstance(
chunk, anthropic.types.raw_content_block_start_event.RawContentBlockStartEvent

View File

@@ -72,6 +72,28 @@ class DifyServiceAPIRunner(runner.RequestRunner):
content = f'<think>\n{thinking_content}\n</think>\n{content}'.strip()
return content, thinking_content
def _extract_dify_text_output(self, value: typing.Any) -> str:
"""Extract text content from Dify output payload."""
if value is None:
return ''
if isinstance(value, dict):
content = value.get('content')
if isinstance(content, str):
return content
return json.dumps(value, ensure_ascii=False)
if isinstance(value, str):
text = value.strip()
if not text:
return ''
try:
parsed = json.loads(text)
except json.JSONDecodeError:
return value
if isinstance(parsed, dict) and isinstance(parsed.get('content'), str):
return parsed['content']
return value
return str(value)
async def _preprocess_user_message(self, query: pipeline_query.Query) -> tuple[str, list[dict]]:
"""预处理用户消息,提取纯文本,并将图片/文件上传到 Dify 服务
@@ -192,7 +214,8 @@ class DifyServiceAPIRunner(runner.RequestRunner):
if mode == 'workflow':
if chunk['event'] == 'node_finished':
if chunk['data']['node_type'] == 'answer':
content, _ = self._process_thinking_content(chunk['data']['outputs']['answer'])
answer = self._extract_dify_text_output(chunk['data']['outputs'].get('answer'))
content, _ = self._process_thinking_content(answer)
yield provider_message.Message(
role='assistant',
@@ -405,6 +428,7 @@ class DifyServiceAPIRunner(runner.RequestRunner):
for f in upload_files
]
mode = 'basic'
basic_mode_pending_chunk = ''
inputs = {}
@@ -417,6 +441,7 @@ class DifyServiceAPIRunner(runner.RequestRunner):
is_final = False
think_start = False
think_end = False
yielded_final = False
remove_think = self.pipeline_config['output'].get('misc', '').get('remove-think')
@@ -430,11 +455,12 @@ class DifyServiceAPIRunner(runner.RequestRunner):
):
self.ap.logger.debug('dify-chat-chunk: ' + str(chunk))
# if chunk['event'] == 'workflow_started':
# mode = 'workflow'
# if mode == 'workflow':
# elif mode == 'basic':
# 因为都只是返回的 message也没有工具调用什么的暂时不分类
if chunk['event'] == 'workflow_started':
mode = 'workflow'
elif chunk['event'] in ('node_started', 'node_finished', 'workflow_finished'):
# Some Dify deployments may omit workflow_started in streamed chunks.
mode = 'workflow'
if chunk['event'] == 'message':
message_idx += 1
if remove_think:
@@ -457,14 +483,30 @@ class DifyServiceAPIRunner(runner.RequestRunner):
if chunk['event'] == 'message_end':
is_final = True
elif chunk['event'] == 'workflow_finished':
is_final = True
if chunk['data'].get('error'):
raise errors.DifyAPIError(chunk['data']['error'])
if is_final or message_idx % 8 == 0:
if mode == 'workflow' and chunk['event'] == 'node_finished':
if chunk['data'].get('node_type') == 'answer':
answer = self._extract_dify_text_output(chunk['data'].get('outputs', {}).get('answer'))
if answer:
basic_mode_pending_chunk = answer
if (
not yielded_final
and (is_final or message_idx % 8 == 0)
and (basic_mode_pending_chunk != '' or is_final)
):
# content, _ = self._process_thinking_content(basic_mode_pending_chunk)
yield provider_message.MessageChunk(
role='assistant',
content=basic_mode_pending_chunk,
is_final=is_final,
)
if is_final:
yielded_final = True
if chunk is None:
raise errors.DifyAPIError('Dify API 没有返回任何响应请检查网络连接和API配置')

View File

@@ -4,6 +4,7 @@ import json
import copy
import typing
from .. import runner
from ..modelmgr import requester as modelmgr_requester
import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
import langbot_plugin.api.entities.builtin.provider.message as provider_message
import langbot_plugin.api.entities.builtin.rag.context as rag_context
@@ -26,29 +27,114 @@ Respond in the same language as the user's input.
@runner.runner_class('local-agent')
class LocalAgentRunner(runner.RequestRunner):
"""本地Agent请求运行器"""
"""Local agent request runner"""
class ToolCallTracker:
"""工具调用追踪器"""
async def _get_model_candidates(
self,
query: pipeline_query.Query,
) -> list[modelmgr_requester.RuntimeLLMModel]:
"""Build ordered list of models to try: primary model + fallback models."""
candidates = []
def __init__(self):
self.active_calls: dict[str, dict] = {}
self.completed_calls: list[provider_message.ToolCall] = []
# Primary model
if query.use_llm_model_uuid:
try:
primary = await self.ap.model_mgr.get_model_by_uuid(query.use_llm_model_uuid)
candidates.append(primary)
except ValueError:
self.ap.logger.warning(f'Primary model {query.use_llm_model_uuid} not found')
# Fallback models
fallback_uuids = (query.variables or {}).get('_fallback_model_uuids', [])
for fb_uuid in fallback_uuids:
try:
fb_model = await self.ap.model_mgr.get_model_by_uuid(fb_uuid)
candidates.append(fb_model)
except ValueError:
self.ap.logger.warning(f'Fallback model {fb_uuid} not found, skipping')
return candidates
async def _invoke_with_fallback(
self,
query: pipeline_query.Query,
candidates: list[modelmgr_requester.RuntimeLLMModel],
messages: list,
funcs: list,
remove_think: bool,
) -> tuple[provider_message.Message, modelmgr_requester.RuntimeLLMModel]:
"""Try non-streaming invocation with sequential fallback. Returns (message, model_used)."""
last_error = None
for model in candidates:
try:
msg = await model.provider.invoke_llm(
query,
model,
messages,
funcs if model.model_entity.abilities.__contains__('func_call') else [],
extra_args=model.model_entity.extra_args,
remove_think=remove_think,
)
return msg, model
except Exception as e:
last_error = e
self.ap.logger.warning(f'Model {model.model_entity.name} failed: {e}, trying next fallback...')
raise last_error or RuntimeError('No model candidates available')
async def _invoke_stream_with_fallback(
self,
query: pipeline_query.Query,
candidates: list[modelmgr_requester.RuntimeLLMModel],
messages: list,
funcs: list,
remove_think: bool,
) -> tuple[typing.AsyncGenerator, modelmgr_requester.RuntimeLLMModel]:
"""Try streaming invocation with sequential fallback. Returns (stream_generator, model_used).
Fallback is only possible before any chunks have been yielded to the client.
Once streaming starts, the model is committed.
"""
last_error = None
for model in candidates:
try:
stream = model.provider.invoke_llm_stream(
query,
model,
messages,
funcs if model.model_entity.abilities.__contains__('func_call') else [],
extra_args=model.model_entity.extra_args,
remove_think=remove_think,
)
# Attempt to get the first chunk to verify the stream works
first_chunk = await stream.__anext__()
async def _chain_stream(first, rest):
yield first
async for chunk in rest:
yield chunk
return _chain_stream(first_chunk, stream), model
except StopAsyncIteration:
# Empty stream — treat as success (model returned nothing)
async def _empty_stream():
return
yield # make it a generator
return _empty_stream(), model
except Exception as e:
last_error = e
self.ap.logger.warning(f'Model {model.model_entity.name} stream failed: {e}, trying next fallback...')
raise last_error or RuntimeError('No model candidates available')
async def run(
self, query: pipeline_query.Query
) -> typing.AsyncGenerator[provider_message.Message | provider_message.MessageChunk, None]:
"""运行请求"""
"""Run request"""
pending_tool_calls = []
# Get knowledge bases list (new field)
kb_uuids = query.pipeline_config['ai']['local-agent'].get('knowledge-bases', [])
# Fallback to old field for backward compatibility
if not kb_uuids:
old_kb_uuid = query.pipeline_config['ai']['local-agent'].get('knowledge-base', '')
if old_kb_uuid and old_kb_uuid != '__none__':
kb_uuids = [old_kb_uuid]
# Get knowledge bases list from query variables (set by PreProcessor,
# may have been modified by plugins during PromptPreProcessing)
kb_uuids = query.variables.get('_knowledge_base_uuids', [])
user_message = copy.deepcopy(query.user_message)
@@ -74,15 +160,14 @@ class LocalAgentRunner(runner.RequestRunner):
self.ap.logger.warning(f'Knowledge base {kb_uuid} not found, skipping')
continue
# Get top_k based on KB type
if kb.get_type() == 'internal':
top_k = kb.knowledge_base_entity.top_k
elif kb.get_type() == 'external':
top_k = 5 # external kb's top_k is managed by plugin config
else:
top_k = 5 # default fallback
result = await kb.retrieve(user_message_text, top_k)
result = await kb.retrieve(
user_message_text,
settings={
'bot_uuid': query.bot_uuid or '',
'sender_id': str(query.sender_id),
'session_name': f'{query.session.launcher_type.value}_{query.session.launcher_id}',
},
)
if result:
all_results.extend(result)
@@ -97,9 +182,9 @@ class LocalAgentRunner(runner.RequestRunner):
if content.type == 'text' and content.text is not None:
texts.append(f'[{idx}] {content.text}')
idx += 1
rag_context = '\n\n'.join(texts)
rag_context_text = '\n\n'.join(texts)
final_user_message_text = rag_combined_prompt_template.format(
rag_context=rag_context, user_message=user_message_text
rag_context=rag_context_text, user_message=user_message_text
)
else:
@@ -121,51 +206,51 @@ class LocalAgentRunner(runner.RequestRunner):
remove_think = query.pipeline_config['output'].get('misc', '').get('remove-think')
use_llm_model = await self.ap.model_mgr.get_model_by_uuid(query.use_llm_model_uuid)
# Build ordered candidate list (primary + fallbacks)
candidates = await self._get_model_candidates(query)
if not candidates:
raise RuntimeError('No LLM model configured for local-agent runner')
self.ap.logger.debug(
f'localagent req: query={query.query_id} req_messages={req_messages} use_llm_model={query.use_llm_model_uuid}'
f'localagent req: query={query.query_id} req_messages={req_messages} '
f'candidates={[m.model_entity.name for m in candidates]}'
)
if not is_stream:
# 非流式输出,直接请求
msg = await use_llm_model.provider.invoke_llm(
# Non-streaming: invoke with fallback
msg, use_llm_model = await self._invoke_with_fallback(
query,
use_llm_model,
candidates,
req_messages,
query.use_funcs,
extra_args=use_llm_model.model_entity.extra_args,
remove_think=remove_think,
remove_think,
)
yield msg
final_msg = msg
else:
# 流式输出,需要处理工具调用
# Streaming: invoke with fallback
tool_calls_map: dict[str, provider_message.ToolCall] = {}
msg_idx = 0
accumulated_content = '' # 从开始累积的所有内容
accumulated_content = ''
last_role = 'assistant'
msg_sequence = 1
async for msg in use_llm_model.provider.invoke_llm_stream(
stream_src, use_llm_model = await self._invoke_stream_with_fallback(
query,
use_llm_model,
candidates,
req_messages,
query.use_funcs,
extra_args=use_llm_model.model_entity.extra_args,
remove_think=remove_think,
):
remove_think,
)
async for msg in stream_src:
msg_idx = msg_idx + 1
# 记录角色
if msg.role:
last_role = msg.role
# 累积内容
if msg.content:
accumulated_content += msg.content
# 处理工具调用
if msg.tool_calls:
for tool_call in msg.tool_calls:
if tool_call.id not in tool_calls_map:
@@ -177,21 +262,18 @@ class LocalAgentRunner(runner.RequestRunner):
),
)
if tool_call.function and tool_call.function.arguments:
# 流式处理中工具调用参数可能分多个chunk返回需要追加而不是覆盖
tool_calls_map[tool_call.id].function.arguments += tool_call.function.arguments
# continue
# 每8个chunk或最后一个chunk时输出所有累积的内容
if msg_idx % 8 == 0 or msg.is_final:
msg_sequence += 1
yield provider_message.MessageChunk(
role=last_role,
content=accumulated_content, # 输出所有累积内容
content=accumulated_content,
tool_calls=list(tool_calls_map.values()) if (tool_calls_map and msg.is_final) else None,
is_final=msg.is_final,
msg_sequence=msg_sequence,
)
# 创建最终消息用于后续处理
final_msg = provider_message.MessageChunk(
role=last_role,
content=accumulated_content,
@@ -206,7 +288,8 @@ class LocalAgentRunner(runner.RequestRunner):
req_messages.append(final_msg)
# 持续请求,只要还有待处理的工具调用就继续处理调用
# Once a model succeeds, commit to it for the tool call loop
# (no fallback mid-conversation — different models may interpret tool results differently)
while pending_tool_calls:
for tool_call in pending_tool_calls:
try:
@@ -247,7 +330,6 @@ class LocalAgentRunner(runner.RequestRunner):
req_messages.append(msg)
except Exception as e:
# 工具调用出错,添加一个报错信息到 req_messages
err_msg = provider_message.Message(role='tool', content=f'err: {e}', tool_call_id=tool_call.id)
yield err_msg
@@ -255,39 +337,38 @@ class LocalAgentRunner(runner.RequestRunner):
req_messages.append(err_msg)
self.ap.logger.debug(
f'localagent req: query={query.query_id} req_messages={req_messages} use_llm_model={query.use_llm_model_uuid}'
f'localagent req: query={query.query_id} req_messages={req_messages} '
f'use_llm_model={use_llm_model.model_entity.name}'
)
if is_stream:
tool_calls_map = {}
msg_idx = 0
accumulated_content = '' # 从开始累积的所有内容
accumulated_content = ''
last_role = 'assistant'
msg_sequence = first_end_sequence
async for msg in use_llm_model.provider.invoke_llm_stream(
tool_stream_src = use_llm_model.provider.invoke_llm_stream(
query,
use_llm_model,
req_messages,
query.use_funcs,
query.use_funcs if use_llm_model.model_entity.abilities.__contains__('func_call') else [],
extra_args=use_llm_model.model_entity.extra_args,
remove_think=remove_think,
):
)
async for msg in tool_stream_src:
msg_idx += 1
# 记录角色
if msg.role:
last_role = msg.role
# 第一次请求工具调用时的内容
# Prepend first-round content on first chunk of tool-call round
if msg_idx == 1:
accumulated_content = first_content if first_content is not None else accumulated_content
# 累积内容
if msg.content:
accumulated_content += msg.content
# 处理工具调用
if msg.tool_calls:
for tool_call in msg.tool_calls:
if tool_call.id not in tool_calls_map:
@@ -299,15 +380,13 @@ class LocalAgentRunner(runner.RequestRunner):
),
)
if tool_call.function and tool_call.function.arguments:
# 流式处理中工具调用参数可能分多个chunk返回需要追加而不是覆盖
tool_calls_map[tool_call.id].function.arguments += tool_call.function.arguments
# 每8个chunk或最后一个chunk时输出所有累积的内容
if msg_idx % 8 == 0 or msg.is_final:
msg_sequence += 1
yield provider_message.MessageChunk(
role=last_role,
content=accumulated_content, # 输出所有累积内容
content=accumulated_content,
tool_calls=list(tool_calls_map.values()) if (tool_calls_map and msg.is_final) else None,
is_final=msg.is_final,
msg_sequence=msg_sequence,
@@ -320,12 +399,12 @@ class LocalAgentRunner(runner.RequestRunner):
msg_sequence=msg_sequence,
)
else:
# 处理完所有调用,再次请求
# Non-streaming: use committed model directly (no fallback in tool loop)
msg = await use_llm_model.provider.invoke_llm(
query,
use_llm_model,
req_messages,
query.use_funcs,
query.use_funcs if use_llm_model.model_entity.abilities.__contains__('func_call') else [],
extra_args=use_llm_model.model_entity.extra_args,
remove_think=remove_think,
)

View File

@@ -22,12 +22,12 @@ class KnowledgeBaseInterface(metaclass=abc.ABCMeta):
pass
@abc.abstractmethod
async def retrieve(self, query: str, top_k: int) -> list[rag_context.RetrievalResultEntry]:
async def retrieve(self, query: str, settings: dict | None = None) -> list[rag_context.RetrievalResultEntry]:
"""Retrieve relevant documents from the knowledge base
Args:
query: The query string
top_k: Number of top results to return
settings: Optional per-request retrieval settings overrides
Returns:
List of retrieve result entries
@@ -45,8 +45,8 @@ class KnowledgeBaseInterface(metaclass=abc.ABCMeta):
pass
@abc.abstractmethod
def get_type(self) -> str:
"""Get the type of knowledge base (internal/external)"""
def get_knowledge_engine_plugin_id(self) -> str:
"""Get the Knowledge Engine plugin ID"""
pass
@abc.abstractmethod

View File

@@ -1,85 +0,0 @@
"""External knowledge base implementation"""
from __future__ import annotations
from langbot.pkg.core import app
from langbot.pkg.entity.persistence import rag as persistence_rag
from langbot_plugin.api.entities.builtin.rag import context as rag_context
from .base import KnowledgeBaseInterface
class ExternalKnowledgeBase(KnowledgeBaseInterface):
"""External knowledge base that queries via HTTP API or plugin retriever"""
external_kb_entity: persistence_rag.ExternalKnowledgeBase
# Plugin retriever instance ID
retriever_instance_id: str | None
def __init__(self, ap: app.Application, external_kb_entity: persistence_rag.ExternalKnowledgeBase):
super().__init__(ap)
self.external_kb_entity = external_kb_entity
self.retriever_instance_id = None
async def initialize(self):
"""Initialize the external knowledge base"""
# Use KB UUID as instance ID
# Instance creation is now handled by the unified sync mechanism
# when LangBot connects to runtime
self.retriever_instance_id = self.external_kb_entity.uuid
self.ap.logger.info(
f'Initialized external KB {self.external_kb_entity.uuid}, instance will be created by sync mechanism'
)
async def retrieve(self, query: str, top_k: int = 5) -> list[rag_context.RetrievalResultEntry]:
"""Retrieve documents from external knowledge base via plugin retriever"""
if not self.retriever_instance_id:
self.ap.logger.error(f'No retriever instance for KB {self.external_kb_entity.uuid}')
return []
try:
results = await self.ap.plugin_connector.retrieve_knowledge(
self.external_kb_entity.plugin_author,
self.external_kb_entity.plugin_name,
self.external_kb_entity.retriever_name,
self.retriever_instance_id,
{'query': query},
)
# Convert plugin results to RetrievalResultEntry
retrieval_entries = []
for result in results:
retrieval_entries.append(rag_context.RetrievalResultEntry(**result))
return retrieval_entries
except Exception as e:
self.ap.logger.error(f'Plugin retriever error: {e}')
import traceback
traceback.print_exc()
return []
def get_uuid(self) -> str:
"""Get the UUID of the external knowledge base"""
return self.external_kb_entity.uuid
def get_name(self) -> str:
"""Get the name of the external knowledge base"""
return self.external_kb_entity.name
def get_type(self) -> str:
"""Get the type of knowledge base"""
return 'external'
async def dispose(self):
"""Clean up resources"""
# Trigger sync to immediately delete the instance from plugin process
# This ensures instance is cleaned up without waiting for next LangBot restart
try:
await self.ap.plugin_connector.sync_polymorphic_component_instances()
self.ap.logger.info(
f'Disposed external KB {self.external_kb_entity.uuid}, triggered sync to delete instance'
)
except Exception as e:
self.ap.logger.error(f'Failed to sync after disposing KB: {e}')

View File

@@ -1,18 +1,19 @@
from __future__ import annotations
import mimetypes
import os.path
import traceback
import uuid
import zipfile
import io
from .services import parser, chunker
from typing import Any
from langbot.pkg.core import app
from langbot.pkg.rag.knowledge.services.embedder import Embedder
from langbot.pkg.rag.knowledge.services.retriever import Retriever
import sqlalchemy
from langbot.pkg.entity.persistence import rag as persistence_rag
from langbot.pkg.core import taskmgr
from langbot_plugin.api.entities.builtin.rag import context as rag_context
from .base import KnowledgeBaseInterface
from .external import ExternalKnowledgeBase
class RuntimeKnowledgeBase(KnowledgeBaseInterface):
@@ -20,28 +21,16 @@ class RuntimeKnowledgeBase(KnowledgeBaseInterface):
knowledge_base_entity: persistence_rag.KnowledgeBase
parser: parser.FileParser
chunker: chunker.Chunker
embedder: Embedder
retriever: Retriever
def __init__(self, ap: app.Application, knowledge_base_entity: persistence_rag.KnowledgeBase):
super().__init__(ap)
self.knowledge_base_entity = knowledge_base_entity
self.parser = parser.FileParser(ap=self.ap)
self.chunker = chunker.Chunker(ap=self.ap)
self.embedder = Embedder(ap=self.ap)
self.retriever = Retriever(ap=self.ap)
# 传递kb_id给retriever
self.retriever.kb_id = knowledge_base_entity.uuid
async def initialize(self):
pass
async def _store_file_task(self, file: persistence_rag.File, task_context: taskmgr.TaskContext):
async def _store_file_task(
self, file: persistence_rag.File, task_context: taskmgr.TaskContext, parser_plugin_id: str | None = None
):
try:
# set file status to processing
await self.ap.persistence_mgr.execute_async(
@@ -50,31 +39,46 @@ class RuntimeKnowledgeBase(KnowledgeBaseInterface):
.values(status='processing')
)
task_context.set_current_action('Parsing file')
# parse file
text = await self.parser.parse(file.file_name, file.extension)
if not text:
raise Exception(f'No text extracted from file {file.file_name}')
task_context.set_current_action('Processing file')
task_context.set_current_action('Chunking file')
# chunk file
chunks_texts = await self.chunker.chunk(text)
if not chunks_texts:
raise Exception(f'No chunks extracted from file {file.file_name}')
# Get file size from storage
file_size = await self.ap.storage_mgr.storage_provider.size(file.file_name)
task_context.set_current_action('Embedding chunks')
# Detect MIME type from extension
mime_type, _ = mimetypes.guess_type(file.file_name)
if mime_type is None:
mime_type = 'application/octet-stream'
embedding_model = await self.ap.model_mgr.get_embedding_model_by_uuid(
self.knowledge_base_entity.embedding_model_uuid
)
# embed chunks
await self.embedder.embed_and_store(
kb_id=self.knowledge_base_entity.uuid,
file_id=file.uuid,
chunks=chunks_texts,
embedding_model=embedding_model,
# If a parser plugin is specified, call it before ingestion
parsed_content = None
if parser_plugin_id:
task_context.set_current_action('Parsing file')
file_bytes = await self.ap.storage_mgr.storage_provider.load(file.file_name)
parse_context = {
'mime_type': mime_type,
'filename': file.file_name,
'metadata': {},
}
parsed_content = await self.ap.plugin_connector.call_parser(parser_plugin_id, parse_context, file_bytes)
# Call plugin to ingest document
result = await self._ingest_document(
{
'document_id': file.uuid,
'filename': file.file_name,
'extension': file.extension,
'file_size': file_size,
'mime_type': mime_type,
},
file.file_name, # storage path
parsed_content=parsed_content,
)
# Check plugin result status
if result.get('status') == 'failed':
error_msg = result.get('error_message', 'Plugin ingestion returned failed status')
raise Exception(error_msg)
# set file status to completed
await self.ap.persistence_mgr.execute_async(
sqlalchemy.update(persistence_rag.File)
@@ -97,16 +101,17 @@ class RuntimeKnowledgeBase(KnowledgeBaseInterface):
# delete file from storage
await self.ap.storage_mgr.storage_provider.delete(file.file_name)
async def store_file(self, file_id: str) -> str:
async def store_file(self, file_id: str, parser_plugin_id: str | None = None) -> str:
# pre checking
if not await self.ap.storage_mgr.storage_provider.exists(file_id):
raise Exception(f'File {file_id} not found')
file_name = file_id
extension = file_name.split('.')[-1].lower()
_, ext = os.path.splitext(file_name)
extension = ext.lstrip('.').lower() if ext else ''
if extension == 'zip':
return await self._store_zip_file(file_id)
return await self._store_zip_file(file_id, parser_plugin_id=parser_plugin_id)
file_uuid = str(uuid.uuid4())
kb_id = self.knowledge_base_entity.uuid
@@ -126,7 +131,7 @@ class RuntimeKnowledgeBase(KnowledgeBaseInterface):
# run background task asynchronously
ctx = taskmgr.TaskContext.new()
wrapper = self.ap.task_mgr.create_user_task(
self._store_file_task(file_obj, task_context=ctx),
self._store_file_task(file_obj, task_context=ctx, parser_plugin_id=parser_plugin_id),
kind='knowledge-operation',
name=f'knowledge-store-file-{file_id}',
label=f'Store file {file_id}',
@@ -134,7 +139,7 @@ class RuntimeKnowledgeBase(KnowledgeBaseInterface):
)
return wrapper.id
async def _store_zip_file(self, zip_file_id: str) -> str:
async def _store_zip_file(self, zip_file_id: str, parser_plugin_id: str | None = None) -> str:
"""Handle ZIP file by extracting each document and storing them separately."""
self.ap.logger.info(f'Processing ZIP file: {zip_file_id}')
@@ -150,7 +155,8 @@ class RuntimeKnowledgeBase(KnowledgeBaseInterface):
if file_info.is_dir() or file_info.filename.startswith('.'):
continue
file_extension = file_info.filename.split('.')[-1].lower()
_, file_ext = os.path.splitext(file_info.filename)
file_extension = file_ext.lstrip('.').lower()
if file_extension not in supported_extensions:
self.ap.logger.debug(f'Skipping unsupported file in ZIP: {file_info.filename}')
continue
@@ -159,18 +165,18 @@ class RuntimeKnowledgeBase(KnowledgeBaseInterface):
file_content = zip_ref.read(file_info.filename)
base_name = file_info.filename.replace('/', '_').replace('\\', '_')
extension = base_name.split('.')[-1]
file_name = base_name.split('.')[0]
file_stem, file_ext = os.path.splitext(base_name)
extension = file_ext.lstrip('.')
if file_name.startswith('__MACOSX'):
if file_stem.startswith('__MACOSX'):
continue
extracted_file_id = file_name + '_' + str(uuid.uuid4())[:8] + '.' + extension
extracted_file_id = file_stem + '_' + str(uuid.uuid4())[:8] + '.' + extension
# save file to storage
await self.ap.storage_mgr.storage_provider.save(extracted_file_id, file_content)
task_id = await self.store_file(extracted_file_id)
task_id = await self.store_file(extracted_file_id, parser_plugin_id=parser_plugin_id)
stored_file_tasks.append(task_id)
self.ap.logger.info(
@@ -189,21 +195,28 @@ class RuntimeKnowledgeBase(KnowledgeBaseInterface):
return stored_file_tasks[0] if stored_file_tasks else ''
async def retrieve(self, query: str, top_k: int) -> list[rag_context.RetrievalResultEntry]:
embedding_model = await self.ap.model_mgr.get_embedding_model_by_uuid(
self.knowledge_base_entity.embedding_model_uuid
)
return await self.retriever.retrieve(self.knowledge_base_entity.uuid, query, embedding_model, top_k)
async def retrieve(self, query: str, settings: dict | None = None) -> list[rag_context.RetrievalResultEntry]:
# Merge stored retrieval_settings with per-request overrides
stored = self.knowledge_base_entity.retrieval_settings or {}
merged = {**stored, **(settings or {})}
if 'top_k' not in merged:
merged['top_k'] = 5 # fallback default
response = await self._retrieve(query, merged)
results_data = response.get('results', [])
entries = []
for r in results_data:
if isinstance(r, dict):
entries.append(rag_context.RetrievalResultEntry(**r))
elif isinstance(r, rag_context.RetrievalResultEntry):
entries.append(r)
return entries
async def delete_file(self, file_id: str):
# delete vector
await self.ap.vector_db_mgr.vector_db.delete_by_file_id(self.knowledge_base_entity.uuid, file_id)
# delete chunk
await self.ap.persistence_mgr.execute_async(
sqlalchemy.delete(persistence_rag.Chunk).where(persistence_rag.Chunk.file_id == file_id)
)
await self._delete_document(file_id)
# Also cleanup DB record
await self.ap.persistence_mgr.execute_async(
sqlalchemy.delete(persistence_rag.File).where(persistence_rag.File.uuid == file_id)
)
@@ -216,32 +229,295 @@ class RuntimeKnowledgeBase(KnowledgeBaseInterface):
"""Get the name of the knowledge base"""
return self.knowledge_base_entity.name
def get_type(self) -> str:
"""Get the type of knowledge base"""
return 'internal'
def get_knowledge_engine_plugin_id(self) -> str:
"""Get the Knowledge Engine plugin ID"""
return self.knowledge_base_entity.knowledge_engine_plugin_id or ''
async def dispose(self):
await self.ap.vector_db_mgr.vector_db.delete_collection(self.knowledge_base_entity.uuid)
"""Dispose the knowledge base, notifying the plugin to cleanup."""
await self._on_kb_delete()
# ========== Plugin Communication Methods ==========
async def _on_kb_create(self) -> None:
"""Notify plugin about KB creation."""
plugin_id = self.knowledge_base_entity.knowledge_engine_plugin_id
if not plugin_id:
return
try:
config = self.knowledge_base_entity.creation_settings or {}
self.ap.logger.info(
f'Calling RAG plugin {plugin_id}: on_knowledge_base_create(kb_id={self.knowledge_base_entity.uuid})'
)
await self.ap.plugin_connector.rag_on_kb_create(plugin_id, self.knowledge_base_entity.uuid, config)
except Exception as e:
self.ap.logger.error(f'Failed to notify plugin {plugin_id} on KB create: {e}')
raise
async def _on_kb_delete(self) -> None:
"""Notify plugin about KB deletion."""
plugin_id = self.knowledge_base_entity.knowledge_engine_plugin_id
if not plugin_id:
return
try:
self.ap.logger.info(
f'Calling RAG plugin {plugin_id}: on_knowledge_base_delete(kb_id={self.knowledge_base_entity.uuid})'
)
await self.ap.plugin_connector.rag_on_kb_delete(plugin_id, self.knowledge_base_entity.uuid)
except Exception as e:
self.ap.logger.error(f'Failed to notify plugin {plugin_id} on KB delete: {e}')
async def _ingest_document(
self,
file_metadata: dict[str, Any],
storage_path: str,
parsed_content: dict[str, Any] | None = None,
) -> dict[str, Any]:
"""Call plugin to ingest document."""
kb = self.knowledge_base_entity
plugin_id = kb.knowledge_engine_plugin_id
if not plugin_id:
self.ap.logger.error(f'No RAG plugin ID configured for KB {kb.uuid}. Ingestion failed.')
raise ValueError('RAG Plugin ID required')
self.ap.logger.info(f'Calling RAG plugin {plugin_id}: ingest(doc={file_metadata.get("filename")})')
# Inject knowledge_base_id into file metadata as required by SDK schema
file_metadata['knowledge_base_id'] = kb.uuid
context_data = {
'file_object': {
'metadata': file_metadata,
'storage_path': storage_path,
},
'knowledge_base_id': kb.uuid,
'collection_id': kb.collection_id or kb.uuid,
'creation_settings': kb.creation_settings or {},
'parsed_content': parsed_content,
}
try:
result = await self.ap.plugin_connector.call_rag_ingest(plugin_id, context_data)
return result
except Exception as e:
self.ap.logger.error(f'Plugin ingestion failed: {e}')
raise
async def _retrieve(
self,
query: str,
settings: dict[str, Any],
) -> dict[str, Any]:
"""Call plugin to retrieve documents.
Raises:
ValueError: If no RAG plugin is configured for this KB.
Exception: If the plugin retrieval call fails.
"""
kb = self.knowledge_base_entity
plugin_id = kb.knowledge_engine_plugin_id
if not plugin_id:
raise ValueError(f'No RAG plugin ID configured for KB {kb.uuid}. Retrieval failed.')
# Session context (e.g. session_name) stays in retrieval_settings
# for plugins that need it. Do NOT move them into filters, as filters
# are passed directly to vector_search by some plugins (e.g. LangRAG)
# and would cause empty results when the metadata field doesn't exist.
filters = settings.pop('filters', {})
retrieval_context = {
'query': query,
'knowledge_base_id': kb.uuid,
'collection_id': kb.collection_id or kb.uuid,
'retrieval_settings': settings,
'creation_settings': kb.creation_settings or {},
'filters': filters,
}
result = await self.ap.plugin_connector.call_rag_retrieve(
plugin_id,
retrieval_context,
)
return result
async def _delete_document(self, document_id: str) -> bool:
"""Call plugin to delete document."""
kb = self.knowledge_base_entity
plugin_id = kb.knowledge_engine_plugin_id
if not plugin_id:
return False
self.ap.logger.info(f'Calling RAG plugin {plugin_id}: delete_document(doc_id={document_id})')
try:
return await self.ap.plugin_connector.call_rag_delete_document(plugin_id, document_id, kb.uuid)
except Exception as e:
self.ap.logger.error(f'Plugin document deletion failed: {e}')
return False
class RAGManager:
ap: app.Application
knowledge_bases: list[KnowledgeBaseInterface]
knowledge_bases: dict[str, KnowledgeBaseInterface]
def __init__(self, ap: app.Application):
self.ap = ap
self.knowledge_bases = []
self.knowledge_bases = {}
async def initialize(self):
await self.load_knowledge_bases_from_db()
async def get_all_knowledge_base_details(self) -> list[dict]:
"""Get all knowledge bases with enriched Knowledge Engine details."""
# 1. Get raw KBs from DB
result = await self.ap.persistence_mgr.execute_async(sqlalchemy.select(persistence_rag.KnowledgeBase))
knowledge_bases = result.all()
# 2. Get all available Knowledge Engines for enrichment
engine_map = {}
if self.ap.plugin_connector.is_enable_plugin:
try:
engines = await self.ap.plugin_connector.list_knowledge_engines()
engine_map = {e['plugin_id']: e for e in engines}
except Exception as e:
self.ap.logger.warning(f'Failed to list Knowledge Engines: {e}')
# 3. Serialize and enrich
kb_list = []
for kb in knowledge_bases:
kb_dict = self.ap.persistence_mgr.serialize_model(persistence_rag.KnowledgeBase, kb)
self._enrich_kb_dict(kb_dict, engine_map)
kb_list.append(kb_dict)
return kb_list
async def get_knowledge_base_details(self, kb_uuid: str) -> dict | None:
"""Get specific knowledge base with enriched Knowledge Engine details."""
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_rag.KnowledgeBase).where(persistence_rag.KnowledgeBase.uuid == kb_uuid)
)
kb = result.first()
if not kb:
return None
kb_dict = self.ap.persistence_mgr.serialize_model(persistence_rag.KnowledgeBase, kb)
# Fetch engines
engine_map = {}
if self.ap.plugin_connector.is_enable_plugin:
try:
engines = await self.ap.plugin_connector.list_knowledge_engines()
engine_map = {e['plugin_id']: e for e in engines}
except Exception as e:
self.ap.logger.warning(f'Failed to list Knowledge Engines: {e}')
self._enrich_kb_dict(kb_dict, engine_map)
return kb_dict
@staticmethod
def _to_i18n_name(name) -> dict:
"""Ensure name is always an I18nObject-compatible dict.
If *name* is already a dict (with ``en_US`` / ``zh_Hans`` keys) it is
returned as-is. A plain string is wrapped into an I18nObject so the
frontend ``extractI18nObject`` helper never receives an unexpected type.
"""
if isinstance(name, dict):
return name
return {'en_US': str(name), 'zh_Hans': str(name)}
def _enrich_kb_dict(self, kb_dict: dict, engine_map: dict) -> None:
"""Helper to inject engine info into KB dict."""
plugin_id = kb_dict.get('knowledge_engine_plugin_id')
# Default fallback structure — name must be I18nObject for frontend compatibility
fallback_name = self._to_i18n_name(plugin_id or 'Internal (Legacy)')
fallback_info = {
'plugin_id': plugin_id,
'name': fallback_name,
'capabilities': [],
}
if not plugin_id:
kb_dict['knowledge_engine'] = fallback_info
return
engine_info = engine_map.get(plugin_id)
if engine_info:
kb_dict['knowledge_engine'] = {
'plugin_id': plugin_id,
'name': self._to_i18n_name(engine_info.get('name', plugin_id)),
'capabilities': engine_info.get('capabilities', []),
}
else:
kb_dict['knowledge_engine'] = fallback_info
async def create_knowledge_base(
self,
name: str,
knowledge_engine_plugin_id: str,
creation_settings: dict,
retrieval_settings: dict | None = None,
description: str = '',
) -> persistence_rag.KnowledgeBase:
"""Create a new knowledge base using a RAG plugin."""
# Validate that the Knowledge Engine plugin exists
if self.ap.plugin_connector.is_enable_plugin:
try:
engines = await self.ap.plugin_connector.list_knowledge_engines()
engine_ids = [e.get('plugin_id') for e in engines]
if knowledge_engine_plugin_id not in engine_ids:
raise ValueError(f'Knowledge Engine plugin {knowledge_engine_plugin_id} not found')
except ValueError:
raise
except Exception as e:
self.ap.logger.warning(f'Failed to validate Knowledge Engine plugin existence: {e}')
kb_uuid = str(uuid.uuid4())
# Use UUID as collection ID by default for isolation
collection_id = kb_uuid
kb_data = {
'uuid': kb_uuid,
'name': name,
'description': description,
'knowledge_engine_plugin_id': knowledge_engine_plugin_id,
'collection_id': collection_id,
'creation_settings': creation_settings,
'retrieval_settings': retrieval_settings or {},
}
# Create Entity
kb = persistence_rag.KnowledgeBase(**kb_data)
# Persist
await self.ap.persistence_mgr.execute_async(sqlalchemy.insert(persistence_rag.KnowledgeBase).values(kb_data))
# Load into Runtime
runtime_kb = await self.load_knowledge_base(kb)
# Notify Plugin — rollback DB record and runtime entry on failure
try:
await runtime_kb._on_kb_create()
except Exception:
self.knowledge_bases.pop(kb_uuid, None)
await self.ap.persistence_mgr.execute_async(
sqlalchemy.delete(persistence_rag.KnowledgeBase).where(persistence_rag.KnowledgeBase.uuid == kb_uuid)
)
raise
self.ap.logger.info(f'Created new Knowledge Base {name} ({kb_uuid}) using plugin {knowledge_engine_plugin_id}')
return kb
async def load_knowledge_bases_from_db(self):
self.ap.logger.info('Loading knowledge bases from db...')
self.knowledge_bases = []
self.knowledge_bases = {}
# Load internal knowledge bases
# Load knowledge bases
result = await self.ap.persistence_mgr.execute_async(sqlalchemy.select(persistence_rag.KnowledgeBase))
knowledge_bases = result.all()
@@ -253,86 +529,37 @@ class RAGManager:
f'Error loading knowledge base {knowledge_base.uuid}: {e}\n{traceback.format_exc()}'
)
# Load external knowledge bases
external_result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_rag.ExternalKnowledgeBase)
)
external_kbs = external_result.all()
for external_kb in external_kbs:
try:
# Don't trigger sync during batch loading - will sync once after LangBot connects to runtime
await self.load_external_knowledge_base(external_kb, trigger_sync=False)
except Exception as e:
self.ap.logger.error(
f'Error loading external knowledge base {external_kb.uuid}: {e}\n{traceback.format_exc()}'
)
async def load_knowledge_base(
self,
knowledge_base_entity: persistence_rag.KnowledgeBase | sqlalchemy.Row | dict,
) -> RuntimeKnowledgeBase:
if isinstance(knowledge_base_entity, sqlalchemy.Row):
# Safe access to _mapping for SQLAlchemy 1.4+
knowledge_base_entity = persistence_rag.KnowledgeBase(**knowledge_base_entity._mapping)
elif isinstance(knowledge_base_entity, dict):
knowledge_base_entity = persistence_rag.KnowledgeBase(**knowledge_base_entity)
# Filter out non-database fields (like knowledge_engine which is computed)
filtered_dict = {
k: v for k, v in knowledge_base_entity.items() if k in persistence_rag.KnowledgeBase.ALL_DB_FIELDS
}
knowledge_base_entity = persistence_rag.KnowledgeBase(**filtered_dict)
runtime_knowledge_base = RuntimeKnowledgeBase(ap=self.ap, knowledge_base_entity=knowledge_base_entity)
await runtime_knowledge_base.initialize()
self.knowledge_bases.append(runtime_knowledge_base)
self.knowledge_bases[runtime_knowledge_base.get_uuid()] = runtime_knowledge_base
return runtime_knowledge_base
async def load_external_knowledge_base(
self,
external_kb_entity: persistence_rag.ExternalKnowledgeBase | sqlalchemy.Row | dict,
trigger_sync: bool = True,
) -> ExternalKnowledgeBase:
"""Load external knowledge base into runtime
Args:
external_kb_entity: External KB entity to load
trigger_sync: Whether to trigger sync after loading (default True for manual creation, False for batch loading)
"""
if isinstance(external_kb_entity, sqlalchemy.Row):
external_kb_entity = persistence_rag.ExternalKnowledgeBase(**external_kb_entity._mapping)
elif isinstance(external_kb_entity, dict):
external_kb_entity = persistence_rag.ExternalKnowledgeBase(**external_kb_entity)
external_kb = ExternalKnowledgeBase(ap=self.ap, external_kb_entity=external_kb_entity)
await external_kb.initialize()
self.knowledge_bases.append(external_kb)
# Trigger sync to create the instance immediately (for manual creation)
# Skip sync during batch loading from DB to avoid multiple sync calls
if trigger_sync:
try:
await self.ap.plugin_connector.sync_polymorphic_component_instances()
self.ap.logger.info(f'Triggered sync after loading external KB {external_kb_entity.uuid}')
except Exception as e:
self.ap.logger.error(f'Failed to sync after loading external KB: {e}')
return external_kb
async def get_knowledge_base_by_uuid(self, kb_uuid: str) -> KnowledgeBaseInterface | None:
for kb in self.knowledge_bases:
if kb.get_uuid() == kb_uuid:
return kb
return None
return self.knowledge_bases.get(kb_uuid)
async def remove_knowledge_base_from_runtime(self, kb_uuid: str):
for kb in self.knowledge_bases:
if kb.get_uuid() == kb_uuid:
self.knowledge_bases.remove(kb)
return
self.knowledge_bases.pop(kb_uuid, None)
async def delete_knowledge_base(self, kb_uuid: str):
for kb in self.knowledge_bases:
if kb.get_uuid() == kb_uuid:
await kb.dispose()
self.knowledge_bases.remove(kb)
return
kb = self.knowledge_bases.pop(kb_uuid, None)
if kb is not None:
await kb.dispose()
else:
self.ap.logger.warning(f'Knowledge base {kb_uuid} not found in runtime, skipping plugin notification')

Some files were not shown because too many files have changed in this diff Show More