Files
LangBot/web/tests/e2e/crud-smoke.spec.ts
RockChinQ e1ac5e0fc8 feat(tenancy): add Workspace multi-tenant foundation (#2353)
* Document multi-tenant workspace architecture

* Add OSS and commercial workspace boundaries

* docs: redesign multi-tenant workspace architecture

* feat(tenancy): implement workspace isolation

* docs(tenancy): record verification evidence

* docs(tenancy): revise single-instance SaaS topology

* docs(tenancy): refine architecture options

* docs: finalize cloud v2 multi-tenant decisions

* feat(tenancy): establish cloud isolation foundations

* feat(tenancy): harden shared cloud runtime boundaries

* docs(tenancy): record final isolation verification

* fix(tenancy): close isolation and permission gaps

* docs(tenancy): record final isolation verification

* feat(tenancy): connect cloud workspace control plane

* fix(build): install git for pinned SDK

* docs(cloud): update control plane verification

* chore: update multi-tenant SDK pin

* fix(cloud): skip legacy model sync during startup

* test(cloud): preserve minimal model manager fixtures

* fix(cloud): preserve authenticated account context

* fix(cloud): reuse authenticated account for user info

* feat(cloud): complete Workspace settings navigation

* test(web): cover Workspace dropdown menu

* feat(web): place workspace controls in sidebar

* refactor(web): streamline workspace controls

* style(web): format workspace layout test

* fix(cloud): surface runtime and workspace plan status

* fix(plugin): keep runtime identity stable across restarts

* fix(ui): widen and center workspace switcher

* fix(ui): hide roles from workspace switcher

* fix(ui): align workspace switcher with sidebar entries

* feat(workspace): add in-product collaboration and direct Cloud launch

* style: format collaboration changes

* fix(workspace): bind collaboration APIs to tenant UoW

* fix(cloud): preserve Core-owned collaboration state

* test(cloud): require Space identity for invite registration

* feat(cloud): complete secure invitation experience

* style(web): format invitation flows

* fix(cloud): recover box runtime without unscoped skill reload

* feat(oss): enforce invitation account and owner billing flows

* style: format OSS account service

* test(oss): cover invitation logout handoff

* fix(oss): resolve workspace owner in scoped session

* feat(cloud): harden multi-tenant runtime resources

* fix(cloud): bound runtime restart storms

* fix(cloud): eliminate periodic runtime CPU spikes

* fix(cloud): enforce instance capacity ceilings

* fix(cloud): scope public login capability discovery

* fix(cloud): bound tenant maintenance and monitoring work

* fix(runtime): bound tenant resource amplification

* fix(deps): pin green multi-tenant plugin SDK

* fix(cloud): handle unavailable skill capability

* fix(security): require authentication for image file endpoint (H-2)

- Changed /api/v1/files/image from AuthType.NONE to USER_TOKEN_OR_API_KEY
- Added Permission.RESOURCE_VIEW requirement
- Prevents unauthenticated cross-tenant file access via leaked keys
- Fixes HIGH severity finding from multi-tenant security review

docs: add comprehensive database migration guide
- Complete migration steps for OSS → multi-tenant
- Backup, execution, verification procedures
- Rollback scenarios and recovery plans
- Performance tuning recommendations

* test: add comprehensive cross-tenant isolation tests

Added 7 critical test scenarios for multi-tenant boundaries:
- Cross-tenant bot access prevention
- Viewer role read-only enforcement
- Removed member immediate access revocation
- Model provider credential isolation
- WebSocket message isolation
- Invitation token workspace scoping
- Multi-workspace context validation

These tests address P0-2 coverage gaps for:
- workspaces.py (membership & invitation flows)
- user.py (authentication & authorization)
- websocket_chat.py (real-time isolation)
- plugins.py (resource access control)

docs: finalize database migration guide

* fix(security): resolve M-1, M-2, M-3 security findings

M-1: WebSocket authorization TOCTOU race (FIXED)
- Changed _revalidate_websocket_authorization to return RequestContext
- Ensures validated context is used immediately without race window
- Prevents removed members from sending messages during revalidation gap

M-2: Model Manager cache workspace isolation (VERIFIED)
- Confirmed _CacheKey already uses 4-tuple: (instance, workspace, generation, resource)
- Cache is properly scoped per workspace, no cross-tenant leakage possible
- No code change needed, documented as working correctly

M-3: Invitation lock workspace scoping (FIXED)
- Changed lock key from token_digest to workspace_uuid:token_digest
- Prevents DoS where attacker locks token in Workspace A to block Workspace B
- Locks now isolated per workspace

All MEDIUM severity findings from security review now resolved.

* fix(cloud): unblock tenant CI and enforce knowledge quotas

* fix(tenancy): scope rerank model sync

---------

Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
2026-07-30 21:43:35 +08:00

539 lines
18 KiB
TypeScript

import { expect, Page, test } from '@playwright/test';
import {
installLangBotApiMocks,
makeWorkspaceEntry,
} from './fixtures/langbot-api';
async function save(page: Page) {
const button = page.getByRole('button', { name: /^Save$/ });
await expect(button).toBeEnabled();
await button.click();
}
async function submit(page: Page) {
await page.getByRole('button', { name: /^Submit$/ }).click();
}
async function confirmDelete(page: Page) {
await page
.getByRole('dialog')
.getByRole('button', { name: /^Confirm Delete$/ })
.click();
}
test.describe('frontend CRUD smoke flows', () => {
test('viewer keeps ordinary bot and pipeline monitoring access', async ({
page,
}) => {
const workspace = makeWorkspaceEntry(
'workspace-viewer',
'Viewer Workspace',
'local',
);
await installLangBotApiMocks(page, {
authenticated: true,
workspaces: [workspace],
});
await page.goto('/home/bots?id=new');
await page.locator('input[name="name"]').fill('Viewer Test Bot');
await page
.locator('input[name="description"]')
.fill('Proves monitoring is ordinary resource visibility.');
await page.getByRole('combobox').click();
await page.getByRole('option', { name: 'Playwright Adapter' }).click();
await submit(page);
await expect(page).toHaveURL(/\/home\/bots\?id=bot-1$/);
await page.goto('/home/pipelines?id=new');
await page.locator('input[name="basic.name"]').fill('Viewer Pipeline');
await page
.locator('input[name="basic.description"]')
.fill('Viewer monitoring permission regression.');
await submit(page);
await expect(page).toHaveURL(/\/home\/pipelines\?id=pipeline-1$/);
workspace.membership.role = 'viewer';
workspace.permissions = ['member.view', 'resource.view', 'workspace.view'];
await page.goto('/home/bots?id=bot-1');
await expect(page.getByRole('tab', { name: 'Logs' })).toBeVisible();
await expect(page.getByRole('tab', { name: 'Sessions' })).toBeVisible();
await expect(page.getByRole('button', { name: /^Save$/ })).toHaveCount(0);
await page.getByRole('tab', { name: 'Logs' }).click();
await expect(page.getByText('No logs yet')).toBeVisible();
await page.goto('/home/pipelines?id=pipeline-1');
await expect(page.getByRole('tab', { name: 'Dashboard' })).toBeVisible();
await expect(page.getByRole('tab', { name: 'Debug Chat' })).toHaveCount(0);
await expect(page.getByRole('button', { name: /^Save$/ })).toHaveCount(0);
await page.goto('/home/monitoring');
await expect(
page.getByRole('button', { name: 'Refresh Data' }),
).toBeVisible();
await expect(page.getByRole('button', { name: 'Export Data' })).toHaveCount(
0,
);
});
test('creates, edits, and deletes a bot', async ({ page }) => {
await installLangBotApiMocks(page, { authenticated: true });
await page.goto('/home/bots?id=new');
await expect(page.locator('input[name="name"]')).toBeVisible();
await page.locator('input[name="name"]').fill('Support Bot');
await page
.locator('input[name="description"]')
.fill('Answers customer support questions.');
await page.getByRole('combobox').click();
await page.getByRole('option', { name: 'Playwright Adapter' }).click();
await submit(page);
await expect(page).toHaveURL(/\/home\/bots\?id=bot-1$/);
await page.reload();
await expect(page.locator('input[name="name"]')).toHaveValue('Support Bot');
await page
.locator('input[name="description"]')
.fill('Answers customer support questions with context.');
await save(page);
await expect(page.locator('input[name="description"]')).toHaveValue(
'Answers customer support questions with context.',
);
await page.getByRole('button', { name: /^Delete$/ }).click();
await confirmDelete(page);
await expect(page).toHaveURL(/\/home\/bots$/);
await expect(page.getByText('Select a bot from the sidebar')).toBeVisible();
});
test('creates, edits, and deletes a pipeline', async ({ page }) => {
await installLangBotApiMocks(page, { authenticated: true });
await page.goto('/home/pipelines?id=new');
await expect(page.locator('input[name="basic.name"]')).toBeVisible();
await page.locator('input[name="basic.name"]').fill('Escalation Pipeline');
await page
.locator('input[name="basic.description"]')
.fill('Routes urgent customer issues.');
await submit(page);
await expect(page).toHaveURL(/\/home\/pipelines\?id=pipeline-1$/);
await page.reload();
await expect(page.locator('input[name="basic.name"]')).toHaveValue(
'Escalation Pipeline',
);
await page
.locator('input[name="basic.description"]')
.fill('Routes urgent customer issues to operators.');
await save(page);
await expect(page.locator('input[name="basic.description"]')).toHaveValue(
'Routes urgent customer issues to operators.',
);
await page.getByRole('button', { name: /^Delete$/ }).click();
await confirmDelete(page);
await expect(page).toHaveURL(/\/home\/pipelines$/);
await expect(
page.getByText('Select a pipeline from the sidebar'),
).toBeVisible();
});
test('opens pipeline AI capabilities with malformed model options', async ({
page,
}) => {
await installLangBotApiMocks(page, { authenticated: true });
await page.goto('/home/pipelines?id=pipeline-ai');
await expect(page.locator('input[name="basic.name"]')).toBeVisible();
await page.getByRole('button', { name: /^AI$/ }).click();
await expect(page.getByText('Runtime')).toBeVisible();
await expect(
page.locator('[data-slot="card-title"]').filter({
hasText: 'Built-in Agent',
}),
).toBeVisible();
await expect(
page.locator('label').filter({
hasText: 'Model',
}),
).toBeVisible();
await expect(page.getByText('A <Select.Item')).toHaveCount(0);
await expect(page.getByText('500')).toHaveCount(0);
});
test('creates, edits, and deletes a knowledge base', async ({ page }) => {
await installLangBotApiMocks(page, { authenticated: true });
await page.goto('/home/knowledge?id=new');
await expect(page.locator('input[name="name"]')).toBeVisible();
await page.locator('input[name="name"]').fill('Support Knowledge');
await page
.locator('input[name="description"]')
.fill('Source material for support answers.');
await submit(page);
await expect(page).toHaveURL(/\/home\/knowledge\?id=knowledge-1$/);
await page.reload();
await expect(page.locator('input[name="name"]')).toHaveValue(
'Support Knowledge',
);
await page.waitForTimeout(600);
await page
.locator('input[name="description"]')
.fill('Updated source material for support answers.');
await save(page);
await expect(page.locator('input[name="description"]')).toHaveValue(
'Updated source material for support answers.',
);
await page.getByRole('button', { name: /^Delete$/ }).click();
await confirmDelete(page);
await expect(page).toHaveURL(/\/home\/knowledge$/);
await expect(
page.getByText('Select a knowledge base from the sidebar'),
).toBeVisible();
});
test('creates, edits, and deletes an MCP server', async ({ page }) => {
await installLangBotApiMocks(page, { authenticated: true });
await page.goto('/home/mcp?id=new');
await expect(page.locator('input[name="name"]')).toBeVisible();
await page.locator('input[name="name"]').fill('playwright-mcp');
await page
.locator('input[name="url"]')
.fill('https://mcp.example.test/sse');
await submit(page);
await expect(page).toHaveURL(/\/home\/mcp\?id=playwright-mcp$/);
await page.reload();
await expect(page.locator('input[name="name"]')).toHaveValue(
'playwright-mcp',
);
await page
.locator('input[name="url"]')
.fill('https://mcp.example.test/updated-sse');
await save(page);
await expect(page.locator('input[name="url"]')).toHaveValue(
'https://mcp.example.test/updated-sse',
);
await page.getByRole('button', { name: /^Delete$/ }).click();
await confirmDelete(page);
await expect(page).toHaveURL(/\/home\/mcp$/);
await expect(
page.getByText('Select an MCP server from the sidebar'),
).toBeVisible();
});
test('updates and deletes a manually-created skill', async ({ page }) => {
await installLangBotApiMocks(page, { authenticated: true });
await page.goto('/home/skills?action=create');
await page.locator('#display_name').fill('Release Notes');
await page.locator('#name').fill('release_notes');
await page.locator('#description').fill('Drafts release notes.');
await page
.locator('#instructions')
.fill('Summarize merged changes for the next release.');
await save(page);
await expect(page).toHaveURL(/\/home\/skills\?id=release_notes$/);
await page.reload();
await expect(page.locator('#description')).toHaveValue(
'Drafts release notes.',
);
await page
.locator('#description')
.fill('Drafts concise release notes for maintainers.');
await expect(page.locator('#description')).toHaveValue(
'Drafts concise release notes for maintainers.',
);
await save(page);
await page.reload();
await expect(page.locator('#description')).toHaveValue(
'Drafts concise release notes for maintainers.',
);
await expect(page.locator('#instructions')).toHaveValue(
'Summarize merged changes for the next release.',
);
await page.getByRole('button', { name: /^Delete$/ }).click();
await confirmDelete(page);
await expect(page).toHaveURL(/\/home\/add-extension$/);
});
});
test.describe('bot advanced flows', () => {
test('toggles bot enable/disable state', async ({ page }) => {
await installLangBotApiMocks(page, { authenticated: true });
// Create a bot first
await page.goto('/home/bots?id=new');
await page.locator('input[name="name"]').fill('Toggle Test Bot');
await page.getByRole('combobox').click();
await page.getByRole('option', { name: 'Playwright Adapter' }).click();
await submit(page);
await expect(page).toHaveURL(/\/home\/bots\?id=bot-1$/);
// Wait for the enable switch to load (it's fetched via getBot)
await expect(page.locator('#bot-enable-switch')).toBeVisible({
timeout: 5000,
});
// Verify initial state is enabled
await expect(page.locator('#bot-enable-switch')).toBeChecked();
// Toggle to disabled
await page.locator('#bot-enable-switch').click();
await expect(page.locator('#bot-enable-switch')).not.toBeChecked();
// Reload and verify state persisted
await page.reload();
await expect(page.locator('#bot-enable-switch')).not.toBeChecked();
});
test('switches between bot detail tabs', async ({ page }) => {
await installLangBotApiMocks(page, { authenticated: true });
// Create a bot
await page.goto('/home/bots?id=new');
await page.locator('input[name="name"]').fill('Tab Test Bot');
await page.getByRole('combobox').click();
await page.getByRole('option', { name: 'Playwright Adapter' }).click();
await submit(page);
// Verify we're on the Configuration tab
await expect(
page.getByRole('tab', { name: /Configuration/ }),
).toHaveAttribute('data-state', 'active');
await expect(page.locator('input[name="name"]')).toBeVisible();
// Switch to Logs tab
await page.getByRole('tab', { name: /Logs/ }).click();
await expect(page.getByRole('tab', { name: /Logs/ })).toHaveAttribute(
'data-state',
'active',
);
// Switch to Sessions tab
await page.getByRole('tab', { name: /Sessions/ }).click();
await expect(page.getByRole('tab', { name: /Sessions/ })).toHaveAttribute(
'data-state',
'active',
);
// Switch back to Configuration
await page.getByRole('tab', { name: /Configuration/ }).click();
await expect(page.locator('input[name="name"]')).toBeVisible();
});
test('save button is disabled when form is clean', async ({ page }) => {
await installLangBotApiMocks(page, { authenticated: true });
// Create a bot
await page.goto('/home/bots?id=new');
await page.locator('input[name="name"]').fill('Clean Form Bot');
await page.getByRole('combobox').click();
await page.getByRole('option', { name: 'Playwright Adapter' }).click();
await submit(page);
// After creation, save button should be disabled (form is clean)
const saveButton = page.getByRole('button', { name: /^Save$/ });
await expect(saveButton).toBeDisabled();
// Edit the form
await page.locator('input[name="description"]').fill('New description');
await expect(saveButton).toBeEnabled();
// Save
await saveButton.click();
await expect(saveButton).toBeDisabled();
});
test('shows validation error when bot name is empty', async ({ page }) => {
await installLangBotApiMocks(page, { authenticated: true });
await page.goto('/home/bots?id=new');
// Select adapter but leave name empty
await page.getByRole('combobox').click();
await page.getByRole('option', { name: 'Playwright Adapter' }).click();
await submit(page);
// Should show validation error for name (zod validation)
await expect(page.getByText(/cannot be empty/i)).toBeVisible();
await expect(page).toHaveURL(/\/home\/bots\?id=new$/);
});
});
test.describe('pipeline advanced flows', () => {
test('switches to monitoring tab from pipeline detail', async ({ page }) => {
await installLangBotApiMocks(page, { authenticated: true });
// Create a pipeline
await page.goto('/home/pipelines?id=new');
await page.locator('input[name="basic.name"]').fill('Tab Test Pipeline');
await submit(page);
// Verify we're on the Configuration tab
await expect(
page.getByRole('tab', { name: /Configuration/ }),
).toHaveAttribute('data-state', 'active');
// Switch to Monitoring tab (labeled "Dashboard" in the pipeline context)
// Skip Debug tab as it requires WebSocket connection
await page.getByRole('tab', { name: /Dashboard/ }).click();
await expect(page.getByRole('tab', { name: /Dashboard/ })).toHaveAttribute(
'data-state',
'active',
);
// Switch back to Configuration
await page.getByRole('tab', { name: /Configuration/ }).click();
await expect(page.locator('input[name="basic.name"]')).toBeVisible();
});
test('save button reflects form dirty state', async ({ page }) => {
await installLangBotApiMocks(page, { authenticated: true });
// Create a pipeline
await page.goto('/home/pipelines?id=new');
await page.locator('input[name="basic.name"]').fill('Dirty Form Pipeline');
await submit(page);
// Wait for the page to fully load and form to reset
await page.waitForTimeout(500);
// Edit the form - use the name field which definitely triggers dirty state
await page
.locator('input[name="basic.name"]')
.fill('Dirty Form Pipeline Updated');
const saveButton = page.getByRole('button', { name: /^Save$/ });
await expect(saveButton).toBeEnabled();
// Save
await saveButton.click();
// Wait for save to complete
await page.waitForTimeout(500);
});
test('shows validation error when pipeline name is empty', async ({
page,
}) => {
await installLangBotApiMocks(page, { authenticated: true });
await page.goto('/home/pipelines?id=new');
// Submit without filling name
await submit(page);
// Should show validation error for name (zod validation)
await expect(page.getByText(/cannot be empty/i)).toBeVisible();
await expect(page).toHaveURL(/\/home\/pipelines\?id=new$/);
});
});
test.describe('cross-resource flows', () => {
test('creates a pipeline then binds it to a bot', async ({ page }) => {
await installLangBotApiMocks(page, { authenticated: true });
// Create a pipeline first
await page.goto('/home/pipelines?id=new');
await page.locator('input[name="basic.name"]').fill('Production Pipeline');
await submit(page);
await expect(page).toHaveURL(/\/home\/pipelines\?id=pipeline-1$/);
// Create a bot
await page.goto('/home/bots?id=new');
await page.locator('input[name="name"]').fill('Bound Bot');
await page.getByRole('combobox').click();
await page.getByRole('option', { name: 'Playwright Adapter' }).click();
await submit(page);
await expect(page).toHaveURL(/\/home\/bots\?id=bot-1$/);
// Wait for form to fully load
await expect(page.locator('input[name="name"]')).toHaveValue('Bound Bot');
// Find the pipeline select by its label "Bind Pipeline"
const pipelineCard = page.getByText('Bind Pipeline').locator('..');
await expect(pipelineCard).toBeVisible({ timeout: 5000 });
// Click on the select trigger within the pipeline binding card
// The select trigger shows "Select Pipeline" placeholder initially
const pipelineSelectTrigger = page.getByText('Select Pipeline').first();
await pipelineSelectTrigger.click();
// Select the pipeline option
await page.getByRole('option', { name: 'Production Pipeline' }).click();
// Save the bot
await save(page);
// Reload and verify binding persisted
await page.reload();
// The pipeline name should appear in the select trigger (not in sidebar or options)
await expect(
page
.locator('[data-slot="select-trigger"]')
.filter({ hasText: 'Production Pipeline' }),
).toBeVisible();
});
});
test.describe('empty states', () => {
test('shows empty state when no bots exist', async ({ page }) => {
await installLangBotApiMocks(page, { authenticated: true });
await page.goto('/home/bots');
await expect(page.getByText('Select a bot from the sidebar')).toBeVisible();
});
test('shows empty state when no pipelines exist', async ({ page }) => {
await installLangBotApiMocks(page, { authenticated: true });
await page.goto('/home/pipelines');
await expect(
page.getByText('Select a pipeline from the sidebar'),
).toBeVisible();
});
test('shows empty state when no knowledge bases exist', async ({ page }) => {
await installLangBotApiMocks(page, { authenticated: true });
await page.goto('/home/knowledge');
await expect(
page.getByText('Select a knowledge base from the sidebar'),
).toBeVisible();
});
test('shows empty state when no MCP servers exist', async ({ page }) => {
await installLangBotApiMocks(page, { authenticated: true });
await page.goto('/home/mcp');
await expect(
page.getByText('Select an MCP server from the sidebar'),
).toBeVisible();
});
});