perf(clients): scale add/delete and bulk client operations

Follow-up to the SyncInbound bulk rewrite, fixing the remaining O(M*N)
and O(M)-round-trip behaviour in the add/delete and bulk paths that made
them time out on large inbounds (worst case minutes), especially on
PostgreSQL.

- compactOrphans: chunk the "email IN (...)" lookup (400/batch) instead
  of binding every email at once. A single huge IN exceeded PostgreSQL's
  65535-parameter limit (and SQLite's) and made the planner pathological,
  so add/delete failed outright past ~100k clients.

- emailsUsedByOtherInbounds: new batched form used by delInboundClients
  (BulkDetach) and bulkDelInboundClients (BulkDelete), replacing a
  per-email global JSON scan (O(M*N)) with one scan, and skipped entirely
  when keepTraffic is set.

- BulkCreate: rewritten to validate/dedup in one pass, then group clients
  by inbound and add them in a single addInboundClient call per inbound
  (one getAllEmailSubIDs, one settings rewrite, one SyncInbound) instead
  of running the full single-create pipeline per client.

- Bulk delete/adjust: batch DelClientStat/DelClientIPs with IN deletes
  and wrap the settings Save + SyncInbound in one transaction, so the
  per-row writes share a single fsync instead of one per row.

Measured on PostgreSQL 16 (one inbound, M=2000 affected clients):
  - create: 8m35s (M=500) -> ~1-5s
  - detach: 52s -> ~4s (flat in N)
  - delete: ~16s -> ~1-4s
  - adjust: ~20s -> ~7-10s
add/delete of a single client on a 200k-client inbound stays in seconds.

sync_scale_postgres_test.go adds skip-gated benchmarks (XUI_DB_TYPE=
postgres) for the single add/delete and the five bulk operations.
This commit is contained in:
MHSanaei
2026-06-04 19:41:00 +02:00
parent 756746dbca
commit f185d3315c
6 changed files with 426 additions and 80 deletions
@@ -249,7 +249,7 @@ export default function ClientBulkAddModal({
)}
{form.emailMethod < 2 && (
<Form.Item label={t('pages.clients.clientCount')}>
<InputNumber value={form.quantity} min={1} max={100} onChange={(v) => update('quantity', Number(v) || 1)} />
<InputNumber value={form.quantity} min={1} max={1000} onChange={(v) => update('quantity', Number(v) || 1)} />
</Form.Item>
)}
+2 -4
View File
@@ -71,6 +71,7 @@ import type { ClientFilters } from './filters';
import './ClientsPage.css';
const FILTER_STATE_KEY = 'clientsFilterState';
const DISABLED_PAGE_SIZE = 200;
function UngroupIcon() {
return (
@@ -276,10 +277,7 @@ export default function ClientsPage() {
const activeCount = activeFilterCount(filters);
useEffect(() => {
if (pageSize > 0) {
setTablePageSize(pageSize);
}
setTablePageSize(pageSize > 0 ? pageSize : DISABLED_PAGE_SIZE);
}, [pageSize]);
const onlineSet = useMemo(() => new Set(onlines || []), [onlines]);
+1 -1
View File
@@ -182,7 +182,7 @@ export const ClientBulkAddFormSchema = z.object({
lastNum: z.number().int().min(1),
emailPrefix: z.string(),
emailPostfix: z.string(),
quantity: z.number().int().min(1).max(100),
quantity: z.number().int().min(1).max(1000),
subId: z.string(),
group: z.string(),
comment: z.string(),