perf(scale): speed up traffic, auto-renew, and node bulk ops at 50k-100k clients

Local hot paths:
- autoRenewClients: replace the O(clients x expired) inner scan with an
  email->traffic map lookup (quadratic at scale).
- node traffic sync: scope the client_traffics email-membership query to the
  snapshot's emails instead of plucking the whole table every poll.
- add a (expiry_time, reset) index for the per-tick auto-renew filter.
- SQLite: add cache_size/mmap_size/temp_store pragmas (env-tunable); keep the
  single-file DELETE journal and synchronous=FULL defaults.
- scale benchmarks now run on SQLite too via XUI_SCALE_TEST=1 (shared
  setupScaleDB/resetScaleTables helpers), not just Postgres.

Node paths:
- bulk add/delete/adjust on a node-attached inbound folded one HTTP RPC per
  client; above nodeBulkPushThreshold (32) mark the node dirty and let one
  ReconcileNode push converge it instead of O(M) sequential round-trips.
  Small ops keep the live per-client path. Also hoist nodePushPlan out of the
  per-email delete loop.
- ReconcileNode skips inbounds whose wire payload is unchanged (per-tag
  fingerprint on Remote), guarded by node-side tag presence so a restarted
  node is still re-seeded.

Tests: auto-renew multi-inbound correctness, node-path dispatch (large ops
fold to dirty, small ops push live) via a manager runtime override seam, and
reconcile delta-skip.
This commit is contained in:
MHSanaei
2026-06-20 10:35:46 +02:00
parent e079490144
commit 6a032bcb2a
15 changed files with 623 additions and 137 deletions
+2 -2
View File
@@ -11,8 +11,8 @@ type ClientTraffic struct {
SubId string `json:"subId" form:"subId" gorm:"-" example:"i7tvdpeffi0hvvf1"`
Up int64 `json:"up" form:"up" example:"1048576"`
Down int64 `json:"down" form:"down" example:"2097152"`
ExpiryTime int64 `json:"expiryTime" form:"expiryTime" example:"1735689600000"`
ExpiryTime int64 `json:"expiryTime" form:"expiryTime" gorm:"index:idx_client_traffics_renew,priority:1" example:"1735689600000"`
Total int64 `json:"total" form:"total" example:"10737418240"`
Reset int `json:"reset" form:"reset" gorm:"default:0" example:"0"`
Reset int `json:"reset" form:"reset" gorm:"default:0;index:idx_client_traffics_renew,priority:2" example:"0"`
LastOnline int64 `json:"lastOnline" form:"lastOnline" gorm:"default:0" example:"1735680000000"`
}