Files
3x-ui/internal/web/service/node_dirty_test.go
T
mrchatam 6f7a305239 fix(node): stop stale expiry sync from undoing client extensions (#6228) (#6231)
* fix(node): stop stale expiry sync from undoing client extensions (#6228)

After an expired client is extended on the master, a lagging node could
overwrite client_traffics with an older absolute expiry and latch
enable=false. Reject older absolute expiries on merge, ignore
expiry-stale disables when the master is not over quota, lift stale
lifecycle fields out of adopted settings, stamp reconcile fingerprints
from the pre-lift node blob, and mark the node dirty so the next tick
re-pushes.

* fix(node): lockstep client_traffics expiry/enable SQL with review fixes (#6228)

Make expiry merge keep any master absolute (node only activates when master
is unset/duration). Include this tick's up/down deltas in the enable
stale-disable quota check so a crossing-tick disable is not dropped.

* fix(node): add settings absolute helper for renew/lift guards (#6228)

Expose settingsClientAbsoluteExpiry so traffic merge can tell a real
node auto-renew (settings+stats later) from lagging ClientStats after a
master shorten. Trim lift godoc to the invariant.

* fix(node): authority-aware lifecycle merge for multi-node sync (#6228)

While config_dirty, accumulate traffic only — do not adopt node
expiry/enable/total/reset (and preserve dipped baselines so a false
renew cannot fire after clear). On clean ticks, master absolute expiry
wins; node auto-renew still goes through nodeClientRenewed when settings
also show the later deadline. Defer settings lifecycle lift until after
traffic deltas land, align SyncInbound via applyMasterClientLifecycle,
and avoid re-MarkNodeDirty when already dirty.

* fix(node): clear config_dirty only after the post-reconcile traffic merge (#6228)

After a successful ReconcileNode, keep the node dirty through the same
tick's SetRemoteTraffic so lagging ClientStats cannot clobber the
just-pushed master lifecycle, then ClearNodeDirty.

* test(node): cover dirty-gate, master-absolute, and renew false-positives (#6228)

Add regressions for extend/shorten while dirty, clean-sibling shorten,
settings vs lagging disable, renew recovery after dirty, renew with
matching settings, and shorten+Reset lagging stats not treated as renew.

* fix(node): address the review findings on the lifecycle merge (#6228)

The automated review on #6231 flagged a blocking regression and six smaller
issues. All of them are fixed here.

Blocking: making the master's absolute expiry always win left nodeClientRenewed
as the only channel for a node-side auto-renew, and that required a counter dip.
A client that used no traffic in the period never dips, so its renewal was
dropped, the master kept the expired deadline and disableInvalidClients removed
it with no way back (master-side autoRenewClients skips node inbounds). The node
bumps reset_count on every renewal, so that counter is now an independent
renewal signal and is persisted with the renewal so it keeps converging.

The deferred ClearNodeDirty made every reconcile-success tick merge in dirty
mode, which suppressed inbound adoption, new client_traffics rows, the orphan
sweeps and the whole SyncInbound record loop -- and left the node dirty forever
whenever SetRemoteTraffic errored. The clear goes back to where it was; a
separate justPushed flag now freezes only the client lifecycle merge for the
tick whose push just landed.

staleNodeDisable only recognised a lagging disable by an older expiry, so a
quota top-up (raise totalGB, leave the expiry alone) was re-latched to disabled
by the next lagging snapshot -- the #6228 symptom on a second axis. The
reviewer's suggestion of dropping the expiry precondition outright fails
TestNodeQuotaDisable_SameExpiryStillLatches, because the master's own counters
legitimately sit below a node's after a seeded-at-zero adoption. nodeDisableIsStale
instead compares the limits the node judged the client against with the master's
own: matching limits mean a genuine verdict that still latches (#4917), differing
limits mean the node has not seen the master's change yet. It also now measures
the master deadline against wall-clock now, so an expired master row no longer
looks "extended" merely because the node's copy is older still.

Also: the settings lift now writes enable in both directions, so a blob fetched
before a master disable cannot carry enable=true back into central settings and
on to the node; the renewal guard parses the inbound settings once per inbound
instead of once per renewing client; the adoption loop only writes settings when
they actually changed; and two comments that described mechanisms the code does
not use were corrected.

The test deadlines are now relative to the run: the merge compares against now,
so fixed timestamps would have rotted into the wrong side of it.

---------

Co-authored-by: mrchatam <287639636+mrchatam@users.noreply.github.com>
Co-authored-by: Sanaei <ho3ein.sanaei@gmail.com>
2026-08-23 23:17:56 +02:00

327 lines
11 KiB
Go

package service
import (
"errors"
"testing"
"gorm.io/gorm"
"github.com/mhsanaei/3x-ui/v3/internal/database"
"github.com/mhsanaei/3x-ui/v3/internal/database/model"
"github.com/mhsanaei/3x-ui/v3/internal/web/runtime"
)
// While a node is config-dirty (a local edit committed before it could be
// mirrored to the node), the traffic pull must not overwrite the central
// inbound's config columns from the node's stale snapshot — only traffic
// counters may advance. Otherwise a reconnecting node reverts the edit.
func TestSetRemoteTraffic_DirtyPreservesConfig(t *testing.T) {
setupConflictDB(t)
db := database.GetDB()
node := &model.Node{Name: "n1", Address: "127.0.0.1", Port: 2096, ApiToken: "tok", Enable: true, Status: "online"}
if err := db.Create(node).Error; err != nil {
t.Fatalf("create node: %v", err)
}
id := node.Id
const desiredSettings = `{"clients":[{"email":"a@x"}]}`
central := &model.Inbound{
UserId: 1,
NodeID: &id,
Tag: "in-443-tcp",
Enable: true,
Port: 443,
Protocol: model.VLESS,
Settings: desiredSettings,
}
if err := db.Create(central).Error; err != nil {
t.Fatalf("create inbound: %v", err)
}
snap := &runtime.TrafficSnapshot{
Inbounds: []*model.Inbound{{
Tag: "in-443-tcp",
Enable: true,
Port: 443,
Protocol: model.VLESS,
Settings: `{"clients":[{"email":"b@x"}]}`,
Up: 500,
Down: 700,
}},
}
svc := InboundService{}
if _, err := svc.setRemoteTrafficLocked(id, snap, true, false); err != nil {
t.Fatalf("setRemoteTrafficLocked dirty: %v", err)
}
var got model.Inbound
if err := db.First(&got, central.Id).Error; err != nil {
t.Fatalf("reload inbound: %v", err)
}
if got.Settings != desiredSettings {
t.Fatalf("dirty pull overwrote settings: want %q got %q", desiredSettings, got.Settings)
}
if got.Up != 500 || got.Down != 700 {
t.Fatalf("traffic counters not applied while dirty: up=%d down=%d", got.Up, got.Down)
}
}
func TestSetRemoteTraffic_MissingDisabledInboundIsNotSwept(t *testing.T) {
setupConflictDB(t)
db := database.GetDB()
node := &model.Node{Name: "disabled-snapshot", Address: "127.0.0.1", Port: 2096, ApiToken: "tok", Enable: true, Status: "online"}
if err := db.Create(node).Error; err != nil {
t.Fatal(err)
}
disabled := &model.Inbound{
UserId: 1, NodeID: &node.Id, Tag: "disabled", Enable: false,
Port: 24443, Protocol: model.VLESS, Settings: `{"clients":[]}`,
}
reported := &model.Inbound{
UserId: 1, NodeID: &node.Id, Tag: "reported", Enable: true,
Port: 24444, Protocol: model.VLESS, Settings: `{"clients":[]}`,
}
if err := db.Create(disabled).Error; err != nil {
t.Fatal(err)
}
if err := db.Create(reported).Error; err != nil {
t.Fatal(err)
}
snap := &runtime.TrafficSnapshot{Inbounds: []*model.Inbound{{
Tag: reported.Tag, Enable: true,
Port: reported.Port, Protocol: reported.Protocol, Settings: reported.Settings,
}}}
if _, err := (&InboundService{}).setRemoteTrafficLocked(node.Id, snap, false, false); err != nil {
t.Fatal(err)
}
var count int64
if err := db.Model(&model.Inbound{}).Where("id=?", disabled.Id).Count(&count).Error; err != nil {
t.Fatal(err)
}
if count != 1 {
t.Fatalf("disabled inbound rows=%d, want 1", count)
}
}
// Deleting a *disabled* client attached to a node inbound must still propagate
// to the node. The node's own DB carries the (disabled) client, so the central
// panel has to mark the node dirty (→ reconcile) instead of dropping the delete
// and letting the next traffic snapshot resurrect the client. Regression for
// the enable-flag gate that used to skip the node path entirely (#5352).
func TestDelInboundClientByEmail_DisabledNodeClientMarksDirty(t *testing.T) {
setupConflictDB(t)
db := database.GetDB()
// Offline node so nodePushPlan reports dirty without needing a live runtime.
node := &model.Node{Name: "n1", Address: "127.0.0.1", Port: 2096, ApiToken: "tok", Enable: true, Status: "offline"}
if err := db.Create(node).Error; err != nil {
t.Fatalf("create node: %v", err)
}
id := node.Id
central := &model.Inbound{
UserId: 1,
NodeID: &id,
Tag: "in-443-tcp",
Enable: true,
Port: 443,
Protocol: model.VLESS,
Settings: `{"clients":[{"email":"a@x","enable":false}]}`,
}
if err := db.Create(central).Error; err != nil {
t.Fatalf("create inbound: %v", err)
}
inboundSvc := &InboundService{}
clientSvc := &ClientService{}
if _, err := clientSvc.DelInboundClientByEmail(inboundSvc, central.Id, "a@x", false, false); err != nil {
t.Fatalf("DelInboundClientByEmail: %v", err)
}
if _, _, dirty, _, err := (&NodeService{}).NodeSyncState(id); err != nil {
t.Fatalf("NodeSyncState: %v", err)
} else if !dirty {
t.Fatal("deleting a disabled node client must mark the node dirty (#5352)")
}
}
// An online, enabled node that is merely config-dirty must NOT be reported as
// pending: every node-backed edit marks the node dirty as the reconcile
// self-heal marker, so keying the "saved, node offline, will sync" toast off
// the dirty flag fired it on every save to a healthy online node.
func TestIsNodePending_OnlineDirtyNodeIsNotPending(t *testing.T) {
setupConflictDB(t)
db := database.GetDB()
node := &model.Node{Name: "n1", Address: "127.0.0.1", Port: 2096, ApiToken: "tok", Enable: true, Status: "online"}
if err := db.Create(node).Error; err != nil {
t.Fatalf("create node: %v", err)
}
nodeSvc := NodeService{}
if nodeSvc.IsNodePending(node.Id) {
t.Fatal("a clean online node must not be pending")
}
if err := nodeSvc.MarkNodeDirty(node.Id); err != nil {
t.Fatalf("MarkNodeDirty: %v", err)
}
if nodeSvc.IsNodePending(node.Id) {
t.Fatal("an online, enabled node must not be pending just because it is config-dirty")
}
}
// Offline or disabled nodes are genuinely deferred and must report pending so
// the "saved, node offline, will sync" toast still surfaces for them.
func TestIsNodePending_OfflineOrDisabledIsPending(t *testing.T) {
setupConflictDB(t)
db := database.GetDB()
offline := &model.Node{Name: "off", Address: "127.0.0.1", Port: 2096, ApiToken: "tok", Enable: true, Status: "offline"}
disabled := &model.Node{Name: "dis", Address: "127.0.0.1", Port: 2097, ApiToken: "tok", Enable: false, Status: "online"}
for _, n := range []*model.Node{offline, disabled} {
if err := db.Create(n).Error; err != nil {
t.Fatalf("create node %s: %v", n.Name, err)
}
}
// Node.Enable carries gorm default:true, so Create({Enable:false}) persists
// TRUE — force the column off to actually exercise the disabled path.
if err := db.Model(&model.Node{}).Where("id = ?", disabled.Id).Update("enable", false).Error; err != nil {
t.Fatalf("force-disable node: %v", err)
}
nodeSvc := NodeService{}
if !nodeSvc.IsNodePending(offline.Id) {
t.Fatal("an offline node must be pending")
}
if !nodeSvc.IsNodePending(disabled.Id) {
t.Fatal("a disabled node must be pending")
}
}
// ClearNodeDirty must be a compare-and-swap on config_dirty_at so a concurrent
// edit that re-dirties the node during a reconcile is not silently cleared.
func TestNodeDirty_ClearIsCASOnDirtyAt(t *testing.T) {
setupConflictDB(t)
db := database.GetDB()
node := &model.Node{Name: "n2", Address: "127.0.0.1", Port: 2096, ApiToken: "tok", Enable: true, Status: "online"}
if err := db.Create(node).Error; err != nil {
t.Fatalf("create node: %v", err)
}
nodeSvc := NodeService{}
if err := nodeSvc.MarkNodeDirty(node.Id); err != nil {
t.Fatalf("MarkNodeDirty: %v", err)
}
_, _, dirty, dirtyAt, err := nodeSvc.NodeSyncState(node.Id)
if err != nil {
t.Fatalf("NodeSyncState: %v", err)
}
if !dirty {
t.Fatal("node should be dirty after MarkNodeDirty")
}
if err := nodeSvc.ClearNodeDirty(node.Id, dirtyAt-1); err != nil {
t.Fatalf("ClearNodeDirty stale token: %v", err)
}
if _, _, stillDirty, _, _ := nodeSvc.NodeSyncState(node.Id); !stillDirty {
t.Fatal("stale-token clear must not clear the dirty flag")
}
if err := nodeSvc.ClearNodeDirty(node.Id, dirtyAt); err != nil {
t.Fatalf("ClearNodeDirty matching token: %v", err)
}
if _, _, stillDirty, _, _ := nodeSvc.NodeSyncState(node.Id); stillDirty {
t.Fatal("matching-token clear must clear the dirty flag")
}
}
func TestMarkNodeDirtyTxRollsBackWithTransaction(t *testing.T) {
setupConflictDB(t)
db := database.GetDB()
node := &model.Node{Name: "n3", Address: "127.0.0.1", Port: 2096, ApiToken: "tok", Enable: true, Status: "online"}
if err := db.Create(node).Error; err != nil {
t.Fatalf("create node: %v", err)
}
nodeSvc := NodeService{}
rollbackErr := errors.New("force rollback")
if err := db.Transaction(func(tx *gorm.DB) error {
if err := nodeSvc.MarkNodeDirtyTx(tx, node.Id); err != nil {
return err
}
return rollbackErr
}); !errors.Is(err, rollbackErr) {
t.Fatalf("rollback tx: got %v want %v", err, rollbackErr)
}
if _, _, dirty, _, err := nodeSvc.NodeSyncState(node.Id); err != nil {
t.Fatalf("NodeSyncState after rollback: %v", err)
} else if dirty {
t.Fatal("dirty flag escaped a rolled-back transaction")
}
if err := db.Transaction(func(tx *gorm.DB) error {
return nodeSvc.MarkNodeDirtyTx(tx, node.Id)
}); err != nil {
t.Fatalf("commit tx: %v", err)
}
if _, _, dirty, _, err := nodeSvc.NodeSyncState(node.Id); err != nil {
t.Fatalf("NodeSyncState after commit: %v", err)
} else if !dirty {
t.Fatal("dirty flag should commit with its transaction")
}
}
// Editing a node must mark it config-dirty so the next traffic-sync tick
// reconciles (pushes the panel's inbounds to the remote) before pulling a
// snapshot. Without the dirty flag, re-pointing a node to a fresh server
// makes the orphan sweep delete every central inbound absent from the empty
// snapshot (#5461).
func TestNodeService_UpdateMarksNodeDirty(t *testing.T) {
setupConflictDB(t)
db := database.GetDB()
node := &model.Node{
Name: "n1",
Address: "10.0.0.1",
Port: 2096,
ApiToken: "tok",
Enable: true,
Status: "online",
}
if err := db.Create(node).Error; err != nil {
t.Fatalf("create node: %v", err)
}
edited := &model.Node{
Name: node.Name,
Address: "10.0.0.2",
Port: 2097,
ApiToken: node.ApiToken,
Enable: true,
}
nodeSvc := NodeService{}
if err := nodeSvc.Update(node.Id, edited); err != nil {
t.Fatalf("Update: %v", err)
}
_, _, dirty, _, err := nodeSvc.NodeSyncState(node.Id)
if err != nil {
t.Fatalf("NodeSyncState: %v", err)
}
if !dirty {
t.Fatal("Update must mark the node config-dirty so sync reconciles before snapshot sweep (#5461)")
}
var got model.Node
if err := db.First(&got, node.Id).Error; err != nil {
t.Fatalf("reload node: %v", err)
}
if got.Address != "10.0.0.2" || got.Port != 2097 {
t.Fatalf("node row not updated: address=%q port=%d", got.Address, got.Port)
}
}