feat(node): node hardening — mTLS, hashed+zstd reconcile transport, per-node net metrics (#5382)

* fix(api-docs): document clientIpsByGuid route Restores a green `go test ./...` baseline: TestAPIRoutesDocumented flagged POST /panel/api/clients/clientIpsByGuid (added in 9385b6c6) as undocumented in endpoints.ts. * test(node): characterize current node TLS + API auth behavior Phase 0 regression net for the mTLS work. These pass on unchanged production code and lock the pre-mTLS contracts so later phases can be proven additive: - tlsConfigForNode: skip -> InsecureSkipVerify (no VerifyConnection); pin -> VerifyConnection installed. - checkAPIAuth: bearer match -> Next + api_authed; unauthenticated -> 401 (XHR) / 404; valid session -> Next. - panel HTTPS listener with no ClientAuth accepts a client that presents no client certificate (the browsers-keep-working invariant). * feat(crypto): node-auth CA + client-cert minting (TDD) Stdlib-only ECDSA P-256 helpers for the node mTLS work: - GenerateNodeCA: self-signed CA (IsCA, CertSign, path len 0) - IssueClientCert: client-auth leaf (ExtKeyUsageClientAuth) signed by CA - LoadCAFromPEM: parse a CA cert+key for issuing / trust-pool building Tests assert the contract (leaf verifies against the issuing CA with ExtKeyUsageClientAuth), seen failing on the assertion before impl. * feat(node): lazy node mTLS CA + client cert in settings (TDD) SettingService gains opt-in mTLS material, all stored as Setting rows with empty defaults and kept out of entity.AllSetting (so private keys never reach the settings UI/export): - EnsureNodeMtlsCA: mint+persist the node-auth CA once, reuse thereafter - EnsureMasterClientCert: issue the master client cert from the CA, idempotent - NodeMtlsClientCAPool: ClientCAs trust pool for the listener; nil when unconfigured so the no-mTLS path is unchanged Tests assert idempotency and that the client cert verifies against the CA for client auth; seen failing on the assertion before impl. * feat(node): mtls client TLS config + master-cert provider (TDD) tlsConfigForNode gains an 'mtls' branch that presents the master client certificate and verifies the node server against system roots (no InsecureSkipVerify, no custom RootCAs). The cert is supplied via an injected MasterClientCertProvider so runtime need not import service; it fails closed when unconfigured. skip/pin contracts unchanged. * feat(node): allow tokenless mtls nodes in remote do() (TDD) mtls nodes authenticate with a client certificate, so the bearer token becomes optional for them: do() no longer rejects an empty ApiToken when TlsVerifyMode is mtls, and the Authorization header is omitted when no token is set. Every other mode still requires a token (regression kept). * feat(node): authenticate verified client certs in checkAPIAuth (TDD) A completed mTLS handshake (non-empty r.TLS.VerifiedChains) now authenticates an API request, equivalent to a valid bearer token, and sets api_authed so the CSRF middleware lets cert-authed mutations through. Bearer/session/reject paths unchanged. The accept-path assert was mutation-checked (guard flipped -> test red -> reverted). * feat(node): opt-in mTLS on the panel listener (TDD; mutation-checked) web.go now applies VerifyClientCertIfGiven + ClientCAs to the HTTPS listener when a node trust CA is configured, and wires the master client cert provider for outbound mtls calls. With no CA the listener is byte-identical to before (browsers unaffected). applyNodeMtls is covered end-to-end: no-cert client handshakes (browsers keep working), a CA-signed client cert verifies, a foreign-CA cert is rejected at the handshake. Mutation-checked: - RequireAndVerifyClientCert -> no-cert client rejected (red) -> reverted - drop ClientCAs -> master cert no longer trusted (red) -> reverted * feat(node): accept mtls verify-mode + CA reveal endpoint (TDD) - model.Node.TlsVerifyMode validator now accepts 'mtls' - normalize() preserves mtls and requires the node scheme to be https (fail closed), instead of clamping mtls back to verify - NodeService.NodeMtlsCaCert + POST /panel/api/nodes/mtls/ca return this panel's node-auth CA cert (public) to paste into a node, minting the CA + master client cert on first call - endpoints.ts documents the new route (doc-sync test) No model column added (enum is a string), so no migration/codegen. * feat(node): node mTLS UI + trust-CA setter (TDD) Backend: - NodeService.SetNodeMtlsTrustCA + POST /panel/api/nodes/mtls/trustCA store the CA this panel trusts for incoming node-API client certs (validates PEM, empty clears); applied on next restart - endpoints.ts + regenerated openapi.json document both mtls routes Frontend: - node form: 'mtls' TLS-verify option + setup hint (zod enum updated) - Nodes page 'Node mTLS' card: copy this panel's CA, and paste/save the trusted parent CA - en-US i18n keys (other locales fall back to en-US) Gates green: go build (native+windows), vet, go test ./...; frontend typecheck, lint, vitest (541). * style(node): gofmt web_mtls_test doc comment * feat(node): hashed+zstd reconcile transport (TDD, negotiated, mixed-version safe) Adds an integrity + compression envelope to node config pushes: - internal/util/wirecodec: shared zstd codec (bomb-capped decode) + SHA-256 hashing + the header/capability constants - Remote.do(): always attaches X-Config-Sha256 of the uncompressed body; zstd-compresses only when the node advertised support (learned from its X-3x-Node-Caps response header) and the body is >=1KiB - ConfigEnvelopeMiddleware on /panel/api: advertises the cap, decompresses and verifies the hash (handler not invoked on mismatch) before binding Mixed-version safe: old nodes never advertise the cap -> plain bodies; the hash header is verify-if-present so any panel/node mix interoperates (existing reconcile tests stay green). klauspost/compress promoted to a direct dep. Hash-mismatch reject was mutation-checked (compare defeated -> test red -> reverted). * feat(node): per-node network throughput metrics (TDD) The node status response already carries gopsutil netIO.up/down (summed non-virtual interfaces), so no node-side change is needed: - probe() parses netIO.up/down into HeartbeatPatch.NetUp/NetDown - Node gains net_up/net_down columns (AutoMigrate); UpdateHeartbeat persists them and appends netUp/netDown to the per-node metric history - NodeMetricKeys whitelists netUp/netDown so the history endpoint serves them - NodeHistoryPanel renders Net Up/Down sparklines (KB/s, no 0-100 clamp) - regenerated frontend types + openapi.json for the new Node fields * feat(node): move node mTLS controls into a toolbar button + modal The Node mTLS panel was an always-visible card cluttering the nodes page. Replace it with a 'Node mTLS' button beside 'Add node' that opens a modal with the same copy-CA + trusted-parent-CA controls; the modal closes on a successful save. No backend/i18n changes. * i18n(node): translate mTLS + net-metrics keys for all locales Adds the node mTLS strings (tlsMtls, mtlsFormHint, mtls.* dialog + the saveMtls toast) and the netUp/netDown chart labels to all 12 non-English catalogs (ar, es, fa, id, ja, pt, ru, tr, uk, vi, zh-CN, zh-TW), matching each catalog's existing terminology. Technical tokens (mTLS/TLS/CA/API/ KB/s) kept verbatim. * fix(node): address Copilot review on node-hardening PR - setting_mtls: fail closed on a half-present CA/master-cert pair instead of silently regenerating (which would rotate the CA and break fleet trust). - config_envelope: reject non-zstd Content-Encoding on the envelope path rather than hashing/forwarding a still-encoded body to the handler. - node mTLS: support tokenless mTLS end-to-end — apiToken is now required_unless tlsVerifyMode=mtls (model) with matching conditional validation in NodeFormSchema, so the runtime allowance is actually reachable. - NodesPage: add a catch block to onSaveTrustCa so save failures surface.
2026-08-11 13:50:59 +00:00 · 2026-06-16 12:19:33 +02:00
parent f3eba04ed8
commit 37c5e0bfd2
51 changed files with 3073 additions and 1014 deletions
@@ -175,7 +175,7 @@ var SystemMetricKeys = []string{
 }

 // NodeMetricKeys lists the per-node metric names NodeHeartbeatJob writes.
-var NodeMetricKeys = []string{"cpu", "mem"}
+var NodeMetricKeys = []string{"cpu", "mem", "netUp", "netDown"}

 // XrayMetricKeys lists series sourced from xray's /debug/vars expvar
 // endpoint. Populated by XrayMetricsService.Sample on the same 2s cadence
@@ -35,7 +35,11 @@ type HeartbeatPatch struct {
 	CpuPct        float64
 	MemPct        float64
 	UptimeSecs    uint64
-	LastError     string
+	// NetUp/NetDown are the node's current interface throughput (bytes/sec),
+	// summed over non-virtual interfaces, read from its status response.
+	NetUp     uint64
+	NetDown   uint64
+	LastError string
 	// XrayState and XrayError come from the remote /panel/api/server/status when the
 	// panel API is reachable. They allow distinguishing panel connectivity from
 	// Xray core health on the node.
@@ -275,9 +279,12 @@ func (s *NodeService) normalize(n *model.Node) error {
 	if n.Scheme != "http" && n.Scheme != "https" {
 		n.Scheme = "https"
 	}
-	if n.TlsVerifyMode != "skip" && n.TlsVerifyMode != "pin" {
+	if n.TlsVerifyMode != "skip" && n.TlsVerifyMode != "pin" && n.TlsVerifyMode != "mtls" {
 		n.TlsVerifyMode = "verify"
 	}
+	if n.TlsVerifyMode == "mtls" && n.Scheme != "https" {
+		return common.NewError("mtls requires the node scheme to be https")
+	}
 	n.PinnedCertSha256 = strings.TrimSpace(n.PinnedCertSha256)
 	if n.InboundSyncMode != "selected" {
 		n.InboundSyncMode = "all"
@@ -555,6 +562,8 @@ func (s *NodeService) UpdateHeartbeat(id int, p HeartbeatPatch) error {
 		"cpu_pct":        p.CpuPct,
 		"mem_pct":        p.MemPct,
 		"uptime_secs":    p.UptimeSecs,
+		"net_up":         p.NetUp,
+		"net_down":       p.NetDown,
 		"last_error":     p.LastError,
 		"xray_state":     p.XrayState,
 		"xray_error":     p.XrayError,
@@ -571,6 +580,8 @@ func (s *NodeService) UpdateHeartbeat(id int, p HeartbeatPatch) error {
 		now := time.Unix(p.LastHeartbeat, 0)
 		nodeMetrics.append(nodeMetricKey(id, "cpu"), now, p.CpuPct)
 		nodeMetrics.append(nodeMetricKey(id, "mem"), now, p.MemPct)
+		nodeMetrics.append(nodeMetricKey(id, "netUp"), now, float64(p.NetUp))
+		nodeMetrics.append(nodeMetricKey(id, "netDown"), now, float64(p.NetDown))
 	}
 	return nil
 }
@@ -823,6 +834,10 @@ func (s *NodeService) probe(ctx context.Context, n *model.Node, proxyURL string)
 			PanelVersion string `json:"panelVersion"`
 			PanelGuid    string `json:"panelGuid"`
 			Uptime       uint64 `json:"uptime"`
+			NetIO        struct {
+				Up   uint64 `json:"up"`
+				Down uint64 `json:"down"`
+			} `json:"netIO"`
 		} `json:"obj"`
 	}
 	if err := json.NewDecoder(resp.Body).Decode(&envelope); err != nil {
@@ -844,6 +859,8 @@ func (s *NodeService) probe(ctx context.Context, n *model.Node, proxyURL string)
 	patch.PanelVersion = o.PanelVersion
 	patch.Guid = o.PanelGuid
 	patch.UptimeSecs = o.Uptime
+	patch.NetUp = o.NetIO.Up
+	patch.NetDown = o.NetIO.Down
 	return patch, nil
 }

@@ -0,0 +1,43 @@
+package service
+
+import (
+	"crypto/x509"
+	"encoding/pem"
+	"strings"
+
+	"github.com/mhsanaei/3x-ui/v3/internal/util/common"
+)
+
+// NodeMtlsCaCert returns the PEM of this panel's node-auth CA certificate (the
+// public half) to copy into a node's mTLS trust setting, minting the CA and the
+// master client cert on first call so the panel is ready to present a client
+// certificate to mtls nodes.
+func (s *NodeService) NodeMtlsCaCert() (string, error) {
+	settings := SettingService{}
+	ca, err := settings.EnsureNodeMtlsCA()
+	if err != nil {
+		return "", err
+	}
+	if _, err := settings.EnsureMasterClientCert(); err != nil {
+		return "", err
+	}
+	return string(ca.CertPEM), nil
+}
+
+// SetNodeMtlsTrustCA stores the CA certificate this panel trusts for incoming
+// node-API client certificates. An empty value clears it (mTLS off). A
+// non-empty value must be a PEM certificate (fail closed). Takes effect on the
+// next panel restart, when the listener's ClientCAs is rebuilt.
+func (s *NodeService) SetNodeMtlsTrustCA(caPem string) error {
+	caPem = strings.TrimSpace(caPem)
+	if caPem != "" {
+		block, _ := pem.Decode([]byte(caPem))
+		if block == nil || block.Type != "CERTIFICATE" {
+			return common.NewError("trust CA must be a PEM-encoded certificate")
+		}
+		if _, err := x509.ParseCertificate(block.Bytes); err != nil {
+			return common.NewError("invalid trust CA certificate: " + err.Error())
+		}
+	}
+	return (&SettingService{}).setString(settingNodeMtlsClientCA, caPem)
+}
@@ -0,0 +1,111 @@
+package service
+
+import (
+	"crypto/x509"
+	"encoding/pem"
+	"testing"
+
+	"github.com/go-playground/validator/v10"
+	"github.com/mhsanaei/3x-ui/v3/internal/database/model"
+)
+
+func TestNormalizeKeepsMtls(t *testing.T) {
+	s := &NodeService{}
+	cases := []struct {
+		name     string
+		in       model.Node
+		wantMode string
+		wantErr  bool
+	}{
+		{"mtls over https preserved", model.Node{Name: "n", Address: "node.example.com", Port: 2053, Scheme: "https", TlsVerifyMode: "mtls"}, "mtls", false},
+		{"mtls over http rejected", model.Node{Name: "n", Address: "node.example.com", Port: 2053, Scheme: "http", TlsVerifyMode: "mtls"}, "", true},
+		{"unknown mode clamped to verify", model.Node{Name: "n", Address: "node.example.com", Port: 2053, Scheme: "https", TlsVerifyMode: "bogus"}, "verify", false},
+	}
+	for _, c := range cases {
+		t.Run(c.name, func(t *testing.T) {
+			n := c.in
+			err := s.normalize(&n)
+			if c.wantErr {
+				if err == nil {
+					t.Fatal("expected an error")
+				}
+				return
+			}
+			if err != nil {
+				t.Fatalf("normalize: %v", err)
+			}
+			if n.TlsVerifyMode != c.wantMode {
+				t.Fatalf("TlsVerifyMode = %q, want %q", n.TlsVerifyMode, c.wantMode)
+			}
+		})
+	}
+}
+
+func TestNodeTlsVerifyModeValidatorAcceptsMtls(t *testing.T) {
+	v := validator.New(validator.WithRequiredStructEnabled())
+	base := model.Node{Name: "n", Address: "node.example.com", Port: 2053, Scheme: "https", ApiToken: "t"}
+
+	for _, m := range []string{"verify", "skip", "pin", "mtls"} {
+		n := base
+		n.TlsVerifyMode = m
+		if err := v.Struct(n); err != nil {
+			t.Fatalf("validator rejected valid TlsVerifyMode %q: %v", m, err)
+		}
+	}
+	bad := base
+	bad.TlsVerifyMode = "bogus"
+	if err := v.Struct(bad); err == nil {
+		t.Fatal("validator must reject an unknown TlsVerifyMode")
+	}
+}
+
+func TestNodeMtlsCaCert(t *testing.T) {
+	_ = setupSettingMtlsDB(t)
+
+	got, err := (&NodeService{}).NodeMtlsCaCert()
+	if err != nil {
+		t.Fatalf("NodeMtlsCaCert: %v", err)
+	}
+	block, _ := pem.Decode([]byte(got))
+	if block == nil || block.Type != "CERTIFICATE" {
+		t.Fatalf("NodeMtlsCaCert must return a CERTIFICATE PEM, got %q", got)
+	}
+	cert, err := x509.ParseCertificate(block.Bytes)
+	if err != nil {
+		t.Fatalf("parse returned cert: %v", err)
+	}
+	if !cert.IsCA {
+		t.Fatal("NodeMtlsCaCert must return the CA certificate (IsCA)")
+	}
+}
+
+func TestSetNodeMtlsTrustCA(t *testing.T) {
+	_ = setupSettingMtlsDB(t)
+	ns := &NodeService{}
+	settings := SettingService{}
+
+	ca, err := settings.EnsureNodeMtlsCA()
+	if err != nil {
+		t.Fatalf("EnsureNodeMtlsCA: %v", err)
+	}
+
+	if err := ns.SetNodeMtlsTrustCA(string(ca.CertPEM)); err != nil {
+		t.Fatalf("SetNodeMtlsTrustCA(valid): %v", err)
+	}
+	pool, err := settings.NodeMtlsClientCAPool()
+	if err != nil || pool == nil {
+		t.Fatalf("valid trust CA must persist + build a pool: pool=%v err=%v", pool, err)
+	}
+
+	if err := ns.SetNodeMtlsTrustCA("not a certificate"); err == nil {
+		t.Fatal("invalid PEM must be rejected (fail closed)")
+	}
+
+	if err := ns.SetNodeMtlsTrustCA(""); err != nil {
+		t.Fatalf("clearing the trust CA must be allowed: %v", err)
+	}
+	pool, _ = settings.NodeMtlsClientCAPool()
+	if pool != nil {
+		t.Fatal("cleared trust CA must yield a nil pool (mTLS off)")
+	}
+}
@@ -0,0 +1,72 @@
+package service
+
+import (
+	"context"
+	"net/http"
+	"net/http/httptest"
+	"net/url"
+	"slices"
+	"strconv"
+	"testing"
+	"time"
+
+	"github.com/mhsanaei/3x-ui/v3/internal/database"
+	"github.com/mhsanaei/3x-ui/v3/internal/database/model"
+)
+
+func TestProbeParsesNetIO(t *testing.T) {
+	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
+		w.Header().Set("Content-Type", "application/json")
+		_, _ = w.Write([]byte(`{"success":true,"obj":{"cpu":5,"mem":{"current":1,"total":2},"netIO":{"up":1000,"down":2000},"panelGuid":"g","uptime":42}}`))
+	}))
+	defer srv.Close()
+
+	u, err := url.Parse(srv.URL)
+	if err != nil {
+		t.Fatalf("parse url: %v", err)
+	}
+	port, _ := strconv.Atoi(u.Port())
+	n := &model.Node{Scheme: "http", Address: u.Hostname(), Port: port, BasePath: "/", ApiToken: "t", AllowPrivateAddress: true}
+
+	patch, err := (&NodeService{}).probe(context.Background(), n, "")
+	if err != nil {
+		t.Fatalf("probe: %v", err)
+	}
+	if patch.NetUp != 1000 || patch.NetDown != 2000 {
+		t.Fatalf("net throughput not parsed from status: up=%d down=%d", patch.NetUp, patch.NetDown)
+	}
+}
+
+func TestUpdateHeartbeatStoresNetMetrics(t *testing.T) {
+	_ = setupSettingMtlsDB(t)
+	s := &NodeService{}
+
+	n := &model.Node{Name: "netn", Address: "1.2.3.4", Port: 2053, Scheme: "https", ApiToken: "t"}
+	if err := database.GetDB().Create(n).Error; err != nil {
+		t.Fatalf("create node: %v", err)
+	}
+
+	patch := HeartbeatPatch{Status: "online", LastHeartbeat: time.Now().Unix(), NetUp: 111, NetDown: 222}
+	if err := s.UpdateHeartbeat(n.Id, patch); err != nil {
+		t.Fatalf("UpdateHeartbeat: %v", err)
+	}
+
+	var got model.Node
+	if err := database.GetDB().First(&got, n.Id).Error; err != nil {
+		t.Fatalf("reload node: %v", err)
+	}
+	if got.NetUp != 111 || got.NetDown != 222 {
+		t.Fatalf("net columns not persisted: up=%d down=%d", got.NetUp, got.NetDown)
+	}
+	if len(s.AggregateNodeMetric(n.Id, "netUp", 2, 60)) == 0 {
+		t.Fatal("expected netUp history points after an online heartbeat")
+	}
+}
+
+func TestNodeMetricKeysIncludesNet(t *testing.T) {
+	for _, k := range []string{"netUp", "netDown"} {
+		if !slices.Contains(NodeMetricKeys, k) {
+			t.Fatalf("NodeMetricKeys must include %q so the history endpoint accepts it", k)
+		}
+	}
+}
@@ -29,15 +29,24 @@ import (
 var xrayTemplateConfig string

 var defaultValueMap = map[string]string{
-	"xrayTemplateConfig":          xrayTemplateConfig,
-	"webListen":                   "",
-	"webDomain":                   "",
-	"webPort":                     "2053",
-	"webCertFile":                 "",
-	"webKeyFile":                  "",
-	"secret":                      random.Seq(32),
-	"panelGuid":                   uuid.NewString(),
-	"apiToken":                    "",
+	"xrayTemplateConfig": xrayTemplateConfig,
+	"webListen":          "",
+	"webDomain":          "",
+	"webPort":            "2053",
+	"webCertFile":        "",
+	"webKeyFile":         "",
+	"secret":             random.Seq(32),
+	"panelGuid":          uuid.NewString(),
+	"apiToken":           "",
+	// Node mTLS material (opt-in). All default empty: the CA + master client
+	// cert are minted lazily on first use, and the node-side trust CA is pasted
+	// in by the operator. Kept out of entity.AllSetting so private keys never
+	// reach the settings UI/export.
+	"nodeMtlsCaCertPem":           "",
+	"nodeMtlsCaKeyPem":            "",
+	"nodeMtlsClientCertPem":       "",
+	"nodeMtlsClientKeyPem":        "",
+	"nodeMtlsClientCAPem":         "",
 	"webBasePath":                 normalizeBasePath(getEnv("XUI_INIT_WEB_BASE_PATH", "/")),
 	"sessionMaxAge":               "360",
 	"trustedProxyCIDRs":           "127.0.0.1/32,::1/128",
@@ -0,0 +1,106 @@
+package service
+
+import (
+	"crypto/x509"
+
+	"github.com/mhsanaei/3x-ui/v3/internal/util/common"
+	"github.com/mhsanaei/3x-ui/v3/internal/util/crypto"
+)
+
+const (
+	settingNodeMtlsCaCert     = "nodeMtlsCaCertPem"
+	settingNodeMtlsCaKey      = "nodeMtlsCaKeyPem"
+	settingNodeMtlsClientCert = "nodeMtlsClientCertPem"
+	settingNodeMtlsClientKey  = "nodeMtlsClientKeyPem"
+	settingNodeMtlsClientCA   = "nodeMtlsClientCAPem"
+)
+
+// EnsureNodeMtlsCA returns this panel's node-auth CA, minting and persisting it
+// on first use and reusing the stored pair thereafter. The CA private key never
+// leaves the panel.
+func (s *SettingService) EnsureNodeMtlsCA() (crypto.CertKeyPEM, error) {
+	certPem, err := s.getString(settingNodeMtlsCaCert)
+	if err != nil {
+		return crypto.CertKeyPEM{}, err
+	}
+	keyPem, err := s.getString(settingNodeMtlsCaKey)
+	if err != nil {
+		return crypto.CertKeyPEM{}, err
+	}
+	if certPem != "" && keyPem != "" {
+		return crypto.CertKeyPEM{CertPEM: []byte(certPem), KeyPEM: []byte(keyPem)}, nil
+	}
+	// Fail closed on a half-present pair: regenerating here would silently rotate
+	// the CA and break trust on nodes that already hold the old cert. Only mint
+	// when neither half exists (first use).
+	if certPem != "" || keyPem != "" {
+		return crypto.CertKeyPEM{}, common.NewError("node mTLS CA is incomplete: one of cert/key is missing; refusing to regenerate")
+	}
+	ca, err := crypto.GenerateNodeCA("3x-ui node mTLS CA")
+	if err != nil {
+		return crypto.CertKeyPEM{}, err
+	}
+	if err := s.saveSetting(settingNodeMtlsCaCert, string(ca.CertPEM)); err != nil {
+		return crypto.CertKeyPEM{}, err
+	}
+	if err := s.saveSetting(settingNodeMtlsCaKey, string(ca.KeyPEM)); err != nil {
+		return crypto.CertKeyPEM{}, err
+	}
+	return ca, nil
+}
+
+// EnsureMasterClientCert returns the client certificate this panel presents when
+// calling its nodes over mTLS, issuing it from the node CA on first use and
+// reusing the stored pair thereafter.
+func (s *SettingService) EnsureMasterClientCert() (crypto.CertKeyPEM, error) {
+	certPem, err := s.getString(settingNodeMtlsClientCert)
+	if err != nil {
+		return crypto.CertKeyPEM{}, err
+	}
+	keyPem, err := s.getString(settingNodeMtlsClientKey)
+	if err != nil {
+		return crypto.CertKeyPEM{}, err
+	}
+	if certPem != "" && keyPem != "" {
+		return crypto.CertKeyPEM{CertPEM: []byte(certPem), KeyPEM: []byte(keyPem)}, nil
+	}
+	// Half a stored pair signals corrupted settings; reissuing would rotate the
+	// master client credential (and indirectly the CA). Only mint on first use.
+	if certPem != "" || keyPem != "" {
+		return crypto.CertKeyPEM{}, common.NewError("master client cert is incomplete: one of cert/key is missing; refusing to reissue")
+	}
+	ca, err := s.EnsureNodeMtlsCA()
+	if err != nil {
+		return crypto.CertKeyPEM{}, err
+	}
+	client, err := crypto.IssueClientCert(ca, "3x-ui master")
+	if err != nil {
+		return crypto.CertKeyPEM{}, err
+	}
+	if err := s.saveSetting(settingNodeMtlsClientCert, string(client.CertPEM)); err != nil {
+		return crypto.CertKeyPEM{}, err
+	}
+	if err := s.saveSetting(settingNodeMtlsClientKey, string(client.KeyPEM)); err != nil {
+		return crypto.CertKeyPEM{}, err
+	}
+	return client, nil
+}
+
+// NodeMtlsClientCAPool builds the trust pool used as the panel listener's
+// ClientCAs for incoming node-API client certificates. It returns (nil, nil)
+// when no trust CA is configured, so mTLS stays off and the listener behaves
+// exactly as before.
+func (s *SettingService) NodeMtlsClientCAPool() (*x509.CertPool, error) {
+	caPem, err := s.getString(settingNodeMtlsClientCA)
+	if err != nil {
+		return nil, err
+	}
+	if caPem == "" {
+		return nil, nil
+	}
+	pool := x509.NewCertPool()
+	if !pool.AppendCertsFromPEM([]byte(caPem)) {
+		return nil, common.NewError("nodeMtlsClientCAPem is not a valid certificate")
+	}
+	return pool, nil
+}
@@ -0,0 +1,125 @@
+package service
+
+import (
+	"bytes"
+	"crypto/x509"
+	"encoding/pem"
+	"path/filepath"
+	"testing"
+
+	"github.com/mhsanaei/3x-ui/v3/internal/database"
+)
+
+func setupSettingMtlsDB(t *testing.T) *SettingService {
+	t.Helper()
+	dbDir := t.TempDir()
+	t.Setenv("XUI_DB_FOLDER", dbDir)
+	if err := database.InitDB(filepath.Join(dbDir, "x-ui.db")); err != nil {
+		t.Fatalf("InitDB: %v", err)
+	}
+	t.Cleanup(func() { _ = database.CloseDB() })
+	return &SettingService{}
+}
+
+func TestEnsureNodeMtlsCA_Idempotent(t *testing.T) {
+	s := setupSettingMtlsDB(t)
+
+	first, err := s.EnsureNodeMtlsCA()
+	if err != nil {
+		t.Fatalf("EnsureNodeMtlsCA (first): %v", err)
+	}
+	block, _ := pem.Decode(first.CertPEM)
+	if block == nil {
+		t.Fatal("CA cert is not valid PEM")
+	}
+	caCert, err := x509.ParseCertificate(block.Bytes)
+	if err != nil {
+		t.Fatalf("parse CA cert: %v", err)
+	}
+	if !caCert.IsCA {
+		t.Fatal("stored CA must have IsCA=true")
+	}
+
+	second, err := s.EnsureNodeMtlsCA()
+	if err != nil {
+		t.Fatalf("EnsureNodeMtlsCA (second): %v", err)
+	}
+	if !bytes.Equal(first.CertPEM, second.CertPEM) || !bytes.Equal(first.KeyPEM, second.KeyPEM) {
+		t.Fatal("EnsureNodeMtlsCA must be idempotent: second call returned different PEMs")
+	}
+}
+
+func TestEnsureMasterClientCert_VerifiesAndIdempotent(t *testing.T) {
+	s := setupSettingMtlsDB(t)
+
+	ca, err := s.EnsureNodeMtlsCA()
+	if err != nil {
+		t.Fatalf("EnsureNodeMtlsCA: %v", err)
+	}
+	client, err := s.EnsureMasterClientCert()
+	if err != nil {
+		t.Fatalf("EnsureMasterClientCert: %v", err)
+	}
+
+	cblock, _ := pem.Decode(client.CertPEM)
+	if cblock == nil {
+		t.Fatal("client cert is not valid PEM")
+	}
+	leaf, err := x509.ParseCertificate(cblock.Bytes)
+	if err != nil {
+		t.Fatalf("parse client cert: %v", err)
+	}
+	caBlock, _ := pem.Decode(ca.CertPEM)
+	roots := x509.NewCertPool()
+	roots.AddCert(mustParse(t, caBlock.Bytes))
+	if _, err := leaf.Verify(x509.VerifyOptions{
+		Roots:     roots,
+		KeyUsages: []x509.ExtKeyUsage{x509.ExtKeyUsageClientAuth},
+	}); err != nil {
+		t.Fatalf("master client cert must verify against the node CA for client auth: %v", err)
+	}
+
+	again, err := s.EnsureMasterClientCert()
+	if err != nil {
+		t.Fatalf("EnsureMasterClientCert (second): %v", err)
+	}
+	if !bytes.Equal(client.CertPEM, again.CertPEM) || !bytes.Equal(client.KeyPEM, again.KeyPEM) {
+		t.Fatal("EnsureMasterClientCert must be idempotent")
+	}
+}
+
+func TestNodeMtlsClientCAPool(t *testing.T) {
+	s := setupSettingMtlsDB(t)
+
+	pool, err := s.NodeMtlsClientCAPool()
+	if err != nil {
+		t.Fatalf("NodeMtlsClientCAPool (unset): %v", err)
+	}
+	if pool != nil {
+		t.Fatal("with no trust CA configured, the pool must be nil (mTLS off; listener unchanged)")
+	}
+
+	ca, err := s.EnsureNodeMtlsCA()
+	if err != nil {
+		t.Fatalf("EnsureNodeMtlsCA: %v", err)
+	}
+	if err := s.setString("nodeMtlsClientCAPem", string(ca.CertPEM)); err != nil {
+		t.Fatalf("set trust CA: %v", err)
+	}
+	pool, err = s.NodeMtlsClientCAPool()
+	if err != nil {
+		t.Fatalf("NodeMtlsClientCAPool (set): %v", err)
+	}
+	if pool == nil {
+		t.Fatal("with a trust CA configured, the pool must be non-nil")
+	}
+}
+
+func mustParse(t *testing.T, der []byte) *x509.Certificate {
+	t.Helper()
+	c, err := x509.ParseCertificate(der)
+	if err != nil {
+		t.Fatalf("ParseCertificate: %v", err)
+	}
+	return c
+}