Files
3x-ui/internal/sub/json_routing.go
T
DIMFLIX 2dd903ea8e feat(sub): bake Happ/INCY routing profiles into the JSON subscription (#6402)
* feat(sub): parse generic Happ/INCY routing payloads for the JSON subscription

Accepts the routing-rules format emitted for Happ and INCY (inline JSON,
happ:// or incy:// deeplink, or a remote https:// URL resolved through the
existing remote routing cache). The JSON subscription will bake these
rules into its documents so header-ignoring clients still get routing.

* feat(sub): bake Happ/INCY routing profiles into JSON subscription documents

When subJsonRoutingRules is set, every emitted document (per-inbound and
balancer alike) carries the profile's dns and routing rules baked in, so
header-ignoring clients like Happ and INCY still get routing; the legacy
simple-rules merge only applies when no profile is set. The balancer
document builder keeps rewriting proxy-tag rules to the balancer.

* feat(sub): add the subJsonRoutingRules setting

Plumbed from the settings store through the subscription server into
SubJsonService, so admins can set a routing profile once and every JSON
subscription document carries it.

* chore(api): regenerate OpenAPI artifacts for subJsonRoutingRules

* feat(web): routing profile editor for the JSON subscription

A textarea inside the JSON card accepts the routing profile (inline JSON,
happ/incy deeplink, or https URL) with a remote-source badge; the badge
helper moves to a shared module. Keys added to all 13 locales.

* fix(sub): warm and lazily resolve the baked JSON routing source

The routing profile was resolved once at service construction: a remote
URL that was cold at that moment baked default routing forever, and the
cron job never warmed it. The job now warms the subJsonRoutingRules URL,
and the profile resolves per request with an in-memory memo (a failed
resolve is not cached), so a warmed cache takes effect without a restart.

* feat(sub): fall back to the JSON routing profile for the Routing header

Happ and INCY download the geo files a routing profile references
through the Routing response header. When the Happ header setting was
blank the header stayed unset, and clients fetched no geo files even
though a JSON routing profile was configured. A blank setting now falls
back to the JSON profile: happ/incy deeplinks pass through, inline JSON
and remote URLs are normalized to a happ:// deeplink; an unusable or
oversized value leaves the header unset. Locale captions mention the
fallback.

* fix(sub): pass routingRules arg at call sites added by main

Main gained four NewSubJsonService call sites after this branch forked;
update them to the five-arg signature so internal/sub builds again.

* fix(sub): address code review findings on the baked JSON routing

The memoised baked template never invalidated, so an edited remote
profile kept serving the superseded dns/routing subtrees until a panel
restart; bakedTemplate now re-resolves the spec per request and rebuilds
only when the payload actually changed (regression-tested).

subJsonRoutingRules shared the happ persistence row with
subRoutingRules, so only the last-written setting survived a restart;
it now resolves under its own jsonhapp kind with the same validation
and size caps. The setting also joins validateSettingsURLs, so remote
values are canonicalised and bad URLs are rejected on save.

Also: drop the unreachable half of the remote-source guard, cut the
overlong comment blocks to the two-line convention, and deduplicate
remoteSourceBadge in the General tab. Merges upstream/main (call sites
for the widened NewSubJsonService signature).

* style(sub): gofumpt the json_routing imports

* fix(sub): accept happ add/ deeplinks and bound the routing warning

The baked-JSON routing parser only recognised happ://routing/onadd/, but
normalizeHappRouting treats happ://routing/add/ as an equally valid routing
deeplink. An operator pasting the add/ form got the Routing header set, so
the panel looked configured, while every JSON subscription document silently
carried the default routing instead of their profile.

resolveJsonRoutingSpec logged one warning per call and bakedTemplate calls it
once per emitted document, so a single fetch of an unusable profile wrote one
identical warning per document. On the public subscription server that floods
the 10240-entry buffer the panel's log view reads, evicting real entries. Log
only when the message changes, and reset on a successful resolve so a profile
that recovers and fails again is still reported.

Also resolve the template once in buildBalancerConfig: two resolves could
straddle a profile refresh and pair one revision's dns with the other's
routing.
2026-09-10 17:05:54 +02:00

367 lines
11 KiB
Go

package sub
import (
"encoding/json"
"errors"
"fmt"
"maps"
"slices"
"strings"
"sync/atomic"
"github.com/mhsanaei/3x-ui/v3/internal/logger"
"github.com/mhsanaei/3x-ui/v3/internal/util/common"
)
// jsonRoutingSpec is the canonical form of the generic Happ/INCY routing
// payload (inline JSON, happ:// or incy:// deeplink, or remote URL).
type jsonRoutingSpec struct {
DomainStrategy string
RemoteDNSDomain string
RemoteDNSIP string
DomesticDNSDomain string
DomesticDNSIP string
DnsHosts map[string]string
RouteOrder []string // e.g. {"block","proxy","direct"}; default {"block","direct","proxy"}
DirectSites []string
DirectIp []string
ProxySites []string
ProxyIp []string
BlockSites []string
BlockIp []string
}
func (s jsonRoutingSpec) empty() bool {
return s.DomainStrategy == "" && s.RemoteDNSDomain == "" && s.RemoteDNSIP == "" &&
s.DomesticDNSDomain == "" && s.DomesticDNSIP == "" && len(s.DnsHosts) == 0 &&
len(s.RouteOrder) == 0 && len(s.DirectSites) == 0 && len(s.DirectIp) == 0 &&
len(s.ProxySites) == 0 && len(s.ProxyIp) == 0 && len(s.BlockSites) == 0 && len(s.BlockIp) == 0
}
// equal reports whether two specs carry the same routing payload, so a
// rebuilt template only replaces the memoised one when the profile changed.
func (s jsonRoutingSpec) equal(other jsonRoutingSpec) bool {
return s.DomainStrategy == other.DomainStrategy &&
s.RemoteDNSDomain == other.RemoteDNSDomain && s.RemoteDNSIP == other.RemoteDNSIP &&
s.DomesticDNSDomain == other.DomesticDNSDomain && s.DomesticDNSIP == other.DomesticDNSIP &&
maps.Equal(s.DnsHosts, other.DnsHosts) && slices.Equal(s.RouteOrder, other.RouteOrder) &&
slices.Equal(s.DirectSites, other.DirectSites) && slices.Equal(s.DirectIp, other.DirectIp) &&
slices.Equal(s.ProxySites, other.ProxySites) && slices.Equal(s.ProxyIp, other.ProxyIp) &&
slices.Equal(s.BlockSites, other.BlockSites) && slices.Equal(s.BlockIp, other.BlockIp)
}
// Both happ forms appear in the wild; normalizeHappRouting accepts each.
var jsonRoutingDeeplinkPrefixes = []string{
"happ://routing/onadd/", "happ://routing/add/", "incy://routing/onadd/",
}
// bakedTemplate resolves once per emitted document, so an unusable profile
// must not write one identical warning per document on every public fetch.
var lastJsonRoutingWarning atomic.Value
// resolveJsonRoutingSpec parses the routing payload, degrading to an empty
// spec on error — a bad setting must never take the subscription server down.
func resolveJsonRoutingSpec(raw string) jsonRoutingSpec {
spec, remote, err := parseJsonRoutingSpec(raw)
if err != nil {
warning := "subJsonRoutingRules: " + err.Error()
if remote {
warning = "subJsonRoutingRules: remote source unavailable, emitting default routing"
}
if previous, _ := lastJsonRoutingWarning.Load().(string); previous != warning {
lastJsonRoutingWarning.Store(warning)
logger.Warning(warning)
}
return jsonRoutingSpec{}
}
lastJsonRoutingWarning.Store("")
return spec
}
// jsonRoutingHeaderSource turns the JSON routing setting into a Routing header
// value; blank means unusable, and the header then stays unset.
func jsonRoutingHeaderSource(raw string) string {
trimmed := strings.TrimSpace(raw)
if trimmed == "" {
return ""
}
if strings.HasPrefix(trimmed, "incy://") {
_, rest, ok := cutAnyPrefix(trimmed, jsonRoutingDeeplinkPrefixes)
if !ok {
return ""
}
decoded, err := decodeRoutingBase64(rest)
if err != nil {
return ""
}
if _, err := validateAndCompactJSONObject(decoded); err != nil || len(trimmed) > remoteRoutingHappMaxValue {
return ""
}
return trimmed
}
resolved, _, err := resolveRoutingSource(remoteRoutingJson, trimmed)
if err != nil {
return ""
}
content, err := normalizeHappRouting([]byte(resolved))
if err != nil || len(content) > remoteRoutingHappMaxValue {
return ""
}
return content
}
// parseJsonRoutingSpec resolves raw (inline JSON, happ:// or incy:// deeplink,
// or https:// URL) into a spec; the caller degrades on error, never fails.
func parseJsonRoutingSpec(raw string) (jsonRoutingSpec, bool, error) {
trimmed := strings.TrimSpace(raw)
if trimmed == "" {
return jsonRoutingSpec{}, false, nil
}
if _, remote, err := common.ParseRemoteRoutingURL(trimmed); remote {
if err != nil {
return jsonRoutingSpec{}, true, err
}
resolved, remote, err := resolveRoutingSource(remoteRoutingJson, trimmed)
if err != nil || !remote {
return jsonRoutingSpec{}, true, err
}
trimmed = resolved
}
payload := []byte(trimmed)
if _, rest, ok := cutAnyPrefix(trimmed, jsonRoutingDeeplinkPrefixes); ok {
decoded, err := decodeRoutingBase64(rest)
if err != nil {
return jsonRoutingSpec{}, false, fmt.Errorf("invalid routing deeplink payload: %w", err)
}
payload = decoded
} else if !strings.HasPrefix(trimmed, "{") {
return jsonRoutingSpec{}, false, errors.New("routing payload must be a JSON object or a happ/incy deeplink")
}
var object map[string]any
if err := json.Unmarshal(payload, &object); err != nil {
return jsonRoutingSpec{}, false, fmt.Errorf("invalid routing payload JSON: %w", err)
}
if object == nil {
return jsonRoutingSpec{}, false, errors.New("routing payload must be a JSON object")
}
spec, err := buildJsonRoutingSpec(object)
if err != nil {
return jsonRoutingSpec{}, false, err
}
return spec, false, nil
}
func cutAnyPrefix(s string, prefixes []string) (string, string, bool) {
for _, prefix := range prefixes {
if after, ok := strings.CutPrefix(s, prefix); ok {
return prefix, after, true
}
}
return "", "", false
}
func buildJsonRoutingSpec(object map[string]any) (jsonRoutingSpec, error) {
spec := jsonRoutingSpec{}
var err error
if spec.DomainStrategy, err = routingString(object, "DomainStrategy"); err != nil {
return spec, err
}
if spec.RemoteDNSDomain, err = routingString(object, "RemoteDNSDomain"); err != nil {
return spec, err
}
if spec.RemoteDNSIP, err = routingString(object, "RemoteDNSIP"); err != nil {
return spec, err
}
if spec.DomesticDNSDomain, err = routingString(object, "DomesticDNSDomain"); err != nil {
return spec, err
}
if spec.DomesticDNSIP, err = routingString(object, "DomesticDNSIP"); err != nil {
return spec, err
}
if spec.DirectSites, err = routingList(object, "DirectSites"); err != nil {
return spec, err
}
if spec.DirectIp, err = routingList(object, "DirectIp"); err != nil {
return spec, err
}
if spec.ProxySites, err = routingList(object, "ProxySites"); err != nil {
return spec, err
}
if spec.ProxyIp, err = routingList(object, "ProxyIp"); err != nil {
return spec, err
}
if spec.BlockSites, err = routingList(object, "BlockSites"); err != nil {
return spec, err
}
if spec.BlockIp, err = routingList(object, "BlockIp"); err != nil {
return spec, err
}
if spec.DnsHosts, err = routingHosts(object, "DnsHosts"); err != nil {
return spec, err
}
order, err := routingString(object, "RouteOrder")
if err != nil {
return spec, err
}
for _, segment := range strings.Split(order, "-") {
switch segment {
case "block", "proxy", "direct":
spec.RouteOrder = append(spec.RouteOrder, segment)
}
}
return spec, nil
}
func routingString(object map[string]any, key string) (string, error) {
value, ok := object[key]
if !ok || value == nil {
return "", nil
}
text, ok := value.(string)
if !ok {
return "", fmt.Errorf("routing field %q must be a string", key)
}
return text, nil
}
func routingList(object map[string]any, key string) ([]string, error) {
value, ok := object[key]
if !ok || value == nil {
return nil, nil
}
entries, ok := value.([]any)
if !ok {
return nil, fmt.Errorf("routing field %q must be an array of strings", key)
}
list := make([]string, 0, len(entries))
for _, entry := range entries {
text, ok := entry.(string)
if !ok {
return nil, fmt.Errorf("routing field %q must be an array of strings", key)
}
list = append(list, text)
}
return list, nil
}
func routingHosts(object map[string]any, key string) (map[string]string, error) {
value, ok := object[key]
if !ok || value == nil {
return nil, nil
}
raw, ok := value.(map[string]any)
if !ok {
return nil, fmt.Errorf("routing field %q must be a string-to-string map", key)
}
hosts := make(map[string]string, len(raw))
for name, entry := range raw {
address, ok := entry.(string)
if !ok {
return nil, fmt.Errorf("routing field %q must be a string-to-string map", key)
}
hosts[name] = address
}
return hosts, nil
}
func routeOrderGroups(order []string) []string {
if len(order) == 0 {
return []string{"block", "direct", "proxy"}
}
return order
}
// applyJsonRouting patches the base template with the spec's dns and routing
// subtrees, mirroring the njs patcher panel admins previously ran behind nginx.
func applyJsonRouting(configJson map[string]any, spec jsonRoutingSpec) {
domestic := spec.DomesticDNSDomain
if domestic == "" {
ip := spec.DomesticDNSIP
if ip == "" {
ip = "77.88.8.8"
}
domestic = "https://" + ip + "/dns-query"
}
remote := spec.RemoteDNSDomain
if remote == "" {
ip := spec.RemoteDNSIP
if ip == "" {
ip = "8.8.8.8"
}
remote = "https://" + ip + "/dns-query"
}
dns := map[string]any{
"tag": "dns_out",
"queryStrategy": "UseIP",
"servers": []any{},
}
if len(spec.DirectSites) > 0 {
dns["servers"] = append(dns["servers"].([]any), map[string]any{
"address": domestic,
"domains": spec.DirectSites,
})
}
dns["servers"] = append(dns["servers"].([]any), map[string]any{
"address": remote,
"skipFallback": false,
})
if len(spec.DnsHosts) > 0 {
dns["hosts"] = spec.DnsHosts
}
domainStrategy := spec.DomainStrategy
if domainStrategy == "" {
domainStrategy = "IPIfNonMatch"
}
groups := map[string][]map[string]any{
"block": {
{"domain": stringList(spec.BlockSites), "outboundTag": "block"},
{"ip": stringList(spec.BlockIp), "outboundTag": "block"},
},
"direct": {
{"domain": stringList(spec.DirectSites), "outboundTag": "direct"},
{"ip": stringList(spec.DirectIp), "outboundTag": "direct"},
},
"proxy": {
{"domain": stringList(spec.ProxySites), "outboundTag": "proxy"},
{"ip": stringList(spec.ProxyIp), "outboundTag": "proxy"},
},
}
rules := make([]any, 0, len(routeOrderGroups(spec.RouteOrder))*2+1)
for _, group := range routeOrderGroups(spec.RouteOrder) {
for _, rule := range groups[group] {
var key string
if _, ok := rule["domain"]; ok {
key = "domain"
} else {
key = "ip"
}
if len(rule[key].([]string)) == 0 {
continue
}
entry := map[string]any{"type": "field", key: rule[key], "outboundTag": rule["outboundTag"]}
rules = append(rules, entry)
}
}
rules = append(rules, map[string]any{"type": "field", "network": "tcp,udp", "outboundTag": "proxy"})
configJson["dns"] = dns
configJson["routing"] = map[string]any{
"domainStrategy": domainStrategy,
"rules": rules,
}
}
func stringList(list []string) []string {
if len(list) == 0 {
return nil
}
return list
}