Files
3x-ui/internal/web/service/integration/custom_geo.go
T
Sanaei 41645255f1 refactor: focused service files, leaf subpackages, and an internal/ layout (#5167)
* refactor(service): split client.go into focused files

client.go had grown to 4455 lines mixing ~10 responsibilities. Split it
verbatim into cohesive same-package files (no behavior change):

  client.go            foundation: ClientService, ClientWithAttachments,
                       ClientCreatePayload, ErrClientNotInInbound, sqlInChunk
  client_locks.go      inbound mutation locks, delete tombstones, compactOrphans
  client_lookup.go     read-only lookups (GetByID, List, EffectiveFlow, ...)
  client_link.go       inbound association sync (SyncInbound, DetachInbound, ...)
  client_crud.go       single-client CRUD + validation + protocol defaults
  client_inbound_apply.go  low-level inbound-settings mutators + by-email setters
  client_bulk.go       bulk attach/detach/adjust/delete/create + DelDepleted
  client_traffic.go    traffic-reset paths
  client_groups.go     client group management
  client_paging.go     paged listing, filtering, sorting, summary

Every declaration moved unchanged (verified: identical func/type/const/var
signature set before vs after). Imports redistributed per file via goimports.
go build ./..., go vet, and go test ./web/service/... all pass.

* refactor(service): split inbound.go into focused files

inbound.go was 4100 lines. Split it verbatim into cohesive same-package
files (no behavior change):

  inbound.go             core inbound CRUD + InboundService (keeps pkg doc)
  inbound_protocol.go    protocol / stream capability helpers
  inbound_node.go        node/runtime/remote coordination + online tracking
  inbound_traffic.go     traffic accounting, reset, client stats
  inbound_client_ips.go  per-client IP tracking
  inbound_clients.go     client lookups within inbounds + copy-clients
  inbound_disable.go     auto-disable invalid inbounds/clients
  inbound_migration.go   DB migrations
  inbound_sublink.go     subscription link providers
  inbound_util.go        generic slice/string helpers

Identical func/type/const/var signature set before vs after; package doc
comment preserved on inbound.go. Imports redistributed via goimports.
Build, vet, and go test ./web/service/... all pass.

* refactor(service): split tgbot.go into focused files

tgbot.go was 3738 lines dominated by a 1246-line answerCallback. Split it
verbatim into cohesive same-package files (no behavior change):

  tgbot.go           lifecycle, bot setup, caches, small utils
  tgbot_router.go    incoming update / command / callback dispatch
  tgbot_send.go      outbound messaging primitives
  tgbot_client.go    client views, actions, subscription links
  tgbot_inbound.go   inbound listing / pickers
  tgbot_report.go    server usage, exhausted, online, backups, notifications

Identical func/type/const/var signature set before vs after. Imports
redistributed via goimports. Build, vet, and go test ./web/service/... pass.

* refactor(client): dedupe single-field by-email setters

ResetClientIpLimitByEmail, ResetClientExpiryTimeByEmail, and
ResetClientTrafficLimitByEmail shared an identical ~50-line body that
resolves the inbound by email, confirms the client exists, rewrites a
single-client settings payload, and delegates to UpdateInboundClient.

Extract that into applyClientFieldByEmail(inboundSvc, email, mutate) and
reduce each setter to a 3-line wrapper. Behavior is unchanged: same checks
and error strings, same single-client payload contract, same totalGB guard.

SetClientTelegramUserID (resolves by traffic id, different error text) and
ToggleClientEnableByEmail/SetClientEnableByEmail (different return shape and
a pre-read of the old state) intentionally keep their own bodies.

* refactor(service): extract panel/ subpackage

Move the panel-administration leaf services out of the flat service
package into web/service/panel/ (package panel):

  user.go         UserService (auth / 2FA / LDAP)
  panel.go        PanelService (restart / self-update) + version helpers
  panel_other.go  non-unix RestartPanel
  panel_unix.go   unix RestartPanel
  api_token.go    ApiTokenService
  websocket.go    WebSocketService
  panel_test.go   version/shellQuote unit tests

These are leaves: they depend on core (SettingService, Release) but no
core file references them, so the extraction creates no import cycle.
Core references are now qualified (service.SettingService, service.Release);
callers in main.go, web/web.go, and web/controller/* updated to panel.*.
Build, vet, and go test ./web/... pass.

* refactor(service): extract integration/ subpackage

Move the external-provider integration leaves into web/service/integration/
(package integration):

  warp.go        WarpService (Cloudflare WARP)
  nord.go        NordService (NordVPN)
  custom_geo.go  CustomGeoService (custom geo asset management)
  *_test.go      custom_geo / panel-proxy tests

These depend on core (SettingService, ServerService, XraySettingService) but
no core file references them. xray_setting.go stays in core because it calls
the unexported SettingService.saveSetting. The shared isBlockedIP SSRF helper
(used by core url_safety.go and by custom_geo) now has a small copy in each
package rather than being exported. Core references qualified; callers in
web/web.go, web/job/*, and web/controller/* updated to integration.*.
Build, vet, and go test ./web/... pass.

* refactor(service): extract tgbot/ subpackage

Move the Telegram bot (6 files + test) into web/service/tgbot/ (package
tgbot). It is a leaf: it embeds five core services (Inbound/Client/Setting/
Server/Xray) and the core never references it, so no import cycle.

To support the package boundary without changing behavior:
  - core exposes XrayProcess() *xray.Process so tgbot keeps calling the
    exact same running-process methods it used via the package-level `p`;
  - three core methods tgbot calls are exported: ClientService.checkIs-
    EnabledByEmail -> CheckIsEnabledByEmail, InboundService.getAllEmails ->
    GetAllEmails (callers updated in-package);
  - tgbot's embedded-field types and the few core type refs (Status,
    ClientCreatePayload, SanitizePublicHTTPURL) are now service-qualified.

Callers in main.go, web/web.go, web/job/*, and web/controller/* updated to
tgbot.*. Build, vet, and go test ./web/... pass.

* refactor(service): extract outbound/ subpackage

OutboundService (outbound.go) imports only neutral packages (config,
database, model, xray) and its production code is referenced by no core or
sibling service file — only by web/controller/xray_setting.go and
web/job/xray_traffic_job.go. Move it to web/service/outbound/ (package
outbound); no core qualification needed inside. Callers updated to outbound.*.

The one coupling was a tiny pure test helper, outboundsContainTag, used by
both outbound.go and the core outbound_subscription_test.go; it now has a
small copy in that test file rather than being shared across the boundary.
Build, vet, and go test ./web/... pass.

* refactor(util): move wireguard into its own subpackage

util/wireguard.go was the lone file of the root `util` package (24 lines,
one exported func GenerateWireguardKeypair), while every other util concern
lives in a focused subpackage (util/common, util/crypto, util/netsafe, ...).
Move it to util/wireguard/ (package wireguard) for consistency; its only
importer, web/service/integration/warp.go, is updated. The root `util`
package no longer exists.

* refactor(sub): drop redundant sub prefix from filenames

Inside package sub the subXxx.go prefix just repeats the package name
(like client_*.go did inside service). Rename for consistency; content and
type names are unchanged:

  subController.go    -> controller.go
  subService.go       -> service.go
  subClashService.go  -> clash_service.go
  subJsonService.go   -> json_service.go
  (+ matching _test.go files)

* refactor(controller): rename xui.go -> spa.go

XUIController serves the panel's single-page-app shell; spa.go names that
role plainly (the other controller files are domain-named). File rename only
— the type stays XUIController. api_docs_test.go keys route base paths by
filename, so its "xui.go" case is updated to "spa.go".

* refactor: move backend packages under internal/

Adopt the idiomatic Go application layout: the backend packages now live
under internal/ (a boundary the toolchain enforces), signalling private
implementation instead of a library-style flat root. No runtime behavior
changes — only import paths and a few build/config paths move.

Moved: config, database, logger, mtproto, sub, util, web, xray -> internal/.
main.go stays at the repo root and tools/openapigen stays under tools/ (both
still import internal/* because the internal rule keys off the module root).
The module path github.com/mhsanaei/3x-ui/v3 is unchanged; 149 .go files had
their import prefix rewritten to .../internal/<pkg>.

Couplings the Go compiler can't see, updated to the new layout:
  - frontend i18n imports of web/translation (react.ts, setup.components.ts)
  - vite outDir + eslint/tsconfig ignore globs -> internal/web/dist
  - Dockerfile COPY paths for web/dist and web/translation
  - locale.go os.DirFS("web") disk fallback -> "internal/web"
  - .gitignore and ci.yml go:embed stub for internal/web/dist
  - api_docs_test.go repo-root relative walk (one level deeper)
  - tools/openapigen filesystem package paths; ApiTokenView repointed to the
    web/service/panel subpackage and codegen regenerated (clears a stale
    type the ci.yml codegen check was failing on)

Verified: go build/vet/test (all packages), and frontend typecheck, lint,
vitest (478 tests), and production build into internal/web/dist.

* fix(config): keep test runs from writing logs into the source tree

GetLogFolder() returns a CWD-relative "./log" on Windows. Under `go test`
the working directory is each package's own folder, so InitLogger (called by
tests in web/job, web/service, xray, web/websocket) created stray log/
directories scattered through the source tree (e.g. internal/web/job/log/).

Redirect to a shared temp folder when testing.Testing() reports a test run.
Production behavior is unchanged: Windows still uses ./log next to the binary
and Linux /var/log/x-ui. The log files were always gitignored (*.log) and
never committed; this just stops the noise at the source.

* docs: move subscription-template guide out of root into docs/

sub_templates/ was a top-level folder holding only a README and no actual
templates (3x-ui ships none by design), referenced nowhere and unlinked from
any doc — it read like an empty placeholder cluttering the repo root.

Move the guide to docs/custom-subscription-templates.md (a proper docs home),
reword its intro to read as documentation rather than a folder note, link it
from the Features list in README.md, and drop the empty sub_templates/ folder.

* fix: update stale web/ path references after the internal/ move

The internal/ migration rewrote Go import paths but left some references to
the old top-level layout in docs, comments, and a few runtime disk paths.

Functional (dev-mode only): the disk-serving fallbacks that read the Vite
build from disk when running from source still pointed at web/dist/, which
moved to internal/web/dist/ — so `os.DirFS`/`os.Stat`/`os.ReadFile` in
internal/web/web.go and internal/sub/{sub,controller}.go are corrected.
Production was unaffected (it serves the embedded FS; verified by the Docker
build), but `go run` with a live frontend build silently fell back to embed.

Docs/comments: frontend/README.md, CONTRIBUTING.md, the claude-issue-bot and
release workflows, the openapigen -root help text, and assorted Go comments
now reference internal/web, internal/database, internal/sub, internal/xray,
etc. Package-name mentions (the "web" package), root paths (main.go,
frontend/, install scripts, /etc/x-ui), routes (/panel/api/xray), and the
historical "web/assets no longer exists" note were intentionally left as-is.

* refactor(web): remove the legacy /xui -> /panel redirect middleware

RedirectMiddleware existed only for backward compatibility with the old
`/xui` URL scheme (301-redirecting /xui and /xui/API to /panel and
/panel/api). That cutover was long ago, so drop the middleware, its
registration in initRouter, and the now-inaccurate "URL redirection"
mention in the middleware package doc. Old /xui URLs now 404 like any other
unknown path. HTTPS auto-redirect and auth redirects are unrelated and stay.

* build: fix .dockerignore for internal/ layout and exclude runtime dir

- web/dist -> internal/web/dist: the embedded frontend moved under internal/,
  so the stale exclude no longer matched and the locally-built dist could be
  sent to the build context (the frontend stage rebuilds it fresh anyway).
- exclude x-ui/: the local runtime directory (SQLite db, geo .dat files, xray
  binaries, certs — ~150MB) was being shipped into the build context for no
  reason. Verified the pattern excludes only the directory and still keeps
  x-ui.sh, which the Dockerfile copies to /usr/bin/x-ui.
2026-06-10 15:19:22 +02:00

785 lines
23 KiB
Go

package integration
import (
"context"
"errors"
"fmt"
"io"
"net"
"net/http"
"net/url"
"os"
"path/filepath"
"regexp"
"strings"
"time"
"github.com/mhsanaei/3x-ui/v3/internal/config"
"github.com/mhsanaei/3x-ui/v3/internal/database"
"github.com/mhsanaei/3x-ui/v3/internal/database/model"
"github.com/mhsanaei/3x-ui/v3/internal/logger"
"github.com/mhsanaei/3x-ui/v3/internal/util/netproxy"
"github.com/mhsanaei/3x-ui/v3/internal/util/netsafe"
"github.com/mhsanaei/3x-ui/v3/internal/web/service"
)
const (
customGeoTypeGeosite = "geosite"
customGeoTypeGeoip = "geoip"
minDatBytes = 64
customGeoProbeTimeout = 12 * time.Second
)
var (
customGeoAliasPattern = regexp.MustCompile(`^[a-z0-9_-]+$`)
reservedCustomAliases = map[string]struct{}{
"geoip": {}, "geosite": {},
"geoip_ir": {}, "geosite_ir": {},
"geoip_ru": {}, "geosite_ru": {},
}
ErrCustomGeoInvalidType = errors.New("custom_geo_invalid_type")
ErrCustomGeoAliasRequired = errors.New("custom_geo_alias_required")
ErrCustomGeoAliasPattern = errors.New("custom_geo_alias_pattern")
ErrCustomGeoAliasReserved = errors.New("custom_geo_alias_reserved")
ErrCustomGeoURLRequired = errors.New("custom_geo_url_required")
ErrCustomGeoInvalidURL = errors.New("custom_geo_invalid_url")
ErrCustomGeoURLScheme = errors.New("custom_geo_url_scheme")
ErrCustomGeoURLHost = errors.New("custom_geo_url_host")
ErrCustomGeoDuplicateAlias = errors.New("custom_geo_duplicate_alias")
ErrCustomGeoNotFound = errors.New("custom_geo_not_found")
ErrCustomGeoDownload = errors.New("custom_geo_download")
ErrCustomGeoSSRFBlocked = errors.New("custom_geo_ssrf_blocked")
ErrCustomGeoPathTraversal = errors.New("custom_geo_path_traversal")
)
type CustomGeoUpdateAllItem struct {
Id int `json:"id"`
Alias string `json:"alias"`
FileName string `json:"fileName"`
}
type CustomGeoUpdateAllFailure struct {
Id int `json:"id"`
Alias string `json:"alias"`
FileName string `json:"fileName"`
Err string `json:"error"`
}
type CustomGeoUpdateAllResult struct {
Succeeded []CustomGeoUpdateAllItem `json:"succeeded"`
Failed []CustomGeoUpdateAllFailure `json:"failed"`
}
type CustomGeoService struct {
serverService *service.ServerService
updateAllGetAll func() ([]model.CustomGeoResource, error)
updateAllApply func(id int, onStartup bool) (string, error)
updateAllRestart func() error
getPanelProxy func() (string, error)
}
func NewCustomGeoService() *CustomGeoService {
s := &CustomGeoService{
serverService: &service.ServerService{},
}
s.updateAllGetAll = s.GetAll
s.updateAllApply = s.applyDownloadAndPersist
s.updateAllRestart = func() error { return s.serverService.RestartXrayService() }
s.getPanelProxy = (&service.SettingService{}).GetPanelProxy
return s
}
func NormalizeAliasKey(alias string) string {
return strings.ToLower(strings.ReplaceAll(alias, "-", "_"))
}
func (s *CustomGeoService) fileNameFor(typ, alias string) string {
if typ == customGeoTypeGeoip {
return fmt.Sprintf("geoip_%s.dat", alias)
}
return fmt.Sprintf("geosite_%s.dat", alias)
}
func (s *CustomGeoService) validateType(typ string) error {
if typ != customGeoTypeGeosite && typ != customGeoTypeGeoip {
return ErrCustomGeoInvalidType
}
return nil
}
func (s *CustomGeoService) validateAlias(alias string) error {
if alias == "" {
return ErrCustomGeoAliasRequired
}
if !customGeoAliasPattern.MatchString(alias) {
return ErrCustomGeoAliasPattern
}
if _, ok := reservedCustomAliases[NormalizeAliasKey(alias)]; ok {
return ErrCustomGeoAliasReserved
}
return nil
}
func (s *CustomGeoService) sanitizeURL(raw string) (string, error) {
if raw == "" {
return "", ErrCustomGeoURLRequired
}
u, err := url.Parse(raw)
if err != nil {
return "", ErrCustomGeoInvalidURL
}
if u.Scheme != "http" && u.Scheme != "https" {
return "", ErrCustomGeoURLScheme
}
if u.Host == "" {
return "", ErrCustomGeoURLHost
}
if err := checkSSRF(context.Background(), u.Hostname()); err != nil {
return "", err
}
// Reconstruct URL from parsed components to break taint propagation.
clean := &url.URL{
Scheme: u.Scheme,
Host: u.Host,
Path: u.Path,
RawPath: u.RawPath,
RawQuery: u.RawQuery,
Fragment: u.Fragment,
}
return clean.String(), nil
}
func localDatFileNeedsRepair(path string) bool {
safePath, err := sanitizeDestPath(path)
if err != nil {
return true
}
fi, err := os.Stat(safePath)
if err != nil {
return true
}
if fi.IsDir() {
return true
}
return fi.Size() < int64(minDatBytes)
}
func CustomGeoLocalFileNeedsRepair(path string) bool {
return localDatFileNeedsRepair(path)
}
func isBlockedIP(ip net.IP) bool {
return netsafe.IsBlockedIP(ip)
}
// checkSSRFDefault validates that the given host does not resolve to a private/internal IP.
// It is context-aware so that dial context cancellation/deadlines are respected during DNS resolution.
func checkSSRFDefault(ctx context.Context, hostname string) error {
ips, err := net.DefaultResolver.LookupIPAddr(ctx, hostname)
if err != nil {
return fmt.Errorf("%w: cannot resolve host %s", ErrCustomGeoSSRFBlocked, hostname)
}
for _, ipAddr := range ips {
if isBlockedIP(ipAddr.IP) {
return fmt.Errorf("%w: %s resolves to blocked address %s", ErrCustomGeoSSRFBlocked, hostname, ipAddr.IP)
}
}
return nil
}
// checkSSRF is the active SSRF guard. Override in tests to allow localhost test servers.
var checkSSRF = checkSSRFDefault
func ssrfSafeTransport() http.RoundTripper {
base, ok := http.DefaultTransport.(*http.Transport)
if !ok {
base = &http.Transport{}
}
cloned := base.Clone()
cloned.DialContext = func(ctx context.Context, network, addr string) (net.Conn, error) {
host, _, err := net.SplitHostPort(addr)
if err != nil {
return nil, fmt.Errorf("%w: %v", ErrCustomGeoSSRFBlocked, err)
}
if err := checkSSRF(ctx, host); err != nil {
return nil, err
}
var dialer net.Dialer
return dialer.DialContext(ctx, network, addr)
}
return cloned
}
func (s *CustomGeoService) httpClient(timeout time.Duration) *http.Client {
proxyURL := ""
if s.getPanelProxy != nil {
if p, err := s.getPanelProxy(); err != nil {
logger.Warning("custom geo: read panel proxy:", err)
} else {
proxyURL = strings.TrimSpace(p)
}
}
if proxyURL != "" {
client, err := netproxy.NewHTTPClient(proxyURL, timeout)
if err != nil {
logger.Warningf("custom geo: invalid panel proxy %q, using direct connection: %v", proxyURL, err)
} else {
return client
}
}
return &http.Client{Timeout: timeout, Transport: ssrfSafeTransport()}
}
func (s *CustomGeoService) probeCustomGeoURLWithGET(rawURL string) error {
sanitizedURL, err := s.sanitizeURL(rawURL)
if err != nil {
return err
}
client := s.httpClient(customGeoProbeTimeout)
req, err := http.NewRequest(http.MethodGet, sanitizedURL, nil)
if err != nil {
return err
}
req.Header.Set("Range", "bytes=0-0")
resp, err := client.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
_, _ = io.Copy(io.Discard, io.LimitReader(resp.Body, 256))
switch resp.StatusCode {
case http.StatusOK, http.StatusPartialContent:
return nil
default:
return fmt.Errorf("get range status %d", resp.StatusCode)
}
}
func (s *CustomGeoService) probeCustomGeoURL(rawURL string) error {
sanitizedURL, err := s.sanitizeURL(rawURL)
if err != nil {
return err
}
client := s.httpClient(customGeoProbeTimeout)
req, err := http.NewRequest(http.MethodHead, sanitizedURL, nil)
if err != nil {
return err
}
resp, err := client.Do(req)
if err != nil {
return err
}
_ = resp.Body.Close()
sc := resp.StatusCode
if sc >= 200 && sc < 300 {
return nil
}
if sc == http.StatusMethodNotAllowed || sc == http.StatusNotImplemented {
return s.probeCustomGeoURLWithGET(rawURL)
}
return fmt.Errorf("head status %d", sc)
}
func (s *CustomGeoService) EnsureOnStartup() {
list, err := s.GetAll()
if err != nil {
logger.Warning("custom geo startup: load list:", err)
return
}
n := len(list)
if n == 0 {
logger.Info("custom geo startup: no custom geofiles configured")
return
}
logger.Infof("custom geo startup: checking %d custom geofile(s)", n)
for i := range list {
r := &list[i]
sanitizedURL, err := s.sanitizeURL(r.Url)
if err != nil {
logger.Warningf("custom geo startup id=%d: invalid url: %v", r.Id, err)
continue
}
r.Url = sanitizedURL
s.syncLocalPath(r)
localPath := r.LocalPath
if !localDatFileNeedsRepair(localPath) {
logger.Infof("custom geo startup id=%d alias=%s path=%s: present", r.Id, r.Alias, localPath)
continue
}
logger.Infof("custom geo startup id=%d alias=%s path=%s: missing or needs repair, probing source", r.Id, r.Alias, localPath)
if err := s.probeCustomGeoURL(r.Url); err != nil {
logger.Warningf("custom geo startup id=%d alias=%s url=%s: probe: %v (attempting download anyway)", r.Id, r.Alias, r.Url, err)
}
_, _ = s.applyDownloadAndPersist(r.Id, true)
}
}
func (s *CustomGeoService) downloadToPath(resourceURL, destPath string, lastModifiedHeader string) (skipped bool, newLastModified string, err error) {
safeDestPath, err := sanitizeDestPath(destPath)
if err != nil {
return false, "", fmt.Errorf("%w: %v", ErrCustomGeoDownload, err)
}
skipped, lm, err := s.downloadToPathOnce(resourceURL, safeDestPath, lastModifiedHeader, false)
if err != nil {
return false, "", err
}
if skipped {
if _, statErr := os.Stat(safeDestPath); statErr == nil && !localDatFileNeedsRepair(safeDestPath) {
return true, lm, nil
}
return s.downloadToPathOnce(resourceURL, safeDestPath, lastModifiedHeader, true)
}
return false, lm, nil
}
// sanitizeDestPath ensures destPath is inside the bin folder, preventing path traversal.
// It resolves symlinks to prevent symlink-based escapes.
// Returns the cleaned absolute path that is safe to use in file operations.
func sanitizeDestPath(destPath string) (string, error) {
baseDirAbs, err := filepath.Abs(config.GetBinFolderPath())
if err != nil {
return "", fmt.Errorf("%w: %v", ErrCustomGeoPathTraversal, err)
}
// Resolve symlinks in base directory to get the real path.
if resolved, evalErr := filepath.EvalSymlinks(baseDirAbs); evalErr == nil {
baseDirAbs = resolved
}
destPathAbs, err := filepath.Abs(destPath)
if err != nil {
return "", fmt.Errorf("%w: %v", ErrCustomGeoPathTraversal, err)
}
// Resolve symlinks for the parent directory of the destination path.
destDir := filepath.Dir(destPathAbs)
if resolved, evalErr := filepath.EvalSymlinks(destDir); evalErr == nil {
destPathAbs = filepath.Join(resolved, filepath.Base(destPathAbs))
}
// Verify the resolved path is within the safe base directory using prefix check.
safeDirPrefix := baseDirAbs + string(filepath.Separator)
if !strings.HasPrefix(destPathAbs, safeDirPrefix) {
return "", ErrCustomGeoPathTraversal
}
return destPathAbs, nil
}
func (s *CustomGeoService) downloadToPathOnce(resourceURL, destPath string, lastModifiedHeader string, forceFull bool) (skipped bool, newLastModified string, err error) {
safeDestPath, err := sanitizeDestPath(destPath)
if err != nil {
return false, "", fmt.Errorf("%w: %v", ErrCustomGeoDownload, err)
}
sanitizedURL, err := s.sanitizeURL(resourceURL)
if err != nil {
return false, "", fmt.Errorf("%w: %v", ErrCustomGeoDownload, err)
}
var req *http.Request
req, err = http.NewRequest(http.MethodGet, sanitizedURL, nil)
if err != nil {
return false, "", fmt.Errorf("%w: %v", ErrCustomGeoDownload, err)
}
if !forceFull {
if fi, statErr := os.Stat(safeDestPath); statErr == nil && !localDatFileNeedsRepair(safeDestPath) {
if !fi.ModTime().IsZero() {
req.Header.Set("If-Modified-Since", fi.ModTime().UTC().Format(http.TimeFormat))
} else if lastModifiedHeader != "" {
if t, perr := time.Parse(http.TimeFormat, lastModifiedHeader); perr == nil {
req.Header.Set("If-Modified-Since", t.UTC().Format(http.TimeFormat))
}
}
}
}
client := s.httpClient(10 * time.Minute)
// lgtm[go/request-forgery]
resp, err := client.Do(req)
if err != nil {
return false, "", fmt.Errorf("%w: %v", ErrCustomGeoDownload, err)
}
defer resp.Body.Close()
var serverModTime time.Time
if lm := resp.Header.Get("Last-Modified"); lm != "" {
if parsed, perr := time.Parse(http.TimeFormat, lm); perr == nil {
serverModTime = parsed
newLastModified = lm
}
}
updateModTime := func() {
if !serverModTime.IsZero() {
_ = os.Chtimes(safeDestPath, serverModTime, serverModTime)
}
}
if resp.StatusCode == http.StatusNotModified {
if forceFull {
return false, "", fmt.Errorf("%w: unexpected 304 on unconditional get", ErrCustomGeoDownload)
}
updateModTime()
return true, newLastModified, nil
}
if resp.StatusCode != http.StatusOK {
return false, "", fmt.Errorf("%w: unexpected status %d", ErrCustomGeoDownload, resp.StatusCode)
}
binDir := filepath.Dir(safeDestPath)
if err = os.MkdirAll(binDir, 0o755); err != nil {
return false, "", fmt.Errorf("%w: %v", ErrCustomGeoDownload, err)
}
safeTmpPath, err := sanitizeDestPath(safeDestPath + ".tmp")
if err != nil {
return false, "", fmt.Errorf("%w: %v", ErrCustomGeoDownload, err)
}
out, err := os.Create(safeTmpPath)
if err != nil {
return false, "", fmt.Errorf("%w: %v", ErrCustomGeoDownload, err)
}
n, err := io.Copy(out, resp.Body)
closeErr := out.Close()
if err != nil {
_ = os.Remove(safeTmpPath)
return false, "", fmt.Errorf("%w: %v", ErrCustomGeoDownload, err)
}
if closeErr != nil {
_ = os.Remove(safeTmpPath)
return false, "", fmt.Errorf("%w: %v", ErrCustomGeoDownload, closeErr)
}
if n < minDatBytes {
_ = os.Remove(safeTmpPath)
return false, "", fmt.Errorf("%w: file too small", ErrCustomGeoDownload)
}
if err = os.Rename(safeTmpPath, safeDestPath); err != nil {
_ = os.Remove(safeTmpPath)
return false, "", fmt.Errorf("%w: %v", ErrCustomGeoDownload, err)
}
updateModTime()
if newLastModified == "" && resp.Header.Get("Last-Modified") != "" {
newLastModified = resp.Header.Get("Last-Modified")
}
return false, newLastModified, nil
}
func (s *CustomGeoService) resolveDestPath(r *model.CustomGeoResource) string {
if r.LocalPath != "" {
return r.LocalPath
}
return filepath.Join(config.GetBinFolderPath(), s.fileNameFor(r.Type, r.Alias))
}
func (s *CustomGeoService) syncLocalPath(r *model.CustomGeoResource) {
p := filepath.Join(config.GetBinFolderPath(), s.fileNameFor(r.Type, r.Alias))
r.LocalPath = p
}
func (s *CustomGeoService) syncAndSanitizeLocalPath(r *model.CustomGeoResource) error {
s.syncLocalPath(r)
safePath, err := sanitizeDestPath(r.LocalPath)
if err != nil {
return err
}
r.LocalPath = safePath
return nil
}
func removeSafePathIfExists(path string) error {
safePath, err := sanitizeDestPath(path)
if err != nil {
return err
}
if _, err := os.Stat(safePath); err == nil {
if err := os.Remove(safePath); err != nil {
return err
}
}
return nil
}
func (s *CustomGeoService) Create(r *model.CustomGeoResource) error {
if err := s.validateType(r.Type); err != nil {
return err
}
if err := s.validateAlias(r.Alias); err != nil {
return err
}
sanitizedURL, err := s.sanitizeURL(r.Url)
if err != nil {
return err
}
r.Url = sanitizedURL
var existing int64
database.GetDB().Model(&model.CustomGeoResource{}).
Where("geo_type = ? AND alias = ?", r.Type, r.Alias).Count(&existing)
if existing > 0 {
return ErrCustomGeoDuplicateAlias
}
if err := s.syncAndSanitizeLocalPath(r); err != nil {
return err
}
skipped, lm, err := s.downloadToPath(r.Url, r.LocalPath, r.LastModified)
if err != nil {
return err
}
now := time.Now().Unix()
r.LastUpdatedAt = now
r.LastModified = lm
if err = database.GetDB().Create(r).Error; err != nil {
_ = removeSafePathIfExists(r.LocalPath)
return err
}
logger.Infof("custom geo created id=%d type=%s alias=%s skipped=%v", r.Id, r.Type, r.Alias, skipped)
if err = s.serverService.RestartXrayService(); err != nil {
logger.Warning("custom geo create: restart xray:", err)
}
return nil
}
func (s *CustomGeoService) Update(id int, r *model.CustomGeoResource) error {
var cur model.CustomGeoResource
if err := database.GetDB().First(&cur, id).Error; err != nil {
if database.IsNotFound(err) {
return ErrCustomGeoNotFound
}
return err
}
if err := s.validateType(r.Type); err != nil {
return err
}
if err := s.validateAlias(r.Alias); err != nil {
return err
}
sanitizedURL, err := s.sanitizeURL(r.Url)
if err != nil {
return err
}
r.Url = sanitizedURL
if cur.Type != r.Type || cur.Alias != r.Alias {
var cnt int64
database.GetDB().Model(&model.CustomGeoResource{}).
Where("geo_type = ? AND alias = ? AND id <> ?", r.Type, r.Alias, id).
Count(&cnt)
if cnt > 0 {
return ErrCustomGeoDuplicateAlias
}
}
oldPath := s.resolveDestPath(&cur)
r.Id = id
if err := s.syncAndSanitizeLocalPath(r); err != nil {
return err
}
if oldPath != r.LocalPath && oldPath != "" {
if err := removeSafePathIfExists(oldPath); err != nil && !errors.Is(err, ErrCustomGeoPathTraversal) {
logger.Warningf("custom geo remove old path %s: %v", oldPath, err)
}
}
_, lm, err := s.downloadToPath(r.Url, r.LocalPath, cur.LastModified)
if err != nil {
return err
}
r.LastUpdatedAt = time.Now().Unix()
r.LastModified = lm
err = database.GetDB().Model(&model.CustomGeoResource{}).Where("id = ?", id).Updates(map[string]any{
"geo_type": r.Type,
"alias": r.Alias,
"url": r.Url,
"local_path": r.LocalPath,
"last_updated_at": r.LastUpdatedAt,
"last_modified": r.LastModified,
}).Error
if err != nil {
return err
}
logger.Infof("custom geo updated id=%d", id)
if err = s.serverService.RestartXrayService(); err != nil {
logger.Warning("custom geo update: restart xray:", err)
}
return nil
}
func (s *CustomGeoService) Delete(id int) (displayName string, err error) {
var r model.CustomGeoResource
if err := database.GetDB().First(&r, id).Error; err != nil {
if database.IsNotFound(err) {
return "", ErrCustomGeoNotFound
}
return "", err
}
displayName = s.fileNameFor(r.Type, r.Alias)
p := s.resolveDestPath(&r)
if _, err := sanitizeDestPath(p); err != nil {
return displayName, err
}
if err := database.GetDB().Delete(&model.CustomGeoResource{}, id).Error; err != nil {
return displayName, err
}
if p != "" {
if err := removeSafePathIfExists(p); err != nil {
logger.Warningf("custom geo delete file %s: %v", p, err)
}
}
logger.Infof("custom geo deleted id=%d", id)
if err := s.serverService.RestartXrayService(); err != nil {
logger.Warning("custom geo delete: restart xray:", err)
}
return displayName, nil
}
func (s *CustomGeoService) GetAll() ([]model.CustomGeoResource, error) {
var list []model.CustomGeoResource
err := database.GetDB().Order("id asc").Find(&list).Error
return list, err
}
func (s *CustomGeoService) applyDownloadAndPersist(id int, onStartup bool) (displayName string, err error) {
var r model.CustomGeoResource
if err := database.GetDB().First(&r, id).Error; err != nil {
if database.IsNotFound(err) {
return "", ErrCustomGeoNotFound
}
return "", err
}
displayName = s.fileNameFor(r.Type, r.Alias)
if err := s.syncAndSanitizeLocalPath(&r); err != nil {
return displayName, err
}
sanitizedURL, sanitizeErr := s.sanitizeURL(r.Url)
if sanitizeErr != nil {
return displayName, sanitizeErr
}
skipped, lm, err := s.downloadToPath(sanitizedURL, r.LocalPath, r.LastModified)
if err != nil {
if onStartup {
logger.Warningf("custom geo startup download id=%d: %v", id, err)
} else {
logger.Warningf("custom geo manual update id=%d: %v", id, err)
}
return displayName, err
}
now := time.Now().Unix()
updates := map[string]any{
"last_modified": lm,
"local_path": r.LocalPath,
"last_updated_at": now,
}
if err = database.GetDB().Model(&model.CustomGeoResource{}).Where("id = ?", id).Updates(updates).Error; err != nil {
if onStartup {
logger.Warningf("custom geo startup id=%d: persist metadata: %v", id, err)
} else {
logger.Warningf("custom geo manual update id=%d: persist metadata: %v", id, err)
}
return displayName, err
}
if skipped {
if onStartup {
logger.Infof("custom geo startup download skipped (not modified) id=%d", id)
} else {
logger.Infof("custom geo manual update skipped (not modified) id=%d", id)
}
} else {
if onStartup {
logger.Infof("custom geo startup download ok id=%d", id)
} else {
logger.Infof("custom geo manual update ok id=%d", id)
}
}
return displayName, nil
}
func (s *CustomGeoService) TriggerUpdate(id int) (string, error) {
displayName, err := s.applyDownloadAndPersist(id, false)
if err != nil {
return displayName, err
}
if err = s.serverService.RestartXrayService(); err != nil {
logger.Warning("custom geo manual update: restart xray:", err)
}
return displayName, nil
}
func (s *CustomGeoService) TriggerUpdateAll() (*CustomGeoUpdateAllResult, error) {
var list []model.CustomGeoResource
var err error
if s.updateAllGetAll != nil {
list, err = s.updateAllGetAll()
} else {
list, err = s.GetAll()
}
if err != nil {
return nil, err
}
res := &CustomGeoUpdateAllResult{}
if len(list) == 0 {
return res, nil
}
for _, r := range list {
var name string
var applyErr error
if s.updateAllApply != nil {
name, applyErr = s.updateAllApply(r.Id, false)
} else {
name, applyErr = s.applyDownloadAndPersist(r.Id, false)
}
if applyErr != nil {
res.Failed = append(res.Failed, CustomGeoUpdateAllFailure{
Id: r.Id, Alias: r.Alias, FileName: name, Err: applyErr.Error(),
})
continue
}
res.Succeeded = append(res.Succeeded, CustomGeoUpdateAllItem{
Id: r.Id, Alias: r.Alias, FileName: name,
})
}
if len(res.Succeeded) > 0 {
var restartErr error
if s.updateAllRestart != nil {
restartErr = s.updateAllRestart()
} else {
restartErr = s.serverService.RestartXrayService()
}
if restartErr != nil {
logger.Warning("custom geo update all: restart xray:", restartErr)
}
}
return res, nil
}
type CustomGeoAliasItem struct {
Alias string `json:"alias"`
Type string `json:"type"`
FileName string `json:"fileName"`
ExtExample string `json:"extExample"`
}
type CustomGeoAliasesResponse struct {
Geosite []CustomGeoAliasItem `json:"geosite"`
Geoip []CustomGeoAliasItem `json:"geoip"`
}
func (s *CustomGeoService) GetAliasesForUI() (CustomGeoAliasesResponse, error) {
list, err := s.GetAll()
if err != nil {
logger.Warning("custom geo GetAliasesForUI:", err)
return CustomGeoAliasesResponse{}, err
}
var out CustomGeoAliasesResponse
for _, r := range list {
fn := s.fileNameFor(r.Type, r.Alias)
ex := fmt.Sprintf("ext:%s:tag", fn)
item := CustomGeoAliasItem{
Alias: r.Alias,
Type: r.Type,
FileName: fn,
ExtExample: ex,
}
if r.Type == customGeoTypeGeoip {
out.Geoip = append(out.Geoip, item)
} else {
out.Geosite = append(out.Geosite, item)
}
}
return out, nil
}