Files
3x-ui/deploy/lightsail/build-snapshot.sh
T
Sanaei 7c2598fae9 feat: release-driven golden-image & unattended-install deployment pipeline (#5323)
* feat(install): add non-interactive install path for cloud/golden-image use

Trigger non-interactive mode when XUI_NONINTERACTIVE=1 or stdin is not a
TTY (curl | bash, cloud-init). Every prompt is then replaced by an env var
or a sane default; interactive prompts stay byte-for-byte identical.

Honored env vars: XUI_USERNAME, XUI_PASSWORD, XUI_PANEL_PORT,
XUI_WEB_BASE_PATH (unset => random, as before), XUI_SSL_MODE=none|ip|domain
(default none), XUI_DOMAIN, XUI_ACME_EMAIL, XUI_DB_TYPE/XUI_DB_DSN, plus
additive XUI_ACME_HTTP_PORT, XUI_SSL_IPV6, XUI_SERVER_IP.

On success, write /etc/x-ui/install-result.env (mode 600) with the panel
creds + access URL + api token, in both interactive and non-interactive
modes, so cloud-init/MOTD can surface them. Postgres in non-interactive
mode requires XUI_DB_DSN or installs locally; never silently downgrades.

* feat(deploy): add first-boot per-instance credential generation

Golden images ship with no x-ui.db. x-ui-firstboot.sh runs once (guarded by
/etc/x-ui/.firstboot-done), before x-ui.service, and replaces the seeded
admin/admin with fresh random username/password on a random high port,
regenerates the session secret/panel GUID via 'x-ui setting -reset', mints an
API token, and writes the creds to /etc/x-ui/credentials.txt (600) + /etc/motd.

Idempotent: skips regeneration if a non-default admin already exists. The
oneshot unit is ordered After=network-online/cloud-init and Before=x-ui.service
so the panel never serves default credentials.

* chore(deploy): force LF for cloud-image deploy assets (.service/.hcl/.yaml)

* feat(deploy): add Packer config + provisioning scripts for golden image

One build, two sources: amazon-ebs (AWS AMI, Canonical Ubuntu 24.04 base via
source_ami_filter) and qemu (qcow2 + raw, NoCloud-seeded for build-time SSH).
Provisioner order is fixed: provision.sh -> harden.sh -> cleanup.sh.

- provision.sh: downloads the released x-ui tarball (no Go build), installs the
  panel + firstboot unit, enables but does NOT start services, creates NO DB.
- harden.sh: key-only SSH, no root password login, locks default account
  passwords, enables unattended-upgrades (scanner-compliant).
- cleanup.sh: wipes any DB/creds, SSH host keys, authorized_keys, machine-id,
  cloud-init state, logs and history; fails the build if any secret survives.

packer fmt -check clean; packer validate passes for both sources.

* feat(deploy): add generic cloud-init user-data for unattended install

cloud-init.yaml installs the latest 3x-ui non-interactively (XUI_NONINTERACTIVE=1)
on any cloud-init platform, generating unique per-instance credentials and
surfacing them via /etc/x-ui/install-result.env, serial console and MOTD.
README documents per-provider usage (Hetzner/AWS/DO/Vultr/GCP/Azure/Oracle)
and all XUI_* knobs.

* ci: add image.yml to build cloud images on release

On release: published (or workflow_dispatch with a tag), waits for the
x-ui-linux-amd64.tar.gz asset (handles the release-matrix upload race), then:
- qemu-image (always): builds the qcow2 with Packer and attaches a compressed
  .qcow2.xz + sha256 to the GitHub release. Uses KVM when /dev/kvm exists,
  else TCG.
- ami-image (gated): builds the AWS AMI only when AWS creds exist (OIDC role
  preferred, else access keys), so forks skip cleanly. Prints the AMI ID to the
  job summary. No secrets or AMI IDs are committed.

* test(deploy): add container smoke tests for install + firstboot

smoke-noninteractive.sh: runs install.sh piped (no TTY) with
XUI_NONINTERACTIVE=1 in an Ubuntu container; asserts install-result.env (600)
holds random non-default creds, hasDefaultCredential is false, and the panel
serves HTTP.

smoke-firstboot.sh: installs the released binary with no DB, runs
x-ui-firstboot.sh; asserts per-instance creds + credentials.txt (600) + MOTD,
no admin/admin, and that a second run is a no-op (sentinel honored).

smoke.yml runs both as gated jobs on PRs/pushes touching install.sh or deploy/**.
Both pass locally against the v3.3.1 release binary.

* docs(deploy): add Packer/marketplace docs and link from README

- deploy/README.md: index of the cloud-deploy tooling and the two models
- deploy/packer/README.md: how to build locally, variables, first-boot behavior
- deploy/marketplace/aws/README.md: seller registration -> AMI scan ->
  limited-visibility preview -> go-public checklist
- deploy/marketplace/hetzner/README.md: cloud-init-first guidance + snapshot
  caveat (delete x-ui.db first) + hetznercloud/apps reference
- README.md: link the unattended-install / cloud-image docs from Quick Start

* feat(deploy): build golden images for arm64 as well as amd64

The install path was already multi-arch (install.sh auto-detects arch); this
extends the golden image + CI to arm64:

- packer: xui_arch (amd64|arm64, validated) now derives the base AMI filter and
  the Ubuntu cloud image; the qemu source switches to qemu-system-aarch64 + virt
  machine + AAVMF UEFI firmware for arm64. amd64 path unchanged.
- image.yml: arch matrix. AMIs for amd64 (t3.small) + arm64 (t4g.small/Graviton)
  from one runner; qcow2 for amd64 on a standard runner and arm64 on a native
  ubuntu-24.04-arm runner. Waits for both release tarballs.
- smoke.yml: run install + firstboot smoke tests on amd64 and arm64 runners;
  smoke-firstboot.sh now resolves the arch tarball via dpkg.
- docs updated for both arches.

packer fmt/validate pass for amd64 and arm64; actionlint + shellcheck clean.
Verified locally: non-interactive install AND firstboot run on the real arm64
release binary under emulation (ELF aarch64, no admin/admin).

* chore(deploy): default AWS region to eu-central-1 (Frankfurt)

Replace the us-east-1 fallback in image.yml (4 sites) and the Packer 'region'
default + doc examples. Still overridable via the AWS_REGION repo variable / the
-var 'region=...' flag.

* feat(deploy): add Amazon Lightsail support (launch script + snapshot builder)

Lightsail can't launch from an EC2 AMI and its blueprint list isn't
self-publishable, so add the two self-service paths instead:

- launch-script.sh: paste into Lightsail 'Add launch script' (or --user-data) to
  install 3x-ui non-interactively with unique per-instance credentials.
- snapshot-userdata.sh + build-snapshot.sh: AWS CLI pipeline that provisions a
  build instance (panel installed, NO DB, firstboot enabled), runs the shared
  cleanup.sh, then snapshots it. Instances launched from the snapshot mint their
  own credentials on first boot. Optional --panel-port pins a known port for the
  Lightsail firewall.
- README documents both paths, the firewall caveat, and the blueprint reality.

EC2 AMI / Marketplace path kept untouched alongside. All scripts shellcheck-clean.

* fix(deploy): address Copilot PR review findings

- install.sh + firstboot: write install-result.env / credentials.txt values with
  printf %q so the files stay safe to source even if creds are pinned with shell
  metacharacters (no-op for the alphanumeric random defaults).
- firstboot: fail closed if 'x-ui setting -show' can't be parsed to true/false —
  exit without writing the sentinel so the next boot retries, instead of silently
  skipping regeneration and risking admin/admin.
- firstboot + cloud-init + lightsail launch-script: keep secrets out of the
  world-readable /etc/motd (show URL + username only; full creds via the mode-600
  file / serial console).
- lightsail build-snapshot: handle download-default-key-pair returning either a
  PEM or base64, and assert a valid PEM before using it for SSH.
- image.yml: pin hashicorp/setup-packer@v3 (was @main).
- deploy/README: document XUI_ACME_HTTP_PORT / XUI_SSL_IPV6 / XUI_SERVER_IP.

Both container smoke tests still pass; shellcheck + actionlint clean.
2026-06-14 18:08:35 +02:00

193 lines
7.4 KiB
Bash

#!/usr/bin/env bash
#
# build-snapshot.sh — build a reusable Amazon Lightsail snapshot of 3x-ui.
#
# Flow (mirrors the Packer golden-image model, via the Lightsail API):
# 1. create an Ubuntu Lightsail instance with snapshot-userdata.sh
# (installs the panel, NO database, enables the first-boot unit)
# 2. wait for provisioning, then (optionally) pin a known panel port and run
# the shared cleanup.sh (wipes any DB/creds/keys/host-keys/cloud-init state)
# 3. stop the instance and create an instance snapshot
# 4. delete the build instance (unless --keep-instance)
#
# Every instance you later launch from the snapshot generates its OWN unique
# credentials on first boot (see deploy/firstboot/). The snapshot is private to
# your AWS account.
#
# Requirements: awscli v2, jq, ssh. AWS credentials with Lightsail permissions.
# Usage:
# deploy/lightsail/build-snapshot.sh --region eu-central-1 [options]
# Options:
# --region <r> AWS region (default: $AWS_REGION or eu-central-1)
# --blueprint-id <id> Lightsail blueprint (default: ubuntu_24_04)
# --bundle-id <id> Lightsail bundle/size (default: small_3_0)
# --availability-zone <z> AZ (default: <region>a)
# --panel-port <p> Pin the panel port in the snapshot so you can pre-open
# it in the Lightsail firewall (default: random per instance)
# --snapshot-name <n> Snapshot name (default: 3x-ui-ubuntu-24.04-<timestamp>)
# --keep-instance Do not delete the build instance afterwards
set -euo pipefail
REGION="${AWS_REGION:-eu-central-1}"
BLUEPRINT="ubuntu_24_04"
BUNDLE="small_3_0"
AZ=""
PANEL_PORT=""
SNAPSHOT_NAME=""
KEEP_INSTANCE=0
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
STAMP="$(date +%Y%m%d-%H%M%S)"
INSTANCE_NAME="3xui-build-${STAMP}"
KEY_FILE=""
log() { echo "[build-snapshot] $*"; }
die() {
echo "[build-snapshot] ERROR: $*" >&2
exit 1
}
while [ $# -gt 0 ]; do
case "$1" in
--region) REGION="$2"; shift 2 ;;
--blueprint-id) BLUEPRINT="$2"; shift 2 ;;
--bundle-id) BUNDLE="$2"; shift 2 ;;
--availability-zone) AZ="$2"; shift 2 ;;
--panel-port) PANEL_PORT="$2"; shift 2 ;;
--snapshot-name) SNAPSHOT_NAME="$2"; shift 2 ;;
--keep-instance) KEEP_INSTANCE=1; shift ;;
-h | --help) sed -n '2,40p' "$0"; exit 0 ;;
*) die "unknown option: $1" ;;
esac
done
[ -n "$AZ" ] || AZ="${REGION}a"
[ -n "$SNAPSHOT_NAME" ] || SNAPSHOT_NAME="3x-ui-ubuntu-24.04-${STAMP}"
for cmd in aws jq ssh; do
command -v "$cmd" > /dev/null 2>&1 || die "'$cmd' is required"
done
SSH_OPTS=(-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectTimeout=10 -o LogLevel=ERROR)
cleanup() {
[ -n "$KEY_FILE" ] && rm -f "$KEY_FILE"
if [ "$KEEP_INSTANCE" -eq 0 ]; then
aws lightsail delete-instance --instance-name "$INSTANCE_NAME" --region "$REGION" > /dev/null 2>&1 || true
fi
}
trap cleanup EXIT
wait_state() {
local want="$1" tries="${2:-60}" st
for _ in $(seq 1 "$tries"); do
st=$(aws lightsail get-instance-state --instance-name "$INSTANCE_NAME" --region "$REGION" \
--query 'state.name' --output text 2> /dev/null || echo "")
[ "$st" = "$want" ] && return 0
sleep 5
done
return 1
}
log "creating build instance ${INSTANCE_NAME} (${BLUEPRINT}/${BUNDLE}) in ${REGION}..."
aws lightsail create-instances \
--instance-names "$INSTANCE_NAME" \
--availability-zone "$AZ" \
--blueprint-id "$BLUEPRINT" \
--bundle-id "$BUNDLE" \
--user-data "file://${SCRIPT_DIR}/snapshot-userdata.sh" \
--region "$REGION" > /dev/null
log "waiting for instance to run..."
wait_state running 60 || die "instance did not reach 'running'"
IP=$(aws lightsail get-instance --instance-name "$INSTANCE_NAME" --region "$REGION" \
--query 'instance.publicIpAddress' --output text)
if [ -z "$IP" ] || [ "$IP" = "None" ]; then die "no public IP"; fi
log "instance IP: ${IP}"
KEY_FILE="$(mktemp)"
# download-default-key-pair returns the key in 'privateKeyBase64'. Despite the
# name, the CLI historically emits the plaintext PEM (-----BEGIN...); the API
# docs describe it as base64. Handle both: write PEM as-is, else base64-decode.
KEY_RAW="$(aws lightsail download-default-key-pair --region "$REGION" \
--query 'privateKeyBase64' --output text)"
[ -n "$KEY_RAW" ] && [ "$KEY_RAW" != "None" ] || die "failed to download default key pair"
case "$KEY_RAW" in
*-----BEGIN*) printf '%s\n' "$KEY_RAW" > "$KEY_FILE" ;;
*) printf '%s' "$KEY_RAW" | base64 -d > "$KEY_FILE" 2> /dev/null \
|| die "private key is neither PEM nor valid base64" ;;
esac
grep -q -- "-----BEGIN" "$KEY_FILE" || die "downloaded key is not a valid PEM private key"
chmod 600 "$KEY_FILE"
log "waiting for provisioning to finish (this installs the panel)..."
ok=0
for _ in $(seq 1 72); do # ~12 min
if ssh "${SSH_OPTS[@]}" -i "$KEY_FILE" "ubuntu@${IP}" \
'test -f /var/lib/3xui-provision-done' 2> /dev/null; then
ok=1
break
fi
sleep 10
done
[ "$ok" -eq 1 ] || die "provisioning did not complete in time"
log "provisioning complete."
if [ -n "$PANEL_PORT" ]; then
log "pinning panel port ${PANEL_PORT} (username/password stay random)..."
ssh "${SSH_OPTS[@]}" -i "$KEY_FILE" "ubuntu@${IP}" \
"echo 'XUI_PANEL_PORT=${PANEL_PORT}' | sudo tee -a /etc/default/x-ui >/dev/null"
fi
log "stripping instance state (shared cleanup.sh)..."
ssh "${SSH_OPTS[@]}" -i "$KEY_FILE" "ubuntu@${IP}" \
'curl -fsSL https://raw.githubusercontent.com/MHSanaei/3x-ui/main/deploy/packer/scripts/cleanup.sh | sudo bash'
log "stopping instance..."
aws lightsail stop-instance --instance-name "$INSTANCE_NAME" --region "$REGION" > /dev/null
wait_state stopped 60 || die "instance did not stop"
log "creating snapshot ${SNAPSHOT_NAME}..."
aws lightsail create-instance-snapshot \
--instance-name "$INSTANCE_NAME" \
--instance-snapshot-name "$SNAPSHOT_NAME" \
--region "$REGION" > /dev/null
log "waiting for snapshot to become available..."
snap_ok=0
for _ in $(seq 1 120); do # ~20 min
state=$(aws lightsail get-instance-snapshot --instance-snapshot-name "$SNAPSHOT_NAME" \
--region "$REGION" --query 'instanceSnapshot.state' --output text 2> /dev/null || echo "")
[ "$state" = "available" ] && {
snap_ok=1
break
}
sleep 10
done
[ "$snap_ok" -eq 1 ] || die "snapshot did not become available"
log "DONE."
echo
echo "================================================================"
echo " Lightsail snapshot ready: ${SNAPSHOT_NAME} (region ${REGION})"
echo "================================================================"
echo " Launch an instance from it:"
echo " aws lightsail create-instances-from-snapshot \\"
echo " --instance-snapshot-name ${SNAPSHOT_NAME} \\"
echo " --instance-names my-3xui-1 --bundle-id ${BUNDLE} \\"
echo " --availability-zone ${AZ} --region ${REGION}"
if [ -n "$PANEL_PORT" ]; then
echo
echo " Then open the panel port (pinned to ${PANEL_PORT}):"
echo " aws lightsail open-instance-public-ports --region ${REGION} \\"
echo " --instance-name my-3xui-1 \\"
echo " --port-info fromPort=${PANEL_PORT},toPort=${PANEL_PORT},protocol=TCP"
else
echo
echo " Each instance picks a RANDOM panel port. After it boots, read it from"
echo " sudo cat /etc/x-ui/credentials.txt"
echo " and open that TCP port in the instance's Lightsail IPv4 firewall."
fi
echo "================================================================"