feat(backups): añadir stack de backups cifrados con Kopia + Cloudflare R2

- init-r2-repository.sh: inicializa repositorio Kopia en R2
- run-backup.sh: ejecuta backup de rutas definidas en backup-paths.txt
- pre-backup-dumps.sh: vuelca bases de datos (Postgres, MariaDB) antes del backup
- kopia-common.sh: funciones compartidas de autenticación y configuración
- restore-smoke-test.sh: test de restauración para verificar integridad
- backup-paths.txt / excludes.txt: rutas y exclusiones del backup
- systemd/: unidades kopia-server, kopia-docker-backup (service + timer)
- traefik-kopia.yml: exposición del servidor Kopia vía Traefik
- env.example: plantilla de variables (credenciales R2, contraseña repo)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Eduardo David Paredes Vara
2026-05-25 05:48:38 +00:00
parent 6ba696ff2b
commit 616b20f285
14 changed files with 614 additions and 0 deletions
+4
View File
@@ -0,0 +1,4 @@
kopia.env
*.log
staging/
cache/
+124
View File
@@ -0,0 +1,124 @@
# Kopia backups to Cloudflare R2
This directory defines restore-focused backups for the Docker services that are actually running on the host. It uses Kopia with Cloudflare R2 as an S3-compatible repository.
## Policy
- Use Kopia as the only backup engine.
- Store the repository in Cloudflare R2.
- Keep secrets outside this repo in `/opt/kopia/kopia.env`.
- Snapshot data needed to restore services and configuration.
- Generate logical database dumps before each snapshot.
- Include Paperless documents and Gitea repositories.
- Exclude Nextcloud user data/multimedia until storage usage and R2 cost are explicitly accepted.
Current size check on 2026-04-27:
| Path | Size |
| --- | ---: |
| `/opt/paperless` | 316M |
| `/opt/gitea` | 6.0M |
| `/opt/nextcloud` | 2.8G |
| `/opt/onlyoffice` | 4.3M |
| `/opt/authentik` | 8.3M |
| `/opt/trilium` | 3.6M |
| `/opt/karakeep` | 5.2M |
| `/opt/spintools` | 335M |
## What is backed up
The snapshot paths are listed in `backup-paths.txt`. The important choices are:
- Paperless: `/opt/paperless/data`, `/opt/paperless/media`, export and consume directories.
- Gitea: `/opt/gitea/data` and runner data.
- Nextcloud: config, themes and a MariaDB dump only. `/opt/nextcloud/data`, `/opt/nextcloud/custom_apps` and `/opt/nextcloud/html` are excluded.
- Databases: PostgreSQL and MariaDB dumps are written to `/opt/kopia/staging/dumps/latest`.
- Service configuration: deployment repo, Traefik, Authentik, n8n, Memos, Vikunja, Trilium, Karakeep, AdGuard, WireGuard, mail relay, Beszel, Homepage, Dozzle, Kasm and OpenCode.
- Local active deployments: Scheduler, Spintools, THSCOM, OpenClaw restore state and OpenCode runtime state.
Live database directories are excluded when a logical dump exists. Kasm's PostgreSQL database is backed up with `pg_dump`; Docker image/layer storage under `/opt/kasm/opt/docker` is intentionally excluded because it is rebuildable and large. OpenProject's embedded PostgreSQL data is not dumped by this first version; treat its restore as best-effort unless a service-specific dump command is added.
## Install Kopia
Install Kopia on the host using the package method appropriate for the server. Then create the runtime directory:
```bash
sudo mkdir -p /opt/kopia
sudo cp backups/kopia/env.example /opt/kopia/kopia.env
sudo chmod 600 /opt/kopia/kopia.env
```
Edit `/opt/kopia/kopia.env` and set the R2 bucket, endpoint, access keys and `KOPIA_PASSWORD`.
Cloudflare R2 endpoint format:
```text
<account_id>.r2.cloudflarestorage.com
```
## Initialize
```bash
backups/kopia/init-r2-repository.sh
```
## Run manually
```bash
backups/kopia/run-backup.sh
```
The script:
1. Creates database dumps.
2. Snapshots the paths in `backup-paths.txt`.
3. Applies retention: latest 7, daily 7, weekly 4, monthly 6.
4. Runs `kopia repository validate-provider`.
## Enable timer
Install the scripts under `/opt/kopia/scripts`; systemd should not execute them directly from a home directory on SELinux-enforcing hosts.
```bash
sudo install -d -m 0755 /opt/kopia/scripts
sudo install -m 0755 backups/kopia/*.sh /opt/kopia/scripts/
sudo install -m 0644 backups/kopia/backup-paths.txt backups/kopia/excludes.txt /opt/kopia/scripts/
sudo cp backups/kopia/systemd/kopia-docker-backup.* /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now kopia-docker-backup.timer
```
## Web UI
Kopia UI is exposed at:
```text
https://kopia.sherlockhomeless.net
```
The service listens only on the Docker host gateway `10.0.0.1:51515` and is published through Traefik using `traefik-kopia.yml`.
Credentials are stored outside the repo in `/opt/kopia/kopia.env`:
```bash
sudo awk -F= '/^KOPIA_SERVER_USERNAME|^KOPIA_SERVER_PASSWORD/ {print}' /opt/kopia/kopia.env
```
Install or update the service and Traefik route:
```bash
sudo cp backups/kopia/systemd/kopia-server.service /etc/systemd/system/
sudo cp backups/kopia/traefik-kopia.yml /opt/traefik/dynamic/kopia.yml
sudo systemctl daemon-reload
sudo systemctl enable --now kopia-server.service
```
## Restore smoke test
Requires `jq`.
```bash
backups/kopia/restore-smoke-test.sh /tmp/kopia-restore-test
```
For real restores, restore into a temporary directory first, stop the target service, move data into place with correct ownership, then start the stack and validate the application.
+77
View File
@@ -0,0 +1,77 @@
# Filesystem paths included in Kopia snapshots.
# Database contents are captured through dumps in /opt/kopia/staging/dumps.
#
# Nextcloud policy: include DB dump plus app/config/runtime metadata only.
# Do not include /opt/nextcloud/data, custom_apps or html until storage cost is reviewed.
/opt/kopia/staging/dumps
# Paperless: keep documents and app data.
/opt/paperless/data
/opt/paperless/media
/opt/paperless/export
/opt/paperless/consume
/opt/paperless-ai/data
/opt/rclone
# Gitea: keep repositories, attachments, runner config and DB dump.
/opt/gitea/data
/opt/gitea/runner
# App configuration and small persistent data.
/opt/n8n/data
/opt/authentik/redis
/opt/memos/data
/opt/vikunja/files
/opt/trilium/data
/opt/karakeep/data
/opt/karakeep/meili_data
/opt/adguard/conf
/opt/adguard/work
/opt/wg-easy
/opt/traefik/dynamic
/opt/traefik/letsencrypt
/opt/traefik/certs
/opt/mail-relay/opendkim
/opt/mail-relay/secrets
/opt/beszel/data
/opt/beszel/agent
/opt/homepage/config
/opt/dozzle/data
/opt/guacamole/admin-credentials.env
# Kasm Workspaces: DB dump plus install/config state required for restore.
# Docker image/layer storage under /opt/kasm/opt/docker is intentionally omitted.
/opt/kasm/1.18.1/bin
/opt/kasm/1.18.1/certs
/opt/kasm/1.18.1/conf
/opt/kasm/1.18.1/docker
/opt/kasm/1.18.1/file_mappings
/opt/kasm/1.18.1/licenses
/opt/kasm/1.18.1/plugins
# OpenCode Web UI service configuration and runtime state.
/etc/opencode-web.env
/home/felidae/.config/opencode
/home/felidae/.local/share/opencode
# Nextcloud: no user multimedia/data and no app tree. DB dump is generated separately.
/opt/nextcloud/config
/opt/nextcloud/themes
/opt/onlyoffice/data
/opt/onlyoffice/lib
/opt/onlyoffice/postgresql
# Scheduler: keep PostgreSQL data volume and logical dump, but not source tree.
/var/lib/docker/volumes/scheduler_app_postgres_data/_data
# Spintools / THSCOM.
/opt/spintools/prod
/opt/spintools/openproject/assets
/opt/spintools/openproject-token
/opt/spintools/authentik/redis
/var/lib/docker/volumes/spintools_ruleta_static/_data
/var/lib/docker/volumes/thscom-backend-media/_data
# OpenClaw runtime state, excluding workspaces via exclude file.
/home/felidae/.openclaw
+23
View File
@@ -0,0 +1,23 @@
# Copy this file to /opt/kopia/kopia.env and fill the real values there.
# Do not commit the real file.
# Kopia repository encryption password.
KOPIA_PASSWORD=change-me
# Cloudflare R2 S3-compatible repository.
# Endpoint format: <account_id>.r2.cloudflarestorage.com
KOPIA_R2_BUCKET=change-me
KOPIA_R2_ENDPOINT=change-me.r2.cloudflarestorage.com
KOPIA_R2_PREFIX=docker-host
KOPIA_R2_ACCESS_KEY_ID=change-me
KOPIA_R2_SECRET_ACCESS_KEY=change-me
# Local Kopia state and dump staging.
KOPIA_CONFIG_FILE=/opt/kopia/repository.config
KOPIA_CACHE_DIRECTORY=/opt/kopia/cache
KOPIA_LOG_DIR=/opt/kopia/logs
BACKUP_STAGING_DIR=/opt/kopia/staging
# Optional overrides.
BACKUP_PATHS_FILE=
KOPIA_EXCLUDES_FILE=
+54
View File
@@ -0,0 +1,54 @@
# Global excludes for Kopia snapshots.
# Never include Nextcloud user data until reviewed for R2 cost.
/opt/nextcloud/data
/opt/nextcloud/custom_apps
/opt/nextcloud/html
/opt/paperless/media/.thumbnails
# Live database directories are handled by logical dumps when possible.
/var/lib/docker/volumes/kasm_db_1.18.1
/opt/kasm/opt/docker
/opt/kasm/1.18.1/log
/opt/kasm/1.18.1/tmp
/opt/paperless/pgdata
/opt/gitea/postgres
/opt/n8n/postgres
/opt/authentik/postgres
/opt/memos/postgres
/opt/vikunja/postgres
/opt/nextcloud/db
/opt/spintools/authentik/postgres
/opt/spintools/openproject/pgdata
# Rebuildable caches and dependencies.
**/node_modules
node_modules
frontend/*/node_modules
/home/felidae/scheduler_app/frontend/*/node_modules
**/.next/cache
frontend/*/.next
/home/felidae/scheduler_app/frontend/*/.next
**/__pycache__
**/.pytest_cache
**/.mypy_cache
/home/felidae/scheduler_app/.mypy_cache
**/.ruff_cache
**/.cache
**/cache
**/tmp
**/*.tmp
**/dist
**/build
# OpenClaw workspaces can become large and are not required for host restore.
/home/felidae/.openclaw/workspace
# OpenCode generated dependencies and noisy runtime artifacts.
/home/felidae/.config/opencode/node_modules
/home/felidae/.local/share/opencode/log
/home/felidae/.local/share/opencode/tool-output
# Logs are useful locally but noisy and usually not restore-critical.
**/*.log
**/logs
+30
View File
@@ -0,0 +1,30 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
# shellcheck disable=SC1091
source "$SCRIPT_DIR/kopia-common.sh"
require_cmd kopia
require_real_env KOPIA_PASSWORD
require_real_env KOPIA_R2_BUCKET
require_real_env KOPIA_R2_ENDPOINT
require_real_env KOPIA_R2_ACCESS_KEY_ID
require_real_env KOPIA_R2_SECRET_ACCESS_KEY
ensure_runtime_dirs
if kopia_cmd repository status >/dev/null 2>&1; then
log "Kopia repository already connected: $KOPIA_CONFIG_FILE"
exit 0
fi
log "Creating Kopia repository in Cloudflare R2 bucket $KOPIA_R2_BUCKET"
kopia_cmd repository create s3 \
--bucket="$KOPIA_R2_BUCKET" \
--endpoint="$KOPIA_R2_ENDPOINT" \
--prefix="${KOPIA_R2_PREFIX:-docker-host}" \
--access-key="$KOPIA_R2_ACCESS_KEY_ID" \
--secret-access-key="$KOPIA_R2_SECRET_ACCESS_KEY"
log "Repository created"
+58
View File
@@ -0,0 +1,58 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
ENV_FILE="${KOPIA_ENV_FILE:-/opt/kopia/kopia.env}"
if [[ -f "$ENV_FILE" ]]; then
# shellcheck disable=SC1090
source "$ENV_FILE"
fi
KOPIA_CONFIG_FILE="${KOPIA_CONFIG_FILE:-/opt/kopia/repository.config}"
KOPIA_CACHE_DIRECTORY="${KOPIA_CACHE_DIRECTORY:-/opt/kopia/cache}"
KOPIA_LOG_DIR="${KOPIA_LOG_DIR:-/opt/kopia/logs}"
BACKUP_STAGING_DIR="${BACKUP_STAGING_DIR:-/opt/kopia/staging}"
BACKUP_PATHS_FILE="${BACKUP_PATHS_FILE:-$SCRIPT_DIR/backup-paths.txt}"
KOPIA_EXCLUDES_FILE="${KOPIA_EXCLUDES_FILE:-$SCRIPT_DIR/excludes.txt}"
export KOPIA_PASSWORD
export KOPIA_CACHE_DIRECTORY
require_cmd() {
command -v "$1" >/dev/null 2>&1 || {
echo "Missing required command: $1" >&2
exit 1
}
}
require_env() {
local name="$1"
if [[ -z "${!name:-}" ]]; then
echo "Missing required environment variable: $name" >&2
exit 1
fi
}
require_real_env() {
local name="$1"
require_env "$name"
case "${!name}" in
change-me|change-me.*|*.change-me|*change-me*)
echo "Environment variable $name still contains a placeholder value" >&2
exit 1
;;
esac
}
kopia_cmd() {
kopia --config-file="$KOPIA_CONFIG_FILE" "$@"
}
log() {
printf '[%s] %s\n' "$(date -u +'%Y-%m-%dT%H:%M:%SZ')" "$*"
}
ensure_runtime_dirs() {
mkdir -p "$KOPIA_CACHE_DIRECTORY" "$KOPIA_LOG_DIR" "$BACKUP_STAGING_DIR"
}
+87
View File
@@ -0,0 +1,87 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
# shellcheck disable=SC1091
source "$SCRIPT_DIR/kopia-common.sh"
require_cmd docker
ensure_runtime_dirs
DUMP_DIR="$BACKUP_STAGING_DIR/dumps"
RUN_DIR="$DUMP_DIR/latest"
mkdir -p "$RUN_DIR"
find "$RUN_DIR" -type f -name '*.dump' -delete
find "$RUN_DIR" -type f -name '*.sql.gz' -delete
container_running() {
[[ "$(docker inspect -f '{{.State.Running}}' "$1" 2>/dev/null || true)" == "true" ]]
}
container_env() {
local container="$1"
local key="$2"
docker exec "$container" sh -lc "printenv $key 2>/dev/null || true" | tr -d '\r'
}
dump_postgres() {
local container="$1"
local fallback_user="$2"
local fallback_db="$3"
if ! container_running "$container"; then
log "Skip PostgreSQL dump for $container: container not running"
return 0
fi
local user db out
user="$(container_env "$container" POSTGRES_USER)"
db="$(container_env "$container" POSTGRES_DB)"
user="${user:-$fallback_user}"
db="${db:-$fallback_db}"
out="$RUN_DIR/${container}.dump"
log "Dumping PostgreSQL $container/$db"
docker exec "$container" pg_dump -U "$user" -d "$db" -Fc > "$out"
}
dump_mariadb() {
local container="$1"
if ! container_running "$container"; then
log "Skip MariaDB dump for $container: container not running"
return 0
fi
local out
out="$RUN_DIR/${container}.sql.gz"
log "Dumping MariaDB $container"
docker exec "$container" sh -lc '
set -eu
db="${MYSQL_DATABASE:-nextcloud}"
user="${MYSQL_USER:-nextcloud}"
pass="${MYSQL_PASSWORD:-}"
if command -v mariadb-dump >/dev/null 2>&1; then
MYSQL_PWD="$pass" mariadb-dump --single-transaction --quick -u "$user" "$db"
else
MYSQL_PWD="$pass" mysqldump --single-transaction --quick -u "$user" "$db"
fi
' | gzip -c > "$out"
}
dump_postgres "paperless-db" "paperless" "paperless"
dump_postgres "gitea-postgres" "gitea" "gitea"
dump_postgres "n8n-pg" "n8n" "n8n"
dump_postgres "vikunja-db" "vikunja" "vikunja"
dump_postgres "memos-db" "memos" "memos"
dump_postgres "ths-authentik-postgres" "authentik" "authentik"
dump_postgres "scheduler-db" "scheduler" "scheduler"
dump_postgres "guacamole-postgres" "guacamole" "guacamole"
dump_postgres "kasm_db" "kasmapp" "kasm"
dump_postgres "thscom-db" "postgres" "postgres"
dump_postgres "spintools-db" "postgres" "postgres"
dump_postgres "spintools-authentik-postgres" "authentik" "authentik"
dump_mariadb "nextcloud-db"
log "Database dumps written to $RUN_DIR"
+28
View File
@@ -0,0 +1,28 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
# shellcheck disable=SC1091
source "$SCRIPT_DIR/kopia-common.sh"
require_cmd kopia
require_cmd jq
require_real_env KOPIA_PASSWORD
TARGET="${1:-/tmp/kopia-restore-smoke-test}"
SNAPSHOT_PATH="${2:-/opt/kopia/staging/dumps}"
mkdir -p "$TARGET"
log "Finding latest snapshot for $SNAPSHOT_PATH"
SNAPSHOT_ID="$(kopia_cmd snapshot list "$SNAPSHOT_PATH" --json | jq -r '.[0].snapshots[-1].id')"
if [[ -z "$SNAPSHOT_ID" || "$SNAPSHOT_ID" == "null" ]]; then
echo "No snapshot found for $SNAPSHOT_PATH" >&2
exit 1
fi
log "Restoring $SNAPSHOT_ID to $TARGET"
kopia_cmd snapshot restore "$SNAPSHOT_ID" "$TARGET"
log "Restore smoke test completed: $TARGET"
+69
View File
@@ -0,0 +1,69 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
# shellcheck disable=SC1091
source "$SCRIPT_DIR/kopia-common.sh"
require_cmd kopia
require_cmd docker
require_real_env KOPIA_PASSWORD
ensure_runtime_dirs
"$SCRIPT_DIR/pre-backup-dumps.sh"
if ! kopia_cmd repository status >/dev/null 2>&1; then
echo "Kopia repository is not connected. Run $SCRIPT_DIR/init-r2-repository.sh first." >&2
exit 1
fi
mapfile -t SNAPSHOT_PATHS < <(
while IFS= read -r path; do
[[ -z "$path" || "$path" =~ ^[[:space:]]*# ]] && continue
if [[ -e "$path" ]]; then
printf '%s\n' "$path"
else
log "Skip missing path: $path" >&2
fi
done < "$BACKUP_PATHS_FILE"
)
if [[ "${#SNAPSHOT_PATHS[@]}" -eq 0 ]]; then
echo "No backup paths found from $BACKUP_PATHS_FILE" >&2
exit 1
fi
mapfile -t IGNORE_PATTERNS < <(
while IFS= read -r pattern; do
[[ -z "$pattern" || "$pattern" =~ ^[[:space:]]*# ]] && continue
printf '%s\n' "$pattern"
done < "$KOPIA_EXCLUDES_FILE"
)
POLICY_ARGS=(
--global \
--keep-latest=7 \
--keep-daily=7 \
--keep-weekly=4 \
--keep-monthly=6 \
--compression=zstd \
--clear-ignore
)
for pattern in "${IGNORE_PATTERNS[@]}"; do
POLICY_ARGS+=(--add-ignore="$pattern")
done
log "Applying retention, compression and ignore policy"
kopia_cmd policy set "${POLICY_ARGS[@]}"
log "Starting Kopia snapshot with ${#SNAPSHOT_PATHS[@]} paths"
kopia_cmd snapshot create \
--description="docker-host restore backup" \
--tags="source:docker-host,type:restore" \
"${SNAPSHOT_PATHS[@]}"
log "Validating R2 provider"
kopia_cmd repository validate-provider
log "Backup completed"
@@ -0,0 +1,10 @@
[Unit]
Description=Kopia backup for Docker services to Cloudflare R2
Wants=docker.service
After=docker.service network-online.target
[Service]
Type=oneshot
Environment=KOPIA_ENV_FILE=/opt/kopia/kopia.env
WorkingDirectory=/opt/kopia/scripts
ExecStart=/opt/kopia/scripts/run-backup.sh
@@ -0,0 +1,10 @@
[Unit]
Description=Daily Kopia backup for Docker services
[Timer]
OnCalendar=*-*-* 03:30:00
RandomizedDelaySec=30m
Persistent=true
[Install]
WantedBy=timers.target
@@ -0,0 +1,15 @@
[Unit]
Description=Kopia web UI server
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
EnvironmentFile=/opt/kopia/kopia.env
Environment=KOPIA_CHECK_FOR_UPDATES=false
ExecStart=/usr/bin/kopia --config-file=/opt/kopia/repository.config server start --ui --insecure --address=http://10.0.0.1:51515 --cache-directory=/opt/kopia/cache --log-dir=/opt/kopia/logs
Restart=on-failure
RestartSec=10
[Install]
WantedBy=multi-user.target
+25
View File
@@ -0,0 +1,25 @@
http:
routers:
kopia:
rule: Host(`kopia.sherlockhomeless.net`)
entryPoints:
- websecure
tls:
certResolver: letsencrypt
middlewares:
- crowdsec-bouncer@file
- ths-authentik@docker
- kopia-inject-auth
service: kopia
middlewares:
kopia-inject-auth:
headers:
customRequestHeaders:
Authorization: "Basic <redacted-kopia-basic-auth>"
services:
kopia:
loadBalancer:
servers:
- url: "http://host.docker.internal:51515"