diff --git a/docs/deployment-guide.md b/docs/deployment-guide.md new file mode 100644 index 00000000..ea6c3ee5 --- /dev/null +++ b/docs/deployment-guide.md @@ -0,0 +1,230 @@ +# Deployment Guide + +## Overview + +Production target uses a hybrid deployment: + +- GoClaw gateway runs as a bare-metal `systemd` service. +- PostgreSQL 18 + pgvector runs in Docker. +- Nginx reverse proxies public HTTP/HTTPS traffic to GoClaw on localhost. +- Codex CLI is installed on the host for future agent-controlled CLI work. + +Current VPS shape: + +| Item | Value | +|---|---| +| Host | Private deployment value | +| SSH | Private deployment value | +| Domain | Private deployment value | +| Gateway listen | `127.0.0.1:18790` | +| Public proxy | Nginx on `80/443` | +| Postgres | Docker container `goclaw-postgres` | +| Postgres bind | `127.0.0.1:5432` | + +Keep concrete hostnames, IPs, SSH users, SSH ports, and credentials in private operator notes or environment variables outside this public repository. + +For command examples below: + +```bash +export GOCLAW_HOST= +export GOCLAW_SSH_USER= +export GOCLAW_SSH_PORT= +export GOCLAW_DOMAIN= +``` + +## Server Layout + +| Path | Purpose | +|---|---| +| `/opt/goclaw/releases/` | Immutable GoClaw release directory | +| `/opt/goclaw/current` | Symlink to active release | +| `/opt/goclaw/shared/docker-compose.postgres.yml` | Postgres compose file | +| `/opt/goclaw/shared/postgres.env` | Postgres env file | +| `/opt/goclaw/backups/` | Uploaded DB backup files | +| `/etc/goclaw/config.json` | Gateway config | +| `/etc/goclaw/goclaw.env` | Gateway env and secrets | +| `/etc/goclaw/r2-backup.env` | R2 backup env and secrets | +| `/var/lib/goclaw/data` | GoClaw persistent data | +| `/var/lib/goclaw/workspace` | Agent workspace | +| `/var/lib/goclaw/postgres` | Postgres Docker data | +| `/usr/local/bin/goclaw-deploy` | Release switch, upgrade, health-check, rollback | +| `/usr/local/bin/goclaw-issue-ssl` | Certbot wrapper for the deployment domain | +| `/usr/local/bin/goclaw-backup-r2` | Postgres dump, R2 upload, retention cleanup | + +Secrets are stored only in server env files. Do not copy tokens or database passwords into repo docs. + +## Runtime Services + +Check status: + +```bash +ssh -p "$GOCLAW_SSH_PORT" "$GOCLAW_SSH_USER@$GOCLAW_HOST" +systemctl status goclaw --no-pager +systemctl status nginx --no-pager +systemctl status goclaw-backup-r2.timer --no-pager +docker ps --filter name=goclaw-postgres +curl -fsS http://127.0.0.1:18790/health +``` + +Expected health: + +```json +{"status":"ok","protocol":3} +``` + +## Initial Deployment Record + +Completed on 2026-05-17: + +- Installed Docker, Docker Compose v2, Nginx, Certbot, Node.js 22, Codex CLI. +- Added the operator workstation SSH public key to the deployment user. +- Installed Codex CLI. User still needs to run `codex --login` interactively. +- Restored the latest private PostgreSQL backup into Docker Postgres. +- Upgraded restored schema from `57` to `65`. +- Started the initial GoClaw release by `systemd`. +- Added automated database backup to a private Cloudflare R2 bucket. +- Verified local and Nginx health endpoints. + +Current verification snapshot: + +```text +goclaw_active=active +nginx_active=active +health_local={"status":"ok","protocol":3} +health_nginx={"status":"ok","protocol":3} +docker=goclaw-postgres pgvector/pgvector:pg18 healthy +schema=65 +codex=codex-cli 0.130.0 +``` + +## DNS And SSL + +Cloudflare DNS record: + +| Type | Name | Value | Proxy | +|---|---|---|---| +| `A` | Deployment subdomain | Deployment host IP | Proxied | + +SSL was issued with Certbot for the deployment domain on 2026-05-17 and Certbot installed automatic renewal. + +Verify HTTPS: + +```bash +ssh -p "$GOCLAW_SSH_PORT" "$GOCLAW_SSH_USER@$GOCLAW_HOST" +curl -fsS "https://$GOCLAW_DOMAIN/health" +``` + +Re-issue manually if needed: + +```bash +sudo /usr/local/bin/goclaw-issue-ssl +``` + +## Deploy A New Release + +Build locally with embedded web UI: + +```bash +cd ui/web +pnpm install --frozen-lockfile +pnpm build +cd ../.. +rm -rf internal/webui/dist +mkdir -p internal/webui/dist +cp -r ui/web/dist/* internal/webui/dist/ +CGO_ENABLED=0 GOOS=linux GOARCH=amd64 \ + go build -tags embedui \ + -ldflags="-s -w -X github.com/nextlevelbuilder/goclaw/cmd.Version=" \ + -o dist/goclaw-linux-amd64 . +``` + +Upload: + +```bash +release= +ssh -p "$GOCLAW_SSH_PORT" "$GOCLAW_SSH_USER@$GOCLAW_HOST" "mkdir -p /opt/goclaw/releases/$release/migrations" +scp -P "$GOCLAW_SSH_PORT" dist/goclaw-linux-amd64 "$GOCLAW_SSH_USER@$GOCLAW_HOST:/opt/goclaw/releases/$release/goclaw" +scp -P "$GOCLAW_SSH_PORT" migrations/* "$GOCLAW_SSH_USER@$GOCLAW_HOST:/opt/goclaw/releases/$release/migrations/" +ssh -p "$GOCLAW_SSH_PORT" "$GOCLAW_SSH_USER@$GOCLAW_HOST" "chmod +x /opt/goclaw/releases/$release/goclaw && sudo /usr/local/bin/goclaw-deploy /opt/goclaw/releases/$release" +``` + +`goclaw-deploy` does: + +1. Validate binary and migrations. +2. Switch `/opt/goclaw/current`. +3. Run `goclaw upgrade`. +4. Restart `goclaw`. +5. Poll `/health`. +6. Roll back symlink and restart if health fails. + +## Backup And Restore + +Automated backups: + +| Item | Value | +|---|---| +| Timer | `goclaw-backup-r2.timer` | +| Schedule | Every 6 hours: `00:00`, `06:00`, `12:00`, `18:00` server time | +| Source | Docker Postgres container `goclaw-postgres` | +| Format | PostgreSQL custom dump, `pg_dump -Fc` | +| Local directory | `/opt/goclaw/backups/` | +| R2 bucket | Private deployment value | +| R2 prefix | Private deployment value | +| Retention | Keep latest 20 backups locally and in R2 | + +Check timer and latest logs: + +```bash +systemctl list-timers goclaw-backup-r2.timer --no-pager +journalctl -u goclaw-backup-r2.service -n 80 --no-pager +``` + +Run a manual backup: + +```bash +sudo systemctl start goclaw-backup-r2.service +``` + +Create a database dump on server: + +```bash +ts=$(date +%Y%m%d-%H%M%S) +docker exec goclaw-postgres pg_dump -U goclaw -Fc -d goclaw > /opt/goclaw/backups/goclaw-$ts.dump +``` + +Restore a dump: + +```bash +systemctl stop goclaw +docker exec -i goclaw-postgres pg_restore -U goclaw -d goclaw --clean --if-exists --no-owner < /opt/goclaw/backups/.dump +sudo /usr/local/bin/goclaw-deploy /opt/goclaw/current +``` + +## Operational Notes + +- Gateway runs as Linux user `goclaw`. +- Host-control exceptions are deployment-specific and must be documented privately, not in this public runbook. +- Workspace restriction settings are deployment-specific and must be reviewed before enabling agent-controlled host operations. +- Postgres is bound only to localhost. +- UFW allows the private SSH port, `80/tcp`, and `443/tcp`. +- Reboot is recommended later because the VPS reports a pending kernel upgrade. + +## Troubleshooting + +Logs: + +```bash +journalctl -u goclaw -n 200 --no-pager +docker logs goclaw-postgres --tail 100 +tail -n 100 /var/log/nginx/error.log +``` + +Restart: + +```bash +systemctl restart goclaw +docker compose --env-file /opt/goclaw/shared/postgres.env -f /opt/goclaw/shared/docker-compose.postgres.yml restart postgres +systemctl reload nginx +``` + +Unresolved questions: none. diff --git a/docs/project-changelog.md b/docs/project-changelog.md index ff035427..cfebf205 100644 --- a/docs/project-changelog.md +++ b/docs/project-changelog.md @@ -6,6 +6,19 @@ Significant changes, features, and fixes in reverse chronological order. ## 2026-05-17 +### Deployment: VPS hybrid GoClaw setup + +**Operations** + +- Deployed GoClaw to a VPS using bare-metal `systemd` gateway plus Dockerized PostgreSQL 18 pgvector. +- Restored the latest private PostgreSQL backup, then upgraded schema from `57` to `65`. +- Installed Node.js 22 and Codex CLI on the host; interactive `codex --login` remains manual. +- Configured Cloudflare-proxied deployment domain and issued SSL through Certbot/Nginx. +- Added `goclaw-backup-r2.timer` to dump PostgreSQL every 6 hours, upload to private Cloudflare R2 storage, and retain the latest 20 backups. +- Added deployment runbook in `docs/deployment-guide.md`. + +--- + ### CI/CD: dev branch beta automation **Features**