Commit Graph
1581 Commits
Author SHA1 Message Date
Duy /zuey/ faa195156e feat(ci): deploy zuey beta releases (#40)
* feat(ci): deploy zuey beta releases

Add automatic zuey VPS deployment to the dev beta release workflow after prerelease assets are published. Publish beta checksums and make the host upgrade script tolerate beta asset naming and checksum fallback.

* fix(deploy): allow active beta release reruns

Treat an existing target release as success only when it is already the active release and still contains the expected binary and migrations. This keeps automated beta deploy reruns from failing after a prior successful deploy while preserving fail-closed behavior for stale or partial release directories.
2026-05-22 19:49:43 +07:00
Duy /zuey/ 81d96ae014 feat(skills): clarify access modes and repair file paths (#31)
* feat(skills): add bulk management actions

* feat(skills): improve operations UX

* feat(skills): clarify access modes and repair file paths
2026-05-21 16:37:50 +07:00
Duy /zuey/ 56e227c403 feat(api): unblock goclaw-cli P6 trace follow and provider reconnect (#37)
* feat(api): unblock trace follow and provider reconnect

* test(cron): stabilize run log execution test
2026-05-21 16:37:19 +07:00
Duy /zuey/ e51fe80212 fix(http): harden automation API contracts (#36) 2026-05-20 22:17:13 +07:00
Duy /zuey/ ce5ea10d46 fix(runtime): repair package installs, archives, logs, and telegram startup
Merge runtime/package install fixes, archive attachment handling, operator log access, and Telegram startup IPv4 fallback.
2026-05-20 21:26:10 +07:00
Duy /zuey/ 532ff91d8e fix(security): harden upstream critical surfaces (#32)
* fix(security): harden upstream critical surfaces

Refs #30

* fix(security): close pre-landing review gaps

Refs #30

* fix(security): close official release blockers
2026-05-20 16:33:49 +07:00
Duy /zuey/ 2a48e62c6b feat(skills): add bulk management actions
Adds Skills bulk actions, Grant all agents support, header-level skill version selector, and upload write validation.
2026-05-19 18:31:29 +07:00
Duy /zuey/ 1858cebe9c feat(skills): add management metadata and deeplinks
Closes #26
2026-05-19 16:35:56 +07:00
Duy /zuey/ 328cbf3d61 Merge pull request #25 from digitopvn/codex/securecli-openrouter-alias
fix(secure-cli): resolve runtime npm binary aliases
2026-05-19 10:23:14 +07:00
Duy /zuey/ 951733c136 Merge pull request #24 from digitopvn/codex/feat-wait-tool
feat(tools): add built-in wait tool
2026-05-19 09:57:42 +07:00
Goon a08c626867 fix(secure-cli): resolve runtime npm binary aliases 2026-05-19 09:54:21 +07:00
Goon f55ea3dc26 fix(pipeline): remove BOM from deps source 2026-05-19 09:45:50 +07:00
Goon 703c0b2df0 fix(config): remove BOM from channels config 2026-05-19 09:28:13 +07:00
Goon ed558e6f89 merge: bring dev into wait tool branch 2026-05-19 09:18:29 +07:00
Duy Nguyen d48ae43e4f feat(tools): add built-in wait tool 2026-05-19 06:53:45 +07:00
Duy /zuey/ 069f984f2f Merge pull request #19 from digitopvn/fix/github-installer-runtime-bin
fix(packages): use runtime dir for GitHub binaries
2026-05-19 06:51:22 +07:00
Duy Nguyen c174279e01 fix(packages): use runtime dir for GitHub binaries 2026-05-18 21:43:03 +07:00
Duy /zuey/ 077e81c990 Merge pull request #16 from digitopvn/fix/skill-grants-tenant-scope
fix(skills): enforce tenant scope on agent grants
2026-05-18 20:50:02 +07:00
Duy Nguyen 8ef8fc4f73 fix(skills): scope agent grant status joins 2026-05-18 20:44:45 +07:00
Duy /zuey/ 52aa110e1f feat(providers): default ChatGPT OAuth to GPT-5.5
Default ChatGPT Subscription (OAuth) provider model selection to GPT-5.5 and update model metadata, tests, and docs.
2026-05-18 20:33:40 +07:00
Duy /zuey/ 134cc7119f docs(deployment): document codex service user auth (#17) 2026-05-18 20:23:41 +07:00
Goon 2686a6c989 docs: add skill grants beta ship journal 2026-05-18 17:43:25 +07:00
Goon 3a62bb50e8 fix(skills): enforce tenant scope on agent grants
Reject cross-tenant skill grant and revoke operations before grant rows or skill visibility can be changed. Clean legacy invalid grant rows in PostgreSQL and SQLite migrations, hide owner IDs from skill API/UI responses, and cover the tenant-isolation cases with PG and SQLite regression tests.
2026-05-18 17:40:41 +07:00
Goon 8f5aad5c07 Merge remote-tracking branch 'origin/dev' into fix/skill-grants-tenant-scope 2026-05-18 17:18:19 +07:00
Goon 99460a7f8a fix(packages): repack npm fallback without scripts
Create sanitized npm tarballs directly in Go so workspace dependency fallback does not run package lifecycle scripts or create global symlinks to temporary folders.
2026-05-18 16:04:12 +07:00
Goon d8fdf8fcd3 fix(packages): install sanitized npm tarballs
Avoid npm global symlinks to temporary fallback directories by repacking rewritten workspace dependency packages before install.
2026-05-18 15:58:57 +07:00
Goon 1de07cf8c4 fix(packages): support npm workspace dependency installs
Retry npm global installs that fail on workspace protocol dependencies by packing the registry tarball, rewriting workspace ranges to published versions, and installing the sanitized package folder.
2026-05-18 15:53:42 +07:00
Goon b377282164 Merge branch 'dev' of github.com:digitopvn/goclaw into dev 2026-05-18 13:13:47 +07:00
Goon 27cf2d2a20 chore: codex AGENTS.md 2026-05-18 13:13:42 +07:00
Duy Nguyen a33de08119 fix(packages): support bare-metal runtime installs 2026-05-18 06:56:17 +07:00
Duy Nguyen f09b08e44f feat(gateway): add remote release upgrade trigger 2026-05-18 06:56:17 +07:00
Duy Nguyen 0d6c5bbb7c fix(skills): add agent manage grants
Add explicit per-agent manage grants for skills so granted agents can patch/delete skills when ownership identity drifts.

Expose skill owner and manage-grant controls in the web skills UI, and add PostgreSQL/SQLite migrations plus coverage for preserve/revoke behavior.
2026-05-17 23:15:52 +07:00
Duy /zuey/ ba3b535286 fix(agents): handle null JSON config updates (#13)
* fix(agents): handle null JSON config updates

* docs(changelog): note agent provider switch fix

* docs(journal): record agent provider switch fix
2026-05-17 22:34:19 +07:00
Duy Nguyen 0b6ecdecfe docs(deploy): add sanitized VPS deployment runbook 2026-05-17 20:47:33 +07:00
Duy Nguyen 3084896dc3 fix(ci): publish beta release without checkout 2026-05-17 20:11:11 +07:00
Duy Nguyen d931dc04a3 fix(ci): allow ghcr-only beta releases 2026-05-17 18:57:54 +07:00
Duy Nguyen 6a253c46ca fix(test): stabilize ci unit tests 2026-05-17 18:47:46 +07:00
Duy Nguyen 9a6fd2dec9 feat(ci): automate dev beta releases 2026-05-17 18:36:13 +07:00
Duy Nguyen d57d3810e4 fix(cli-credentials): clarify agent grant env controls 2026-05-17 18:01:41 +07:00
Duy Nguyen f1277af8c9 style(protocol): format method constants 2026-05-17 17:20:50 +07:00
Duy Nguyen 9a3aad51ba chore(repo): add agent instructions and ignore backups 2026-05-17 17:07:50 +07:00
Duy Nguyen 506e96cd63 feat(agents): add model fallback configuration 2026-05-17 15:51:56 +07:00
Duy Nguyen 5a189330f9 fix(test): support windows test execution 2026-05-17 15:33:55 +07:00
Duy Nguyen 2fa715ba4e fix(context): share agent scoped context data 2026-05-17 15:33:27 +07:00
Duy Nguyen 536ab4ac6c feat(permissions): add agent channel permission matrix 2026-05-17 15:09:49 +07:00
Duy Nguyen c029e4f6bf feat(cli-credentials): support per-agent env grants
- enforce binary/grant parent checks on nested grant routes
- validate grant binary/agent tenant scope on create
- fail closed on invalid per-user env and preserve per-user precedence
- remove duplicate CLI Credentials sidebar entry while keeping Packages tab route
- refs #12
2026-05-17 14:35:34 +07:00
Duy /zuey/ 425cecb9a3 feat(packages): Phase 2b — apk update flow + pkg-helper v2 protocol (#900) (#7)
* feat(packages): add apk update flow + pkg-helper v2 protocol

- APK update checker/executor via helper IPC (runtime detection, upgrade scan via apk list --upgradable)
- BREAKING: pkg-helper v2 protocol (5 actions: check_apk/check_pip/check_npm/exec_apk/exec_pip, code/data fields, renewable 10min deadline, apkMutex, 1MB scanner)
- Edition gating: SupportsApk + IsAlpineRuntime double-gate (Standard/Full only)
- Backend 3-branch wiring: alpine/apt/yum routes + update_registry, dep_installer helpers
- i18n: 5 apk keys (EN/VI/ZH catalogs)
- Frontend: source pill Alpine badge, APK in updates-list/summary-bar/update-all modal
- E2E tests: apk_e2e build tag covering checker/executor/helper protocol
- Docs: packages-apk.md, security/changelog updates
- Plans + reports under plans/260417-1500-packages-update-phase2b-apk-pkghelper/ + plans/reports/

* docs(packages): journal Phase 2b apk + pkg-helper v2
2026-05-11 15:41:27 +07:00
Duy /zuey/ 6e5e51a18b feat(packages): Phase 2a — pip + npm update flow (#900) (#6)
* feat(packages): backend pip + npm update flow (#900)

Extend Phase 1 update infrastructure to pip + npm sources. Register
checkers/executors behind edition gate (Lite edition stays github-only).
Per-source sentinel errors + stderr classifier; strict package-name
validators reject @version suffix. Shared PackageLocker serializes
install + update paths. HTTP response surfaces per-source availability
from LookPath detection.

Closes part of #900 (Phase 2a).

* feat(packages): frontend multi-source updates UI (#900)

Unified flat updates list with source pill (github/pip/npm) + filter
dropdown. Summary bar shows per-source counts, hiding sources whose
backend availability=false. 30 i18n keys with full en/vi/zh parity.
Mobile-safe table (overflow-x-auto + min-w-[600px]).

Part of #900 (Phase 2a).

* test(packages): pip + npm integration e2e (#900)

Optional real-runtime integration test behind `pipnpm_e2e` build tag.
Skipped by default CI; exercises full check + apply cycle with real
pip3/npm in Alpine container.

Part of #900 (Phase 2a).

* docs(packages): document pip + npm update flow (#900)

Adds packages-pip-npm.md covering command matrix, exit codes, stderr
error classes, pre-release handling, availability detection, runbook
for EACCES/ERESOLVE/externally-managed, min versions, fixture regen.
Cross-link from packages-github.md. Changelogs updated.

Part of #900 (Phase 2a).

* fix(packages): set exec bit on testdata npm/pip scripts
2026-05-11 15:31:32 +07:00
Duy /zuey/ 4472c607b8 feat(workstation): Remote Workstation Runtime — SSH exec + security + audit (#4)
* feat(packages): add update flow for GitHub binaries (#900)

Closes #900. Proactive update-check + atomic swap for GitHub-installed
binaries on the Runtime & Packages page. Interfaces prepared for pip/npm/apk
extension in Phase 2.

- UpdateCache + UpdateRegistry + PackageLocker (ctx-aware keyed mutex)
- GitHubUpdateChecker: ETag-aware, distinct /latest vs /list ETag keys,
  semver-correct ordering via golang.org/x/mod/semver, non-semver fallback
  that refuses to downgrade, pre-release + stable candidate fusion for
  the v1.0.0-rc.1 -> v1.0.0 transition
- GitHubUpdateExecutor: two-phase .bak swap with hadBackup-aware rollback,
  manifest save retry (3x, 100ms/500ms/1s backoff), nil-safe meta access,
  explicit ScratchDir, 0755 set pre-rename
- HTTP: GET /v1/packages/updates (SWR), POST /v1/packages/updates/refresh,
  POST /v1/packages/update, POST /v1/packages/updates/apply-all
  (always 200, failed[] is error source). Master-scope gated.
- WS events package.update.{checked,started,succeeded,failed} forwarded to
  owner clients via event_filter.go
- Frontend: useUpdates hook + 3 components (summary bar, update-all modal,
  row button), master-scope-gated disabled state
- i18n: 8 backend keys + 17 frontend keys x en/vi/zh
- Config: packages.github_token (reserved), updates_check_ttl, scratch_dir
- 45+ new tests, race-clean, BenchmarkCheckAll10Packages ~1.1ms/op warm

* docs(packages): document update flow + Phase 1 completion

- packages-github.md: "Updating Installed Packages" section with UI + API
  contract, troubleshooting runbook (corrupt cache, rate-limit, scratch dir,
  mid-swap recovery)
- 17-changelog.md + CHANGELOG.md: Phase 1 entry
- 14-skills-runtime.md: cross-ref to update flow
- journal entry capturing CRIT fixes (double-write, lock-key mismatch,
  rollback false-alarm) + design wins (keyed locks, red-team pre-flight)

* feat(workstation): remote workstation runtime — SSH exec + security + audit

Adds generic Remote Workstation Runtime enabling agents to execute commands
on user-owned SSH workstations. Includes registry (DB + API + UI), SSH backend
with connection pool and circuit breaker, workstation.exec + claude_remote tools,
NFKC + binary-name allowlist security, and audit logging.

Standard edition only. Closes #941.

* fix(workstation): address 3 critical + 5 important code review findings

- C1: Add json:"-" to Metadata/DefaultEnv fields; use SanitizedView() in
  all API responses to prevent SSH private key leakage
- C2: Wire CheckEnv into PermCheckFn; LD_PRELOAD/PATH injection now blocked
- C3: SSH Setenv fallback — prepend `export K=V;` when server rejects Setenv
- I1: BackendCache sync.RWMutex → sync.Mutex (fix data race on lastUsed)
- I2: Validate metadata shape in handleUpdate before store write
- I3: Include command in exec-done event; activity sink uses actual cmd hash
- I4: Wrap pool release in sync.Once (idempotent double-call safety)
- I5: Verify workstation tenant ownership before adding permissions

* fix(packages): bypass HTTPS+IP validation in update executor tests

Test httptest servers bind to http://127.0.0.1 which fails both the
HTTPS scheme check and literal-IP SSRF guard. Add testSkipDownloadValidation
flag (same pattern as existing withTestDownloadHosts) to skip full URL
validation in test context.

* fix(workstation): address Claude review findings — tenant isolation + pool leak + dead code

- Activity list: add workstation ownership check before listing
  (prevents cross-tenant activity enumeration via known UUID)
- SSH pool: clean up p.sem + p.circuits maps in CloseWorkstation,
  prune, and Close to prevent unbounded map growth
- RPC handlers: return ErrInvalidRequest on JSON unmarshal failure
  instead of silently using zero-value params
- Remove unused containsControlChars function in normalize.go
- HTTP tests: add 10s context timeout to prevent CI package timeout

* fix(workstation): DefaultEnv JSON parse, backend cache leak, perm ownership check

- DefaultEnv: replace KEY=VALUE text parse with json.Unmarshal (stored as
  JSON by HTTP handler, was silently ignored)
- BackendCache: close losing backend on concurrent cache miss to prevent
  pruneLoop goroutine leak
- Backend interface: add Close() error method; SSHBackend delegates to
  pool.Close()
- handlePermList: add wsStore.GetByID ownership check (prevents cross-tenant
  UUID enumeration returning empty array vs 404)
- scanRows: log scan errors instead of silently skipping

* fix(workstation): wire activity sink shutdown + remove misleading comment

- WireActivitySink: capture cleanup func, register in gateway shutdown
  (was discarded → retention goroutine leaked + buffered rows lost)
- Add Stop() to WorkstationActivityStore interface (PG+SQLite already had it)
- wireWorkstationTools returns cleanup func; gateway.go defers it
- Remove misleading "re-validate env" comment in allowlist.go Check()

* ci: bump unit test timeout from 90s to 120s

hooks/handlers package (goja script tests) consumes ~85s on cold CI
runners, leaving insufficient headroom for HTTP retry tests with 1s
backoff. 120s provides adequate breathing room without masking real
deadlocks.

* fix: compile errors in integration tests + allowlist docstring

- packages_update_test: add missing lockKey arg to registry.Apply
- mcp_grant_revoke_test: remove unused fakeMCPClient struct
- allowlist.go: fix Check() docstring to match actual 3-step pipeline

* fix(test): relax mcp grant revoke assertion for pre-Phase02 state

Execute-time grant checking not yet wired — test correctly gets an
error but the message is "no active client" (nil clientPtr) rather
than "grant revoked". Accept any error as valid regression guard.

* chore: trigger CI on digitopvn/goclaw fork

* ci: retrigger workflows

* fix(permissions): classify workstation methods in RBAC policy
2026-05-11 14:58:19 +07:00
Duy /zuey/ ddf8e1099f feat(webhooks): HTTP webhooks to trigger agents with HMAC auth + durable callbacks (#2)
* feat(webhooks): HTTP webhooks to trigger agents with HMAC auth and durable callbacks

Add multi-tenant HTTP webhook endpoints for agent triggering:
- /v1/webhooks/message: send messages to channels
- /v1/webhooks/llm: sync/async LLM prompts with HMAC-signed callbacks
- HMAC-256 + bearer token authentication
- Rate limiting and tenant isolation
- Durable callback worker with exponential backoff
- PG 000056 + SQLite schema v25 migrations
- Unit + integration tests, P0 tenant isolation invariants
- Channel media capability helpers for attachment routing
- Comprehensive webhook documentation and i18n strings

* fix(webhooks): address post-review findings (K1-K10)

Comprehensive post-merge fixes addressing 10 blocking code review issues
and 2 adversarial re-audit findings in webhook-agent-triggering feature:

K1: Fix auth middleware tenant context lookup sequencing — move
    tenant context injection before authenticate() call to prevent
    unscoped secret lookups.

K2: Canonicalize JSON payload format for jsonb compatibility across
    PostgreSQL and SQLite — ensure consistent serialization without
    whitespace variance to prevent hash mismatches.

K3: Add fail-closed JSON parsing in body hash extraction with explicit
    error handling for malformed payloads before HMAC verification.

K4: Fix worker queue wedge by properly draining slot reservations
    when delivery succeeds, preventing permanent slot occupancy.

K5: Implement lease-token optimistic concurrency control to prevent
    duplicate webhook delivery under high concurrency or retry storms.

K6: Add AES-256-GCM encrypted secret storage at rest with fail-fast
    skip-mount when GOCLAW_ENCRYPTION_KEY environment variable unset.

K7: Implement IP allowlist enforcement supporting both CIDR ranges
    and exact IP matching with proper X-Forwarded-For parsing.

K8: Add HMAC replay nonce cache (5min expiry, non-blocking async flush)
    to prevent request replay attacks on webhook handler.

K9: Fix invariant test schema selection — replace hardcoded assumption
    with explicit schema name from config to support multi-schema testing.

K10: Consolidate rate limiters into single shared instance to prevent
     per-endpoint limiter starvation and ensure fair rate limiting.

New database migrations:
- 000057: webhook_calls.lease_token for optimistic concurrency
- 000058: webhooks.encrypted_secret_key for AES-256-GCM encryption

New i18n keys: MsgWebhookIPDenied, MsgWebhookEncryptionUnavailable
(with English, Vietnamese, Chinese translations).

New modules:
- internal/http/webhooks_payload.go: JSON canonicalization + body hash
- internal/http/webhooks_nonce.go: Replay nonce cache implementation
- internal/http/webhooks_idempotency_test.go: Integration tests

Documentation updates:
- docs/webhooks.md: §13-14 security sections, encryption flow
- docs/00-architecture-overview.md: webhook subsystem security overview
- docs/codebase-summary.md: webhook security patterns
- docs/project-changelog.md: webhook fixes changelog

Test coverage: 53 webhook tests + 4 P0 invariant tests all passing.
No tenant isolation violations. All security gates enforced.

* docs(journals): webhook feature ship + fix cycle entries

* fix(webhooks): address Claude review findings

- webhooks_llm.go: remove misleading ptr() helper; use &completedAt
  pattern for error-path audit rows (matches success path)
- webhooks_auth.go: wrap TouchLastUsed context in WithoutCancel so
  background DB update isn't cancelled when HTTP response completes
- store GetByIDUnscoped (PG+SQLite): add NOT revoked / revoked = 0
  filter for defense-in-depth parity with GetByHashUnscoped
- webhooks/sign.go: fix package doc — HMAC key is raw plaintext
  secret bytes, not hex-decoded SHA-256
- webhooks_admin.go: check auth before encKey guard to avoid leaking
  config state to unauthenticated callers
- webhooks_ratelimit.go: two-phase Load→LoadOrStore to avoid per-call
  entry allocation on the hot path

* docs(webhooks): fix Sign() function doc to match actual key input

Function-level comment still referenced hex-decoded SecretHash after
the package-level doc was corrected. Align with actual caller usage
([]byte(rawSecret)).

* fix(webhooks): use WithoutCancel for worker execute DB updates

Terminal status writes in execute() ran through the worker main-loop
ctx, which is cancelled on graceful shutdown. If the outbound send
completed but the status update raced with shutdown, the row stayed
in 'running' and got re-delivered via reclaimStale. WithoutCancel
lets the DB write survive worker cancellation while preserving
propagated values (tenant ID, etc.).

* fix(webhooks): move tctx init before panic defer in worker execute

Panic recovery called updateRetry with raw ctx (no tenant ID), making
requireTenantID fail and the reset-to-retry DB write silently drop.
Row stayed 'running' until reclaimStale (~90s delay). Init tctx first
so defer closure captures tenant-scoped non-cancellable context.

* fix(webhooks): pass tenant-scoped tctx to invokeAgent in worker

execute() was passing the raw worker-loop ctx (no tenant ID) to
invokeAgent → router.Get → PGAgentStore.GetByID. GetByID reads
TenantIDFromContext which returned uuid.Nil, making every lookup
return 'agent not found'. Async LLM webhook calls silently failed
all retries. Pass tctx (already tenant-scoped + WithoutCancel) so
the router resolves the agent correctly.

* fix(tests): resolve integration test compile errors

- Remove duplicate contains() in mcp_grant_revoke_test.go (already
  defined in tts_gemini_live_test.go)
- Update webhooks_admin_test.go RotateSecret call to match current
  5-arg signature (newSecretHash, newPrefix, newEncryptedSecret)

* fix(webhooks): default nil scopes/ip_allowlist to empty slice in Create

PG columns are NOT NULL DEFAULT '{}'. Explicit NULL from pqStringArray(nil)
violated the constraint, breaking TestWebhookAdminCRUD/TenantIsolation.
Coerce nil slices to empty []string{} so the default applies at the DB layer.

* chore: trigger CI on digitopvn/goclaw fork

* ci: retrigger workflows

* fix(webhooks): renumber migrations to 000059-000061 for merge train
2026-05-11 13:29:24 +07:00