* feat(packages): add GitHub Releases binary installer
New runtime source `github:owner/repo[@tag]` for installing Linux CLI
binaries from GitHub Releases. Admin-only, SHA256-verified, ELF-validated.
Backend:
- GitHub API client with 10-min cache + rate-limit mapping
- SSRF-guarded streaming downloader (HTTPS + host allowlist, re-validated
on every redirect hop, literal-IP rejection)
- Checksums.txt / SHA256SUMS lookup with constant-time verify
- Archive extract (tar.gz / zip / raw) with path-traversal + zip-bomb
guards, symlink skip
- ELF magic + 64-bit class + runtime-arch validation
- Atomic manifest persistence (temp + rename)
HTTP:
- POST /v1/packages/install accepts github: spec
- GET /v1/packages/github-releases for picker UI (viewer+, arch-filtered)
- Extended InstalledPackages response with github field
- github-bin runtime probe
Infra:
- Dockerfile creates /app/data/.runtime/bin (goclaw:goclaw 0755)
- docker-entrypoint.sh prepends bin dir to PATH
- Env-only config (never config.json): token, max size, org allowlist,
bin dir, manifest path
UI:
- GitHub Binaries section + release picker modal
- Dismissable musl/glibc compatibility warning (localStorage)
- i18n keys across en/vi/zh
Docs: docs/packages-github.md user guide + 14-skills-runtime.md cross-ref.
Closes#741
* refactor(packages): revert validPkgName broadening + drop unused sentinel
Code review cleanup:
- validPkgName regex had `:` added defensively, but github: specs are
validated separately via skills.ParseGitHubSpec before reaching this
check — the broadening was dead attack surface.
- Drop unused ErrUnknownArchive sentinel + the `_ = ErrUnknownArchive`
stub in extractRaw.
* feat(packages): per-user rate limit on /v1/packages/github-releases
Cap picker endpoint at 30 req/min/user (burst 10) to protect the shared
GitHub API quota. Key is userID (header X-GoClaw-User-Id) or remote IP
for anonymous callers. Returns 429 + Retry-After: 60 when tripped.
Standalone token-bucket limiter (stale-entry cleanup every 5 min) lives
in internal/http rather than importing internal/gateway, which would
create a package cycle.
* fix(ui): guard split()[0] for noUncheckedIndexedAccess strict TS
CI pnpm build failed on TS2345: `.split('@')[0]` returns
`string | undefined` under strict index access. Default to empty
string to satisfy the type checker; runtime behaviour unchanged
because the downstream regex rejects empty strings.
* fix(packages): address Claude review — medium + low findings
Medium
- rate limiter: atomic.Int64 lastSeen + amortized sweep replaces
goroutine-based cleanup → fixes data race on lastSeen and the
goroutine leak when tests swap the package-level limiter.
- checksum pipeline: slog.Warn on ReadFile and ParseChecksums failures
(previously silent). "asset not listed" stays warn+proceed but is now
documented as the publisher's choice — ELF validation remains the
final gate.
- downloader: drop http.Client.Timeout (30s capped the whole request
including body read, aborting large downloads on slow links). Context
deadline from install timeout (5 min) is the correct bound.
Low / style / UI
- extractRaw honors maxUncompressed (ErrFileTooLarge on overflow) so
the helper is safe outside the hot path.
- cmd/gateway_github_installer.go: drop the explicit cfg.Defaults()
call — NewGitHubInstaller already invokes it.
- GitHubPackageEntry: remove unpopulated InstalledBy field + document
why.
- owner regex tightened to 39-char GitHub limit (was 40).
- mu lock comment corrected: serializes only the disk-write phase.
- UI: shared stripPrefixAndTag helper + owner regex mirrors the backend
39-char cap; destructure-with-default kills the split()[0] ?? ""
awkwardness while still satisfying noUncheckedIndexedAccess.
Verified: go build (pg + sqliteonly) · go vet · go test -race
./internal/skills ./internal/http · pnpm build.
* fix(packages): address Claude review round 2
Medium
- validRepoPath now rejects trailing hyphens in the owner segment and
caps at 39 chars, matching gitHubSpecRE exactly. Previously a subtle
drift between the two validators could let `foo-/repo` slip to the
GitHub API and surface as a 502 instead of a clean 400.
- handleGitHubReleases no longer forwards raw err.Error() from the
upstream call. Maps sentinel errors:
ErrGitHubRateLimited → 429 + Retry-After
ErrGitHubNotFound → 404
ErrGitHubUnauthorized → 502 "github authentication failed"
default → 502 "failed to fetch releases"
Avoids leaking rate-limit reset timestamps / server internals to
viewer-tier callers.
Low / UX
- Install response now returns the manifest entry for github: specs
(new lookupGitHubEntry helper; nil-safe fallback to {ok:true}). Lets
the UI display "installed: lazygit v0.42.0" without a list refresh.
- gitHubSpecRE tag segment capped at 1..255 chars (git ref-name bound).
UI isValidFullSpec mirrors the same cap.
* fix(packages): address Claude review round 3
Medium
- github_api: URL-encode owner, repo, and tag via url.PathEscape when
building API paths. Previously a tag containing '#' would be stripped
as a URL fragment and '?' would inject a query parameter, silently
hitting the wrong release.
Low / polish
- Uninstall via full "github:owner/repo[@tag]" spec now falls back to
manifest lookup by owner/repo, handling packages whose binary name
differs from the repo name (cli/cli → gh).
- GitHubClient.cache sweeps expired entries opportunistically when the
map grows past 256 entries (prevents theoretical unbounded growth
over long uptime).
- handleInstall for github: specs now calls GitHubInstaller.Install
directly and returns the freshly-created manifest entry, eliminating
the double manifest read via List() from the lookupGitHubEntry helper.
- pickBinaries comment corrected — actual behavior excludes paths
matched by nonBinaryPathRE rather than enforcing a single-depth limit.
* fix(packages): address Claude review round 4 (style + ordering)
All 4 findings are Low severity:
- github_api.go: replace interface{} with any across the cache type,
cacheGet return, cacheSet param, and doJSON out param.
- doJSON: rename local `url` to `apiURL` to avoid shadowing the
"net/url" package import used by GetRelease/ListReleases.
- Uninstall: save the updated manifest BEFORE removing binaries on
disk. If saveManifest fails we now bail out without leaving a
manifest entry that still claims binaries which have been deleted
(a retried Uninstall would otherwise hit ErrPackageNotInstalled
after the first attempt wiped the files). Disk removal stays
best-effort and warn-on-error, which matches the idempotent intent.
- pickBinaries: inline comment corrected to reflect actual behavior —
depth is not enforced; nonBinaryPathRE filter + downstream ELF
validation are the real gates.
* fix(packages): address Claude review round 5
All 3 findings are Low severity:
- handleInstall github fast-path now wraps the context with
skills.InstallTimeout (5 min) before calling gh.Install and emits the
same "skills: installing dep" / "dep installed" / "github install
failed" log lines as the generic InstallSingleDep path, so
operator-observability is identical between github: and pip:/npm:
install flows.
- installTimeout promoted to exported InstallTimeout so the http layer
shares the single source of truth rather than duplicating the
5-minute constant.
- cacheMaxEntries comment clarifies it is a soft sweep trigger, not a
hard cap — when every entry is still within TTL the map can briefly
exceed the threshold by one insert.
* fix(packages): address Claude review round 6 (final Lows)
Both findings are Low severity (reviewer marked the PR "ready to merge"
already):
- github_installer: "no checksum asset available" downgraded from
slog.Warn to slog.Info. Many popular upstream releases (jq, fzf,
older ripgrep, etc.) ship no checksum file at all — that is publisher
policy, not a problem with the install. The suspicious cases
(checksum file unreadable, unparseable, or missing this asset) stay
at Warn so they stand out.
- handleGitHubReleases response now uses a narrow assetPreview DTO
(name + size_bytes) instead of embedding the full GitHubAsset type
which also carried browser_download_url. The picker UI never rendered
the URL; trimming the response keeps the viewer-tier surface minimal.
UI AssetPreview interface realigned to match.
* fix(packages): address Claude review round 7
Narrow the GET /v1/packages GitHub entry to a viewer-safe projection
(repo/tag/binaries/name/installed_at), mirroring the assetPreview fix
from round 6. Strips asset_url, sha256, and asset_name from the list
response — viewer-level callers no longer see CDN download URLs or
checksum metadata for installed packages. UI types realigned; the
removed fields were never rendered.
Finding #2 (install writes binary before manifest save) left as noted —
reviewer confirmed informational only, self-heals on retry, no security
impact since binaries pass ELF validation before being written.
* fix(packages): address Claude review round 8
Map HTTP 429 (GitHub secondary rate limits — abuse detection,
unauthenticated bursts, search) to ErrGitHubRateLimited in the API
client so the picker endpoint renders 429 "rate limit reached" with
Retry-After: 60 instead of falling through to 502 "failed to fetch
releases". Primary rate limits (403 + X-RateLimit-Remaining: 0) were
already handled; this covers the secondary class documented at
https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api#about-secondary-rate-limits
* fix(packages): address Claude review round 9
Two defensive hardenings flagged as Very Low:
- ParseChecksums: strip leading `./` from checksum filenames.
`sha256sum ./file` emits `./file` in the name column; the caller
looks up by bare asset basename so `./`-prefixed entries would
silently miss. Real release checksums almost never use this form,
but the guard is essentially free.
- doJSON: cap response body at 8 MiB via io.LimitReader before JSON
decode. Current GitHub list/release payloads are well under this
(~1 MiB at per_page=100). Guards against future call sites or a
misbehaving upstream returning an oversized document.
* fix(cron): eliminate cross-test race on runLoopTickInterval
`Service.Stop()` closes stopChan but does not wait for the runLoop
goroutine to exit. In the test suite, test A's `defer cs.Stop()` can
return before the spawned runLoop has reached
`ticker := time.NewTicker(runLoopTickInterval)`. If test B then calls
`setFastTick()` to mutate the package-level var, the race detector
correctly flags it:
Read at runLoopTickInterval by goroutine A (runLoop ticker init)
Previous write by goroutine B (setFastTick in test B)
Fix: snapshot `runLoopTickInterval` inside `Start()` under the mutex
before spawning the goroutine, and pass the value as a parameter to
`runLoop`. The spawned goroutine no longer reads the package-level
var, so the cross-test window is closed. Production behavior
unchanged.
Verified: `go test -race -count=3 ./internal/cron/...` passes three
times in a row; the CI failure on PR #898 reproduced before the fix
and is gone after.
* fix(packages): address review P0/P1/P2 + new DoS vector
P0.1 — UI uninstall 400: parseAndValidatePackage now accepts
github:<bare-name> (manifest Name form, no owner/repo) in addition
to the full spec. UI sends github:${pkg.name} from the manifest;
dispatcher already tolerated bare names — the HTTP validator was
the only gate rejecting them. Install path re-validates strictly
via ParseGitHubSpec and bare-name install returns 400 now (was 500).
P1.1 — ExtractArchive raw-ELF fallback name: add ExtractArchiveAs(
path, fallbackName, max). Installer passes parsed.Repo so raw
(non-archive) ELF assets no longer end up recorded as
/tmp/goclaw-gh-asset-XXXX.bin — that basename would leak into the
manifest Binaries entry and break PATH lookup.
P1.3 — Archive entry count cap: maxArchiveEntries = 10_000 +
ErrTooManyEntries sentinel. Tar: count ALL headers seen (incl.
symlinks/dirs we skip) to block the gzip-bomb-of-headers DoS —
header bytes don't count against maxUncompressed for zero-size
entries. Zip: pre-check via peekZipEntryCount reads the EOCD
record manually and rejects oversized archives BEFORE
zip.OpenReader allocates []*zip.File of declared capacity (this
was a fresh red-team finding; stdlib would otherwise alloc ~1GB
for a crafted 200MB zip claiming 4M entries).
P1.4 — Rate-limit install/uninstall: packagesWriteLimiter
(10/min/user, burst 3). Admin-only mitigates but a compromised
token could otherwise flood upstream (GitHub/pip/npm) or spam
manifest mutations.
P1.6 — Non-Linux early reject: ErrUnsupportedOS guard at the top
of Install(). Windows/macOS hosts no longer waste bandwidth
fetching a Linux asset just to fail at the ELF machine check.
P1.7 — Manifest fsync: OpenFile → Write → Sync → Close → Rename →
dir Sync, with tmp cleanup on every error path. POSIX doesn't
guarantee durability via rename alone; XFS / ext4 with async
journal can reorder.
P2.1 — Belt-and-suspenders zip runtime break when cumulative bytes
reach the cap (pre-declared check already covers it but the
streaming loop now bails immediately).
P2.6 — Binary-name collision warn: slog.Warn when a different repo
already owns the basename we're about to overwrite. Last-writer-
wins unchanged; operator now gets a signal instead of silence.
Hardening — rate-limit key: rateLimitKeyFromRequest prefers
store.UserIDFromContext over the raw X-GoClaw-User-Id header so
an admin can't rotate the header mid-session to dodge the bucket.
Header/IP fallback retained for pre-auth / test callers.
Tests: 9 new cases on parseAndValidatePackage (github full/bare/
empty/traversal/injection/space/leading-hyphen);
TestExtractArchiveAs_RawELFUsesFallbackName;
TestExtractTarGz_EntryCountCap + TestExtractZip_EntryCountCap;
TestPeekZipEntryCount (DoS pre-check path).
Verified: go build ./... && go build -tags sqliteonly ./... &&
go vet ./... && go test -race ./internal/skills/... ./internal/http/...
---------
Co-authored-by: viettranx <viettranx@gmail.com>
Slow tests were dominating CI feedback time and AI dev loop because they
waited through real exponential backoffs and 1s ticker intervals.
Test-only override pattern keeps production behavior 100% identical.
Speed wins (no-race wall-clock per package):
- internal/vault 16.3s -> 0.6s (-15.7s)
- internal/cron 11.7s -> 1.5s (-10.2s)
- internal/channels/facebook 6.3s -> 3.0s (-3.3s)
- Full -race ./... suite 90s+ -> 51s
Changes:
- vault: new fastBackoffsForTest(t) helper overrides enrichRetryBackoffs
+ enrichRetryTimeouts to 1ms in 3 retry tests; drop 2 duplicate tests
(FirstAttemptSuccess, MaxRetriesConstant)
- cron: extract runLoopTickInterval as package var (default 1s); test-only
setFastTick(t) helper shortens to 20ms so 6 scheduler tests no longer
sleep 1.5s each waiting for a tick
- facebook: extract graphBackoffBase as package var (default 1s); newFakeGraph
helper shortens to 1ms so HTTP retry tests don't burn 6s of real waits
Coverage ratchet removed:
- Delete scripts/check_coverage.go + scripts/coverage_thresholds.json
- Remove "Coverage ratchet gate" CI step
- Keep coverage profile + go tool cover summary as informational only
- Philosophy: signal over coverage %. Forced tests to bump % were the
root cause of the slowness this commit unwinds.
Production behavior unchanged. Coverage profile shows isolated package
coverage matches prior thresholds (vault 27.4%, cron 73.7%, facebook 81.9%).
* fix(cron): prevent scheduler loop from blocking when a job hangs
The cron scheduler's runLoop calls checkAndRunDueJobs() every second,
which previously used wg.Wait() to block until ALL claimed jobs complete.
If any single job hung (LLM provider timeout, agent loop stuck, network
issue), wg.Wait() would block indefinitely, preventing the scheduler
from ever checking for new due jobs — effectively killing all cron
scheduling until a container restart.
Changes:
- Remove wg.Wait() from both PG and SQLite cron schedulers — jobs now
run as independent goroutines that don't block the check loop
- Add panic recovery to PG runLoop (safeCheckAndRunDueJobs wrapper)
and per-job goroutines, matching the existing safego.Recover pattern
in the SQLite scheduler
- Add 10-minute context timeout to the cron job handler so a hung
agent run is cancelled instead of blocking forever
- Use select with context.Done() in the handler to respect the timeout
when waiting for the scheduler outcome
- Invalidate PG job cache per-job on completion instead of after the
(now-removed) batch wait
The SQLite scheduler already had safego.Recover on job goroutines but
still used wg.Wait() — this commit removes that blocking wait as well.
* fix(cron): make job timeout configurable + add SQLite runLoop panic recovery
- Add `cron.job_timeout` config field (Go duration string, default "10m")
so operators can tune the per-job timeout for complex agent workloads
without code changes
- Add `safeCheckJobs` panic recovery wrapper to SQLite cron runLoop,
matching the PG scheduler's `safeCheckAndRunDueJobs` for consistency
- Use dynamic timeout string in error message for better diagnostics
* fix: remove unused "time" import from gateway_cron.go
* fix(cron): apply same fixes to SQLite DB scheduler (sqlitestore)
The SQLite DB-backed scheduler (used by desktop edition with SQLite
backend) had the exact same wg.Wait() blocking issue and missing
panic recovery as the PG scheduler. Apply identical fixes:
- Remove wg.Wait() — jobs run as independent goroutines
- Add safeCheckAndRunDueJobs panic recovery wrapper for runLoop
- Add per-job panic recovery and cache invalidation
---------
Co-authored-by: Luvu182 <208665161+Luvu182@users.noreply.github.com>
## Summary
- **Flood after restart**: `recomputeStaleJobs()` now advances past-due jobs (not just NULL) to their next future run time, preventing all missed jobs from firing simultaneously on first tick
- **Interval drift**: Anchor-based scheduling computes next run from original scheduled time instead of execution end time, preserving per-job offsets
- **O(1) advance**: Replaced O(N) loop with modular arithmetic to prevent CPU starvation after prolonged downtime with short-interval jobs
- **SQLite RunJob fix**: Added `next_run_at = NULL` claim + `reloadClaimed` param to match PG store behavior — manual runs were silently skipped
- **Manual RunJob consistency**: Nil-anchor guard ensures manual triggers use `now + interval` (not anchor) across all stores
- **Zombie at-job fix**: JSON store Start() now disables past-due one-time `at` jobs instead of leaving them enabled with nil NextRunAtMS
All fixes applied across JSON, PostgreSQL, and SQLite store implementations.
4 new unit tests covering flood prevention, anchor arithmetic, RunJob scheduling, and at-job disabling.
Extract wake_heartbeat and stateless from JSON payload into first-class
columns on cron_jobs. Adds migration 000033 with backfill from existing
payload data. Updates PG + SQLite stores, RPC handlers, and UI i18n.
- Enable merge UI for linking channel contacts to tenant_users
- Contact → tenant_user resolution with cached lookup (60s TTL)
- MCP per-user credentials via user-keyed connection pool
- Secure CLI per-user credentials with AES-256-GCM encryption
- Unified UserPickerCombobox searching contacts + tenant_users
- Group contact collection with chat title in all channels
- Group permission inheritance via wildcard user_id="*"
- Fix heartbeat using wrong userID in group chats
- Filter internal senders from contact collection
- Add contact_type column (user/group) to channel_contacts
- SQLite schema v2 migration for desktop edition
* fix: add panic recovery to tool, cron, and summarization goroutines
A panicking tool, cron job, or auto-summarization goroutine crashes
the entire server process — disconnecting all clients and aborting
all running agents. Add defer/recover at three levels:
- safeExecute() in tools/registry.go: catches panics from any tool's
Execute method and returns an error result to the LLM
- Agent loop goroutine (loop.go): defense-in-depth for non-tool code
in the parallel tool execution goroutine
- Cron execution goroutine (service_execution.go): prevents a single
panicking cron job from taking down the server
- Summarization goroutine (loop_history.go): prevents background
auto-summarization panics from crashing the process
All recovery points log the panic value and stack trace at ERROR level
for debugging. Tool execution panics return actionable error messages
to the LLM so the agent can recover gracefully.
* refactor: extract panic recovery into safego.Recover helper
DRY the duplicated recover+stack+log boilerplate across 4 goroutine
sites into a shared safego.Recover(onPanic, attrs...) function.
Stack buffer bumped from 4096 to 8192 bytes.
---------
Co-authored-by: viettranx <viettranx@gmail.com>
1. providers/anthropic_stream.go — check scanner.Err() after SSE loop;
truncated responses returned silently on network errors
2. config/config_load.go — trim whitespace in GOCLAW_OWNER_IDS;
trailing spaces caused owner exact-match failures
3. cron/service.go — handle saveUnsafe() errors at 5 call sites;
disk I/O failures lost cron state silently
4. tasks/task_ticker.go — separate timeouts for followups (15s) and
recovery (30s); shared context risked followups starving recovery
5. hooks/use-ws-event.ts — useRef pattern for handler to prevent
subscribe/unsubscribe churn with inline functions
6. markdown-renderer.tsx — AbortController for file fetch with abort
on new fetch and on unmount
7. mcp-form-dialog.tsx — auto-split Command field when user pastes
full command; fix splitShellTokens to preserve commas inside quotes
- Update go.mod and Dockerfile to Go 1.26
- Apply `go fix ./...` stdlib modernizations across 170+ files
- Add `go fix` to post-implementation checklist in CLAUDE.md
- Fix go fix misapplied rewrite in loop_history.go
- Simplify cron session key to `agent:{agentId}:cron:{jobID}` (remove redundant `:run:{runID}`)
- Route cron jobs through scheduler's cron lane instead of calling loop.Run() directly
- Scheduler enforces per-session maxConcurrent=1, preventing same job from running concurrently
- Parallelize due job execution with goroutines + WaitGroup (PG and file store)
- Move scheduler creation before cron setup in gateway.go initialization order
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Multi-agent AI gateway with WebSocket RPC, HTTP API, and messaging channel integrations.
Go port of OpenClaw with multi-tenant PostgreSQL, per-user isolation, security hardening,
and production observability.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Goon