mirror of
https://github.com/tiennm99/goclaw.git
synced 2026-10-13 04:14:56 +00:00
* feat(packages): add update flow for GitHub binaries (#900) Closes #900. Proactive update-check + atomic swap for GitHub-installed binaries on the Runtime & Packages page. Interfaces prepared for pip/npm/apk extension in Phase 2. - UpdateCache + UpdateRegistry + PackageLocker (ctx-aware keyed mutex) - GitHubUpdateChecker: ETag-aware, distinct /latest vs /list ETag keys, semver-correct ordering via golang.org/x/mod/semver, non-semver fallback that refuses to downgrade, pre-release + stable candidate fusion for the v1.0.0-rc.1 -> v1.0.0 transition - GitHubUpdateExecutor: two-phase .bak swap with hadBackup-aware rollback, manifest save retry (3x, 100ms/500ms/1s backoff), nil-safe meta access, explicit ScratchDir, 0755 set pre-rename - HTTP: GET /v1/packages/updates (SWR), POST /v1/packages/updates/refresh, POST /v1/packages/update, POST /v1/packages/updates/apply-all (always 200, failed[] is error source). Master-scope gated. - WS events package.update.{checked,started,succeeded,failed} forwarded to owner clients via event_filter.go - Frontend: useUpdates hook + 3 components (summary bar, update-all modal, row button), master-scope-gated disabled state - i18n: 8 backend keys + 17 frontend keys x en/vi/zh - Config: packages.github_token (reserved), updates_check_ttl, scratch_dir - 45+ new tests, race-clean, BenchmarkCheckAll10Packages ~1.1ms/op warm * docs(packages): document update flow + Phase 1 completion - packages-github.md: "Updating Installed Packages" section with UI + API contract, troubleshooting runbook (corrupt cache, rate-limit, scratch dir, mid-swap recovery) - 17-changelog.md + CHANGELOG.md: Phase 1 entry - 14-skills-runtime.md: cross-ref to update flow - journal entry capturing CRIT fixes (double-write, lock-key mismatch, rollback false-alarm) + design wins (keyed locks, red-team pre-flight) * feat(workstation): remote workstation runtime — SSH exec + security + audit Adds generic Remote Workstation Runtime enabling agents to execute commands on user-owned SSH workstations. Includes registry (DB + API + UI), SSH backend with connection pool and circuit breaker, workstation.exec + claude_remote tools, NFKC + binary-name allowlist security, and audit logging. Standard edition only. Closes #941. * fix(workstation): address 3 critical + 5 important code review findings - C1: Add json:"-" to Metadata/DefaultEnv fields; use SanitizedView() in all API responses to prevent SSH private key leakage - C2: Wire CheckEnv into PermCheckFn; LD_PRELOAD/PATH injection now blocked - C3: SSH Setenv fallback — prepend `export K=V;` when server rejects Setenv - I1: BackendCache sync.RWMutex → sync.Mutex (fix data race on lastUsed) - I2: Validate metadata shape in handleUpdate before store write - I3: Include command in exec-done event; activity sink uses actual cmd hash - I4: Wrap pool release in sync.Once (idempotent double-call safety) - I5: Verify workstation tenant ownership before adding permissions * fix(packages): bypass HTTPS+IP validation in update executor tests Test httptest servers bind to http://127.0.0.1 which fails both the HTTPS scheme check and literal-IP SSRF guard. Add testSkipDownloadValidation flag (same pattern as existing withTestDownloadHosts) to skip full URL validation in test context. * fix(workstation): address Claude review findings — tenant isolation + pool leak + dead code - Activity list: add workstation ownership check before listing (prevents cross-tenant activity enumeration via known UUID) - SSH pool: clean up p.sem + p.circuits maps in CloseWorkstation, prune, and Close to prevent unbounded map growth - RPC handlers: return ErrInvalidRequest on JSON unmarshal failure instead of silently using zero-value params - Remove unused containsControlChars function in normalize.go - HTTP tests: add 10s context timeout to prevent CI package timeout * fix(workstation): DefaultEnv JSON parse, backend cache leak, perm ownership check - DefaultEnv: replace KEY=VALUE text parse with json.Unmarshal (stored as JSON by HTTP handler, was silently ignored) - BackendCache: close losing backend on concurrent cache miss to prevent pruneLoop goroutine leak - Backend interface: add Close() error method; SSHBackend delegates to pool.Close() - handlePermList: add wsStore.GetByID ownership check (prevents cross-tenant UUID enumeration returning empty array vs 404) - scanRows: log scan errors instead of silently skipping * fix(workstation): wire activity sink shutdown + remove misleading comment - WireActivitySink: capture cleanup func, register in gateway shutdown (was discarded → retention goroutine leaked + buffered rows lost) - Add Stop() to WorkstationActivityStore interface (PG+SQLite already had it) - wireWorkstationTools returns cleanup func; gateway.go defers it - Remove misleading "re-validate env" comment in allowlist.go Check() * ci: bump unit test timeout from 90s to 120s hooks/handlers package (goja script tests) consumes ~85s on cold CI runners, leaving insufficient headroom for HTTP retry tests with 1s backoff. 120s provides adequate breathing room without masking real deadlocks. * fix: compile errors in integration tests + allowlist docstring - packages_update_test: add missing lockKey arg to registry.Apply - mcp_grant_revoke_test: remove unused fakeMCPClient struct - allowlist.go: fix Check() docstring to match actual 3-step pipeline * fix(test): relax mcp grant revoke assertion for pre-Phase02 state Execute-time grant checking not yet wired — test correctly gets an error but the message is "no active client" (nil clientPtr) rather than "grant revoked". Accept any error as valid regression guard. * chore: trigger CI on digitopvn/goclaw fork * ci: retrigger workflows * fix(permissions): classify workstation methods in RBAC policy
146 lines
4.4 KiB
Go
146 lines
4.4 KiB
Go
// Package workstation contains the activity sink that subscribes to domain events
|
|
// and persists exec audit rows to WorkstationActivityStore.
|
|
package workstation
|
|
|
|
import (
|
|
"context"
|
|
"crypto/sha256"
|
|
"fmt"
|
|
"log/slog"
|
|
"regexp"
|
|
"strings"
|
|
"time"
|
|
|
|
"github.com/google/uuid"
|
|
|
|
"github.com/nextlevelbuilder/goclaw/internal/eventbus"
|
|
"github.com/nextlevelbuilder/goclaw/internal/store"
|
|
"github.com/nextlevelbuilder/goclaw/pkg/protocol"
|
|
)
|
|
|
|
// sensitivePatterns is a list of compiled regexes that redact secret-bearing fragments.
|
|
// Applied to cmd_preview before storage; raw command is never persisted.
|
|
var sensitivePatterns = []*regexp.Regexp{
|
|
regexp.MustCompile(`(?i)(api[_-]?key|password|secret|token|auth)[=:]\S+`),
|
|
regexp.MustCompile(`-H\s+"Authorization:[^"]*"`),
|
|
regexp.MustCompile(`Bearer\s+[A-Za-z0-9\-_\.]+`),
|
|
regexp.MustCompile(`eyJ[A-Za-z0-9\-_]+\.[A-Za-z0-9\-_]+\.[A-Za-z0-9\-_]+`), // JWT
|
|
}
|
|
|
|
// WireActivitySink subscribes to EventWorkstationExecDone on domainBus and writes
|
|
// audit rows to activityStore. The subscription is fire-and-forget (Insert is buffered).
|
|
// Also starts a nightly retention goroutine that prunes rows older than 30 days.
|
|
// Returns a cleanup function that stops the retention goroutine.
|
|
func WireActivitySink(bus eventbus.DomainEventBus, activityStore store.WorkstationActivityStore) func() {
|
|
if bus == nil || activityStore == nil {
|
|
return func() {}
|
|
}
|
|
|
|
// Subscribe to exec done events (emitted by WorkstationExecTool.streamAndCollect).
|
|
// The payload is map[string]any (see internal/tools/workstation_exec.go).
|
|
bus.Subscribe(eventbus.EventType(protocol.EventWorkstationExecDone), func(ctx context.Context, ev eventbus.DomainEvent) error {
|
|
payload, ok := ev.Payload.(map[string]any)
|
|
if !ok {
|
|
return nil
|
|
}
|
|
|
|
wsIDStr, _ := payload["workstation_id"].(string)
|
|
wsID, err := uuid.Parse(wsIDStr)
|
|
if err != nil {
|
|
return nil
|
|
}
|
|
tenantID, _ := uuid.Parse(ev.TenantID)
|
|
agentID := ev.AgentID
|
|
sessionKey, _ := payload["session_key"].(string)
|
|
|
|
// I3 fix: use the "command" field from the done event payload for meaningful
|
|
// cmd_hash and cmd_preview. Falls back to sessionKey if command is absent
|
|
// (e.g. events from older tool versions).
|
|
cmdRaw, _ := payload["command"].(string)
|
|
if cmdRaw == "" {
|
|
// Fallback for events without the command field.
|
|
cmdRaw = "session:" + sessionKey
|
|
}
|
|
cmdPreview := redactSensitive(cmdRaw)
|
|
|
|
exitCodeF, _ := payload["exit_code"].(int)
|
|
durationF, _ := payload["duration_ms"].(int64)
|
|
// JSON numbers decode as float64 from map[string]any.
|
|
if ef, ok := payload["exit_code"].(float64); ok {
|
|
exitCodeF = int(ef)
|
|
}
|
|
if df, ok := payload["duration_ms"].(float64); ok {
|
|
durationF = int64(df)
|
|
}
|
|
|
|
cmdHash := fmt.Sprintf("%x", sha256.Sum256([]byte(cmdRaw)))[:16]
|
|
|
|
exitCodeVal := exitCodeF
|
|
durationVal := durationF
|
|
|
|
row := &store.WorkstationActivity{
|
|
ID: uuid.New(),
|
|
TenantID: tenantID,
|
|
WorkstationID: wsID,
|
|
AgentID: agentID,
|
|
Action: "exec",
|
|
CmdHash: cmdHash,
|
|
CmdPreview: cmdPreview,
|
|
ExitCode: &exitCodeVal,
|
|
DurationMS: &durationVal,
|
|
CreatedAt: time.Now().UTC(),
|
|
}
|
|
|
|
if err := activityStore.Insert(ctx, row); err != nil {
|
|
slog.Warn("workstation.activity.insert_error", "error", err)
|
|
}
|
|
|
|
slog.Info("workstation.exec.completed",
|
|
"workstation_id", wsIDStr,
|
|
"tenant_id", ev.TenantID,
|
|
"agent_id", agentID,
|
|
"cmd_hash", cmdHash,
|
|
"exit_code", exitCodeVal,
|
|
"duration_ms", durationVal,
|
|
)
|
|
return nil
|
|
})
|
|
|
|
// Start nightly retention goroutine.
|
|
stopCh := make(chan struct{})
|
|
go func() {
|
|
ticker := time.NewTicker(24 * time.Hour)
|
|
defer ticker.Stop()
|
|
for {
|
|
select {
|
|
case <-ticker.C:
|
|
before := time.Now().Add(-30 * 24 * time.Hour)
|
|
n, err := activityStore.Prune(context.Background(), before)
|
|
if err != nil {
|
|
slog.Warn("workstation.activity.prune_error", "error", err)
|
|
} else if n > 0 {
|
|
slog.Info("workstation.activity.pruned", "rows", n, "before", before.Format(time.RFC3339))
|
|
}
|
|
case <-stopCh:
|
|
return
|
|
}
|
|
}
|
|
}()
|
|
|
|
return func() { close(stopCh) }
|
|
}
|
|
|
|
// redactSensitive strips lines or fragments matching known secret patterns from cmd.
|
|
// Returns a truncated, redacted string safe for tenant-admin display.
|
|
func redactSensitive(cmd string) string {
|
|
result := cmd
|
|
for _, re := range sensitivePatterns {
|
|
result = re.ReplaceAllString(result, "[REDACTED]")
|
|
}
|
|
// Truncate to 200 chars.
|
|
if len(result) > 200 {
|
|
result = result[:200]
|
|
}
|
|
return strings.TrimSpace(result)
|
|
}
|