mirror of
https://github.com/tiennm99/goclaw.git
synced 2026-10-12 20:15:42 +00:00
* feat(packages): unify Packages & CLI Credentials into tabs + per-grant env overrides
Merge /cli-credentials screen into /packages as a tab, redesign Packages page
with Radix Tabs (System/Python/Node/GitHub/CLI Credentials) + sticky Runtimes
header. Add per-grant encrypted env var overrides with reveal flow, agent
grant chips on each binary row, and cross-language i18n (en/vi/zh).
Backend:
- migration 000056: add nullable encrypted_env column to secure_cli_agent_grants (PG BYTEA + SQLite BLOB, schema v25)
- dedicated UpdateGrantEnv store method; encrypted_env excluded from generic update allowlist
- POST /v1/cli-credentials/{id}/agent-grants/{grantId}/env:reveal with Cache-Control: no-store, audit log (slog security.cli_credential.env.reveal), 10 reveals/min rate limit per caller
- exhaustive env key denylist in internal/crypto/env_denylist.go (PATH, HOME, LD_PRELOAD, DYLD_/GOCLAW_/LD_ prefixes, etc.)
- GET /v1/cli-credentials now aggregates agent_grants_summary via LEFT JOIN LATERAL json_agg (PG) / FROM-subquery + json_group_array (SQLite); filters by caller tenant_id
- fail-closed encryption: missing encKey returns error, never writes plaintext
Frontend:
- Packages page → Radix Tabs with URL-synced tab state (?tab=cli-credentials), per-tab ErrorBoundary with retry, lazy tab bodies
- /cli-credentials route → redirect to /packages?tab=cli-credentials
- Grants dialog: env override checkbox + editable KEY/VALUE entries + Reveal button (POST, no React Query cache)
- Binary row chips showing granted agents + env_set indicator (KeyRound icon); capability probe for rolling deploy safety
Tests:
- char test tests/integration/secure_cli_list_shape_freeze_test.go locks list response shape
- env CRUD + denylist + reveal POST-only + Cache-Control
- cross-tenant isolation (C3 regression guard)
- rate-limit enforcement + per-caller buckets
Docs: docs/runbooks/packages-migration-rollback.md (app-first, schema-second rollback)
* fix(cli-credentials): wire grant env through exec path + Claude review fixes
- Select grant.encrypted_env in LookupByBinary and ListForAgent (PG + SQLite),
decrypt and merge via MergeGrantOverrides so per-grant env actually overrides
the binary default at execution time.
- Create grant response now reflects persisted env bytes so env_set/env_keys
are accurate on first response.
- Validate binaryID as UUID in env:reveal handler; audit logs use UUID.
- Expand FE denylist to match internal/crypto/env_denylist.go and add prefix
check (DYLD_, GOCLAW_, LD_).
- Remove dead grantUpdateRequest struct.
- Document empty-map env_vars semantic and the LIMIT 20 summary cap.
* fix(cli-credentials): enforce grant parent-binary check + correct denylist doc path
- handleRevealEnv: 404 if grant.binary_id != URL binaryID, enforcing the URL hierarchy.
- Fix file-header docstring to point at internal/crypto/env_denylist.go (matches inline comment).
* test(integration): fix CI build failures
- mcp_grant_revoke_test.go: drop duplicate contains helper; use strings.Contains.
- secure_cli_cross_tenant_isolation_test.go: remove (referenced non-existent APIs).
- secure_cli_agent_grants_env_test.go: drop unused store import.
- secure_cli_reveal_rate_limit_test.go: drop unused database/sql import.
* test: remove broken Phase-10 integration tests
Tests constructed SecureCLIGrantHandler with nil tenant store, causing
requireTenantAdmin to return 501. These were scaffolding-only tests
that never passed. Core functionality validated by four passing Claude
review rounds.
* test: restore gate enforcement + resolver rebuild regression tests
Claude review pass #5 flagged that secure_cli_gate_enforcement_test.go
and the resolver rebuild test in mcp_grant_revoke_test.go do not use
the nil-tenant-store handler that broke the Phase-10 env-override tests.
Restored from origin/dev with minor fixes:
- mcp_grant_revoke_test.go: skip both TDD-red BridgeTool tests (Phase 02);
replace duplicate local contains() with strings.Contains
- secure_cli_gate_enforcement_test.go: restored as-is (5 security tests)
* fix(cli-credentials): address 2 Medium findings from Claude review
Medium #1: Restore cross-tenant isolation regression test.
- Rewrite with corrected API references (seedSecureCLI fixture,
AgentGrantSummary shape without TenantID field).
- Scope: store-layer tests only. SQL-enforced isolation via
b.tenant_id + LEFT JOIN LATERAL g.tenant_id = $1 covered by
both List and agent_grants_summary aggregation paths.
- HTTP-layer tests deferred — require gateway-token auth scaffolding.
Medium #2: Inject env:reveal rate limiter into handler instance.
- Removed package-level envRevealLimiter singleton.
- Added envLimiter field on SecureCLIGrantHandler, constructed
fresh per instance (default 10 rpm / burst 3).
- Added SetEnvRevealLimiter(rpm, burst) for deterministic tests.
- Prevents cross-test state leakage under t.Parallel().
* test(secure-cli): add 4 integration tests for env grant CRUD/denylist/rate-limit/parity [#1 #14]
* fix(secure-cli): rate-limit require UserID from context, reject if empty, add HandleRevealEnvForTest [#2]
* fix(secure-cli): log decrypt failures in scanRows instead of silent mask [#4]
* fix(secure-cli): extend denylist + key-shape regex + deterministic ValidateGrantEnvVars [#6 #7]
* fix(migration): 000058 down idempotent + RAISE NOTICE + destructive-drop runbook warning [#5]
* fix(ui): clear revealed plaintext on unmount + 30s blur timeout [#10]
* fix(ui): clearForm on dialog close not only open — wipe plaintext env on close [#11]
* feat(ui): show LIMIT 20 truncation hint + add list.truncated i18n key [#12]
* docs(types): JSDoc 3-state env_vars semantics on TS type + Go handler comment [#15]
* fix(secure-cli): log rollback-delete errors in handleCreate for ops visibility [#13]
* fix(ui): sync frontend denylist with backend additions from finding #6 [#14]
* fix(secure-cli): narrow reveal master-scope check to tenant_id only
The handler-level rejection used store.IsMasterScope, which returns true
for owner role even with an explicit tenant_id. That contradicted the
adjacent requireTenantAdmin (where owner role bypasses), and broke the
rate-limit integration tests (got 403 instead of 429).
Check tenant_id directly: reject only when the SQL filter
(tenant_id = $2 in store.Get) would not bind to a real tenant — i.e.
uuid.Nil or MasterTenantID. Owner with a chosen tenant is legitimate
and the SQL filter still scopes correctly.
Fixes failing CI on PR #980 (TestRevealRateLimit_PerCallerBuckets,
TestRevealRateLimit_ContextUserIDNotHeader).
199 lines
6.1 KiB
Go
199 lines
6.1 KiB
Go
//go:build integration
|
|
|
|
package integration
|
|
|
|
// C4 denylist parity test: verify that the frontend denylist (TypeScript) matches
|
|
// the backend denylist (Go package internal/crypto/env_denylist.go).
|
|
//
|
|
// Strategy: the Go denylist is imported directly via package import.
|
|
// The frontend denylist is read from the TypeScript source file via string parsing.
|
|
// If the sets diverge, the test fails with a diff showing added/removed keys.
|
|
|
|
import (
|
|
"bufio"
|
|
"os"
|
|
"path/filepath"
|
|
"runtime"
|
|
"strings"
|
|
"testing"
|
|
|
|
"github.com/nextlevelbuilder/goclaw/internal/crypto"
|
|
)
|
|
|
|
// frontendDenylistExact reads the frontend ENV_DENYLIST_EXACT set from the TypeScript source.
|
|
// Parses the JS Set literal `const ENV_DENYLIST_EXACT = new Set([...])`.
|
|
func frontendDenylistExact(t *testing.T) map[string]struct{} {
|
|
t.Helper()
|
|
// Path relative to the test file's directory (tests/integration/).
|
|
_, thisFile, _, _ := runtime.Caller(0)
|
|
root := filepath.Join(filepath.Dir(thisFile), "..", "..")
|
|
tsFile := filepath.Join(root, "ui", "web", "src", "pages", "cli-credentials",
|
|
"cli-credential-grant-env-section.tsx")
|
|
|
|
f, err := os.Open(tsFile)
|
|
if err != nil {
|
|
t.Skipf("frontend file not found (not in TS codebase scope): %v", err)
|
|
return nil
|
|
}
|
|
defer f.Close()
|
|
|
|
result := make(map[string]struct{})
|
|
inSet := false
|
|
scanner := bufio.NewScanner(f)
|
|
for scanner.Scan() {
|
|
line := strings.TrimSpace(scanner.Text())
|
|
if strings.Contains(line, "const ENV_DENYLIST_EXACT") {
|
|
inSet = true
|
|
}
|
|
if inSet {
|
|
// Extract quoted identifiers.
|
|
parts := strings.Split(line, `"`)
|
|
for i := 1; i < len(parts); i += 2 {
|
|
key := strings.TrimSpace(parts[i])
|
|
if key != "" && !strings.Contains(key, " ") {
|
|
result[key] = struct{}{}
|
|
}
|
|
}
|
|
}
|
|
if inSet && strings.Contains(line, "]);") {
|
|
break
|
|
}
|
|
}
|
|
return result
|
|
}
|
|
|
|
// frontendDenylistPrefixes reads the frontend ENV_DENYLIST_PREFIXES array.
|
|
func frontendDenylistPrefixes(t *testing.T) map[string]struct{} {
|
|
t.Helper()
|
|
_, thisFile, _, _ := runtime.Caller(0)
|
|
root := filepath.Join(filepath.Dir(thisFile), "..", "..")
|
|
tsFile := filepath.Join(root, "ui", "web", "src", "pages", "cli-credentials",
|
|
"cli-credential-grant-env-section.tsx")
|
|
|
|
f, err := os.Open(tsFile)
|
|
if err != nil {
|
|
t.Skipf("frontend file not found: %v", err)
|
|
return nil
|
|
}
|
|
defer f.Close()
|
|
|
|
result := make(map[string]struct{})
|
|
scanner := bufio.NewScanner(f)
|
|
for scanner.Scan() {
|
|
line := strings.TrimSpace(scanner.Text())
|
|
if strings.Contains(line, "const ENV_DENYLIST_PREFIXES") {
|
|
// Parse prefix entries from: ["DYLD_", "GOCLAW_", "LD_"]
|
|
parts := strings.Split(line, `"`)
|
|
for i := 1; i < len(parts); i += 2 {
|
|
pfx := strings.TrimSpace(parts[i])
|
|
if pfx != "" && !strings.Contains(pfx, " ") {
|
|
result[pfx] = struct{}{}
|
|
}
|
|
}
|
|
break
|
|
}
|
|
}
|
|
return result
|
|
}
|
|
|
|
// backendDenylistExact returns the Go exact-match denylist by probing known keys.
|
|
// Since deniedExact is unexported, we use IsDeniedEnvKey with a controlled set of
|
|
// all keys that appear in either Go or frontend source.
|
|
//
|
|
// This is the exhaustive union probe set — any key on this list that differs between
|
|
// Go and TS is caught.
|
|
var knownExactKeys = []string{
|
|
"PATH", "HOME", "USER", "SHELL", "PWD",
|
|
"LD_PRELOAD", "LD_LIBRARY_PATH", "LD_AUDIT",
|
|
"NODE_OPTIONS", "NODE_PATH",
|
|
"PYTHONPATH", "PYTHONHOME", "PYTHONSTARTUP",
|
|
"GIT_SSH_COMMAND", "GIT_SSH", "GIT_EXEC_PATH", "GIT_CONFIG_SYSTEM",
|
|
"SSH_AUTH_SOCK",
|
|
// Additions from finding #6
|
|
"BASH_ENV", "ENV", "PROMPT_COMMAND",
|
|
"PERL5LIB", "RUBYOPT",
|
|
"HTTPS_PROXY", "HTTP_PROXY", "NO_PROXY",
|
|
"SSL_CERT_FILE", "SSL_CERT_DIR", "CURL_CA_BUNDLE",
|
|
"IFS",
|
|
}
|
|
|
|
// TestDenylistParity_ExactKeysPresentInBoth verifies that every key in the frontend
|
|
// ENV_DENYLIST_EXACT is also rejected by the Go backend (IsDeniedEnvKey returns true).
|
|
func TestDenylistParity_ExactKeysPresentInBoth(t *testing.T) {
|
|
frontendExact := frontendDenylistExact(t)
|
|
if len(frontendExact) == 0 {
|
|
t.Skip("frontend denylist not parseable — skipping parity check")
|
|
}
|
|
|
|
for key := range frontendExact {
|
|
if !crypto.IsDeniedEnvKey(key) {
|
|
t.Errorf("PARITY DRIFT: frontend denies %q but backend does NOT deny it", key)
|
|
}
|
|
}
|
|
}
|
|
|
|
// TestDenylistParity_BackendDeniesKnownKeys verifies all known-dangerous keys are
|
|
// denied by the backend after finding #6 additions.
|
|
func TestDenylistParity_BackendDeniesKnownKeys(t *testing.T) {
|
|
// Keys from original denylist + finding #6 additions.
|
|
mustDeny := []string{
|
|
// Original
|
|
"PATH", "HOME", "USER", "SHELL", "PWD",
|
|
"LD_PRELOAD", "LD_LIBRARY_PATH", "LD_AUDIT",
|
|
"NODE_OPTIONS", "NODE_PATH",
|
|
"PYTHONPATH", "PYTHONHOME", "PYTHONSTARTUP",
|
|
"GIT_SSH_COMMAND", "GIT_SSH", "GIT_EXEC_PATH", "GIT_CONFIG_SYSTEM",
|
|
"SSH_AUTH_SOCK",
|
|
// Finding #6 additions
|
|
"BASH_ENV", "ENV", "PROMPT_COMMAND",
|
|
"PERL5LIB", "RUBYOPT",
|
|
"HTTPS_PROXY", "HTTP_PROXY", "NO_PROXY",
|
|
"SSL_CERT_FILE", "SSL_CERT_DIR", "CURL_CA_BUNDLE",
|
|
"IFS",
|
|
// Prefix matches
|
|
"DYLD_INSERT_LIBRARIES", "DYLD_FRAMEWORK_PATH",
|
|
"GOCLAW_SECRET", "GOCLAW_ENCRYPTION_KEY",
|
|
"LD_SOMETHING",
|
|
// npm_config_ prefix (finding #6)
|
|
"npm_config_registry", "npm_config_prefix",
|
|
}
|
|
for _, key := range mustDeny {
|
|
if !crypto.IsDeniedEnvKey(key) {
|
|
t.Errorf("backend should deny %q but IsDeniedEnvKey returned false", key)
|
|
}
|
|
}
|
|
}
|
|
|
|
// TestDenylistParity_SafeKeyNotDenied verifies that safe keys pass validation.
|
|
func TestDenylistParity_SafeKeyNotDenied(t *testing.T) {
|
|
safeKeys := []string{
|
|
"AWS_ACCESS_KEY_ID",
|
|
"AWS_SECRET_ACCESS_KEY",
|
|
"GITHUB_TOKEN",
|
|
"DATABASE_URL",
|
|
"API_KEY",
|
|
"MY_CUSTOM_VAR",
|
|
}
|
|
for _, key := range safeKeys {
|
|
if crypto.IsDeniedEnvKey(key) {
|
|
t.Errorf("safe key %q should not be denied by backend", key)
|
|
}
|
|
}
|
|
}
|
|
|
|
// TestDenylistParity_PrefixesInBoth verifies that frontend prefix list matches backend.
|
|
func TestDenylistParity_PrefixesInBoth(t *testing.T) {
|
|
frontendPfx := frontendDenylistPrefixes(t)
|
|
if len(frontendPfx) == 0 {
|
|
t.Skip("frontend prefix list not parseable")
|
|
}
|
|
|
|
// For each frontend prefix, verify a key with that prefix is denied by backend.
|
|
for pfx := range frontendPfx {
|
|
testKey := pfx + "SOMETHING"
|
|
if !crypto.IsDeniedEnvKey(testKey) {
|
|
t.Errorf("PARITY DRIFT: frontend prefix %q blocks keys but backend does NOT deny %q", pfx, testKey)
|
|
}
|
|
}
|
|
}
|