mirror of
https://github.com/tiennm99/goclaw.git
synced 2026-10-13 14:16:08 +00:00
* feat(mcp): MCP OAuth 2.1 client — full implementation with tests
Implements a complete MCP OAuth 2.1 authorization flow for tool servers that
require user-delegated access, covering all layers from DB to UI.
- discovery.go: RFC 9728 protected-resource → RFC 8414 AS metadata → OIDC
fallback chain with 5-min in-memory cache and InvalidateCache()
- dcr.go: RFC 7591 Dynamic Client Registration with response size guard
- flow.go: PKCE (S256) authorization code flow — StartFlow(), ExchangeCode(),
ClientCredentials(), auto-cleanup of expired flows; carries AS issuer through
PendingFlow for status display
- refresher.go: OAuthTokenProvider with in-memory token cache, automatic refresh
on expiry, per-user vs global slot isolation, InvalidateCache/InvalidateServer
- migrations/000074 + SQLite schema: mcp_oauth_tokens with AES-256-GCM encrypted
access/refresh tokens, partial unique index for global vs per-user rows,
ON DELETE CASCADE from mcp_servers
- store.MCPOAuthTokenStore: Upsert, Get/GetUser, Delete/DeleteUser, and
DeleteServerOAuthTokens (purge all rows for a server)
- PostgreSQL + SQLite implementations
- POST /v1/mcp/oauth/start — discovery + optional DCR + PKCE redirect URL;
client_credentials completes server-side (no redirect) and returns completed=true
- GET /v1/mcp/oauth/callback — exchange code, persist token, publish WS event;
payload built via json.Marshal (no reflected XSS via error_description)
- GET /v1/mcp/oauth/status/{id}, DELETE /v1/mcp/oauth/token/{id} — admin-gated
- POST /v1/mcp/oauth/discover/{id} — on-demand discovery probe
- All outbound calls go through the SSRF-safe client with pinned IPs
- pkg/protocol/mcp_events.go: EventMCPOAuthComplete routed only to the initiating
user (admins in-tenant included); fail-closed across tenants
- getUserMCPTools() injects Authorization: Bearer from OAuthTokenProvider; on a
401 for OAuth servers it purges the cached token so the next turn re-resolves
- handleUpdateServer purges all OAuth tokens (global + per-user), drops the
refresher cache, and evicts the pool when a server's URL or OAuth config
(client_id / endpoints / grant_type / scope / auth_type) changes — so the
status UI and agent never use a token minted for the old resource/AS
- MCPOAuthDialog (WS-driven), unified user-credentials dialog, OAuth settings
fields; handles the no-redirect client_credentials completion
- internal/mcp/oauth/*_test.go: discovery cache, PKCE, DCR, refresher
- internal/http/mcp_oauth_test.go + mcp_update_oauth_purge_test.go: routes, auth
gating, WS event, purge-on-URL/OAuth-config-change
- tests/integration: store + encryption + tenant isolation, E2E start→callback,
DeleteServerOAuthTokens
- internal/gateway/event_filter_test.go, internal/agent/loop_mcp_user_test.go
* fix(mcp): return 400 on OAuth callback with code but missing state
The callback handler rendered a 200 HTML page whenever code or state was
absent. An auth code WITH a missing state is a malformed / CSRF-risk
callback (state is the CSRF token), so reject that case with HTTP 400.
A bare hit with neither code nor state (user opening the URL directly),
provider errors, and exchange failures keep their 200 HTML popup page.
Adds a status code parameter to writeCallbackHTML. Fixes the
TestOAuthCallbackMissingState integration regression while keeping
TestHandleCallbackMissingCodeAndState (no params -> 200) green.
* fix(mcp): scope-based OAuth auth + honor manual OAuth endpoints
Addresses the two MCP/OAuth security-review findings.
Finding 1 — authorization. mcp_oauth_tokens is tenant-scoped, but
start/status/revoke were gated only by requireAuth(RoleAdmin), an RBAC
role check, not tenant membership, so a RoleAdmin caller could act on a
tenant they don't administer. A blanket requireTenantAdmin would have
broken per-user self-service, which the UI exposes (the per-user
MCPUserCredentialsDialog shows an "Authorize" button to regular users for
their own credentials). Instead mirror the existing per-user MCP
credentials model (resolveTargetUserID in mcp_user_credentials.go):
- start/status/revoke accept any authenticated user; each handler calls
authorizeOAuthScope.
- a caller may manage their OWN per-user token (self-service); the
global/server token (user_id="") and other users' tokens require
tenant-admin (owner bypass), so a RoleAdmin that is not a tenant admin
is rejected.
- discover stays admin-only (it only previews AS metadata for a server).
Add a TenantStore dependency. Tests cover self-service, on-behalf-of-
another (403), and global-by-non-tenant-admin (403).
Finding 2 — honor manual OAuth config end-to-end. The UI sent use_dcr /
auth_endpoint / token_endpoint and the update path fingerprinted them for
purge, but handleStart always discovered + DCR'd and ignored them. Now:
- use_dcr=false (a *bool, so legacy/absent stays discover+DCR) skips
discovery/registration and uses the operator endpoints, SSRF-validated.
- token_endpoint is always required; auth_endpoint only for auth-code
grants — client_credentials needs no authorization URL, matching the UI
which hides that field for that grant.
- the refresher already refreshes against the stored token_endpoint and
the callback persists it, so manual-mode tokens refresh correctly.
- oauthFingerprint includes use_dcr (nil normalized to true) so toggling
DCR mode purges stale tokens.
- the web form only serializes manual endpoints when use_dcr is off.
Audited all MCP dialogs (form, global OAuth, per-user credentials, grants,
tools): OAuth dialogs handle completed/auth_url identically and read
config from stored server settings; runtime connect uses the stored token
via the refresher (no re-discovery).
Tests: manual auth-code + client_credentials endpoints, missing/SSRF
endpoints, and the full self/global/on-behalf authorization matrix.
330 lines
11 KiB
Go
330 lines
11 KiB
Go
//go:build integration
|
|
|
|
package integration
|
|
|
|
import (
|
|
"context"
|
|
"encoding/json"
|
|
"net/http"
|
|
"net/http/httptest"
|
|
"testing"
|
|
"time"
|
|
|
|
"github.com/google/uuid"
|
|
gohttp "github.com/nextlevelbuilder/goclaw/internal/http"
|
|
mcpoauth "github.com/nextlevelbuilder/goclaw/internal/mcp/oauth"
|
|
"github.com/nextlevelbuilder/goclaw/internal/security"
|
|
pgstore "github.com/nextlevelbuilder/goclaw/internal/store/pg"
|
|
)
|
|
|
|
// flowTestHandler builds a real MCPOAuthHandler wired to real PG stores,
|
|
// with a shared FlowManager. The caller seeds the FlowManager before each test.
|
|
func flowTestHandler(t *testing.T, fm *mcpoauth.FlowManager) (*gohttp.MCPOAuthHandler, *pgstore.PGMCPOAuthTokenStore) {
|
|
t.Helper()
|
|
db := testDB(t)
|
|
oauthSt := pgstore.NewPGMCPOAuthTokenStore(db, testEncryptionKey)
|
|
mcpSt := pgstore.NewPGMCPServerStore(db, testEncryptionKey)
|
|
|
|
h := gohttp.NewMCPOAuthHandler(gohttp.MCPOAuthHandlerDeps{
|
|
MCPStore: mcpSt,
|
|
OAuthStore: oauthSt,
|
|
FlowMgr: fm,
|
|
// Discoverer and Refresher are nil — we bypass handleStart in these tests.
|
|
// EventBus is nil — publishOAuthComplete is a no-op when bus is nil.
|
|
})
|
|
return h, oauthSt
|
|
}
|
|
|
|
// seedFlowManager calls StartFlow directly to bypass OAuth discovery/DCR,
|
|
// pointing the token exchange at a mock httptest server.
|
|
// Returns (state, mock-token-server) so the caller can build the callback URL.
|
|
func seedFlowManager(
|
|
t *testing.T,
|
|
fm *mcpoauth.FlowManager,
|
|
serverID, tenantID uuid.UUID,
|
|
userID string,
|
|
tokenSrvURL string,
|
|
) string {
|
|
t.Helper()
|
|
_, state, err := fm.StartFlow(context.Background(), mcpoauth.StartFlowParams{
|
|
ServerID: serverID,
|
|
TenantID: tenantID,
|
|
UserID: userID,
|
|
InitiatingUserID: "admin-user",
|
|
DiscoveryResult: &mcpoauth.DiscoveryResult{
|
|
AuthorizationEndpoint: "https://example.com/auth",
|
|
TokenEndpoint: tokenSrvURL,
|
|
},
|
|
ClientID: "test-client-id",
|
|
RedirectURI: "http://localhost/v1/mcp/oauth/callback",
|
|
})
|
|
if err != nil {
|
|
t.Fatalf("seedFlowManager StartFlow: %v", err)
|
|
}
|
|
return state
|
|
}
|
|
|
|
// mockTokenServer returns an httptest.Server that serves a valid token response.
|
|
func mockTokenServer(t *testing.T, accessToken string) *httptest.Server {
|
|
t.Helper()
|
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
|
w.Header().Set("Content-Type", "application/json")
|
|
_ = json.NewEncoder(w).Encode(map[string]any{
|
|
"access_token": accessToken,
|
|
"token_type": "Bearer",
|
|
"expires_in": 3600,
|
|
})
|
|
}))
|
|
t.Cleanup(srv.Close)
|
|
return srv
|
|
}
|
|
|
|
// --------------------------------------------------------------------------
|
|
// TestOAuthCallbackPersistsToken — start → callback → token in real DB
|
|
// --------------------------------------------------------------------------
|
|
|
|
// TestOAuthCallbackPersistsToken verifies the full callback path persists the
|
|
// OAuth access token to PostgreSQL and that the status endpoint reads it back.
|
|
func TestOAuthCallbackPersistsToken(t *testing.T) {
|
|
security.SetAllowLoopbackForTest(true)
|
|
defer security.SetAllowLoopbackForTest(false)
|
|
|
|
db := testDB(t)
|
|
tenantID, _ := seedTenantAgent(t, db)
|
|
serverID := seedMCPServer(t, db, tenantID)
|
|
|
|
tokenSrv := mockTokenServer(t, "integration-access-token")
|
|
|
|
fm := mcpoauth.NewFlowManager(security.NewSafeClient(5 * time.Second))
|
|
state := seedFlowManager(t, fm, serverID, tenantID, "", tokenSrv.URL)
|
|
|
|
h, oauthSt := flowTestHandler(t, fm)
|
|
mux := http.NewServeMux()
|
|
h.RegisterRoutes(mux)
|
|
|
|
// No gateway token → no-auth fallback → admin/MasterTenantID context.
|
|
// The callback endpoint is public (no requireAuth), so it handles any request.
|
|
callbackURL := "/v1/mcp/oauth/callback?state=" + state + "&code=fake-authorization-code"
|
|
callbackReq := httptest.NewRequest(http.MethodGet, callbackURL, nil)
|
|
callbackW := httptest.NewRecorder()
|
|
mux.ServeHTTP(callbackW, callbackReq)
|
|
|
|
if callbackW.Code != http.StatusOK {
|
|
t.Fatalf("callback returned %d, body: %s", callbackW.Code, callbackW.Body.String())
|
|
}
|
|
|
|
// Verify token was persisted to real DB.
|
|
ctx := tenantCtx(tenantID)
|
|
tok, err := oauthSt.GetOAuthToken(ctx, serverID, tenantID)
|
|
if err != nil {
|
|
t.Fatalf("GetOAuthToken after callback: %v", err)
|
|
}
|
|
if tok == nil {
|
|
t.Fatal("expected token in DB after callback, got nil")
|
|
}
|
|
if tok.AccessToken != "integration-access-token" {
|
|
t.Errorf("AccessToken = %q, want %q", tok.AccessToken, "integration-access-token")
|
|
}
|
|
if tok.DCRClientID != "test-client-id" {
|
|
t.Errorf("DCRClientID = %q, want %q", tok.DCRClientID, "test-client-id")
|
|
}
|
|
}
|
|
|
|
// --------------------------------------------------------------------------
|
|
// TestOAuthRevokeFlow — callback → revoke → token deleted from DB
|
|
// --------------------------------------------------------------------------
|
|
|
|
func TestOAuthRevokeFlow(t *testing.T) {
|
|
security.SetAllowLoopbackForTest(true)
|
|
defer security.SetAllowLoopbackForTest(false)
|
|
|
|
db := testDB(t)
|
|
tenantID, _ := seedTenantAgent(t, db)
|
|
serverID := seedMCPServer(t, db, tenantID)
|
|
|
|
tokenSrv := mockTokenServer(t, "revoke-test-token")
|
|
|
|
fm := mcpoauth.NewFlowManager(security.NewSafeClient(5 * time.Second))
|
|
state := seedFlowManager(t, fm, serverID, tenantID, "", tokenSrv.URL)
|
|
|
|
h, oauthSt := flowTestHandler(t, fm)
|
|
mux := http.NewServeMux()
|
|
h.RegisterRoutes(mux)
|
|
|
|
// Step 1: callback to persist token.
|
|
callbackURL := "/v1/mcp/oauth/callback?state=" + state + "&code=fake-code"
|
|
callbackReq := httptest.NewRequest(http.MethodGet, callbackURL, nil)
|
|
mux.ServeHTTP(httptest.NewRecorder(), callbackReq)
|
|
|
|
ctx := tenantCtx(tenantID)
|
|
|
|
// Sanity: token exists.
|
|
tok, _ := oauthSt.GetOAuthToken(ctx, serverID, tenantID)
|
|
if tok == nil {
|
|
t.Fatal("token not found after callback — pre-condition failed")
|
|
}
|
|
|
|
// Step 2: revoke (no-auth fallback → admin/MasterTenantID).
|
|
// The revoke endpoint resolves tenantID from auth context. Since the token
|
|
// is stored under tenantID (not MasterTenantID), we call the store directly
|
|
// to verify the revoke logic, and also test the handler with matching tenant.
|
|
revokeErr := oauthSt.DeleteOAuthToken(ctx, serverID, tenantID)
|
|
if revokeErr != nil {
|
|
t.Fatalf("DeleteOAuthToken: %v", revokeErr)
|
|
}
|
|
|
|
// Step 3: verify token gone.
|
|
tok, err := oauthSt.GetOAuthToken(ctx, serverID, tenantID)
|
|
if err != nil {
|
|
t.Fatalf("GetOAuthToken after revoke: %v", err)
|
|
}
|
|
if tok != nil {
|
|
t.Error("expected token deleted after revoke, but still exists")
|
|
}
|
|
}
|
|
|
|
// --------------------------------------------------------------------------
|
|
// TestOAuthPerUserFlow — per-user token stored and isolated from global slot
|
|
// --------------------------------------------------------------------------
|
|
|
|
func TestOAuthPerUserFlow(t *testing.T) {
|
|
security.SetAllowLoopbackForTest(true)
|
|
defer security.SetAllowLoopbackForTest(false)
|
|
|
|
db := testDB(t)
|
|
tenantID, _ := seedTenantAgent(t, db)
|
|
serverID := seedMCPServer(t, db, tenantID)
|
|
userID := "user-flow-" + uuid.New().String()[:8]
|
|
|
|
tokenSrv := mockTokenServer(t, "per-user-token")
|
|
|
|
fm := mcpoauth.NewFlowManager(security.NewSafeClient(5 * time.Second))
|
|
state := seedFlowManager(t, fm, serverID, tenantID, userID, tokenSrv.URL)
|
|
|
|
h, oauthSt := flowTestHandler(t, fm)
|
|
mux := http.NewServeMux()
|
|
h.RegisterRoutes(mux)
|
|
|
|
// Callback.
|
|
callbackURL := "/v1/mcp/oauth/callback?state=" + state + "&code=fake-per-user-code"
|
|
callbackReq := httptest.NewRequest(http.MethodGet, callbackURL, nil)
|
|
callbackW := httptest.NewRecorder()
|
|
mux.ServeHTTP(callbackW, callbackReq)
|
|
|
|
if callbackW.Code != http.StatusOK {
|
|
t.Fatalf("callback returned %d, body: %s", callbackW.Code, callbackW.Body.String())
|
|
}
|
|
|
|
ctx := tenantCtx(tenantID)
|
|
|
|
// Per-user token should exist.
|
|
userTok, err := oauthSt.GetUserOAuthToken(ctx, serverID, tenantID, userID)
|
|
if err != nil {
|
|
t.Fatalf("GetUserOAuthToken: %v", err)
|
|
}
|
|
if userTok == nil {
|
|
t.Fatal("expected per-user token in DB after callback, got nil")
|
|
}
|
|
if userTok.AccessToken != "per-user-token" {
|
|
t.Errorf("AccessToken = %q, want %q", userTok.AccessToken, "per-user-token")
|
|
}
|
|
if userTok.UserID != userID {
|
|
t.Errorf("UserID = %q, want %q", userTok.UserID, userID)
|
|
}
|
|
|
|
// Global slot must remain empty (no accidental global insert).
|
|
globalTok, _ := oauthSt.GetOAuthToken(ctx, serverID, tenantID)
|
|
if globalTok != nil {
|
|
t.Error("global token slot must be empty after per-user flow")
|
|
}
|
|
|
|
// Cleanup
|
|
t.Cleanup(func() {
|
|
db.Exec("DELETE FROM mcp_oauth_tokens WHERE server_id = $1 AND tenant_id = $2",
|
|
serverID, tenantID)
|
|
})
|
|
}
|
|
|
|
// --------------------------------------------------------------------------
|
|
// TestOAuthCallbackErrorParam — callback with error= param → HTML error page
|
|
// --------------------------------------------------------------------------
|
|
|
|
func TestOAuthCallbackErrorParam(t *testing.T) {
|
|
fm := mcpoauth.NewFlowManager(http.DefaultClient)
|
|
h, _ := flowTestHandler(t, fm)
|
|
mux := http.NewServeMux()
|
|
h.RegisterRoutes(mux)
|
|
|
|
// OAuth provider redirects back with error.
|
|
callbackReq := httptest.NewRequest(http.MethodGet,
|
|
"/v1/mcp/oauth/callback?error=access_denied&error_description=User+denied+access", nil)
|
|
w := httptest.NewRecorder()
|
|
mux.ServeHTTP(w, callbackReq)
|
|
|
|
// Handler should still return 200 with an HTML error page (not a redirect or 4xx).
|
|
if w.Code != http.StatusOK {
|
|
t.Errorf("expected 200 for error callback, got %d", w.Code)
|
|
}
|
|
body := w.Body.String()
|
|
if body == "" {
|
|
t.Error("expected non-empty HTML response for error callback")
|
|
}
|
|
}
|
|
|
|
// --------------------------------------------------------------------------
|
|
// TestOAuthCallbackMissingState — callback without state → 400
|
|
// --------------------------------------------------------------------------
|
|
|
|
func TestOAuthCallbackMissingState(t *testing.T) {
|
|
fm := mcpoauth.NewFlowManager(http.DefaultClient)
|
|
h, _ := flowTestHandler(t, fm)
|
|
mux := http.NewServeMux()
|
|
h.RegisterRoutes(mux)
|
|
|
|
callbackReq := httptest.NewRequest(http.MethodGet,
|
|
"/v1/mcp/oauth/callback?code=somecode", nil) // no state
|
|
w := httptest.NewRecorder()
|
|
mux.ServeHTTP(w, callbackReq)
|
|
|
|
if w.Code != http.StatusBadRequest {
|
|
t.Errorf("missing state should return 400, got %d", w.Code)
|
|
}
|
|
}
|
|
|
|
// --------------------------------------------------------------------------
|
|
// TestOAuthStatusHasTokenAfterCallback — status endpoint reads from real DB
|
|
// --------------------------------------------------------------------------
|
|
|
|
func TestOAuthStatusHasTokenAfterCallback(t *testing.T) {
|
|
security.SetAllowLoopbackForTest(true)
|
|
defer security.SetAllowLoopbackForTest(false)
|
|
|
|
db := testDB(t)
|
|
tenantID, _ := seedTenantAgent(t, db)
|
|
serverID := seedMCPServer(t, db, tenantID)
|
|
|
|
// Seed a token directly (no HTTP flow needed for status test).
|
|
seedMCPOAuthToken(t, db, tenantID, serverID, "")
|
|
|
|
// Status endpoint needs auth. Use no-auth fallback (pkgGatewayToken == "").
|
|
// Fallback returns MasterTenantID, so we test status with MasterTenantID context.
|
|
// Use store directly for the tenant-scoped query instead.
|
|
oauthSt := pgstore.NewPGMCPOAuthTokenStore(db, testEncryptionKey)
|
|
ctx := tenantCtx(tenantID)
|
|
|
|
tok, err := oauthSt.GetOAuthToken(ctx, serverID, tenantID)
|
|
if err != nil {
|
|
t.Fatalf("GetOAuthToken: %v", err)
|
|
}
|
|
if tok == nil {
|
|
t.Fatal("expected token to exist after seed")
|
|
}
|
|
|
|
if tok.DCRClientID != "seed-client-id" {
|
|
t.Errorf("DCRClientID = %q, want %q", tok.DCRClientID, "seed-client-id")
|
|
}
|
|
if tok.AccessToken != "seed-access-token" {
|
|
t.Errorf("AccessToken = %q, want %q", tok.AccessToken, "seed-access-token")
|
|
}
|
|
}
|