feat(plan): outline passive channel memory extraction

plans/260528-1844-passive-channel-memory-extraction/phase-01-research-and-characterization-tests.md

@@ -0,0 +1,87 @@
+---
+phase: 1
+title: "Research and Characterization Tests"
+status: pending
+priority: P1
+effort: "1d"
+dependencies: []
+---
+
+# Phase 1: Research and Characterization Tests
+
+## Context Links
+
+- Issue: `digitopvn/goclaw#64`
+- Existing docs: `docs/06-store-data-model.md`, `docs/07-bootstrap-skills-memory.md`, `docs/09-security.md`, `docs/18-http-api.md`
+- Channel buffer: `internal/channels/history.go`, `internal/store/pending_message_store.go`
+- Memory/KG: `internal/consolidation/episodic_worker.go`, `internal/consolidation/semantic_worker.go`
+
+## Overview
+
+Lock current behavior with tests before changing schemas or workers. This phase proves the feature can reuse existing channel pending-message buffers and Memory/KG pipeline without breaking session memory.
+
+## Requirements
+
+- Functional: document exact current data flow for group pending messages, channel instance config update, episodic creation, KG extraction, and activity logging.
+- Non-functional: tests first, no implementation until expected failures describe the desired behavior.
+
+## Architecture
+
+No runtime architecture added yet. The output is a verified map and failing tests for v1 contracts:
+
+1. channel instance config can carry `passive_memory`
+2. pending messages are tenant/channel/history-key scoped
+3. extraction candidates exclude DMs/private history in v1
+4. session-created episodic flow remains unchanged
+
+## Related Code Files
+
+- Read: `internal/channels/history.go`
+- Read: `internal/channels/history_compaction.go`
+- Read: `internal/store/pending_message_store.go`
+- Read: `internal/store/channel_instance_store.go`
+- Read: `internal/consolidation/episodic_worker.go`
+- Read: `internal/consolidation/semantic_worker.go`
+- Read: `internal/http/channel_instances.go`
+- Create: `internal/consolidation/channel_memory_config_test.go`
+- Create: `internal/consolidation/channel_memory_worker_test.go`
+- Create: `internal/http/channel_memory_handlers_test.go`
+- Create or extend: store tests under `internal/store/pg/` and `internal/store/sqlitestore/`
+
+## Implementation Steps
+
+1. Re-scout channel adapters to list which currently call `PendingHistory.Record`.
+2. Re-scout `channel_pending_messages` PG/SQLite implementations and confirm tenant predicates.
+3. Write failing config parser tests for default-disabled `passive_memory`.
+4. Write failing candidate-selection tests: group-only, enabled-only, interval-or-cap trigger, minimum useful content gate.
+5. Write failing redaction tests using representative secret/PII strings.
+6. Write failing API tests for unauthorized settings/run/review actions.
+7. Write a short `plans/.../reports/scout-report.md` with verified file/line anchors.
+
+## Todo List
+
+- [ ] Enumerate all pending-history recording call sites.
+- [ ] Enumerate HTTP + WS channel instance update surfaces.
+- [ ] Add failing config parser tests.
+- [ ] Add failing worker trigger tests.
+- [ ] Add failing redaction tests.
+- [ ] Add failing admin API permission tests.
+- [ ] Save scout report under this plan.
+
+## Success Criteria
+
+- [ ] Tests fail for missing passive-memory config, worker, queue, and handlers.
+- [ ] Scout report lists exact files to modify and no fabricated APIs.
+- [ ] Plan assumptions match live code after re-grep.
+
+## Risk Assessment
+
+Main risk is planning from stale scout notes. Mitigation: every claim in later phases must cite paths found in this phase, especially schema versions and route wiring.
+
+## Security Considerations
+
+Tests must include cross-tenant pending-message isolation and reject non-admin write access before any handler implementation exists.
+
+## Next Steps
+
+Proceed to schema/store only after tests describe the contracts.

plans/260528-1844-passive-channel-memory-extraction/phase-02-schema-and-store-contracts.md

@@ -0,0 +1,100 @@
+---
+phase: 2
+title: "Schema and Store Contracts"
+status: pending
+priority: P1
+effort: "2d"
+dependencies: [1]
+---
+
+# Phase 2: Schema and Store Contracts
+
+## Context Links
+
+- Phase 1 tests and scout report
+- PG migration latest during planning: `000073_secure_cli_credential_type.up.sql`
+- PG schema version: `internal/upgrade/version.go`
+- SQLite schema: `internal/store/sqlitestore/schema.sql`, `internal/store/sqlitestore/schema.go`
+- Store registry: `internal/store/stores.go`, `internal/store/pg/factory.go`, `internal/store/sqlitestore/factory.go`
+
+## Overview
+
+Add durable run and review-item stores for passive extraction. Keep channel settings inside `channel_instances.config` to avoid another settings subsystem.
+
+## Requirements
+
+- Functional: persist extraction runs, pending extracted items, approval/rejection/deletion state, source ranges, redaction stats, and write metadata.
+- Non-functional: tenant-scoped SQL, PG + SQLite parity, deterministic source IDs, no raw message body in extraction item after redaction.
+
+## Architecture
+
+Use two new store concepts:
+
+1. `ChannelMemoryExtractionRun`: one batch attempt per channel/history key.
+2. `ChannelMemoryExtractionItem`: one extracted candidate fact or KG relation bundle pending review.
+
+`channel_instances.config.passive_memory` carries settings. Store layer exposes typed config helpers so HTTP/WS/UI do not mutate arbitrary JSON blindly.
+
+## Related Code Files
+
+- Create: `migrations/000074_channel_memory_extraction.up.sql`
+- Create: `migrations/000074_channel_memory_extraction.down.sql`
+- Modify: `internal/upgrade/version.go`
+- Modify: `internal/store/sqlitestore/schema.sql`
+- Modify: `internal/store/sqlitestore/schema.go`
+- Create: `internal/store/channel_memory_extraction_store.go`
+- Create: `internal/store/pg/channel_memory_extraction.go`
+- Create: `internal/store/sqlitestore/channel_memory_extraction.go`
+- Modify: `internal/store/stores.go`
+- Modify: `internal/store/pg/factory.go`
+- Modify: `internal/store/sqlitestore/factory.go`
+
+## Implementation Steps
+
+1. Add PG migration `000074` with tenant-scoped tables and indexes:
+   - `channel_memory_extraction_runs`
+   - `channel_memory_extraction_items`
+2. Add unique constraints for idempotency:
+   - run: `(tenant_id, channel_instance_id, history_key, source_start_id, source_end_id)`
+   - item: `(tenant_id, run_id, item_hash)`
+3. Add statuses:
+   - run: `pending`, `running`, `completed`, `failed`, `cancelled`
+   - item: `pending_review`, `approved`, `rejected`, `written`, `deleted`
+4. Add source metadata fields: channel instance ID/name, history key, platform message ID range, time range, extractor run ID.
+5. Add redaction summary fields: count, categories, skipped reason.
+6. Add SQLite schema and migration map patch; bump `SchemaVersion` from 42 to 43.
+7. Bump PG `RequiredSchemaVersion` from 73 to 74.
+8. Implement PG and SQLite stores using existing scope patterns.
+9. Wire stores into `store.Stores` and factories.
+10. Make Phase 1 store tests pass.
+
+## Todo List
+
+- [ ] Add PG migration + version bump.
+- [ ] Add SQLite schema + migration map + version bump.
+- [ ] Add store interface types.
+- [ ] Add PG store implementation.
+- [ ] Add SQLite store implementation.
+- [ ] Wire factories.
+- [ ] Add idempotency tests.
+- [ ] Add tenant isolation tests.
+
+## Success Criteria
+
+- [ ] Fresh PG migration applies and rolls back.
+- [ ] Fresh SQLite DB contains tables.
+- [ ] Existing SQLite DB migrates 42 to 43.
+- [ ] Store tests pass for create/list/update/claim/status transitions.
+- [ ] No SQL string concatenation with user input.
+
+## Risk Assessment
+
+Migration conflict risk is high on active `dev`. Before implementation, re-run `ls migrations/*.up.sql | sort | tail` and renumber if another migration claims `000074`.
+
+## Security Considerations
+
+All read/write methods require tenant context unless explicitly cross-tenant. Review item payload must store redacted text only; raw body stays in existing pending-message table subject to retention.
+
+## Next Steps
+
+Worker can claim runs only after store contracts exist.

plans/260528-1844-passive-channel-memory-extraction/phase-03-extractor-worker-and-redaction.md

@@ -0,0 +1,114 @@
+---
+phase: 3
+title: "Extractor Worker and Redaction"
+status: pending
+priority: P1
+effort: "2d"
+dependencies: [1, 2]
+---
+
+# Phase 3: Extractor Worker and Redaction
+
+## Context Links
+
+- Existing compaction: `internal/channels/history_compaction.go`
+- Existing KG extractor: `internal/knowledgegraph/extractor.go`
+- Background provider resolver: `internal/providerresolve`
+- Usage caps: `internal/usage/caps`
+- Runtime wiring: `cmd/gateway.go`
+
+## Overview
+
+Build the passive extraction worker. It scans enabled channel configs, selects eligible message batches, redacts sensitive content, calls LLM extraction/summarization once per batch, and stores review items.
+
+## Requirements
+
+- Functional: interval/message-cap/manual trigger, group-only v1, minimum useful content gate, redaction before durable writes, retry-safe run status.
+- Non-functional: bounded worker concurrency, cost controlled by batching, no panic on provider failure, graceful shutdown.
+
+## Architecture
+
+Add `internal/consolidation/channel_memory_worker.go` or a focused `internal/channelmemory/` package if the file would exceed 200 LOC.
+
+Data flow:
+
+1. scan `ChannelInstanceStore` for enabled channels with `config.passive_memory.enabled=true`
+2. inspect `PendingMessageStore.ListGroups`
+3. choose groups where interval elapsed or message cap reached
+4. load `PendingMessageStore.ListByKey`
+5. redact
+6. create/claim extraction run
+7. call extractor prompt for durable facts only
+8. persist `pending_review` items
+9. log activity + `slog.Info/Warn`
+
+## Related Code Files
+
+- Create: `internal/consolidation/channel_memory_config.go`
+- Create: `internal/consolidation/channel_memory_redactor.go`
+- Create: `internal/consolidation/channel_memory_worker.go`
+- Create: `internal/consolidation/channel_memory_prompt.go`
+- Modify: `internal/consolidation/workers.go` or gateway wiring if worker stays separate from existing worker bundle
+- Modify: `cmd/gateway.go`
+- Modify: `cmd/gateway_managed.go` if managed gateway has separate wiring
+
+## Implementation Steps
+
+1. Implement `PassiveMemoryConfig` parser with defaults:
+   - disabled
+   - review mode true
+   - interval 360 minutes
+   - cap 100 messages
+   - retention 168 hours
+   - allowed types default set
+2. Implement redactor with tests for:
+   - API keys/tokens/passwords
+   - env-like secrets
+   - connection strings
+   - payment/banking numbers
+   - phone/address-like PII
+   - configured exclude user IDs and regex patterns
+3. Implement candidate selector with tests:
+   - skip disabled
+   - skip DM/private keys by v1 policy
+   - skip too few useful messages
+   - trigger on interval or message cap
+4. Implement LLM prompt and parser for durable facts:
+   - output JSON array of items
+   - allowed types only
+   - confidence score
+   - concise memory text
+   - optional entity/relation hints
+5. Use `usageCaps.Chat` with purpose `channel-memory-extraction`.
+6. Store run/item status transitions.
+7. Emit audit activity for run started/completed/failed.
+8. Wire worker start/stop in gateway with conservative ticker and concurrency.
+
+## Todo List
+
+- [ ] Config parser tests pass.
+- [ ] Redactor tests pass.
+- [ ] Candidate selector tests pass.
+- [ ] Worker run lifecycle tests pass.
+- [ ] Provider failure leaves run `failed` and does not block gateway.
+- [ ] Usage cap purpose covered.
+- [ ] Gateway wiring starts worker only when channels + stores are available.
+
+## Success Criteria
+
+- [ ] Worker creates review items from eligible group messages.
+- [ ] No raw unredacted message text stored in new tables.
+- [ ] Duplicate scans do not create duplicate runs/items.
+- [ ] Provider failures are non-fatal and auditable.
+
+## Risk Assessment
+
+The redactor can false-positive and reduce extraction quality. Accept for v1. Better to drop sensitive content than store it.
+
+## Security Considerations
+
+Prompt must explicitly reject secrets and low-confidence guesses. Logs must include counts and IDs, not raw message body.
+
+## Next Steps
+
+Approved review items can write to Memory/KG after queue semantics are stable.

plans/260528-1844-passive-channel-memory-extraction/phase-04-memory-kg-review-queue-integration.md

@@ -0,0 +1,99 @@
+---
+phase: 4
+title: "Memory KG Review Queue Integration"
+status: pending
+priority: P1
+effort: "2d"
+dependencies: [2, 3]
+---
+
+# Phase 4: Memory KG Review Queue Integration
+
+## Context Links
+
+- Episodic store: `internal/store/episodic_store.go`, `internal/store/pg/episodic_summaries.go`, `internal/store/sqlitestore/episodic.go`
+- Semantic worker: `internal/consolidation/semantic_worker.go`
+- KG store: `internal/store/knowledge_graph_store.go`
+- Activity store: `internal/store/activity_store.go`
+
+## Overview
+
+Add approve/reject/delete semantics and connect approved channel extraction items to existing episodic Memory and KG ingestion. Keep review queue default-on.
+
+## Requirements
+
+- Functional: list pending items, approve, reject, delete, manual write, idempotent re-approve, source metadata preserved.
+- Non-functional: approved summaries become searchable like episodic memory; KG receives only approved/redacted text; all writes are tenant/agent scoped.
+
+## Architecture
+
+Approval writes one episodic summary with:
+
+- `SourceType`: `channel`
+- `SourceID`: deterministic `channel:<run_id>:<item_id>`
+- `SessionKey`: synthetic, e.g. `channel:<channel_name>:<history_key>`
+- `Summary`: approved redacted memory text
+- `KeyTopics`: extracted normalized topics/entities
+- `UserID`: approving admin or configured channel agent owner user, decided in implementation after re-checking current auth model
+
+Then publish `EventEpisodicCreated` so `semanticWorker` extracts KG facts through the existing path.
+
+## Related Code Files
+
+- Modify: `internal/consolidation/episodic_worker.go` only if helpers can be reused without changing session behavior.
+- Create: `internal/consolidation/channel_memory_writer.go`
+- Modify: `internal/eventbus/event_types.go` only if channel-specific payload is genuinely needed; prefer existing `EventEpisodicCreated`.
+- Modify: new `ChannelMemoryExtractionStore` from Phase 2.
+- Create: tests in `internal/consolidation/channel_memory_writer_test.go`.
+
+## Implementation Steps
+
+1. Write failing tests for approve idempotency:
+   - first approve creates episodic and marks item written/approved
+   - second approve no-ops
+   - rejected/deleted item cannot be approved unless restored is explicitly implemented, which v1 does not need
+2. Implement writer service that depends on:
+   - `ChannelMemoryExtractionStore`
+   - `EpisodicStore`
+   - `DomainEventBus`
+   - `ActivityStore`
+3. Create episodic summary from approved item with deterministic source ID.
+4. Publish existing `EventEpisodicCreated` with redacted summary.
+5. Record activity:
+   - `channel_memory.item.approved`
+   - `channel_memory.item.rejected`
+   - `channel_memory.item.deleted`
+   - `channel_memory.item.written`
+6. Add delete/forget behavior:
+   - v1 deletes queued item if not written
+   - if written, mark deleted and delete associated episodic summary when source ID matches
+   - KG deletion for already-ingested facts may be best-effort by source metadata if existing KG store supports it; otherwise flag as v1 limitation in UI/docs
+
+## Todo List
+
+- [ ] Add writer service tests.
+- [ ] Implement approve/reject/delete transitions.
+- [ ] Implement episodic write.
+- [ ] Publish existing semantic event.
+- [ ] Add activity logs.
+- [ ] Verify session episodic tests remain unchanged.
+
+## Success Criteria
+
+- [ ] Approved item appears in episodic search.
+- [ ] KG extraction path is triggered through existing event bus.
+- [ ] Re-approve cannot duplicate episodic/KG data.
+- [ ] Reject/delete are auditable.
+- [ ] Tenant mismatch tests fail closed.
+
+## Risk Assessment
+
+Deleting KG facts after approval may require source-level relation tracking not present today. If exact deletion is not supported, v1 must be honest: delete queued item + episodic source, and mark KG cleanup as best-effort/manual dedup.
+
+## Security Considerations
+
+Approval endpoint must require tenant admin for tenant-scoped channel rows. Master-scope owner is not required because rows are tenant-scoped.
+
+## Next Steps
+
+Expose the review queue and settings through API/UI.

plans/260528-1844-passive-channel-memory-extraction/phase-05-channel-settings-ui-and-admin-controls.md

@@ -0,0 +1,111 @@
+---
+phase: 5
+title: "Channel Settings UI and Admin Controls"
+status: pending
+priority: P2
+effort: "2d"
+dependencies: [2, 3, 4]
+---
+
+# Phase 5: Channel Settings UI and Admin Controls
+
+## Context Links
+
+- HTTP channel handler: `internal/http/channel_instances.go`
+- WS channel methods: `internal/gateway/methods/channel_instances.go`
+- Channel UI: `ui/web/src/pages/channels/channels-page.tsx`, `ui/web/src/pages/channels/channel-detail/`
+- UI types: `ui/web/src/types/channel.ts`
+- i18n: `ui/web/src/i18n/locales/{en,vi,zh}/channels.json`
+
+## Overview
+
+Expose passive memory settings and review queue in channel detail. Keep the UI operator-focused: no global dashboard in v1.
+
+## Requirements
+
+- Functional: configure, manually run extraction, show last run, show counts, approve/reject/delete pending items.
+- Non-functional: mobile-safe controls, no raw secret/PII display beyond already-redacted item text, i18n in EN/VI/ZH.
+
+## Architecture
+
+HTTP endpoints under channel instance scope:
+
+- `GET /v1/channels/instances/{id}/memory-extraction`
+- `PUT /v1/channels/instances/{id}/memory-extraction/settings`
+- `POST /v1/channels/instances/{id}/memory-extraction/run`
+- `GET /v1/channels/instances/{id}/memory-extraction/items`
+- `POST /v1/channels/instances/{id}/memory-extraction/items/{itemID}/approve`
+- `POST /v1/channels/instances/{id}/memory-extraction/items/{itemID}/reject`
+- `DELETE /v1/channels/instances/{id}/memory-extraction/items/{itemID}`
+
+WS methods are optional in v1. Prefer HTTP because current web hooks already use HTTP for channel instances.
+
+## Related Code Files
+
+- Modify: `internal/http/channel_instances.go`
+- Create: `internal/http/channel_memory_extraction_handlers.go`
+- Create: `internal/http/channel_memory_extraction_handlers_test.go`
+- Modify: `ui/web/src/types/channel.ts`
+- Create: `ui/web/src/pages/channels/channel-detail/passive-memory-section.tsx`
+- Create: `ui/web/src/pages/channels/hooks/use-channel-memory-extraction.ts`
+- Modify: `ui/web/src/pages/channels/channel-detail/channel-detail-page.tsx`
+- Modify: locale files in `ui/web/src/i18n/locales/en/`, `vi/`, `zh/`
+
+## Implementation Steps
+
+1. Add HTTP handler tests first:
+   - viewer can read status/items
+   - admin required for settings/run/approve/reject/delete
+   - tenant mismatch returns not found/forbidden without leaking row existence
+2. Add settings validation:
+   - interval min/max
+   - message cap min/max
+   - retention min/max
+   - allowed types from enum
+   - exclude patterns bounded length/count
+3. Add routes to `ChannelInstancesHandler.RegisterRoutes`.
+4. Implement UI hook with TanStack Query and invalidation on mutation.
+5. Build `PassiveMemorySection`:
+   - toggle
+   - interval and cap numeric inputs
+   - review mode toggle
+   - allowed type checkboxes
+   - last run summary
+   - manual run button
+   - pending extracted items list
+   - approve/reject/delete buttons
+6. Add i18n keys to EN/VI/ZH channel namespace.
+7. Ensure mobile rules:
+   - inputs use `text-base md:text-sm`
+   - tables/lists horizontally safe
+   - buttons have sufficient hit area
+
+## Todo List
+
+- [ ] Add HTTP route tests.
+- [ ] Implement handler validation.
+- [ ] Add UI hook.
+- [ ] Add channel detail section.
+- [ ] Add i18n strings.
+- [ ] Add basic UI tests if existing setup supports it; otherwise rely on build + manual browser verification.
+
+## Success Criteria
+
+- [ ] Admin can enable passive memory for a channel.
+- [ ] Admin can force a run.
+- [ ] Pending items appear after extraction.
+- [ ] Admin can approve/reject/delete items.
+- [ ] Viewer cannot mutate settings/items.
+- [ ] Web build passes.
+
+## Risk Assessment
+
+UI can balloon. Keep it as one compact channel-detail section. Do not add global review inbox until v1 proves useful.
+
+## Security Considerations
+
+UI must show redaction status and avoid displaying raw source messages. If source preview is needed later, it must be separate permissioned work.
+
+## Next Steps
+
+Run full verification, docs, and issue handoff.

plans/260528-1844-passive-channel-memory-extraction/phase-06-verification-documentation-and-ship-readiness.md

@@ -0,0 +1,99 @@
+---
+phase: 6
+title: "Verification Documentation and Ship Readiness"
+status: pending
+priority: P1
+effort: "1d"
+dependencies: [1, 2, 3, 4, 5]
+---
+
+# Phase 6: Verification Documentation and Ship Readiness
+
+## Context Links
+
+- Post-implementation checklist in `CLAUDE.md`
+- Docs to update: `docs/05-channels-messaging.md`, `docs/06-store-data-model.md`, `docs/07-bootstrap-skills-memory.md`, `docs/09-security.md`, `docs/18-http-api.md`, `docs/project-changelog.md`
+- Local verification memory: `go build ./...`, `go build -tags sqliteonly ./...`, `go vet ./...`
+
+## Overview
+
+Prove the feature is safe, documented, and ready for implementation PR review. This phase owns the final consistency sweep and release notes.
+
+## Requirements
+
+- Functional: all new tests pass; user-facing docs describe opt-in, review queue, redaction, retention, deletion limits.
+- Non-functional: compile in PG and SQLite builds, no known tenant isolation gaps, no stale plan references in code comments.
+
+## Architecture
+
+No new architecture. This phase verifies all layers together:
+
+1. schema migration
+2. store contracts
+3. worker batching/redaction
+4. review queue writer
+5. HTTP/UI controls
+6. docs
+
+## Related Code Files
+
+- Modify: `docs/05-channels-messaging.md`
+- Modify: `docs/06-store-data-model.md`
+- Modify: `docs/07-bootstrap-skills-memory.md`
+- Modify: `docs/09-security.md`
+- Modify: `docs/18-http-api.md`
+- Modify: `docs/project-changelog.md`
+- Possibly modify: `docs/codebase-summary.md`
+
+## Implementation Steps
+
+1. Run focused Go tests:
+   - `go test ./internal/consolidation`
+   - `go test ./internal/http`
+   - `go test ./internal/store/pg`
+   - `go test -tags sqliteonly ./internal/store/sqlitestore`
+2. Run compile/static checks:
+   - `go build ./...`
+   - `go build -tags sqliteonly ./...`
+   - `go vet ./...`
+3. Run web checks:
+   - `cd ui/web && pnpm build`
+4. If UI changed, run local dev server and browser-check channel detail at desktop/mobile widths.
+5. Update docs listed above.
+6. Update `docs/project-changelog.md`.
+7. Whole-plan consistency sweep:
+   - search code and docs for stale names such as `passiveExtraction`, `channel_memory`, `memory-extraction`
+   - verify route names match docs and UI hooks
+   - verify schema version numbers match migrations
+8. Prepare PR summary with test results and privacy caveats.
+
+## Todo List
+
+- [ ] Focused tests pass.
+- [ ] PG build passes.
+- [ ] SQLite build passes.
+- [ ] Vet passes.
+- [ ] Web build passes.
+- [ ] Browser verification done if UI implemented.
+- [ ] Docs updated.
+- [ ] Changelog updated.
+- [ ] Whole-plan consistency sweep clean.
+
+## Success Criteria
+
+- [ ] Feature satisfies all issue #64 acceptance criteria selected for v1.
+- [ ] No failing tests ignored.
+- [ ] Any unsupported v1 behavior documented explicitly.
+- [ ] PR is ready for code review.
+
+## Risk Assessment
+
+Integration tests with PG/pgvector may require local service. If unavailable, document exactly which integration tests were skipped and why; do not claim full verification.
+
+## Security Considerations
+
+Final review must include a targeted scan for raw message body persistence, secrets in logs, missing tenant predicates, and unauthorized mutation paths.
+
+## Next Steps
+
+After plan approval, execute with `/ck:cook <plan-path>` or equivalent implementation workflow.

plans/260528-1844-passive-channel-memory-extraction/plan.md

@@ -0,0 +1,92 @@
+---
+title: "Passive Channel Memory Extraction"
+description: "Add disabled-by-default, reviewable passive extraction from group channel buffers into episodic Memory and Knowledge Graph."
+status: pending
+priority: P1
+issue: 64
+branch: "codex/issue-64-passive-channel-memory-extraction"
+tags: [channels, memory, knowledge-graph, privacy, tdd, issue-64]
+blockedBy: []
+blocks: []
+created: "2026-05-28T11:45:20.128Z"
+createdBy: "ck:plan"
+source: skill
+---
+
+# Passive Channel Memory Extraction
+
+## Overview
+
+Implement issue #64 as a privacy-first v1. Agents may learn durable context from group/channel conversations they already receive, but only after a channel admin enables the feature. Extraction is batched, redacted, reviewable, tenant-scoped, and writes to Memory/KG only after approval or explicit high-confidence auto-approve.
+
+Hard boundary: no realtime per-message extraction, no default enablement, no permanent raw full-history store, and no cross-channel memory sharing.
+
+## Current Codebase Anchors
+
+| Area | Existing anchor |
+|------|-----------------|
+| Channel instances | `channel_instances.config`, `internal/store/channel_instance_store.go`, `internal/http/channel_instances.go`, `internal/gateway/methods/channel_instances.go` |
+| Group buffer | `channel_pending_messages`, `internal/store/pending_message_store.go`, `internal/channels/history.go`, `internal/channels/history_compaction.go` |
+| Memory/KG pipeline | `internal/consolidation/episodic_worker.go`, `internal/consolidation/semantic_worker.go`, `internal/knowledgegraph/extractor.go` |
+| Activity audit | `internal/store/activity_store.go`, `internal/http/activity.go`, `emitAudit(...)` call sites |
+| Runtime wiring | `cmd/gateway.go`, `cmd/gateway_http_handlers.go`, `cmd/gateway_http_wiring.go`, `cmd/gateway_channels_setup.go` |
+| Web UI | `ui/web/src/pages/channels/`, `ui/web/src/pages/memory/`, `ui/web/src/types/channel.ts` |
+| Schema versions | PG next migration `000074`; bump `internal/upgrade/version.go` from 73. SQLite bump `internal/store/sqlitestore/schema.go` from 42 to 43 |
+
+## Phases
+
+| Phase | Name | Status |
+|-------|------|--------|
+| 1 | [Research and Characterization Tests](./phase-01-research-and-characterization-tests.md) | Pending |
+| 2 | [Schema and Store Contracts](./phase-02-schema-and-store-contracts.md) | Pending |
+| 3 | [Extractor Worker and Redaction](./phase-03-extractor-worker-and-redaction.md) | Pending |
+| 4 | [Memory KG Review Queue Integration](./phase-04-memory-kg-review-queue-integration.md) | Pending |
+| 5 | [Channel Settings UI and Admin Controls](./phase-05-channel-settings-ui-and-admin-controls.md) | Pending |
+| 6 | [Verification Documentation and Ship Readiness](./phase-06-verification-documentation-and-ship-readiness.md) | Pending |
+
+## Dependencies
+
+- GitHub issue: `digitopvn/goclaw#64`
+- Existing group pending-message persistence must stay intact for channel context.
+- Existing Memory/KG workers must keep session behavior unchanged.
+- Standard edition only for channels. Lite has no channels, so SQLite migrations still must compile/apply, but UI should naturally hide channel surfaces through existing edition gates.
+
+## Recommended Defaults
+
+| Setting | Default | Reason |
+|---------|---------|--------|
+| `enabled` | `false` | Sensitive feature; explicit opt-in only |
+| `review_mode` | `true` | Prevent accidental durable PII/secret storage |
+| `interval_minutes` | `360` | Cheap daily-ish cadence without long wait |
+| `message_cap` | `100` | Batching per issue request |
+| `retention_hours` | `168` | 7-day raw buffer cap, same spirit as pending-message TTL |
+| `allowed_types` | people, projects, decisions, todos, preferences, events | Durable work context only |
+| `group_only` | `true` for v1 | Avoid DM privacy ambiguity |
+
+## Success Criteria
+
+- [ ] Passive extraction is disabled by default and opt-in per channel instance.
+- [ ] Extraction runs only by interval, message cap, or manual trigger.
+- [ ] Raw channel messages are read from tenant-scoped pending buffers and are not stored permanently outside existing retention.
+- [ ] Redaction blocks common secrets, tokens, credentials, payment/banking data, phone/address patterns, and configurable excluded users/patterns before durable writes.
+- [ ] Extracted items go to a review queue by default.
+- [ ] Approved items write to `episodic_summaries` with `source_type='channel'` and deterministic `source_id`.
+- [ ] KG ingestion receives only approved/redacted summaries and records source metadata.
+- [ ] Review reject/delete removes queued items and prevents later writes.
+- [ ] Tenant/channel/agent boundaries covered by tests.
+- [ ] PG and SQLite schemas both migrate and compile.
+- [ ] UI exposes settings, last run status, pending items, approve/reject/delete, and manual run.
+
+## Key Risks
+
+| Risk | Mitigation |
+|------|------------|
+| Durable sensitive-data leak | Redaction before queue, review default, no raw full-history durable copy |
+| Duplicate facts | Deterministic run/item source IDs, store-level uniqueness, reuse KG dedup worker |
+| Cross-tenant leak | Tenant-scoped queries and tests before implementation |
+| Cost spike | Batched triggers, minimum content gate, usage caps purpose string |
+| UX overbuild | Channel detail section only in v1; no separate global console |
+
+## Open Questions
+
+- None for v1. Assumptions: review queue required, group channels first, UI in channel detail.