9 Commits
Author SHA1 Message Date
Duy Nguyen 0d6c5bbb7c fix(skills): add agent manage grants
Add explicit per-agent manage grants for skills so granted agents can patch/delete skills when ownership identity drifts.

Expose skill owner and manage-grant controls in the web skills UI, and add PostgreSQL/SQLite migrations plus coverage for preserve/revoke behavior.
2026-05-17 23:15:52 +07:00
Duy /zuey/ 2c2e01644c feat(skills): privacy/visibility controls for agent-owned skills (#1)
* feat(skills): add privacy/visibility controls for agent-owned skills

Closes #1009

- Add private/public visibility enum with validator + normalizer
  (internal/skills/visibility.go)
- Add IsSkillVisibleTo/FilterVisibleSkills authorization helper with
  three-identity ownership check (actor/user/sender) matching #915
- Propagate owner_id into SkillInfo and all PG/SQLite SELECTs so the
  filter has the data it needs
- Agent injection path (FilterSkills, nil allowList) now hides private
  skills owned by other users — fixes the leak vector across tenant
  members
- publish_skill: accept visibility param (defaults to private), replaces
  hardcoded literal
- skill_manage: visibility settable on create and editable via patch,
  including a content-less visibility-only patch that skips version bump
- skills.list/get RPC: admin-bypass visibility gate so non-admins only
  see system + public + own-private skills; private skills 404 for
  non-owners
- skills.update RPC: validate + normalize visibility enum before persist
  (fail closed on unknown values)

* fix(skills): address PR review — i18n error, normalize visibility, auth-first

- Add MsgInvalidVisibility i18n key (en/vi/zh) and use it in skills.update
  RPC instead of raw validator error text.
- Reorder skills.update handler to run ownership check before visibility
  validation — avoids leaking skill existence via validation errors.
- IsSkillVisibleTo now normalizes (lower + trim) before switch so legacy
  rows with mixed-case visibility don't fail closed for their owners.
- Extend TestIsSkillVisibleTo with uppercase/whitespace cases.
2026-05-11 13:03:49 +07:00
viettranx 9e5cec03ba feat: preserve acting sender + migrate ACTOR sites across tool chain
Resolves issue #915 (Telegram group write_file permission denied after
/addwriter) by reconstructing the real sender on announce re-ingress
and migrating ownership/audit call sites off the scope-coupled UserID.

Propagation chain (new MetaOriginSenderID key):
  SubagentTask.OriginSenderID
  → AnnounceMetadata.OriginSenderID
  → InboundMessage.Metadata[MetaOriginSenderID]
  → subagentAnnounceRouting.SenderID
  → RunRequest.SenderID

Same chain for delegate_tool.announceToParent, team_tool_dispatch,
sessions_send, team_tasks_blocker escalation.

ACTOR migration (owner/audit now uses ActorIDFromContext):
- publish_skill: skill owner = individual sender
- skill_manage create/patch/delete: owner = sender; ownership check
  accepts actor or legacy UserID for pre-migration rows
- delegate_tool: DelegateRequest.UserID + DomainEvent.UserID = actor
- team_tasks_blocker: blocker attribution + escalation = actor
- team_tool_cache: team access-policy check = actor (fixes per-user
  allow/deny lists in group chats)

Scope-intentional (unchanged, commented): cron.go, team_tasks_create.go.
2026-04-16 14:17:48 +07:00
viettranx 731a98221a fix(teams): gracefully handle task status races in progress/complete
- executeProgress: early-exit on terminal tasks using pre-fetched status,
  check TaskActionFlags before re-querying DB, auto re-assign on stale recovery
- executeComplete: handle already-completed/failed/cancelled gracefully,
  auto re-assign pending tasks reset by stale recovery ticker
- Increase taskLockDuration 30min→60min, heartbeat 5min (was 10min)
  for 12x safety margin against stale recovery race conditions
- Add diagnostic logging in UpdateTaskProgress when 0 rows affected
2026-03-26 18:00:09 +07:00
viettranx 9168e4b4a7 fix(skills): enforce tenant isolation in skill upload, versioning, and grants
- Inject WithTenantSlug into both HTTP (enrichContext) and WS (MethodRouter.Handle)
  context — previously WithTenantSlug was defined but never called, causing all
  non-master tenants to share the same filesystem path (data isolation breach)
- Scope GetNextVersion, GetNextVersionLocked, and CreateSkillManaged version
  queries by tenant_id to prevent cross-tenant version collision
- Make advisory lock tenant-aware (hash tenant_id + slug) so different tenants
  uploading the same slug don't contend on the same lock
- Add agent tenant_id validation in GrantToAgent to prevent cross-tenant grant
  injection
- Fix IsSystemSkill to filter by MasterTenantID only
- Fix StoreMissingDeps to work for both system and custom skills
2026-03-26 13:32:28 +07:00
viettranx 81c8414423 refactor(store): fix goroutine safety issues across cron, sessions, skills, and gateway
- Replace 16x bare db.Query/db.Exec with context-aware variants to prevent
  orphan DB operations when parent context is cancelled
- Thread baseCtx through PGCronStore lifecycle (Start/Stop) for all internal
  scheduler, executor, and job cache operations
- Add ctx parameter to StoreMissingDeps and ListSystemSkillDirs, update all callers
- Add sync.WaitGroup tracking to fire-and-forget goroutines in gateway consumer
  handlers (subagent announce, teammate message) with safego.Recover panic protection
- Add safego.Recover to 3 Slack channel goroutines to prevent WaitGroup hang on panic
2026-03-25 08:20:10 +07:00
viettranx 6e80a1af61 fix(skills): unblock agent read_file on skill paths and fix web UI file browser
- Add builtinSkillsDir + tenant-scoped dirs to read_file AllowPaths
- Use DB file_path in HTTP skill file handlers instead of hardcoded baseDir
- Add bundled dir fallback (system skills only) when managed copy is missing
- Tenant-scope publish_skill tool destination directory
- Extract walkSkillFiles/readSkillFile/skillSlugDir helpers (DRY)
- Return is_system from GetSkillFilePath for security-guarded fallback
2026-03-23 22:06:39 +07:00
viettranx 843b550651 feat: runtime packages UI, pkg-helper, configurable shell deny groups (#244)
Runtime package management with security hardening:

- pkg-helper: root-privileged daemon for apk install/uninstall via Unix socket
- HTTP API: /v1/packages (list/install/uninstall/runtimes), admin role required for writes
- Shell deny groups: 15 configurable groups (per-agent overrides via context)
- Packages UI: Web page for managing system/pip/npm packages with confirmation dialogs
- Docker: privilege separation (root entrypoint → su-exec drop), init for zombie reaping
- Security: umask socket creation, persist file validation, deny pattern hardening
  (Node.js fetch/http, Python from/import, curl localhost, sensitive env vars)
- Auth: empty gateway token → admin role (dev/single-user mode)
2026-03-17 19:50:26 +07:00
Viet TranandGoon ace07509b7 feat(skills): system skills integration — toggle, dep checking, per-item install (#161)
* feat(infra): add runtime package support for skills

Install nodejs, npm, pandoc, github-cli + pre-install Python packages
(openpyxl, pandas, python-pptx, markitdown) and Node packages
(docx, pptxgenjs). Configure runtime dirs for agent pip/npm installs
with PIP_TARGET, NPM_CONFIG_PREFIX, NODE_PATH to enable dynamic
package installation in read-only container environment.

* feat(infra): add bundled skills with runtime package support

- Add 5 bundled skills: docx, pdf, pptx, xlsx, skill-creator from container skills-store
- Wire GOCLAW_BUILTIN_SKILLS_DIR env var in gateway and CLI
- Support optional runtime packages alongside dynamic skill loading
- Update Dockerfile to COPY bundled-skills at /app/bundled-skills/
- Add PIP_CACHE_DIR in docker-entrypoint.sh for clean pip installs
- Document bundled skills in 14-skills-runtime.md section 6

* feat(infra): remove ai-multimodal skill directory from bundled skills

Remove the ai-multimodal skill package as part of consolidating runtime
package support for bundled skills. This directory is no longer needed
in the bundled skills structure.

* feat(ci): add semantic release and Docker Hub publishing

Add go-semantic-release workflow to auto-create semver tags on merge to
main. Extend docker-publish to push all variants to both GHCR and
Docker Hub (digitop/goclaw).

* feat(skills): add system skills infrastructure with is_system column, dep scanning, and seeder

- Migration 000017: add is_system boolean column with partial index
- Store layer: UpsertSystemSkill, delete protection, IsSystemSkill
- ListAccessible auto-includes system skills (no grants needed)
- ListWithGrantStatus returns is_system field
- Dependency scanner: auto-detect deps from scripts/ or skill-manifest.json
- Dependency checker: verify system binaries, Python/Node packages
- Seeder: seed bundled skills into DB on startup (idempotent via hash)
- Gateway wiring: GOCLAW_BUNDLED_SKILLS_DIR env for bundled skills
- HTTP: delete guard (403), slug conflict check (409), rescan-deps endpoint
- UI: System badge, hide delete for system skills, rescan deps button
- Agent skills tab: "Always available" for system skills
- i18n: en/vi/zh keys for system skills, deps scanning

* feat(skills): conditional system prompt, skill manifests, and Zip Slip fix

- System prompt: only show package list when python3/node are available
- Add skill-manifest.json for pdf, docx, xlsx, pptx bundled skills
- Fix Zip Slip vulnerability in office/unpack.py (all 3 copies)

* refactor(skills): extract shared office code to _shared/ and deduplicate

Move office scripts (pack, unpack, validate, schemas, validators) from
duplicated copies in docx/xlsx/pptx to skills/_shared/office/ with
symlinks. Remove soffice.py (non-functional in containers) and update
SKILL.md references to use soffice binary directly. Update seeder
copyDir to follow symlinks.

Removes ~45K lines of duplicate code across 3 skills.

* fix(skills): address code review findings for system skills integration

- H1: Remove dead symlink branch in copyDir (filepath.Walk follows symlinks)
- H3: Fix rescan-deps to query ALL skills (including archived) and re-activate
  when deps become available; add ListAllSkills() + Status field to SkillInfo
- H4: Add Status field to SkillCreateParams, stop overloading Visibility
- M1: Batch Python/Node dep checks into single subprocess per runtime
- M4: Add rows.Err() check in ListSkills to prevent caching partial results

* feat(skills): async dep checking with realtime WS events

Split Seed() into sync DB upsert + async CheckDepsAsync() goroutine.
Gateway startup no longer blocks on Python/Node subprocess dep checks.

- Seed() returns seeded skills list, all initially status="active"
- CheckDepsAsync() runs in background, emits skill.deps.checked per-skill
- skill.deps.complete event emitted when all checks finish
- Each failed dep check: archives skill + BumpVersion() for immediate
  cache invalidation so next agent turn picks up the change
- UI: use-query-invalidation listens to skill.deps.* events → auto-refresh
  skills list in realtime

* feat(skills): system skills integration with toggle, dep checking, and per-item install

- Add is_system, deps, enabled columns to skills table (migration 017)
- Seed bundled core skills (pdf, docx, pptx, xlsx, skill-creator) on startup
- PYTHONPATH-based dep detection — eliminates false positives from local modules
- Per-item dep install UI with individual status (installing/success/error)
- Enable/disable toggle for core and custom skills (independent of dep status)
- Re-run dep check when skill is toggled back on
- Inline skill thresholds: 40 skills / 5000 tokens before switching to search mode
- Fix UpsertSystemSkill: backfill null file_hash without bumping DB version
- Remove redundant skill-manifest.json files (replaced by deps JSONB column)
- Show author from frontmatter in custom skills tab
- Runtime checker for python3/pip3/node/npm availability
- WS events for dep checking/installing progress
- docs: add 15-core-skills-system.md, 16-skill-publishing.md

---------

Co-authored-by: Goon <duy@wearetopgroup.com>
2026-03-12 09:20:41 +07:00