tiennm99/blog
1
0 Fork 0
mirror of https://github.com/tiennm99/blog.git synced 2026-10-18 18:15:18 +00:00
Code Issues Packages Projects Releases Wiki Activity

refactor(skills): invoke the Node engine and centralise the command reference

Browse Source 
Switches every call site to `node scripts/newsletter` and, in the same pass,
fixes three structural problems in the skill layer.

The seven subcommands were spelled out in five files; they are now enumerated
once in docs/newsletter/engine-commands.md, and every other mention is a link
except where a skill inlines the one or two commands it actually invokes. The
shared post mechanics move out of mt-add-url's directory into docs/newsletter/,
so no skill owns another skill's documentation and both runtimes reach it by a
repository-relative path. The Codex adapters become symlinks to their canonical
counterparts, which removes the parallel frontmatter that could drift.

Two skills are renamed for a uniform mt-<verb>-<object> scheme: mt-add-post
becomes mt-add-article, since it adds an article to a post, and mt-webfetch
becomes mt-fetch-url. The other four already conformed. Note that $mt-add-post
and mt-webfetch no longer resolve; no alias directories are left behind, because
two names for one skill is the duplication this change removes.

Trigger wording in the frontmatter descriptions is left alone apart from the
renamed tokens and the corrected fetch-tier summary: those strings are how the
runtimes decide whether to invoke a skill.

Setup documentation now states the npm ci prerequisite. The engine has
dependencies, so unlike its predecessor it does not run on a bare clone.
This commit is contained in:
tiennm99 committed 2026-09-18 16:24:34 +07:00
1 parent 682d5239b8
commit baab3d443a
19 files changed
+258 -134
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/multi-tool-usage.md
+9 -9
View File
@@ -1,29 +1,29 @@
# Multi-tool usage (Claude Code · OpenCode · Codex)
This repo is usable from three AI coding tools off **one shared engine**. The newsletter engine lives in `scripts/newsletter/` (Go, stdlib only) and every tool calls it as `go run ./scripts/newsletter <command>` from the repo root. Project instructions live once in `AGENTS.md`.
This repo is usable from three AI coding tools off **one shared engine**. The newsletter engine lives in `scripts/newsletter/` (Node ESM) and every tool calls it as `node scripts/newsletter <command>` from the repo root — the commands are documented once in [newsletter/engine-commands.md](newsletter/engine-commands.md). Project instructions live once in `AGENTS.md`.
## Per-tool setup & invocation
| Tool | Instruction file | How the workflow surfaces | Setup | Invocation example |
|------|------------------|---------------------------|-------|--------------------|
| **Claude Code** | `CLAUDE.md` → imports `AGENTS.md` | Native skill auto-dispatch from `.claude/skills/` | None (works as-is) | Paste a URL; `mt-add-url` auto-runs |
| **OpenCode** | `AGENTS.md` (auto-read) | `.claude/skills/` auto-discovered via the `skill` tool, governed by `opencode.json` | None beyond `opencode.json` (committed) | Ask to add a URL; pick/`skill` `mt-add-url` |
| **Codex** | `AGENTS.md` (auto-read) | Repository skills discovered from `.agents/skills/` | None (works as-is) | Ask to add a URL or invoke `$mt-add-url` |
| **Claude Code** | `CLAUDE.md` → imports `AGENTS.md` | Native skill auto-dispatch from `.claude/skills/` | `npm ci` once | Paste a URL; `mt-add-url` auto-runs |
| **OpenCode** | `AGENTS.md` (auto-read) | `.claude/skills/` auto-discovered via the `skill` tool, governed by `opencode.json` | `npm ci` once; `opencode.json` is committed | Ask to add a URL; pick/`skill` `mt-add-url` |
| **Codex** | `AGENTS.md` (auto-read) | Repository skills discovered from `.agents/skills/` | `npm ci` once | Ask to add a URL or invoke `$mt-add-url` |
**Shared engine:** all three call `go run ./scripts/newsletter <command>` from repo root — no per-tool script copies.
**Shared engine:** all three call `node scripts/newsletter <command>` from repo root — no per-tool script copies. It has npm dependencies, so run `npm ci` once after cloning.
### Notes per tool
- **Claude Code** — `CLAUDE.md` is a thin file that imports `AGENTS.md` (`@AGENTS.md`); all instruction content lives in `AGENTS.md`. Permissions in `.claude/settings.json`.
- **OpenCode** — reads `.claude/skills/<name>/SKILL.md` natively (multi-location search up the worktree), so the same `mt-*` skills work with zero duplication. `opencode.json` mirrors the Claude allow-list intent (node/git/hugo/edit). No MCP block — relies on OpenCode's built-in web fetch/search. Restart OpenCode after editing `opencode.json` (config is not hot-reloaded).
- **Codex** — reads repository skills from `.agents/skills/` and can invoke them explicitly (`$mt-add-url`) or implicitly from their descriptions. The checked-in Codex skills delegate to the canonical workflow definitions in `.claude/skills/`, so workflow behavior stays shared without an install or sync step.
- **Codex** — reads repository skills from `.agents/skills/` and can invoke them explicitly (`$mt-add-url`) or implicitly from their descriptions. Each `.agents/skills/<name>/SKILL.md` is a **symlink** to its `.claude/skills/` counterpart, so there is one file to edit and nothing that can drift. If a checkout cannot follow symlinks (Windows without developer mode), replace them with thin pointer files that say "read and follow `.claude/skills/<name>/SKILL.md`".
## Teardown — pick one, remove the rest
Once you commit to one tool, delete the others' config:
- **Keep Claude Code only** → remove `opencode.json` and `.agents/`. Keep `.claude/`.
- **Keep Claude Code only** → remove `opencode.json` and `.agents/`. Keep `.claude/` and `docs/newsletter/`.
- **Keep OpenCode only** → remove `.claude/settings*.json` and `.agents/`. **Keep `.claude/skills/`** — OpenCode reads it. If you want to drop the `.claude/` dir entirely, first move skills to `.opencode/skills/` (scripts are path-neutral in `scripts/newsletter/`, so the move breaks nothing). Optionally make `AGENTS.md` the only instruction file (delete `CLAUDE.md`).
- **Keep Codex only** → keep `.agents/skills/`, `AGENTS.md`, `scripts/newsletter/`, and the canonical `.claude/skills/` files referenced by the Codex skills. You may remove Claude-specific settings and `opencode.json`.
- **Keep Codex only** → keep `.agents/skills/`, `AGENTS.md`, `scripts/newsletter/`, `docs/newsletter/`, and the canonical `.claude/skills/` files the Codex skills symlink to. You may remove Claude-specific settings and `opencode.json`.
In every case: **keep `scripts/newsletter/`, `.claude/skills/`, and `AGENTS.md`** — they are the shared engine, canonical workflows, and instructions.
In every case: **keep `scripts/newsletter/`, `.claude/skills/`, `docs/newsletter/`, and `AGENTS.md`** — they are the shared engine, canonical workflows, shared documentation, and instructions.
docs/newsletter/engine-commands.md
+166
View File
@@ -0,0 +1,166 @@
# Newsletter engine commands
The portable newsletter engine lives in `scripts/newsletter/` — Node ESM, one
module per subcommand. **This file is the only place the commands are
enumerated**; skills and docs link here rather than restating the list.
## Prerequisite
The engine has three runtime dependencies (`cheerio`, `defuddle`, `linkedom`), so
a fresh clone needs one setup step:
```bash
npm ci
```
Skip it and the content-parsing subcommands print
`newsletter engine: dependencies missing — run 'npm ci' from the repo root`
and exit 1 rather than a module-resolution stack trace. `fetch-via-defuddle` is
the exception: its local tier needs the dependencies, so without them it falls
through to the `defuddle.md` proxy and still succeeds.
## Invocation
Always from the repo root — repo-relative paths (`content/post`) resolve from the
process working directory:
```bash
node scripts/newsletter <command> [args]
```
JSON output is 2-space indented with a trailing newline, and is not HTML-escaped
(URLs keep their `&` readable). Optional fields are **omitted** when empty rather
than emitted as `""`.
## Commands
### `add-url <url>`
Classifies and deduplicates one URL — the single entry point per URL.
```json
{
"original_url": "…",
"clean_url": "…",
"http_status": "200",
"accessible": true,
"duplicate": false,
"route": "article",
"title": "…",
"author": "…"
}
```
- `route` ∈ `youtube | image | video | document | article`
- `clean_url` has tracking parameters stripped (`utm_*`, `fbclid`, `gclid`,
`ref`, …); surviving parameters keep their original order and encoding
- For `youtube`, `clean_url` is the canonical `https://www.youtube.com/watch?v=ID`
and `title` / `author` come from oEmbed. Both keys are absent for every other
route, and absent when oEmbed fails.
- `duplicate` is boundary-aware: a URL that is merely a *prefix* of one already
stored is not a duplicate. Substack images dedupe on their image uuid, so
transform and size variants collapse onto one identity.
### `find-newsletter-number`
Prints the next newsletter number (plain text). Scans `content/post/YYYY/MM/DD/`
newest-year-first and stops at the first year that holds a `Newsletter #N`
heading, so a stale higher number in an older year does not win. An empty tree
yields `1`.
### `list-existing-tags`
Prints tag frequencies, most-used first, top 40, as `%6d %s` — the count
right-aligned in six columns. Ties keep first-seen (lexical walk) order.
### `detect-image-source <url>`
Detects a Substack-hosted image and extracts its S3 image uuid.
```json
{
"original_url": "…",
"clean_url": "…",
"isSubstack": true,
"uuid": "…",
"innerUrl": "…"
}
```
`uuid` and `innerUrl` are present only when the URL is a Substack image and the
value is non-empty. A malformed percent sequence in the CDN wrapper falls back to
the raw substring rather than failing.
### `find-substack-post --uuid <uuid> [--deep]`
Finds which Substack post embeds an image uuid, and extracts a label. Only the
double-dash flag form is accepted.
Searches each publication in `scripts/newsletter/config/substack-publications.json`
(editable, read at runtime) — the RSS feed first, then, with `--deep`, a sitemap
crawl back ~3 months sharing a 40-fetch budget across all publications.
On a hit:
```json
{
"found": true,
"source": "rss",
"publication": "…",
"postTitle": "…",
"postUrl": "…",
"caption": "…",
"candidates": ["…"]
}
```
On a miss: `{"found": false}`, or with `--deep`
`{"found": false, "source": "sitemap", "scanned": 0, "budget": 40, "cutoff": null}`
where `cutoff` is the crawl boundary date, or `null` when no sitemap could be
fetched at all.
### `fetch-via-defuddle <url>`
Fallback fetcher for pages that blocked the built-in fetch. Two tiers, in order:
local defuddle extraction, then the `defuddle.md` proxy (which fetches from its
own IP — the tier that matters when this machine's IP is the blocked one).
Both tiers write YAML frontmatter followed by the markdown body, so the caller
parses one shape either way; stderr names which tier failed and why. An
extraction that looks like a bot challenge rather than the requested page counts
as a local-tier failure, so the proxy still gets its turn.
| Exit | Meaning |
|---|---|
| 0 | content returned |
| 1 | every tier failed |
| 2 | bad arguments |
### `post-stats <path/to/index.md>`
Counts what a post already holds, so a handler can report a running tally.
```json
{
"post": "content/post/2026/09/10/index.md",
"newsletter": 132,
"articles": 8,
"images": 3,
"videos": 1,
"documents": 0,
"total": 12
}
```
`newsletter` is `0` when the post carries no `Newsletter #N` heading.
## Tests
```bash
npm test # deterministic tier, offline, ~2s
NEWSLETTER_NET=1 npm test # adds the live-upstream tier
```
Golden outputs under `scripts/newsletter/__fixtures__/golden/` were captured from
the previous Go implementation; see
`plans/reports/parity-260918-newsletter-js-migration-report.md`.
docs/newsletter/post-mechanics.md
+134
View File
@@ -0,0 +1,134 @@
# Newsletter Post Mechanics (shared)
Shared procedure used by the newsletter handler skills (`mt-add-article`, `mt-add-video`, `mt-add-image`).
The shared engine lives in `scripts/newsletter/` — see [engine-commands.md](engine-commands.md) for every command and the `npm ci` prerequisite.
**Project Context:**
- Hugo static site, theme `hugo-theme-stack`
- Language: Vietnamese (99%) + minimal English tech terms (1%)
- Timezone: Asia/Ho_Chi_Minh (UTC+7)
- Content path: `content/post/YYYY/MM/DD/index.md`
## 1. Find / create the target post
The target defaults to today's date but the user can pin a different one.
**Target date resolution:**
- Default: current date in `YYYY-MM-DD` (UTC+7).
- **Pinned target:** when the user names a post or date to keep working on — typically an unpublished draft started on an earlier day ("keep adding to the 7/9 post") — that post is the target for the rest of the session, including across a date rollover mid-session. Do not silently start a new post for the new day; if a pin might have lapsed, ask before creating one.
Check `content/post/YYYY/MM/DD/index.md` for the resolved target date:
- **Exists** → update this file.
- **Missing** → create it (new newsletter number; template below). Create directories as needed.
The newsletter number always comes from the target post, not from today's date.
## 2. Newsletter number
```bash
node scripts/newsletter find-newsletter-number
```
Searches backwards from today for the most recent newsletter and returns the next number. Only needed when **creating** a new post — when the target post already exists, read its number from its own `title`.
## 2a. Terminology
Refer to each numbered publication as a **newsletter**, not an issue. Use wording like `Newsletter #120`, `add newsletter 120`, or `add Newsletter #120` in reports and commit messages.
## 3. New post template
```markdown
---
title: "Newsletter #[number]"
date: YYYY-MM-DD
tags: ["AI-Assisted"]
categories: ["Newsletter"]
---
*Mời bạn thưởng thức Newsletter #[number].*
```
(Handlers then append their own content block — article body, or Bonus entry.)
## 4. Section insertion (no clobbering)
**Never rewrite the whole `index.md`.** Always insert by anchoring an Edit on an existing string (e.g. `### Bonus`, `**Videos:**`) and prepending/appending around it. Multiple URLs targeting the same day's post must be applied **sequentially** so one edit doesn't clobber another.
**Articles go before the `### Bonus` section; Bonus assets go inside it.**
Article headings must use the original source title. Do not translate/localize article titles; keep the source-language wording, capitalization, punctuation, and proper nouns from metadata (`og:title`, page title, or fetcher frontmatter). Summaries stay Vietnamese and are prose paragraphs only — no key-points bullet list.
To insert an article safely, anchor the Edit on `### Bonus` and prepend:
```
old_string: "### Bonus"
new_string: "## [Original Article Title](clean_url)\n\n[Summary]\n\n### Bonus"
```
If the post has **no `### Bonus`** yet:
- Article → append at end of file. Do NOT create an empty Bonus section.
- Bonus asset (video/image/doc) → create the `### Bonus` section now (only because there's an asset to put in it).
**Bonus section format:**
```markdown
### Bonus
**Images:**
![title](url)
**Videos:**
[Tiêu đề video tiếng Việt](https://www.youtube.com/watch?v=ID)
> Tóm tắt 1-2 câu về nội dung video.
[Video: title](url) <!-- direct video file -->
**Documents:**
[PDF: title](url)
```
When a subsection (e.g. `**Videos:**`) already exists, append under it; otherwise create it. Keep subsections in this order: **Images** → **Videos** → **Documents**.
## 4a. Post tally (report line)
After every successful insertion, report the target post's running totals so the user can see what the post now holds:
```bash
node scripts/newsletter post-stats content/post/YYYY/MM/DD/index.md
```
Output (JSON): `{ post, newsletter, articles, images, videos, documents, total }`.
Render it as a single TL;DR line in the handler's report, omitting zero counts:
```
📊 4 articles · 2 videos · 1 image
```
Use the counts the command returns — do not tally by hand. This is a **report-only** line: never write it into `index.md`.
## 5. Post content language guidelines
These rules apply only to text written into `content/post/**/index.md`. Keep user-facing questions, status updates, reports, and final responses in English unless the user explicitly requests another language.
**Primary**: Vietnamese (99%) | **Secondary**: English (1% — unavoidable technical terms only)
**Allow English when no Vietnamese equivalent exists:**
- Technology: AI, API, GitHub, Rust, Go, Docker, Kubernetes
- Databases: PostgreSQL, MongoDB, Redis
- Companies: Google, Microsoft, Databricks
- Acronyms: MCP, CI/CD, JSON, SQL, HTTP
**Always translate to Vietnamese:**
- Common terms: code → mã nguồn, software → phần mềm, deployment → triển khai, performance → hiệu năng
- Action verbs: use → sử dụng, check → kiểm tra, build → xây dựng, test → kiểm thử
- Any other term: translate if a common Vietnamese equivalent exists
**Preserve original source language:**
- Article headings and image labels must retain the source wording, capitalization, and punctuation.
- Do not translate or localize article headings or image labels unless the user explicitly asks.
- Video titles follow the `mt-add-video` handler's localization rules.
**Content requirements:** audience = junior developers; tone = professional, clear, accessible; verify content is ≥99% Vietnamese before saving.
## 6. Error handling
| Problem | Action |
|-------|--------|
| URL inaccessible | Skip, note in report |
| Duplicate URL | Skip, note in report |
| Extraction fails | Skip, treat as error |
| Directory missing | Create directories |