mirror of
https://github.com/tiennm99/blog.git
synced 2026-10-11 03:13:10 +00:00
refactor(scripts): migrate newsletter engine from Node to Go
One Go binary with a subcommand per former script, invoked as 'go run ./scripts/newsletter <command>' from repo root. Stdlib only; publications config embedded via go:embed (go run recompiles on edit). Behavior parity verified side-by-side before deleting the JS — see plans/reports/parity-260818-newsletter-go-migration-report.md. Permissions: allow 'Bash(go *)', drop 'Bash(node *)' and the redundant 'Write(content)' rule (Edit rules cover all file-editing tools).
This commit is contained in:
1 parent
211863b982
commit
e8451d8c24
34 files changed
+1303
-762
No files matched your search
@@ -21,24 +21,24 @@ A clean image URL (passed by `mt-add-url`, or given directly).
|
||||
|
||||
### 1. Detect source
|
||||
```bash
|
||||
node scripts/newsletter/detect-image-source.js "<url>"
|
||||
go run ./scripts/newsletter detect-image-source "<url>"
|
||||
```
|
||||
→ `{ original_url, clean_url, isSubstack, uuid?, innerUrl? }`.
|
||||
|
||||
When invoked **directly** (not via `mt-add-url`), first run the router to get accessibility + duplicate status and skip accordingly:
|
||||
```bash
|
||||
node scripts/newsletter/add-url.js "<url>" # expect route:image; skip if duplicate/!accessible
|
||||
go run ./scripts/newsletter add-url "<url>" # expect route:image; skip if duplicate/!accessible
|
||||
```
|
||||
(When dispatched by `mt-add-url`, that check already ran — don't repeat it.)
|
||||
|
||||
### 2a. Substack image (`isSubstack: true` with `uuid`)
|
||||
Find the source post:
|
||||
```bash
|
||||
node scripts/newsletter/find-substack-post.js --uuid <uuid>
|
||||
go run ./scripts/newsletter find-substack-post --uuid <uuid>
|
||||
```
|
||||
- `found: false` → retry with the deeper sitemap crawl (slower — fetches posts ~3 months back, capped at 40 fetches total across all publications; warn the user it may take a while):
|
||||
```bash
|
||||
node scripts/newsletter/find-substack-post.js --uuid <uuid> --deep
|
||||
go run ./scripts/newsletter find-substack-post --uuid <uuid> --deep
|
||||
```
|
||||
On a miss the result reports `scanned` (posts fetched), `budget` (the 40-fetch cap), and `cutoff` (oldest date looked at) — mention how far back it looked.
|
||||
- `found: false` after `--deep` → no source post; go to step 3 (ask) and/or step 4 (add publication).
|
||||
|
||||
@@ -13,7 +13,7 @@ Shared scripts: `scripts/newsletter/`. Shared procedure: `../mt-add-url/referenc
|
||||
|
||||
A clean article URL (passed by `mt-add-url`, or given directly). If a raw URL is provided directly, you may run the classifier to clean/dedup it first:
|
||||
```bash
|
||||
node scripts/newsletter/add-url.js "<url>"
|
||||
go run ./scripts/newsletter add-url "<url>"
|
||||
```
|
||||
Trust `route: article`; skip if `duplicate` or not `accessible`.
|
||||
|
||||
|
||||
@@ -50,7 +50,7 @@ Read the full post body. Identify:
|
||||
|
||||
Before generating, run:
|
||||
```bash
|
||||
node scripts/newsletter/list-existing-tags.js
|
||||
go run ./scripts/newsletter list-existing-tags
|
||||
```
|
||||
When a proposed tag matches an existing one case-insensitively, use the existing casing.
|
||||
-->
|
||||
|
||||
@@ -20,7 +20,7 @@ Everything else (direct `video` file, `document`, or anything unrecognized) is *
|
||||
|
||||
For every URL the user provides:
|
||||
```bash
|
||||
node scripts/newsletter/add-url.js "<url>"
|
||||
go run ./scripts/newsletter add-url "<url>"
|
||||
```
|
||||
Output (JSON): `{ original_url, clean_url, http_status, accessible, duplicate, route, title?, author? }`.
|
||||
|
||||
|
||||
@@ -18,7 +18,7 @@ Get current date in `YYYY-MM-DD` (UTC+7). Check `content/post/YYYY/MM/DD/index.m
|
||||
## 2. Newsletter number
|
||||
|
||||
```bash
|
||||
node scripts/newsletter/find-newsletter-number.js
|
||||
go run ./scripts/newsletter find-newsletter-number
|
||||
```
|
||||
Searches backwards from today for the most recent newsletter and returns the next number. Only needed when **creating** a new post.
|
||||
|
||||
|
||||
@@ -19,7 +19,7 @@ A clean YouTube URL (passed by `mt-add-url`, or given directly).
|
||||
|
||||
1. **Classify / fetch title** — run the router to get the canonical URL + title:
|
||||
```bash
|
||||
node scripts/newsletter/add-url.js "<url>"
|
||||
go run ./scripts/newsletter add-url "<url>"
|
||||
```
|
||||
Confirm `route: youtube`; skip if `duplicate` or not `accessible`. Use the returned `clean_url` (canonical `watch?v=ID`) and `title`.
|
||||
- If `title` is missing (oEmbed failed), fetch the title via WebFetch on the watch URL.
|
||||
|
||||
@@ -28,7 +28,7 @@ Use this skill only after a WebFetch attempt returned one of:
|
||||
1. Confirm WebFetch already failed on the target URL
|
||||
2. Run the fetch script:
|
||||
```bash
|
||||
node scripts/newsletter/fetch-via-defuddle.js "<target_url>"
|
||||
go run ./scripts/newsletter fetch-via-defuddle "<target_url>"
|
||||
```
|
||||
Alternatively, use WebFetch with the defuddle-prefixed URL:
|
||||
```
|
||||
@@ -70,7 +70,7 @@ Never loop. Never retry more than once.
|
||||
User wanted to extract content from https://example.com/article
|
||||
WebFetch returned: "Request failed with status code 403"
|
||||
→ Trigger mt-webfetch
|
||||
→ node scripts/newsletter/fetch-via-defuddle.js "https://example.com/article"
|
||||
→ go run ./scripts/newsletter fetch-via-defuddle "https://example.com/article"
|
||||
→ Parse markdown output
|
||||
→ Summarize as usual
|
||||
```
|
||||
Reference in new issue
Block a user