mirror of
https://github.com/tiennm99/noitu.git
synced 2026-10-11 03:13:45 +00:00
76 lines
2.7 KiB
YAML
76 lines
2.7 KiB
YAML
# Monthly dictionary refresh.
|
|
#
|
|
# Wikimedia regenerates the Wiktionary tiếng Việt dump every month. This
|
|
# downloads the current one, regenerates data/dictionary.txt from it — the
|
|
# committed corpus the image is built from — and commits it straight to main,
|
|
# which deploys it. The commit is the month's diff of the word list.
|
|
#
|
|
# The dump run that starts on the 1st takes a few days to publish, hence the
|
|
# 10th. The builder fails below its word, page and meaning-coverage floors, so
|
|
# a broken month commits nothing rather than a corpus with holes in it. A month
|
|
# whose dump matches the committed corpus commits nothing either.
|
|
#
|
|
# A push made with GITHUB_TOKEN does not trigger ci.yml. The build here
|
|
# already verifies the database it derives; the full suite runs on the next
|
|
# push to main.
|
|
name: refresh-dictionary
|
|
|
|
on:
|
|
schedule:
|
|
- cron: "0 3 10 * *"
|
|
workflow_dispatch:
|
|
|
|
permissions:
|
|
contents: write
|
|
|
|
concurrency:
|
|
group: refresh-dictionary
|
|
cancel-in-progress: false
|
|
|
|
jobs:
|
|
refresh:
|
|
name: Regenerate the corpus from the current dump
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v7
|
|
with:
|
|
ref: main
|
|
- uses: actions/setup-go@v7
|
|
with:
|
|
go-version: stable
|
|
cache-dependency-path: server/go.sum
|
|
|
|
- name: Download the dump
|
|
run: make fetch-dict
|
|
|
|
- name: Regenerate the corpus
|
|
run: make refresh-dict
|
|
|
|
# The commit message reports the dump's date and hash from the corpus
|
|
# header, and how many words came and went.
|
|
- name: Commit and push
|
|
run: |
|
|
set -eu
|
|
if git diff --quiet -- data/dictionary.txt; then
|
|
echo "The corpus matches the current dump; nothing to commit."
|
|
exit 0
|
|
fi
|
|
header() { sed -n "s/^#@$1 //p" data/dictionary.txt; }
|
|
fetched="$(header source_fetched_at)"
|
|
added="$(git diff -U0 -- data/dictionary.txt | grep -c '^+[^+#]' || true)"
|
|
removed="$(git diff -U0 -- data/dictionary.txt | grep -c '^-[^-#]' || true)"
|
|
words="$(grep -vc '^#\|^$' data/dictionary.txt)"
|
|
git config user.name 'github-actions[bot]'
|
|
git config user.email '41898282+github-actions[bot]@users.noreply.github.com'
|
|
git add data/dictionary.txt
|
|
git commit -F - <<MSG
|
|
chore(data): refresh the dictionary from the ${fetched%%T*} dump
|
|
|
|
Regenerated data/dictionary.txt from the Wiktionary tiếng Việt dump of ${fetched}.
|
|
|
|
- Dump SHA-256: $(header source_sha256)
|
|
- Words in the corpus: ${words}
|
|
- Lines added: ${added}, lines removed: ${removed} (a word whose meanings changed counts as both)
|
|
MSG
|
|
git push origin HEAD:main
|