mirror of
https://github.com/tiennm99/tiennm99bot.git
synced 2026-10-11 12:28:54 +00:00
fix(doantu): retune calibration for phow2sim's wider cosine distribution
format.js was inherited from semantle (bge-m3 transformer) whose raw cosines live in a narrow 0.4-0.55 band for unrelated words. phow2sim runs on PhoW2V word2vec — related pairs sit at 0.3-0.5, synonyms at 0.55-0.80 — so the FLOOR=0.4 cutoff was dumping real signal (làng/đất =0.38, làng/phố=0.38) to a displayed 0. Retune: FLOOR=0.1, CENTER=0.4, SCALE=6. Now 0.38 → 39, 0.52 → 64, 0.80 → 93.
This commit is contained in:
1 parent
6373a2f882
commit
7725f5a3ba
2 files changed
+17
-7
No files matched your search
@@ -43,8 +43,10 @@ Override the base URL for local dev via `PHOW2SIM_API_URL`.
|
||||
- `state.js` — KV persistence for game + stats. Same shape as semantle.
|
||||
- `lookup.js` — guess normalization + shape validation. Accepts Unicode
|
||||
letters + combining marks + single internal spaces.
|
||||
- `format.js` — warmth-percent and emoji-bucket formatters (identical
|
||||
to semantle/format.js — score display is language-agnostic).
|
||||
- `format.js` — warmth-percent and emoji-bucket formatters. Calibration
|
||||
constants differ from semantle/format.js: phow2sim (word2vec) has a
|
||||
wider cosine distribution than bge-m3's narrow transformer cone, so
|
||||
`FLOOR=0.1`, `CENTER=0.4`, `SCALE=6` here vs 0.4/0.6/8 there.
|
||||
- `render.js` — Telegram HTML `<pre>` monospace board with a 🇻🇳 header.
|
||||
- `handlers.js` — subject resolution + the three command entry points.
|
||||
Fast-path dedup (exact text OR prior canonical) skips wasted API calls
|
||||
|
||||
@@ -1,12 +1,20 @@
|
||||
/**
|
||||
* @file Display formatting helpers for similarity scores.
|
||||
* Identical to semantle/format.js — score calibration is language-agnostic
|
||||
* because bge-m3 runs on both modules with the same cosine distribution.
|
||||
*
|
||||
* Tuned for phow2sim (PhoW2V word2vec): its cosine distribution is wider
|
||||
* than bge-m3's narrow transformer cone that semantle/format.js calibrates
|
||||
* against. Typical word2vec ranges:
|
||||
* unrelated pairs: 0.00–0.15
|
||||
* loosely related: 0.15–0.30
|
||||
* clearly related: 0.30–0.55
|
||||
* synonyms/same-topic: 0.55–0.80
|
||||
* identical: 1.00
|
||||
* Mapping: raw 0.10 → 0, 0.30 → ~25, 0.40 → ~45, 0.50 → ~60, 0.70 → ~85, 1.00 → 100.
|
||||
*/
|
||||
|
||||
const FLOOR = 0.4;
|
||||
const CENTER = 0.6;
|
||||
const SCALE = 8.0;
|
||||
const FLOOR = 0.1;
|
||||
const CENTER = 0.4;
|
||||
const SCALE = 6.0;
|
||||
|
||||
const sigmoid = (x) => 1 / (1 + Math.exp(-x));
|
||||
const FLOOR_SIG = sigmoid(SCALE * (FLOOR - CENTER));
|
||||
|
||||
Reference in new issue
Block a user