refactor: split into web/crawler/go-parser and drop the 2017 archives

Browse Source

Move the frontend into web/, the repo's only npm workspace, and replace the
JS crawler with a Go module covering both remaining datasets. The crawler
writes to a .part file and renames on completion: writing straight to the
destination left truncated files that the skip-if-present check would then
skip forever.

Remove the 2017-old and 2017-old2 datasets. They were successive publications
of the same exam, kept side by side so the disagreement stayed inspectable;
the current 2017 supersedes them and they remain in git history.

Recover the 2016 crawler source from the Internet Archive's copy of the
aggregator article, whose original host no longer resolves. All 119 filenames
are verified against data/2016 in both directions, but no archive captured the
spreadsheets themselves, so the host still serving them is unconfirmed and
data/2016 remains the only confirmed copy.

Filenames are load-bearing throughout: go-parser sorts inputs bytewise and
inserts last-wins, so they decide which row survives a duplicate exam number.

...

This commit is contained in:

tiennm99 committed 2026-08-13 21:45:05 +07:00

1 parent 8530df0154

commit ceb694a747

163 files changed

+1317 -510

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

data/2017-old/10_BinhThuan_RIIW.xls.xlsx

BIN

Binary file not shown.

data/2017-old/10_Ca_Mau_BKXT.xls.xlsx

BIN

Binary file not shown.

data/2017-old/10_LamDong_GNFT.xls.xlsx

BIN

Binary file not shown.

data/2017-old/10_Soc_Trang_XCGJ.xls.xlsx

BIN

Binary file not shown.

data/2017-old/11_BinhDuong_RYQL.xls.xlsx

BIN

Binary file not shown.

data/2017-old/11_LaoCai_GMSU.xls.xlsx

BIN

Binary file not shown.

data/2017-old/12_BenTre_DKWF.xls.xlsx

BIN

Binary file not shown.

data/2017-old/12_LongAn_ZZUK.xls.xlsx

BIN

Binary file not shown.

data/2017-old/13_NamDinh_ESEL.xls.xlsx

BIN

Binary file not shown.

data/2017-old/13_TraVinh_LKUJ.xls.xlsx

BIN

Binary file not shown.

data/2017-old/14_NgheAn_BSLY.xls.xlsx

BIN

Binary file not shown.

data/2017-old/15_PhuTho_ABWQ.xls.xlsx

BIN

Binary file not shown.

data/2017-old/16_QuangBinh_KGEU.xls.xlsx

BIN

Binary file not shown.

data/2017-old/17_QuangNam_AMTK.xls.xlsx

BIN

Binary file not shown.

data/2017-old/18_QuangNgai_KOFP.xls.xlsx

BIN

Binary file not shown.

data/2017-old/19_QuangTri_OMZF.xls.xlsx

BIN

Binary file not shown.

data/2017-old/1_BaRia_VungTau_HJKG.xls.xlsx

BIN

Binary file not shown.

data/2017-old/1_Da_Nang_AHWJ.xls.xlsx

BIN

Binary file not shown.

data/2017-old/1_Ha_Noi_CVXG.xls.xlsx

BIN

Binary file not shown.

data/2017-old/1_Son_La_JIDP.xls.xlsx

BIN

Binary file not shown.

data/2017-old/1_TuyenQuang_JBYF.xls.xlsx

BIN

Binary file not shown.

data/2017-old/20_TayNinh_ILFA.xls.xlsx

BIN

Binary file not shown.

data/2017-old/21_ThaiBinh_FTVG.xls.xlsx

BIN

Binary file not shown.

data/2017-old/22_ThaiNguyen_TLTW.xls.xlsx

BIN

Binary file not shown.

data/2017-old/23_HaiPhong_HXBV.xls.xlsx

BIN

Binary file not shown.

data/2017-old/24_HCM_XULN.xls.xlsx

BIN

Binary file not shown.

data/2017-old/2_BacKan_GFVQ.xls.xlsx

BIN

Binary file not shown.

data/2017-old/2_Ha_Giang_QNCM.xls.xlsx

BIN

Binary file not shown.

data/2017-old/2_Ninh_Thuan_VHLY.xls.xlsx

BIN

Binary file not shown.

data/2017-old/2_Thanh_Hoa_AUFV.xls.xlsx

BIN

Binary file not shown.

data/2017-old/2_VinhPhuc_GUDK.xls.xlsx

BIN

Binary file not shown.

data/2017-old/3_BacGiang_TOIF.xls.xlsx

BIN

Binary file not shown.

data/2017-old/3_BinhPhuoc_YFMU.xls.xlsx

BIN

Binary file not shown.

data/2017-old/3_Cao_Bang_CIEY.xls.xlsx

BIN

Binary file not shown.

data/2017-old/3_Dong_Thap_GSXQ.xls.xlsx

BIN

Binary file not shown.

data/2017-old/3_Thua_Thien_Hue_HMDB.xls.xlsx

BIN

Binary file not shown.

data/2017-old/4_An_Giang_JNOS.xls.xlsx

BIN

Binary file not shown.

data/2017-old/4_BacNinh_STLR.xls.xlsx

BIN

Binary file not shown.

data/2017-old/4_Binh_Dinh_WWZW.xls.xlsx

BIN

Binary file not shown.

data/2017-old/4_DienBien_SOJG.xls.xlsx

BIN

Binary file not shown.

data/2017-old/4_Lang_Son_NYQL.xls.xlsx

BIN

Binary file not shown.

data/2017-old/5_Bac_Lieu_XEKH.xls.xlsx

BIN

Binary file not shown.

data/2017-old/5_Gia_Lai_ABZI.xls.xlsx

BIN

Binary file not shown.

data/2017-old/5_HaiDuong_WWWG.xls.xlsx

BIN

Binary file not shown.

data/2017-old/5_Hanam_QGJS.xls.xlsx

BIN

Binary file not shown.

data/2017-old/5_Yen_Bai_FAQR.xls.xlsx

BIN

Binary file not shown.

data/2017-old/6_Dong_Nai_WOTM.xls.xlsx

BIN

Binary file not shown.

data/2017-old/6_Hau_Giang_KWDM.xls.xlsx

BIN

Binary file not shown.

data/2017-old/6_HoaBinh_TPZY.xls.xlsx

BIN

Binary file not shown.

data/2017-old/6_NinhBinh_IGFT.xls.xlsx

BIN

Binary file not shown.

data/2017-old/6_Quang_Ninh_DQCJ.xls.xlsx

BIN

Binary file not shown.

data/2017-old/7_HaTinh_DDHD.xls.xlsx

BIN

Binary file not shown.

data/2017-old/7_HungYen_LTIK.xls.xlsx

BIN

Binary file not shown.

data/2017-old/7_Kon_Tum_RSLR.xls.xlsx

BIN

Binary file not shown.

data/2017-old/7_Tien_Giang_EFHX.xls.xlsx

BIN

Binary file not shown.

data/2017-old/8_Can_Tho_RQZM.xls.xlsx

BIN

Binary file not shown.

data/2017-old/8_Dak_Lak_YKPR.xls.xlsx

BIN

Binary file not shown.

data/2017-old/8_KienGiang_OTOB.xls.xlsx

BIN

Binary file not shown.

data/2017-old/8_PhuYen_OGIM.xls.xlsx

BIN

Binary file not shown.

data/2017-old/9_DakNong_FBOP.xls.xlsx

BIN

Binary file not shown.

data/2017-old/9_Khanh_Hoa_KPKQ.xls.xlsx

BIN

Binary file not shown.

data/2017-old/9_LaiChau_ALKN.xls.xlsx

BIN

Binary file not shown.

data/2017-old/9_Vinh_Long_OLWA.xls.xlsx

BIN

Binary file not shown.

data/2017-old2/1.BaRia-VungTau_PGZT.xlsx

BIN

Binary file not shown.

data/2017-old2/1.Da Nang_ABWU.xlsx

BIN

Binary file not shown.

data/2017-old2/1.Son La_XLFN.xlsx

BIN

Binary file not shown.

data/2017-old2/1.TuyenQuang_PTMR.xlsx

BIN

Binary file not shown.

data/2017-old2/10.BinhThuan_MVVG.xlsx

BIN

Binary file not shown.

data/2017-old2/10.LamDong_YUQA.xlsx

BIN

Binary file not shown.

data/2017-old2/10.Soc Trang_LQWU.xlsx

BIN

Binary file not shown.

data/2017-old2/11.BinhDuong_HVAH.xlsx

BIN

Binary file not shown.

data/2017-old2/11.LaoCai_ZTBP.xlsx

BIN

Binary file not shown.

data/2017-old2/12.BenTre_NQTU.xlsx

BIN

Binary file not shown.

data/2017-old2/12.LongAn_PDRH.xlsx

BIN

Binary file not shown.

data/2017-old2/13.NamDinh_NAYR.xlsx

BIN

Binary file not shown.

data/2017-old2/13.TraVinh_FODZ.xlsx

BIN

Binary file not shown.

data/2017-old2/14.NgheAn_HTKD.xlsx

BIN

Binary file not shown.

data/2017-old2/15.PhuTho_IJZW.xlsx

BIN

Binary file not shown.

data/2017-old2/17.QuangNam_NQMG.xlsx

BIN

Binary file not shown.

data/2017-old2/18.QuangNgai_IUPY.xlsx

BIN

Binary file not shown.

data/2017-old2/19.QuangTri_MKNN.xlsx

BIN

Binary file not shown.

data/2017-old2/2.BacKan_YQNX.xlsx

BIN

Binary file not shown.

data/2017-old2/2.Ha Giang_PIYK.xlsx

BIN

Binary file not shown.

data/2017-old2/2.Ninh Thuan_BAGG.xlsx

BIN

Binary file not shown.

data/2017-old2/2.Thanh Hoa_UOPE.xlsx

BIN

Binary file not shown.

data/2017-old2/2.VinhPhuc_QZJK.xlsx

BIN

Binary file not shown.

data/2017-old2/20.TayNinh_KJAQ.xlsx

BIN

Binary file not shown.

data/2017-old2/21.ThaiBinh_KTQN.xlsx

BIN

Binary file not shown.

data/2017-old2/22.ThaiNguyen_BKIF.xlsx

BIN

Binary file not shown.

data/2017-old2/24.HCM_UTLQ.xlsx

BIN

Binary file not shown.

data/2017-old2/3.BacGiang_SAVS.xlsx

BIN

Binary file not shown.

data/2017-old2/3.BinhPhuoc_IPHL.xlsx

BIN

Binary file not shown.

data/2017-old2/3.Cao Bang_WMUU.xlsx

BIN

Binary file not shown.

data/2017-old2/3.Dong Thap_HKJX.xlsx

BIN

Binary file not shown.

data/2017-old2/3.Thua Thien -Hue_MAET.xlsx

BIN

Binary file not shown.

data/2017-old2/4.An Giang_PMJD.xlsx

BIN

Binary file not shown.

data/2017-old2/4.BacNinh_NNIS.xlsx

BIN

Binary file not shown.

data/2017-old2/4.Binh Dinh_VOMJ.xlsx

BIN

Binary file not shown.

data/2017-old2/4.DienBien_FYGN.xlsx

BIN

Binary file not shown.

data/2017-old2/4.Lang Son_QWOG.xlsx

BIN

Binary file not shown.

Loaded 100 of 163 files, more files were not shown because too many files have changed in this diff. Show more