2 Commits
Author SHA1 Message Date
tiennm99 aea29826c2 fix: detect truncated uploads, and stop reporting unreachable work as retryable
Verification matched on name and non-zero size, so an upload that died partway
was counted archived permanently. Comparing against the size Telegram reports
found six such objects in the live archive, one of them 221 MiB standing in for
a 2006 MiB video. They are folded into the outstanding set; rclone overwrites a
size mismatch, so another pass repairs them.

A zero Report claimed the archive was complete — nothing expected, nothing
missing — which is the value both commands hold before their Telegram callback
populates it, so any early return printed COMPLETE and exited 0 on an untouched
chat. A Report now knows whether it ran.

A name that can never be written kept the run outstanding forever while the
download set deliberately excluded it, so a driver looping on "incomplete"
walked the whole history and re-indexed the whole remote every pass for work
that could not be done. Such a run now reports STALLED and exits 4.

A destination that stopped accepting uploads was reported and then discarded,
exiting 1. The same driver would retry against a full or unreachable remote
indefinitely, downloading gigabytes each pass to upload none. It exits 3.

Free space is re-checked during the run, not only before it. An archive this
size runs for hours, and the remote can fill in the middle; discovering it
through five failed multi-gigabyte uploads wastes the download for all of them.

Also: parseSize silently wrapped to a negative or zero on a large input, which
reads downstream as "no cap"; list printed attacker-chosen filenames raw, so a
tab shifted the columns and an escape sequence reached the terminal; a missing
backend blamed credentials rather than the build; humanBytes indexed past its
unit table above 1 PiB; a second Init reported success against a config that
never loaded; and sync did not surface the basename collisions verify warned
about, though sync is the command that acts on the verdict.

The env-override test could not observe what it claimed: rclone reads RCLONE_*
at package init, so t.Setenv came too late and the assertion held with the
guard removed. It runs in a subprocess now, as does the new one covering the
index against inherited filters.
2026-09-06 20:44:08 +07:00
tiennm99 1393abdae2 feat: index a remote and verify a chat against it in-process
Third slice: verify-export.sh, without the subprocess or the python.

One rclone listing builds an in-memory index, and the report is computed from
it. missing-ids.txt and gap.json are gone; so is every python3 heredoc.

Presence is answered from a whole name and never from a message id. The
id-keyed map exists only to tell "absent" apart from "absent, but a stale copy
under an older name is sitting there", and it stays unexported so nothing can
reach for it as an answer. That distinction is the bug this rewrite exists to
remove, so it is enforced by structure rather than by comment.

Names are checked for path containment before use. Storing them verbatim means
a filename chosen by whoever uploaded the file can contain a separator or a
parent reference, and tdl never had to care because its template rewrote those
away. Over-long names are refused for the same reason: the filesystem would
reject them at create time, and a file that can never be written would be
reported absent on every pass forever.

Objects are addressed by the path rclone knows them by, not by the basename
used for matching. The two differ once a remote has directory structure, and
deleting by basename would miss the object or remove a same-named one from the
root. Basenames appearing at more than one path make the snapshot ambiguous, so
they are reported rather than silently resolved.

Indexing runs at full depth with filters cleared. Inheriting RCLONE_MAX_DEPTH
or RCLONE_EXCLUDE would not fail, it would quietly report archived files as
absent and fetch them all again.

Deleting stale copies stays opt-in and confirmed; a non-interactive stdin
declines rather than proceeding. Filenames are quoted wherever they are
printed, so an embedded escape cannot redraw the list an operator approves.

Verified against the live remote: identical to verify-export.sh on the same
state — 18155 expected, 15548 present, 2607 absent, ids 9857-18013, exit 1.
2026-09-06 18:50:53 +07:00