The real run was dead. Removing Config.max_flood_wait left cli._real_run
still passing it, so every non-dry-run export raised an uncaught
TypeError after taking both locks and writing title.txt and the state
file, but before downloading anything. Dry-run was unaffected, which is
why it looked healthy.
That escaped 180 passing tests because every one of them constructed
Config by hand and none invoked main(). test_cli_dispatch.py now drives
the whole dispatch against a fake client - real run, resume, dry run,
--limit 0, exit 7, exit 3, and an unexpected exception still honoring
the exit contract. That test file, not the one-line fix, is the remedy.
Also:
- AuthKeyNotFound subclasses plain Exception and matched no clause, yet
it is what actually arrives when the connection drops mid-download:
MTProtoSender sets it on every in-flight request. It now aborts with
exit 4 like its rarer RPC-level sibling.
- CdnFileTamperedError was swallowed as a network error and retried
three times. A hash mismatch on CDN-served bytes is a trust event, not
transient noise, so it gets its own clause and its own message.
- Unicode Cf/Zl/Zp characters survived filename sanitization. U+202E
RIGHT-TO-LEFT OVERRIDE renders 'a<RLO>gpj.exe' as 'a.exe.jpg' in any
terminal or file manager - the ANSI vector was closed while the older
extension-spoofing one stayed open. Stripping by Unicode category
subsumes the previous control-character table.
- A run where every file failed and none succeeded no longer sets
completed_at. A dead session looks exactly like that from inside the
loop, and the field is supposed to answer 'did my export finish?'
without guessing.
- Network errors reaching main() are no longer labelled 'filesystem',
and a catch-all keeps any unexpected exception inside the exit-code
contract.
- The retry ladder no longer sleeps after its final attempt.
Verified unchanged: filenames for ordinary extensions, so no previously
downloaded file is orphaned on resume.
Phases 2-6 complete, phase 1 partial, phase 7 pending.
Records what was settled without live credentials: offset_id is an
exclusive lower bound under reverse=True, read from Telethon's iterator
rather than inferred from one sample, so the cursor stores the last
handled id. Also logs the defects found along the way and the two
critical findings from the post-implementation review, plus the one
finding left unresolved rather than decided unilaterally - whether a
file that exhausted its retries should stay abandoned behind a set
completed_at.
Covers why the Bot API structurally cannot do this, what .env may and
may not hold, why the export directory is a chat id, and the sidecar's
union-over-records rule - reading it as last-write-wins reports a
narrower file list than what is on disk after a filter change.
Also states plainly that flood waits have no ceiling by default: the
tool sleeps as long as Telegram demands and logs the computed wake time,
so a long sleep reads as a sleep rather than a hang.
180 tests, all offline against synthetic message stubs: no credentials,
no network, no live group.
The cases that matter are the ones where 'looks like success' and 'is
success' diverge - a cursor ahead of what is on disk, a split album, a
zero-byte file renamed to its target, an empty filter set serialized as
no filter at all.
Two guards are tested for their ability to fail rather than assumed:
the AST scan that forbids parallelism primitives is checked against a
sample that must trip it, and lock contention is proven with a real
second process, since two acquisitions in one process would succeed and
prove nothing.
Exports all media from a group to disk, one folder per logical post with
albums collapsed, alongside a messages.jsonl sidecar linking every file
to its message.
Three properties the design is built around:
- Identity is filter-invariant. A post's folder is the lowest message id
in the full grouped_id group and filenames key on message id, so
--types photo and --types video address the same files. A positional
index would shift whenever a member message was deleted.
- The cursor commits only after a complete post and is unreachable from
every error path, so it can never point past in-flight work.
- Filters are stored verbatim; a mismatch refuses the run and names what
changed, because the cursor means 'handled through here under these
filters'.
Exactly one untrusted string becomes a path component: the filename
leaf, sanitized stem and extension both. Every other component is an int
enforced by type. The export directory is the chat id rather than the
group title, which makes traversal via a renamed group structurally
impossible and stops a mid-export rename from orphaning the download.
Downloads are sequential. Telethon does not parallelize transfers, and
concurrency mainly accelerates flood-wait escalation from seconds to
hours.
Telethon 2.0 is alpha with a renamed API surface, hence the <2 bound.
requirements.txt records 1.44.0, the version the API probes were run
against, so a later resolver bump cannot silently change the semantics
the traversal depends on.
tiennm99