Project.gather_source_files walks the tree with os.walk, which hands back
directories and files separately, and then calls is_ignored_path on each
one. That method re-derived file-ness from the filesystem: an
os.path.exists and an os.path.isfile in _is_ignored_relative_path, plus
an os.path.isdir in match_path. Three syscalls per path, for an answer
the caller already had.
is_ignored_path, _is_ignored_relative_path and match_path now accept an
optional hint, and gather_source_files supplies it from the os.walk
split. The parameter defaults to None, which determines file-ness from
the filesystem exactly as before, so no existing caller changes
behaviour.
Measured on a repository with 97,549 tracked source files out of 707,889
total (Unreal Engine source plus three game projects; Windows 11,
Python 3.13):
gather_source_files() 67.8s -> 11.6s
The returned file list is byte-identical before and after (sha256 over
the sorted relative paths, 10,390,077 bytes).
For context on where the time went: a bare os.walk of the whole 708k-file
tree takes 10.7s, so this was never I/O-bound. cProfile over 15,000 real
is_ignored_path calls attributed 3.41s to nt._path_exists,
nt._path_isfile and nt._path_isdir - about 82% of the per-call cost.
The new test asserts equivalence rather than specific verdicts: for every
path in a fixture tree, across several ignore configurations, the hinted
call must agree with the unhinted one. It covers the two cases where
guessing file-ness from the name would go wrong - a directory with a
suffix, and an extensionless file - and was checked against a deliberate
inversion of the hint, which it catches.
Refs #2077
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Dominik Jain <dominik.jain@oraios-ai.de>
