tiennm99/open-domains
1
0 Fork 0
mirror of https://github.com/tiennm99/open-domains.git synced 2026-10-11 12:19:07 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
fda164e41f0dee1fec4a9fdb5cd7fa175166a0b7
open-domains/web/robots.txt
T
andrewstech fda164e41f Add files via upload
2022-12-01 18:14:48 +00:00

5 lines
51 B
Plaintext
Raw Blame History

User-agent: ia_archiver
Disallow: /
User-agent: *
Reference in new issue View Git Blame Copy Permalink