Skip to content
Changelog
fixdeveloper-web

robots.txt Validator v1.1 — the URL tester takes full URLs

v1.1

Pasting a full address into the URL tester, such as https://example.com/admin, tested it as the path /https://example.com/admin.

That path matches no rule, so the tester answered Allowed even when the file disallowed /admin. A full URL is now tested by its path and query, the part robots.txt rules apply to. Spaces around a typed path are ignored too.

All changes