NAVINESSEO LAB
Menu
LanguageEnglish

NAVINES SEO LAB

Robots.txt builder and rule tester

Generate deterministic groups and explain the winning rule for a crawler and path.

Robots.txt builder and rule tester

Generate deterministic groups and explain the winning rule for a crawler and path.

Enter robots rules and a test path
robots.txt

robots.txt ≤ 500 KiB · Allow / Disallow ≤ 10,000

Runs in the browser unless you explicitly load a public robots.txt file.

Robots rules control crawling, not authorization, and do not guarantee deindexing.

Method

Generate deterministic groups and explain the winning rule for a crawler and path.

Privacy

Runs in the browser unless you explicitly load a public robots.txt file.

Limitations

Robots rules control crawling, not authorization, and do not guarantee deindexing.

Primary sources