JoliTools

Robots.txt Generator

Build a correct robots.txt visually

Rules

robots.txt

User-agent: *
Disallow: /admin/

Generated entirely in your browser. Nothing is sent or stored anywhere.

About this tool

A malformed robots.txt can accidentally block your entire site from Google. Building it visually removes that risk.

Add allow and disallow rules per user-agent, attach your sitemap, set crawl delay, and download a syntactically correct file ready for your domain root.

How to use the robots.txt generator

  1. 1Add rules: user-agent, allow and disallow paths.
  2. 2Add your sitemap URL and optional crawl delay.
  3. 3Preview the file live.
  4. 4Copy or download robots.txt.

Frequently asked questions

It tells search engine crawlers which parts of your site they may or must not crawl. It is advisory, not access control.

Related tools

More free tools you might find useful.

New

Sitemap Generator

Generate XML sitemaps from a URL list

Developer Tools
New

Meta Tag Generator

Generate SEO meta tags in seconds

Developer Tools
New

HTTP Status Codes

Look up every HTTP status code with plain-English meanings.

Developer Tools
New

Regex Tester

Test regular expressions live with highlighted matches

Developer Tools

Part of Developer Tools on JoliTools, free tools, no sign-up.