GeneratorsFree

robots.txt AI Generator

Choose which AI crawlers may access your site and get a ready robots.txt file.

Input

AI crawlers

Google-Extended and Applebot-Extended only control use of your content for AI training. They do not change normal search crawling.

robots.txt

# robots.txt generated by Techorizone
# Site: https://your-site.com

User-agent: *
Allow: /

# AI crawlers
User-agent: GPTBot
Allow: /

User-agent: OAI-SearchBot
Allow: /

User-agent: ChatGPT-User
Allow: /

User-agent: ClaudeBot
Allow: /

User-agent: Claude-SearchBot
Allow: /

User-agent: Claude-User
Allow: /

User-agent: PerplexityBot
Allow: /

User-agent: Perplexity-User
Allow: /

User-agent: Google-Extended
Allow: /

User-agent: Applebot-Extended
Allow: /

User-agent: CCBot
Allow: /

User-agent: Bytespider
Allow: /

User-agent: Amazonbot
Allow: /

User-agent: meta-externalagent
Allow: /

Sitemap: https://your-site.com/sitemap.xml

Related Generators tools

About robots.txt AI Generator

Choose which AI crawlers may access your site and get a ready robots.txt file.

How to use

  1. Fill in the form.
  2. Watch the preview update as you type.
  3. Copy the output or download the file.

Good to know

This tool runs in your browser. What you enter is not sent to our server.