Skip to content

Can AI actually reach your site?

See which AI crawlers your robots.txt allows, which it blocks, and whether you have an llms.txt.

  • Free
  • No signup
  • Fetches the live page

Ready for ChatGPT, Geminiand Perplexity?

01

Separates AI search bots (which decide if you can be cited) from AI training bots (which only affect model training).

02

Flags when Googlebot or Bingbot are blocked, which also removes you from AI Overviews and Copilot.

03

Checks for an llms.txt file, an emerging convention for guiding AI assistants.

How it works

  1. 01

    Search bots vs training bots

    Blocking GPTBot or ClaudeBot opts you out of model training. Blocking OAI-SearchBot, Claude-SearchBot or PerplexityBot removes you from their AI search answers. Most sites want to allow the second group.

  2. 02

    Where the data comes from

    We fetch your live /robots.txt and /llms.txt and evaluate each published AI user-agent against your rules, using the same longest-match logic crawlers use.

  3. 03

    A voluntary standard

    robots.txt declares your intent. Reputable crawlers follow it, but it is not access control. This tool reports what you have declared.

Questions, answered

That is a business decision about training. It does not affect ChatGPT search citations, which use OAI-SearchBot.