Type a domain and it reads that site's robots.txt the way a crawler does, then tells you which AI bots are allowed in and quotes the line that decides it.
Two files, both of them yours. It reads robots.txt at the root of the domain and works out, for each AI crawler in turn, whether that file lets it in. Then it reads the home page and looks for a noindex instruction, in the meta tag and in the HTTP header, and measures how much readable text is in the HTML before any JavaScript runs.
Every verdict quotes the line that produced it. If there is no line to quote, it says the crawler is not mentioned, which is a real answer too: an unmentioned crawler is an allowed one.
Whether your site is in an AI's index or its training data. OpenAI, Anthropic, Google and Perplexity all publish which user agents they crawl with, and none of them publish what they kept. There is no lookup, no API and no back door, so any tool that hands you a confident yes or no on that question is not reading anything.
What you can check is whether you are letting them in, and that is the half you control. A blocked crawler is a certainty; an allowed one is a chance.
Most sites that turn AI crawlers away never decided to. A Disallow: / under User-agent: * written years ago for a staging site, a security plugin with an AI-blocking toggle switched on by default, a CDN rule somebody enabled and forgot: the file says no and nobody in the building knows it does.
Which is why this quotes your own file back at you rather than summarising it. If the answer is not the one you wanted, the line to change is right there in the result.
Three things, all of them read from your own site. It fetches the robots.txt at the root of your domain and works out, for each AI crawler in turn, whether that file lets it in, quoting the line responsible. It fetches your home page and looks for a noindex instruction, both in the meta robots tag and in the X-Robots-Tag header. And it measures how much readable text is in that page's HTML before any JavaScript runs, because most AI crawlers do not run any.
No, and neither can anything else. OpenAI, Anthropic, Google and Perplexity all publish the user agents they crawl with, and none of them publish what they kept. There is no lookup and no API for it. A tool that answers that question with a confident yes or no is not reading anything, it is guessing. What can be established is whether you are letting them in, which is the half you control.
OpenAI uses GPTBot for training data, OAI-SearchBot for its search index and ChatGPT-User when someone asks about a specific page. Anthropic uses ClaudeBot, Claude-SearchBot and Claude-User for the same three jobs. Google uses Google-Extended for AI alongside ordinary Googlebot, Perplexity uses PerplexityBot, and Apple, Amazon, Meta, ByteDance and Common Crawl each have their own. Thirteen are checked here. Each is named separately in robots.txt, so allowing one does not allow the rest.
Not necessarily. Googlebot and the AI crawlers are separate, and a site can rank perfectly well in search while being blocked to every one of them.
No. Google-Extended governs AI training and Gemini answers only. Ordinary Googlebot crawling and your position in search results are unaffected by it.
Add an Allow rule for each user agent in the robots.txt at the root of your domain. Crawlers re-read that file on their own schedule, usually within a day, so a change is not instant.
Being crawlable is the first step, not the last. Pages get cited when they answer a question directly, in plain language, with the answer near the top rather than buried under a preamble. Clear headings that match real questions do more than any amount of keyword work.