· 3 min read
How to get cited by ChatGPT and Perplexity
The three technical prerequisites for appearing in AI answers, with the exact robots.txt lines, an llms.txt template, and how to verify each one is working.
Being cited by an AI engine has a technical floor. Before content quality matters at all, three things have to be true: the engine's crawler can fetch your page, it can read the text without running JavaScript, and it can tell what your site is. This guide covers exactly those three, with copy-paste examples.
Step 1: let the search crawlers in
AI engines use two kinds of bots. Search crawlers fetch pages at answer time so the engine can cite them. Training crawlers collect text to train models. They have different names, and you can treat them differently.
Search crawlers you almost certainly want allowed:
User-agent: OAI-SearchBot
User-agent: ChatGPT-User
User-agent: Claude-SearchBot
User-agent: PerplexityBot
User-agent: Applebot
User-agent: DuckAssistBot
Allow: /
Training crawlers you may block without losing AI search visibility:
User-agent: GPTBot
User-agent: ClaudeBot
User-agent: Google-Extended
User-agent: CCBot
Disallow: /
Two traps. First, a blanket User-agent: * with Disallow: / earlier in the file wins for any bot you did not name explicitly. Check the whole file, not just the section you added. Second, Google's AI Overviews use plain Googlebot; blocking Google-Extended does nothing to them either way.
Verify it: paste your robots.txt into a tester and check the path / for each crawler name above. RankRocket's free robots.txt tester does this per crawler, and the site audit reports every AI bot's verdict in one table.
Step 2: make the text readable without JavaScript
Most AI crawlers fetch HTML and stop. They do not run your framework. If your page is an empty shell that fills in after load, the crawler sees the shell.
Verify it: fetch the page with a plain HTTP client and look for your body text in the response:
curl -s https://example.com/your-page | grep -c "a sentence from the page"
A zero means the text is not in the server response. The fix is server-side rendering or static generation for any page you want cited. This is also why RankRocket's AI readiness score includes a "content readable without JavaScript" check.
Step 3: tell them what you are
llms.txt
Put a Markdown file at /llms.txt. It is a convention rather than a standard, so support varies by engine. But it is cheap, and it is your one chance to describe your site to a model in your own words. A minimal template:
# Your Company
> One sentence on what you do and for whom.
Two or three sentences of plain description: the product, the audience, what is free.
## Product
- [How it works](https://example.com/how-it-works): One line.
- [Pricing](https://example.com/pricing): One line.
## Guides
- [Your best article](https://example.com/blog/best-article): One line.
Keep it short, keep every link real, and regenerate it when pages change. Ours is at rankrocket.co.in/llms.txt and is generated from the same registries as the sitemap so it never drifts.
Entity schema on the homepage
Add Organization (or Person) and WebSite JSON-LD to the homepage, with name, url, logo and sameAs links to real profiles. If your brand name is shared with other businesses (common, and true of ours), this plus a consistent name on every page is what lets an engine tell you apart.
Authorship and dates
Every article should carry a visible author and published date, mirrored in BlogPosting schema with author, datePublished and dateModified. Engines drop undated, anonymous sources first.
Step 4: shape the pages that should be cited
With the floor in place, structure decides who gets quoted. On any page whose job is to answer a question: put the question in a heading, give the complete short answer in the first sentence beneath it, then elaborate. Add a FAQ section with FAQ schema. Keep one H1 and a sensible heading order. That is the whole trick, and it is most of what "AEO" means.
How do I know it worked?
Ask. Take ten questions your customers ask, put them to ChatGPT, Perplexity and Gemini, and note who is named. Repeat monthly. Referrals from chatgpt.com and perplexity.ai in your analytics are the other tell.
If you would rather not do that by hand, RankRocket's AI mention monitoring runs a saved set of prompts through Gemini, ChatGPT and Perplexity and reports mentions and competitors, and every crawl scores the four steps above as an AI readiness score with a fix list.
Frequently asked questions
Will llms.txt get me cited on its own?
No. It helps engines that read it understand you; it does not override crawler access, unreadable content or vague pages. Do steps one and two first.
Should I block GPTBot?
That is a policy choice about training, not a visibility choice. Blocking GPTBot does not affect whether ChatGPT search cites you. That is OAI-SearchBot and ChatGPT-User.
How long until I see mentions?
Engines that fetch live can cite a page within days of it becoming reachable and clear. Being recommended as a brand takes longer, because it depends on the entity signals accumulating across your site and the web.