When someone asks ChatGPT, Gemini, Claude or Perplexity for "a software agency in London" or "the best accountant for startups", the answer is assembled from what the assistant can find and believe. You cannot buy your way into it. You can make yourself much easier to include.
1. Let the crawlers in
Each assistant reads the web through its own crawler, and the major ones say they respect your
robots.txt. A blanket block, often added by a security plug-in or a cautious hosting
default, takes you out of their view entirely.
- Check
/robots.txtfor rules that disallow agents such as GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot or Google-Extended. - Check the site actually loads for a crawler: no redirect loops, no challenge pages, no login wall in front of public content.
Blocking is a legitimate choice for some publishers. It should be a choice, not an accident.
2. Make the content readable without JavaScript
Many AI crawlers fetch the HTML and do not run scripts. If your page assembles its text in the browser — a common pattern on JavaScript-heavy sites — the crawler may receive an empty shell. View your page's source (not the inspector) and check the words a customer reads are actually there.
3. Write answers that survive being quoted
Assistants lift sentences and paragraphs, not whole pages. A paragraph that starts "We've been doing this for years" means nothing out of context — who is "we"? Write the important answers so each one stands alone:
- Name the subject: "Codexlava builds custom software for UK startups", not "We build software".
- Answer the question in the first sentence, then add detail.
- Use the words your customers use. If they ask "how much does an app cost", say "cost", not "investment".
- Put real questions and answers on the page — an FAQ built from questions people actually ask.
4. Mark up the facts
Structured data (schema.org markup in JSON-LD) states facts in a form machines do not have to interpret: your organisation, address, phone, services and FAQs. Two rules matter more than coverage:
- Only mark up what is visible. Schema describing content a visitor cannot see is cloaking, and search engines treat it as such.
- Match the page exactly. If the markup says one phone number and the footer says another, you have taught the machine that your facts are unreliable.
5. Be consistent everywhere else
Assistants cross-check. Your business is described on directories, data aggregators, Companies House, your Google Business Profile, review sites and social profiles. Where those disagree on your name, address, phone or what you do, an assistant has less reason to say anything confident.
- Search your business name and list every profile that describes you.
- Correct the ones that are wrong — an old address, the wrong industry, an out-of-date phone number.
- Claim the profiles you do not control yet, starting with Google Business Profile.
6. Earn mentions elsewhere
Assistants weigh what others say about you, not just what you say about yourself: industry directories, review platforms, press coverage, partner pages. This is the slowest part and the one most like traditional SEO. There is no shortcut, and anyone selling one is selling something else.
Where llms.txt fits
A small, cheap extra. It summarises your site for AI tools in plain Markdown, but Google says its search does not need it, and no assistant has confirmed reading it on other people's sites. Do the five steps above first. Our explainer covers what llms.txt is and when it helps.
What nobody can promise
No one can guarantee that an assistant will mention you, cite you, or rank you in a particular position. Assistants do not sell placements, and their answers change between questions, users and model versions. Anyone promising otherwise is describing something they do not control. What can be done is removing every technical and factual reason to leave you out.