AI agentsNovember 7, 20269 min read

Is your website ready for AI agents? A 2026 checklist

AI agents now read, compare and fill out forms on business websites for buyers. A 2026 checklist to make sure ChatGPT, Claude and Gemini can finish the job.

GDBy the GrowDesk team

Your website is ready for AI agents if software can read it without running JavaScript, find one clearly labeled next step, fill out your form without guessing, and get through without hitting a CAPTCHA. Agents in ChatGPT, Claude, Gemini in Chrome and Perplexity's Comet now do this for buyers. Add structured data for prices, hours and contact details, and make sure your robots.txt and firewall let them in.

Key takeaways

AI agents don't just read your site. They click, compare and fill out forms for a person. If a step breaks, the agent moves on to a competitor's site.
The basics matter most: content in the HTML, buttons with plain labels, form fields with real labels, and no CAPTCHA on the main contact or quote form.
Bot protection can turn agents away without anyone noticing. Cloudflare now treats "Agent" traffic as its own category, and since September 15, 2026, some settings block it on pages that show ads.
Structured data (schema.org) gives agents a clean, unambiguous copy of your prices, hours, address and phone number.
Agent commerce protocols (OpenAI's ACP, Google's UCP, Chrome's WebMCP) are real but early. Most small businesses should fix the basics first.

Which AI agents visit business websites today?

An AI agent is an assistant that acts on the web for someone. Ask it to "book a furnace tune-up under $150 near Dayton this week," and it opens sites, compares them and fills in the booking form. The main ones as of November 2026:

Agent	Company	How it reaches your site	Status, November 2026
ChatGPT (ChatGPT Work)	OpenAI	OpenAI's cloud browser, which signs its requests, plus a Chrome extension	Launched July 9, 2026; the Atlas browser was retired in August
Claude in Chrome	Anthropic	The person's own Chrome: reads, clicks, types, fills forms	On all paid Claude plans since August 26, 2026
Chrome auto browse	Google	The person's Chrome; Google's servers use a `Google-Agent` fetcher	US preview for Google AI Pro and Ultra
Comet	Perplexity	Perplexity's own browser	Mac, Windows, iOS and Android
AI Mode booking	Google	Booking partners like OpenTable, then hands off to the person	Rolling out by country

Google says auto browse can fill out forms, schedule appointments and fill a cart, pausing before a purchase.

Why should a small business care?

Because the agent picks the site that works. Picture a homeowner asking an assistant to get quotes from three HVAC companies. Company A's quote form loads through a script and labels its fields only with gray placeholder text. Company B has a CAPTCHA. Company C has a plain form with labeled fields and a "Request a quote" button. Company C gets the lead.

The same goes for a manufacturer whose buyers ask an agent to request pricing on 304 stainless sheet, or a dental clinic whose new patients ask an agent to book a cleaning. To get recommended in the first place, see how to get recommended by ChatGPT. This checklist covers what happens after the agent arrives.

Check your site in 20 seconds

See whether AI assistants can read you and AI agents can use you. Free, no account.

The checklist, item by item

1. Can your pages be read without JavaScript?

Browser-based agents run JavaScript, but many AI fetchers don't. A December 2024 Vercel analysis found that none of the major AI crawlers it studied rendered JavaScript, including OpenAI's and Anthropic's. If your services, prices or phone number only appear after a script runs, those fetchers see an empty page.

How to test: in Chrome DevTools, turn off JavaScript and reload your main pages. If the text disappears, ask your developer about server-side rendering.

2. Is there one clear next step on every page?

Agents look for a link or button that matches the task. "Get a quote," "Book an appointment" and "Buy now" are clear. "Learn more" or an icon with no text are not. Use a real link or <button> with visible text, not a clickable image. Keep one primary action per page, near the top and again at the bottom.

3. Are your form fields labeled?

An agent fills in a form by matching each field to what it knows: name, email, ZIP code, square footage. Placeholder text that disappears when you type is not a label. The W3C forms tutorial shows how to connect a <label> to each input, which also helps people using screen readers. Mark required fields in text, not only by color, and make error messages say what to fix ("Enter a 5-digit ZIP code").

4. Are CAPTCHAs and bot blocking turning agents away?

A CAPTCHA is built to stop software, and an agent is software. At best, the agent stops and asks the person to finish. At worst, it gives up and tries the next company. Spam is a real problem, so look at lighter options: a hidden honeypot field, rate limits, or an invisible check like Cloudflare Turnstile (test it with an agent before you rely on it). Keep a hard CAPTCHA only on forms that attract heavy abuse.

The bigger risk is blocking that nobody chose on purpose. In July 2026, Cloudflare split AI traffic into separate controls for Search, Agent and Training, and it changed the defaults on September 15. Its July 2026 announcement says the Agent category includes chat fetchers like ChatGPT-User and browser agents like "Gemini or Claude driving Chrome." According to Cloudflare's September post, sites that had the older "Block AI bots" setting turned on were moved to Agent: Block on pages with ads, and new domains with ads are offered the same preset. If your site shows ads anywhere, check this setting.

You can let trusted agents through without opening the door to every scraper. Web Bot Auth, a standard being developed at the IETF, lets a bot sign its requests so a site can verify the sender. OpenAI's help center says its cloud browser signs requests with a Signature-Agent header set to https://chatgpt.com, which Cloudflare, Akamai, HUMAN and Vercel can verify. Cloudflare's verified bots documentation now lists signed agent services separately.

5. Are your prices, hours and contact details machine-readable?

Structured data removes the guesswork. Add schema.org markup in JSON-LD:

LocalBusiness (or a subtype like HVACBusiness or Dentist): name, address, phone, openingHoursSpecification, service area.
Product with Offer: price, currency, availability. This matters most for manufacturers and stores.

Google's AI features documentation says AI Overviews and AI Mode need no special markup. Structured data is still the clearest way to state facts so software can't misread them, as long as it matches the page. Our schema markup guide has examples. Keep your phone number and address in plain text too, not inside an image.

6. Does your robots.txt let user-triggered agents in?

When a person asks an assistant about your business, it may fetch your page with ChatGPT-User, Claude-User or Perplexity-User. Anthropic says blocking Claude-User prevents retrieval of your content when users ask. OpenAI and Perplexity say their user-triggered fetchers may not follow robots.txt, because a person asked for the page. Either way, don't block them. Our guide to AI crawlers and robots.txt has a recommended file.

What about agent commerce protocols like ACP, UCP and MCP?

Several standards let agents browse catalogs and check out through code instead of clicking through pages:

Protocol	From	What it does	Status, November 2026
Agentic Commerce Protocol (ACP)	OpenAI and Stripe	Product feeds, agentic checkout and delegated payments for ChatGPT	Live. OpenAI pulled back from in-chat Instant Checkout in March 2026 and now focuses ACP on product discovery, with merchants running their own checkout.
Universal Commerce Protocol (UCP)	Google, Shopify, Etsy, Walmart and others	Catalog, cart, checkout and orders; payments through Google's Agent Payments Protocol (AP2)	Announced January 2026; Google says Chrome will support it
WebMCP	Chrome team	Lets a page expose its forms and actions as tools an agent can call	Experimental origin trial from Chrome 149
MCP server	Open standard	Connects AI assistants directly to your data or booking system	Common for software companies, rare for local businesses

If you sell through Shopify or another big platform, check what it already supports. A machine shop or a plumbing company doesn't need any of these today. Readable pages, working forms and correct structured data matter more. An llms.txt file is optional and no substitute for them.

Printable agent-readiness checklist

#	Check	Quick test
1	Key pages readable without JavaScript	Turn off JavaScript and reload
2	One clear next step per page	Name the main button in three seconds
3	Button text says what happens	"Get a quote," not "Submit"
4	Every form field has a real label	Click the label; the cursor jumps to the field
5	No CAPTCHA on the main contact or quote form	Fill out the form in a private window
6	Plain-text errors and confirmation	Submit with a blank field
7	Phone, email and address in text, not images	Select the text with your cursor
8	LocalBusiness schema with hours and contact details	Google's Rich Results Test
9	Product schema with prices, if you sell products	Google's Rich Results Test
10	robots.txt allows ChatGPT-User, Claude-User, Perplexity-User	Open yoursite.com/robots.txt
11	Firewall doesn't block the Agent category	Cloudflare: AI Crawler and Agent access policies
12	Quote or booking flow works end to end	Ask an agent to try it

GrowDesk's free check at growdesk.io tests six of these: JavaScript-free readability, a clear next step, labeled form fields, CAPTCHA, machine-readable contact details and robots.txt blocks on AI agents. To track whether assistants mention you afterward, see how to measure AI visibility.

FAQ

Do AI agents follow robots.txt?

It depends on the company. Anthropic says its bots, including Claude-User, honor robots.txt. OpenAI, Perplexity and Google say their user-triggered fetchers may not, because a person asked for the page.

Will removing my CAPTCHA flood me with spam?

Not usually, if you replace it with something lighter. A hidden honeypot field and rate limits stop most form spam without asking every visitor, human or agent, to solve a puzzle.

Do I need ACP, UCP or an MCP server to work with AI agents?

No. These protocols are early and mostly matter for online stores on big platforms. Agents can already use any site with readable pages, clear buttons and labeled forms.

How do I know if Cloudflare is blocking AI agents on my site?

In the Cloudflare dashboard, open Security settings and find AI Crawler and Agent access policies. Check that Agent is set to allow, then look at AI Crawl Control to see which agents reached your site and which were blocked.