What Is Google-CloudVertexBot? The Vertex AI Crawler and robots.txt
Google Cloud's Vertex AI Search crawler, explained: what it fetches, why it hit your logs, and how to write the robots.txt rule.
Topic
AI SEO
A collection of 11 issues
How to Add Google's Preferred Sources Button to Your Site
Google's new interactive preferred sources button ships as two lines of HTML. Where it goes, what the data-theme and data-lang attributes do, who qualifies, and what the preferred badge actually changes in Top Stories, AI Mode, and AI Overviews.
llms.txt v2 (August 2026): how to pass Lighthouse's llms.txt audit
llms.txt v2 (modified August 10, 2026) adds link-relation discovery and blesses both .md URL forms. Lighthouse's agentic-browsing audit fails the file only on a server error - a 404 is Not Applicable.
Cloudflare AI bot policies can block ChatGPT and Perplexity citations
Cloudflare's Search and Agent presets catch OAI-SearchBot, PerplexityBot, and Perplexity-User. The fix, plus the September 15, 2026 defaults deadline, in one pass.
Why Is Your Site Not Showing in Google AI Overviews?
A site can rank in regular Search yet stay absent from AI Overviews. The cause is almost always one of five eligibility blockers, and one is a Search Console toggle most guides never mention.
llms.txt across the top 1,000 SaaS sites in May 2026: who ships, what's inside, what changed
About 10% of sites ship llms.txt. Developer docs and AI tooling lead adoption by a wide margin. But the bots that drive AI citations almost never read the file - its real consumers are IDE agents and MCP servers.
FAQPage Schema: The Highest-ROI Signal for AI Overviews
The FAQ rich result is gone, but the markup is now a higher-ROI answer-engine citation signal. What changed, and what to keep.
AI search invisibility: 7 reasons your site isn't cited (and how to fix each)
Most 'not cited by ChatGPT' diagnoses oversell the robots.txt fix. The seven real blockers cluster into access, extraction, and authority - here's the diagnostic for each.
How to get cited by ChatGPT, Perplexity, and Claude in 2026
Per-bot robots.txt rules, answer-extractable page structure, and JSON-LD schema - the three layers that actually move ChatGPT, Perplexity, and Claude citations in 2026.
Launching Citation Intelligence MCP: free, self-hosted AI citation tracking
Free, self-hosted MCP server that reports which URLs ChatGPT, Claude, Perplexity, Google AI Overviews, Gemini, and Bing cite for any query. 12 tools, BYO API keys, no backend.