Skip to content
Start a Project

Webot & the Bots

Googlebot, Bingbot, ChatGPT, Claude — Webot knows what every bot likes, and keeps this site a place they are happy to visit.

Friend-check running — see live status below.

Run the friend-check

The friend group

Bots Webot calls friends.

Every bot wants something slightly different. Here is what each one looks for — and what Webot does about it, on every page, every day.

Googlebot

Google Search

  • sitemap
  • canonical
  • schema

What it loves

  • A valid robots.txt with a sitemap pointer
  • One clear canonical URL per page
  • Fast pages and honest meta tags

What Webot does for it

  • Serves robots.txt with the sitemap linked
  • Prints exactly one canonical per page
  • Ships structured data and clean headings

Bingbot

Bing / Copilot

  • crawl-clean
  • unique-meta
  • no-cloaking

What it loves

  • Crawlable pages without dead ends
  • Descriptive titles and descriptions
  • The same respect Google gets

What Webot does for it

  • Keeps every route crawlable, 404s honest
  • Writes unique titles and metas per page
  • No cloaking — bots see what people see

DuckDuckBot

DuckDuckGo

  • tracker-light
  • semantic-dom
  • fast-html

What it loves

  • Privacy-respecting, tracking-light pages
  • Clean HTML it can parse quickly

What Webot does for it

  • No trackers beyond the analytics you opt into
  • Semantic, dependency-free markup

GPTBot

OpenAI / ChatGPT

  • llms.txt
  • robots-allow
  • plain-markdown

What it loves

  • An llms.txt that orients it instantly
  • Plain-text summaries over heavy markup
  • Permission signals it can respect

What Webot does for it

  • Publishes llms.txt and llms-full.txt
  • Serves page summaries in plain markdown
  • Explicitly welcomes it in robots.txt

ClaudeBot

Anthropic / Claude

  • clean-h1-h3
  • no-fluff
  • human-prose

What it loves

  • Well-structured pages with real headings
  • Honest content, no fluff to wade through

What Webot does for it

  • Keeps a clean heading hierarchy everywhere
  • Writes like a human — because bots read it too

PerplexityBot

Perplexity

  • faq-ready
  • live-pricing
  • fact-first

What it loves

  • Answer-shaped content: prices, FAQs, facts
  • A sitemap that lists what changed

What Webot does for it

  • Answers common questions right on the page
  • Keeps pricing and services machine-readable

Live status

Is this site bot-friendly right now?

Webot checks its own public promises — not marketing claims. These are live lookups of this site's robots.txt, sitemap, llms.txt and page head, cached for an hour.

webot — friend-check --live

robots.txt is served ✓ PASS

Points every bot to the sitemap

XML sitemap is live ✓ PASS

Googlebot, Bingbot and friends can map the site

llms.txt is published ✓ PASS

AI bots get a curated tour of the site

One canonical per page ✓ PASS

No duplicate-signal confusion

Social share card ready ✓ PASS

Links preview with the brand image

WebMCP tools available ✓ PASS

Agent browsers can ask the site directly

> All promises verified. The bots are happy.

PUBLIC FILE robots.txt Who is welcome, in plain text PUBLIC FILE sitemap.xml Every page worth crawling PUBLIC FILE llms.txt A guided tour for AI assistants

Being friends with bots is not a trick. It is the same work that makes a site fast and honest for people: clear structure, quick pages, no clutter, and explicit invitations like robots.txt and llms.txt. If you can read this page easily, so can Googlebot — and ChatGPT.

See the work the bots rank

Got a website in mind?Let’s build it.

Start Your ProjectTalk to Webot