nytimes.com · agent readability

nytimes.com · scanned 22h ago · 5 pages

What each client got, same URL, same second

Three of four AI clients were refused. Chrome was not.

browser

Chrome

200

served

AI client

Claude

403

refused

AI client

GPT

403

refused

AI client

Perplexity

403

refused

AI client

Google

200

served

Your robots.txt also asks Claude, GPT, Perplexity, Google not to come — which is a separate line to change from whatever your edge is doing.

refusedA rule at your edge refuses this user agent. Allow it, or allow the address ranges its vendor publishes.

2 things are costing you the rest

Three agent clients are refused at the edge
When somebody asks an assistant about you, it answers from whatever it can reach. That is not your site.
robots.txt blocks four reading agents and lets search crawlers through
Your own robots.txt is what turns them away. Nothing else on the site gets a chance to be read.

0/ 100

Grade Cscoring v1.5Media · 4 not counted

Three of the Five AI clients can't read this page at all.

Get the fix pack$15

Retrievability

49%

Discovery

38%

Representation

48%

Structure

100%

Actionability

Freshness

100%

109th of 177108 sites read better than yours

+2 points clears 4 sites, starting with kenmorevet.com at 59.See the chart

Already written for nytimes.com

6 of your 9 findings have a file below with your own URLs in it. The rest are a punch list ordered by effort, and a prompt that applies the lot one commit at a time.

By hand: 1 × days, 5 × hours, 3 × minutes
In the pack: 10 files, 900 lines
Points it addresses: all 42

$ claude "apply botready-fixes.md"

Get the fix pack$15

Covers nytimes.com. Another domain later is $5.

Your files open here in the browser the second you pay. We email a copy too, so check your spam folder for it: we are a new domain and filters have not met us yet.

Serve this at https://nytimes.com/llms.txt. It tells a reading agent which of your pages matter, in the order they matter.

# Breaking News, US News, World News and Videos - The New York Times
> Live news, investigations, opinion, photos and video by the journalists of The New York Times from more than 150 countries around the world. Subscribe for coverage of U.S. and international news, politics, business, technology, science, health, arts, sports and more.

## Company
- [The New York Times](https://www.nytimes.com/privacy/manage-settings)

## Other pages
- [The New York Times International - Breaking News, US News, World News, Videos](https://www.nytimes.com/international/): The New York Times seeks the truth and helps people understand the world. With 1,700 journalists reporting from more than 150 countries, we provide live updates, investigations, photos and video of international and regional news, politics, business, technology, science, health, arts, sports and opinion.
- [The New York Times Canada - Breaking News, US News, World News, Videos](https://www.nytimes.com/ca/): The New York Times provides live news, investigations, opinion and video from the United States, Canada and around the world. Our 1,700 journalists report on politics from Washington and Ottawa, global business, technology, science, health, arts and sports from more than 150 countries.
- [Site Map - The New York Times](https://www.nytimes.com/sitemap/)

complete · written for nytimes.compreview

Three of four AI clients were refused. Chrome was not.

2 things are costing you the rest

Already written for nytimes.com

What to fix

Retrievability

Discovery

Representation

Structure

Actionability

How this was measured

Agent readability for nytimes.com: grade C, 58 out of 100

Three of four AI clients were refused. Chrome was not.

2 things are costing you the rest

Already written for nytimes.com

What to fix

Retrievability

Discovery

Representation

Structure

Actionability

How this was measured