Commerce discovery files in the shared crawl
Of 178 commerce-category domains we could probe in the crawl of 4 Nov 2026, 15 served a valid UCP profile at /.well-known/ucp. Across the top 10,000 the figure is 47 of 7,401. The UCP row comes from a follow-up pass that did not check robots.txt per path, and answers of 403 or 429 are counted, not dropped.
Results
E-commerce and Commerce Tools category domains
| Probe path | Robots.txt checked per path | Domains probed | Answered 403 or 429 | Status 200 | JSON | Valid file | Share valid |
|---|---|---|---|---|---|---|---|
| /.well-known/ucp | no, follow-up pass | 178 | 6 | 35 | 15 | 15 | 8.43 percent |
| /.well-known/ucp.json | yes | 177 | 7 | 31 | 13 | 13 | 7.34 percent |
| /.well-known/acp.json | no, follow-up pass | 178 | 6 | 19 | 1 | 1 | 0.56 percent |
| /.well-known/x402 | no, follow-up pass | 178 | 6 | 20 | 0 | 0 | 0 percent |
| /.well-known/ard.json | yes | 177 | 7 | 19 | 1 | 1 | 0.56 percent |
| /openapi.json | yes | 176 | 8 | 24 | 5 | 5 | 2.84 percent |
| /agents.json | yes | 176 | 8 | 19 | 1 | 0 | 0 percent |
| /.well-known/web-bot-auth/directory | no, follow-up pass | 178 | 6 | 21 | 0 | 0 | 0 percent |
Domains with at least one valid file in this group: anakin.io, apparel.monster, chewy.com, cratejoy.com, flaviar.com, flexfits.com, hypeandvice.com, ikea.com, matagora.com, provenskincare.com, riveterhq.com, sellfy.com, shoplocale.com, shoptiques.com, socksoho.com, soteriskin.com, theessential.com, vendah.com.br, zinc.com.
Tranco top 10,000 (all categories)
| Probe path | Robots.txt checked per path | Domains probed | Answered 403 or 429 | Status 200 | JSON | Valid file | Share valid |
|---|---|---|---|---|---|---|---|
| /.well-known/ucp | no, follow-up pass | 7401 | 1264 | 1037 | 51 | 47 | 0.64 percent |
| /.well-known/ucp.json | yes | 7442 | 1255 | 943 | 24 | 18 | 0.24 percent |
| /.well-known/acp.json | no, follow-up pass | 7408 | 1233 | 927 | 5 | 1 | 0.01 percent |
| /.well-known/x402 | no, follow-up pass | 7404 | 1233 | 991 | 4 | 0 | 0 percent |
| /.well-known/ard.json | yes | 7438 | 1268 | 939 | 24 | 18 | 0.24 percent |
| /openapi.json | yes | 7472 | 1125 | 1075 | 24 | 21 | 0.28 percent |
| /agents.json | yes | 7485 | 1120 | 1042 | 7 | 2 | 0.03 percent |
| /.well-known/web-bot-auth/directory | no, follow-up pass | 7409 | 1258 | 995 | 5 | 0 | 0 percent |
Domains with at least one valid file in this group: academy.com, admaster.cc, alchemy.com, amplitude.com, anker.com, appsflyer.com, att.com, august.com, aylanetworks.com, barnesandnoble.com, beehiiv.com, bitdefender.com, bluehost.com, boozt.com, cal.com, chewy.com, chiikawamarket.jp, cloudflare.com, cloudx.io, cm.com, cookieyes.com, daisonet.com, dev.to, discoverpilgrim.com, dmac-solutions.net, dns-finder.com, dpboss.tax, dpbossss.boston, dpbosssss.boston, ecoflow.com, elementor.com, eufy.com, eufylife.com, expiredwixdomain.com, farfetch.com, fineartamerica.com, forter.com, gameanalytics.com, govee.com, guard.io, haveibeenpwned.com, homedepot.com, hootsuite.com, hp.com, ikea.com, linksys.com, mattel.com, mediaexpert.pl, monday.com, monetixads.com, mysterium.network, nanit.com, netlify.app, netlify.com, nic.ai, nishatlinen.com, openlibrary.org, openrouter.ai, padlet.com, paypal.com, paypalobjects.com, pixai.art, pixels.com, pixhost.to, pocketcasts.com, post-gazette.com, preply.com, programmaticx.ai, railway.app, saya.pk, supabase.co, supabase.com, surveymonkey.com, sweetwater.com, theiconic.com.au, thredup.com, tradedoubler.com, ukrainealarm.com, usercentrics.eu, wiki.gg, withings.net, wyze.com, wyzecam.com, zellbury.com.
Payments and Fintech category domains
| Probe path | Robots.txt checked per path | Domains probed | Answered 403 or 429 | Status 200 | JSON | Valid file | Share valid |
|---|---|---|---|---|---|---|---|
| /.well-known/ucp | no, follow-up pass | 399 | 11 | 40 | 4 | 4 | 1 percent |
| /.well-known/ucp.json | yes | 393 | 11 | 33 | 0 | 0 | 0 percent |
| /.well-known/acp.json | no, follow-up pass | 399 | 11 | 34 | 1 | 1 | 0.25 percent |
| /.well-known/x402 | no, follow-up pass | 399 | 11 | 37 | 0 | 0 | 0 percent |
| /.well-known/ard.json | yes | 393 | 11 | 36 | 3 | 3 | 0.76 percent |
| /openapi.json | yes | 395 | 13 | 42 | 3 | 3 | 0.76 percent |
| /agents.json | yes | 395 | 12 | 37 | 1 | 0 | 0 percent |
| /.well-known/web-bot-auth/directory | no, follow-up pass | 399 | 11 | 36 | 0 | 0 | 0 percent |
Domains with at least one valid file in this group: cypherhq.io, fintoc.com, forter.com, getlago.com, itseasy.co, joinforge.app, mercury.com, onefin.in, paypal.com, ramp.com.
The follow-up crawl probed /.well-known/ucp, the path the UCP specification publishes, on the same domains. It held 10508 rows when this slice was cut, and the UCP row above uses it.
How it was built
The shared crawl read 13,071 domains on 4 Nov 2026 with a plain-fetch client and the user agent agent-readiness-crawl/1.0. Its main pass checked robots.txt for every path; the follow-up pass that supplies the UCP, ACP, x402 and web-bot-auth rows did not check robots.txt per path, and the table marks those rows. This slice keeps the domain, category, rank and the answer to each commerce probe. A file counts only when the answer is status 200, parses as JSON and carries the top-level key its format defines. The slicing script is scripts/slice-crawl.mjs and a test recomputes the headline number from the raw rows.
Caveats
- A file counts as valid only when the answer is status 200, parsed as JSON and carries the top-level key the format defines (ucp, protocol, x402Version, entries, openapi or swagger, agents, keys). A status 200 HTML page, a soft 404 or an unrelated JSON error does not count.
- A probe with no response at all (timeout or connection error) is left out of the denominator. A probe that answered 403 or 429 is counted as probed and not valid, because a bot wall or a rate limit may hide a file; the 403 or 429 column shows how many, so those answers lower the share.
- On the main crawl pass a path that robots.txt disallowed was not fetched and is left out of the denominator. The follow-up pass (the /.well-known/ucp, acp.json, x402 and web-bot-auth rows) did not check robots.txt per path, and the rows say so.
- The category groups are small: they hold the domains that a public agent-readiness leaderboard files under E-commerce or Commerce Tools, plus the commerce brands named in the research. Read them as a sample, not as the commerce web.
- Neither the ACP nor the x402 specification requires a discovery path. /.well-known/acp.json and /.well-known/x402 are conventions seen in the wild, so a missing file says nothing about support; x402 itself shows as a 402 response on a paid endpoint.
Cite this
Cite this: Agent Ready Checkout. Commerce discovery file prevalence, crawl of 4 Nov 2026. Edition of 4 Nov 2026. https://agentreadycheckout.com/data/commerce-discovery-files
Sources
- UCP specification overview, read 4 Nov 2026
Last reviewed