--- title: "Commerce discovery files in the crawl | Agent Ready Checkout" description: "How many commerce and top-10,000 domains serve a valid UCP profile, agent registry, OpenAPI or agents.json file, from the shared crawl of 4 Nov 2026." canonical: https://agentreadycheckout.com/data/commerce-discovery-files site: "Agent Ready Checkout" last_updated: 2026-11-04 date: 2026-11-04 section: "Method and data" --- # Commerce discovery files in the crawl | Agent Ready Checkout > How many commerce and top-10,000 domains serve a valid UCP profile, agent registry, OpenAPI or agents.json file, from the shared crawl of 4 Nov 2026. Source: https://agentreadycheckout.com/data/commerce-discovery-files Site: Agent Ready Checkout Last reviewed: 2026-11-04 # Commerce discovery files in the shared crawl In short Of 178 commerce-category domains we could probe in the crawl of 4 Nov 2026, 15 served a valid UCP profile at /.well-known/ucp. Across the top 10,000 the figure is 47 of 7,401. The UCP row comes from a follow-up pass that did not check robots.txt per path, and answers of 403 or 429 are counted, not dropped. ## Results ### E-commerce and Commerce Tools category domains 184 domains in the group, 181 reachable. | Probe path | Robots.txt checked per path | Domains probed | Answered 403 or 429 | Status 200 | JSON | Valid file | Share valid | | --- | --- | --- | --- | --- | --- | --- | --- | | /.well-known/ucp | no, follow-up pass | 178 | 6 | 35 | 15 | 15 | 8.43 percent | | /.well-known/ucp.json | yes | 177 | 7 | 31 | 13 | 13 | 7.34 percent | | /.well-known/acp.json | no, follow-up pass | 178 | 6 | 19 | 1 | 1 | 0.56 percent | | /.well-known/x402 | no, follow-up pass | 178 | 6 | 20 | 0 | 0 | 0 percent | | /.well-known/ard.json | yes | 177 | 7 | 19 | 1 | 1 | 0.56 percent | | /openapi.json | yes | 176 | 8 | 24 | 5 | 5 | 2.84 percent | | /agents.json | yes | 176 | 8 | 19 | 1 | 0 | 0 percent | | /.well-known/web-bot-auth/directory | no, follow-up pass | 178 | 6 | 21 | 0 | 0 | 0 percent | Domains with at least one valid file in this group: anakin.io, apparel.monster, chewy.com, cratejoy.com, flaviar.com, flexfits.com, hypeandvice.com, ikea.com, matagora.com, provenskincare.com, riveterhq.com, sellfy.com, shoplocale.com, shoptiques.com, socksoho.com, soteriskin.com, theessential.com, vendah.com.br, zinc.com. ### Tranco top 10,000 (all categories) 9999 domains in the group, 7849 reachable. | Probe path | Robots.txt checked per path | Domains probed | Answered 403 or 429 | Status 200 | JSON | Valid file | Share valid | | --- | --- | --- | --- | --- | --- | --- | --- | | /.well-known/ucp | no, follow-up pass | 7401 | 1264 | 1037 | 51 | 47 | 0.64 percent | | /.well-known/ucp.json | yes | 7442 | 1255 | 943 | 24 | 18 | 0.24 percent | | /.well-known/acp.json | no, follow-up pass | 7408 | 1233 | 927 | 5 | 1 | 0.01 percent | | /.well-known/x402 | no, follow-up pass | 7404 | 1233 | 991 | 4 | 0 | 0 percent | | /.well-known/ard.json | yes | 7438 | 1268 | 939 | 24 | 18 | 0.24 percent | | /openapi.json | yes | 7472 | 1125 | 1075 | 24 | 21 | 0.28 percent | | /agents.json | yes | 7485 | 1120 | 1042 | 7 | 2 | 0.03 percent | | /.well-known/web-bot-auth/directory | no, follow-up pass | 7409 | 1258 | 995 | 5 | 0 | 0 percent | Domains with at least one valid file in this group: academy.com, admaster.cc, alchemy.com, amplitude.com, anker.com, appsflyer.com, att.com, august.com, aylanetworks.com, barnesandnoble.com, beehiiv.com, bitdefender.com, bluehost.com, boozt.com, cal.com, chewy.com, chiikawamarket.jp, cloudflare.com, cloudx.io, cm.com, cookieyes.com, daisonet.com, dev.to, discoverpilgrim.com, dmac-solutions.net, dns-finder.com, dpboss.tax, dpbossss.boston, dpbosssss.boston, ecoflow.com, elementor.com, eufy.com, eufylife.com, expiredwixdomain.com, farfetch.com, fineartamerica.com, forter.com, gameanalytics.com, govee.com, guard.io, haveibeenpwned.com, homedepot.com, hootsuite.com, hp.com, ikea.com, linksys.com, mattel.com, mediaexpert.pl, monday.com, monetixads.com, mysterium.network, nanit.com, netlify.app, netlify.com, nic.ai, nishatlinen.com, openlibrary.org, openrouter.ai, padlet.com, paypal.com, paypalobjects.com, pixai.art, pixels.com, pixhost.to, pocketcasts.com, post-gazette.com, preply.com, programmaticx.ai, railway.app, saya.pk, supabase.co, supabase.com, surveymonkey.com, sweetwater.com, theiconic.com.au, thredup.com, tradedoubler.com, ukrainealarm.com, usercentrics.eu, wiki.gg, withings.net, wyze.com, wyzecam.com, zellbury.com. ### Payments and Fintech category domains 401 domains in the group, 399 reachable. | Probe path | Robots.txt checked per path | Domains probed | Answered 403 or 429 | Status 200 | JSON | Valid file | Share valid | | --- | --- | --- | --- | --- | --- | --- | --- | | /.well-known/ucp | no, follow-up pass | 399 | 11 | 40 | 4 | 4 | 1 percent | | /.well-known/ucp.json | yes | 393 | 11 | 33 | 0 | 0 | 0 percent | | /.well-known/acp.json | no, follow-up pass | 399 | 11 | 34 | 1 | 1 | 0.25 percent | | /.well-known/x402 | no, follow-up pass | 399 | 11 | 37 | 0 | 0 | 0 percent | | /.well-known/ard.json | yes | 393 | 11 | 36 | 3 | 3 | 0.76 percent | | /openapi.json | yes | 395 | 13 | 42 | 3 | 3 | 0.76 percent | | /agents.json | yes | 395 | 12 | 37 | 1 | 0 | 0 percent | | /.well-known/web-bot-auth/directory | no, follow-up pass | 399 | 11 | 36 | 0 | 0 | 0 percent | Domains with at least one valid file in this group: cypherhq.io, fintoc.com, forter.com, getlago.com, itseasy.co, joinforge.app, mercury.com, onefin.in, paypal.com, ramp.com. The follow-up crawl probed /.well-known/ucp, the path the UCP specification publishes, on the same domains. It held 10508 rows when this slice was cut, and the UCP row above uses it. ## How it was built The shared crawl read 13,071 domains on 4 Nov 2026 with a plain-fetch client and the user agent agent-readiness-crawl/1.0. Its main pass checked robots.txt for every path; the follow-up pass that supplies the UCP, ACP, x402 and web-bot-auth rows did not check robots.txt per path, and the table marks those rows. This slice keeps the domain, category, rank and the answer to each commerce probe. A file counts only when the answer is status 200, parses as JSON and carries the top-level key its format defines. The slicing script is scripts/slice-crawl.mjs and a test recomputes the headline number from the raw rows. ### Caveats - A file counts as valid only when the answer is status 200, parsed as JSON and carries the top-level key the format defines (ucp, protocol, x402Version, entries, openapi or swagger, agents, keys). A status 200 HTML page, a soft 404 or an unrelated JSON error does not count. - A probe with no response at all (timeout or connection error) is left out of the denominator. A probe that answered 403 or 429 is counted as probed and not valid, because a bot wall or a rate limit may hide a file; the 403 or 429 column shows how many, so those answers lower the share. - On the main crawl pass a path that robots.txt disallowed was not fetched and is left out of the denominator. The follow-up pass (the /.well-known/ucp, acp.json, x402 and web-bot-auth rows) did not check robots.txt per path, and the rows say so. - The category groups are small: they hold the domains that a public agent-readiness leaderboard files under E-commerce or Commerce Tools, plus the commerce brands named in the research. Read them as a sample, not as the commerce web. - Neither the ACP nor the x402 specification requires a discovery path. /.well-known/acp.json and /.well-known/x402 are conventions seen in the wild, so a missing file says nothing about support; x402 itself shows as a 402 response on a paid endpoint. Download: [CSV](https://agentreadycheckout.com/data/commerce-discovery-prevalence-2026-11-04.csv), [JSON](https://agentreadycheckout.com/data/commerce-discovery-prevalence-2026-11-04.json). ## Cite this **Cite this:** Agent Ready Checkout. Commerce discovery file prevalence, crawl of 4 Nov 2026. Edition of 4 Nov 2026. https://agentreadycheckout.com/data/commerce-discovery-files ## Sources 1. [UCP specification overview](https://ucp.dev/latest/specification/overview/), read 4 Nov 2026 Last reviewed 4 Nov 2026 ## Edition Matrix edition of 4 Nov 2026. Reviewed 4 Nov 2026, [1 sources](#sources-h). ## Method [How cells, sources and read dates are defined](https://agentreadycheckout.com/method)