A growing share of research and comparison is being done by software acting for a customer. Software does not admire your homepage. It parses it, or it fails to.

In short

• Buying assistants and agents need structured, unambiguous facts. Most commercial sites do not provide them.

• A study of US retail found product pages only 66% machine readable on average, with homepages at 75%.

• Open protocols for agent-led purchasing are emerging, and they assume merchants can supply clean data about price, availability, delivery and returns.

• This is a commercial readiness issue, not an IT project. The audit takes a day and the gaps are usually cheap to close.

  
  

For thirty years, every page on the internet was written for a person with eyes. That assumption is quietly ending, and most marketing teams have not noticed because nothing has visibly broken yet.

The change is that a growing share of research, comparison and even purchasing is being carried out by software acting on someone's behalf. It reads pages, extracts facts, compares them and returns a shortlist. It is not impressed by a hero image, it cannot interpret text baked into a graphic, and it will not wait for a script to finish before deciding your page contains nothing useful.

How bad is the gap

Adobe put numbers on this by scoring how much of a page's content was actually available to a language model across US retail sites.

Two of those numbers describe a problem and one describes the opportunity. A third of what is published on the most commercially important page type is invisible to the software doing the comparing. Meanwhile the visitors that do arrive through these routes convert considerably better than average, because they have already been filtered.

You are not competing for a ranking any more. You are competing to be parseable.

What machine readability actually means

The term sounds technical and mostly is not. It comes down to four questions about any important page on your site.

Can it be fetched at all?

Automated clients identify themselves, and many sites block them without ever deciding to. A default firewall rule, an aggressive bot filter at the content delivery network, or a blanket disallow written years ago for a different purpose.

The detail that catches people out is that these clients do different jobs. Some collect data for training models. Others fetch your page at the moment a customer asks a question, in order to answer it. They are frequently separate identifiers with separate rules. Blocking everything removes you from live answers while doing very little about the training question, and most teams that have done it did not intend to.

Does the content exist without a browser?

If the meaningful text on a page is assembled by scripts after loading, a simple fetch returns a shell. The same happens with content hidden behind tabs that only populate on click, infinite-scroll listings, and the very common case of specifications presented as an image.

A crude but effective test: view the raw source of your most important page and read it. If the specification, the price and the eligibility criteria are not there in plain text, they do not exist as far as most automated readers are concerned.

Is it structured, or just written?

A paragraph saying delivery is usually quick is not comparable. A field stating a delivery window, with a cut-off time and a region, is. Structured data markup exists precisely for this purpose and remains one of the cheapest improvements available, because it removes interpretation from the process.

The categories worth structuring first are the ones a comparison depends on: price and currency, availability, specification, eligibility, delivery options and timings, returns window and conditions, and location or service area.

Is it consistent everywhere?

If your site says one thing, your listing on a marketplace says another, and your entry in an industry directory says a third, software has no way to decide which is right. It will usually resolve the conflict by favouring whichever source it considers more reliable, which is often not you.

The protocols arriving underneath this

Agent-led purchasing has moved from speculation to standards work. Open protocols are being developed to let software agents interact with commerce systems consistently rather than by scraping each site differently, with the Universal Commerce Protocol among the named efforts.

The strategic point is not which protocol wins. It is what they all assume. Every one of them takes for granted that a merchant can supply clean, current, machine-readable facts about what is for sale, what it costs in total, whether it is available, how it will arrive and what happens if it is returned.

Businesses that cannot supply those facts today will not be able to participate, regardless of which standard prevails. The preparation is identical either way, which makes this an unusually low-risk thing to start on.

Audit

Eight checks, about a day of work

1. Fetch your top ten pages as plain text. Is the substance there, or only navigation and a cookie notice?

2. Review your robots file line by line and list every automated client you currently block. Decide each one deliberately.

3. Check your server logs for which automated clients reached you last month and what they received.

4. Confirm price, availability and specification are in text rather than in images.

5. Validate your structured data for the page types that matter commercially.

6. Compare your own product facts against your listings on third-party platforms and fix every contradiction.

7. Check your returns, delivery and eligibility terms are stated as specifics rather than as reassurance.

8. Confirm the pages a comparison would need are reachable without a login or a form.

What not to over-invest in

Every shift of this kind attracts proposals that sound authoritative and achieve very little. Two are worth naming.

The first is the idea that a single new file placed on your server will make you preferred by automated readers. Formats of this type have been proposed. At the time of writing there is no evidence that major systems act on them, and it has been stated publicly that none currently do. Add one if it takes an hour. Do not let anyone sell it as a strategy.

The second is rewriting all your content to sound like it was written for a machine. The opposite is closer to correct. Software extracts the sentence that answers a question, so writing clearly for a person and putting the answer near the top serves both audiences. Stripped-down, keyword-heavy text reads badly to humans and offers nothing extra to software.

Why this belongs to marketing

The natural instinct is to send this list to engineering and consider it handled. That usually fails, because most of the items are not engineering decisions.

Which automated clients to allow is a commercial judgement about visibility against control. Which facts to publish in structured form is a decision about what you are willing to be compared on. Whether returns terms are stated plainly or written defensively is a brand decision with a legal input. Engineering can implement all of it and should not be asked to decide any of it.

The practical division that works: marketing decides what should be readable and consistent, engineering makes it so, and someone monitors afterwards, because a single release can empty a template without anyone noticing for months.

The unglamorous conclusion

There is no clever tactic here. The work is making true things about your business available in a form that software can read without guessing, and keeping them accurate.

That is not a new skill. It is the same discipline that made catalogues usable and product data reliable long before any of this existed. What has changed is the consequence of neglecting it. A page a machine cannot read is not merely ranked lower. It is absent from a comparison the customer never even sees.

Frequently asked questions