What an automated run actually proves
Automated testing answers one question well: does the website work. AI tools crawl the whole site and check that links resolve, forms submit, buttons lead where they claim, images load and the layout survives a hundred browser and device combinations. They measure load time, flag accessibility violations and catch pages that contradict each other. All of that is observable in code, and all of it is worth doing — a contact form that sends emails to nowhere is a lost enquiry, and nobody finds those by eye.
The report that comes out reads like a readiness signal, but it is narrower than it looks. Everything it checks was written down beforehand: a test encodes what the team already believes the website is for. It confirms the machine’s version of the project.
The visitor who leaves without a trace
The problems that cost you enquiries do not appear in a test report; they appear on someone’s face. A visitor looking for what a service costs opens the menu three times and then goes back to the search results. A page says “complex solution” where a competitor states what is included and from when. Two menu items look like the same page, so neither gets clicked. On a phone, the call to action sits below a long introduction and the visitor never reaches it.
None of those are bugs. The page renders, the form works, the automated pass is green — and the person who was about to send an enquiry leaves without a trace. Analytics records a bounce; it does not record the hesitation in front of it.
This is the gap AI cannot close from the inside. A model reading your website already knows what the company does, what the offer is supposed to mean and where the important part is. A first-time visitor knows none of that, and no automated check produces that ignorance on demand.
A broken link can be found by a machine. A sentence that makes a customer hesitate cannot, because the hesitation happens in a person and not in the page.
Five people, one afternoon
A usability test is not a research project; it is five short conversations in front of a screen. The version that works:
Pick three tasks from the questions customers actually ask you. “Find out what this would cost for a company like ours”, “check whether they work in our region”, “send an enquiry with our requirements”. Not “browse the homepage” — that only tests whether people can scroll.
Recruit five people who resemble your customers, not your colleagues. Same size of company, same role, same level of familiarity with the topic. Your colleagues already know what you do, and that knowledge is exactly what the test needs to be missing.
Give a scenario, then stay quiet. Describe a situation rather than an instruction: “your current supplier has raised prices and you are looking for an alternative — work out whether this company is a fit.” Ask them to think aloud and resist explaining anything. Every explanation destroys the observation you came to make.
Write down the moment, not the opinion. “At 02:14 she scrolled back to the menu and said, I guess this is it” is data. “The menu is confusing” is a conclusion you have not earned yet.
Twenty minutes per person, two afternoons including notes. Research on usability testing — the Nielsen and Landauer model, now more than thirty years old — suggests that five participants typically surface around 85% of the problems for one kind of user. It also suggests two rounds of five beat one round of twenty, because you fix what the first round found before you look again.
AI earns its place around this test, not inside it. It can pull the three tasks out of a year of real customer emails, draft the scenarios, cluster your handwritten notes afterwards and turn recordings into a ranked list of what to fix first. It cannot sit in the chair: it has already read the website with full knowledge of what every word means.
While the copy is still an edit
Run this before launch, when a fix still costs a sentence. Rewording a heading in a draft takes ten minutes. The same fix after launch means a change request, a deploy and a reindex, and the enquiries lost in between are not recoverable. Teams that skip the test usually do it in the week after launch, once the goodwill for “one more round of changes” is already spent.
Run it again in the first weeks after launch with genuinely new visitors — people who arrived from search or a link and have never heard of you. That second round shows whether the message survives contact with someone who was not primed by a referral.
What belongs in the brief
One line is enough: before launch, five people from our target audience test the three key tasks, and we fix everything they could not do. Put a date on it while the project is planned, not while the site is already waiting for release. Budget an afternoon rather than a study, and name the person who recruits participants, because that is the step teams drop when nobody owns it.
How we fit it into the project
In our projects the test happens in the week before launch, with tasks written from the client’s own inbox and participants recruited from people who have never seen the site. Findings are fixed the same week, before the budget for changes runs out.
Automated testing tells you the website works. Five real visitors tell you whether it sells. The first you can schedule any time; the second only pays off before launch — which is exactly when it is cheapest to arrange.
Related service: Web design and content strategy