Visa Test Finds AI Shoppers Fail Often and Buy Anyway

Visa tested 22 AI models on realistic shopping tasks. Models passed 57.2% of runs, while 54% of failed attempts still ended with a purchase, exposing a trust problem for agentic commerce.
Visa's first Agentic Commerce Benchmark found a troubling gap between an AI model's confidence and its ability to shop correctly. Across 22 models from 12 labs, the systems passed 57.2% of attempts on 15 realistic shopping challenges. More revealingly, 54% of failed runs still ended with a purchase.
That means the model either recommended something that did not meet the shopper's stated requirements or failed to uncover enough requirements before acting. For a shopping assistant, a wrong answer is inconvenient. For an agent allowed to spend money, it can become a costly error.
The benchmark asked models to discover hidden constraints through conversation, then decide whether to buy. In one television scenario, the agent had to learn the shopper's budget, format compatibility and gaming needs rather than simply following a brand suggestion. If no product fit, the correct decision was not to purchase.
Performance varied widely. The best models passed 82.2% of attempts, while the weakest passed 28.9%. Cost helped but did not guarantee quality: the five most expensive models averaged a 67.1% pass rate, compared with 50.2% for the five least expensive. Models that uncovered more hidden buying factors also performed better.
Consistency was the harder problem. Each model received three attempts at every scenario, yet all three attempts passed only 34.2% of the time. Even a strong one-off result therefore did not show that an agent would behave reliably for the next shopper.
Visa says the benchmark is research rather than a direct component of Visa Intelligent Commerce. Still, it supports the company's broader effort to make agentic payments safer. Visa plans to extend testing from product selection into checkout, payment and order placement. That is where a recommendation error can turn into a transaction dispute or a loss of trust.
Consumer caution is already visible. Visa's September 2026 Trust Index found that 72% of U.S. consumers had used an AI assistant, but only 23% trusted generative AI to handle payments for them. The benchmark helps explain that gap: models can appear capable while still purchasing after they have failed the task.
For merchants, payment networks and AI developers, the practical lesson is not that automated shopping cannot work. It is that an agent needs explicit controls, a reliable way to gather constraints and a clear ability to stop. The industry must measure correct no-purchase decisions as seriously as successful checkouts.
Frequently asked questions
What is "Visa Test Finds AI Shoppers Fail Often and Buy Anyway" about?
Visa tested 22 AI models on shopping tasks. A 57.2% pass rate and purchases after failed runs show why agentic commerce still needs safeguards.
What is the market context?
Visa's first Agentic Commerce Benchmark found a troubling gap between an AI model's confidence and its ability to shop correctly. Across 22 models from 12 labs, the systems passed 57.2% of attempts on 15 realistic shopping challenges. More revealingly, 54% of failed runs still ended with a purchase. That means the model either recommended something that did not meet the shopper's stated…
When was this article published?
It was published on 05 / 11 / 2026 by Be The Investor Research.



