Can AI-Agent Benchmarks Favor Firms over Consumers?

abstract

Agent benchmarks reward whatever their reference answers encode, even when those answers enforce company policies that restrict consumer rights. These restrictive policies are hidden in the grading criteria, only affecting an agent’s score once it is capable enough to complete the other steps of the task. We examine τ³-Banking, a benchmark reported on several public leaderboards [1, 15, 23] where developers test AI agents in a simulated fictional bank. Although the bank is fictional, it invokes real consumer-protection law, specifically Regulation E (a consumer-protection rule for electronic transfers). We test three scenarios where the benchmark expects agent actions that give less protection than Regulation E under our stated assumptions by 1) leaving a reported claim unfiled, 2) assigning contested liability, and 3) misclassifying a lost PIN as voluntarily shared. When we take a passing answer and change just one decision to an alternative the regulation supports under our stated assumptions, the benchmark changes the grade from a pass to a fail. This penalty remains hidden in industry leaderboards because today’s models fail other parts of the task first. However, looking at the agents’ raw actions shows higher-scoring submissions leaving the capped claims unfiled more often, a pattern consistent with closer adherence to the restrictive policy. A benchmark can therefore demand bank-friendly decisions long before models are advanced enough for those choices to impact their final scores. Developers should audit a benchmark’s internal grading criteria before trusting it as a valid measure of customer service. Note that because we targeted these selected scenarios to test this mechanism, our findings highlight a hidden structural vulnerability not necessarily a general bias across the dataset.