LLMs as anomaly detectors: catching fraud at 100k tx/day with AI agents in production

For most of 2024, if you asked me whether LLMs had a role in real-time fraud detection at scale, I would have said no. Latency was wrong, cost per inference was wrong, hallucination rate was wrong. In 2025 we tried anyway. In 2026 it's a core part of our fraud stack, catching things our classical models miss and doing it cheaply enough to run on every transaction. This is what changed my mind, and what I'd tell anyone thinking about doing the same.

Why classical ML wasn't enough

Our baseline was a well-tuned gradient-boosted model on ~150 hand-engineered features. It caught the obvious patterns — velocity anomalies, geo-mismatches, known-bad device fingerprints — with reasonable precision. It failed at three specific things:

• Narrative fraud — where the transactions were individually normal but the sequence, in the context of the account's history, told a story a human would immediately recognise as fraud.
• Novel patterns — anything a fraudster invented after our last training run was invisible to us for weeks.
• Contextual thin-signal — small accounts where we didn't have enough behavioural history to make the ML confident, but where a plain-English summary of the account state would trip any human analyst's instincts.

These are exactly the failure modes where LLMs, in theory, should shine. The theory turned out to be right. The engineering was harder than the theory.

The agent architecture we shipped

The naive version — send every transaction to GPT-4 or Claude with a big prompt, get back a fraud score — does not work. It's too slow, too expensive, and the models are terrible at outputting well-calibrated numerical scores. What works is a routed architecture with specialist agents:

1. Router (small, cheap model). Every transaction hits a fast Haiku-class router that decides whether the LLM stack sees it at all. Roughly 5% of traffic passes this gate; the rest goes straight to the ML model. This alone drops cost by 95%.
2. Context assembler (deterministic code, not LLM). Given a transaction that passes the gate, this pulls the account history, recent related transactions, device state, and known risk signals into a structured summary. This is not an agent — it's a boring SQL + serialisation function. Getting this right is where most of the engineering work lives.
3. Specialist agents (mid-tier model, structured output). Depending on the router's classification, one of three specialist agents runs: account-takeover, merchant-collusion, or synthetic-identity. Each is a dedicated prompt with dedicated eval data.
4. Adjudicator (larger model, only when disagreement). If two specialists disagree, or if the specialist disagrees with the classical model with high confidence, an adjudicator agent gets both perspectives and outputs a final decision with reasoning. This runs on ~0.3% of traffic.

The point of this shape is that most transactions are cheap, only the ambiguous ones get expensive scrutiny, and every decision has a paper trail.

What burned tokens for nothing

The mistakes were expensive and instructive:

• Chain-of-thought without token limits. Early on, the specialist agents were allowed to "think" freely before answering. Some transactions cost 40x the median. We capped reasoning at 500 tokens and quality did not measurably drop.
• Free-form output. We started with "explain your reasoning and give a score." Parsing the score reliably was a permanent bug source. Switching to structured outputs (JSON schema with an enum for decision, a bounded float for confidence, and a bounded text for rationale) killed an entire class of failures.
• Sending the raw ledger. The context assembler initially dumped 5,000 tokens of transaction history. Most of it was noise. Summarising the last 30 days into a 300-token narrative — deterministically, with code, not an LLM call — cut cost and improved accuracy.
• Retries on "I don't know." Not retrying is the right answer. "I don't know" is signal; escalate to the adjudicator.

The eval harness

The single thing that made this system safe to ship was an eval harness we ran on every prompt change, model change, and quarterly baseline. The core loop:

• A golden set of ~4,000 transactions with human-labelled ground truth, curated over months and continuously updated.
• Automated metrics: precision, recall, false-positive rate at fixed thresholds, cost per detected fraud dollar, latency p50/p95/p99.
• Regression gates: any prompt change that drops recall by more than 2% or increases cost per detection by more than 15% blocks the deploy automatically.
• Shadow mode for every change: two weeks running in parallel with the old version before promotion, with per-decision comparison logged.

This is boring MLOps applied to LLM prompts. It is also the difference between an LLM system you can operate and an LLM system that operates you.

Real cost numbers

At 100k transactions per day, our LLM stack costs approximately $180/day in inference. Compared to the fraud dollars it catches that our classical model missed — averaging around $22,000/day — this is not a close call. Two things made the economics work:

• The router keeps 95% of traffic out of the LLM path entirely.
• Model pricing dropped ~4x between mid-2024 and mid-2026. What was uneconomic 18 months ago is now boring infrastructure.

Lessons for anyone building this

1. Route aggressively. Do not send every request to a big model. Cheap gates in front of expensive models are how you make the economics work.
2. Deterministic context assembly beats agentic context gathering. The temptation to have the agent "figure out what data it needs" is huge and expensive. Resist it. Give the agent the same 15 well-chosen fields every time.
3. Structured outputs are non-negotiable. If your production system depends on parsing free-form model output, you have a latent bug waiting to break at scale.
4. Eval before you ship, always. If you don't have an eval harness that gates deploys, you don't have a production LLM system — you have a proof of concept running in prod.
5. Cost per outcome, not cost per token. "$180/day sounds expensive" is meaningless without "and it catches $22k/day of fraud." Frame the ROI in the units the business cares about.

The bigger point is that LLMs in production aren't magic — they're a new component in the same engineering discipline you already do. Ship boring architectures, measure everything, and stop believing the demos.