65% of Trajectories Had Safety Threats. The Planner Still Scored Well.

I initially thought this was primarily a benchmarking paper, a "here's a better leaderboard" kind of contribution. But after sitting with it, I think the implications go further. If the metrics that labs, regulators, and the public use to assess these systems are systematically missing safety-relevant failures, that's not just an academic problem.

And then there's the second paper, which explains part of why the metrics fail.

A separate team published CADET on arXiv, and it attacks a related but distinct problem: causal confusion in end-to-end planners.

The short version is this. When you train a driving model by imitation, showing it expert demonstrations and asking it to learn the underlying policy, it will sometimes learn the wrong thing. Not wrong in an obvious way. Wrong in the sense that it associates the right actions with the wrong causes.

The paper's example is illustrative. A planner might learn to slow down not because there's a pedestrian crossing, but because there's a particular type of building facade that tends to appear in pedestrian-heavy areas. In training data, the two co-occur. The model latches onto the facade. In a new environment with different architecture, the model fails. This is what the researchers call a "spurious cue," and it's essentially invisible to standard metrics because in most test scenarios, the spurious cue and the real causal variable appear together anyway.

CADET is designed to audit for exactly this. The framework, notably, is training-free. It doesn't require retraining the model or even modifying it. It works on already-deployed planners, which matters a lot practically. Retraining large end-to-end models is expensive. Being able to audit and patch spurious reliance without touching the weights is a meaningful capability.

You might be wondering how common causal confusion actually is in practice. Honestly, I'm not sure this holds up at scale yet, and the paper is based on specific benchmarks rather than real-world deployment data. It's too early to say how much of real-world autonomous driving failure is attributable to spurious cues versus other causes. But the mechanism is real, the examples are plausible, and the fact that standard metrics don't catch it is, tbh, kind of alarming.

What both papers are really saying, taken together.

There's a thread connecting these two pieces of work. End-to-end autonomous driving planners are increasingly powerful, increasingly capable of passing the tests we set for them, and increasingly opaque about what they've actually learned. The metrics we use are dominated by easy signal, basically ego vehicle kinematics, and don't probe the harder questions about causal understanding or long-tail safety.

This raises questions about... well, multiple things. Regulatory standards. Public trust. The internal evaluation practices of labs that are already deploying these systems on public roads.

I don't want to overstate it. Both papers are early-stage research. FluidTest's human annotation protocol needs to be validated across more annotator pools and more diverse scenarios. CADET's deconfounding approach hasn't been stress-tested in deployment. These are research contributions, not finished products.

But the direction of travel feels important. The field is starting to take seriously the gap between "scores well on benchmarks" and "is actually safe." That gap, it turns out, might be larger than the leaderboards suggest.

Sarah Williams · 19 Jun · 3 min