At 8:17 p.m. on a wet Thursday in Ann Arbor, a test vehicle stopped 30 feet before an empty right-turn lane, waited through a complete signal cycle, then proceeded only after a following driver honked. That is not a science-fiction scenario. It is the kind of ordinary failure that matters when evaluating self driving taxis. The vehicle avoided a collision, but its behavior was inefficient, confusing, and difficult for a human road user to predict.
I evaluate autonomous systems the way I evaluate an inference model: define the input, record the output, repeat the scenario, and preserve the failure. Marketing language does not enter the scorecard. A smooth ride is useful evidence, but it is not proof that a system understands the road. Your car talks. I check his homework.
What self driving taxis actually do today
The phrase self driving taxis covers several very different operating designs. Waymo operates a geofenced service in selected cities with a remote assistance function for unusual situations. Zoox is developing a purpose-built vehicle with no conventional front or rear driving position. Cruise has tested similar ideas, although its public operations and regulatory status have changed over time. These are not interchangeable products, and a result from one should not be generalized to all autonomous fleets.
A production robotaxi typically combines cameras, radar, lidar, high-definition maps, localization, prediction, planning, and low-level control. The stack must estimate where objects are, infer what they might do, choose a trajectory, and execute that trajectory within tight latency limits. My useful distinction is not whether the vehicle has a steering wheel. It is whether the system can maintain a safe, legible policy when the scene falls outside its most common patterns.
The operational design matters just as much. A geofence reduces the number of road types, weather conditions, construction layouts, and traffic cultures the model must handle. That is a legitimate engineering strategy, not a shortcut. But a successful ride inside a mapped service area does not demonstrate general autonomous driving.
My test harness for self driving taxis
I use a five-part scorecard: perception, prediction, planning, control, and rider communication. Each category receives a severity rating from 1 to 5. A level 1 event is a harmless hesitation. Level 3 is a behavior that forces another road user to react. Level 5 means a collision, near miss, or dangerous policy that would make continued operation unacceptable.
The test begins before the vehicle moves. I record weather, lighting, road markings, map condition, construction, passenger load, and whether the ride is manually assigned or selected from a normal customer account. During the trip, I log timestamps, speed changes, stops, lane positioning, yielding behavior, and remote-assistance prompts. If video is available, I annotate the relevant frame instead of relying on memory.
Repeatability is the difficult part. A vehicle can make a correct left turn five times and still fail on the sixth because a pedestrian pauses at the curb or a delivery van blocks the usual sightline. I therefore separate capability from luck. One run tells me what happened. Three or more runs begin to tell me whether the behavior is a policy or noise.
The failure modes that matter
The most revealing problems are rarely dramatic. They are small gaps between what the system sees and what a human expects. A robotaxi might stop beside a temporary cone instead of moving around it, hesitate when a cyclist rides along the edge of a lane, or treat a hand gesture from a traffic officer as ambiguous. None of those events necessarily creates immediate danger, but each exposes uncertainty in perception or planning.
One recurring pattern is excessive conservatism. The vehicle chooses a safe trajectory that is so slow or hesitant that surrounding drivers begin to route around it. That can create a new hazard. A human driver understands that a four-way stop requires a clear turn-taking convention; a planner may understand the legal rule but struggle with the social timing.
Another pattern is poor communication. A car that stops unexpectedly has not completed the task simply because it avoided impact. Brake lights, turn signals, positioning, and forward progress are part of the interface. I score a confusing stop higher when it gives the passenger no explanation and forces a following vehicle to make the next decision.
How to compare a ride with a benchmark
Do not compare self driving taxis by ride smoothness alone. Record the route, trip duration, number of unnecessary stops, intervention or assistance events, and any time another road user had to compensate. A seven-mile ride that arrives three minutes late with no confusing behavior can be better than a fast ride containing two unreported freezes.
The comparison should also control for operating domain. Downtown Phoenix, San Francisco hills, suburban Chandler, and an airport access road place different demands on the stack. Weather, road geometry, pedestrian density, and map freshness can change the result more than a software version number. When an operator announces a major update, I want before-and-after runs on matched routes, not a screenshot of a new dashboard.
For engineers, the most valuable artifact is the replayable edge case. Save the sensor context, planner output if available, and the expected action. Then write the gap in plain language: Here is what happened. Here is what it should have done. Here is the gap. Without that record, a vague claim about improvement is impossible to audit.
What riders should inspect before booking
A passenger should know the service boundary, weather limitations, support process, and pickup behavior before entering a self-driving vehicle. Check whether the service operates during heavy rain, whether a remote specialist can assist, and what happens if the vehicle cannot complete the route. A support button is useful only if it connects quickly and the rider can communicate the problem clearly.
Watch the first five minutes instead of immediately trusting the novelty. Does the vehicle merge decisively? Does it leave reasonable space around parked cars? Does it signal early? When it stops, can you understand why? These observations are more informative than a polished launch video. I would also avoid treating a successful ride as permission to distract yourself. The operator remains responsible for its automated driving system, but a passenger still needs to follow safety instructions and report unsafe behavior.
Pricing is another practical variable. Fares can look competitive with a standard rideshare during promotions, then change with demand, wait time, or service availability. The cheapest ride is not automatically the best benchmark. A comparable human-driven trip can reveal whether the automated service is genuinely useful for your route or merely interesting once.
The scorecard, not the slogan
Self driving taxis are an important deployment test because they expose autonomy to ordinary passengers, messy streets, and repeated operational pressure. They are not evidence that every personal vehicle is ready to drive anywhere. The strongest claims will come from transparent logs, clear geofences, safety reports, and reproducible comparisons across versions.
My current standard is simple: show the route, show the failure, explain the limitation, and rerun the test after the update. A system that improves its level-4 hesitation to level 2 deserves credit. A system that hides the edge case behind an average completion rate does not.
The homework is unfinished, which is normal for a difficult engineering problem. The useful question is not whether self driving taxis look futuristic from the back seat. It is whether their decisions remain safe, predictable, and explainable when the road stops cooperating. That is the test I will keep running.