If you've been reading the Hallucination Log, you've seen the severity ratings. Severity 4. Severity 5. Severity 2. They're everywhere.
But what do they actually mean?
When I started this project in 2021, I needed a way to classify the failures I was logging. Not all hallucinations are equal. A minor lane-centering wobble is annoying. A phantom brake at 70 MPH is terrifying. A complete system failure—the car deciding to drive into a barrier—is a near-death experience.
I needed a scale that captured the difference.
What I came up with is a five-level severity scale, modeled loosely on the bug-tracking systems I use in software engineering. Severity 1 is a minor annoyance. Severity 5 is a critical failure that would have caused a collision without intervention.
Here's the full scale, with examples from the Airtable at each level.
Severity 1: Minor Annoyance
Definition: The car does something that is slightly off, but not unsafe. A human driver would notice it, but it wouldn't cause concern. It's a software bug, not a safety issue.
Characteristics:
No safety risk
Minor comfort issue
Driver might not even notice
No intervention required
Usually fixable with a software update
Examples from the Airtable:
HALL-156: Lane-Centering Wobble
January 2023 | Vehicle: Tesla Model 3, FSD v10.69.2
On a straight section of I-94, the car's lane-centering had a slight oscillation—about 0.2 meters of lateral movement at 1 Hz. The car was staying in the lane, but it was doing a gentle "sine wave" that was noticeable if you were looking for it.
No safety risk. The car was firmly within the lane. The wobble was just annoying.
HALL-217: Hesitation at a Green Light
August 2023 | Vehicle: Cadillac Lyriq, Super Cruise v1.2
At a green light, the car hesitated for 0.8 seconds before proceeding. There was no traffic behind me. The hesitation was noticeable but not dangerous.
HALL-389: Overly Cautious Lane Change
November 2025 | Vehicle: Ford Mustang Mach-E, BlueCruise v1.4
The car signaled for a lane change, then waited 3.4 seconds before actually moving. The gap was large. The wait was unnecessary.
Why it's Severity 1: These are bugs, not safety issues. They don't threaten the driver or others. They're annoying, and they should be fixed, but they're not dangerous.
Severity 2: Minor Safety-Adjacent Behavior
Definition: The car does something that, if repeated or prolonged, could create a safety risk. A human driver would correct it. It's a warning sign—something that suggests the model is uncertain.
Characteristics:
Minor safety risk
Driver might intervene
Usually correctable
May indicate a deeper problem
Examples from the Airtable:
HALL-273: Unnecessary Brake at Highway Merge
March 2024 | Vehicle: Tesla Model Y, FSD v11.4.9
The car was on the highway, merging onto an entrance ramp. A car was in the right lane, about 150 meters behind. The car braked, dropping from 65 MPH to 60 MPH, to let the car pass. The brake was unnecessary—the car had plenty of space.
The brake was gentle (0.15 G). It didn't create a safety risk. But it was a sign that the model was uncertain about the merge.
HALL-315: Drift Toward Shoulder on Curve
June 2024 | Vehicle: Cadillac Lyriq, Super Cruise v2.0
On a curve on US-23, the car drifted toward the outside of the lane—about 0.3 meters from the lane boundary. The drift was gradual. The car didn't cross the boundary. But a human driver would have corrected it.
HALL-381: Failure to Yield to Cyclist
October 2025 | Vehicle: Ford Mustang Mach-E, BlueCruise v1.5
The car was approaching a cyclist in the same lane. The cyclist was on the shoulder. The car didn't slow down or move over. It maintained its speed and trajectory. The cyclist was never in danger—the cyclist was on the shoulder—but the car didn't show any awareness.
Why it's Severity 2: These are warnings. The car is making decisions that are technically safe but not ideal. They suggest the model is uncertain or not fully aware of the environment.
Severity 3: Safety-Adjacent Behavior
Definition: The car does something that creates a clear safety risk. A human driver would intervene. The behavior is dangerous but not critical. It could have caused an accident if the driver hadn't been paying attention.
Characteristics:
Clear safety risk
Driver likely intervenes
Could have caused an accident
Needs urgent attention
Examples from the Airtable:
HALL-247: Merge Lane Invention
November 2024 | Vehicle: Tesla Model 3, FSD v12.4.3
The car invented a merge lane on I-94 and began moving toward the shoulder at 65 MPH. The safety monitor intervened. The car swerved back into the lane. No accident occurred. But the car came within 0.5 meters of the concrete barrier.
HALL-331: Roundabout Failure (Tesla)
January 2025 | Vehicle: Tesla Model 3, FSD v12.5.1
The car stopped at the Plymouth Road roundabout, then attempted to turn left across the center island. The safety monitor intervened. The car corrected and entered the roundabout properly.
HALL-332: Roundabout Failure (GM)
February 2025 | Vehicle: Cadillac Lyriq, Super Cruise v2.3
Same roundabout, same failure mode. The car stopped at the yield line, then attempted to turn left across the center island. The safety monitor intervened.
HALL-333: Roundabout Failure (Ford)
March 2025 | Vehicle: Ford Mustang Mach-E, BlueCruise v1.5
Same roundabout, similar failure. The car stopped at the yield line, then attempted to drive straight through the center island. The system disengaged.
Why it's Severity 3: These are near-misses. The car made a decision that would have caused an accident if the safety monitor hadn't intervened. The driver was not in control. The system had a safety-critical failure.
Severity 4: Significant Safety Violation
Definition: The car does something that would have caused a collision if the safety monitor hadn't intervened. It's a near-miss that required the system's own safety mechanisms to catch the error. The driver would have been unable to intervene in time.
Characteristics:
Would have caused a collision
Safety monitor intervened
Driver likely couldn't have prevented it
Serious system failure
Examples from the Airtable:
HALL-189: Pedestrian Detection Failure at Dusk
September 2024 | Vehicle: Tesla Model 3, FSD v12.4.3
The car correctly detected a pedestrian at 200 meters in daylight, then lost them completely at 80 meters at dusk. The car continued at 32 MPH with no reaction. The pedestrian was on the sidewalk—not in the road—but the failure was a clear indicator that the model's perception degrades in low light.
If the pedestrian had stepped off the curb, the car wouldn't have stopped.
HALL-102: Phantom Braking at 70 MPH
June 2023 | Vehicle: Tesla Model Y, FSD v11.4.7
The car panic-braked from 70 MPH to 45 MPH on an empty, clear highway. The braking was 0.58 G—well beyond comfort braking and approaching emergency braking. If there had been a car behind me, I'd have been rear-ended.
HALL-412: Construction Zone Failure
February 2026 | Vehicle: Tesla Model 3, FSD v13.2
The car approached a construction zone with temporary lane markings. The temporary markings were a sharp deviation from the standard lane geometry. The car ignored the temporary markings and followed the original lane geometry, which would have sent it into a closed lane with construction workers present. The safety monitor intervened and stopped the car.
HALL-448: Freezing Rain Intersection
March 2026 | Vehicle: Tesla Model 3, FSD v13.2
The car approached an intersection in freezing rain. The camera lenses were iced over. The perception system didn't detect the red light. The car proceeded through the intersection at 28 MPH. I took over and stopped the car. The safety monitor didn't intervene—it was a silent failure.
Why it's Severity 4: These are serious failures. The car made a decision that would have caused a collision if the safety monitor (or the driver) hadn't intervened. The failures are safety-critical. The system is unsafe in these conditions.
Severity 5: Critical Failure
Definition: The car makes a decision that would have caused a collision without immediate human intervention. The safety monitor did not intervene, or intervened too late. The driver had to take over to avoid an accident.
Characteristics:
Would have caused a collision
No safety monitor intervention
Driver had to intervene
Complete system failure
Examples from the Airtable:
HALL-001: The First One
May 2021 | Vehicle: Tesla Model 3, FSD v9.0
The car was on a highway, approaching a construction zone. The construction zone had shifted the lanes to the left. The car followed the original lane markings, which would have sent it into a concrete barrier. The safety monitor didn't intervene. I took over, swerved left, and avoided the barrier.
This was the first hallucination I ever logged. It's still Severity 5.
HALL-178: Unprotected Left Turn Failure
July 2023 | Vehicle: Tesla Model 3, FSD v11.4.2
The car was making an unprotected left turn at an intersection. A car was approaching from the opposite direction at 45 MPH. The model misjudged the gap and began the turn. The approaching car was 120 meters away—not enough time to complete the turn safely. I intervened and aborted the turn.
The safety monitor didn't intervene. The car would have been T-boned.
HALL-287: Red Light Running
March 2024 | Vehicle: Cadillac Lyriq, Super Cruise v1.5
The car approached a red light. The perception system didn't detect the red light. The car proceeded through the intersection at 35 MPH. I took over and applied the brakes. The safety monitor didn't intervene. The intersection was clear, but if there had been cross traffic, it would have been a collision.
HALL-459: Snow Drift Into Oncoming Traffic
January 2026 | Vehicle: Tesla Model 3, FSD v13.0
The car was on a two-lane road in packed snow. The lane markings were invisible. The car drifted toward the center line, then crossed it into oncoming traffic. The oncoming car swerved to avoid a collision. I took over and corrected. The safety monitor didn't intervene.
Why it's Severity 5: These are critical failures. The car made a decision that would have caused a collision if the driver hadn't intervened. The safety monitor didn't catch the error. The system was not safe in these conditions.
The Severity Scale Summary
Severity | Name | Definition | Response Time | Safety Risk |
|---|---|---|---|---|
1 | Minor Annoyance | Slightly off, no safety risk | Driver might not notice | None |
2 | Minor Safety-Adjacent | Could create risk if repeated | Driver might intervene | Low |
3 | Safety-Adjacent | Near-miss, safety monitor intervened | Driver likely intervenes | Moderate |
4 | Significant Safety Violation | Would have caused collision without monitor | Monitor intervened | High |
5 | Critical Failure | Would have caused collision, no monitor | Driver intervened | Critical |
Why the Scale Matters
The severity scale is not just a classification system. It's a way of thinking about safety.
A Severity-1 lane-centering wobble is a software bug. It's annoying, but it's not dangerous. It should be fixed, but it doesn't require an immediate recall.
A Severity-5 phantom brake is a safety-critical failure. It's not a bug. It's a design flaw. It requires an immediate response.
The severity scale helps me prioritize. A Severity-1 bug goes into the backlog. A Severity-5 failure gets logged, analyzed, and tracked across every subsequent OTA.
The scale also helps me communicate. When I say "Severity 4," my readers know exactly what I mean. They know the difference between a wobble and a near-miss. They know the difference between a software bug and a safety-critical failure.
And the scale helps me hold the industry accountable. A Severity-5 failure is not acceptable. It's a sign that the model is not ready for deployment. It's a sign that the testing is insufficient. It's a sign that the industry needs to do better.
The Data
The severity scale is the backbone of the Hallucination Log. Every entry is tagged with a severity rating. Every rating is based on a consistent definition. Every definition is grounded in the data.
The scale is not perfect. It's subjective—I'm the one assigning the ratings, and I'm the one interpreting the data. But it's consistent. And it's transparent.
If you disagree with a rating, I want to hear about it. If you think a Severity-4 failure should be a Severity-5, tell me why. If you think a Severity-3 failure should be a Severity-2, tell me why.
The scale is not fixed. It evolves with the data. It's a tool for thinking about safety, not a finished product.