We Estimate the number of deaths and major crashes Autopilot would have caused by now if every vehicle in America had the driver-assist system.
We decided to calculate the number of deaths and major crashes that would have occurred since 2016 if Tesla’s autopilot had been installed on every passenger vehicle in America over the past three years. Every model that Tesla sells has now killed an occupant while the vehicle operating on Autopilot. The deaths are not behind us. The latest crash was just nine weeks ago in Tesla’s newest model, the Model 3, and since Tesla updates its cars automagically using telematics, we can safely assume this vehicle had Tesla’s latest and greatest version of the driver assist program.
In March, a Tesla Model 3, the company’s newest vehicle, killed its occupant while it was operating autonomously on Autopilot. This according to the latest report by the National Transportation Safety Board, NTSB. The fatal crash was almost identical to one that occurred three years ago in 2016. Thus, it is only fair to assume that Tesla’s Autopilot still contains the underlying flaw, lack of safeguards, or lack of safety defaults needed to prevent the same accident from happening again. Tesla fans jump to the defense of the company and its Autopilot system. Many blame the driver. Others the driver of the other vehicle. Others the world at large in the case of a Tesla Model X that steered straight into a highway median killing an Apple employee. Many point out that Autopilot is worth the risk since they guesstimate it will save lives if widely adopted. They argue it is safer than a human driver.
What If Every Vehicle Had Autopilot?
That is where we got the idea to simply multiply up the number of deaths and crashes that would have occurred if Autopilot was on every passenger vehicle in America. Tesla’s Autopilot is only found on a tiny fraction of the vehicles in the U.S. passenger fleet. Less than half a million Teslas in a U.S. fleet of over 263 million registered vehicles. The number is so tiny it is hard to even work with. Luckily, an MIT researcher has already saved us much of the work. Lex Fridman has been tracking Tesla’s Autopilot for years. Fridman calculates that as of April 2019, Tesla’s Autopilot has driven its occupants 1,356,442,569 miles (primarily in America). So, in other words, about 1.4 billion miles in total since Autopilot was first deployed at the end of 2015.
Tesla Autopilot Miles Driven
Since it was deployed, Tesla’s Autopilot has been linked to seven major crash events, and we will focus only on those. They include non-fatal, but very serious crash accidents into the back of three firetrucks and one police SUV. The fatal U.S. crashes included two into semi-trailers and one into a highway median barrier. Tesla owners report non-fatal and non-serious Autopilot crashes frequently. They even post videos of these Autopilot crashes. To make things as fair as possible for Tesla, we will ignore all of these minor crashes and just count the seven very serious ones widely documented. We will count all Autopilot miles driven in all countries, but will only count passenger vehicle miles in the U.S. to make the number appear more favorable to Tesla. By counting just the U.S. miles in total, and the total global Autopilot miles driven, the calculated Autopilot crash rate will be lower.
U.S. Passenger-Miles Driven
Here is the other piece of the puzzle. We know the passenger miles driven in total in the U.S. It is a widely available number. 3.2 trillion in 2018, 3.2 trillion in 2017, and 3.1 trillion in 2016. To again make the numbers more favorable to Tesla, we will not count any miles in 2015. We use 1 trillion miles as the number driven in 2019 thus far. So the total number of passenger miles since the start of 2016 we will multiply by will be 3.2+3.2+3.1+1 trillion, or a total of 10.5 trillion miles. We know that Tesla has a serious Autopilot crash about seven times per every 1.356 billion miles. That equals 9.492 serious crashes per 1 billion Autopilot miles driven. A trillion is one thousand billions. So, multiplying by one thousand, we can say that the expected rate of serious autopilot crashes is about 9,492 per trillion miles driven.
Autopilot Crash Rate Calulation
Thus, here are some conclusions from this very non-scientific analysis:
Serious Crashes By Autopilot If Every Passenger Vehicle In America had Autopilot Since 2016 = 99,666, or about 30,000 per year.
10.5 trillion x 9,492 crashes per trillion miles = 99,666 Divide that by 3.3. to get the annual number.
Autopilot Deaths If Every Passenger Vehicle In America had Autopilot Since 2016 = 42,714 or about 13,000 per year.
3/7 x 99,666 = 42,714 Divide that by 3.3. to get annual number.
Here we must add a multiplier to reduce the rate of Autopilot deaths. The reason is that not all the miles driven in America were “qualifying” Autopilot miles. In other words, many of those miles were city miles and single lane roads where Autopilot is not supposed to be used (though it often is). Reduce the number by whatever factor you like. 1/5? Your call.
Autopilot Crash Rate Analysis
At first glance, these numbers may look favorable to Tesla’s Autopilot since the total actual fatalities per year in America are about 37,000 per year (and the number per mile driven has been falling). However, Teslas equipped with Autopilot do not travel all of their miles using the system. It doesn’t work in all situations. So, the deaths by speeding, deaths related to alcohol, deaths caused by driver distraction, and all of the other reasons people are killed in auto-related incidents would still happen to Tesla cars (when not operating on Autopilot). Also, the 37,000 auto-related deaths in American include a substantial number of motorcycles (about 1/7th the total) as well as ATVs, commercial trucks, buses, etc. The number of auto-related deaths due to cars that crash into something is just a fraction of the total deaths each year.
Draw whatever conclusion you like from this analysis. Find the goodness in it if you can. However, we can’t imagine a world where vehicles operating autonomously drive their occupants at high speed into things on a daily basis being allowed by regulators.
If our numbers are wrong, we apologize in advance. Please feel free to extrapolate how many deaths and serious high speed crashes you think would be caused by Autopilot if every passenger vehicle in America had used it for the past few years. Show your math. It is likely to be better than ours. If we have any simple math errors we will correct the story as they are found.
Top of page image courtesy of NTSB.
Wow John, you must really
Wow John, you must really like doing math. How about this much simpler math. Autopilot has been installed in Tesla cars for about 4 years, and they have collectively been driven 1.4 billion miles using autopilot, and 3 people have died in that time (in the U.S.) and one more elsewhere in the world. Or an average of less than one U.S. death in a Tesla with autopilot per year. So there are 37,000 U.S. automotive deaths per year, and only One (1) of them was in a Tesla using autopilot. That sounds like pretty encouraging numbers to me. And a good sales pitch for autopilot. It sounds to me like someone should give Tesla a good safety award for this notable achievement.
Haha. Believe me, I would
Haha. Believe me, I would prefer simpler math. The problem with your analysis is that Tesla has a very very tiny percentage of the US Car population. 500K cars for Tesla of 273 Million overall. So 1/546th of the population are Teslas. Another simple way to calculate the Autopilot deaths that would have occurred if all cars had it would be to just multiply 4 by 546. The total is 2,184. This number is too low though because in 2016 and 2017 Tesla had dramatically fewer cars in the population. The math I did takes into account actual miles driven. Hold off on the safety award. The rate of driver deaths per mile driven due to Autopilot seems huge. Not small.
Why not use the most simple
Why not use the most simple metric, fatalities per mile?
1.4 billion miles / 3 fatalities = 466,666,666 miles between each fatality
3.2 trillion miles / 37,000 fatalities = 86,486,486 miles between each fatality
466,666,666 / 86,486,486 = 440% safer with autopilot
3.2 trillion / 466,666,666 = 6,857 deaths if all vehicles had autopilot (30,142 lives saved per year)
These simpler calculations take in a few assumptions. Some of them are fewer motorcycles crashing into vehicles due to autopilot collision avoidance. People with autopilot would switch it on while drunk, distracted, or speeding, increasing their safety by 440% and reducing their chance of death by 81.5%
Another factor to pay close attention to is that if autopilot ever has fewer than 1 fatality per 86.5m miles, then there is a good argument to implement it, since not implementing it is directly related to allowing people to die that otherwise would have lived, even if it means 1 person may die due to Autopilot error for every 5.3 deaths saved from human error.
Its the classic trolley problem from psychology. We can do nothing and let ~5 people die and one live, but if we choose save 5 peoples lives we are choosing to let 1 die.
The Tesla miles ARE ALL AP
The Tesla miles ARE ALL AP miles. Check your own link. And the comparison is human vs. autopilot, so alcohol, motorcycle, etc. is not a relative factor.
Not sure I follow you.Tesla's
Not sure I follow you.Tesla's Autopilot is not always engaged. I have driven with it off. So of course, during the time that Autopilot is not in control of a Tesla a human is. Tesla's have been crashed due to misbehavior and intention with Autopilot not engaged.
The issue the previous post
The issue the previous post is bringing up is that you assume some portion of the 1.4 billion miles wasn't driven with auto pilot on. But 1.4 billion miles is the number of miles driven with AP on. The total number of miles driven with a Tesla with/without Tesla on was ~13 billion.
Sorry. Lost me. Is this the
Sorry. Lost me. Is this the part you are referring to? "Here we must add a multiplier to reduce the rate of Autopilot deaths. The reason is that not all the miles driven in America were “qualifying” Autopilot miles. In other words, many of those miles were city miles and single lane roads where Autopilot is not supposed to be used (though it often is). Reduce the number by whatever factor you like. 1/5? Your call." Here I am saying that we need to reduce the estimate because not all of the total vehicle population miles driven would have been qualifying Autopilot miles.
But it doesn't really work
But it doesn't really work out that you can just assume that the accident and death rate would remain the same if ALL cars had autopilot. The reality is that in total only Three (3) people have died in Teslas in the U.S. with autopilot engaged over the last 4 years, compared to roughly 150,000 people who died over those same 4 years without Tesla's autopilot safety technology. Again, (3) deaths vs. (150,000) deaths seems like very good odds to me. There was one death in 2016, NONE in 2017, one in 2018, and one so far in 2019 even with the Tesla autopilot installed population increasing dramatically. Again, one death per year vs. 37,000 deaths in the U.S. overall is very good.
I wish I could agree, but the
I wish I could agree, but the tiny population of Teslas negates that argument. We also have seen many other accidents involving Teslas on AP that very well could have been fatal. It's not just three or four serious crashes. I understand that 3 is a very small number and that when you compare it to a much larger one it seems small. However, Tesla's vehicle population is tiny compared to the overall vehicle population. Tesla represents about 500,000/263,000,000 of the vehicles on the road. Now. During 2016, 2017, and half of 2018 it had a much smaller number.
The autopilot Tesla
The autopilot Tesla population may be relatively small compared to ALL cars, but it is easily the LARGEST population of cars in the world actively deploying a higher level of automated driving technology. It works out positively regardless if you compare the small number of Tesla/autopilot deaths to the Tesla population alone, or if you compare to the larger population of all deaths in vehicles. In both cases the death percentages seem relatively tiny. Remember that Tesla's autopilot at this point is really supposed to have driver's attention/assistance at all times, and from what I've read the drivers were not paying attention at all. It is frankly too early to speculate on the relative safety of self driving cars. Once the technology evolves to that stage and they are deployed with full self driving autonomy. For all of the foolish accidents that I see commuting many miles every day due to impatience, substance abuse, and driver distraction, I eagerly away self driving cars, and autopilot safety technology. But I do understand that it is not perfected yet, and I am eager for the blame/fear finger pointing to die down so that the implementation of more automated technology can move forward and evolve.
Well put! These drivers were
Well put! These drivers were supposed to be at attention but it is also easy to drift into fully trusting the system. Nevertheless, Tesla rules are for the driver to be alert at all times. In any event, it is somewhat early and as we get closer to the levels 3+ & 4, I am sure total accidents in general will reduced dramatically. I personally have been in situations where I wished either my vehicle or the other was on auto pilot.
I don't get your math
I don't get your math completely.
Tesla AP has travelled 1.4 billion miles with 3 deaths (in the US).
Also in the US, 3.2 trillion miles have been driven with 37,000 deaths.
So, there is 1 death per 86 million miles.
Thus, we would expect to have seen 16 deaths in the 1.4 billion miles that Tesla AP has driven.
However, we only see 3.
Am I missing something?
Many things. For a start,
Many things. For a start, your 37,000 death figure includes deaths caused by motorcycles, buses, ATVs, and commercial trucks. If AP caused 3 deaths and represented just 0.2% of the vehicles on the road, driving only a fraction of the time the vehicle was operated, how many AP deaths would you project if all those miles by every vehicle had been driven by AP? Let's make it even simpler. How many deaths would you project if AP had driven 10X the number of miles. How many deaths if AP had driven 100X the number of miles. And how many if it had driven 500 times the number of miles?
I think it is completely
I think it is completely unrealistic that AP should be held to a perfect no-deaths standard. As long as AP does not cause significantly more deaths than human drivers under the conditions that it is used, I don't see how anyone could call AP dangerous. We need to always look at relative risk rather than absolute risk. This projection of number of AP-caused deaths is completely disregarding the concept of relative risk and is ignoring the number of deaths that would have occurred without AP.
That being said, I think AP can do much better than that its current state. Clearly Tesla's AP has trouble properly detecting stationary objects. Why exactly is this the case? Why can't radar detect large stationary objects in the highway such as fire trucks? It doesn't make sense to me - clearly radar waves can bounce off of fire trucks just fine! It looks to me like a flawed error-checking algorithm that needs to be modified. I find it surprising that this has not yet been solved - it must be a harder problem than I realize.
If this is some sort of fundamental problem, then Tesla may need to consider alternatives such as lidar. Clearly this problem will need to be 100% solved if Tesla is ever going to be able to claim that it has achieved FSD. I can only assume that Tesla's engineers are working super hard on this problem.
Yeah, but that logic doesn't
Yeah, but that logic doesn't really work, because you are doing math based on projected assumptions of failure. The only facts that we know are that Teslas have driven 1.4 billion miles using autopilot, and with all those miles driven there have been 4 deaths in the whole world. Also most likely those extremely rare deaths could have been avoided if the drivers were actually paying attention, and if they were using autopilot as an assisting safety technology, as they were advised to. The most stern recommendation that I would make to Tesla would be to use eye-tracking technology to confirm that drivers are actually paying attention to the road while autopilot is engaged. I understand that Tesla is actively marketing the prospect of fully self driving cars, but considering that many people think that self driving cars are only acceptable with 100% flawless driving capability (not that humans come close to that) Tesla will probably have to delay the implementation of those more advanced self driving technologies until the lawyers can come to terms with what is an acceptable loss rate for injuries and deaths. Otherwise naysayers will only fear/project the worst, and ignore the progress that has already been made in computer aided driving systems for cars.
I agree that Tesla needs more
I agree that Tesla needs more safety systems for Autopilot. However, there are more facts than just the 4 deaths in the whole world. We also know that Tesla's on Autopilot have slammed full speed into the back of four parked first responder vehicles. We also know, from Tesla owner published videos (one of which I link to) that Tesla's still crash into highway median barriers. Clearly, Autopilot is still crashing onto things it is not meant to. Projecting more deaths over more driven miles is a projected assumption of failure. But so too is any assumption of improved safety claimed by Tesla.
In the past I worked on
In the past I worked on sensor systems and visual tracking software for self driving vehicles, and some of the hardest problems that we faced were random movement or objects out of place. Like a deer jumping out in front of a car, or a stopped car in the middle of the road around a corner. During the recent torrent of raging fires in California some firefighters had parked their firetrucks in the middle of the road as they were fighting the blazes. What is interesting to me is that there was no accounting of normal cars that ran into fire trucks during that time, because it is not money making news the way that Tesla is. As far as the Tesla cars that ran into the highway barriers goes, in every one of those instances the CalTrans workers moved the concrete barriers into the roadway, crossing into the lane, and the Tesla was just following the road. I have experienced this issue firsthand locally and I personally saw several accidents due to the thoughtless roadwork and poor planning. The bigger point is that I am certain that Tesla has learned from those issues and adapted their software to anticipate those issues in the future. Accident avoidance and life saving due to automated safety equipment takes a lot of years to prove, as it has with stability control, anti-lock brakes, safety belts, airbags, tire pressure sensors, etc. And there have always been people opposed to the inclusion of such safety equipment, refuting it's value.
Just having LIDAR instead of
Just having LIDAR instead of RADAR is not necessarily a better solution for detecting stationary objects. LIDAR is similar to RADAR but uses lasers as opposed to radio waves. LIDAR systems "see" more detail, and have better angular accuracy, but RADAR is better at object detection at close distances and it "sees" better in rain, fog or snow conditions. Plus LIDAR sensors are usually larger and cost MUCH more than RADAR systems. RADAR often has the issue of merging close objects together into one larger object, but it does have the advantage of being able to discern different object's relative speed using Doppler frequency shift. When most people see these stories they imagine an empty highway with a flashing fire truck off to the side of the road, and the Tesla blindly running into the big red truck, but most likely the Tesla was in it's own driving lane and there was a car driving right in front of it, then that car quickly veered around the stationary fire truck parked in the driving lane, and neither the human driver, nor automated computer driving system could turn, or stop in time. It is one of the challenges that we saw with video camera tracking, where you cannot see through the object in front of you to detect the object (or vehicle) in front of them. And also you cannot see around an obstructed corner. The complication is that vehicle object tracking usually relies on actively ignoring most all of the static objects all around you, and instead concentrating on staying in the lane and carefully watching all of the other moving vehicles on the road. We attained the highest accuracy by using multiple, simultaneous sensors, from video, to ultrasonic, to RADAR, to LIDAR. Each sensor technology has their own advantages and disadvantages, so more of them working at the same time is better. If not 100% accurate.
Good analysis. The Telsa AP
Good analysis. The Telsa AP would kill 13,000 people per year, compared to 32,000 per year.
Thank you Bohica. I
Thank you Bohica. I appreciate the time you took to figure it out. I will admit the math had my head spinning. Cheers,
This analysis also doesn’t
This analysis also doesn’t take in to account the fact that if every vehicle was capable of full AP, then they would all be ‘aware’ of each other.
This would mean there was a much more detailed picture, each vehicle would know it’s full relative position & speed and the would be no surprises like fire trucks in a highway lane, this would be reported to the network as a closed lane as soon as the truck stopped.
I hope we will soon see closed lanes that are AP only where vehicles will be able to join a pod travelling at 200mph bumper to bumper, that will be when the true strength can be shown.
If Autopilot employed V to V
If Autopilot employed V to V technology. True. Of course, non-Autopilot cars can and will employ V to V technology too. Vehicles travelling bumper to bumper in a pod travelling 200 MPH have been around now for many decades. We call them trains. Good luck "joining" the pod and then exiting at 200 MPH.
Sir, your math here is wrong:
Sir, your math here is wrong:
"We know that Tesla has a serious Autopilot crash about seven times per every 1.356 billion miles. That equals 9.492 serious crashes per 1 billion Autopilot miles driven."
If you have 7 crashes per 1.3 billion you can't have MORE than seven crashes per 1.0 billion. Your erroneously multiplied 7 by 1.356, instead of dividing by 1.356 to get the correct per-1.0b number which is 5.16 - nearly half the figure you incorrectly cited.
It IS a relief. Normally I
It IS a relief. Normally I don't break out the protractor for minor arithmetic mistakes, but that specific excerpt of your article made it to among the top featured items on the Google search results page when I searched "how many Tesla autopilot deaths" or something of that sort, so you should really correct it.
Looking at the deaths, not
Looking at the deaths, not the "serious accidents", on Tesla autopilot we have 3 deaths per 1.356 billion miles. That's 1 death per 466 million miles. There are 1.25 deaths by motor vehicle per million miles driven in the US (includes peds and cyclists). I don't know that this is the best way to look at this, but it's a simple way to look at this and seems pretty compelling to me. Nonetheless, the sample of deaths is very low (which is why "serious accidents" is probably a better way to do this, if there were good statistics. Quick googling didn't not provide any figures on "serious accidents per miles driven".) I could easily imagine that with a bigger sample, the death rate on Tesla autopilot might be found to be double, or alternatively, one third. Definitely an interesting problem.
John Goreham,
John Goreham,
You haven't yet corrected your math mistake of 9.492 serious crashes per billion AP miles, with it actually being almost *half* at 5.16 serious crashes per billion AP miles.
Exactly- he multiplied 7
Exactly- he multiplied 7 deaths by 1.356 billion when it should’ve been 7 / 1.356 billion.
Oops I screwed up on math and made things twice as deadly
I would really appreciate it
I would really appreciate it if someone would do an analysis of unintended accelerations in a Tesla versus other cars. Tesla is saying that people are simply pressing the accelerator instead of the brake, but I highly doubt that.
I wonder if someone driving
I wonder if someone driving without autopilot could have avoided these instances, or would it have been an inevitable fatal incident?
What do you think of autopilot when it is calculated as fatalities per mile? (Updated math from earlier response)
Tesla = 5.1 billion miles driven/ 9 autopilot fatalities = 566,666,666 miles between each fatality
Usa = 3.2 trillion miles driven/ 37,000 fatalities = 86,486,486 miles between each fatality
566,666,666 / 86,486,486 = 555% safer with autopilot
3.2 trillion / 566,666,666 = 5,647 deaths if all vehicles had autopilot (31,352 lives saved per year)
These calculations take in a few assumptions. Fewer motorcycles crashing into vehicles due to autopilot collision avoidance. People with autopilot would switch it on while drunk, distracted, or speeding, increasing their safety by 555% and reducing their chance of death by 85%
Another factor to pay close attention to is if autopilot has fewer than 1 fatality per 86.5m miles, then there is a good argument to implement it, since not implementing it is directly related to allowing people to die that otherwise would have lived, even if it means 1 person may die due to Autopilot error for every 6.6 deaths saved from human error.
It’s the classic trolley problem from psychology. We can do nothing and let ~7 people die and one live, but if we choose save 7 peoples lives we are choosing to let 1 die.