Consumer Reports Critiques Tesla’s Full Self Driving Beta 9 Software

Consumer Reports’ car safety experts along with experts from MIT and other academic institutions recently critiqued Tesla’s latest update of their full self-driving Beta software and have this to say about Tesla and what it is doing wrong.

Complaints About Safety and Lack of Consent

According to a recent article from Consumer Reports, their car safety experts and some academic engineering experts are concerned that Tesla’s FSD Beta Software used on public roads lacks safeguards, and that it is putting others at unnecessary risk as Tesla continues to use vehicle owners as beta testers for its new features.

Beta testers play an important role for Tesla as it attempts to gain momentum not just toward interest in their vehicles like the recently released Model S Plaid, but also toward acceptance that EVs with full self-driving capabilities are the next big leap in automotive tech…and safety.

There are numerous videos demonstrating just how well Tesla FSD operates in real life traffic posted by Tesla owners/enthusiasts that are remarkably impressive. However, there has also been some negative publicity involving both real and questionable accidents as well as at least one Tesla owner arrested for breaking California law by riding in the back seat rather than remaining in control of his Tesla while under Autopilot. And, not a few quirks found with Autopilot including a headlight high beam issue.

Recently, the latest update to Tesla’s FSD suite known as “FSD Beta 9” was released and a whirlwind of new videos showcasing Beta 9’s magic have been downloaded to YouTube. However, according to Consumer Reports, their experts have been paying close attention to those videos and are finding faults that include:

• Missing turns
• Scraping against bushes
• Heading toward parked cars.
• Taking the wrong lane after a turn
• “Meandering” from side to side like a drunk driver

In fact, Consumer Reports writes that, “Selika Josiah Talbott, a professor at the American University School of Public Affairs in Washington, D.C., who studies autonomous vehicles, said that the FSD beta 9-equipped Teslas in videos she has seen act ‘almost like a drunk driver,’ struggling to stay between lane lines. ‘It’s meandering to the left; it’s meandering to the right,’ she says. ‘While its right-hand turns appear to be fairly solid, the left-hand turns are almost wild.’”

While the popularity of FSD in Tesla is fueled by it’s fanbase, safety experts at CR point out that their faith in FSD Beta 9 is not warranted and that their fervor for FSD is driving insufficiently proven tech to be released too soon.

“Videos of FSD beta 9 in action don’t show a system that makes driving safer or even less stressful,” says Jake Fisher, senior director of CR’s Auto Test Center. “Consumers are simply paying to be test engineers for developing technology without adequate safety protection.”

Consumer Reports safety experts are not alone in this view and point to other experts who state that much of the public is unaware that they’re part of an ongoing experiment that they did not consent to.

“While drivers may have some awareness of the increased risk that they are assuming, other road users—drivers, pedestrians, cyclists, etc.—are unaware that they are in the presence of a test vehicle and have not consented to take on this risk,” stated Bryan Reimer, a professor at MIT and founder of the Advanced Vehicle Technology (AVT) consortium, a group that researches vehicle automation.

CR points out that in comparison to other developers of self-driving car technology, such as Argo AI, Cruise, and Waymo, that Tesla’s approach uses its fans as test drivers on public streets whereas the aforementioned companies limit their software testing to private courses and/or use trained safety drivers as monitors. Moreover, safety experts point out that at the very least, Tesla needs to insure that their drivers are actually paying attention to the road at all times.

“Tesla just asking people to pay attention isn’t enough—the system needs to make sure people are engaged when the system is operational,” says Fisher, who recommends that Tesla use in-car driver monitoring systems to ensure that drivers have their eyes on the road. “We already know that testing developing self-driving systems without adequate driver support can—and will—end in fatalities.”

As an example of such a fatality, CR writes:

An Uber self-driving test vehicle struck and led to the death of 49-year-old Elaine Herzberg in 2018 as she crossed a street in the Phoenix area. An investigation found that the driver monitor in the back seat of the test vehicle was distracted at the time and did not override the system to stop. A federal investigation partially blamed insufficient regulation and inadequate safety policies at Uber for the collision. After that tragedy, many companies enacted stricter internal safety policies for testing self-driving systems, such as monitoring drivers to ensure that they aren’t distracted and relying more on simulations and closed test tracks.

FSD Is Not Here Yet

To most, seeing those YouTube videos is believing. However, to more critical and trained eyes, FSD is not there yet as Consumer Reports acknowledges from other experts:

“It’s hard to know just by watching these videos what the exact problem is, but just watching the videos it’s clear (that) it’s having an object detection and/or classification problem,” says Missy Cummings, an automation expert who is director of the Humans and Autonomy Laboratory at Duke University in Durham, N.C. In other words, she says, the car is struggling to determine what the objects it perceives are or what to do with that information, or both.

According to Cummings, it may be possible for Tesla to eventually build a self-driving car, but the amount of testing the automaker has done so far would be insufficient to build that capacity within the existing software. “I’m not going to rule out that at some point in the future that’s a possible event. But are they there now? No. Are they even close? No.”

An Attitude Fix Toward a Safer FSD Approach

The problem may lie in that Tesla has taken a Beta version approach that while is good for Apple, is the wrong one for Tesla when it comes to public safety. CR’s interview with Cummings says that Silicon Valley ethos is wrong for Tesla.

“It’s a very Silicon Valley ethos to get your software 80 percent of the way there and then release it and then let your users figure out the problems…And maybe that’s okay for your cell phone, but it’s not okay for a safety critical system,” stated Cummings.

Another point made is that Tesla has not made enough effort to educate its fans, and might be misleading them with marketing terms such as “Full Self-Driving,” that could provide a false impression with many owners excited by the new tech that the vehicles can drive without human intervention.

While developers of other automated driving development would not comment directly on how they feel about Tesla’s approach to testing out their FSD product, they appear to all share a commonality of taking their time, ensuring what works works, and doing so under controlled conditions without putting the public at risk. More specifically, using trained safety drivers and closed test courses to validate vehicle software before testing on public roads.

Can or will Tesla undergo an attitude change toward their approach? The experts did not sound hopeful about this happening and indicate that it is another case of the law lagging behind the technology. It will likely take regulation to enforce change, but will it be too little, too late?

“Car technology is advancing really quickly, and automation has a lot of potential, but policymakers need to step up to get strong, sensible safety rules in place,” says William Wallace, manager of safety policy at CR. “Otherwise, some companies will just treat our public roads as if they were private proving grounds, with little holding them accountable for safety.

For more about how other experts feel about how Beta 9 handles, check out this article and video featuring automotive expert Sandy Munro. In addition, here is a Tesla Vision No Radar Road Test Challenge that recently showcased how well Tesla FSD is doing without radar.

Coming Up Next: The Most Discounted New Cars to Buy Right Now

Timothy Boyer is Torque News Tesla and EV reporter based in Cincinnati. Experienced with early car restorations, he regularly restores older vehicles with engine modifications for improved performance. Follow Tim on Twitter at @TimBoyerWrites for daily Tesla and electric vehicle news.