Tesla FSD Needs These 5 Things Before Robotaxis Can Launch

Work for Torque News, follow on Twitter, Youtube and Facebook.

There was a recent launch of Tesla's FSD Beta v9 software. There are many videos of it and it's clear that it is improving. But there are 5 things that need to be done much better before cars can drive themselves without human intervention and in order to awaken the robotaxi fleet.

We know the Tesla FSD Beta v9 is slick and smooth. But there's much more that needs to be done to awaken the robotaxi fleet and for people to get paid when they aren't using their Tesla's. Here's what it is:

1: Confidence and Precision With Unprotected Left Hand Turns

The YouTube user, Chuck Cook, has a very difficult unprotected left hand turn he can take going out of his neighborhood where he lives. He's done many videos on this, the most recent being, the three unprotected left hand turns. The turn is difficult. The car has to have room on the left for traffic going left to right, and then there is another set of traffic beyond that going right to left, which the car then needs to merge into.

In my opinion, this litmus test of making a very difficult unprotected left hand turn is one of the ultimate signs of full self-driving being ready for level 5. If the car can regularly make this turn, it can probably make most other difficult turns. Here's another video of the car failing to make an unprotected left hand turn.

In order to make this turn, I believe the car is going to need better distance viewing from its cameras, and possibly cameras on the side of its headlights. It's going to need to be able to see the traffic far on the left and right from both sets of traffic. That's traffic coming from 2 different directions on two different sections of road. It will need to be able to predict the velocity of each vehicle from those sections of road and deduce what speed and angle it needs to make while driving through. I'm very excited to see further updates and how the car does here. This turn is very difficult even for a human driver.

2: The Ability to Change Routes When Traffic Blocks the Current Route

Tying in with #1, sometimes there are just busy and dangerous unprotected left hand turns. If the car can't make the turn, it should do what a human would - make a right hand turn and take an extra minute or two on its drive. In the case of the above video, if traffic is so dense and not easing up after a minute, the car should be able to consider alternate and more safe routes.

If the car is in dense traffic and that traffic has blocked it off on the left and it needs to make a left hand turn, the car doesn't have to just sit there and do nothing (which is what it does now). The car can make a right hand turn, make a u-turn, or take a little extra time and change its route as necessary. Even better is if the car plans ahead and is adaptable, which brings me to point #3:

3: Better Adaptability and Accuracy

The ability to adapt quickly to construction, speed bumps, unnatural road markings, accuracy with being in the right lane and avoiding unnecessary lane changes. There are just a few of the issues with Tesla FSD needing to adapt to all situations.

In this video from an earlier version of the beta, we can see FSD having trouble with cones and construction.. Also in the video, there's a case where a construction worker is waving the car forward, but the FSD system isn't recognizing that and is instead doing the wrong thing and staying put.

Here's an example of an unnecessary lane change that didn't need to happen. The car has 0.5 miles to go and needed to turn left at that point. Instead, the car changed to a lane to the right in sparse traffic. There wasn't any need to do that. The car should have stayed in its lane, or moved to the left lane.

Here's an example of the FSD taking a left turn too wide. This just requires more adaptability as the turn wasn't a normal turn and there were some construction objects directly ahead that could have thrown it off.

Here's a video of Tesla FSD taking a speed bump a little too fast. It should take the speed bumps at 15 miles per hour or less. It did this one at 19 miles per hour. It just needs to be able to see and recognize them sooner and act appropriately.

There's also an example of the Tesla FSD v9 Beta handling construction well. In this case, it turned into some construction, rendered it on the UI, and didn't try to merge into it like the previous video.

Here's an example of Tesla FSD handling a roundabout really well. It has made progress with roundabouts.

The car also merges a lot better with traffic. Here's a video clip of Tesla FSD v9 merging nicely with dense traffic.

Lastly, the software is going to need to adopt different driving styles for different situations and vehicles. For example, a semi-truck will drive a little slower, especially going down hill and likely stay to the right on the freeway more. A robotaxi car will drive a little faster and not be as restricted. A UPS delivery vehicle will need to drive cautiously and make frequent stops.

4: More Accurate Visualizations

Currently, I'm not seeing sidewalks rendered appropriately. I also don't see all signs, poles, or objects being rendered. Not all vehicles are rendered exactly as they are. Here is a video of a train that was rendered as several different vehicle types going across.

The software should render it as a train crossing and treat it exactly as that. The software should treat a train crossing with an actual train very cautiously and slow down much sooner than the video showed. In addition, the software showed cars behind the driver in the video stacked on top of each other, which was incorrect. The better the reflection of reality, the greater chance of fully solving autonomy.

5: Handing of All the Edge Cases

The final test of level 5 full self-driving is going to be the ability to handle all the edge cases. I'm going to list quite a bit here:

  • What happens if the car has a tire that blows out. What should it do?
  • What happens if someone else crashes into the car, what should happen?
  • What happens if there is complete obstruction of the roadway on the route the car needs to take?
  • What happens if there is a family of ducks crossing the road - or other animals like a deer?
  • What happens if the weather is extremely dense like a blizzard and the car can barely see, what will happen?
  • What happens if there is an accident up ahead that occurs while the car is driving - how does it handle that?
  • What happens if there are no lane markings, what should the car do?
  • What happens if there is a police blockade, and the police are giving instructions - can the car handle that?
  • What happens if there is construction and the route ahead is blocked. Can the car turn around and go another way?
  • What happens if the car hits a dead end and it is difficult to turn around. What does the car do?

These are just some examples of corner and edges cases. There are many many more. The car is going to need to have some element of human-like decision making in order to handle these. It doesn't need to have a human mind or reach the level of a Turing Test, but it will need to be able to exercise some level of judgment.

I think there is going to need to be an element of programming for full self-driving that includes human code that teaches the car what to do in certain situations that are unnatural. Think of this as protocol to follow. An edge case module. Or a module for tricky situations. If the car is disabled or has a tire blow out, execute the red alert code, which puts on the hazard lights and calls a tow truck - that kind of thing.

What do you think about Tesla's current FSD software? Is it going to reach level 5 soon? If so, how many years will it take?

Leave your comments below, share the article with friends and tweet it out to your followers.

Jeremy Johnson is a Tesla investor and supporter. He first invested in Tesla in 2017 after years of following Elon Musk and admiring his work ethic and intelligence. Since then, he's become a Tesla bull, covering anything about Tesla he can find, while also dabbling in other electric vehicle companies. Jeremy covers Tesla developments at Torque News. You can follow him on Twitter, Facebook, LinkedIn and Instagram to stay in touch and follow his Tesla news coverage on Torque News.