Tesla Patents "Supplementing Vision-Based System Training With Simulated Content" For Its FSD Software

Work for Torque News, follow on Twitter, Youtube and Facebook.

Tesla just filed a patent that will supplement its vision based software system with simulated data. This patent includes a variety of things, including the complete process of how simulated data is used.

Tesla Patents Simulated Data for Its Vision Based FSD

Tesla is known for creating patents. One patent I can remember writing about a while ago was for a laser based system to clean the windshield of a car. Tesla has been very quiet about the progress (or lack thereof) of this patent, but recently submitted a new patent for something else.

This patent is a 21-page document that was filed on May 19, 2023. I first heard the news about it today, however, and it appears that this is because the publication of this patent is today (November 23, 2023).

The inventors included on this patent are Tesla team members that likely work at Giga Texas and are:

  • Matthew Wilson, an Autopilot software engineer at Tesla
  • Artem Brizitskiy, a staff artist at Tesla
  • Yazan Haddedin, a former senior software engineer at Tesla
  • Charles Henden, a manager on Autopilot simulation training at Tesla

The abstract of the patent reads:

"Systems and methods for training machine learning algorithms utilized for autonomous driving. An example method includes creating driving environments based at least on vision based image data received from a vehicle simulating driving a virtual vehicle by importing driving environment scenario; in response to completing the simulation, analyzing simulated results; determining optimized driving parameters associated with the driving environment scenario; and generating a set of machine learning algorithms training data based on the determined optimized driving parameters. The driving environment scenario is based at least on the created driving environment."

That was a mouth full, but seems to encompass creating simulated environments that will help train Autopilot and FSD on not just real world video, but computer generated video.

I think it could also mean that Tesla is going to take real videos from the real world and then make a variety of additional simulations from that real world video and train FSD on those variations too.

The process is as follows:

  1. Start collected vision and simulated content processing routine
  2. Obtain collected vision information
  3. Obtain generated simulated content information
  4. Process obtained information
  5. Data combination
  6. Combined data processing
  7. Training and updated machine learning model
  8. End

There are additional documents in the patent that include how the architecture is run in order to make all this happen.

The Reason For This Patent

Tesla is creating this patent because I think they want to control the system that trains how a car should drive itself so that other companies license their technology, rather than trying to copy the technology.

It's not clear if Tesla is going to eventually end up using simulated data, but I think they want to be the first to get there, just in case it is an important part of getting FSD to level 5 autonomy.

Elon Musk even mentioned that FSD should be able to avoid the debris of a crashing UFO in front of the car - and I think this would require simulated data to do - that is unless there are actually UFOs...

Tesla may have found a way to accelerate the training of their FSD software using simulated data, which is good news for FSD getting regulatory approval for driverless use. Let's hope that comes sooner than later!

I'll be sure to update more as I learn about this patent, in the meantime, check back later for more Tesla news!

What do you think about this patent from Tesla - is this a smart move to protect Tesla and keep their technology proprietary?

Share this article with friends and family and on social media! Thank you so much for your support!

Hi! My name is Jeremy Johnson, and I am a Tesla investor and supporter. I first invested in Tesla in 2017 after years of following Elon Musk and admiring his work ethic and intelligence. Since then, I've become a Tesla bull, covering anything about Tesla I can find, while also dabbling in other electric vehicle companies like Aptera. I cover Tesla and EV developments at Torque News. You can follow me on X.COM or LinkedIn to stay in touch and follow my Tesla news coverage. Image Credit, Tesla, Screenshot