Tesla Director of Artificial Intelligence Adrej Karpathy Explains Neural Net Development in FSD

Fans of All-Things-Tesla have been anxiously awaiting the latest release of FSD Beta version 9 and the promise of the magic it will hold for making true Level 5 FSD a reality and herald in a new era of automotive experience unlike any other.

Unfortunately, however, the information Tesla enthusiasts are wanting has been slow in coming and not altogether clear on why some changes to Tesla’s FSD capabilities have changed; nor, what to expect or when to expect Level 5 FSD will become a reality in production vehicles for the public. The good news is a lot of this is about to change.

Recently Tesla’s Director of Artificial Intelligence, Adrej Karpathy---who oversees development of Tesla Autopilot---revealed some of the things we would like to know about the where’s and why’s of what is going on with Autopilot and FSD while attending the 2021 Conference on Computer Vision and Pattern Recognition (CVPR) being held from June 19 to June 25 this year.

According to the Zoom video he speaks in, what is going on is really about neural network development in which meat computers are being replaced with silicon computers because not only can silicon computers respond faster than the human brain and body can react to input, but in time will also outperform with the correct response to visual input that will make driving safer.

The Real Reason Why Radar Was Dropped

We’ve all heard that seeing is believing. However, when other human senses come into play such as hearing, sound and taste, the final determination of what is going on at times might be confusing and not entirely accurate. In fact, the amount of sensation signal you are receiving---rather than supplementing the other senses---can actually result in a type of sensor domination where one sensation/sensor attempts to override the other sensation sensors and thereby your response to reality.

And, with artificial neural networks it’s not that much different. Hence, radar was eventually dropped because what the radar “saw” and what the camera “sees” at times became conflicting data that had to be processed, but was not always processed accurately because of this aforementioned sensor domination. The video identifies the problem primarily being based on depth sensing differences between radar and camera systems.

This is just one nugget of information explained in the Zoom video, but there is a lot more to it

Neural Network Simulations

The majority of the video goes into explaining how Tesla is approaching bringing silicon-based neural network control of Autopilot and FSD closer to reality by showing how that petabyte amounts of information are being collected and then used in simulation programs to enable Autopilot and FSD “to learn” how to respond to varying road conditions and driving scenarios.

Tesla reveals that out of 15 million miles worth of data used in recently modified scenario simulation, they have observed zero crashes from Autopilot in 1.7 million miles of that 15 million total of data. In the past with the legacy Autopilot system, they have observed about one crash every 5 million miles or so. However, Karpathy to his credit does acknowledge that since the latest simulations are ongoing and still developing the correct responses, that they do expect some crash scenarios of which the neural network will learn from.

Conclusion

The take-away message from the video is that Tesla Vision without the aid (or interference) of radar, is perfectly capable of accurately sensing depth and that simulation testing of enormous amounts of data Tesla is collecting will demonstrate that a silicon brain will become much safer than a meat brain when it comes to driving.

“…We (uniquely) have all of the puzzle pieces for FSD,” ends the Zoom presentation.

For More In-Depth Information, Here is the Zoom Presentation for Your Enjoyment and Education About What Tesla Has Been Up To:

Coming Up Next: Does Your Tesla Have This Feature?

Timothy Boyer is Torque News Tesla and EV reporter based in Cincinnati. Experienced with early car restorations, he regularly restores older vehicles with engine modifications for improved performance. Follow Tim on Twitter at @TimBoyerWrites for daily Tesla and electric vehicle news.