What was Tesla’s process for establishing the “GPT moment” in smart driving technology?

Author | Zhou Yongliang

Editor | Jingyu

On May 15, in response to the news that the paid option rate of Tesla FSD (Full-Self Driving) was only 2%, Tesla CEO Musk responded on the X platform that the actual situation far exceeded this number.

This response was made in response to Gary Black, a well-known Tesla investor, who questioned the future acceptance rate of FSD. According to data provided by credit card data provider YipitData, among American Tesla owners who tried FSD for a month, only 2% of users chose to continue subscribing to the service, far lower than the expected 6%.

Black believes that this may be because the added value of the FSD service is not high, or the subscription price ($99 per month) is too high. He called on Tesla to seriously evaluate these factors to better cater to the needs of car owners.

Meanwhile, Musk’s visit to China at the end of April sparked discussions about the implementation of Tesla’s FSD in China. However, according to China Daily, despite Tesla’s proposal to launch “driverless taxis” in China, the Chinese government has not yet fully approved the full implementation of FSD in China, and may first support its testing and demonstration in China.

So, what is the actual experience of Tesla FSD, which is at the center of the storm? Will it trigger the catfish effect again?

01、Intelligent driving ChatGPT Moments

As new energy vehicles enter the second half of the intelligent era, intelligent driving has always been one of the technological high grounds pursued by the industry. People are not only concerned about the passing rate of driving scenarios and commuting efficiency, but also more concerned about whether intelligent driving is more humane.

The previous V11 version of Tesla FSD, like many intelligent driving systems, had a rigid speed control, and would suddenly brake or accelerate when dealing with emergencies, with a distinct mechanical feel. Especially in special circumstances such as narrow roads or bad weather, this rigid speed control would make users feel uneasy.

But now, the emergence of Tesla FSD V12 has changed this situation. Zhihu blogger “EatElephant”, who has experienced Tesla FSD V12, said that one of its biggest advantages is its anthropomorphic ability. The biggest improvement of the V12 version is the smoothness of speed and steering control. Even if passengers sit in the back row, they can hardly feel any frustration when starting and stopping at traffic lights and turning at intersections.

Second, V12 has made great improvements in handling some unstructured scenarios (such as the lack of lane lines and traffic rules), such as the timing and amplitude of turns, the specific degree of deceleration, etc. For example, in the V11 version, when the vehicle encounters a vehicle turning left in the distance ahead while driving straight, the intelligent driving system responds by significantly decelerating. Although this can avoid the risk of collision, the deceleration is usually too large, resulting in a noticeable sense of frustration and increasing the risk of being rear-ended by the vehicle behind.

In the V12 version, the system seems to be able to more accurately judge the route and speed of the vehicle ahead in similar situations. Therefore, it can slow down at a very appropriate speed, making it almost imperceptible to the passengers in the car, while leaving enough safety distance to avoid unnecessary discomfort and the risk of rear-end collision with the vehicle behind.

third,V12's ability to cope with various scenarios has been significantly improved, greatly reducing the number of times manual intervention is required. For example, Tesla FSD V12 can not only identify and bypass obstacles such as iron sheets on the road, but also drive on temporary roads according to guidance signs during road construction, just like a human driver. Even if you need to drive into the opposite lane, you will not be restricted by the reverse sign and the center yellow line. When driving at night, it can turn right like a human and drive between two parked cars on the side to cleverly avoid oncoming cars. After the oncoming car leaves, it turns left and continues to move forward, showing excellent driving skills.

At the same time, Zhihu blogger “EatElephant” also mentioned that FSD V12 demonstrated some capabilities similar to the emergence of wisdom. During the test, the vehicle encountered such a problem: there was a lane in front of the intersection, and the V12 could not use the reverse gear to complete the U-turn. While the passengers were waiting, the V12 found a small parking lot and decisively changed the route to bypass it, trying to replace the standard U-turn. Although the system prompted to take over at the last exit, this behavior was a major breakthrough in autonomous driving technology, because the system usually strictly follows the navigation route, and the behavior of deviating from the navigation on its own is almost unacceptable.

These improvements are also reflected in the number of manual interventions. Compared with the previous version,The V12.3 version has significantly increased the non-critical takeover mileage in urban environments, from about 100 miles (about 160 kilometers) to 386.7 miles (about 622 kilometers).

In contrast, Xiong Lu, professor at Tongji University and vice dean of the School of Automotive Engineering, once said that autonomous driving companies in Beijing, Shanghai, Guangzhou and other places,On average, the vehicle needs to be taken over 3.5 to 10 times per 100 kilometers. Basically, it needs to be taken over manually every ten or dozens of kilometers..

At present, Tesla FSD may be having its own “ChatGPT moment”, although there are still some shortcomings. For example, the system sometimes makes people feel too close to the roadside when driving, which makes people a little worried; the recognition and avoidance of oncoming vehicles are not timely enough, such as the reaction when encountering fire trucks is not sensitive enough; the recognition of some special road signs or lanes may be wrong, and there are also some problems with the recognition and avoidance of potholes.

However, this does not prevent Tesla from significantly increasing its investment in smart driving. It is understood that by the end of this year, Tesla plans to invest a total of $10 billion in autonomous driving technology. Considering that the total expenditure from 2016 to 2023 is around $2 billion, this means that Tesla's expenditure on smart driving this year will reach around $8 billion. This is a huge investment, which shows Tesla's determination to further improve FSD technology.

02.How was FSD V12 created?

The significant improvement in the experience of Tesla FSD V12 is the result of the gradual convergence of its technological path.

Tesla has been leading the development of smart driving technology since the launch of the FSD function in 2020. Unlike the traditional method of relying on lidar and high-precision maps, Tesla insists on using pure vision technology, enabling vehicles to better understand the surrounding environment.

In the following two years, Tesla announced a large number of solutions for FSD technology through a series of activities, such as AI Day. These solutions involve concepts such as data closed loop, shared backbone network, BEV perception, and network occupation. Although these technologies are at the forefront of the industry, previous versions have been criticized by users for not significantly improving the user experience.

However, as FSD has developed to the V12 stage, the situation has changed. Compared with FSD 11, the biggest change in FSD V12 is the use of end-to-end neural network technology (End-to-End Neural Network, E2E NN). This technology enables the system to better understand and handle complex driving environments, reduce driver intervention, and improve the accuracy and automation of autonomous driving.

In the past, the basic process of FSD usually includes three stages: perception, decision-making, and execution. In early versions, the perception stage needs to obtain information about surrounding objects through vision or radar, and identify and classify them, while the decision-making stage relies on pre-written control rules.

However, in FSD V12, end-to-end neural network technology is used, and these steps have undergone a revolutionary change: the perception stage no longer requires manual identification and classification of objects, and the decision-making stage no longer requires pre-written control rules. The system only needs to learn through a large amount of video input to let the neural network make the right decisions in different situations. This allows Tesla to reduce a lot of code in FSD V12, making the system lighter and more flexible, while also being able to operate normally in unfamiliar environments even without a network connection.

Musk said at the end of last year that Tesla's FSD Beta V12 was not programmed from beginning to end.No programmer wrote a line of code to identify concepts such as roads and pedestrians. All was left to the neural network to think for itself. There are only 2,000 lines of C++ code, while V11 has 300,000 lines.

In fact, the end-to-end model is not a new concept. It has been proposed before. However, many people have doubts about the interpretability and reliability of neural networks. Although the end-to-end system has improved the upper limit of the model's capabilities,However, it also magnifies the unexplainable nature of neural networks as “black boxes”, which will bring difficulties to R&D iteration and problem solving.Big challengesTherefore, many companies dare not try it easily.

Tesla FSD shows everyone the potential of end-to-end models. Of course, this process is not achieved overnight. Tesla has always emphasized the “purity” of end-to-end in its autonomous driving technology. Starting from version V10.9, they removed the post-processing code for lane line perception and changed it to direct output of lane lines by the model. At AI Day, Tesla also demonstrated how to introduce learning trajectory generation and neural network decision models into the planning and control module.

However, Tesla's technology updates are mainly concentrated in the middle and upper reaches of the technology stack, such as perception and prediction, while the decision-making and planning modules that control driving functions have rarely changed. This has led to a lack of significant improvement in user experience.

The major breakthrough of V12 is that it opens up the last link of the entire technology stack (decision-making planning), enabling the system to be end-to-end data-driven, ultimately achieving more natural and intelligent driving behavior.

As V12 adopts an end-to-end technical architecture and directly optimizes the planning control output, the improvement of user experience will be faster. In early May, Musk announced that Tesla's FSD system will soon usher in three major update versions, namely V12.4, V12.5 and V12.6.

Among them, version V12.4 is expected to be launched in mid-May. This version will fully update the model training to improve the accuracy and reliability of the system. Secondly, in response to the problems of excessive acceleration and excessive braking reported by users, V12.4 and subsequent versions will focus on optimizing driving comfort, thereby improving the driving experience of passengers.

Back to the beginning of this article, Musk's visit to China seems to make the “landing of FSD in China” more credible. There are even rumors on the Internet that BYD and Tesla will cooperate on FSD, which will further release the potential of FSD in China – what will be the result of the cooperation between the two largest new energy vehicle companies in the world?

If the rumors come true, will Tesla FSD change the current situation in the field of intelligent driving? How will the intelligent driving “story” of Chinese automakers continue to develop?

This is probably a question that everyone fantasizes about but dares not ask.

Advertising Statement: The external jump links contained in the article (including but not limited to hyperlinks, QR codes, passwords, etc.) are used to convey more information and save selection time. The results are for reference only. All articles in Gamingdeputy contain this statement.