Mercedes-Benz announced the integration of ChatGPT into the vehicle system, the car can finally understand "human language"

Author: Vicky Xiao

Original title: Mercedes-Benz can play: Bring ChatGPT into the car, so that the car can finally understand “human language”

We knew ChatGPT was going to take everything by storm, but no one thought that you would be able to see it in your car so soon.

Mercedes has announced that owners in the US will be able to add a new luxury to their already luxurious vehicles tomorrow via an OTA update: ChatGPT. Owners of Mercedes-Benz cars equipped with the MBUX in-vehicle system only need to apply on their mobile phones, or say, “Hey Mercedes, I want to join the beta program” in the car, and they can activate the ChatGPT function from tomorrow, June 16.

Not to mention, Mercedes-Benz + ChatGPT are not related at first glance, but they can really produce a magical chemical reaction together. Not only can it answer simple preprogrammed commands like “turn up the temperature” or “forecast the weather,” but it can also engage in natural conversations on almost any topic, including contextual follow-up questions.

At the event held by Mercedes-Benz this week, Silicon Star was exclusively invited to experience the feeling of using ChatGPT in the car.

Photo by Silicon Star

First-hand experience: My car finally understands “human language”

After saying the Hey Mercedes password, you can ask it, “I want to bury my feet in the sand, can you recommend some nearby beaches?”

ChatGPT will return the names of some popular nearby beaches, and say that these beaches are good places to relax and enjoy, and you can choose one of them to start navigating.

You can continue to ask, “What’s the weather like at my destination?”, “What’s there to do in these places?”, “Are there sharks near these beaches?” ChatGPT can answer these questions, And, on request, tell a bad shark joke.

If you often use in-vehicle voice assistants, you know that most in-vehicle voice assistants are clumsy and can only understand a single command, and in many cases you have to use the simplest and most straightforward commands for it to understand , not to mention being able to connect with the context and have a natural conversation with you.

But this time, Mercedes-Benz uses its own “Hey Mercedes” voice assistant to integrate ChatGPT, which provides a completely different level of experience. You can say colloquialisms, such as “these” and “my”, which require context. You can have a continuous dialogue with it, and you can even say after it finishes telling a joke, “It’s not funny, tell me another one”, and it can fully understand and answer. Although, as in the video, occasionally there will be bugs due to unstable network signals, but most of the time, the dialogue is very smooth and natural.

And think about it, why do you need a “smart assistant” that can understand human speech more than driving?

Mercedes developers told Silicon Star that they are using Microsoft’s Azure OpenAI service to bring large language models to its Hey Mercedes in-car voice assistant. The current testing period of this service is three months, and it only supports English dialogue, and it is limited to owners of Mercedes-Benz cars equipped with MBUX in the United States for trial use. The upgrade and use are free.

They also told Silicon Star that after the official launch, they will continue to improve based on user feedback. They are still exploring the ChatGPT plug-in function to see if they can allow car owners to book restaurants or Book movie tickets. If the feedback is good, a wider and more permanent launch of this new experience will also be considered in the future.

It is not surprising that large language models such as ChatGPT came to the car, but they came so fast, but it was really beyond the expectations of Silicon Star.

Mercedes-Benz developers told Silicon Star that they started the development of this project in February. Although it is a very small team, the speed is very fast. And car owners don’t have to worry about privacy issues, “All conversations are only stored locally and in Mercedes-Benz cloud services, and we will not share these data with any third-party services.”

“I don’t even know what’s going to happen in the next nine months”

Markus Schäfer, Member of the Board of Directors and Chief Technology Officer of Mercedes-Benz Group AG, also chatted with Silicon Star.

Looking back on the integration of ChatGPT, Schäfer said that they held an investor meeting in Sunnyvale (the location of the Mercedes-Benz Silicon Valley R&D Center) in February, “At that time, I said that we would quickly integrate ChatGPT into the car. Come on, and we need to be very fast.”

He said that in the past two or three years, he went to many technology companies to discuss cooperation, such as Amazon and Google many times. Because Mercedes-Benz has attached great importance to cooperation with major technology companies since it decided to build MBUX, its own in-vehicle infotainment system.

"It’s like we built a foundation, built our own house, and there are many rooms for different application services and different partners. In this way, we can quickly adopt all new technologies. The MBUX system is universal Yes, this is a very flexible operating system, and each region can choose its favorite applications to add.” He said, for example, that Chinese developers are also joining in to create a speech recognition system for Chinese users. In addition, Chinese users like to order food in the car, sing K, etc., and they also provide corresponding services with their partners

"ChatGPT is something that people like, and it’s something that everyone is talking about, so we quickly added it to the car. There are other big language models in the world, etc., and we are also thinking about cooperation. I am most excited, Combining all these new technologies with electric technology. I have been in this industry for 30 years. In the previous 25 years, the car basically did not change, but in the past few years, new things and different elements have been added every day. Come in, turn into something new. I can’t even predict what’s going to be new on the car in the next nine months.”

When talking about another aspect of the car revolution - autonomous driving, Schäfer also said that Mercedes-Benz is making very fast progress in this area.

“There are many emerging auto brands in China. They are developing very fast and the competition is fierce. But we have a history of more than 130 years, and we have many stories to tell, but we are also maintaining technological innovation.” He said, "Mercedes-Benz It is the first car manufacturer to obtain L3 level autonomous driving licenses in Europe, Nevada and California. American car owners can start ordering cars equipped with this technology in the middle of this year. These cars are already on the road in Germany, and the feedback is very good. "

This means that after the Mercedes-Benz automatic driving system is turned on, the vehicle can drive by itself within the specified speed. The driver can send text messages, watch videos, and play games. If there is an accident, Mercedes-Benz will be responsible. From this perspective, Mercedes-Benz’s L3 system truly realizes “semi-autonomous driving”, while other brands such as Tesla’s Autopilot can only be regarded as “assisted driving”, and the driver is responsible for the accident.

Schäfer said, “We are the first car brand that can do this. Of course, safety is the first priority for us, and we will always monitor the performance of this automatic driving system.” He also said that he is now working with Beijing Communicate with relevant departments to see if they can get the L3 test license in Beijing. “Chinese consumers like new technologies very much, and we are also looking forward to more new technologies appearing on cars sold in China.”