Mercedes-Benz brings Augmented Reality to its service department

Article content

Imagine this: You’re a technician working on an over-engineered, complex vehicle with a one-of-a-kind problem that hasn’t been seen in your shop, or at least recorded before. Where do you start? The first thing you do is glance down and tap your wrist. A menu pops up, floating in the air, awaiting your input.

Article content

You tap a selection or two, and there’s a call in the corner of your eye with another expert who says they’ve seen this kind of issue before. He sees everything you see, and can circle the trouble spots, guiding you past any time-wasting red herrings. A few more taps in the air and you get the wiring diagram in your field of vision, helping you work on the vehicle with this reference with you the whole time.

We apologize, but this video has failed to load.

Try refreshing your browser, or
tap here to see other videos from our team.

Mercedes-Benz brings Augmented Reality to its service department Back to video

What used to be a frustrating, lengthy affair for both customers and technicians has been shortened considerably thanks to fancy new technology. It’s no longer science-fiction — it’s (augmented) reality.

As cars get more complex, the process to repair them will get more advanced as well. Mercedes-Benz Canada is hoping to be ahead of the curve when it comes to dealing with complex repairs with a high-tech augmented reality headset that is now used in every one of the brand’s dealerships across the country.

Article content

There are two parts to this futuristic solution: The Microsoft HoloLens 2 headset; and the Microsoft Dynamics 365 Remote Assist conferencing and collaboration system. The HoloLens 2 is a wireless augmented reality headset, featuring four visible light cameras for head tracking; two infrared cameras for eye tracking; a depth sensor; five-channel microphone array; and built-in speakers. Together and in the hands of Mercedes’ technicians, they make up Mercedes-Benz Virtual Remote Support.

The technology was piloted back in 2019, as a way to address challenging vehicle repairs. Typically, when a customer comes in with a puzzling issue, it can take a long time to diagnose the issue, let alone get the right parts and apply a permanent fix. Lengthy wait times aren’t what Mercedes-Benz customers signed up for when they bought the brand’s latest piece of metal, and the delays in returning a car into the hands of an owner could turn off a customer for life.

Article content

How It Works

When a car comes in with a rarely seen, out-of-the-ordinary issue, Mercedes used to fly specialists to a dealership to check it out. This meant it could take a while before a car was seen, diagnosed and fixed, never mind the costs and environmental impacts associated with flying all over the country.

With this technology, they can collaborate remotely and assist in the repair. The local technician dons the headset and dials his support team, showing and and telling them the problem in real time. The device can broadcast the technician’s view to other specialists and experts, who can speak to and hear from the technician on-site. It’s like a fancy Zoom or Microsoft Teams Meeting, except instead of sharing your screen, you’re sharing what you see . The headset can also project vital information like wiring diagrams and schematics so the technician can work on the vehicle at the same time.

Article content

“It’s like having the right expert over your shoulder the minute you need them,” said Joseph Lagrasta, Shop Foreman at Mercedes-Benz Brampton. “Since deploying this technology in July, we are getting cars back to the owners – diagnosed, fixed, and ready to roll – in a fraction of the time.”

It’s the result of a pilot that started back in 2019, with Mercedes-Benz Canada looking into how it could address these challenging vehicle repairs.

“It’s one part of a massive digital transformation involving investment into our dealerships,” said Nikhil Ondhia, VP of Customer Services, Mercedes-Benz Canada. “The investment into HoloLens and Remote Virtual Assistance is part of that. The product sells a customer the first car, but service sells the second, third, and fourth ones.”

Article content

While HoloLens may seem like a weird science experiment going on behind the scenes of the dealership, Ondhia explains that the technology will directly impact the service department and customers. His gut feeling is that the technology will improve efficiency and turnaround times in the service department by as much as 30 per cent.

“We can’t add service bays to a dealership,” he said. “But with tech like this we can get cars and customers in and out quicker, especially those tougher situations which had people waiting days for a flying doctor to come in.”

How Complex Are We Talking?

Some of the more skeptical readers may be asking “Why won’t Mercedes just make more reliable vehicles, rather than spend all this money on fancy gadgets?”

Article content

With the recent trend toward building electric vehicles, the HoloLens solution may be a smart move going forward. While automakers may be quick to point out that EVs have fewer mechanical components and will be less complicated to repair, they have a ton of code and wiring, which requires a completely different approach to repairs than combustion-powered vehicles. Looking through the wiring diagrams hands-free while working on the vehicle should make things easier for the technicians.

Learning and Growing

The technology is pretty intuitive — even I was managing to get the menus and calls going within minutes of putting the headset on. That’s the beauty of it, and Ondhia thinks it will shift perceptions in the service industry. “Moving the focus of the trade from mechanics to technology will open things up considerably,” he said. It dispels a stereotype about technicians. And sends a signal to those interested in cutting-edge technology.

Mercedes-Benz thinks this futuristic technology has applications beyond the service department. It’s going to use it for sales training and other venues to better arm its dealerships to deal with customers, repairs, and vehicles. The tech may even make it to the consumer, letting them look at the vehicle they’re building right there in the showroom.

Augmented Reality

Article content

This is just the latest showcase of augmented reality from Mercedes-Benz. Currently available on the brand’s cars is the augmented navigation system, which broadcasts navigation instructions on top of a video feed from a front-facing camera on the dashboard. The technology leapt from the dashboard to the windshield with an augmented reality head-up display on the new S-Class.

With HoloLens 2, that technology is extending beyond the car and into the service department, hoping to improve the experience for owners and technicians alike.