At NIO, NOMI is the physical embodiment of the joy we hope to bring our users. We offer NOMI in all our cars, and most of our users choose to welcome her along their journey. She’s a helpful companion to our users as a small ball-shaped dashtop device in the form of a head and face. She was designed to imitate the subtleties of human interaction. 

At NIO, we are keenly interested in the relationship between machines and people, especially the relationship between a brand and the user. We started by thinking about our relationship between future users and the car as a friend. Beyond offering high quality vehicles or providing the best automotive services, NIO is about building lifelong relationships with our users and creating an emotional connection and attachment throughout the whole vehicle life. This is what inspired our fundamental conceptualization of NIO. 

In considering future-facing technologies like voice commands, functionality and relationship building are our top priorities. However, our company founder, William Li, spotted the disconnection. If the car is truly a companion to the user, why must the user shout into thin air when communicating with the vehicle? Typically, when talking to another person, they look at each other – they’re not talking to the air.

Why the decision to make NOMI move?

A companion is mobile. In our view, since the car moves, so should the in-vehicle companion. The car not only has to talk to you, but you also have to feel it. Life is not static. There are different ways to make the system seem alive. You could have a graphic that pops up when you talk and then goes away, but we felt that didn’t build that companionship. We didn’t just want a digital figure to represent the relationship; we wanted the system to feel alive and to feel part of the car, so we came up with this head and face aritifical intelligence helper live on the dashboard in the vehicle.

When we started developing the system, there was a feeling that having NOMI move and talk could alarm people. There was an idea that it would just move, gesture in a way, and make sounds to acknowledge that it was listening. Eventually, we figured out that talking was necessary to help it achieve functionality. Otherwise, it would just be a cute device that didn’t add much value.

How important is natural movement?

In the beginning, we wanted perfect, resembling human head movements, so in the latest NOMI Mate, there are two motors: one motor controls the horizontal rotation, and the other controls the vertical. When combined, they create 3D motion. To achieve the natural feel, we looked to the film and animation industry to see how we could design the whole character so that the movements are not simply robotic and mechanical but driven by emotions. We invented hardware allowing tiny, almost invisible movements – the motors, the belts, all of that – with the action determined with millisecond accuracy.

How important is the software element of the system?

The essential software that enables the system is called our emotion engine. We developed our software stack with active speech recognition, which is the front end. In the middle, we have natural language understanding. Some of that is processed in-vehicle, but most are done in the cloud. That part of the system is updated and improved almost daily. The final component is how the system speaks to you, the natural language generation. The software makes continuous improvements via Firamware Over the Air (FOTA) updates. 

What are the boundaries of NOMI’s involvement in the driving experience?

Initially, we set a clear boundary where NOMI couldn’t execute driving-related and safety-critical tasks. For example, the system cannot change the drive mode. You don’t want a child or other passengers to say, “Change drive mode,” and then that changes. The driving mode alters the pedal force and changes the steering wheel to interfere or impact your driving behavior.

There are also some other subtleties. For example, NOMI does not operate the tailgate. However, this year we have come to understand that some constraints are unnecessary, and we can work out ways to add functionality. For example, hopefully, we can talk to NOMI and say, “Drive for me,” which would activate the Navigation on Pilot (NOP) driving functions.