NVIDIA has demoed its ACE technology in an actual game for the very first time, showcasing AI-powered interactions in Mecha BREAK.
Mecha BREAK Is Getting NVIDIA ACE Support For Lifelike Interactions Between Players & In-Game Characters Using AI
Update [21/8/2024] - We wanted to clarify that the game will still include a full cast of real-human voice-acted characters. The ACE technology showcased by NVIDIA is a demo to visualize how ACE can be implemented in games. The game does support DLSS 3 support at launch as we reported here.
At Gamescom 2024, NVIDIA is once again demoing its ACE technology but this time, the demo isn't a hand-crafted experience like Covert Protocol or Ramen Shop but an actual game.
This is possible using the latest small language model for game characters known as Nemotron-4 which has 4B (4 Billion parameter) NIM. The small and less complicated models are easy to run on platforms such as gaming PCs and offer quick responses versus traditional LLMs (Large Language Models).
The Nemotron-4 4B NIM model was shown integrated within the very first title, Mecha BREAK by Amazing Seasun Games. The demo not only features Nemotron-4 but also leverages Audio2Face-3D NIM which is used to offer dynamic facial animations.
Besides these, there are also 3rd party models integrated within the game such as Whisper which provides speech recognition, and Elevenlabs which provides realistic voices (runs on the cloud).
ACE & digital human technologies showcased in first game, Mecha BREAK NVIDIA ACE is a revolutionary suite of digital human technologies that bring digital humans to life. ACE made its debut in the Ramen Shop demo at Computex 2023, but ACE has evolved rapidly.
At Gamescom 2024, we announced that its first digital human small language model is being demonstrated in Mecha BREAK, a new multiplayer mech game developed by Amazing Seasun Games, to bring its characters to life and provide a more dynamic and immersive gameplay experience on GeForce RTX AI PCs. Mecha BREAK, developed by Amazing Seasun Games, a Kingsoft Corporation game subsidiary, is showcasing the NVIDIA Nemotron-4 4B Instruct NIM running on device in the first showcase of ACE-powered game interactions.
And this is just the start, as more developers are incorporating NVIDIA ACE technologies into their games and applications. Global game publisher and developer Perfect World Games is advancing its NVIDIA ACE and digital human technology demo, Legends, with new AI-powered vision capabilities. Within the demo, the character Yun Ni can see gamers and identify people and objects in the real world using the computer’s camera powered by ChatGPT-4o, adding an augmented reality layer to the gameplay experience. These capabilities unlock a new level of immersion and accessibility for PC games.
via NVIDIA GeForce
Coming back to the game, Mecha BREAK features a 6v6 combat between various mechs that a player can choose from. Each mech has different loadouts and abilities so making the right choice matters a lot before you land in the action. That's where the engineer character in the hangar bay comes in who can assist the player with various things such as mission objectives, preferred mech choice, and more. The NVIDIA ACE technology and NIM models are applied to this segment of the game where you can interact with the character through voice recognition and have a life-like and responsive conversation on the go.
NVIDIA will have a full suite of ACE demos showcased during Gamescom 2024 along with the Mecha BREAK integration. We can't wait to try these out ourselves and we can expect more AI integrations for NPCs in future game titles too.
