PlayStation Is Secretly Working On AI-Powered Characters

The game industry continues to integrate artificial intelligence into the game development process. Some of the biggest players, such as EA and Microsoft, have demoed some of the tools being worked on internally that the companies hope to use in the future. Now, it seems another gaming giant, Sony, is also developing a prototype tool for creating AI enahnced characters.

An internal video has been leaked showing Sharwin Raghoebardajal, a director of software engineering at Sony Interactive Entertainment, demoing the prototype. In it, Raghoebardajal has a real time conversation with Aloy, the protagonist from the Horizon franchise. When asked how she’s doing, Aloy responds with "Hello, I'm managing alright. Just dealing with a sore throat. How have you been?" The conversation continues with him asking Aloy several other questions that she readily answers in a conversational tone.

As expected with a prototype, the demo is still rough around the edges. Aloy’s voice sounds vaguely like it does when actually playing the games, though it's far more robotic. Moreover, the facial animations are fairly stiff, not landing anywhere near the cinematic performance expected from a AAA game experience. Raghoebardajal stresses that “this is just a glimpse of what is possible.”

Sony is taking advantage of several AI technologies to accomplish this. It’s using OpenAI’s Whisper text-to-speech tool to process the speech, then ChatGPT and Llama 3 to create the character’s dialogue and enhance its decision making. It then brings it all together with the company’s own Mockingbird technology for facial animations.

While industry peers have been more open about the work being done on these AI tools, Sony didn’t necessarily want this information to get out -- at least not yet. The video has been pulled from YouTube after a copyright complaint was filed. Likely because AI is a hot button issue in the gaming community.

What all these demos, public or in this case not so public, have shown is that it’s still early days for AI in game development. There’s lots of great potential, but it could hinder the experience and put some people out of work.