Sony Experiments With AI Technology Using Aloy From The Horizon Franchise

In a now deleted YouTube video, Sony demonstrates its new AI software. The demonstration used Aloy from the Horizon franchise and shows her interacting with a PlayStation staff member.

The report came from folks over at GameSpot who was able to spot the YouTube video. According to them, the video was narrated by Sharwin Raghoebardajal, a software engineering director at Sony Interactive Entertainment. Raghoebardajal could be seen interacting with AI Aloy with the AI answering his questions.

According to The Verge, the demo uses Open AI’s Whisper for speech-to-text and both GPT-4 and Llama 3 for conversations and decision making. According to Raghoebardajal, Sony has its own Emotional Voice Synthesis system it uses for speech generation as well as an audio to face animation software powered by Sony’s Mockingbird technology.

Apparently, Sony has already experimented with this technology by running parts of it on PS5 consoles with “little overhead,” according to Raghoebardajal. It’s currently unclear what the AI is made for.

This development of this AI can be seen as problematic. Sony using Open AI’s Whisper technology might be seen as a threat by voice actors. In the demo, AI Aloy’s voice doesn’t use Ashly Burch’s voice. This might be due to contracts that protect the voice actress from having their voice be used on AI projects. Ashly Burch has shown support of the SAG-AFTRA 2023 strike which aimed to protect voice actresses from AI.

The YouTube video was taken down because of a copyright claim by MUSO TNT LTD. It’s unknown whether the video will be uploaded again.