Search

Hypercinema Week 3: Synthetic Media

This week we had to find an example of synthetic media. I had a tricky time finding a good example that wasn't super obvious. But I did find a service that I found interesting and novel. This is similar to the Artificial Talent's modeling agency example that was shown in class, but there's a company called Synthesia which helps you create and edit videos with with humanoid avatars reading typed up scripts used for advertising, training videos, help chatbots along with a myriad of other use-cases where you would want to provide educational media using an AI avatar.


This website allows you to type a script which the avatar will read, pick from a selection of avatars and scale it to fit in with whatever video content you need overlay the avatar onto. You can then clip, cut and edit the video however you need to to finish it as they provide simple video editing software to complete the videos on their website. The company is largely marketing their product for companies to create in-house educational content but it's recently been used to dub music videos into other languages. You could definitely use this software to create native language voice-overs for various types of video/audio content.


This is classified as synthetic media because it relies on AI to generate the audio and the video avatar. These avatars can be customized or be made to look like a specific person (with the consent of the person in question). Currently they offer 50+ languages in both female and male voices and provide a small number of options for "stock" avatars but it seems like more companies create "custom avatars" for their videos. To create custom avatars you have to record the person for a few minutes reading the script and the software will map the speakers facial expressions or even their actual face onto an avatar body.


I unfortunately am having a hard time finding out how the technology works for Synthesia and am not sure if they're using machine learning and various models to naturalize the voiceovers and the facial expression.


I think Synthesia is being very cautious when marketing their product by advertising it's use in small-scale educational content but it does look like they're branching out into advertising and voice-dubs for the entertainment industry. The first pro they mention about using these avatars is saving time and money in creating videos using actors or members of your staff who don't have any acting or script reading experience. Of course there can be many ethical ramifications when using this platform. For one, if you're creating a custom avatar using a real person or a real person's voice, those with access to that data could abuse it and use voices/facial for creating malicious content.

The company does say that they review every video before it's released to make sure nobody is abusing their platform, but that system won't scale in the future when the platform becomes more popular.


If used in the film industry, you could argue that this technology could put voice-over actors out of business in the future, and could take a chunk of opportunities away from actors being hired by these companies to create these videos.


Furthermore, the company currently only offers a handful of "stock" avatars that you don't have to pay extra to use. They avatars are likely based on real people, but the ethnic, racial and age diversity of said people will always be lacking.