Lip-Sync Fuse Character

Snebjorn

Will try and hack something together tomorrow!

UnityGames

Also, how does salsa support facial expressions? I see random eyes and some lip sync but can you change the facial expression as well? Thanks!

Snebjorn

I believe there's also support for facial expressions, but I haven't tried using that part of the system myself, so can't really comment on it.

Sorry, couldn't find the time to put a lipsync demo together today - Other Stuff[TM] got in the way, but I'll get something online very soon.

Snebjorn

@UnityGames : here's a quick webplayer demo I threw together almost from scratch in about 90 minutes (including the brilliant voice acting, ahem)  showing Salsa lipsync using Fuse characters. 

http://www.spectralstar.com/Test/AC_SalsaDemo.html

UnityGames

Wow, great demo!! Funny stuff!  I just need to figure out how to use facial expressions as well. The lip sync is pretty good! If you have any facial expression ideas let me know. The blend shapes are there of course but the question is if they can be used while playing a audio file in some way.

Snebjorn

My dear son inforrms me that it's not a demo, but a "click experience", because "being completely pointless, it can't be a point and click adventure, and calling it an adventure would probably be lying" ...

 :-)

Anyway, I'm currently working on a VERY short actual game to establish a good workflow for integrating some of my homemade tools, and it's going to be using Fuse characters throughout, so I'll be experimenting a bit more to see what's possible in terms of using those ready-made blendshapes. I'm a big fan of anything that saves just a bit of time!

Snebjorn

BTW, to get good lipsyncing with this on-the-fly system, it's quite important to have all voice audio files normalised to the same level (always a good idea, come to think of it) so you get consistent mouth shapes. A number of parameters can then be fine-tuned for individual characters for optimal results.

Dor_Yahud

Hi, i tew need to use lip sync on a fuse character.. i have an idea, didn't try it yet. What if i use a blend tree , with only the head in the character's avatar , and use the generated phonims to translate it to values of some floats or integers, that changes from one facial expression to another in the animator? (maybe i can add the volume of the speech to the parameters as well..)
How do i access and get the generated phonims from AC? I belive that the generated phonims from the ' from speech ' option, will do...

Dor_Yahud

By the way, can i use phonims in other languages like Hebrew and others that use a difrefrent set of letters?
Can i make multiple phonims sets?

ChrisIceBox

By 'from speech', do you mean the "From Speech Text" lipsyncing method?  The lip sync data for a speech line can be gotten with this function.

AC.KickStarter.dialog.GenerateLipSyncShapes

The parameters that it takes can be found in the scripting guide which I've linked to.   It will return a List of LipSyncShape data, each of which contains a frame and time index.

AC's Speech Manager only allows for one set of phonemes, however you could modify them through script when your language is changed by the player.  They can be accessed with this variable:

AC.KickStarter.speechManager.phonemes