Getting Rogo LipSync to work with AC

Gibbagobba

Since all of the free syncing methods have failed me, I decided to try out Rogo LipSync (Lite version in this case as it just supports phonemes). However, despite the clip preview showing the syncing working correctly on the model, it doesn't play when running. I've named all the files as the speech manager requires, tried exporting the .asset file as a .txt instead, tried disabling auto-naming and just plugging the audio and sync data directly, and nothing seems to work. Is there a certain way the LipSync component needs to be configured or am I missing something else?

Gibbagobba

For further explanation, this is the console output I get:

No LipSync data found for Player, line ID 121

 -> AC debug logger

UnityEngine.Debug:LogWarning(Object, Object)

AC.ACDebug:LogWarning(Object, Object) (at Assets/AdventureCreator/Scripts/Static/ACDebug.cs:29)

AC.RogoLipSyncIntegration:Play(Char, Int32, String) (at Assets/AdventureCreator/Scripts/Static/RogoLipSyncIntegration.cs:91)

AC.Speech:.ctor(Char, String, Int32, String, Boolean, Boolean) (at Assets/AdventureCreator/Scripts/Speech/Speech.cs:142)

AC.Dialog:StartDialog(Char, String, Boolean, Int32, Boolean) (at Assets/AdventureCreator/Scripts/Speech/Dialog.cs:128)

AC.ActionSpeech:StartSpeech() (at Assets/AdventureCreator/Scripts/Actions/ActionSpeech.cs:486)

AC.ActionSpeech:Run() (at Assets/AdventureCreator/Scripts/Actions/ActionSpeech.cs:120)

AC.<RunAction>c__Iterator2:MoveNext() (at Assets/AdventureCreator/Scripts/ActionList/ActionList.cs:342)

UnityEngine.MonoBehaviour:StartCoroutine(String, Object)

AC.ActionList:ProcessAction(Int32) (at Assets/AdventureCreator/Scripts/ActionList/ActionList.cs:306)

AC.ActionList:ProcessActionEnd(ActionEnd, Int32, Boolean) (at Assets/AdventureCreator/Scripts/ActionList/ActionList.cs:522)

AC.ActionList:EndAction(Action) (at Assets/AdventureCreator/Scripts/ActionList/ActionList.cs:435)

AC.<RunAction>c__Iterator2:MoveNext() (at Assets/AdventureCreator/Scripts/ActionList/ActionList.cs:404)

UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)

This is from using a Rogo LipSync asset file. This was the same with both auto-naming and without as well as changing the asset to a .txt.

ChrisIceBox

See the end of the "Extracing phonemes" section in the Manual's Lip Syncing chapter: RogoLipSync files should rename .asset files - only Papagayo, Pamela and SAPI files need to be renamed to .txt.

If you already tried it with .asset, did you manually assign the files in the Speech Manager when auto-naming was unchecked?

What version numbers of AC, LipSync and Unity?

Gibbagobba

Yes, I tried manually assigned the audio and .asset files when auto-naming was unchecked.

AC version is 1.60.7

Unity version is 2017.1.1f1

LipSync Lite version is 1.401

Gibbagobba

I just updated Unity to 2017.3.0f3 and the issue still persists.

ChrisIceBox

I'm afraid I can't recreate - if I import both, enable RogoLipSync in AC's Speech Manager and manually drag the "Gettysburg" asset into one of the Player's speech lines, it doesn't throw an error when the line is played in-game.

If you check auto-name, you should be getting an additional error of the form:

"No lipsync file found.  Looking for 'Resources/(file name')"

Were they appearing as well with the same filepath as you'd placed the files in?

Gibbagobba

Yeah, this was happening in both situations.

ChrisIceBox

The additional error should only appear if "auto-name" is checked.

Try using the 3D Demo as I did above:

1. Load the 3D Demo via the link in the top toolbar
2. Drag TinPot (from /AdventureCreator/Demo/Resources) into the scene Hierarchy
3. Attach the LipSync component to TinPot's base, but don't configure it
4. View the Speech Manager and uncheck Auto-name speech audio files?
5. Set Lip Syncing to Rogo Lip Sync
6. In the list of "Game text", find a speech line to test with (e.g. the "I can't cut that" line that plays when you use inventory on an unhandled Hotspot), and assign the "Gettysburg" asset in the Lipsync file field
7. Run the demo scene (unpausing when LipSync throughs a "No BlendSystem Set" error) and get that line to play

In my testing, the lip-sync won't run (because the LipSync component is unconfigured) but the Console won't report it as missing.  What happens for you?

Gibbagobba

For me, the audio did play and Tin Pot moved his mouth to it, but I also got these console outputs:

The 'RogoLipSyncIsPresent' preprocessor define must be declared in the Player Settings.

etc.

NullReferenceException: Object reference not set to an instance of an object
RogoDigital.Lipsync.BlendSystemUser.OnDestroy ()

If I then define Rogo in the scripting symbols, I get the 2nd message after playing and this one during:

[LipSync - Tin Pot] No BlendSystem set.

UnityEngine.Debug:LogError(Object)

RogoDigital.Lipsync.LipSyncBase:Awake()

I'm also noticing what I think is a bug where the default Gettysburg asset shows up with a Rogo thumbnail, so it's being recognized, where any asset I save from the editor shows up with a generic thumbnail. Yet, plugging it into the lipsync file field still labels it as LipSyncData. If I use this file instead, the console tells me that know data is found for that line.

Gibbagobba

Reported this to Rogo support yesterday and they confirmed that it was a bug with LipSync Lite. You have to select your saved asset, set the inspector to debug mode, and change the "script" variable from "LipSyncDataBase" to "LipSyncData.cs," then set inspector back to normal. They also said that a fix should come some time tomorrow or so, so this solution will be outdated by then.

ChrisIceBox

OK, so this is resolved?

The audio+animation will play on Tin Pot because its already set in the 3D Demo Managers (the animation is done with a separate script, not the standard LipSync tools).

The "No BlendSystem set" is the error I warned about in 7, and can be ignored for testing purposes (just getting AC to recognise the file).

Gibbagobba

Yup, though I would lastly like to know how exactly AC integrates with Rogo for 2D lip sync and expressions?

ChrisIceBox

On a technical level, there's no separation between the implementation for 2D and 3D: AC will play the asset file on any LipSync component it finds on the character.

You'd need to use Sprites Unity Complex animation because you'll need separate control over the head.  I think the best way to do it is to have a separate head sprite for each facing direction (Head_L, Head_D, etc), and then animate the SpriteRenderer's enabled state, instead of changing the sprite of a single object.  That way, you can place a LipSync component on each head, and they'll all be played with each line of speech.