Forum rules - please read before posting.

FMOD Integration

Hi Chris,
we are currently trying to transition to FMOD for the audio of our game. While the general transition isn't too complicated and I have also found custom actions for FMOD audio playback, the main challenge is regarding speech audio.
As Speech audio (especially playing it through IDs) is deeply integrated into AC, I wanted to ask if you are thinking about having this integrated as an option (similar to the adressable option). This could be another define (FMODisPresent), and would enable handling speech audio through the programmer instrument in FMOD.
If it's not something you are planning to include, what would be the best way to tackle this, making sure that things like the speech timeline track etc. still work?

Comments

  • edited December 2020

    It should be possible to override behaviour using events - you can still rely on AC's Speech Manager for assigning / defining the audio data itself.

    When speech is played, it triggers the OnStartSpeech event, which gives information about the character, text, and ID (tutorial here). This is triggered for speech present in Timeline as well as ActionLists.

    With a line ID number, you can get the associated SpeechLine class - and from there the relevant data (i.e. manually-assigned audio clip, or text via the Description box):

    private void StartSpeech (AC.Char character, string lineText, int lineID)
    {
        SpeechLine speechLine = KickStarter.speechManager.GetLine (lineID);
        AudioClip audioClip = speechLine.customAudioClip;
        // or:
        string audioData = speechLine.description;
    }
    

    Once you've gotten the audio data and used it to play it via custom means, you can then prevent AC from playing audio itself by unchecking Auto-play speech audio files? in the Speech Manager.

  • Ah nice, that seems like a great solution. Thank you!

  • @Onat First time post here. It's really nice to see that someone has gotten FMOD to work in AC! I have been searching and waiting for a no-code solution for years that will work with FMOD. It's almost entirely my only hangup with committing to an engine. I'm an audio engineer and can't imagine working on my own game without my "power tools".

    Can you tell any more about how it's working out for you? Was this, by chance, your starting point: https://github.com/duartegarin/Adventure-Creator-FMOD ? Or did you develop your own custom AC actions? Have you come across any issues other than your VO one? How did that resolve for you? Are you able to do all your calls to FMOD without writing any code? As a Unity noob and having no coding chops whatsoever, my biggest concern is hitting a roadblock here.

    Thanks in advance for your insight.

  • Hi @LMo sorry for the late reply. In fact I have just started with the implementaiton, as there were so many other things to do. The VO part seems to work already, It's been just a couple of days, but once I'm done with the implementation, I'll try to upload relevant parts of it in the wiki.

    @ChrisIceBox as far as I can tell, the FaceFx integration is not triggered by playing audio through a custom solution (maybe because of disabling auto-play speech audio files). Is there an official way to still keep all the other functionality, or something else that you would recommend?

  • You can use the FaceFXIntegration script's Play function to manually trigger a facial animation:

    FaceFXIntegration.Play (speakingCharacter, filename, audioClip);
    

    The script below may need to be adapted to suit your purposes, but here's how you'd extract the relevant information from a Speech class as it gets played:

    using UnityEngine;
    using AC;
    
    public class FaceFXExample : MonoBehaviour
    {
    
        private void OnEnable () { EventManager.OnStartSpeech_Alt += StartSpeech; }
        private void OnDisable () { EventManager.OnStartSpeech_Alt -= StartSpeech; }
    
        private void StartSpeech (Speech speech)
        {
            AC.Char speaker = speech.GetSpeakingCharacter ();
            if (speaker)
            {
                AudioClip audioClip = KickStarter.runtimeLanguages.GetSpeechAudioClip (speech.LineID, speaker);
                FaceFXIntegration.Play (speaker, speech.log.speakerName + speech.log.lineID, audioClip);
            }
        }
    }
    
  • Hi, is the above still up to date? And the wiki for FMOD? I am keen to try and integrate FMOD into my project, thanks

  • I can't speak for the Github link or the work of others, but my own comments here should still be valid, yes.

  • edited February 24

    Hello! I have been using Adventure Creator for a few months to develop a small 2D game and I am starting to integrate FMOD with AC. I come from an audio background and don't know much about code. I have been able to get FMOD to work with Unity in general (specifically attaching the Event Emitter/other FMOD components to Game Objects).

    I am attempting to integrate the code from the github links people have referenced above in order to trigger FMOD events through my AC action lists.

    Whenever I use the code below, my action list gets stuck - it does not "complete" the action of playing the sound, and in my debug where you can see which action lists are currently running, the action list stays there as soon as it is initiated and does not fall off the list like it would as it completed its run.

    If the sound was a second in duration, you would think that worst case the action list would not move forward to the next action until the sound completes. However even after the sound finishes it does not move on to the next action in the list. It seems like some sort of release is needed before the action list will move forward.

    I know this code is from 2016 so I do not know really what would have changed, but before I continue troubleshooting, I was hoping that that anyone who has successfully been able to integrate FMOD with AC Action lists to provide insight. Thank you! Below is the code I used from github.

    The error that I receive in the console is "NotImplementedException: The method or operation is not implemented." I have not been able to figure out what this means by googling, or if this provides me with an "obvious" answer of why it is not working.

    I am super new to all of this so I appreciate any help and your patience!

    `using UnityEngine;
    using System.Collections;
    using FMODUnity;
    using FMOD;

    if UNITY_EDITOR

    using UnityEditor;

    endif

    namespace AC
    {

    [System.Serializable]
    public class ActionFMODPlay : Action
    {
        // Intensity of the emitter
        public float intensity;
        // Event Path
        public string eventPath;
        // Event Instance
        public FMOD.Studio.EventInstance backgroundMusic;
    
        //Constructor defining the action
        public ActionFMODPlay ()
        {
            this.isDisplayed = true;
            category = ActionCategory.Custom;
            title = "FMOD: Play Event";
            description = "Plays an FMOD Event.";
        }
    
        //Runs the action
        override public float Run ()
        {
            backgroundMusic = FMODUnity.RuntimeManager.CreateInstance (eventPath);
            backgroundMusic.start ();
            backgroundMusic.setParameterValue ("AT", 2);
            return 0f;
        }
    
    
        #if UNITY_EDITOR
    
        override public void ShowGUI ()
        {
            // Widget to select the scene game object that holds the emitter
            eventPath = EditorGUILayout.TextField ("Event Path", eventPath);
            // Widget to select the intensity when action is triggered
            intensity = EditorGUILayout.FloatField ("Intensity:", intensity);
        }   
    
    
        public override string SetLabel ()
        {
            // Return a string used to describe the specific action's job.
            string labelAdd = "";
            return labelAdd;
        }
    
        #endif
    
    }
    

    }`

  • Welcome to the community, @tamagotchithief.

    Though the Action uses AC's old Action API (the new one being described here), the Run function returns zero so it should just run the once and continue on with the rest of the ActionList.

    Does this issue occur when your ActionList only has this single Action type, and what is the error message you get in full (stacktrace included)? You can get this by opening up the Console window and clicking on the error.

  • Yes, so currently I have it as an OnStart cutscene. I changed the cutscene to "Run in background", and then the only action is Action Type: Custom FMOD: Play Event. There are no other actions in the list.

    I filled the event path and it is set to continue after running. The track plays, but the action list does not complete, it stays on the "Action lists running" list.

    I believe this is what you were asking for, if not please let me know!

    NotImplementedException: The method or operation is not implemented. AC.ActionFMODPlay.Run () (at Assets/Volkov/Actions/ActionFMODPlay.cs:48) AC.ActionList+<RunAction>d__46.MoveNext () (at Assets/AdventureCreator/Scripts/ActionList/ActionList.cs:467) UnityEngine.SetupCoroutine.InvokeMoveNext (System.Collections.IEnumerator enumerator, System.IntPtr returnValueAddress) (at /Users/bokken/build/output/unity/unity/Runtime/Export/Scripting/Coroutines.cs:17) UnityEngine.MonoBehaviour:StartCoroutine(IEnumerator) AC.ActionList:ProcessAction(Int32) (at Assets/AdventureCreator/Scripts/ActionList/ActionList.cs:408)

  • Just to confirm: is line 48 of your script the following?

    backgroundMusic.setParameterValue ("AT", 2);
    

    If so, it looks like this is an issue with FMOD (or the calling of its functions), rather than AC. I suspect the playback interruption of the ActionList is a knock-on effect.

    I'm not hugely knowledgable of FMOD's usage, but try commenting out the line this line, or contacting the FMOD team for advice.

  • I removed that line in the script and the action list completes now. I'm not really sure what function that served, but I guess that is not a problem for right now. Thank you for your help!

Sign In or Register to comment.

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Welcome to the official forum for Adventure Creator.