Integration with other components: Mixamo, FaceFX, Head Look

Integration with other components: Mixamo, FaceFX, Head Look

PostPosted by TonyLi » Sat Jun 02, 2012 12:40 pm

Andy,

Can you help me with some questions about integrating SAGE with other components? In specific: root motion in Mixamo animations, FaceFX facial animation, and Rune Johansen's Head Look Controller.

I bought SAGE to do some quick prototyping. That's been the smoothest part of this project, and boy do I miss it!

I have a large library of Mixamo animations. I'm currently using Mixamo's Root Motion Computer v2, which is an animation state machine that extracts root motion from blended animations. It reads the state machine definition from a JSON text file. Mine are 1000+ lines and a beast to maintain.

Mixamo's RMC, FaceFX's Unity controller, and Rune's Head Look Controller are all playing nicely together right now. Unfortunately, the RMC is REALLY SLOW. Their code is more proof of concept than production quality. I'd MUCH rather use SAGE than rewrite the RMC and have to reapply changes if they ever publish an updated version.

My questions about SAGE:

1. Can I manually apply movement to characters? My animations aren't in-place. Some of them, like leaving cover, move on the X-axis for the first part, then on the Z-axis. If I extract the root motion, can I manually apply it to the character? Would I override the base controller's ApplyMovement() method? If so, I assume in the Animation node I'd check Apply Movement but leave the XYZ at 0.

2. Does SAGE keep its hands off unused clips in the Animation component? My characters have phoneme animations for lip sync. Mixamo's RMC sets weights for all animation clips, even if they're not defined in the state machine's JSON file. This prevents the FaceFX controller from controlling them. The RMC and FaceFX both modify AnimationStates directly, like SAGE, but FaceFX only modifies the facial clips. I modified the RMC to ignore FaceFX clips (my changes don't introduce any measurable overhead), but I'm hoping for a better solution.

3. Can I apply procedural animation in LateUpdate()? I'd like to continue using Rune's Head Look Controller with SAGE.

Thanks in advance!

Tony
TonyLi
 
Posts: 18
Joined: Sun Apr 08, 2012 10:11 am

Re: Integration with other components: Mixamo, FaceFX, Head

PostPosted by TonyLi » Sat Jun 02, 2012 4:16 pm

I tested #2 and #3. Procedural animation in LateUpdate() (HeadLookController) and facial animation (FaceFX controller) work fine.

I'll try to get root motion extraction working now. If anyone has any sage advice (sorry), please do let me know.
TonyLi
 
Posts: 18
Joined: Sun Apr 08, 2012 10:11 am

Re: Integration with other components: Mixamo, FaceFX, Head

PostPosted by AlteredReality » Sat Jun 02, 2012 11:29 pm

Hey there, I'll try and go down the list and answer all your questions! Feel free to reply again if I miss anything.

1. It is definitely possible to apply your own movement to the character controller. However, Sage only supports basic linear movement. I did look into root motion, but I had determined that it was beyond the scope of what I wanted to do with Sage initially. It remains a feature I'd like to work on, but it I'm not sure when that will be at this point.

ApplyMovement in the game controller is intended to let you apply movement in your own way specific to your game. I imagine it would be possible to implement your own movement behaviors while various animations are playing.

2. Sage will only modify the weights of animation clips that are used in it's graphs.

3. It is not something I have personally tried, but I am pretty sure it will work just fine.

While working on Sage, I definitely intended it to work with most other middleware packages, as I knew I wouldn't be able to implement all features. So if you find any issues while working with Sage and other software, please let me know. I'm more than happy to help! :)
Andy - Altered Reality
AlteredReality
Site Admin
 
Posts: 118
Joined: Fri Jan 06, 2012 8:31 pm

Re: Integration with other components: Mixamo, FaceFX, Head

PostPosted by TonyLi » Sun Jun 03, 2012 1:21 pm

Thanks! #2 and #3 worked just fine. I have a question on #1:

How does the ApplyMovement() delegate work? What values should it return?

Is there a way I can get a list of animations that SAGE is currently influencing? I'd like to just iterate through these and sample them. Otherwise I'll have to go through the entire list of animations, or independently maintain a list of animations that can affect motion and iterate through those.

Thanks,
Tony
TonyLi
 
Posts: 18
Joined: Sun Apr 08, 2012 10:11 am

Re: Integration with other components: Mixamo, FaceFX, Head

PostPosted by AlteredReality » Sun Jun 03, 2012 4:18 pm

Yeah, I saw your other post after I wrote mine about #2 and #3. Thanks good to hear!

ApplyMovement() will send the sum of all movement and rotation to be applied. If it returns false, Sage applies it's own movement in that amount. If it returns true, Sage assumes that the movement was applied using custom code.

As for getting the list of animations...that is something that Sage does not have support for. However, it would be relatively easy to add I think. Are you using Sage Dlls or Source Code?

Regardless, I'll add it to the list of things to add.
Andy - Altered Reality
AlteredReality
Site Admin
 
Posts: 118
Joined: Fri Jan 06, 2012 8:31 pm

Re: Integration with other components: Mixamo, FaceFX, Head

PostPosted by TonyLi » Sun Jun 03, 2012 7:31 pm

I'm using the SAGE DLLs. Thanks for the info about ApplyMovement(). I'll give it a shot this week and let you know how it works out.
TonyLi
 
Posts: 18
Joined: Sun Apr 08, 2012 10:11 am

Re: Integration with other components: Mixamo, FaceFX, Head

PostPosted by AlteredReality » Sun Jun 03, 2012 8:45 pm

Sounds good. And I was curious about the Dlls, as it would be relatively easy to add something to the code to keep track of the animations a library is using. I'll add it to the next version.
Andy - Altered Reality
AlteredReality
Site Admin
 
Posts: 118
Joined: Fri Jan 06, 2012 8:31 pm

Re: Integration with other components: Mixamo, FaceFX, Head

PostPosted by TonyLi » Sun Jun 03, 2012 10:54 pm

Could you also track a list of animations for which "Move" is checked? My SAGE library will probably have a lot of in-place animations such as idle, shooting, waving, etc., which wouldn't need to be factored into root motion extraction.
TonyLi
 
Posts: 18
Joined: Sun Apr 08, 2012 10:11 am

Re: Integration with other components: Mixamo, FaceFX, Head

PostPosted by AlteredReality » Mon Jun 04, 2012 8:35 pm

K, I'll keep that in mind too.
Andy - Altered Reality
AlteredReality
Site Admin
 
Posts: 118
Joined: Fri Jan 06, 2012 8:31 pm

Re: Integration with other components: Mixamo, FaceFX, Head

PostPosted by TonyLi » Mon Jun 04, 2012 9:45 pm

Andy,

Well, I just got a batch of models with in-place animations, so I'm going to use SAGE for them and continue with Mixamo's root motion computer (RMC) for the Mixamo models. I was able to squeeze just the right speed out of Mixamo's RMC with a few edits, which to be honest was a lot less work than implementing root motion extraction in SAGE.

If you ever decide to add root motion extraction, Mixamo's RMCv2 code is freely downloadable, and the logic is pretty well commented. A couple guys at CMU have also implemented root motion extraction in Unity. They describe it here: http://wiki.etc.cmu.edu/unity3d/index.php/AMTk_RootTranslationMotor_guide but I'm not sure if their code is available yet.

One more thought: you might consider calling ApplyMovement() in LateUpdate() so it happens after other animation and physics updates.
TonyLi
 
Posts: 18
Joined: Sun Apr 08, 2012 10:11 am

Next

Return to Questions and Discussion

Who is online

Users browsing this forum: No registered users and 0 guests


Style by Gokinstudio | Free forum hosting by ProphpBB | Software by phpBB | Report Abuse | Privacy
cron