Facial Animations

Didimos are imported with a custom animation system, that allows for integration with ARKit, Amazon Polly, and Oculus Lipsync. Internally, this animation system uses Unity's Animation Clips and the Animation component. This is the basis for every didimo's facial animation.

The low-level control of the animation is performed by the LegacyAnimationPoseController component using Unity standard Animation Clips. The didimo animation clips are comprised of a set of facial poses, like the poses specified by ARKit, the visemes required by Polly, Oculus Lipsync, and a few facial expressions.

Each pose acts independently from the others and the resulting facial expression is the additive combination of the poses. This means that you can easily combine simple or complex poses such as smiling and blinking at the same time.

High-level playback of animations can be done through our DidimoAnimator component, which is responsible for playing and blending animations together. It interacts directly with the LegacyAnimationPoseController to blend the required poses for animation playback.

Explore the Meet a didimo example scene if you want to see our didimos animate and how we handle animation, or try it out yourself using any of the DidimoAnimationExample scripts available.

Play an Animation (High-Level control)

In order to play an Animation, you can make use of the DidimoAnimator component with our JSON animations. You can also record your own using our ARKit LiveCapture integration.
Start by adding it to the AnimationCache so that you can use this same animation any time for any didimo and follow up by triggering it.

// Animation Variables
public TextAsset animationJson;
public string animationName;

// Animation Functions
public void LoadAnimation() {
  Didimo.DidimoAnimation animation = Didimo.DidimoAnimation.FromJSONContent(animationName,  animationJson);
  Didimo.AnimationCache.Add(animationName, animation);
}

public void PlayAnimation() {
  didimoComponents.Animator.PlayAnimation(animationName);
}

When you create the animation, you can also control some additional parameters, such as the WrapMode which can be useful to create looping animations such as idles. Additionally, you can also pass in an optional AudioClip parameter if you want to play an animation synced with sound.
The DidimoAnimator component also supports fading in and fading out of animations.

// Animation Fade Variable
float fadeDuration;


// Animation Fade Functions
public void FadeInAnimation() {
  didimoComponents.Animator.FadeInAnimation(animationName, fadeDuration);
}

public void FadeOutAnimation() {
  didimoComponents.Animator.FadeOutAnimation(animationName, fadeDuration);
}

Once you add an animation to the cache, you define its name and subsequently always refer to it by the chosen name, so be sure to provide unique and distinguishable names.

You can try this out yourself, by adding the DidimoAnimationExampleMocap script to a didimo and testing it on Play Mode.

Activate Poses (Low-Level control)

If you require more fine control of the poses, you can also directly control them through the LegacyAnimationPoseController. To ensure that there are no conflicts between the names and that they are easily distinguishable, we use the following convention: source_poseName, e.g.:

• ARKit_jawLeft
• defaultExpressions_Happy
• amazonPolly_phoneme_aa
• oculus_CH

You can see all the available animations for that didimo listed on the LegacyAnimationPoseController, under the AnimationClips array.

From the DidimoComponents component you can directly access the pose controller and subsequently activate any pose and set the desired head rotation:

// Pose Variables
public string poseName = "ARKit_jawLeft";
public string poseValue = 0.5f;
public Quaternion rotation;

// Pose Functions
public void ChangePose() {
	didimoComponents.PoseController.SetWeightForPose(poseName, poseValue);
	didimoComponents.PoseController.SetHeadRotation(rotation);  
}

At the end of the current frame, the resulting final pose will be computed by combining the multiple different poses that were set. Note that you do not need to explicitly call any function to compute the final pose, as this is automatically done by Unity's Animation component.

You can try this out yourself, by adding the DidimoAnimationExamplePoses script to a didimo.