Neural Character Controllers for Character-Object and Character-Character Interactions

Abstract

In this talk, I will cover our recent development of neural network-based character controllers for animating character-object/environment interactions and character-character interactions. Using neural networks for character controllers significantly increases the scalability of the system - the controller can be trained with a large amount of motion capture data while the runtime memory can be kept low. As a result, such controllers are suitable for real-time applications such as computer games and virtual reality systems. The main challenge is in designing an architecture that can produce movements in production-quality and also manage a wide variation of motion classes. Our development about character-object/environment interaction covers goal-driven actions with precise scene interactions, such as sitting on chairs, carrying objects, opening doors and avoiding obstacles. Our proposed deep auto-regressive framework enables modeling of multimodal scene interaction behaviors purely from data. Given high-level instructions such as the goal location and the action to be launched there, our system computes a series of movements and transitions to reach the goal in the desired state. To allow characters to adapt to a wide range of geometry such as different shapes of furniture and obstacles, we incorporate an efficient data augmentation scheme to randomly switch the 3D geometry while maintaining the context of the original motion. To increase the precision to reach the goal during runtime, we introduce a control scheme that combines egocentric inference and goal-centric inference. Our development about character-character interaction focuses on animating characters to play basketball, which involve movements such as catching, dribbling and shooting the ball, while avoiding the opponent player who tries to defend and intercept the ball. To achieve this task we propose a novel feature called local motion phase, that can help neural networks to learn asynchronous movements of each bone and its interaction with external objects such as a ball or an environment. We also propose a novel generative scheme to reproduce a wide variation of movements from abstract control signals given by a gamepad, which can be useful for changing the style of the motion under the same context. Our scheme is useful for animating contact-rich, complex interactions for real-time applications such as computer games. In the end of the talk, I will discuss the open problems and future directions of character animation.