How can we dynamically control frame-by-frame animations using game interactions?

The Problem

The principal gameplay of Floor Kids demands a very large number of frames of hand-drawn character animation, which may be sequenced together in a number of different ways depending on situational factors and player input. We needed a way to manage this complexity in a way that was intuitive, lightweight, and presented as little additional cognitive overhead as possible.

We examined a number of existing solutions for handling character animation in games before deciding to build our own system. The front-runner was Unity Technologies’ own Mecanim system, which is well-maintained and directly integrated into the Unity development environment.

While Mecanim has some faculties for supporting two-dimensional frame-by-frame animation, it was primarily developed for managing complex, semi-procedural 3D “skinned” animation, where a character’s state is often comprised of multiple different “layers” of animations which are blended together to produce a final pose. We found that the abstractions and affordances offered by Mecanim were not flexible enough to be adapted to our specific requirements for Floor Kids. Additionally, it did not offer a simple way to visualize frame-by-frame animation at a glance (for example, styled as an animation “dopesheet”), which significantly reduced its usefulness as a way to manage the complexity of frame-by-frame animations.

We thus chose to develop our own solution to the problem, more suited to our particular demands for this project. Our solution is flexible and simple, and it is easily adapted to a number of different situations where frame-by-frame animation must be sequenced together in nondeterministic or nonlinear ways.

The Solution

An animation graph in Floor Kids consists of any number of discrete “sequences”, which may be positioned freely on an infinite canvas. Sequences are comprised of one or more frames of linear, sequential animation, but may be linked to a number of other sequences via “exit nodes”, transitions which are triggered when certain conditions are met (for example, a transition may be triggered on player input, or if a character’s stamina is below a certain threshold).

Frames are added to sequences via a simple drag-and-drop mechanism from a searchable, filterable database of all the frames of animation in the game. We found that one advantage of this technique over “automatic” asset management (e.g., assembling sequences automatically based on a filename or folder structure) was that it permitted us to use existing frames as placeholders when prototyping new sequences, or to pad out existing animations with extra frames as a way of tweaking the timing or difficulty of the controls. This permitted us to iterate quickly and experiment with alternate designs while minimizing complex asset management or waiting for additional frames of animation to be produced.

Similarly, animations are “wired” together by simply dragging from one sequence’s exit node to another sequence. In this sense, the animation editor can be said to resemble flow-based/visual programming environments such as Max/MSP or Pure Data.

One unconventional choice in the design of the animation editor was to build it as a “runtime” module rather than an extension to the Unity development environment. As a consequence, we are able to visualize, overlay, and even edit a character’s animation during gameplay, correcting problems or tweaking character behaviour instantly and on-the-fly. The editor is even available on mobile devices, and changes made on a device can be easily applied back to the main project. These affordances have, again, proven invaluable for rapidly iterating, prototyping, and experimenting with new ideas as they occur to us.

Challenges and Ongoing Work

The animation editor is far from complete, and will continue to be tweaked and adapted throughout Floor Kids’ production cycle. While intuitive and simple, we found that the node-based “infinite canvas” paradigm presents its own challenges in managing complexity. Complex relationships between sequences can become difficult to read, and analyzing or debugging a particular chain of causality during gameplay can sometimes require careful inspection of the animation graph. While we’ve developed simple solutions to try to mitigate some of these issues, further work might involve “focalizing” the graph editor in certain scenarios, automatically highlighting related sequences or even providing temporary, automatic re-layouting to clarify indirect relationships between sequences.