AnimateDiff Prompt Travel and Examples

A static prompt creates a consistent animation. But what if you want your scene to evolve, morph, or transition from one concept to another? This is where Prompt Travel comes in. It's a powerful AnimateDiff feature that lets you schedule different text prompts to take effect at specific frames in your animation. Instead of one idea guiding the entire clip, you can create a narrative journey within a single generation.

This technique unlocks a huge range of creative possibilities, from subtle changes in lighting to dramatic transformations of a subject. Understanding prompt travel is key to moving from simple looping GIFs to more complex and engaging animated sequences with your AnimateDiff animation projects.

What is Prompt Travel?

Prompt Travel is the process of assigning different text prompts to different keyframes in an AnimateDiff animation. The underlying Stable Diffusion model then smoothly interpolates between these prompts, creating a fluid transition from one described scene to the next. The core idea is to create a "schedule" of prompts that guide the animation's evolution.

The syntax for this is typically a JSON-like object or a specially formatted text block where frame numbers are keys and text prompts are values. For a 64-frame animation, you could have one prompt for the first half and a completely different prompt for the second half. AnimateDiff's motion-aware generation will handle the transition, creating a mesmerizing morphing effect.

The Prompt Travel Syntax

The exact syntax for prompt travel depends on your interface, but the core concept is a map of frame numbers to prompts. In ComfyUI, this is often done with a node like "Batch Prompt Schedule" or a dedicated prompt travel node. In AUTOMATIC1111, this is enabled in the AnimateDiff extension's settings. A typical keyframe block looks like this:

{
  "0": "a closed pink rose bud, morning light, macro photography",
  "16": "a blooming pink rose, soft petals unfurling, morning light",
  "32": "a fully open red rose, vibrant color, golden hour light",
  "48": "a wilting red rose, petals curling at the edges, dusk"
}

In this example for a 64-frame animation, AnimateDiff will start by generating a closed bud. Around frame 16, it will have transitioned to a blooming flower. By frame 32, the flower will be a fully open red rose in different lighting, and by frame 48, it will be wilting. This is a powerful way to tell a small story within a short animation. The use of a clear prompt is very important for the animation.

5 Copy-Pasteable Prompt Travel Examples

Here are several themed prompt travel templates to get you started. Each is designed as a self-contained example. You can copy these directly into your AnimateDiff workflow. Remember to pair them with a suitable base model checkpoint (e.g., a realistic model for the cityscape, a fantasy model for the galaxy zoom).

{
  "0": "a tiny green sprout pushing through dark soil, detailed, macro",
  "24": "a young plant with two small leaves, morning dew, soft focus background",
  "48": "a tall stem with a tightly closed flower bud, side view",
  "72": "a vibrant, fully bloomed sunflower, facing the sun, bright daylight, detailed petals and seeds"
}

{
  "0": "a sprawling cityscape at sunrise, misty morning, golden light on skyscrapers, aerial view, cinematic",
  "32": "the same cityscape at midday, bustling streets, harsh sunlight, sharp shadows, ultra realistic",
  "64": "the cityscape at sunset, fiery orange and purple sky, reflections in windows",
  "96": "the same cityscape at night, millions of glowing lights, trails of car headlights, dark sky, blade runner aesthetic"
}

{
  "0": "a lone oak tree in a field, bare branches against a winter sky, covered in snow",
  "30": "the same oak tree, vibrant green leaves budding, spring morning, soft light",
  "60": "the oak tree in full summer leaf, lush and green, under a bright blue sky",
  "90": "the oak tree's leaves turning brilliant orange and red, autumn afternoon, falling leaves"
}

{
  "0": "a full body shot of a handsome elven warrior, facing forward, fantasy art, detailed armor",
  "16": "a full body shot of a handsome elven warrior, quarter turn right, fantasy art",
  "32": "a full body shot of a handsome elven warrior, side view right, profile shot, fantasy art",
  "48": "a full body shot of a handsome elven warrior, back view, fantasy art",
  "64": "a full body shot of a handsome elven warrior, facing forward again, fantasy art"
}

{
  "0": "calm ocean, gentle ripples on the water surface, clear blue water, sunny day",
  "32": "choppy ocean waves, white foam, dramatic lighting, approaching storm",
  "64": "massive, crashing ocean wave, tidal wave, dark and menacing water, spray and mist",
  "96": "bioluminescent ocean at night, glowing plankton, magical, fantasy"
}

{
  "0": "a vast, starry night sky, Milky Way visible, Hubble space telescope photography",
  "24": "zooming into a colorful nebula, clouds of gas and dust, pillars of creation aesthetic",
  "48": "flying through a dense star cluster, thousands of points of light, lens flare",
  "72": "approaching a swirling spiral galaxy, cosmic art, epic scale"
}

Implementing Prompt Travel in AnimateDiff

As mentioned, the two main platforms for AnimateDiff have different methods for prompt travel:

• In ComfyUI, you'll typically use a node setup. The `ComfyUI-AnimateDiff-Evolved` custom node by Kosinkadink includes a "Batch Prompt Schedule" node that is perfect for this. You can wire your positive and negative prompt schedules into it, and it will feed the results into the KSampler. This gives you maximum flexibility.
• In AUTOMATIC1111, the `sd-webui-animatediff` extension has a dedicated "Prompt Travel" section in its settings. You enable it, and a text box appears where you can paste your JSON-formatted keyframe prompts. The extension handles the parsing and scheduling for you.

Regardless of the platform, the key is experimentation. Start with one of the prompt templates above, see how your chosen model interprets the transitions, and then start tweaking the prompts and frame numbers to create your own unique animation. The combination of a creative prompt schedule and a powerful motion module is what makes AnimateDiff such an expressive tool for AI animation.