Snap Video: Scaled Spatiotemporal Transformers for Text-to-Video Synthesis

Willi Menapace1,2,* Aliaksandr Siarohin1 Ivan Skorokhodov1 Ekaterina Deyneka1 Tsai-Shien Chen1,3,*
Anil Kag1 Yuwei Fang1 Aleksei Stoliar1 Elisa Ricci2,4 Jian Ren1 Sergey Tulyakov1

Snap Inc.1 University of Trento2 UC Merced3 Fondazione Bruno Kessler4 Work performed while interning at Snap Inc.*

Paper Overview Stories

Comparison to Gen-2, Floor33 and PikaLab

We compare results produced by Snap Video against the publicly accessible Gen-2, Floor33 and PikaLab video generators on a selection of 65 prompts from the Evalcrafter benchmark eliciting dynamic scenes.

When evaluated on a user study, our method shows increased photorealism with respect to PikaLab and Floor33, has significantly better video-text alignment and outperforms the baselines on all motion metrics. Results are expressed in percentage of votes in favor of our method.

Photorealism Video-Text Align. Motion Quantity Motion Quality
Gen-2 43.3 81.0 96.0 78.7
PikaLab 61.5 80.3 89.3 70.5
Floor33 76.3 80.9 88.0 79.2

Hover the cursor on the video to reveal the prompt. Note that a selection of prompts issued to Gen-2 resulted in input prompt filtering issues and no output was generated for them.

Snap Video

Gen-2

Two elephants are playing on the beach and enjoying a delicious beef stroganoff meal.
Two elephants are playing on the beach and enjoying a delicious beef stroganoff meal.

Floor33

PikaLab

Two elephants are playing on the beach and enjoying a delicious beef stroganoff meal.
Two elephants are playing on the beach and enjoying a delicious beef stroganoff meal.

Snap Video

Gen-2

A photo of a Corgi dog riding a bike in Times Square. It is wearing sunglasses and a beach hat.
A photo of a Corgi dog riding a bike in Times Square. It is wearing sunglasses and a beach hat.

Floor33

PikaLab

A photo of a Corgi dog riding a bike in Times Square. It is wearing sunglasses and a beach hat.
A photo of a Corgi dog riding a bike in Times Square. It is wearing sunglasses and a beach hat.

Snap Video

Gen-2

Two individuals ride a bike through the temple gates.
Two individuals ride a bike through the temple gates.

Floor33

PikaLab

Two individuals ride a bike through the temple gates.
Two individuals ride a bike through the temple gates.

Snap Video

Gen-2

A young man is riding a bicycle on a sunny day. He is wearing a blue hoodie and black jeans. His hair is brown and styled messily. He has a small scar above his left eye.
A young man is riding a bicycle on a sunny day. He is wearing a blue hoodie and black jeans. His hair is brown and styled messily. He has a small scar above his left eye.

Floor33

PikaLab

A young man is riding a bicycle on a sunny day. He is wearing a blue hoodie and black jeans. His hair is brown and styled messily. He has a small scar above his left eye.
A young man is riding a bicycle on a sunny day. He is wearing a blue hoodie and black jeans. His hair is brown and styled messily. He has a small scar above his left eye.

Snap Video

Gen-2

A couple shares a milkshake with two straws at a retro diner.
A couple shares a milkshake with two straws at a retro diner.

Floor33

PikaLab

A couple shares a milkshake with two straws at a retro diner.
A couple shares a milkshake with two straws at a retro diner.

Snap Video

Gen-2

A man cruises through the city on a motorcycle, feeling the adrenaline rush
A man cruises through the city on a motorcycle, feeling the adrenaline rush

Floor33

PikaLab

A man cruises through the city on a motorcycle, feeling the adrenaline rush
A man cruises through the city on a motorcycle, feeling the adrenaline rush

Snap Video

Gen-2

A thick fog covers a street, making it nearly impossible to see. Cars headlights pierce through the mist as they slowly make their way down the road.
A thick fog covers a street, making it nearly impossible to see. Cars headlights pierce through the mist as they slowly make their way down the road.

Floor33

PikaLab

A thick fog covers a street, making it nearly impossible to see. Cars headlights pierce through the mist as they slowly make their way down the road.
A thick fog covers a street, making it nearly impossible to see. Cars headlights pierce through the mist as they slowly make their way down the road.

Snap Video

Gen-2

large motion, bustling gym with people working out on various equipment and lifting weights.
large motion, bustling gym with people working out on various equipment and lifting weights.

Floor33

PikaLab

large motion, bustling gym with people working out on various equipment and lifting weights.
large motion, bustling gym with people working out on various equipment and lifting weights.

Snap Video

Gen-2

A pod of dolphins gracefully swim and jump in the ocean.
A pod of dolphins gracefully swim and jump in the ocean.

Floor33

PikaLab

A pod of dolphins gracefully swim and jump in the ocean.
A pod of dolphins gracefully swim and jump in the ocean.

Snap Video

Gen-2

Eight fluffy yellow ducklings waddle behind their mother, exploring the edge of a pond.
Eight fluffy yellow ducklings waddle behind their mother, exploring the edge of a pond.

Floor33

PikaLab

Eight fluffy yellow ducklings waddle behind their mother, exploring the edge of a pond.
Eight fluffy yellow ducklings waddle behind their mother, exploring the edge of a pond.

Snap Video

Gen-2

A young woman with glasses is jogging in the park wearing a pink headband.
A young woman with glasses is jogging in the park wearing a pink headband.

Floor33

PikaLab

A young woman with glasses is jogging in the park wearing a pink headband.
A young woman with glasses is jogging in the park wearing a pink headband.

Snap Video

Gen-2

A young woman pushes herself through a tough gym workout, sweat pouring down her face as she lifts weights and jumps on a treadmill.
A young woman pushes herself through a tough gym workout, sweat pouring down her face as she lifts weights and jumps on a treadmill.

Floor33

PikaLab

A young woman pushes herself through a tough gym workout, sweat pouring down her face as she lifts weights and jumps on a treadmill.
A young woman pushes herself through a tough gym workout, sweat pouring down her face as she lifts weights and jumps on a treadmill.

Snap Video

Gen-2

A man lifts weights at the gym, working on his strength.
A man lifts weights at the gym, working on his strength.

Floor33

PikaLab

A man lifts weights at the gym, working on his strength.
A man lifts weights at the gym, working on his strength.

Snap Video

Gen-2

The man will be piloting a small plane for an aerial tour.
The man will be piloting a small plane for an aerial tour.

Floor33

PikaLab

The man will be piloting a small plane for an aerial tour.
The man will be piloting a small plane for an aerial tour.

Snap Video

Gen-2

A busy bakery filled with the aroma of freshly baked bread and pastries.
A busy bakery filled with the aroma of freshly baked bread and pastries.

Floor33

PikaLab

A busy bakery filled with the aroma of freshly baked bread and pastries.
A busy bakery filled with the aroma of freshly baked bread and pastries.

Snap Video

Gen-2

Two sleek white airplanes flew in close formation, cutting through the clouds. They rocked and dipped gracefully, performing an aerial dance.
Two sleek white airplanes flew in close formation, cutting through the clouds. They rocked and dipped gracefully, performing an aerial dance.

Floor33

PikaLab

Two sleek white airplanes flew in close formation, cutting through the clouds. They rocked and dipped gracefully, performing an aerial dance.
Two sleek white airplanes flew in close formation, cutting through the clouds. They rocked and dipped gracefully, performing an aerial dance.

Snap Video

Gen-2

A cute and chubby giant panda is enjoying a bamboo meal in a lush forest. The panda is relaxed and content as it eats, and occasionally stops to scratch its ear with its paw.
A cute and chubby giant panda is enjoying a bamboo meal in a lush forest. The panda is relaxed and content as it eats, and occasionally stops to scratch its ear with its paw.

Floor33

PikaLab

A cute and chubby giant panda is enjoying a bamboo meal in a lush forest. The panda is relaxed and content as it eats, and occasionally stops to scratch its ear with its paw.
A cute and chubby giant panda is enjoying a bamboo meal in a lush forest. The panda is relaxed and content as it eats, and occasionally stops to scratch its ear with its paw.

Snap Video

Gen-2

A horse gallops through a field, kicking up dust with its hooves.
A horse gallops through a field, kicking up dust with its hooves.

Floor33

PikaLab

A horse gallops through a field, kicking up dust with its hooves.
A horse gallops through a field, kicking up dust with its hooves.

Snap Video

Gen-2

teenage girl in red skirt enters a green-lit cave
teenage girl in red skirt enters a green-lit cave

Floor33

PikaLab

teenage girl in red skirt enters a green-lit cave
teenage girl in red skirt enters a green-lit cave

Snap Video

Gen-2

A group of children build a snowman together.
A group of children build a snowman together.

Floor33

PikaLab

A group of children build a snowman together.
A group of children build a snowman together.

Snap Video

Gen-2

A musician strums his guitar, serenading the moonlit night
A musician strums his guitar, serenading the moonlit night

Floor33

PikaLab

A musician strums his guitar, serenading the moonlit night
A musician strums his guitar, serenading the moonlit night

Snap Video

Gen-2

Concert hall with shining lights and energetic audience dancing
Concert hall with shining lights and energetic audience dancing

Floor33

PikaLab

Concert hall with shining lights and energetic audience dancing
Concert hall with shining lights and energetic audience dancing

Snap Video

Gen-2

There's a rush in the subway. People are taking the bus to airports to catch their flights. The airplane passes over them.
There's a rush in the subway. People are taking the bus to airports to catch their flights. The airplane passes over them.

Floor33

PikaLab

There's a rush in the subway. People are taking the bus to airports to catch their flights. The airplane passes over them.
There's a rush in the subway. People are taking the bus to airports to catch their flights. The airplane passes over them.

Snap Video

Gen-2

A dog chews on a bone, gnawing at it with its teeth.
A dog chews on a bone, gnawing at it with its teeth.

Floor33

PikaLab

A dog chews on a bone, gnawing at it with its teeth.
A dog chews on a bone, gnawing at it with its teeth.

Snap Video

Gen-2

A surfer paddles out into the ocean, scanning the waves for the perfect ride.
A surfer paddles out into the ocean, scanning the waves for the perfect ride.

Floor33

PikaLab

A surfer paddles out into the ocean, scanning the waves for the perfect ride.
A surfer paddles out into the ocean, scanning the waves for the perfect ride.

Snap Video

Gen-2

A mother and her child feed ducks at a pond.
A mother and her child feed ducks at a pond.

Floor33

PikaLab

A mother and her child feed ducks at a pond.
A mother and her child feed ducks at a pond.

Snap Video

Gen-2

A girl dances gracefully in the rain, twirling her umbrella.
A girl dances gracefully in the rain, twirling her umbrella.

Floor33

PikaLab

A girl dances gracefully in the rain, twirling her umbrella.
A girl dances gracefully in the rain, twirling her umbrella.

Snap Video

Gen-2

A busy school hallway filled with students hurrying to their classrooms.
A busy school hallway filled with students hurrying to their classrooms.

Floor33

PikaLab

A busy school hallway filled with students hurrying to their classrooms.
A busy school hallway filled with students hurrying to their classrooms.

Snap Video

Gen-2

a snail, large motion
a snail, large motion

Floor33

PikaLab

a snail, large motion
a snail, large motion

Snap Video

Gen-2

a cat drinking beer
a cat drinking beer

Floor33

PikaLab

a cat drinking beer
a cat drinking beer

Snap Video

Gen-2

A cat plays with a ball of yarn, batting it around with its paws.
A cat plays with a ball of yarn, batting it around with its paws.

Floor33

PikaLab

A cat plays with a ball of yarn, batting it around with its paws.
A cat plays with a ball of yarn, batting it around with its paws.

Snap Video

Gen-2

Zombies walking down the street
Zombies walking down the street

Floor33

PikaLab

Zombies walking down the street
Zombies walking down the street

Snap Video

Gen-2

A basketball player dribbles down the court, weaving through defenders before leaping for a slam dunk.
A basketball player dribbles down the court, weaving through defenders before leaping for a slam dunk.

Floor33

PikaLab

A basketball player dribbles down the court, weaving through defenders before leaping for a slam dunk.
A basketball player dribbles down the court, weaving through defenders before leaping for a slam dunk.

Snap Video

Gen-2

A group of teenagers skateboard in an empty parking lot.
A group of teenagers skateboard in an empty parking lot.

Floor33

PikaLab

A group of teenagers skateboard in an empty parking lot.
A group of teenagers skateboard in an empty parking lot.

Snap Video

Gen-2

A child excitedly swings on a rusty swing set, laughter filling the air.
A child excitedly swings on a rusty swing set, laughter filling the air.

Floor33

PikaLab

A child excitedly swings on a rusty swing set, laughter filling the air.
A child excitedly swings on a rusty swing set, laughter filling the air.

Snap Video

Gen-2

A bustling street in London, with double-decker buses and red telephone booths. The famous Big Ben towers in the background.
A bustling street in London, with double-decker buses and red telephone booths. The famous Big Ben towers in the background.

Floor33

PikaLab

A bustling street in London, with double-decker buses and red telephone booths. The famous Big Ben towers in the background.
A bustling street in London, with double-decker buses and red telephone booths. The famous Big Ben towers in the background.

Snap Video

Gen-2

time lapse of flamingos walking around an ornate ballroom, first person perspective
time lapse of flamingos walking around an ornate ballroom, first person perspective

Floor33

PikaLab

time lapse of flamingos walking around an ornate ballroom, first person perspective
time lapse of flamingos walking around an ornate ballroom, first person perspective

Snap Video

Gen-2

a dog wearing vr goggles on a boat
a dog wearing vr goggles on a boat

Floor33

PikaLab

a dog wearing vr goggles on a boat
a dog wearing vr goggles on a boat

Snap Video

Gen-2

A bear rummages through a dumpster, searching for food scraps.
A bear rummages through a dumpster, searching for food scraps.

Floor33

PikaLab

A bear rummages through a dumpster, searching for food scraps.
A bear rummages through a dumpster, searching for food scraps.

Snap Video

Gen-2

man driving in a taxi, angry face
man driving in a taxi, angry face

Floor33

PikaLab

man driving in a taxi, angry face
man driving in a taxi, angry face

Snap Video

Gen-2

A group of two artists collaborate on a painting, each adding their own unique touch to the canvas.
A group of two artists collaborate on a painting, each adding their own unique touch to the canvas.

Floor33

PikaLab

A group of two artists collaborate on a painting, each adding their own unique touch to the canvas.
A group of two artists collaborate on a painting, each adding their own unique touch to the canvas.

Snap Video

Gen-2

A father teaches his son how to ride a bicycle in the backyard
A father teaches his son how to ride a bicycle in the backyard

Floor33

PikaLab

A father teaches his son how to ride a bicycle in the backyard
A father teaches his son how to ride a bicycle in the backyard

Snap Video

Gen-2

Three travelers wander through the bustling streets of Times Square, taking in the dazzling lights and energy.
Three travelers wander through the bustling streets of Times Square, taking in the dazzling lights and energy.

Floor33

PikaLab

Three travelers wander through the bustling streets of Times Square, taking in the dazzling lights and energy.
Three travelers wander through the bustling streets of Times Square, taking in the dazzling lights and energy.

Snap Video

Gen-2

The video depicts the bustling street of Singapore's Chinatown. People are seen walking around under the bright neon lights of the shops and restaurants. The camera pans around to capture the vibrancy of the scene.
The video depicts the bustling street of Singapore's Chinatown. People are seen walking around under the bright neon lights of the shops and restaurants. The camera pans around to capture the vibrancy of the scene.

Floor33

PikaLab

The video depicts the bustling street of Singapore's Chinatown. People are seen walking around under the bright neon lights of the shops and restaurants. The camera pans around to capture the vibrancy of the scene.
The video depicts the bustling street of Singapore's Chinatown. People are seen walking around under the bright neon lights of the shops and restaurants. The camera pans around to capture the vibrancy of the scene.

Snap Video

Gen-2

a cowboy panda riding on the back of a lion, hand-held camera
a cowboy panda riding on the back of a lion, hand-held camera

Floor33

PikaLab

a cowboy panda riding on the back of a lion, hand-held camera
a cowboy panda riding on the back of a lion, hand-held camera

Snap Video

Gen-2

A happy pig rolling in the mud on a sunny day.
A happy pig rolling in the mud on a sunny day.

Floor33

PikaLab

A happy pig rolling in the mud on a sunny day.
A happy pig rolling in the mud on a sunny day.

Snap Video

Gen-2

Three hamster runs on a wheel, exercising in its cage.
Three hamster runs on a wheel, exercising in its cage.

Floor33

PikaLab

Three hamster runs on a wheel, exercising in its cage.
Three hamster runs on a wheel, exercising in its cage.

Snap Video

Gen-2

A woman is walking her dog on the beach at sunset.
A woman is walking her dog on the beach at sunset.

Floor33

PikaLab

A woman is walking her dog on the beach at sunset.
A woman is walking her dog on the beach at sunset.

Snap Video

Gen-2

A group of six dancers perform a ballet on stage, their movements synchronized and graceful.
A group of six dancers perform a ballet on stage, their movements synchronized and graceful.

Floor33

PikaLab

A group of six dancers perform a ballet on stage, their movements synchronized and graceful.
A group of six dancers perform a ballet on stage, their movements synchronized and graceful.

Snap Video

Gen-2

A young couple dances under the moonlight by the lake.
A young couple dances under the moonlight by the lake.

Floor33

PikaLab

A young couple dances under the moonlight by the lake.
A young couple dances under the moonlight by the lake.

Snap Video

Gen-2

A snow avalanche crashed down a mountain peak, causing destruction and mayhem
A snow avalanche crashed down a mountain peak, causing destruction and mayhem

Floor33

PikaLab

A snow avalanche crashed down a mountain peak, causing destruction and mayhem
A snow avalanche crashed down a mountain peak, causing destruction and mayhem

Snap Video

Gen-2

a motorcycle race through the city streets at night
a motorcycle race through the city streets at night

Floor33

PikaLab

a motorcycle race through the city streets at night
a motorcycle race through the city streets at night

Snap Video

Gen-2

large motion, Surrounded by butterflies, a girl walks through a lush garden.
large motion, Surrounded by butterflies, a girl walks through a lush garden.

Floor33

PikaLab

large motion, Surrounded by butterflies, a girl walks through a lush garden.
large motion, Surrounded by butterflies, a girl walks through a lush garden.

Snap Video

Gen-2

In an African savanna, a majestic lion is prancing behind a small timid rabbit. The rabbit tried to run away, but the lion catches up easily.
In an African savanna, a majestic lion is prancing behind a small timid rabbit. The rabbit tried to run away, but the lion catches up easily.

Floor33

PikaLab

In an African savanna, a majestic lion is prancing behind a small timid rabbit. The rabbit tried to run away, but the lion catches up easily.
In an African savanna, a majestic lion is prancing behind a small timid rabbit. The rabbit tried to run away, but the lion catches up easily.

Snap Video

Gen-2

A young woman in her mid-twenties with long blond hair walked briskly down the busy city street, wearing a red coat and black boots.
A young woman in her mid-twenties with long blond hair walked briskly down the busy city street, wearing a red coat and black boots.

Floor33

PikaLab

A young woman in her mid-twenties with long blond hair walked briskly down the busy city street, wearing a red coat and black boots.
A young woman in her mid-twenties with long blond hair walked briskly down the busy city street, wearing a red coat and black boots.

Snap Video

Gen-2

A child excitedly licks an ice cream cone on a sunny day.
A child excitedly licks an ice cream cone on a sunny day.

Floor33

PikaLab

A child excitedly licks an ice cream cone on a sunny day.
A child excitedly licks an ice cream cone on a sunny day.

Snap Video

Gen-2

A couple enjoys a romantic gondola ride in Venice, Italy.
A couple enjoys a romantic gondola ride in Venice, Italy.

Floor33

PikaLab

A couple enjoys a romantic gondola ride in Venice, Italy.
A couple enjoys a romantic gondola ride in Venice, Italy.

Snap Video

Gen-2

A rainy day in the city, raindrops pouring down from grey clouds above, people hurrying on the street with umbrellas.
A rainy day in the city, raindrops pouring down from grey clouds above, people hurrying on the street with umbrellas.

Floor33

PikaLab

A rainy day in the city, raindrops pouring down from grey clouds above, people hurrying on the street with umbrellas.
A rainy day in the city, raindrops pouring down from grey clouds above, people hurrying on the street with umbrellas.

Snap Video

Gen-2

A silver metal train with blue and red stripes, speeding through a mountainous landscape.
A silver metal train with blue and red stripes, speeding through a mountainous landscape.

Floor33

PikaLab

A silver metal train with blue and red stripes, speeding through a mountainous landscape.
A silver metal train with blue and red stripes, speeding through a mountainous landscape.

Snap Video

Gen-2

camera pan from right to left, four people riding in a convertible car
camera pan from right to left, four people riding in a convertible car

Floor33

PikaLab

camera pan from right to left, four people riding in a convertible car
camera pan from right to left, four people riding in a convertible car
Back Next