AI will change movies forever
What happens when you can just build a movie on your laptop from a prompt?
I watched David Fincher’s new movie The Killer yesterday. It’s dark, violent, and evocative. It is, in other words, a David Fincher movie. You can identify his movies from the visual language he uses. And that visual language can be described with natural language—the words and phrases we all speak. We can say of Fincher’s work that it is dark, violent, atmospheric, intriguingly-lit, etc. We can say of his work that his scenes evoke fear and and incipient violence.
To understand what I mean by “visual language” let’s consider some film criticism written about him. Here’s an essay about Fincher’s style. The author notes: “if filmmaking is an art of the precise decisions made to evoke a feeling, then David Fincher is certainly one of the best in the game. His films have become iconic for their dark tones and styles.”
And, true to form, here’s the trailer for The Killer:
Since we can use natural language to explain how a movie looks, we can also use natural language to tell AI technology how to create a movie. Here are RunwayML’s tips on how to create good natural language prompts to make AI-generated movies:
The following generic prompt modifiers generally produce good results:
masterpiece
classic
cinematic
For more action and movement, try adding keywords like:
cinematic action
flying
speeding
running
Try using camera-specific terms:
camera angles (full shot, close up, etc.)
lens types (macro lens, wide angle, etc.)
camera movements (slow pan, zoom, etc.)
In general trying to explain camera movements as if you were explaining to a toddler can help; so instead of ‘dolly shot’, you would say something like ‘zomming in over time’ or ‘growing over time’
RunwayML is telling its users how to use natural languag to evoke (and invoke!) cinematic language. This is not all too dissimilar to telling ChatGPT to evoke a someone’s manner of speaking. We can tell ChatGPT to write a sonnet in the style of Donald Trump:
Sonnet: The Art of the Deal
Upon the grandest stage, I boldly stand,
With golden towers reaching to the sky.
My empure built with firm and steady hand,
In business wars, I’m known as the top guy.
My words, like towers, stand so tall and true,
“I’m really rich,” a phrase I oft repeat.
In every deal, my prowess sees me through,
In boardrooms, I’m the one they can’t defeat.
With confidence, I navigate the storm,
“No loser,” I proclaim with every breath.
In every form, I’m far beyond the norm,
I face the world with bravery, not death.
So listen close, for I will tell you straight,
In life and deals, I’m truly, deeply great.
ChatGPT concludes with this helpful comment: “This sonnet captures the bravado and self-assuredness often associated with Donald Trump, particularly in his public appearances and speeches.” Now, this sonnet isn’t going to win any prize for poetry, and, while it’s certainly filled with bravado, it doesn’t really evoke Trump’s manner of speaking. Large language models, of course, are imperfect tools.
However, and this is where our segue into LLM-generated Tumpian sonnets is relevant, AI tools like RunwayML or ChatGPT allow us to use natural language to prompt the LLM to create something new. The point I am making here is this: we can use natural language prompts to evoke and invoke a movie experience, just as we can use natural language prompts to evoke and invoke a Trumpian sonnet. In either case, the output is not up to par with what a professional could do, but at the same time, “the worst AI you’ll ever use is the one you’re using today.” Given that AI technology is constantly improving, it would be a mistake to look at RunwayML’s output, and conclude that it will never be able to create a whole movie from a prompt.