The MV idea that was impossible to realize ten years ago can now be easily realized with the help of Sora's mysterious power! The great director spliced 55 short clips to easily create a four-minute MV skit!
During the May Day holiday, writer and director Paul Trillo, OpenAI’s designated early-bird Sora user, released “the first official MV produced using OpenAI Sora”!
Advertisement
The 4-minute video was tailor-made for independent musician Washed Out (real name Ernest Weatherly Greene Jr.) and his new single “The Hardest Part.”
In a nutshell, it consists of a sequence of rapid zoom shots, with different scenes tied together through clever splicing.
What's even more shocking is that in order to make this video, Paul Trillo used Sora to generate a total of 700 clips, and then carefully selected 55 of them and spliced them in Adobe Premiere.
Advertisement
In the end, it took about 6 weeks to complete the MV.
In this regard, Trillo said: “This is actually an idea I conceived 10 years ago, but then I had to give up. Now, this idea can finally be realized!”
“I didn’t have to prune my creativity too much this time,” Trillo said. “The freedom to experiment with ideas naturally creates a different kind of story because I can throw out a lot of ideas and see which ones work.”
Will this be Sora’s “ChatGPT” moment?
“The Hardest Part,” a new song from indie pop artist Washed Out, addresses the most human subject of losing love.
Ironically, however, to portray the sense of longing in the song, the musician turned to something far less flesh-and-blood: AI.
According to its record label Sub Pop, with the release of this new song, Washed Out (Ernest Greene) has become the first mainstream music artist to collaborate with a filmmaker and use OpenAI Sora to produce a MV.
The roughly four-minute video, directed by Paul Trillo, brings to life key moments in a couple's lives through a series of quick zoom shots.
Two characters – a red-haired woman and a dark-haired man – go from passionate kissing and smoking in high school in the 1980s to eventual marriage and children.
“Don't cry, everything is okay now,” Greene crooned. “The hardest part is you can't look back.”
According to the producers, this video may herald the beginning of a disruptive trend using AI in video production.
“I think we're at a stage now where we're about to have a big explosion, so I'm really looking forward to incorporating these new technologies and exploring how they impact what I can create,” Greene said in an interview. “If this If it’s groundbreaking, I’d love to be part of that change.”
“The Hardest Part” is the lead single from Greene's new album “Notes From a Quiet Life” to be released on June 28. This is also the longest MV produced using Sora technology to date.
Sora is known for its ability to create short videos based on text prompts, which allows director Trillo to construct scenes at a much lower cost than using real actors, scenes and locations.
“Without the constraints of budget and location, I had 100% freedom to explore the possibilities of this couple’s life,” Trillo said.
At the same time, in order to generate usable video footage, Greene needed to write specific, detailed cues that covered not only the images themselves, but also camera angles and character movements.
“We walked through the bubble, it burst, we walked through the bubble gum and came to an empty football stadium,” Trillo wrote in a caption for one video clip. “The scene changes rapidly, showing the students from a frontal perspective, getting bigger and faster.”
The final music video for “The Hardest Part” shows multiple scenes, including a high school, a grocery store, rolling hills, a hallway hung with fluttering white sheets, and walls engulfed in flames.
Of course, Sora still has its limitations. Sometimes it can perfectly implement Trillo's ideas, and sometimes it just generates some confusing and unusable content.
In particular, there are a lot of inconsistencies – for example, when the scene changes, the appearance of the characters will also change.
However, this dreamlike state is also the artistic charm of this MV – it reviews the life of a couple and shows the vague memories of human beings.
“You have to know how to deal with it,” Trillo said of Sora. “You need to give up your free will to a certain extent to adapt to its chaotic nature.”
Faced with Trillo's “uncanny workmanship”, Greene said with emotion: “It's incredible to be able to splice these fragments into a complete story.”
Both Greene and Trillo believe that the combination of AI and music video fields is very promising because they are usually shorter and the production costs are much lower than feature films and TV series.
Trillo stresses, however, that it's important to him not to use this as his primary creative method, but as just another tool in his toolbox.
“The budgets of many music videos don't allow for large-scale creative attempts,” Trillo said. “I think AI can help the music industry in creating things that even Ernest might not have dared to imagine.”
References:
https://venturebeat.com/ai/the-first-music-video-generated-with-openais-unreleased-sora-model-is-here/
https://www.latimes.com/entertainment-arts/business/story/2024-05-02/first-major-music-artist-uses-openai-sora-to-create-music-video
https://twitter.com/paultrillo/status/1786044776745505209