Filmmakers test-drive OpenAI’s Sora, inspiring awe – and concern

OpenAI has again demonstrated the capabilities of its already impressive text-to-video platform, Sora, by asking creatives to produce their own short films using it. The public’s response has been a mixture of awe and fear about the potential uses of this technology and the real-world impact it may have.

In mid-February, New Atlas discussed OpenAI’s then-new text-to-video generator, Sora, and showcased some of its work. In the article, Loz Blain likened the system to a ‘toddler’ whose growth was likely to be exponential.

Continuing with the toddler analogy, Sora has reached another developmental milestone. OpenAI placed it in the hands of a bunch of creative folks who produced short films using it. The results run the gamut from surreal to stunningly beautiful.

shy kids

Toronto-based shy kids is a small multimedia production company that has made music videos, advertisements, TV series, shorts, and feature films. The team used Sora to create Air Head, an absurdist but strangely touching short film about a man with a yellow balloon in place of his head.

“As great as Sora is at generating things that appear real – what excites us is its ability to make things that are totally surreal,” said Walter Woodman, Sidney Leeder and Patrick Cederberg, the trio that make up shy kids. “We now have the ability to expand on stories we once thought impossible.”

Paul Trillo

Known for films that traverse genres and formats, Paul Trillo‘s short films have amassed a raft of Vimeo Staff Picks. He has now used Sora to unleash his experimental style, creating a frenetic montage that builds in urgency.

“Working with Sora is the first time I’ve felt unchained as a filmmaker,” he said. “Not restricted by time, money, other people’s permission, I can ideate and experiment in bold and exciting ways. Sora is at its most powerful when you’re not replicating the old but bringing to life new and impossible ideas we would have otherwise never had the opportunity to see.”

Josephine Miller

Co-founder and creative director of Oraar Studio in London, Josephine Miller uses extended reality (XR) to empower brands. With Sora’s help, she created a beautiful and incredibly lifelike underwater scene.

“Sora has opened up the potential to bring to life ideas I’ve had for years, ideas that were previously technically impossible,” she said. “The ability to rapidly conceptualize at such a high level of quality is not only challenging my creative process but also helping me evolve in storytelling. It’s enabling me to translate my imagination with fewer technical constraints.”

Nik Kleverov

LA-based Native Foreign is an award-winning creative agency and production company whose Chief Creative Officer, Nik Kleverov, said that Sora allowed him to move beyond budgetary constraints.

“I’m one of those creatives that thinks in motion, so when I’m in Sora it really feels like I can bring any idea to life,” Kleverov said.

While it might be getting harder to distinguish between AI-generated and real-life, there are still some giveaways. Despite being one of the most advanced AI generators, instead of ‘Bicycle Repair,’ Sora rendered the name of the shop that appears 13 seconds into the video ‘Biycle Repaich.’

Don Allen Stevenson III

Don Allen Stevenson III, who honed his creative skill at DreamWorks Animation, is a fan of the weird. That much is clear from his Sora-generated ‘documentary’ featuring a parade of hybrid animals.

“For a long time, I’ve been making augmented reality hybrid creatures that I think would be fun combinations in my head,” Stevenson explained. “Now I have a much easier way of prototyping the ideas before I fully build out the 3-D characters to place in spatial computers. I feel like [Sora] allows me to focus more of my time and energy in the right places … and the emotional impact that I would like my characters to have.”

Naturally, the naysayers are already commenting that ‘Of course, OpenAI chose these artists’ reviews; they’re all positive.’ And they have a point. What about the dark side of Sora?

Sora’s dark side?

The media widely reported that less than a week after OpenAI first previewed Sora’s capabilities a month ago, actor/writer/director Tyler Perry halted his planned US$800 million studio expansion.

As a movie maker, Perry praised Sora for its “mind-blowing” capabilities and noted that the technology may allow productions to be made without needing travel or set-building. But, as a business owner, Perry expressed concern about the technology’s impact on film industry labor.

“Because as I was looking at it, I immediately started thinking of everyone in the industry who would be affected by this, including actors and grip and electric and transportation and sound and editors, and looking at this, I’m thinking this will touch every corner of our industry,” he told The Hollywood Reporter in February.

Perry wasn’t alone in fearing Sora’s rapid rise. But while many were decrying the tech for ending the film industry, the Wall Street Journal‘s Joanna Stern was interviewing Mira Murati, OpenAI’s Chief Technology Officer. Murati stepped in as OpenAI’s CEO in November 2023 during Sam Altman’s very public – and short-lived – ousting from the role.

“The way that I see it, this is a tool for extending creativity,” Murati said when asked whether Sora would negatively impact the film industry. “And we want people in the film industry – creators everywhere – to be a part of how we develop [Sora] further and, also, how we deploy it.”

Clearly, the CTO’s answer was an exercise in diplomacy. However, when Murati was asked what data OpenAI had used to train the AI, Murati gave an answer that, for 80% of Americans, is concerning: “We used publicly available data and licensed data.” Questioned about whether Facebook or Instagram videos were used, Murati tries to backtrack, saying, “If they were publicly available to use, there might be that data. But I’m not sure; I’m not confident about it.”

OpenAI’s returning CEO, Sam Altman, was questioned about the public’s fear that Sora and AI generally are advancing at breakneck speed, too fast to stop, during a recent podcast with Lex Fridman. In it, Fridman asked, “[F]rom an outsider’s perspective, from me just watching, it does feel like there’s leaps. But to you, there isn’t?”

“So part of the reason that we deploy the way we do, we call it iterative deployment …,” Altman responded. “And part of the reason there is, I think, AI and surprise don’t go together. And also the world, people, institutions, whatever you want to call it, need time to adapt and think about these things … But the fact that people like you and others say you still feel like there are these leaps makes me think that maybe we should be doing our releasing even more iteratively. And I don’t know what that would mean, I don’t have an answer ready to go, but our goal is not to have shock updates to the world. The opposite.”

Where to go from here?

Photographers would know how the Canon EOS 5D Mk II, launched in 2008, ‘broke’ the photo industry. And the film industry. The first full-frame DSLR to feature 1080p video recording, it quickly became the ‘must-have’ camera for both independent and Hollywood cinematographers. It was used to shoot scenes in big-budget movies like Black Swan, Drive, Iron Man 2, The Avengers and Mad Max: Fury Road.

Like the 5D Mark II, Sora is poised to redefine filmmaking. It will enable all filmmakers to unleash their imagination and bring their stories to high-definition life in previously unimaginable ways. The tech also has applications outside of filmmaking, particularly in business, particularly marketing and advertising, training, and education. What about online shopping? Being able to virtually try on clothes to see how they look would be a huge advantage. But what will Sora cost us?

Lifelike AI-generated videos could infringe the copyright of existing works. The creation of deepfakes and misleading content is a real threat that raises ethical concerns. We’ve already touched on publicly available (read: personal) information used for data training. And the previously mentioned impact that Sora might have on jobs. In the US, a pair of Senate Democrats have introduced legislation requiring online platforms to get consumers’ consent before using their personal data to train AI models. But has the horse already bolted with respect to Sora? Is there any point in closing the stable door now?

Sora is already capable of incredible feats. That ‘toddler’ will be running before too long.

Source: OpenAI