Superhero and fantasy films don’t need green screens anymore.
From the early years of Hollywood, set designers spent hours painting film backdrops by hand. They tried to make the as realistic as possible, hoping nobody would notice that the trees were suspiciously still, or that the clouds never shifted. Until we shifted to green screens. Here actors had to imagine their surroundings. But now, we can build animated background characters as well as entire mountains and move them to where best suits us during the shoot, all thanks to virtual production.
What is Virtual Production?
In lay terms, virtual production is an innovation that filmmakers can use to lay out a film in a digital environment before commencing the shoot on a physical set. According to Technicolor, this includes defining lensing, set dimensions, asset placement, and exact camera movements.
Instead of a traditional green screen, a real time virtual scene is rendered on an LED wall. This wall surrounds the tangible props and actors, and it shifts persepctive when the camera moves. The key here are the game engines that use gaming technologies to navigate a virtual landscape. The live camera tracking enables accurate camera movements to be realized in real time into the rendering platforms. The unison of movement between camera and imagery creates the illusion of a physical location, enabling filmmakers to capture both live action and CGI in-camera together. This is called parallax.
This enclosed set made up of LED walls and ceilings is a type of new sound stage known as the volume. Currently, 100 LED sound stages exist around the world. Atleast 200 more are in the works and will expand the possibilities for filmmakers who want to replicate real-world scenarios virtually on the sound stage.
The technology
Game engines like Unreal Engine and Unity are being used as the advanced real-time 3D creation software. These are more than online tools; they get final pixels in-camera and on set, in real time.
Numerous industries are using Unreal Engine’s virtual production to create content. From architecture, engineering, construction, games, film, television, to automotive, broadcast, live events, as well as training, simulation, manufacturing and advertising, all these are making use of this brilliant technology.
Initially built for gaming, these new technologies in filmmaking are now influencing how games are being produced. The concept of the avatar was created for game design, but is now being used in feature films. Conversely, game developers are learning camera techniques used by directors of phtography during virtual production film making that is elevating the quality of gaming visuals.
What are the benefits of using Virtual Production?
Reduced production time:
The automation of the volume stage means that production times and set ups are being greatly reduced or even circumvented. You can move around props and assets on your set in real time. Additionally, you can experiment with camera angles and lighting. This prevents multiple recreations and edits. Through a combination of immersive technologies such as virtual reality, augmented reality, CGI and game-engine technologies, filmmakers see their scenes unfold as they are composed and captured on set.
No expensive outdoor shoot:
The team can complete lengthy and expensive location shoots in one trip by capturing the ideal light. The footage can be brought into the studio to recreate backdrops as many times as needed in a controlled environment. The volume sound stage makes set extensions, crowd duplication, and other green screen features easy.
Better context for actors:
Intricate, fantasy worlds can be created by the VFX team. Actors can see and react to these live backgrounds in real time. Unlike the green screens, where actors had to use their imagination.
Remote virtual collaborations:
Virtual production allows for real-time collaboration among filmmakers, regardless of where they are in the world. Almost like a virtual team meeting that occurs daily in corporate offices these days due to the pandemic. They can use tools that simultaneously integrate and manipulate live action and computer-generated assets.
Easier for new filmmakers:
Upcoming and budding filmmakers, with lesser budgets, can use virtual production to make short films. High end effects can be achieved using projectors instead of LED screens in a large sound stage.
Student filmmakers who are learning these new technological practices at the beginning of their careers, are at an advantage against their more experienced counterparts.
Projects which have used Virtual Production
Since the release of the 2019 re-imagining of The Lion King, the term ‘Virtual Production’ has become synonymous with “live-action” films. Jon Favreau, who we all know as Pete Becker, Monica Geller’s boyfriend in Friends, has been leading the direction of virtual production films. From Jungle Book to The Lion King and The Mandalorian, Favreau has spearheaded some of the initial innovations in virtual production, creating masterpieces.
The Star Wars spin-off series directed by Jon Favreau, The Mandalorian used Epic Game’s Unreal Engine. Thanks to a 20-feet high, 270-degree semicircular LED video wall, they could display the ice world of Maldo Kreis to the sinister hideout of The Client. There were three operators working various stations. One station which controlled the master Unreal Engine set up, would record takes and the slate, saving the data. Along with the second, which was dedicated to editing, and another which was for compositing, these three were called the “barin bar” on set. The machines relayed their video feed to a series of screens that was called the ‘video village’.
To light the scenes within the volume stage, the LED screens were used. These were powered by an iPad controller. It produced color correction, exposure control, time of the day, and lights that could be adjusted to any shape or size. The lighting system eliminated issues of light flares, and the overhead screens created the final reflection effects on hte metal suits. This drastically reduced the need for a slate of post effects.
However, the Lion King was another animal altogether. It required a blend of traditional live-action filmmaking techniques, along with state-of-the-art virtual reality tools and the highest level of computer-generated animation. This was a step beyond the techniques Favreau used to bring 2016’s “The Jungle Book” to life, which had earned them an Oscar for the best visual achievement.
For Lion King the filmmakers were literally inside the monitor. They put on a VR headset and were in Africa, where they could walk around and see and feel the filmmaking process as though they were there. Thevirtual characters were brough to life by Artificial Intelligence. The filmmakers created a 3D model of a lion, and then taught the AI to make the lion act hungry, or cold, or as if it’s looking for food.
This form of filmmaking or storytelling is more about “world-building”, according to Ben Grossman, the virtual production supervisor of Lion King.
Future of VFX and Virtual Production
Gaming companies are working in tandem with the film industry. As these virtual and photorealistic technologies improve, both the film and game industries can use the same techniques and sets in all stages of production. Virtual locations of games can be repurposed, with pre-existing rigs to be used in conjunction with motion capture to create original content.
Just as digital cameras simplified filmmaking and made it more affordable in the past 20 years, virtual production will further level the playing field for the entire entertainment industry. No expensive location shoots; effect on-camera; 10 hours of golden lighting. What’s not to like!
Virtual production can also be used for installations and projections for events, marketing, and commercial purposes. Fashion shows, art spaces, exhibitions galas, concerts, broadcast networks, architecture, automotive launches etc. are all innovating with this technology. The possibilities are limitless.
Virtual production is the most significant innovation for filmmaking since digital cameras replaced traditional film camera. It is changing the filmmaking production process forever.
It’s looking good for the future of moviemaking and TV!
Looking for CGI and VFX experts? Aligned Studios has an entire animation vertical. Contact us.