.webp)
Table of contents:
Virtual production has revolutionized the world of filmmaking.
But what exactly is it? And why is it gaining traction in the movie industry?
Let’s find out.
What does virtual production mean?
Virtual production is a filmmaking method that combines virtual and physical worlds to create movies.
You’re probably familiar with the green screen. It’s a tool often used in the background of a shot to enable editors to add visual effects in post-production easily.
In high-end virtual production, the green screen is replaced by a massive LED screen. The high resolution of these screens makes any videos or images on them appear hyper-realistic.
Let’s imagine that this massive LED screen displays an image of a mountain. So when a camera captures footage of this screen, it looks like an actual mountain was captured on video.
Check out the below video to see exactly how virtual production works and how it differs from the green screen.
Using an LED live wall is just one type of virtual production method. Others include visualization and performance capture.
Origins of virtual production
The origin of virtual production began with front and rear projection techniques.
One of the first movies to use rear projection was the film Liliom, which was released in 1930.
Here’s a video that explains how rear-projection works.
The first movie to use front projection was Stanley Kubrik’s 2001: A Space Odessy. Oblivion, which was released in 2013, also used this technology to great effect. While filming the movie, a steady video of moving clouds and the sun was projected onto a large white muslin cloth using projectors.
Other forms of virtual production tech include performance capture and visualization techniques like previs, postvis, and techvis.
In performance capture, an actor's performance is captured and then later translated into a computer-generated 3D character. The technology was first used to capture the movements of Andy Serkis for his role as Gollum in the 2002 Lord of the Rings movie — The Two Towers.
This is a great video to understand how performance capture works. It uses the example of the 2019 movie Alita: Battle Angel.
Now let’s expand on visualization techniques and what each of the three techniques is.
- Previs is a part of pre-production. It involves storyboards, animating the storyboards, concept art, and anything that helps plan out the film.
- Techvis occurs after previs. A techvis artist handles the technical aspects of previs, such as camera placements, shot locations, and necessary green screen measurements for CGI shots.
- Postvis is a part of post-production, where live-action scenes are merged with temporary visual effects to serve as placeholders for the final cut.
An early version of this previsualization was used in Tim Burton's Batman Returns in 1992. It was the 2013 film Gravity that took this tech and visualization techniques to the next level:
In the movie Gravity, the scenes were previsualized shot by shot with the help of highly detailed computer graphics. The film also introduced another cool VFX (visual effects) innovation, the Light Box. This was a 20-foot high enclosure that had 4096 LED bulbs.
The crew programmed these lights with moving pictures of space and the Earth. So when the actors looked at the images projected by the LED bulbs, they could see what they were reacting to in the movie. The lights also provided the proper lighting for different shots.
Gravity pushed VFX boundaries — most of the main shots were digital, and the only real objects were the actors’ faces.
Later on, Rogue One and Solo also used interactive lighting effects to illuminate the actors' faces, using the same technology as Gravity.
These movies also used massive projection screens and high-resolution laser projectors.
One of the most significant virtual production innovations happened during the filming of 2019 The Lion King. The entire movie was filmed in a digital world, and the cinematographers used actual cranes, Steadicam rigs, and drones to control virtual cameras.
Modern virtual productions like The Mandalorian used use LED walls or volumes. These walls display hyper-realistic images. Shows like Star Trek: Discovery and the upcoming Netflix German show 1899 also used this tech.
Benefits of Virtual Production Recordings
Reduced costs
By creating digital backgrounds, virtual production reduces the need for physical sets. It also reduces the need for reshoots and post-production, as most of the work is done during real-time filming. All this massively reduces production costs.
Reduces the need for a physical location
Virtual production eliminates the need for cast and crew to travel to a physical location. And since everything is digital, the crew doesn’t have to worry about changes in environmental lighting and weather that can disrupt the filming schedule.
Working Remotely
Virtual production enables people to work and film remotely as not everyone needs to be on set during shots.
Post-production efficiency
Filmmaking is a long process that usually resembles production in a factory line, with steps like pre-production, production, and post-production.
Unfortunately, this system is inefficient. Any mistake in production increases the time spent in post-production and vice versa. For example, if the director cannot imagine the VFX creature meant to be in the shot, he may shoot the scene incorrectly.
There can be too much uncertainty in the production process without LED walls. And rectifying mistakes in post-production can be costly as well as eat into deadlines.
In contrast, virtual productions allow you to make alterations in real-time. If the director doesn’t like the virtual creature on the LED wall, he can ask the graphics team to change it in real time.
Reduced time on set
When actors work on a green screen, it can be hard for them to imagine what the final shot can look like. However, with LED walls, the actors can see the environment in real-time, which helps them avoid reshoots and excessive time on set.
Creativity
Virtual productions offer endless creative possibilities.
For instance, VFX artists can easily change the glow of the sunset during filming. Or they can remove the sun altogether. If the mountain on the horizon doesn’t look intimidating enough, they can manipulate the mountain to appear bigger.
What do you need for a virtual production setup?
Below are the most important things to purchase if you plan to create a virtual production setup.
Camera tracking technology
A virtual reality headset and tracker like the HTC Vive Pro and Vive Tracker will be needed to track camera movement relative to the LED wall.
iPhone and iPad
Apple's latest devices are excellent for capturing in-engine camera movement, thanks to their LiDAR sensors. These sensors detect the movement of the iPhone camera in space. This is extremely convenient if you want to avoid buying a camera tracker.
With the help of the Unreal iOS app and these sensors, you can move your Apple camera around, and it would change the camera angles in the digital scene. You must connect your PC and Apple device to the same wireless network for this to work.
Real Cameras and Lens
You can use any camera to take the shot, but we highly recommend the Blackmagic Design URSA Mini 4.6K Pro G2 if you have the budget.
Camera Rigs
A camera rig will be needed so you can add accessories like the VR tracker to the camera. Our recommendation is the Neewer DSLR Shoulder Rig.
Capture card
A capture card allows you to stream live camera footage to the computer. This is needed so that the footage can seamlessly stream to the in-game engine in real-time.
An Elgato Cam Link 4K is great for an entry-level card, whereas the Black Magic Decklink 8K Pro is an ideal high-end production card.
LED walls and volumes
LED walls and volumes are made up of multiple banks of LED panels. Each panel is powered by a graphics card like the RTX Quadro.
For budget-conscious people, we suggest the Nvidia Quadro 6000 RTX, whereas, for a high-budget production, the Nvidia Quadro 8000 RTX is preferable.
If you’re a small production house, an LED wall is sufficient, whereas, for larger production houses, we recommend the more expensive LED volume.
Green Screen
You can use a green screen instead of an LED wall if you’re on a tight budget. Just make sure that the screen doesn’t have a lot of wrinkles and that the lighting on it is even throughout.
Unreal Engine
To create the digital world, you will need a game engine like the Unreal Engine 5, which is the industry standard. If you need assets for the digital world, you can use the Unreal Engine marketplace.
How is virtual production done?
Let’s look at how virtual production is done. In short, this is more or less what will happen in a virtual production:
1. Before the filming starts, a couple of key components must be in place: the camera, the camera tracker, the LED wall, and the game engine.
2. Now the filming starts.
The visual effects team needs to first create the scene in the game engine. Let’s suppose the scene is meant to take place in the desert. That means the team will have to create a virtual render of the desert in the engine.
Once that’s created, the render of the desert will be displayed on the LED screen.
3. Next, the actor is placed in front of the screen (which has the render of the desert). So when the camera captures the actor with the LED screen in the background, it appears as if the actor is actually in the desert.
4. If any corrections need to be made in the lighting or color, they can be made in real time with the help of the visual effects team. This kind of real-time editing reduces post-production editing.
And that’s it! This is how virtual production is done in a nutshell.
FAQS on virtual production
What is a virtual set in film?
A virtual set in film uses technologies like green screens and LED walls to create a virtual scene. Real-life actors and objects are then seamlessly incorporated into these scenes.
What is VFX used for?
VFX (visual effects) is used to create on-screen effects and virtual scenes with the help of a computer. Using it, you can create scenes that are difficult to film in real life.
How long has virtual production been around?
Virtual Production has been around for many years. The first instance of the technology is the rear-projection technique used in the 1962 James Bond film, Dr. No.
You can read more about the history of virtual production in the above article.
