Motion-capture has totally altered our movie-going experience in just a few years. From the Na'vi to Gollum to Mark Ruffalo's Hulk, our favorite movie characters are increasingly a blend of computer animation and an actor's real performance.
So what's next? What's the technology in the pipeline that's going to change how you watch movies, as much as mo-cap has? We asked some top visual effects professionals, and here's what they told us.
Eric Barba, Visual Effects Supervisor, Digital Domain (Academy Award-winner for Benjamin Button):
From my perspective, you have a handful of visionary filmmakers, and those visionary filmmakers are usually what usually allows us — not just in visual effects, in pretty much every art in the movie-making process, that pushes us on to the next plane. And then other directors see that and they follow through, and pretty soon it's the norm in film.
Click to view Just look at James Cameron, because he's a visionary film-maker. We did a music video ("Ghosts") with Michael Jackson here at Digital Domain in the mid-1990s, I think Stan Winston directed it. We used this thing called "motion-capture." Jim [Cameron] saw that, and a lot of people saw that. Flash forward, and Digital Domain is working on James Cameron's Titanic, and a lot of that technology is used on Titanic on the digital extras. Certainly, Jim knows about technology, and what he knows about technology, he's applying it forward. Ten years later, he's using that motion-capture technology on Avatar.
Click to view Other directors could kind of fall under that same category. The movie I was able to work on was with David Fincher. I started working with him on commercials, and he was kind of looking at all the little things we did. He liked to use commercials as his R&D and testing ground. And I did a bunch of spots with him, and that ultimately led up to The Curious Case of Benjamin Button, where he believed we were ready to do "digital thespians," as we called them at the time. In order to portray the character of Little Benjamin, we needed to make the character small, and of course Brad Pitt is 6'2, so he believed the only way to do it correctly was digitally. As a visionary, believing in us — that allowed us to push that technology to the next level.
And the rest is kind of history. So that's another example. Obviously Peter Jackson falls into that category, if you think about what he did for Lord of the Rings, and things like the armies fighting each other, and mass quantities of things. And then other film-makers and commercials have used that technology going forward. So it really comes down to those visionary film-makers that help us and believe in us, that push us to the next level.
So motion-capture was a slow transition, if you look at James' progression from Titanic to Avatar. If you want to look forward, you can look at the things that we've done on only a handful of movies that will be the accepted norm down the road.
If you use Benjamin Button and maybe Tron Legacy as an example of digital thespians, it only takes one film-maker to say, "I believe we can do this," and then other film-makers will pick up on it and start using it down the road. It depends what those film-makers want next in their films, and what the screenwriters dream up.
Click to view In Tron, making Jeff Bridges a young version, nobody would have believed we could even attempt that. But that team at Disney, and certainly director Joseph Kosinski, believed we could do the Clu character for that film — which made an interesting story point for the movie, that we hadn't quite seen. [The technique that makes Bridges young] is a derivative [of the Benjamin Button technique], it's the next advancement.
If you use the mo-cap model of that ten-year period [from first use to widespread acceptance], I think it's safe to say that you'll see [digital thespians] more and more, because it becomes more and more accepted.
[Speaking of the Uncanny Valley] I like to say I have property there, that I sometimes visit. Because it's such a long road to get across that valley, that sometimes you just stay there for a good period before you understand what that is. I think it's a fantastic explanation [of this phenomenon] and a fantastic name, and again I spent so many years there. The thing with the Uncanny Valley is, it's like any magician's trick: If it's pulled off to a level where the audience isn't even aware you're doing it, then it's completely successful. But if you tell the audience ahead of time, "Hey, I'm going to do this trick on you, and this is how I'm going to do it," and then you show it to them, then obviously off the bat it's going to be less successful. The most successful tricks are the ones where the audience isn't even aware they're being played.
[The other great advance in technology may be] virtual production. In the old days, we had this thing called pre-production, and we had production, and post-production. We still use that terminology, but when you have a movie like Tron, where 80 percent of the movie is a digital production, then the method in which you go about that film is much closer to an animated film. We're finding more and more of these movies are embracing the idea of fully engaging the visual effects company in the very beginning, to have us on board from the very beginning, to help them plan and develop and think things out so they get the most bang for their buck on the screen.
Wayne Stables, VFX Supervisor, Weta Digital:
There's no doubt that performance capture has changed the way that we make movies. Whereas earlier we might have photographed a plate and then defined a characters performance later in post production, now creative decisions about performance and camera can be made in production with the director, actors and cinematographer.
Click to view I believe the next technology that we need to expand upon, is capturing even more relevant information during production. For instance, if we can replicate accurate lighting in real time then those decisions can also be made at the same time as the camera and performance are being worked out, the same as if we were shooting a live action movie. We need technology that allows a director to make more informed creative decisions about their film, whether it's live action or fully computer generated, when they are shooting instead of having to wait until they are in post production. In this way everything including performance, camera, lighting, effects, and the set can be considered at once.
James David Hattin, Zoic Studios:
You know how, when you stand in the shower, you start solving all of the world's problems? Those times when you dwell on what you are going to do today? I've stood there under the hot water, often wondering, "What's next?" Here's the thing, we've done it all. We've imagined colossus worlds full of vegetation and life forms small and large (Avatar/Star Wars). We've imagined and created creatures too impossible to even have a walk cycle (Lost in Space's three-legged crawlies.)
Click to view We have really done it all. Technology has had a lot to do with it, it has grown as the industry has required it to. No longer, is it simple blue and green screen actors needing to fly through the sky, now we have to make the sky as well — interactive clouds in the sky, that people can fly through. Superman never did that in the 70's.
Click to view Motion capture certainly is one of the latest and greatest technologies to get the performance across. The subtlety of human motion, as translated through the body and face, is virtually impossible to recreate in the computer. So, here we have it. Full body capture as seen in Pirates of the Caribbean, where a whole crew of CG creatures roam a physical set, or the motion capture suits used behind the scenes to create creatures like Falling Skies' Overlord character, one among many hundreds or thousands of photo-real characters that play in movies and TV.
The one thing we have not done to date is recreate a human. We've gotten close, but always an alien face. No one has made a human actor that was digital and convincing. Smeagol is great and expressive, but he's not "human." There has yet to be a [human] character that can stand up to scrutiny and come off as photoreal and alive. To cross the uncanny valley, [the place] where CG characters look more ghoulish than real.
I don't know what technology is going to drive that, I don't know that it's a technological achievement. I think it will be an artist achievement, empowered by the latest tools. We can already render flesh in CG. Clothes aren't a problem. Hair is almost there. There are a host of shaders for all of the 3D rendering packages that bring all the parts together. The missing element, though, is life. That hasn't been done yet. It will also be the biggest game changer. When we can replace actors with digital versions, we will cross that bridge technologically that will allow us to reuse people from any generation and any time. Buster Keaton and a young Johnny Depp acting together.
It will use Motion Capture as part of it, but there will be something that makes the life happen, something in the eyes, something that will fool us the viewer. When we look with even casual eyes, we know something isn't right. That hasn't been overcome. I don't know when. The pieces are there. They just need to be put together in the right order.
