Culture

12
The number of frames per second in ‘Into the Spider-Verse,’ whose distinct visual style shows half the frames of traditional films and is designed to evoke the feeling of comic books.
Culture

Why do film people care so much about frame rates?

On the arbitrary love of choppy-looking visuals in cinema.

It was once only necessary for movies to be presented in their correct “aspect ratios” — meaning compressed into an narrow rectangle in the middle of a TV screen or monitor, with black bars at the top and bottom. Otherwise, the annoying movie guys would say, it would be wrong, and they were so confident about this that no one bothered to ask why. In the ensuing decades, the black bar fixation didn’t go anywhere — in fact, it metastasized into an ever-expanding set of instructions for how to orient your phone when taking or viewing images. But now you can expect an additional earful from the annoying movie guy in your life if the playback on what you’re watching is overly smooth. And it’s not just excessive smoothness. Soon you’ll be getting in trouble because playback on your device is smooth, when you’re meant to be watching deliberately choppy footage.

It’s already happening, and it’s only going to get worse.

Your TV, according to annoying movie guys such as Tom Cruise and the director and screenwriter Christopher McQuarrie, is trying to mess up your cinematic experience. In the video above, Cruise and McQuarrie helpfully explain the whole problem: the default settings on your TV take video content that’s supposed to appear as 24 distinct images, and add mushy, in-between frames that make motion look eerily smooth. “This is sometimes referred to as ‘The Soap Opera Effect,’” Cruise gravely informs the viewer, invoking a type of television program intended for women — the opposite, in other words, of Top Gun: Maverick (coming to a theater near you this June).

This motion smoothing problem has been happening for years already, and we’ve all seen a million variations on what this does. For the most part, the effect is pleasantly psychedelic. Other times, the smoothness triggers childhood memories of cheap daytime TV, which, thanks to something called “interlacing,” appeared at roughly 60 silky-smooth frames per second in the U.S. Unless you turn off your VIZIO XVT3D554SV’s “480Hz SPS Smooth Motion technology,” you’re seeing everything at “80 scenes per second, for clarity of fast action scenes and blur-free images.” As a side-effect, when you see the iconic plane taxiing at the end of Casablanca, your brain will incongruously dredge up childhood memories of Jay Jay the Jet Plane.

It’s precisely for this reason that many of us have decided that when Peter Jackson made his 543-minute adaptation of The Hobbit, the biggest problem with the film wasn’t that Peter Jackson openly admitted he was “winging it” on set and basically didn’t shoot off any storyboards, but that he gave us the option to watch the finished movies at 48 frames per second, triggering the dreaded Soap Opera Effect.

In a Gizmodo article titled “The Hobbit: An Unexpected Masterclass in Why 48 FPS Fails,” Vincent Laforet claimed, claimed that Jackson’s decision “killed a lot of the magic of what makes a film entrance an audience.” Eventually, Laforet did throw Jackson a backhanded compliment, adding, “I did find myself becom[ing] more ‘immersed’ in the three-dimensional environment and all of these details — but to the detriment of the film and the narrative itself.”

But according to YouTuber John Hess, a guru of movie frame rates, 24 frames-per-second is the end-all-be-all of frame rates, and that’s never going to change:

Maybe I'm just an old fogey clinging onto my nostalgia goggles. Maybe I’m just not hip to what’s it. I used to be, but then they changed what ‘it’ was, and now what’s ‘it’ seems weird and scary to me. It’ll happen to you too. But on the topic of frame rate, I’m not going to be diplomatic. I’m not going to say ‘It’ll be interesting to see!’ or some meaningless cop-out. No, 24 is going to be with us for as long as there exists a cinematic medium.

In his video about this, Hess explains why he feels this way. He notes that adding more frames to the filmmaking process adds some expense by enlarging file-sizes, and that cinematographers shooting for increased frame rates have to use more light, but he acknowledges that those are just hurdles that can be surmounted. The real reason is essentially inertia. Filmmakers are nostalgic, and, Hess claims, “When you have that love for films, you want your film to have ‘that look,’ the look of the movies that you fell in love with.”

Indeed, I made movies with my friends in the late ‘90s and early 2000s with a camcorder and my friend’s illegally obtained editing software. Several steps toward the end of the process back then involved “making it look more like a movie.” For instance, we used our pirated version of Adobe After Effects to add huge black bars to the top and bottom of the finished movie to match the George Lucas-approved visual geometry of the Star Wars films. Movies marketed to consumers as “widescreen” on VHS existed because the movies really were intended to be shown on screens wider than TVs, but when we made our films, we were just cutting off the top and bottom arbitrarily. We justified this by claiming that we always intended the finished product to be cropped in this way, even when we were shooting it, so this step was mandatory, we would have argued, if anyone had ever called us on it (they didn’t).

But my much smarter friend did more to our footage than reverse engineer the 2.35:1 aspect ratio we fetishized; he also ran it through software filters that “deinterlaced” it, and reduced the apparent number of frames from 29.97 frames per second down to the cinematic standard of 24 fps. I didn’t understand what any of this wizardry was supposed to accomplish when he was explaining it to me. But like magic, once I was seeing fewer frames per second, our movies looked subtly — but noticeably — more movie-like. Not because we were masters of cinema, but because frame rates have an automatic Pavlovian effect on moviegoers.

Something similar was happening around that time in the world of legitimate film: Mike Figgis was filming the first actual movie on “digital video,” which at the time meant a certain kind of video tape. Figgis’ film, Timecode (the less about which is said the better), was shot with four camcorders at 29.97 FPS, which was then transferred onto old-fashioned film in order to be shown in theaters, where film projectors max out at 24 FPS, meaning that Figgis had to do a (probably) more expensive version of what we did.

Another thing that was happening around this time was the rise of the strobing shutter. Saving Private Ryan came out in 1999, and cinematographer Janusz Kamiński — who won an Oscar for his work on the film — shot the film at the normal 24 frames per second (more on this in a moment), but with a 45-degree shutter, meaning he wasn’t opening the camera shutter very far when shooting, which gave the film its now-iconic choppy feel onscreen. By way of an explanation, just know that narrowing the shutter lets less time elapse as each frame is recorded, and that can make the action look kinda cool in a gritty way.

When there’s a lot of motion on screen at once in Saving Private Ryan, the fast shutter effect doesn’t give the image time to blur when you might otherwise expect it to look blurry, so when you string 24 non-blurry images together, and the images are far apart, they kinda strobe like a moth flying in front of your venetian blinds. This effect is more noticeable in the less shaky scenes toward the end (because keep in mind, the choppiness I’m referring to has nothing to do with camera shakiness), so don’t watch this clip if, for whatever reason, you haven’t seen Saving Private Ryan and you don’t want it spoiled:

Over the next few years, this strobing shutter effect became a fad. It imbued movies like Traffic and The Bourne Identity with documentary realism, and in the case of period films like A Knight’s Tale and Gladiator, it showed up sporadically to make the fight scenes look trendy and cool. It’s now part of the visual vocabulary of movies, symbolizing grittiness. According to Steven Spielberg, “Often, when you see an explosion with a 180-degree shutter it can be a thing of beauty, but a 45-degree shutter looks very frightening.” In other words: this brand of choppiness tells the audience “this isn’t all just fun and games.”

In animation, it doesn’t work this way. Animation sometimes looks a little choppy for whatever reason. As someone who has worked as an animator, I’d say it’s because the people behind the film/show don’t feel like adding any motion blur to the crisp still images they poured blood, sweat, and tears into, and in the end, like Kamiński’s explosions, you end up with a finished product that strobes a bit.

This was particularly true when I was doing stop-motion animation, or “claymation” — a.k.a., “the one where you pose 3D characters in fully-rendered diorama sets to create your scene.” Choppy animation doesn’t convey grit in this context — on the contrary, it seems to convey whimsy. For instance, the stop-motion special effects used to create the animals in 2004’s The Life Aquatic with Steve Zissou make those scenes feel more artsy and fun than if those animals had been computer-generated, which would have made them feel like smooth, fake images trying to be real instead of fake images that knew they were fake (and were therefore cool).

It was important for animation to not look cheap and generic around this time, and the cheapest-looking animated property out there back then was ReBoot, a Canadian animated TV show about eerily smooth pseudo-Na’vi people living in a computer world. ReBoot’s nauseatingly smooth characters, and their smooth, TV-optimized bodily movements seemed related, even though they weren’t — not in any process-related way.

To animation fans, a little bit of strobing subtly implies that more than the usual amount of artistry went into a given product. Speaking for myself as an animator, I was proud of how hard every frame was, and I was glad the jittery finished product reflected that. No matter what you might think of the content itself, the effort was on display — for example, here’s a music video my friend and I made in 2007:

Cavil at Rest - Who's There from Nathaniel Miller on Vimeo.

As with many stop-motion productions, that video was output at the standard, cinematic 24 frames per second, but each frame was taken twice. To use the industry term, we took our 24 frames per second “on twos” — so the 24 FPS only applies in terms of formatting and display compatibility. (Explaining this stuff gets pretty tedious pretty fast, but here’s a link to a good video by John Hess about frame rates, which also explains that 29.97 number I mentioned earlier) But in effect, the frame rate for the whole thing is 12 fps, because that’s the number of distinct images you see per second. And it’s very choppy, isn’t it? But also kinda cool, I hope?

But more to the point, shooting on twos disrupts Hess’s nostalgic definition of cinema that passes by at 24 frames per second. In his defense of the magic number 24, he claims that when you tally up a list of favorite movies, “Every movie on your list was shot at 24 frames per second,” though he adds a small note on the screen pointing out that if you’re a fan of silent films, that might not be true, because they were shot in less than 24 frames per second. He’s also leaving out almost all anime films, which are mostly — but far from always — shot on twos.

Low frame rates aren’t all that rare, even in non-animated movies. For instance, the apparent frame rate in a movie sometimes drops down to 12 FPS to allow a shot that was taken at the normal 24 FPS to play in slow motion, or 12 FPS can be used just to give a scene a different look. The effect is much more tragic, or ominous, or feels otherwise seen through a glass, darkly. And it’s down around the minimum number of frames it takes to convince your eyes that you’re seeing objects in continuous motion rather than a slide show. In animation, shooting on twos or “doubles” is faster and cheaper — I can tell you from experience — but it’s also visually distinctive.

The 2012 introductory textbook Stop Motion: Passion, Process and Performance features several quotations from Dug Calder, an animator who worked at the UK-based stop-motion studio Aardman. At Aardman, Calder says, “the style is doubles, but we use singles if the character is moving fast or the camera is moving. Sometimes we move the camera or the puppet while the camera takes a frame to produce a blurred effect […] There is definitely a place for doubles, and it wins Oscars!!”

But Calder also works on computer animated films. He’s a credited animator on 2014’s The Lego Movie, which clearly went out of its way to look choppy. It’s no secret that The Lego Movie profited from nostalgic appeal for people (myself included) who used to make stop-motion films out of Legos, even though the film itself wasn’t stop-motion. But the lengths the filmmakers were willing to go to in service of that somewhat janky, homemade look were extraordinary. In one scene, Lego waves upon the Lego ocean move at six frames per second by my count. That means the film is mimicking stop-motion shot not on twos, but on threes. Animating on threes is by no means rare, particularly in anime, but it’s not usually so obvious.

But until Spider-Man: Into the Spider-Verse, I don’t think there’s ever been a movie that was choppy throughout, and had no practical reason for being that way.

Even in stylized anime and stop-motion, shooting on twos conserves some of the money and effort involved in making some of the most labor-intensive art in the world. But with Spider-Verse, animating on twos was a stylistic choice, and it caught the animation department at Sony off guard. Bob Persichetti, an Aardman veteran, was one of the directors of the film, and he told an interviewer at the website Cartoon Brew, “The idea of animating on twos for a lot of our animation crew was completely foreign to them. They’re younger kids who have not ever done hand-drawn animation so the concept of every other frame and key pose and all that stuff was a little bit soft for them.”

The patent-pending visual vocabulary of the film also included visible lines, simulated printing flaws, and 2-D elements mixed with the 3-D — all combined, according to Persichetti’s interview with Cartoon Brew, to “find a visual language that feels like it’s derivative of a comic book.” Most of this checks out intuitively — they’re lifting visual elements directly from comics, and applying them to a movie — but the images in comic books are stationary, so it’s hard to conceive of why 12 as opposed to 24 frames per second would be more evocative of comic books, other than, perhaps, a subconscious attraction to subtle strobing effects, encouraged by their presence in prestige cinema for a couple of decades running.

Maybe I’m being too harsh. It helps to compare the Spider-Verse look to the grossly smooth — and, once again, Canadian — computer animation that went into a Spider-Man TV show from 2003. Anything that makes it harder for people to confuse your movie with Spider Man: The New Animated Series is probably a good thing:

Still, can you imagine what it would be like if your annoying cinephile dad caught you watching Into the Spider-Verse on a TV with motion-smoothing turned on?

For my part, I still don’t quite understand why it’s perfectly acceptable in the Cinephile World to enjoy a choppy frame rate, but 48 or 60 — or the rumored 120 frames per second that James Cameron might be attempting for the Avatar sequels — are becoming a taboo.

I have a hunch though. In 2016, Ang Lee released a movie called Billy Lynn’s Long Halftime Walk using a 120 FPS 3-D process, and — despite being an interesting and somewhat satirical take on American heroism — it was pretty much panned. I’ve seen Billy Lynn and all three of Peter Jackson’s Hobbit epics, and the problem is that these are are all (unreleased Avatar sequels included) sentimental stories in the classic cinematic mold, and their uses of high frame rates foster immersion, when what’s called for might not be immersion at all, but a visual style that evokes the fog of nostalgia. Less realism, in other words, not more. And it strikes me that this is exactly the emotional shortcut a filmmaker is taking when they dial down the number of frames rather than dial them up.

Peter Jackson’s Hobbit left Justin Caffier of VICE “feeling as if I was watching the local news.” But plenty of movies — certain thrillers and horror movies in particular — already look like the news. That’s part of what makes them so immersive. Cloverfield is fascinating with the frame rate cranked up. So is Gravity. I know this because I was a little dickens and watched them that way, by choice, on a VIZIO with “480Hz SPS Smooth Motion technology.” I willfully violated the taboo, and you want the truth, Tom Cruise? They didn’t look like soap operas at all. The high frame rate made them even scarier, and I liked it.