You’ve obviously heard the the phrase “a picture is worth a thousand words”, and you probably even have an idea why we say that. But rarely do people delve deeply into the underlying reasons for this truth. And those reasons can be incredibly useful to know. They can tell you a lot about why we communicate they way we do, how art works, and why it’s so damn hard to get a decent novel adaption into theaters.
I’m going to be focusing mostly on that last complaint in this post, but what I’m talking about has all sorts of broad applications to things like good communication at work, how to tell a good story or joke, and how to function best in society.
So, there’s always complaints about how the book or the comic book, or whatever the original was is better than the movie. Or the other way around. And that’s because different artistic media have different strengths in terms of how they convey information. There are two reasons for this:
- Humans have five “senses”. Basically, there are five paths through which we receive information from the world outside our heads. The most obvious one is sight, closely followed by sound. Arguably, touch(which really involves multiple sub-senses, like heat and cold and pain) is the third most important sense, and, in general, taste and smell are battling it out for fourth place. This is an issue of “kind”.
- The second reason has to do with what I’m calling information density. Basically, how much information a sense can transmit to our brains in how much time. This is an issue of “degree”. Sight, at least form humans, probably has the highest information density. It gives is the most information per unit of time.
So how does that effect the strengths of various media? After all, both movies and text mostly enter our brain through sight. You see what’s on the screen and what’s on the page. And neither can directly transmit information about touch, smell, or taste.
The difference is in information density. Movies can transmit visual information(and audio) directly to our brains. But text has to be converted into visual imagery in the brain, and it also takes a lot of text to convey a single piece of visual information.
AI, in the form of image recognition software, is famously bad at captioning photos. Not only does it do a crappy job of recognizing what is in a picture, but it does a crappy job of summarizing it in text. But really, could a human do any better? Sure, you are way better than a computer at recognizing a dog. But what about captioning? It takes you milliseconds at most to see a dog in the picture and figure out it is jumping to catch the frisbee. You know that it’s a black lab, and that it’s in the woods, probably around 4 in the afternoon, and that it’s fall because there’s no leaves on the trees, and it must have rained because there are puddles everywhere, and that…
And now you’ve just spent several seconds at least reading my haphazard description. A picture is worth a thousand words because it takes a relatively longer amount of time for me to portray the same information in a text description. In fact, it’s probably impossibly for me to convey all the same information in text. Just imagine trying to write out every single bit of information explicitly shown in a half-hour cartoon show in text. It would probably take several novels’ worth of words, and take maybe even days to read. No one would read that book. But we have no problem watching TV shows and movies.
Now go back and imagine our poor AI program trying to figure out the important information in the photo of the dog and how to best express it in words. Yikes. But as a human, you might pretty quickly decide that “a dog catches a frisbee” adequately describes the image. Still takes longer than just seeing a picture, but isn’t all that much time or effort. But, you’re summarizing. A picture cannot summarize and really has no reason to. With text(words) you have to summarize. There’s pretty much no way around it. So you lose an enormous amount of detail.
So, movies can’t summarize, and books must summarize. Those are two pretty different constrains on the media in question. Now, imagine a a radio play. It’s possible you’ve never heard one. It’s not the same as an audiobook, despite communicating through the same sense(audio), and it has some serious advantages over books and audiobooks. You don’t have to worry about conveying dialogue, or sound information because you can do that directly. Emotion, accents, sound effects. But of course you can convey visual information like a movie, and unlike in a book or an audiobook, it’s a lot more difficult to just summarize, because you’d have to have a narrator or have the characters include it in dialogue. So raw text still has some serious advantages based on the conventions of the form. Similarly, radio dramas/audio plays/pod casts and movies both have to break convention to include character thoughts in storytelling, while books don’t.
So, audio and television media have major advantages in their specific areas than text, but text is in general far more flexible in making up for any short-comings. And, it can take advantage of the summary nature of the medium when there’s a lot of unnecessary information. Plus, it can count on the reader to be used to filling in details with their imagination.
Film and radio can’t do that. They can use montages, cuts, and voiceovers to try and imitate what text can do, but it’s never quite the same effect. And while language might not limit your ability to understand or experience concepts you have no words for, the chosen medium absolutely influences how effective various story-telling techniques can be.
Consider, an enormous battle scene with lots of action is almost always going to be “better” in a visual medium, because most of the relevant information is audio and video information. An action scene involving riding a dragon through an avalanche while multiple other people try to get out of the way or stop you involves a great deal of visual information, such that a text can’t convey everything a movie could. Watching a tennis match is always going to be more exciting than reading about one, because seeing the events lets you decide without an narrator interference whether a player has a real shot at making a return off that amazing serve. You can look at the ball, and using past experience, imagine yourself in the player’s place and get a feeling of just how impressive that lunging backhand really was. You can’t do the same in text, because even if the writer could describe all the relevant information such that you could imagine the scene exactly in your head, doing so would kill the pacing because of how long reading that whole description would take.
The very best artists in any medium are always going to use that medium to its fullest, exploiting any tricks or hacks as best as possible to make their creation shine. And that means they will (often unconsciously) create a story tailored to best take advantage of the medium they are working in. If and when the time comes to change mediums, a lot of what really made the art work won’t be directly translatable because that other medium will have different strengths and have different “hacks” available to try to imitate actually experiencing events directly. If you play videogames or make software, it’s sort of like how switching platforms or programming languages (porting the game) means some things that worked really well in the original game won’t work in the ported version, because the shortcut in the original programming language doesn’t exist in the new one.
So, if video media have such a drastically higher information density than text, how do really good authors get around these inherent shortcomings to write a book, say? It’s all about understanding audience attention. Say it again, “audience attention.”
While the ways you manipulate it are different in different media, the concept exists in all of them in some form. The most obvious form is “perspective”, or the viewpoint from which the audience perceives the action. In film, this generally refers to the camera, but there’s still the layer of who in the story the audience is watching. Are we following the villain or the hero? The criminal or the detective?
In film, the creator has the ability to include important visual information in a shot that’s actually focused on something else. Because there’s no particular emphasis on a given object or person being included in the shot, things can easily be hidden in plain sight. But in a book, where the author is obviously very carefully choosing what to include in the description in order to control pacing and be efficient with their description, it’s a lot harder to hide something that way. “Chekov’s gun” is the principle that irrelevant information should not be included in the story. “If there’s a rifle hanging on the wall in Act 1, it must be fired in Act 2 or 3.” Readers will automatically pay attention to almost anything the author mentions because why mention it if it’s not relevant?
In a movie, on the other hand, there’s lots of visual and auditory filler because the conceit is that the audience is directly watching events as they actually happened, so a living room with no furniture would seem very odd, even if the cheap Walmart end table plays no significant role in the story. Thus, the viewer isn’t paying particular attention to anything in the shot if the camera isn’t explicitly drawing their eye to it. The hangar at the Rebel Base has to be full of fairly detailed fighter ships even if we only really care about the hero’s. But not novel is going to go in-depth in its description of 30 X-wings that have no real individual bearing on the course of events. They might say as little as “He slipped past the thirty other fighters in the hangar to get to the cockpit where he’d hidden the explosives.” Maybe they won’t even specify a number.
So whereas a movie has an easy time hiding clues, a writer has to straddle the line between giving away the plot twist in the first 5 pages and making it seem like a deus ex machina that comes out of nowhere. But hey, at least your production values for non-cheesy backgrounds and sets are next to nothing! Silver linings.
To get back to the main point, the strengths of the medium to a greater or lesser extent decide what kind of stories can be best told, and so a gimmick that works well in a novel won’t necessarily work well in a movie. The narrator who’s secretly a woman or black, or an alien. Those are pretty simplistic examples, but hopefully they get the point across.
In the second part of this post a couple days from now, I’ll be talking about how what we learned here can help us understand both how to create a more vibrant image in the reader’s head, and why no amount of research is going to allow you to write about a place or culture or subject you haven’t really lived with for most of your life like a someone born to it would.