Does Fiction Affect Reality? Duh.

[CW: content warning for discussion of sexual violence at the end of the post.]

I’ve seen a lot of discussion lately on whether there are moral/ethical consequences to writing certain kinds of fiction. And the answer to me appears to be an unequivocal “yes”. Although regular readers of this blog are probably aware that I have a wide range of interests, the primary purpose of this site is to explore what’s called “speculative linguistics”, that is the combination of real language science and its depiction in fiction, especially speculative fiction such as science fiction and fantasy. Maybe the most famous example of speculative linguistics is the “conlang”, short for constructed language, which is an umbrella term for artificial languages created for a variety of purposes, but most commonly for use as magical or alien languages in speculative fiction/sff(h) literature.

And that’s a fun topic. But today we’re going to take a dive into the science side of speculative linguistics and talk about the relationship not only between fiction(thought) and reality, but also between both of them and the intermediary of language. This could be a dull boring article, or I could use my actual writing style to make my point:

Human beings do not stand on a hard bedrock of objective reality, but rather swim through a vast ocean of narrative, catching in their gaping mouths whatever strands feed their desires of the moment. It’s quite a philosophical argument whether an objective reality even exists, but I’m going to assume one does for the purposes of argument. But even assuming that, there are two layers between objective reality and our perception of it: the first is the channel of our senses, which make different kinds of information about the world around us available to our minds. If you’ve ever taken one of those colorblindness tests as a kid, then you know that not only do these senses only capture limited information, but they are unreliable.

Although we usually talk about “the five senses: touch, taste, hearing, sight, and smell”, in fact what we really have is visible light detection(a narrow band of electromagnetic radiation), sound wave detection(sensing disturbances primarily in air particles, but also solids and liquids), pressure sensitivity, temperature sensitivity, a weak ability to detect airborne chemicals, a moderate ability to differentiate chemicals by out taste buds, and depending on who you listen to, a couple other minor ways of capturing information. So, an actually very limited way of measuring “reality”.

And then, of course, our brain filters out, without any real conscious control on our part, the “unnecessary” information, such as the feeling of our clothes, various background and far away noises, etc. And finally, after all that, we (only) guess at the connections between the various limited streams of sensory input to develop a model of the world and its natural laws.

And then, finally, we condense this information down into words, which are the primary form of passing information between separate human consciousnesses. In modern times we have things like videos or audio files, or memes/gifs. And of course dance, or more importantly music, can be used to communicate.

At this point you may be wondering if I’m actually going to talk about fiction versus reality at all.

But it’s important to understand all these little details of how our brains and senses function, because “fiction” is pleasurable to us because it engages these senses in ways that the real world doesn’t always. Our brains are designed to find useful patterns for navigating “objective reality”/the world based on our limited sensory inputs. And fiction is a way to both create/manipulate and comment on the patterns our brains discover to create a satisfying emotional reaction. Now you know what a “narrative” is. An artificial pattern designed to evoke a specific emotional reaction.

Our brains learn patterns by discovering consistent outcomes to various actions/combinations of sensory input. And we base not only our intentional actions on those patterns, but even our feelings about things are unconscious reflections of those patterns. People are not born with a full and innate set of feelings and emotional responses; we develop them over time based on our experiences.

The goal of fiction is to create a narrative that closely mimics our learned patterns and our emotional responses to those patterns, and to trick us into seeing those narratives as “real” on an emotional level, even if intellectually we know that dragons aren’t real, for example. And because we have studied fiction for a long time, and practiced it, and are surrounded by it, we’ve gotten very good at tricking our brains into treating it as almost the same as patterns we’ve learned from “real life” experiences. If words on a page could not affect they way we respond emotionally to reality, then all of human culture would have been unsuccessful. Propaganda and “fake news” would not be so effective.

Our brains have a great deal of trouble differentiating patterns learned from fiction from those learned from reality. So no, fiction cannot “affect reality”, but it can and absolutely does, even in ways you aren’t aware of at the time, affect our perception of reality on a fundamental level. And because humans and our opinions and culture live almost entirely on a diet of narrative, our perceptions and reality are basically the same thing.

If you watch people behave a certain way and that behavior is almost always met with approval, or at least not disapproval, your brain learns that that behavior is good, or at least acceptable/normal. And as social beings, we base our behavior far more on what we are taught is acceptable than on our own personal reactions. As much as people try to deny it, we do a very poor job of distinguishing between “reality” and fiction, when we look for examples of acceptable behavior. Your brains is almost equally willing to use behavior depicted in stories to determine what is acceptable as behavior you see with your own eyes. Why else would advice columns or r/amitheasshole and r/relationships be so popular? If you trust Dear Abby’s relationship advice as much as your mother’s, why wouldn’t you believe it when behavior shown in a book is clearly approved of by the author?

Your opinions as an individual are based at least as much on the prevailing views of your culture as on your own personal experiences. You’re as like to believe Superman telling you something is okay as you are your father.

I think it’s useful to point out that of course fiction is only one influence on your beliefs, and also that that influence only applies to the situations depicted in the story. Violent videogames won’t make you a killer unless you find yourself on a HALO fighting the Flood. But certainly playing enough Call of Duty or Gears of War will make you look more favorably on war/violence as a solution to certain types of conflict.

And we can also look at other sources of narrative besides prose fiction to prove our point: if all you know of someone is their image on social media, you’re likely to believe that that’s who they are in real life. They’ve created a narrative, a likely partially fictional one, to influence your perception of them, and it works. If you believe someone is an amazing person, it doesn’t really matter if that’s true; we base our actions on our opinions, because of course it’s impossible to actually know every single truth of objective reality.

And finally, we need to remember that the way brains learn means that both quantity of evidence–the number of times you are exposed to a certain narrative–and how long you’ve been exposed to that evidence without counter-evidence is far more important than quality of evidence–your personal experiences on the topic. if you’ve been told your whole life, by parents, friends, television, books, etc, that staking is romantic for example, you won’t immediately realize that’s not true the first time you experience stalking.

If you’ve been told your whole life that “leading someone on” means you owe them sex, the fact that you don’t want to have sex with them, or even the fact that they bullied you into it and you hated it, won’t immediately counteract years of cultural conditioning. You won’t immediately realize that you don’t actually “owe” them sex, or that just because they claimed to feel “lead on” doesn’t mean you actually did so.

To make an extreme example, just because an example of child porn was a cartoon, and therefore “didn’t hurt any real people”, or just because that creepy m/m romance by a straight woman wasn’t about real people and “therefore it can’t be fetishizing”, that doesn’t mean it has no effect in that area. A book or a cartoon or a song still applauds or condemns some form of behavior, and it can and does still reinforce a narrative about what’s okay and what isn’t.

tl;dr– Reality doesn’t matter. Perception of reality matters. If something like a book affects someone’s perception of reality/acceptable behavior/opinion on global warming, that’s just as good as affecting reality, because the person will act on that perception. People often can distinguish between fiction and reality, but that doesn’t mean they actually do, especially if that fiction supports and opinion they already hold.

Tune in next time for a discussion on the actual mechanics of how fiction and use of language can be used to affect people’s perception of reality, emotional response to a subject or scene/character, and maybe a little bit on how you can use this to make a conlang or culture really stand out on the page.

 

Why Is A Picture Worth a Thousand Words? Information Density in Various Media

You’ve obviously heard the the phrase “a picture is worth a thousand words”, and you probably even have an idea why we say that.  But rarely do people delve deeply into the underlying reasons for this truth.  And those reasons can be incredibly useful to know.  They can tell you a lot about why we communicate they way we do, how art works, and why it’s so damn hard to get a decent novel adaption into theaters.

I’m going to be focusing mostly on that last complaint in this post, but what I’m talking about has all sorts of broad applications to things like good communication at work, how to tell a good story or joke, and how to function best in society.

So, there’s always complaints about how the book or the comic book, or whatever the original was is better than the movie.  Or the other way around.  And that’s because different artistic media have different strengths in terms of how they convey information.  There are two reasons for this:

1. Humans have five “senses”.  Basically, there are five paths through which we receive information from the world outside our heads.  The most obvious one is sight, closely followed by sound.  Arguably, touch(which really involves multiple sub-senses, like heat and cold and pain) is the third most important sense, and, in general, taste and smell are battling it out for fourth place.  This is an issue of “kind”.
2. The second reason has to do with what I’m calling information density.  Basically, how much information a sense can transmit to our brains in how much time.  This is an issue of “degree”.  Sight, at least form humans, probably has the highest information density.  It gives is the most information per unit of time.

So how does that effect the strengths of various media?  After all, both movies and text mostly enter our brain through sight.  You see what’s on the screen and what’s on the page.  And neither can directly transmit information about touch, smell, or taste.

The difference is in information density.  Movies can transmit visual information(and audio) directly to our brains.  But text has to be converted into visual imagery in the brain, and it also takes a lot of text to convey a single piece of visual information.

AI, in the form of image recognition software, is famously bad at captioning photos.  Not only does it do a crappy job of recognizing what is in a picture, but it does a crappy job of summarizing it in text.  But really, could a human do any better?  Sure, you are way better than a computer at recognizing a dog.  But what about captioning?  It takes you milliseconds at most to see a dog in the picture and figure out it is jumping to catch the frisbee.  You know that it’s a black lab, and that it’s in the woods, probably around 4 in the afternoon, and that it’s fall because there’s no leaves on the trees, and it must have rained because there are puddles everywhere, and that…

And now you’ve just spent several seconds at least reading my haphazard description.  A picture is worth a thousand words because it takes a relatively longer amount of time for me to portray the same information in a text description.  In fact, it’s probably impossibly for me to convey all the same information in text.  Just imagine trying to write out every single bit of information explicitly shown in a half-hour cartoon show in text.  It would probably take several novels’ worth of words, and take maybe even days to read.  No one would read that book.  But we have no problem watching TV shows and movies.

Now go back and imagine our poor AI program trying to figure out the important information in the photo of the dog and how to best express it in words.  Yikes.  But as a human, you might pretty quickly decide that “a dog catches a frisbee” adequately describes the image.  Still takes longer than just seeing a picture, but isn’t all that much time or effort.  But, you’re summarizing.  A picture cannot summarize and really has no reason to.  With text(words) you have to summarize.  There’s pretty much no way around it.  So you lose an enormous amount of detail.

So, movies can’t summarize, and books must summarize.  Those are two pretty different constrains on the media in question.  Now, imagine a a radio play.  It’s possible you’ve never heard one.  It’s not the same as an audiobook, despite communicating through the same sense(audio), and it has some serious advantages over books and audiobooks.  You don’t have to worry about conveying dialogue, or sound information because you can do that directly.  Emotion, accents, sound effects.  But of course you can convey visual information like a movie, and unlike in a book or an audiobook, it’s a lot more difficult to just summarize, because you’d have to have a narrator or have the characters include it in dialogue.  So raw text still has some serious advantages based on the conventions of the form.  Similarly, radio dramas/audio plays/pod casts and movies both have to break convention to include character thoughts in storytelling, while books don’t.

So, audio and television media have major advantages in their specific areas than text, but text is in general far more flexible in making up for any short-comings.  And, it can take advantage of the summary nature of the medium when there’s a lot of unnecessary information.  Plus, it can count on the reader to be used to filling in details with their imagination.

Film and radio can’t do that.  They can use montages, cuts, and voiceovers to try and imitate what text can do, but it’s never quite the same effect.  And while language might not limit your ability to understand or experience concepts you have no words for, the chosen medium absolutely influences how effective various story-telling techniques can be.

Consider, an enormous battle scene with lots of action is almost always going to be “better” in a visual medium, because most of the relevant information is audio and video information.  An action scene involving riding a dragon through an avalanche while multiple other people try to get out of the way or stop you involves a great deal of visual information, such that a text can’t convey everything a movie could.  Watching a tennis match is always going to be more exciting than reading about one, because seeing the events lets you decide without an narrator interference whether a player has a real shot at making a return off that amazing serve.  You can look at the ball, and using past experience, imagine yourself in the player’s place and get a feeling of just how impressive that lunging backhand really was.  You can’t do the same in text, because even if the writer could describe all the relevant information such that you could imagine the scene exactly in your head, doing so would kill the pacing because of how long reading that whole description would take.

The very best artists in any medium are always going to use that medium to its fullest, exploiting any tricks or hacks as best as possible to make their creation shine.  And that means they will (often unconsciously) create a story tailored to best take advantage of the medium they are working in.  If and when the time comes to change mediums, a lot of what really made the art work won’t be directly translatable because that other medium will have different strengths and have different “hacks” available to try to imitate actually experiencing events directly.  If you play videogames or make software, it’s sort of like how switching platforms or programming languages (porting the game) means some things that worked really well in the original game won’t work in the ported version, because the shortcut in the original programming language doesn’t exist in the new one.

So, if video media have such a drastically higher information density than text, how do really good authors get around these inherent shortcomings to write a book, say?  It’s all about understanding audience attention.  Say it again, “audience attention.”

While the ways you manipulate it are different in different media, the concept exists in all of them in some form.  The most obvious form is “perspective”, or the viewpoint from which the audience perceives the action.  In film, this generally refers to the camera, but there’s still the layer of who in the story the audience is watching.  Are we following the villain or the hero?  The criminal or the detective?

In film, the creator has the ability to include important visual information in a shot that’s actually focused on something else.  Because there’s no particular emphasis on a given object or person being included in the shot, things can easily be hidden in plain sight.  But in a book, where the author is obviously very carefully choosing what to include in the description in order to control pacing and be efficient with their description, it’s a lot harder to hide something that way.  “Chekov’s gun” is the principle that irrelevant information should not be included in the story.  “If there’s a rifle hanging on the wall in Act 1, it must be fired in Act 2 or 3.”  Readers will automatically pay attention to almost anything the author mentions because why mention it if it’s not relevant?

In a movie, on the other hand, there’s lots of visual and auditory filler because the conceit is that the audience is directly watching events as they actually happened, so a living room with no furniture would seem very odd, even if the cheap Walmart end table plays no significant role in the story.  Thus, the viewer isn’t paying particular attention to anything in the shot if the camera isn’t explicitly drawing their eye to it.  The hangar at the Rebel Base has to be full of fairly detailed fighter ships even if we only really care about the hero’s.  But not novel is going to go in-depth in its description of 30 X-wings that have no real individual bearing on the course of events.  They might say as little as “He slipped past the thirty other fighters in the hangar to get to the cockpit where he’d hidden the explosives.”  Maybe they won’t even specify a number.

So whereas a movie has an easy time hiding clues, a writer has to straddle the line between giving away the plot twist in the first 5 pages and making it seem like a deus ex machina that comes out of nowhere.  But hey, at least your production values for non-cheesy backgrounds and sets are next to nothing!  Silver linings.

To get back to the main point, the strengths of the medium to a greater or lesser extent decide what kind of stories can be best told, and so a gimmick that works well in a novel won’t necessarily work well in a movie.  The narrator who’s secretly a woman or black, or an alien.  Those are pretty simplistic examples, but hopefully they get the point across.

In the second part of this post a couple days from now, I’ll be talking about how what we learned here can help us understand both how to create a more vibrant image in the reader’s head, and why no amount of research is going to allow you to write about a place or culture or subject you haven’t really lived with for most of your life like a someone born to it would.

 

Machine “Translation” and What Words Mean in Context

One of the biggest commonly known flaws of mahcine translation is a computer’s inability to understand differing meaning in context.  After all, a machine doesn’t know what a “horse” is.  It knows that “caballo” has (roughly) the same meaning in Spanish as “horse” does in English.  But it doesn’t know what that meaning is.

And it certainly doesn’t know what it means when we say that someone has a “horse-face”(/”face like a horse”).

 

But humans can misunderstand meaning in context, too.  For example, if you don’t know how “machine translation” works, you’d think that machines could actually translate or produce translations.  You would be wrong.  What a human does to produce a translation is not the same as what a machine does to produce a “translation”.  That’s why machine and human translators make different mistakes when trying to render the original meaning in the new language.

 

A human brain converts words from the source language into meaning and the meaning back into words in the target language.  A computer converts words from the source language directly to words in the target language, creating a so-called “literal” translation.  A computer would suck at translating a novel, because the figures of speech that make prose (or poetry) what they are are incomprehensible to a machine.  Machine translation programs lack the deeply associated(inter-connected) knowledge base that humans use when producing and interpreting language.

 

A more realistic machine translation(MT) program would require an information web with connections between concepts, rather than words, such that the concept of horse would be related to the concepts of leg, mane, tail, rider, etc, without any intervening linguistic connection.

Imagine a net of concepts represented as data objects.  These are connected to each other in an enormously complex web.  Then, separately, you have a net of linguistic objects, such as words and grammatical patterns, which are overlaid on the concept net, and interconnected.  The objects representing the words for “horse” and “mane” would not have a connection, but the objects representing the concept of meaning underlying these words would have, perhaps, a “has-a” connection, also represented by a connection or “association” object.

In order to translate between languages like a human would, you need your program to have an approximation of human understanding.  A famous study suggested that in the brain of a human who knows about Lindsay Lohan, there’s an actual “Lindsay” neuron, which lights up whenever you think about Lindsay Lohan.  It’s probably lighting up right now as you read this post.  Similarly, in our theoretical machine translation program information “database”, you have a “horse” “neuron” represented by our concept object concept that I described above.  It’s separate from our linguistic object neuron which contains the idea of the word group “Lindsay Lohan”, though probably connected.

Whenever you dig the concept of horse or Lindsay Lohan from your long-term memory, your brain sort of primes the concept by loading it and related concepts into short-term memory, so your “rehab” neuron probably fires pretty soon after your Lindsay neuron.  Similarly, our translation program doesn’t keep it’s whole data-set in RAM constatnly, but loads it from whatever our storage medium is, based on what’s connected to our currently loaded portion of the web.

Current MT programs don’t translate like humans do.  No matter what tricks or algorithms they use, it’s all based on manipulating sequences of letters and basically doing math based on a set of equivalences such as “caballo” = “horse”.  Whether they do statistical analysis on corpuses of previously-translated phrases and sentences like Google Translate to find the most likely translation, or a straight0forward dictionary look-up one word at a time, they don’t understand what the text they are matching means in either language, and that’s why current approaches will never be able to compare to a reasonably competent human translator.

It’s also why current “artificial intelligence” programs will never achieve true human-like general intelligence.  So, even your best current chatbot has to use tricks like pretending to be a Ukranian teenager with bad English skills on AIM to pass the so-called Turing test.  A side-walk artist might draw a picture perfect crevasse that seems to plunge deep into the Earth below your feet.  But no matter how real it looks, your elevation isn’t going to change.  A bird can;t nest in a picture of tree, no matter how realistically depicted.

Calling what Google Translate does, or any machine “translation” program does translation has to be viewed in context, or else it’s quite misleading.  Language functions properly only in the proper context, and that’s something statistical approaches to machine translation will never be able to imitate, no matter how many billions of they spend on hardware or algorithm development.  Could you eventually get them to where they can probably usually mostly communicate the gist of a short newspaper article?  Sure.  Will you be able to engage live in witty reparte with your mutually-language exclusive acquaintance over Skype?  Probably not.  Not with the kind of system we have now.

Those crude, our theoretical program with knowledge web described above might take us a step closer, but even if we could perfect and polish it, we’re still a long way from truly useful translation or AI software.  After all, we don;t even understand how we do these things ourselves.  How could we create an artificial version when the natural one still eludes our grasp?