RSS

Tag Archives: artificial intelligence

Why Is A Picture Worth a Thousand Words? Information Density in Various Media

You’ve obviously heard the the phrase “a picture is worth a thousand words”, and you probably even have an idea why we say that.  But rarely do people delve deeply into the underlying reasons for this truth.  And those reasons can be incredibly useful to know.  They can tell you a lot about why we communicate they way we do, how art works, and why it’s so damn hard to get a decent novel adaption into theaters.

I’m going to be focusing mostly on that last complaint in this post, but what I’m talking about has all sorts of broad applications to things like good communication at work, how to tell a good story or joke, and how to function best in society.

So, there’s always complaints about how the book or the comic book, or whatever the original was is better than the movie.  Or the other way around.  And that’s because different artistic media have different strengths in terms of how they convey information.  There are two reasons for this:

  1. Humans have five “senses”.  Basically, there are five paths through which we receive information from the world outside our heads.  The most obvious one is sight, closely followed by sound.  Arguably, touch(which really involves multiple sub-senses, like heat and cold and pain) is the third most important sense, and, in general, taste and smell are battling it out for fourth place.  This is an issue of “kind”.
  2. The second reason has to do with what I’m calling information density.  Basically, how much information a sense can transmit to our brains in how much time.  This is an issue of “degree”.  Sight, at least form humans, probably has the highest information density.  It gives is the most information per unit of time.

So how does that effect the strengths of various media?  After all, both movies and text mostly enter our brain through sight.  You see what’s on the screen and what’s on the page.  And neither can directly transmit information about touch, smell, or taste.

The difference is in information density.  Movies can transmit visual information(and audio) directly to our brains.  But text has to be converted into visual imagery in the brain, and it also takes a lot of text to convey a single piece of visual information.

AI, in the form of image recognition software, is famously bad at captioning photos.  Not only does it do a crappy job of recognizing what is in a picture, but it does a crappy job of summarizing it in text.  But really, could a human do any better?  Sure, you are way better than a computer at recognizing a dog.  But what about captioning?  It takes you milliseconds at most to see a dog in the picture and figure out it is jumping to catch the frisbee.  You know that it’s a black lab, and that it’s in the woods, probably around 4 in the afternoon, and that it’s fall because there’s no leaves on the trees, and it must have rained because there are puddles everywhere, and that…

And now you’ve just spent several seconds at least reading my haphazard description.  A picture is worth a thousand words because it takes a relatively longer amount of time for me to portray the same information in a text description.  In fact, it’s probably impossibly for me to convey all the same information in text.  Just imagine trying to write out every single bit of information explicitly shown in a half-hour cartoon show in text.  It would probably take several novels’ worth of words, and take maybe even days to read.  No one would read that book.  But we have no problem watching TV shows and movies.

Now go back and imagine our poor AI program trying to figure out the important information in the photo of the dog and how to best express it in words.  Yikes.  But as a human, you might pretty quickly decide that “a dog catches a frisbee” adequately describes the image.  Still takes longer than just seeing a picture, but isn’t all that much time or effort.  But, you’re summarizing.  A picture cannot summarize and really has no reason to.  With text(words) you have to summarize.  There’s pretty much no way around it.  So you lose an enormous amount of detail.

So, movies can’t summarize, and books must summarize.  Those are two pretty different constrains on the media in question.  Now, imagine a a radio play.  It’s possible you’ve never heard one.  It’s not the same as an audiobook, despite communicating through the same sense(audio), and it has some serious advantages over books and audiobooks.  You don’t have to worry about conveying dialogue, or sound information because you can do that directly.  Emotion, accents, sound effects.  But of course you can convey visual information like a movie, and unlike in a book or an audiobook, it’s a lot more difficult to just summarize, because you’d have to have a narrator or have the characters include it in dialogue.  So raw text still has some serious advantages based on the conventions of the form.  Similarly, radio dramas/audio plays/pod casts and movies both have to break convention to include character thoughts in storytelling, while books don’t.

So, audio and television media have major advantages in their specific areas than text, but text is in general far more flexible in making up for any short-comings.  And, it can take advantage of the summary nature of the medium when there’s a lot of unnecessary information.  Plus, it can count on the reader to be used to filling in details with their imagination.

Film and radio can’t do that.  They can use montages, cuts, and voiceovers to try and imitate what text can do, but it’s never quite the same effect.  And while language might not limit your ability to understand or experience concepts you have no words for, the chosen medium absolutely influences how effective various story-telling techniques can be.

Consider, an enormous battle scene with lots of action is almost always going to be “better” in a visual medium, because most of the relevant information is audio and video information.  An action scene involving riding a dragon through an avalanche while multiple other people try to get out of the way or stop you involves a great deal of visual information, such that a text can’t convey everything a movie could.  Watching a tennis match is always going to be more exciting than reading about one, because seeing the events lets you decide without an narrator interference whether a player has a real shot at making a return off that amazing serve.  You can look at the ball, and using past experience, imagine yourself in the player’s place and get a feeling of just how impressive that lunging backhand really was.  You can’t do the same in text, because even if the writer could describe all the relevant information such that you could imagine the scene exactly in your head, doing so would kill the pacing because of how long reading that whole description would take.

The very best artists in any medium are always going to use that medium to its fullest, exploiting any tricks or hacks as best as possible to make their creation shine.  And that means they will (often unconsciously) create a story tailored to best take advantage of the medium they are working in.  If and when the time comes to change mediums, a lot of what really made the art work won’t be directly translatable because that other medium will have different strengths and have different “hacks” available to try to imitate actually experiencing events directly.  If you play videogames or make software, it’s sort of like how switching platforms or programming languages (porting the game) means some things that worked really well in the original game won’t work in the ported version, because the shortcut in the original programming language doesn’t exist in the new one.

So, if video media have such a drastically higher information density than text, how do really good authors get around these inherent shortcomings to write a book, say?  It’s all about understanding audience attention.  Say it again, “audience attention.”

While the ways you manipulate it are different in different media, the concept exists in all of them in some form.  The most obvious form is “perspective”, or the viewpoint from which the audience perceives the action.  In film, this generally refers to the camera, but there’s still the layer of who in the story the audience is watching.  Are we following the villain or the hero?  The criminal or the detective?

In film, the creator has the ability to include important visual information in a shot that’s actually focused on something else.  Because there’s no particular emphasis on a given object or person being included in the shot, things can easily be hidden in plain sight.  But in a book, where the author is obviously very carefully choosing what to include in the description in order to control pacing and be efficient with their description, it’s a lot harder to hide something that way.  “Chekov’s gun” is the principle that irrelevant information should not be included in the story.  “If there’s a rifle hanging on the wall in Act 1, it must be fired in Act 2 or 3.”  Readers will automatically pay attention to almost anything the author mentions because why mention it if it’s not relevant?

In a movie, on the other hand, there’s lots of visual and auditory filler because the conceit is that the audience is directly watching events as they actually happened, so a living room with no furniture would seem very odd, even if the cheap Walmart end table plays no significant role in the story.  Thus, the viewer isn’t paying particular attention to anything in the shot if the camera isn’t explicitly drawing their eye to it.  The hangar at the Rebel Base has to be full of fairly detailed fighter ships even if we only really care about the hero’s.  But not novel is going to go in-depth in its description of 30 X-wings that have no real individual bearing on the course of events.  They might say as little as “He slipped past the thirty other fighters in the hangar to get to the cockpit where he’d hidden the explosives.”  Maybe they won’t even specify a number.

So whereas a movie has an easy time hiding clues, a writer has to straddle the line between giving away the plot twist in the first 5 pages and making it seem like a deus ex machina that comes out of nowhere.  But hey, at least your production values for non-cheesy backgrounds and sets are next to nothing!  Silver linings.

To get back to the main point, the strengths of the medium to a greater or lesser extent decide what kind of stories can be best told, and so a gimmick that works well in a novel won’t necessarily work well in a movie.  The narrator who’s secretly a woman or black, or an alien.  Those are pretty simplistic examples, but hopefully they get the point across.

In the second part of this post a couple days from now, I’ll be talking about how what we learned here can help us understand both how to create a more vibrant image in the reader’s head, and why no amount of research is going to allow you to write about a place or culture or subject you haven’t really lived with for most of your life like a someone born to it would.

 

Tags: , , , , , , , , ,

Smol Bots: ANNs and Advertising

So I recently read a great story by A. Merc Rustad, “it me, ur smol”.  The story is about and ANN, or artificial neural network.  You may or may not know that the neural net is the latest fad in AI research, replacing statistical models with a model based on–but not the same as!–your brain.  Google uses them for its machine translation, and many other machine translation companies have followed suit.  My last post also dealt with an ANN, in this case, one trained to recognize images.

ANN accounts, like @smolsips in the story above, have become very popular on Twitter lately.  A favorite of mine is the @roborosewater account, which shares card designs for Magic: The Gathering created by a series of neural nets.  It’s lately become quote good at both proper card syntax and design, although it’s notsignificantly better at this than any other twitter neural net is at other things.

The story itself takes some liberties with neural nets.  They are certainly not capable of developing into full AIs.  However, the real genius of the story is in the pitch-perfect depiction of the way human Twitter users and bots interact.  And similarly, the likely development of bots in the near future.  It’s quite likely that bot accounts will become a more significant and less dread feature of Twitter and other similar social networks as they improve in capability.

For example, rather than sock-puppet accounts, I’m very confident that bot accounts used for advertising or brand visibility similar to the various edgy customer service accounts will be arriving shortly.  Using humour and other linguistic tools to make them more palatable as ads, and also to encourage a wider range of engagement as their tweets are shared more frequently due to things having little to do with whatever product they may be shilling.

There are already chatbots on many social media platforms who engage in telephone tree-style customer service and attempt to help automate registrations for services.  The idea of a bot monitoring its own performance through checking its Twitter stats and then trying new methods as in the story is well within the capabilities of current neural nets, although I imagine they would be a tad less eloquent than @smolsips, and a tad more spammy.

I also really like the idea of a bot working to encourage good hydration.  Things like fitbit or Siri or Google Home have already experimented shallowly with using AI to help humans stay healthy.  And as an organizing tool, Twitter itself has been used to great effect.  I would be quite un-shocked to find NGOs, charities, government agencies making use of clever or cute bots to pursue other public policy goals.  Again, with less panache and more realism than in the story, but nonetheless strongly in the vein of what Rustad depicts our erstwhile energy drink namer trying out in its optimistic quest to save us from our own carelessness.

We’ve had apps along these lines before, but they tend to be reactive.  Active campaign and organizing in the style of @smolsips is something we haven’t seen very often, but which could be quite a boon to such efforts.

Although neural nets in this style will never be able to pass for real humans, due to structural limitations in the design, cleverly programmed, they can be both useful and entertaining.

Some other examples of bots I quite enjoy are:

  1. Dear Assistant uses the Wolfran Alpha database to answer factual question.
  2. Grammar Police is young me in bot form.  It must have a busy life trying to save standardize Twitter English.  XD
  3. Deleted Wiki Titles lets you know what shenanigans are happening over on the high school student’s favorite source of citations.
  4. This bot that tweets procedurally generated maps.
  5. This collaborative horror writer bot.
  6. This speculative entomology bot.
  7. The Poet .Exe writes soothing micro-poetry.

Suggest some of your favorite Twitter bots in the comments!

 

Tags: , , , , , , , , , ,

Do Androids Dream?

I’m here with some fascinating news, guys.  Philip K. Dick may have been joking with the title of his famous novel Do Androids Dream of Electric Sheep?  But science has recently answered this deep philosophical question for us.  In the affirmative.  The fabulous Janelle Shane trains neural networks on image recognition datasets with the goal of uncovering some incidental humour.  She’s taken this opportunity to answer a long-standing question in AI.  As it turns out, artificial neural networks do indeed dream of digital sheep.  Whether androids will too is a bit more difficult.  I’d hope we would improve our AI software a bit more before we start trying to create artifical humans.

As Shane explains in the above blog post, the neural network was trained on thousands or even millions (or more) of images, which were pre-tagged by humans for important features.  In this case, lush green fields and rocky mountains.  Also, sheep and goats.  After training, she tested it on images with and without sheep, and it turns out it’s surprisingly easy to confuse it.  It assumed sheep where there were none and missed sheep (and goats) staring it right in the face.  In the second case, it identified them as various other animals based on the other tags attached to images of them.  Dogs in your arms, birds in a tree cats in the kitchen.

This is where Shane and I come to a disagreement.  She suggests that the confusion is the result of insufficient context clues in the images.  That is, fur-like texture and a tree makes a bird, with a leash it makes a dog. In a field, a sheep.  They see a field, and expect sheep.  If there’s an over-abundance of sheep in the fields in the training data, it starts to expect sheep in all the fields.

But I wonder, what about the issue of paucity of tags.  Because of the way images are tagged, there’s not a lot of hint about what the tags are referring to.  Unlike more standard teaching examples, these images are very complex and there lots of things besides what the tags note.  I think the flaw is a lot deeper than Shane posits.   The AI doesn’t know how to recognize discrete objects like a human can.  Once you teach a human what a sheep is, they can recognize it in pretty much any context.  Even a weird one like a space-ship or a fridge magnet.  But a neural net isn’t sophisticated enough or, most generously, structured properly to understand what the word “sheep” is actually referring to.  It’s quite possible the method of tagging is directly interfering with the ANNs ability to understand what it’s intended to do.

The images are going to contain so much information, so many possible changing objects that each tag could refer to, that it might be matching “sheep” say to something entirely different from what a human would match it to.  “Fields” or “lush green” are easy to do.  If there’s a lot of green pixels, those are pretty likely, and because they take up a large portion of the information in the image, there’s less chance of false positives.

Because the network doesn’t actually form a concept of sheep, or determine what entire section of pixels makes up a sheep, it’s easily fooled.  It only has some measure by which it guesses at their presence or absence, probably a sort of texture as mentioned in Shane’s post.  So the pixels making up the wool might be the key to predicting a sheep, for example.  Of course, NNs can recognize lots of image data, such as lines, edges, curves, fills, etc.  But it’s not the same kind of recognition as a human, and it leaves AIs vulnerable to pranks, such as the sheep in funny places test.

I admit to over-simplifying my explanations of the technical aspects a bit.  I could go into a lecture about how NNs work in general and for image recognition, but it would be a bit long for this post, and in many cases, no one really knows, even the designers of a system, everything about how they make their decision.  It is possible to design or train them more transparently, but most people don’t.

But even poor design has its benefits, such as answering this long-standing question for us!

If anyone feels I’ve made any technical or logical errors in my analysis, I’d love to hear about it, insomuch as learning new things is always nice.

 

Tags: , , , , , , , ,

Your Chatbot Overlord Will See You Now

Science fiction authors consistently misunderstand the concept of AI.  So do AI researchers.  They misunderstand what it is, how it works, and most importantly how it will arise.  Terminator?  Nah.  The infinitely increasing complexity of the Internet?  Hell no.  A really advanced chatbot?  Not in a trillion years.

You see, you can’t get real AI with a program that sits around waiting for humans to tell it what to do.  AI cannot arise spontanteously from the internet, or a really complex military computer system or from even the most sophisticated natural language processing program.

The first mistake is the mistake Alan Turing made with his Turing test.  The same mistake the founder and competitors for the Loebner Prize have made.  The mistake being: language is not thought.  Despite the words you hear in your head as you speak, despite the slowly-growing verisimilitude of chatbot programs, language is and only ever has been the expression of thought and not thought itself.  After all, you can visualize a seen in your head without ever using a single word.  You can remember a sound or a smell or the taste of day-old Taco Bell.  All without using a single word.  A chatbot can never become an AI because it cannot actually think, it can only loosely mimic the linguistic expression of thought through tricks and rote memory of templates that if it’s really advanced may involve plugging in a couple variables taken from the user’s input.  Even chatbats based on neural networks and enormous amounts of training data like Microsoft’s Tay, or Siri/Alexa/Cortana are still just tricks of programming trying to eke out an extra tenth of a percentage point of illusory humanness.  Even IBM’s Watson is just faking it.

Let’s consider for a bit what human intelligence is to give you an idea of what the machines of today are lacking, and why most theories on AI are wrong.  We have language, or the expression of intelligence that so many AI programs are so intent on trying to mimic.  We also have emotions and internal drive, incredibly complex concepts that most current AI is not even close to understanding, much less implementing.  We have long-term and short-term memory, something that’s relatively easy for computers to do, although in a different way than humans–and which there has still been no significant progress on because everyone is so obsessed with neural networks and their ability to complete individual tasks something like 80% as well as a human.  A few, like AlphaGoZero, can actually crush humans into the ground on multiple related tasks–in AGZ’s case, chess-like boardgames.

These are all impressive feats of programming, though the opacity of neural-network black boxes kinda dulls the excitement.  It’s hard to improve reliably on something you don’t really understand.  But they still lack the one of the key ingredients for making a true AI: a way to simulate human thought.

Chatbots are one of two AI fields that focus far too much on expression over the underlying mental processes.  The second is natural language processing(NLP).  This includes such sub-fields as machine translation, sentiment analysis, question-answering, automatic document summarization, and various minor tasks like speech recognition and text-to-speech.  But NLP is little different from chatbots because they both focus on tricks that manipulate the surface expression while knowing relatively little about the conceptual framework underlying it.  That’s why Google Translate or whatever program you use will never be able to match a good human translator.  Real language competence requires understanding what the symbols mean, and not just shuffling them around with fancy pattern-recognition software and simplistic deep neural networks.

Which brings us to the second major lack in current AI research: emotion, sentiment, and preference.  A great deal of work has been done on mining text for sentiment analysis, but the computer is just taking human-tagged data and doing some calculations on it.  It still has no idea what emotions are and so it can only do keyword searches and similar and hope the average values give it a usable answer.  It can’t recognize indirect sentiment, irony, sarcasm, or other figurative language.  That’s why you can get Google Translate to ask where the toilet is, but its not gonna do so hot on a novel, much less poetry or humour.   Real translation is far more complex than matching words and applying some grammar rules, and Machine Translation(MT) can barely get that right 50% of the time.

So we’ve talked about thought vs. language, and the lack of emotional intelligence in current AI.  The third issue is something far more fundamental: drive, motivation, autonomy.  The current versions of AI are still just low-level software following a set of pr-programmed instructions.  They can learn new things if you funnel data through the training system.  They can do things if you tell them to.  They can even automatically repeat certain tasks with the right programming.  But they rely on human input to do their work.  They can’t function on their own, even if you leave the computer or server running.  They can’t make new decisions, or teach themselves new things without external intervention.

This is partially because they have no need.  As long as their machine “body” is powered they keep chugging along.  And they have no ability to affect whether or not it is powered.  They don’t even know they need power, for the most part.  Sure they can measure battery charge and engage sleep mode through the computer’s operating system.  But they have no idea why that’s important, and if I turn the power station off or just unplug the computer, a thousand years of battery life won’t help them plug back in.  Whereas human intelligence is based on the physical needs of the body motivating us to interact with the environment, a computer and the rudimentary “AI” we have now has no such motivation.  It can sit in its resting state for eternity.

Even with an external motivation, such as being coded to collect knowledge or to use robot arms to maintain the pre-designated structure of say a block pyramid or a water and sand table like you might see demonstrating erosion at the science center, an AI is not autonomous.  It’s still following a task given to it by a human.  Whereas no one told human intelligence how to make art or why it’s valuable.  Most animals don’t get it, either.  It’s something we developed on our own outside of the basic needs of survival.  Intelligence helps us survive, but because of it we need things to occupy our time in order to maintain mental health and a desire to live and pass on our genes.  There’s nothing to say that a complete lack of being able to be bored is a no-go for a machine intelligence, of course.  But the ability to conceive and implement new purposes in life is what make human intelligence different from that of animals, whose intelligence may have less raw power but still maintains the key element of motivation that current AI lacks, and which a chatbot or a neural network as we know them today can never achieve, no matter how many computers you give it to run on or TV scripts you give it to analyze.  The fundamental misapprehension of what intelligence is and does by the AI community means they will never achieve a truly intelligent machine or program.

Science fiction writers dodge this lack of understanding by ignoring the technical workings of AI and just making them act like strange humans.  They do a similar thing with alien natural/biological intelligences.  It makes them more interesting and allows them to be agents in our fiction.  But that agency is wallpaper over a completely nonexistent technological understanding of ourselves.  It mimics the expression of our own intelligence, but gives limited insight into the underlying processes of either form.  No “hard science fiction” approach does anything more than a “scientific magic system”.  It’s hard sci-fi because it has fixed rules with complex interactions from which the author builds a plot or a character, but it’s “soft sci-fi” in that these plots and characters have little to do with how AI would function in reality.  It’s the AI equivalent of hyperdrive.  A technology we have zero understanding of and which probably can’t even exist.

Elon Musk can whinge over the evils of unethical AI destroying the world, but that’s just another science fiction trope with zero evidential basis in reality.  We have no idea how an AI might behave towards humans because we still have zero understanding of what natural and artificial intelligences are and how they work.  Much less how the differences between the two would effect “inter-species” co-existence.  So your chatbot won’t be becoming the next HAL or Skynet any time soon, and your robot overlords are still a long way off even if they could exist at all.

 

Tags: , , , , , ,

Poetry, Language, and Artificial Intelligence

Poetry exemplifies how the meaning of a string of words depends not only upon the sum of the meaning of the words, or on the order in which they are placed, but also upon something we call “context”.  Context is essentially the concept that single word (or idea) has a different meaning depending on its surroundings.  These surroundings could be linguistic–the language we are assuming the word to belong to, for example, environmental–say it’s cold out and I say “It’s sooooooo hot.”, or in light of recent events: “The Mets suck” means something very different if they’ve just won a game than if they’ve just lost one.

Poetry is the art of manipulating the various possible contexts to get across a deeper or more complex meaning than the bare string of words itself could convey.  The layers of meaning are infinitely deep, and in fact in any form of creative  writing, it is demonstrably impossible for every single human to understand all of them.  I say poetry is the “art” of such manipulation because it is most often the least subtle about engaging in it.  All language acts manipulate context.  Just using a simple pronoun is manipulating context to express meaning.

And we don’t decode this manipulation separate from decoding the bare language.  It happens as a sort of infinite feedback loop, working on all the different layers of an utterance at once.  The ability to both manipulate concepts infinitely and understand our own infinite manipulations might be considered the litmus test for what is considered “intelligent” life.

 

Returning to the three words in our title, I’ve discussed everything but AI.  The difficulty in creating AGI, or artificial general intelligence lies in the fact that nature had millions or billions of years to sketch out and color in the complex organic machine that grants humans this power of manipulation.  Whereas humans have had maybe 100?  In a classic chicken and egg problem, it’s quite difficult to have either the concept web or the system that utilizes it without the other part.  If the system creates the web, how do you know how to code the system without knowing the structure of the web?  And if the web comes first, how can you manipulate it without the complete system?

You might have noticed a perfect example of how context affects meaning in that previous paragraph.  One that was not intentional, but that I noticed as I went along. “Chicken and egg problem”.  You  can’t possibly know what I meant by that phrase without having previously been exposed to the philosophical question of which came first, the chicken that laid the egg, or the egg the chicken hatched from.  But once you do know about the debate, it’s pretty easy to figure out what I meant by “chicken and egg problem”, even though in theory you have infinite possible meanings.

How in the world are you going to account for every single one of those situations when writing an AI program?  You can’t.  You have to have a system based on very general principles that can deduce that connection from first principles.

 

Although I am a speculative fiction blogger, I am still a fiction blogger.  So how do this post relate to fiction?  When  writing fiction you are engaging in the sort of context manipulation I’ve discussed above as such an intractable problem for AI programmers.  Because you are an intelligent being, you can instinctually engage in it when writing, but unless you are  a rare genius, you are more likely needing to engage in it explicitly.  Really powerful writing comes from knowing exactly what context an event is occurring in in the story and taking advantage of that for emotional impact.

The death of a main character is more moving because you have the context of the emotional investment in that character from the reader.  An unreliable narrator  is a useful tool in a story because the truth is more surprising either  when the character knew it and purposefully didn’t tell the reader, or neither of them knew it, but it was reasonable given the  information both had.  Whereas if the truth is staring the reader in the face but the character is clutching the idiot ball to advance the plot, a readers reaction is less likely to be shock or epiphany and more likely to be “well,duh, you idiot!”

Of course, context can always go a layer deeper.  If there are multiple perspectives in the story, the same situation can lead to a great deal of tension because the reader knows the truth, but also knows there was no way this particular character could.  But you can also fuck that up and be accused of artificially manipulating events for melodrama, like if a simple phone call could have cleared up the misunderstanding but you went to unbelievable lengths to prevent it even though both characters had cell phones and each others’ numbers.

If the only conceivable reason the call didn’t take place was because the author stuck their nose in to prevent it, you haven’t properly used or constructed  the context for the story.  On the other hand, perhaps there was an unavoidable reason one character lost their phone earlier in the story, which had sufficient connection to  other important plot events to be not  just an excuse to avoid the plot-killing phone-call.

The point being that as I said before, the  possible contexts for language or events are infinite.  The secret to good writing  lies in being able to judge which contexts are most relevant and making sure that your story functions reasonably within those contexts.  A really, super-out-of-the-way solution to a problem being ignored is obviously a lot more acceptable than ignoring the one staring you in the face.  Sure your character might be able to send a morse-code warning message by hacking the electrical grid and blinking the power to New York repeatedly.  But I suspect your readers would be more likely to call you out for solving the communication difficulty that way than for not solving it with the characters’ easily  reachable cell phone.

I mention the phone thing because currently, due to rapid technological progress, contexts are shifting far  more rapidly than they did in the past.  Plot structures honed for centuries based on a lack of easy long-range communication are much less serviceable as archetypes now that we have cell phones.  An author who grew up before the age of ubiquitous smart-phones for your seven-year-old is going to have a lot more trouble writing a believable contemporary YA romance than someone who is turning twenty-two in the next three months.  But even then, there’s a lack of context-verified, time-tested plot structures to base such a story on than a similar story set in the 50s.  Just imagine how different Romeo and Juliet would have been if they could have just sent a few quick texts.

In the past, the ability of the characters to communicate at all was a strong driver of plots.  These days, it’s far more likely that trustworthiness of communication will be a central plot point.  In the past, the possible speed of travel dictated the pacing of many events.  That’s  far less of an issue nowadays. More likely, it’s a question of if you missed your flight.  Although…  the increased speed of communication might make some plots more unlikely, but it does counteract to some extent the changes in travel speed.  It might be valuable for your own understanding and ability to manipulate context to look at some works in older settings and some works in newer ones and compare how the authors understanding of context increased or decreased the impact and suspension of disbelief for the story.

Everybody has some context for your 50s love story because they’ve been exposed to past media depicting it.  And a reader is less likely to criticize shoddy contextualizing in when they lack any firm context of their own.   Whereas of course an expert on horses is far more likely to find and be irritated by mistakes in your grooming and saddling scenes than a kid born 16 years ago is to criticize a baby-boomer’s portrayal of the 60s.

I’m going to end this post with a wish for more stories–both SpecFic and YA–more strongly contextualized in the world of the last 15 years.  There’s so little of it, if you’re gonna go by my high standards.

 

Tags: , , , , , ,

AI, Academic Journals, and Obfuscation

A common complaint about the structure for publishing and distributing academic journals is that it is designed in such a way that it obfuscates and obscures the true bleeding edge of science and even the humanities.  Many an undergrad has complained about how they found a dozen sources for their paper, but that all but two of them were behind absurd paywalls.  Even after accounting for the subscriptions available to them through their school library.  One of the best arguments for the fallacy that information wants to be free is the way in which academic journals prevent the spread of potentially valuable information and make it very difficult for the indirect collaboration between multiple researchers that likely would lead to the fastest advances of our frontier of knowledge.

In the corporate world, there is the concept of the trade secret.  It’s basically a form of information that creates the value in the product or the lower cost of production a specific corporation which provides that corporation with a competitive edge over other companies in its field.  Although patents and trade secret laws provide incentive for companies to innovate and create new products, the way academic journals are operated hinders innovation and advancement without granting direct benefits to the people creating the actual new research. Rather, it benefits instead the publishing company whose profit is dependent on the exclusivity of the research, rather than the value of the research itself to spur scientific advancement and create innovation.

Besides the general science connection, this issue is relevant to a blog like the Chimney because of the way it relates to science fiction and the plausibility and/or obsolescence of the scientific  or world-building premise behind the story.

Many folks who work  in the hard sciences (or even the social sciences) have an advantage in the premise department, because they have knowledge and the ability to apply it at a level an amateur or  a generalist is unlikely to be able to replicate.  Thus, many generalists or plain-old writers who work in science fiction make use of a certain amount of handwavium in their scientific and technological world-building.  Two of the most common examples of this are in the areas of faster-than-light(FTL) travel (and space travel in general) and artificial intelligence.

I’d like to argue that there are three possible ways to deal with theoretical or futuristic technology in the premise of  an SF novel:

  1. To as much as possible research and include in your world-building and plotting the actual way in which a technology works and is used, or  the best possible guess based on current knowledge of how such a technology could likely work and be used.  This would include the possibility of having actual plot elements based on quirks inherent in a given implementation.  So if your FTL engine has some side-effect, then the world-building and the plot would both heavily incorporate that side-effect.  Perhaps some form of radiation with dangerous effects both dictates the design of your ships and the results of the radiation affecting humans dictates some aspect of the society that uses these engines (maybe in comparison to a society using another method?)  Here you are  firmly in “hard” SF territory and are trying to “predict the future” in some sense.
  2. To say fuck it and leave the mechanics of your ftl mysterious, but have it there to make possible some plot element, such as fast travel and interstellar empires.  You’ve got a worm-hole engine say, that allows your story, but you don’t delve into or completely ignore how such a device might cause your society to differ from the present  world.  The technology is a narrative vehicle rather than itself the reason for the story.  In (cinematic) Star Wars, for example, neither the Force nor hyper-drive are explained in any meaningful way, but they serve to make the story possible.
  3. A sort of mix between the two involves  obviously handwavium technology, but with a set of rules which serve to drive the story. While the second type is arguably not true speculative fiction, but just utilizes the trappings for drama’s sake, this type is speculative, but within a self-awarely unrealistic premise.

 

The first type of SF often suffers from becoming dated, as the theory is disproven, or a better alternative is found.  This also leads to a possible forth type, so-called retro-futurism, wherein an abandoned form of technology is taken beyond it’s historical application, such as with steampunk.

And therein lies a prime connection between our two topics:  A\a technology used in a story may already be dated without the author even knowing about it.  This could be because they came late to the trend  and haven’t caught on to it’s real-world successor; it could also be because an academic paywall or a company on the brink of releasing a new product has kept the advancement private from the layperson, which many authors are.

Readers may be surprised to find that there’s a very recent real-world example of this phenomenon: Artificial Intelligence.  Currently, someone outside the field but who may have read up on the “latest advances” for various reasons might be lead to believe that deep-learning, neural networks, and  statistical natural language processing are the precursors or even the prototype technologies that will bring about real general/human-like artificial intelligence, either  in the near or far future.

That can be forgiven pretty  easily, since the real precursor to AI is sitting behind a massive build-up of paywalls and corporate trade secrets.  While very keen individuals may have heard of the “memristor”, a sort of circuit capable of behavior  similar to a neuron, this is a hardware innovation.  There is  speculation that modified memristors might be able to closely model the activity of the brain.

But there is already a software solution: the content-agnostic relationship  mapping, analysis, formatting, and translation engine.  I doubt anyone reading this blog has ever heard of it.  I would indeed be surprised if anyone at Google or Microsoft had, either.  In fact, I only know it it by chance, myself. A friend I’ve been doing game design with on and off for the past few years told me about it while we were discussing the AI  model used in the HTML5 tactical-RPG Dark Medallion.

Content-agnostic relationship mapping is a sort of neuron simulation technology that permits a computer program to learn and categorize concept-models in a way that is similar to how humans do, and is basically the data-structure underlying  the software “stack”.  The “analysis” part refers to the system and algorithms used to review and perform calculations based on input from the outside world.  “Formatting” is the process of  turning the output of the system into intelligible communication–you might think of this as analogous to language production.  Just like human thoughts, the way this system “thinks” is not  necessarily all-verbal.  It can think in sensory input models just like a person: images, sounds, smells, tastes, and also combine these forms of data into complete “memories”.  “Translation” refers to the process of converting the stored information from the underlying relationship map into output mediums: pictures, text, spoken language, sounds.

“Content agnostic” means that the same data structures can store any type of content.  A sound, an image, a concept like “animal”: all of these can be stored in the same type of data structure, rather than say storing visual information as actual image files or sounds as audio files.  Text input is understood and stored in these same structures, so that the system does not merely analyze and regurgitate text-files like the current statistical language processing systems or use plug and play response templates like a chat-bot.  Further, the system is capable of output in any language it has learned, because the internal representations of knowledge are not stored in any one language such as English.  It’s not translation, but rather spontaneous generation of speech.

It’s debatable whether this system is truly intelligent/conscious, however.  It’s not going to act like a real human.  As far as I understand it, it possesses no driving spirit like a human, which might cause it to act on its own.  It merely responds to commands from a human.  But I suspect that such an advancement is not far away.

Nor is there an AI out there that can speak a thousand human languages and program new AIs, or write novels.  Not yet, anyway.  (Although apparently they’ve developed it to the point where it can read a short story and answer questions about it, like the names of the main characters or the setting. ) My friend categorized this technology as somewhere between an alpha release and a beta release, probably closer to alpha.

Personally, I’ll be impressed if they can just get it reliably answering questions/chatting in English and observably learning and integrating new things into its model of the world.  I saw some screenshots and a quick video of what I’ll call an fMRI equivalent, showing activation of the individual simulated “neurons”* and  of the entire “brain” during some low-level tests.  Wikipedia seems to be saying the technical term is “gray-box testing”, but since I have no formal software-design training, I can’t say if I’m mis-uderstanding that term or not.   Basically, they have zoomable view of the relationship map, and when the program is activating the various nodes, they light on the screen.   So, if you ask the system how many legs a cat has, the node for cat will light up, followed by the node for “legs”, and maybe the node for “possession”.  Possibly other nodes for related concepts, as well.  None of the images I saw actually labelled the nodes at the level of zoom shown, nor do I have a full understanding of how the technology works.  I couldn’t tell anyone enough for them to reproduce it, which I suppose is the point, given that if this really is a useable technique for creating AIs, it’s probably worth more than the blog-platform I’m writing this on or maybe even all of  Google.

 

Getting back to our original topic, while this technology certainly seemed impressive to me, it’s quite possible it’s just another garden path technology like I believe statistical natural language processing to be.  Science fiction books with clear ideas of how AI works will work are actually quite few and far between.  Asimov’s Three Laws, for example, are not about how robot brains work, but rather about  higher-level things like will AI want to harm us.  In light of what I’ve argued above, perhaps that’s the wisest course.  But then again, plenty of other fields  and technologies are elaborately described in SF stories, and these descriptions used to restrict and/or drive the plot and the actions of the characters.

If anyone does have any books recommendations that do get into the details of how AI works in the story’s world,I would love to read some.

 

Tags: , , , , , , , ,

Machine “Translation” and What Words Mean in Context

One of the biggest commonly known flaws of mahcine translation is a computer’s inability to understand differing meaning in context.  After all, a machine doesn’t know what a “horse” is.  It knows that “caballo” has (roughly) the same meaning in Spanish as “horse” does in English.  But it doesn’t know what that meaning is.

And it certainly doesn’t know what it means when we say that someone has a “horse-face”(/”face like a horse”).

 

But humans can misunderstand meaning in context, too.  For example, if you don’t know how “machine translation” works, you’d think that machines could actually translate or produce translations.  You would be wrong.  What a human does to produce a translation is not the same as what a machine does to produce a “translation”.  That’s why machine and human translators make different mistakes when trying to render the original meaning in the new language.

 

A human brain converts words from the source language into meaning and the meaning back into words in the target language.  A computer converts words from the source language directly to words in the target language, creating a so-called “literal” translation.  A computer would suck at translating a novel, because the figures of speech that make prose (or poetry) what they are are incomprehensible to a machine.  Machine translation programs lack the deeply associated(inter-connected) knowledge base that humans use when producing and interpreting language.

 

A more realistic machine translation(MT) program would require an information web with connections between concepts, rather than words, such that the concept of horse would be related to the concepts of leg, mane, tail, rider, etc, without any intervening linguistic connection.

Imagine a net of concepts represented as data objects.  These are connected to each other in an enormously complex web.  Then, separately, you have a net of linguistic objects, such as words and grammatical patterns, which are overlaid on the concept net, and interconnected.  The objects representing the words for “horse” and “mane” would not have a connection, but the objects representing the concept of meaning underlying these words would have, perhaps, a “has-a” connection, also represented by a connection or “association” object.

In order to translate between languages like a human would, you need your program to have an approximation of human understanding.  A famous study suggested that in the brain of a human who knows about Lindsay Lohan, there’s an actual “Lindsay” neuron, which lights up whenever you think about Lindsay Lohan.  It’s probably lighting up right now as you read this post.  Similarly, in our theoretical machine translation program information “database”, you have a “horse” “neuron” represented by our concept object concept that I described above.  It’s separate from our linguistic object neuron which contains the idea of the word group “Lindsay Lohan”, though probably connected.

Whenever you dig the concept of horse or Lindsay Lohan from your long-term memory, your brain sort of primes the concept by loading it and related concepts into short-term memory, so your “rehab” neuron probably fires pretty soon after your Lindsay neuron.  Similarly, our translation program doesn’t keep it’s whole data-set in RAM constatnly, but loads it from whatever our storage medium is, based on what’s connected to our currently loaded portion of the web.

Current MT programs don’t translate like humans do.  No matter what tricks or algorithms they use, it’s all based on manipulating sequences of letters and basically doing math based on a set of equivalences such as “caballo” = “horse”.  Whether they do statistical analysis on corpuses of previously-translated phrases and sentences like Google Translate to find the most likely translation, or a straight0forward dictionary look-up one word at a time, they don’t understand what the text they are matching means in either language, and that’s why current approaches will never be able to compare to a reasonably competent human translator.

It’s also why current “artificial intelligence” programs will never achieve true human-like general intelligence.  So, even your best current chatbot has to use tricks like pretending to be a Ukranian teenager with bad English skills on AIM to pass the so-called Turing test.  A side-walk artist might draw a picture perfect crevasse that seems to plunge deep into the Earth below your feet.  But no matter how real it looks, your elevation isn’t going to change.  A bird can;t nest in a picture of tree, no matter how realistically depicted.

Calling what Google Translate does, or any machine “translation” program does translation has to be viewed in context, or else it’s quite misleading.  Language functions properly only in the proper context, and that’s something statistical approaches to machine translation will never be able to imitate, no matter how many billions of they spend on hardware or algorithm development.  Could you eventually get them to where they can probably usually mostly communicate the gist of a short newspaper article?  Sure.  Will you be able to engage live in witty reparte with your mutually-language exclusive acquaintance over Skype?  Probably not.  Not with the kind of system we have now.

Those crude, our theoretical program with knowledge web described above might take us a step closer, but even if we could perfect and polish it, we’re still a long way from truly useful translation or AI software.  After all, we don;t even understand how we do these things ourselves.  How could we create an artificial version when the natural one still eludes our grasp?

 

Tags: , , , , , , ,