One of the biggest problems in the field of artificial intelligence as it is popularly conceived is that of natural language processing and machine translation. Natural language processing is the processing of human languages, generally in the form of text, but also as audio, by computers and software. Machine translation is a form of natural language processing having to do with the translation of a text from one language into another by a computer. Enormous strides have been made in natural language processing over the last two decades. Text-to-speech, represented often by those annoying robotic phone calls, and speech-to-text such as dictation software, have both improved markedly.
But the big fish of natural language processing is still the universal translator, able to translate between any two given languages with high accuracy, perhaps even perfectly. Which is, currently impossible, despite what people might tell you. For example, Skype Translator looks great in the recent demo, but it’s doubtful it performs quite so well in the field.
There are actually several major issues that machine translation, and translation in general, have to tackle to achieve a high degree of accuracy. Over the next few posts, I’m going to discuss what some of them are, why they are so difficult to solve, and possible ways they might be overcome.
To start with, there’s a very simple problem with most translation: language has meaning on its own, but for human communication purposes, you have to know the context of the words you are trying to translate. Most communication is not factual transmission of information. Even in formal written prose, there are heaps of subtle ambiguities, and most humans can’t always pick up on every one, much less a poor dumb computer. This is a problem not only when a word is used that has two different sense, such as a technical and a general sense. Different languages divide up their semantic space differently, and a word that is a correct translation on paper may fail miserably in the real world.
To give an example, Japanese has pronouns just like English. However, it does not have an exactly equivalent set. The word for “her” in Japanese is “kanojo”. But that word is rarely used, as most Japanese uses a name and an honorific instead. Further complicating the situation, the word “kanojo” is also idiomatic for “girlfriend”. The same goes for “kareshi” which is often translated as “he” on paper. And then we have the various Japanese pronouns technically meaning “you”, or at least referring to the second person singular. The main one given in teaching Japanese is “anata”. However, much like “kanojo” or “kareshi” there is a sense of intimacy not immediately apparent in the use of this word. It is often used between older married couples to mean something like “dear”, and is rarely if ever used as a bare second person, singular pronoun. Rather, again, the person’s name and an honorific are used. However, a great deal would have to be hard-coded into a piece of translation software to cover all of this, and that’s assuming there’s even enough context to make a decision at all. Further, going into Japanese from English, would require the addition of formality information that a computer or even a human couldn’t always be sure of.
It’s often possible to translate conversation/survival language with bare, literal translations, because the basic sense is all that matters. But more formal registers, or translation of a fixed text, would likely prove impossible for today’s machine translators and most humans, because there’s no way to convey all the necessary context that would provide information that in one language would not be included lexically or grammatically, but it all but necessary for appropriate communication in the other.
In the next post, I’ll be looking more at the ways language divide up their semantic space, and how that hinders accurate translation.