Home > Natural language processing
Natural Language Processing (NLP) is a subfield of artificial intelligence and linguistics. It studies the problems inherent in the processing and manipulation of natural language, and, natural language understanding devoted to making computers "understand" statements written in human languages.1 Natural language processing
Early systems such as SHRDLU, working in restricted " blocks world s" with restricted vocabularies, worked extremely well, leading researchers to excessive optimism which was soon lost when the systems were extended to more realistic situations with real-world ambiguity and complexity.
Natural language understanding is sometimes referred to as an AI-complete problem, because natural language recognition seems to require extensive knowledge about the outside world and the ability to manipulate it. The definition of "understanding" is one of the major problems in natural language processing.
Some examples of the problems faced by natural language understanding systems:
- The sentences We gave the monkeys the bananas because they were hungry and We gave the monkeys the bananas because they were over-ripe have the same surface grammatical structure. However, in one of them the word they refers to the monkeys, in the other it refers to the bananas: the sentence cannot be understood properly without knowledge of the properties and behaviour of monkeys and bananas.
- A string of words may be interpreted in myriad ways. For example, the string Time flies like an arrow may be interpreted in a variety of ways:
- time moves quickly just like an arrow does;
- measure the speed of flying insects like you would measure that of an arrow;
- measure the speed of flying insects like an arrow would;
- measure the speed of flying insects that are like arrows;
- a type of flying insect, "time-flies," enjoy arrows (compare Fruit flies like a banana.)
The word "time" alone can be interpreted as three different parts of speech, (noun in the first example, verb in 2, 3, 4, and adjective in 5).
- English is particularly bad in this regard because it has little inflectional morphology to distinguish between parts of speech.
- English and several other languages don't specify which word an adjective applies to. For example, in the string "pretty little girls' school".
- Does the school look little?
- Do the girls look little?
- Do the girls look pretty?
- Does the school look pretty?
To help this problem, some linguists and artificial intelligence researchers have proposed using an artificial language, that is capable of expressing all the nuance and subtlety of the natural languages we are familiar with, but would have mathematically inviolate grammar and spelling rules, to remove all possible confusion about what a sentence is trying to say, even if it were nonsense words. An example of such a constructed language that could be used for higher order human/computer communication is lojban.
2 The major tasks in NLP
3 Some problems which make NLP difficult
- Word boundary detection
- In spoken language, there are usually no gaps between words; where to place the word boundary often depends on what choice makes the most sense grammatically and given the context. In written form, languages like ChineseThe Chinese language (/, /, or ; pinyin: hany, huay, or zhongwen) is a member of the Sino-Tibetan family of languages. Although most Chinese view the many varieties of spoken Chinese as a single language, regional variations in spoken language are compara do not signal word boundaries either.
- Word sense disambiguationIn computational linguistics, word sense disambiguation (WSD) is the problem of determining in which sense a word having a number of distinct senses is used in a given sentence. For example, consider the word "bass", two distinct senses of which are: #a t
- Any given word can have several different meanings; we have to select the meaning which makes the most sense in context.
- Syntactic ambiguitySyntactic ambiguity is a property of sentences which may be parsed in more that one way. It may or may not involve one word having two parts of speech or homonyms. Here are some examples: Bear left at zoo''. Do you turn left when you get to the zoo, or di
- The grammarThis article is about grammar from a linguistic perspective. For English grammar rules see English writing style According to the structuralist point of view, grammar is the study of the rules governing the use of a language. That set of rules is also cal for natural languages is not unambiguous , i.e. there are often multiple possible parse treeTrees (structure) A parse tree is a grammatical structure represented as a tree data structure. A sentence structure represented as a parse tree. See also: X-bar theory document object model data structures.s for a given sentence. Choosing the most appropriate one usually requires semantic and contextual information.
- Imperfect or irregular input
- Foreign or regional accents and vocal impediments in speech; typing or grammatical errors, OCR errors in texts.
- Speech acts and plans
- Sentences often don't mean what they literally say; for instance a good answer to "Can you pass the salt" is to pass the salt; in most contexts "Yes" is not a good answer, although "No" is better and "I'm afraid that I can't see it" is better yet. Or again, if a class was not offered last year, "The class was not offered last year" is a better answer to the question "How many students failed the class last year?" than "None" is.
