Words

• For the sake of this lecture, let's largely stick with English.
• Other languages are just as complex, though they might have more or less complex syntax and morphology.
• What's a word?
• Generally, words are a good and relatively simple place to start language processing.
• Morphology, you can break words down into runs and affixes.
• Other languages, e.g. Arabic and German, have a more complex morphology.
• What do you do with misspellings?
• What do you do with new words?
• Proper Nouns are really important. These can be misspelled and can come in a range of formats. What's my name?
• Gazeteers can help here. These are big lists of names.
• What about speech?
• Here phonemes are the basis, and there a range of problems.