Text Decoding

TEXT DECODING FINDING THE WORDS FROM THE TEXT 

The task of text decoding is to take a tokenised sentence and determine the best sequence of words. In many situations this is a classical disambiguation problem there is one, and only one correct sequence of words which gave rise to the text, and it is our job to determine this. In other situations, especially where we are dealing with non-natural language text such as numbers and dates and so on, there may be a few different acceptable word sequences. 

This process boils down to the following tasks  

Determine the semiotic class (number, date, natural language etc) for each token. 

For each sequence of tokens in the same semiotic class find resolve any ambiguity and find their underlying forms 

---

The process of reading, as we normally understand it, is one of taking some text, decoding it into a message, and then understanding it. For clarity, we call this process reading silently. By and... 

Figure 3.2 The common-form model showing the two processes of text decoding, which finds the words, and speech encoding from those words. For illustration we have shown the different primary levels of form for speech and writing as graphemes and phonemes to show that these are not directly connected.

Chapter 5. Text Decoding Finding the words from the text... 

Figure 5.3 shows the flowchart for the text decoding system described in this chapter. 

We argue that there are a number of advantages to the text decoding model used here. Taking the issue of natural language decoding first, we see that this follows the information theoretic... 

---