Overview of Text Decoding

The task of text decoding is to take a tokenised sentence and determine the best sequence of words. In many situations this is a classical disambiguation problem there is one, and only one correct sequence of words which gave rise to the text, and it is our job to determine this. In other situations, especially where we are dealing with non-natural language text such as numbers and dates and so on, there may be a few different acceptable word sequences. 

This process boils down to the following tasks  

Determine the semiotic class (number, date, natural language etc) for each token. 

For each sequence of tokens in the same semiotic class find resolve any ambiguity and find their underlying forms 

For non-natural language classes, perform a verbalisation process, which converts their underlying forms into a sequence of words which can be spoken. 

---