Statistical parsing

We can therefore identity three common approaches to parsing. 

Define a CFG by hand and use a deterministic search algorithm (i.e. the parser) to find the parse or parses which best fit the input. 

Learn the CFG from data and use this with a probabilistic search algorithm to find the parse (or parses) which is most likely given the input. 

Rgure 5.3 The flow of an entire text-analysis system (only four semiotic decoders/translators are shown for snccinctness). 

The CYK algorithm as just e q)lained will find the set of legal parses for a sentence, but the problem is in general that we wish to find just one parse, not several. We of course want the true parse and by this we mean we want what the author originally intended when the sentence was written. 

Chapter 5. Text Decoding Finding the words from the text 

The major weakness with this type of parsing is that it only considers syntactic factors, semantics is completely ignored. That is why a parser often returns parses that are correct but seem silly. Some more advanced models of S5mtax do incorporate semantic information [78], [177], [312], but in general most syntax models, especially in engineering, only make use of syntactic rules. 

---

A wide variety of methods have been developed to analyze the literature using statistical methods that go beyond text categorization or text clustering. The simplest of these methods is co-occurrence analysis. The main limitation of these methods is that they do not perform syntactic/semantic parsing to extract relations between entities. A benefit to the statistical analysis approach is that it is not constrained to relations presented in a sentence or an anaphoric reference1 from a sentence. As an... 

BioJava is a newer project but has been stable for some time now. The Java framework provides extensive support for sequence manipulation, hie parsing, DAS clients and servers, access to Ensembl and BioSQL databases, and analysis and statistical routines, including a dynamic programming toolkit. 

---