Complexity Shannon information

Another drawback to using Shannon information as a measure of complexity is the fact that it is based on an ensemble of all possible states of a system and therefore cannot describe the information content of a single state. Shannon information thus resembles traditional statistical mechanics - which describes the average or aggregate behavior of, say, a gas, rather than the motion of its constituent molecules - more so than it docs a complexity theory that must address the complexity of individual objects. 

Another way of looking at it is that Shannon information is a formal equivalent of thermodynamic entroi)y, or the degree of disorder in a physical system. As such it essentially measures how much information is missing about the individual constituents of a system. In contrast, a measure of complexity ought to (1) refer to individual states and not ensembles, and (2) reflect how mnc h is known about a system vice what is not. One approach that satisfies both of these requirements is algorithmic complexity theory. 

Within a given sensory modality, it is easy to understand that different stimuU place different demands or loads on information-processing systems. Thus, in order to properly interpret results of performance tests, it is necessary to describe the stimulus. While this remains a topic of ongoing research with inherent controversies, some useful working constructs are available. At issue is not simply a qualitative description, but the measurement of stimulus content (or complexity). Shannon s information theory [1948], which teaches how to measure the amount of information associated with a generalized information source, has been the primary tool used in these efforts. Thus, a stimulus can be characterized in terms of the amount of information present in it Simple stimuli (e.g., a light that is on or off ) possess less information... 

Fig. 15.9. A model of the temperature-dependent molecular teaching the left-side object t student molecule ) is recognize the shape of the right-side molecule ( teacher molecule") in three steps ( lessons ) A - B C - D. Each of the complexes (AE, BE, CE, DE) is characterized by the state of ignorance (Shannon information entropy), while the knowledge learned in each lesson (A —B. B —> C. C -> D) is a measure of diminishing the ignorance of the student molecule fin bitsl.

In contrast, simple patterns are not complex beeause a short computer program ean describe the pattern. But neither simple patterns nor random noise are eonsidered conceptual information. As with Shannon information, there is a diseormeet between Kolmogorov eomplexity and eonceptual information. 

Shannon s Information Static Captures only some of the intuitive flavor of complexity complexity is additive, counter to intuition... 

Given an object com[)osed of N interconnected and interacting [)arts, one might at first be te,m[)ted to equate the complexity of an object with its conventional information content, as defined l)y Shannon [.shann49] ... 

Perhaps the simplest approach to defining a complexity of graphs is to apply Shannon s information measure to its vertices [dres74]. Let Vy = Vi 1... 

I Vy 1= h V) depends on the partition. Let V) be the number of nodes in the set Vi- Then the complexity X G) of the graph G is given by minimizing Shannon s information over all possible partitions of this vertex set ... 

Lloyd and Pagels show that these three requirements lead uniquely to an average complexity of a state proportional to the Shannon entropy of the set of (experimentally determined) trajectories leading to the given state (= EiPi oSzPi)- The thermodynamic depth of a state S to which the system S has evolved via the possible trajectory is equal to the amount of information required to specify that trajectory, or Djj S) Hamiltonian systems, Lloyd and... 

The situation becomes more complex when aspects of the trueness of analytical results are included in the assessment. Trueness of information cannot be considered neither by the classical Shannon model nor by Kullback s divergence measure if information. Instead, a model that takes account of three distributions, viz the uniform expectation range, po(x), the distribution of the measured values, p(x), and that of the true value, r(x), as shown in Fig. 9.5, must be applied. 

What is complexity There is no good general definition of complexity, though there are many. Intuitively, complexity lies somewhere between order and disorder, between regularity and randomness, between perfect crystal and gas. Complexity has been measured by logical depth, metric entropy, information content (Shannon s entropy), fluctuation complexity, and many other techniques some of them are discussed below. These measures are well suited to specific physical or chemical applications, but none describe the general features of complexity. Obviously, the lack of a definition of complexity does not prevent researchers from using the term. 

Model complexity is defined as the ratio between the multivariate entropy Sx of the X-block n objects and p variables) of the model and - Shannon s entropy Hyoi the y response vector, thus also accounting for the information content of the y response [Authors, This Book] ... 

The linear response function [3], R(r, r ) = (hp(r)/hv(r ))N, is used to study the effect of varying v(r) at constant N. If the system is acted upon by a weak electric field, polarizability (a) may be used as a measure of the corresponding response. A minimum polarizability principle [17] may be stated as, the natural direction of evolution of any system is towards a state of minimum polarizability. Another important principle is that of maximum entropy [18] which states that, the most probable distribution is associated with the maximum value of the Shannon entropy of the information theory. Attempts have been made to provide formal proofs of these principles [19-21], The application of these concepts and related principles vis-a-vis their validity has been studied in the contexts of molecular vibrations and internal rotations [22], chemical reactions [23], hydrogen bonded complexes [24], electronic excitations [25], ion-atom collision [26], atom-field interaction [27], chaotic ionization [28], conservation of orbital symmetry [29], atomic shell structure [30], solvent effects [31], confined systems [32], electric field effects [33], and toxicity [34], In the present chapter, will restrict ourselves to mostly the work done by us. For an elegant review which showcases the contributions from active researchers in the field, see [4], Atomic units are used throughout this chapter unless otherwise specified. 

Morowitz information index, —> information index on size, —> information index on molecular symmetry, —> information index on amino acid composition, —> information index on molecular conformations, Bertz complexity index, —> Dosmorov complexity index, Bonchev complexity index, —> atomic information indices, —> dectropy index, —> information theoretic topological index, —> information layer index. Information indices are also the Shannon Entropy Descriptors (SHED), while some information indices are among the GETAWAY descriptors. 

Bonchey, D. (2003b) Shannon s information and complexity, in Complexity Introduction and Fundamentals, Vol. 7 (eds D. Bonchev and D.F. Rouyray), Taylor Francis, London, UK, pp. 157-187. 

Even restricting ourselves to the aforementioned factorization, there is no unique definition for complexity. The reason is that there exist different candidates for being one of the coupled factors which give rise to complexity. The most popular ones are well-known to play a relevant role in an information-theoretic framework. Among them, let us mention the Shannon entropy S, the disequilibrium D, the Fisher information I, and the variance V. 

In this work, we will also analyze, apart from C(LMC) and C(FS), the Cramer-Rao complexity C(CR), also as the product of a local and a global measure, keeping the first one as the Fisher information I, and replacing the Shannon entropy exponential by the variance V, giving rise to... 

Analyzing the main information-theoretic properties of many-electron systems has been a field widely studied by means of different procedures and quantities, in particular, for atomic and molecular systems in both position and momentum spaces. It is worthy to remark the pioneering works of Gadre et al. [62,63] where the Shannon entropy plays a fundamental role, as well as the more recent ones concerning electronic structural complexity [27, 64], the connection between information measures (e.g., disequilibrium, Fisher information) and experimentally accessible quantities such as the ionization potentials or the static dipole polarizabilities [44], interpretation of chemical phenomena from momentum Shannon entropy [65, 66], applications of the LMC complexity [36, 37] and the quantum similarity measure [47] to the study of neutral atoms, and their extension to the FS and CR complexities [52, 60] as well as to ionized systems [39, 54, 59,67]. 

---