Acoustic-distance join costs

It is clear that the acoustic differences between the units being joined have a significant impact on how successful the join will be, and this observation is the basis of the acoustic-distance join cost. Perhaps the simplest way of implementing this is to compare the last frame in die left diphone with the first frame in the right diphone. For instance, in Hunt and Black [227], two cepstral vectors are compared. 

Among the metrics used to calculate the distances were the following. 

Mahalanobis distance. The Euclidean distance treats all components of the vector equally, whereas the Mahalanobis distance compntes a distance in which each component is scaled by the inverse of its variance. It can be thought of as a measure that normahses the space before computing the distance  

Kullback-Leibler-style distances. The Kullback-Leibler divergence is a measure of distance between two probabihty distributions. From this, we can derive an expression that calculates the distance between two spectral vectors [251], [471]  

The above studies shed interesting light on the relationship between acoustic measures and their perception, but also show that there seems to be an upper limit to how far this approach can go. From Table 16.2, we see that the best correlation between an acoustic cost and perceptual judgment is only 0.66, which is far from the type of correlation that we would be happy to accept as a scientific rule. Given the number of studies and that nearly all the well-known acoustic measures (MFCCs, LSFs, formants etc.) and all the distance metrics (Euclidean, Mahalanobis, Kullback-Leibler) have been studied, we can be fairly sure that this area has been investigated thoroughly and that no combination of features and distance metric is likely to improve significantly on the results in Table 16.2. 

---

Another way of improving on the basic acoustic distance join cost is the probabilistic sequence join function, which takes more fi ames into account than just those near the join. Vepa and King [470] used a Kalman filter to model the dynamics of frame evolution, and then converted this into a cost, measured in terms of how far potential joins deviated from this model. A full probabilistic formulation, which avoids the idea of cost altogether was developed by Taylor [438],... 

One possible reason for this is that acoustic distances are limited in that they usually only consider fi-ames near the join, whereas we know that context effects can be felt over a distance of two or three phones. As the categorical features often operate at the phone level or above, they compensate for the short term nature of the acoustic features. As Coorman et al [108] state, it is well known that vowels followed by an [r] are heavily influenced by the [r] and this fact alone is useful in designing the join cost (where we could for instance have a sub-cost which states that units to be joined should either both have an [r] context or both not have an [r] context). Syrdal [427] investigates the hstener responses of a mixed system. 

An acoustic join cost which measures the acoustic distance across a join gives a reasonably indication of how well two imits will join. 

---