The Method of Least Squares

We use the method of least squares to draw the best straight line through experimental data points that have some scatter and do not lie perfectly on a straight line.6 The best line will be such that some of the points lie above and some lie below the line. We will learn to 

SOURCE R. B. Dean atui W J. Dixon, Anal. Chein. 1951, 23, 636 see also D. R. Rorabacher, Anal. Cheni. 1991, 63, 139. 

To evaluate the determinant, multiply the diagonal elements e x h and then subtract the product of the other diagonal elements fx g. 

The procedure we use assumes that the errors in the y values are substantially greater than the errors in the x values.7 This condition is usually true in a calibration curve in which the experimental response (y values) is less certain than the quantity of analyte (x values). A second assumption is that uncertainties (standard deviations) in all the y values are similar. 

Some of the deviations are positive and some are negative. Because we wish to minimize the magnitude of the deviations irrespective of their signs, we square all the deviations so that we are dealing only with positive numbers  

Suppose you measure y at four values of x, plot the data on a graph of y versus x, and draw a line through the data points. 

is positive then the i th data point must be above the line (why ), if d, is negative the point is below the line, and if d, equals zero the line passes through the point. A line is said to fit the data well if the values of most of the residuals arc close to zero. 

There are several ways to determine the line that best fits a set of data, which differ primarily in their definitions of best. The most common method is the method of least squares. 

Suppose there aren plotted points (x[, yi), xi, y2), . ynXso that a line y = ax + b drawn through the points yields a set of n residuals di,d2,---, d . According to the method of least squares, the best line through the data is the one that minimizes the sum of the squares of the residuals The task is therefore to find the values of a and b that minimize 

You can obtain expressions for the best values of a and b in terms of known quantities by differentiating the equation for 4 (Equation A.1-2) with respect to both a and b. setting the derivatives equal to zero, and solving the resulting algebraic equations for a and 6. The results of these calculations are as follows. If we define 

We have already found that the probability function governing observation of a single event x from among a continuous random distribution of possible events x having a population mean p and a population standard deviation a is 

The probability of observing a distribution of events requires that event xi (with probability pi) occur, andxa (with probability P2) occtrr, and so on. The probability of observing events, (xi,X2,X3. x ), is the simultaneous or sequential probability of observing all events in the distribution occtming once, that is, the product of the individual probabilities  

Just as e takes its maximum value when x is at a minimum, the right side of proportion (3-3) is a maximum when its exponent is a minimum. To minimize a fraction with a constant denominator, one minimizes the numerator 

In so doing, we obtain the condition of maximum probability (or, more properly, minimum probable prediction error) for the entire distribution of events, that is, the most probable distribution. The minimization condition [condition (3-4)] requires that the sum of squares of the differences between p and all of the values xi be simultaneously as small as possible. We cannot change the xi, which are experimental measurements, so the problem becomes one of selecting the value of p that best satisfies condition (3-4). It is reasonable to suppose that p, subject to the minimization condition, will be the arithmetic mean, x = )/ provided that 

This method, because it involves minimizing the sum of squares of the deviations xi — p, is called the method of least squares. We have encountered the principle before in our discussion of the most probable velocity of an individual particle (atom or molecule), given a Gaussian distr ibution of particle velocities. It is ver y powerful, and we shall use it in a number of different settings to obtain the best approximation to a data set of scalars (arithmetic mean), the best approximation to a straight line, and the best approximation to parabolic and higher-order data sets of two or more dimensions. 

Before describing this method, let us examine the probability that a measurement will have a particular value (Equation 10.2 with a = l//t /2). The probability that a measurement Xi will lie within an interval dx of the mean value Xq and have a spread (standard deviation) a is 

Consider a set of n measurements, F i)o, where i = 1 to n, and where the subscript o indicates an observed value. The most probable value of this function, F(i)c, where the subscript c indicates a calculated value, is that which minimizes the quantity D, where 

It is important for the reader to understand that in a least-squares refinement of a crystal structure it is the shifts in parameters that are calculated in order to improve the structure, not the parameters themselves. The preliminary parameters that are shifted to more appropriate values come from the trial structures (see Chapters 8 and 9). 

First we will show how the least-squares method can work for a simple linear equation, that is, one in which F i)c is a function of x but no higher powers of x. A typical problem might consist of n observations leading to linear equations (called observational equations) with m 

To calculate the least-squares line through a series of points. 

---

If the experimental error is random, the method of least squares applies to analysis of the set. Minimize the sum of squares of the deviations by differentiating with respect to m. 

Linear regression, also known as the method of least squares, is covered in Section 5C. 

Application. Merriman ( The Method of Least Squares Applied to a Hydraulic Problem, y, Franklin Inst., 23.3-241, October 1877) reported on a study of stream velocity as a function of relative depth of the stream. 

A straight line may be fitted to the (X, Y) or (X, Y) pairs of data when plotted on log-log graph paper from which the slope N and the intercept log K with X = 1 may be read. Alternatively, the method of least squares may be used to estimate the values of K and N, giving the best fit to the available data. 

Nonlinear regression, a technique that fits a specified function of x and y by the method of least squares (i.e., the sum of the squares of the differences between real data points and calculated data points is minimized). 

Once a linear relationship has been shown to have a high probability by the value of the correlation coefficient (r), then the best straight line through the data points has to be estimated. This can often be done by visual inspection of the calibration graph but in many cases it is far better practice to evaluate the best straight line by linear regression (the method of least squares). 

This approach was applied to data obtained by Hausberger, Atwood, and Knight (17). Figure 9 shows the basic temperature profile and feed gas data and the derived composition profiles. Application of the Hougen and Watson approach (16) and the method of least squares to the calculated profiles in Figure 9 gave the following methane rate equation ... 

The method of least squares provides the most powerful and useful procedure for fitting data. Among other applications in kinetics, least squares is used to calculate rate constants from concentration-time data and to calculate other rate constants from the set of -concentration values, such as those depicted in Fig. 2-8. If the function is linear in the parameters, the application is called linear least-squares regression. The more general but more complicated method is nonlinear least-squares regression. These are examples of linear and nonlinear equations ... 

The quantities boo and Soo represent the solution of the problem of drawing parallel lines through the given set of points by the method of least squares. They are obtained relatively easily and can serve to check the whole calculation. The function Sx = f(x) always has a minimum however, the conditions have not been explored when it has only one minimum. This case would be important from the theoretical point of view however, in practice there is no real danger... 

Deutsch and Hansch applied this principle to the sweet taste of the 2-substituted 5-nitroanilines. Using the data available (see Table VII), the calculated regression Eqs. 5-7 (using the method of least squares) optimally expressed the relationship between relative sweetness (RS), the Hammett constant, cr, and the hydrophobic-bonding constant, ir. 

Table 2.3 is used to classify the differing systems of equations, encountered in chemical reactor applications and the normal method of parameter identification. As shown, the optimal values of the system parameters can be estimated using a suitable error criterion, such as the methods of least squares, maximum likelihood or probability density function. 

Gans, P., Data Fitting in the Chemical Sciences by the Method of Least Squares, Wiley, New York, NY, (1992). 

Plackett, R.L., "Studies in the History of Probability and Statistic. XXIX The Discovery of the method of least squares", Biometrika, 59 (2), 239-251 (1972). 

Another advantage of the Savitsky-Golay method is that derivatives of these functions can also be determined from the method of least squares. This method can be used to determine alpha-peak temperatures automatically since the first derivative changes sign at the peak temperature. The advantage of smoothing is that the number of extraneous peaks due to noise has been minimized. 

Data may be fitted to this equation by the method of least squares in order to determine values of the constants log /c, fiA, / B, etc. The goodness of fit may be shown in graphical form by using the values of determined in this manner to calculate (Cf). A plot of the reaction rate versus this function should then meet the criteria of the general method outlined above. 

What is the method of least squares and why is it useful in instrumental analysis ... 

What is it about the calculations involved in the method of least squares that gives this method its name ... 

Give four parameters that are readily obtainable as a result of the method of least squares treatment of a set of data. 

The method of least squares is a procedure by which the best straight line through a series of data points is mathematically determined. More details are given in Section 6.4.4. It is useful because it eliminates guesswork as to the exact placement of the line and provides the slope and y-intercept of the line. 

Linear regression is another name for the process of determining the straight line for a series of data points via the method of least squares. 

Linearity is evaluated by appropriate statistical methods such as the calculation of a regression line by the method of least squares. The linearity results should include the correlation coefficient, y-intercept, slope of the regression line, and residual sum of squares as well as a plot of the data. Also, it is helpful to include an analysis of the deviation of the actual data points for the regression line to evaluate the degree of linearity. 

All contributions at each position of substitution should sum to zero. The series of linear equations thus generated is solved by the method of least squares for terms flj and ji. There must be several more equations than unknowns and each substituent should appear more than once at a position in different combinations with substituents at other positions. The attractiveness of this model, also referred to as the de novo method, is as follows ... 

In order to obtain the calibration curve of pesticide metobromuron, absorbances were measured at the corresponding as a function of concentration and the data were fitted to Lambert-Beer law by the method of least square analysis. The resulting correlation coefficient (R 1.0000) show that the fit to Lambert-Beer law is excellent. The obtained calibration equation was used to convert absorbances into concentrations in kinetic and equilibrium studies. 

---