Residuals and Residual Analysis

If Y is the observed data vector and Y is the model predicted data vector, ordinary residuals are the difference between observed and model predicted values 

An unbiased model should have residuals whose mean value is near zero. For a linear model the residuals always sum to zero, but for a nonlinear model this is not always the case. Conceptually one metric of goodness of fit is the squared difference between observed and predicted values, which has many different names, including the squared residuals, the sum of squares error (SSE), the residual sum of squares, or error sum of squares 

The problem with SSE is that SSE decreases as the number of model parameters increases. Alternatively one could calculate the variance of the residuals called the mean square error (MSE) 

The residuals themselves also contain important information on the quality of the model and a large 

Scatter plot of predicted value (ordinate) versus residual (abscissa). No systematic trend in the residuals should be observed with the data appearing as a shotgun blast. Systematic trends are indicative of model misspe-cification. See Fig. 1.5 for an example. 

---