Errors-in-variables is a robust modeling technique in statistics, which assumes that every variable can have error or noise. Errors-in-variables (EIV) is also referred to as total least squares (TLS), in a broad sense, in the literature of computational mathematics and engineering. However, TLS in a strict sense implies the application of EIV or orthogonal regression to a linear model .

Robust linear regression

In linear regression the least squares (LS) attributes all error to the dependent variables. It has variant versions according to other error configurations such as total least squares (i.e. orthogonal error), data least squares (DLS), constrained or structured TLS and so on. Least squares is a mathematical optimization technique that attempts to find a best fit to a set of data by attempting to minimize the sum of the squares of the differences (called residuals) between the fitted function and the data. ...

Given an observation vector and a data matrix , consider the solution of the overdetermined system of equations . For the square matrix section, see square matrix. ...

The ordinary least square method (OLS) yields the solution that minimizes the Euclidean norm of error or residual . Equivalently, the problem can be solved by In mathematics and astronomy, Euclidean space is a generalization of the 2- and 3-dimensional spaces studied by Euclid. ... An error has different meanings in different domains. ... In general, a residual is a positive or negative numeric difference between two numbers. ...

If the data matrix is also noisy (i.e. error in both the dependent and the explanatory variables), the OLS solution is no longer optimal. In case orthogonal optimization is acceptable, TLS offers a proper formulation:

where is the Frobenius norm (or in human English: the "length" of the vector); and the perturbations and are used to compensate for the noisy signals and , respectively. This formulation of TLS also implies that the errors are identically distributed both in and . Note that the objective can have a weighting matrix according to the distribution of errors if the distribution is known or well-estimated, which is called the constrained or structured TLS. In mathematics, the term matrix norm can have two meanings: A vector norm on matrices, i. ...

In the other case, where the noise is only in , DLS can be used alternatively as

The solution of OLS can be obtained using (pseudo-)inverse of data matrix. The other solutions of TLS or DLS have been shown to be closely connected to a set of singular vectors of (augmented) system-related matrix corresponding to the minimum singular value. For the square matrix section, see square matrix. ...

References

• S. V. Huffel and P. Lemmerling, Total Least Squares and Errors-in-Variables Modeling: Analysis, Algorithms and Applications. Dordrecht, The Netherlands: Kluwer Academic Publishers, 2002.
• S. Jo and S. W. Kim, "Consistent normalized least mean square filtering with noisy data matrix," IEEE Trans. Signal Processing, vol. 53, no. 6, pp. 2112-2123, Jun. 2005.
• R. D. DeGroat and E. M. Dowling, "The data least squares problem and channel equalization," IEEE Trans. Signal Processing, vol. 41, no. 1, pp. 407–411, Jan. 1993.
• T. Abatzoglou and J. Mendel, "Constrained total least squares," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP’87), Apr. 1987, vol. 12, pp. 1485–1488.