I am also using LU factorization to compute the inverse matrix inside my multiple linear regression solver, so what about its numerical stability ?

Generally speaking without geting into much detail:

Of course LU is numerically stable but that has nothing to do with the problem domain itself. SVD is the tool to go since it allows per definition to

do deep error analysis per se. E.g. If you want to invert a "bad" behaving matrix LU will simply tell you "can't do" wheras SVD will provide

you information within the singular values which can be exploited to still find a least squares answer. For example if a singular value is very

small it is set to zero instead of inverted and left as is for further processing and still you have your least squares solution at hand.

If you do it by LU or whatever solution you might get into troubles especially if you start forming some kind of covariance matrix A*A' - if there is a lot of linear

dependence you will badly fail to calculate the inversion properly without doing it by SVD

Check out

http://en.wikipedia.org/wiki/Moore%E2%80%93Penrose_pseudoinverse which shows a few algorithms to calculate the pseudoinverse.

ftp://ftp.demec.ufpr.br/CFD/bibliografia/Higham_2002_Accuracy%20and%20Stability%20of%20Numerical%20Algorithms.pdf
In terms of speed SVD is of course the worst choice but again I favour stability over speed so if the data you are processing is behaving nicely and you

have enough memory by hand do the direct approach if not I would suggest to perform the operating using SVD.

Also in my projects I follow the simple rule to avoid any matrix inversion if possible.

kind regards

Mike

Connect with Us