Numerical Linear Algebra for Applications in Statistics
by
James E. Gentle
Subject Index
A-optimality 171
absolute error 20, 26, 81
{\sl ACM Transactions on Mathematical Software} 198
{\sl ACM Transactions on Modeling and Computer Simulation} 198
algorithm, definition 37
Alta Vista (Web search engine) 201
Amdahl's law 37
AMS MR classification system 198
angle between vectors 51, 74
ANSI (standards) 13
{\sl Applied Statistics} 198, 200
artificial ill-conditioning 79
ASCII code 1
axpy 49
backward error analysis 26, 30
Banachiewicz factorization 93
base 6
base point 5
basis 50, 84
bias, in exponent of floating-point number 7
big endian 16
big O (order) 28, 33
bilinear form 58, 64
bit 1
bitmap 3
BLACS (software) 142
BLAS (software) 140, 142, 154
byte 1
C (programming language) 12, 24, 143
C++ (programming language) 13
{\sl CALGO} 198, 201
cancellation error 23, 31
catastrophic cancellation 22
Cauchy-Schwarz inequality 51, 58
Cayley multiplication 56
CDF (Common Data Format) 2
chaining of operations 21
character data 3
character string 3
characteristic equation 67, 123
characteristic value, {\it see} eigenvalue
chasing 130
Cholesky decomposition 93, 179
column rank 60
column space 53, 60
column-major 81, 137, 138
column-sum norm 72
Common Data Format (CDF) 2
{\sl Communications in Statistics --- Simulation and Computation} 199
companion matrix 123
complete pivoting 90
complex data type 13, 14, 24, 25, 138
COMPSTAT 197, 199
{\sl Computational Statistics} 199
{\sl Computational Statistics and Data Analysis} 199
{\sl Computing Science and Statistics} 199
condition (problem or data) 30
condition number 30, 32, 75, 107, 115, 172
condition number with respect to computing a sample standard deviation 32
condition number with respect to inversion 77, 107
conjugate gradient 105
conjugate transpose 53, 64
conjugate vectors 64
consistency property for matrix norms 72
consistency test for software 157
consistent system of equations 62
constrained least squares, equality constraints 166, 180
convergence criterion 37
convergence ratio 38
correlation matrix 161, 176
cross product, computing 45
Crout method 92
{\sl Current Index to Statistics} 198
curse of dimensionality 39
D-optimality 115, 170, 171
{\tt daxpy} 49
defective matrix 68
deficient matrix 68
derivative with respect to a vector or matrix 79
det$(\cdot)$ 55
determinant of a matrix 55, 115, 171
diag$(\cdot)$ 54, 59
dimension of vector space 49
direct method, linear systems 87, 124
direct sum 50
discretization error 29, 38
divide and conquer 36
dominant eigenvalue 124
Doolittle method 92
dot product of matrices 57
dot product of vectors 50, 58
double precision 11, 16
E-optimality 171
eigenvalue 67, 72
eigenvalue of a polynomial 85
eigenvector 67
EISPACK 142
elementary operator matrix 88
elliptic norm 71
error, absolute 20, 26, 81
error, cancellation 23, 31
error, discretization 29
error, measures of 27, 28, 108
error, relative 20, 26, 81
error, rounding 23, 26
error, rounding, models of 27, 44
error, truncation 29
error bound 28
error of approximation 29
error-free computations 23
errors-in-variables 162
Euclidean matrix norm 73
Euclidean vector norm 71
Euler's constant 42
exact computations 23
exception, in computer operations 19, 23
exponent 6
exponential order 33
extended precision 11
extrapolation 38
fan-in algorithm 21, 36
fast Givens rotation 101
fixed-point representation 5
flat 51
floating-point representation 5
FLOP, or flop 35
FLOPS, or flops 35
Fortran 138, 143, 147
Fortran 77 138
Fortran 90 138
Fortran 95 147
Fortran 2000 147
Frobenius norm 73
full precision 16
full rank 60
$g_1$ inverse 63
$g_2$ inverse 63
$g_4$ inverse (pseudoinverse) 63
GAMS ({\sl Guide to Available Mathematical Software}) 137, 201
GAMS, electronic access 201
Gauss (software) 148
Gauss-Seidel method 103
Gaussian elimination 87, 130
generalized inverse of a matrix 63, 96
generalized least squares 166
generalized least squares with equality constraints 180
generating set 50
gif (graphical interchange file) 3
Givens transformation (rotation) 96, 99, 130
GMRES 106
graceful underflow 8
gradient of a function 80
gradual underflow 8, 23
Gram-Schmidt procedure 102
Gram-Schmidt transformation 74, 102
greedy algorithm 36
guard digit 21
{\sl Guide to Available Mathematical Software}, {\it see} GAMS
Hadamard multiplication 57
half precision 16
hat matrix 66, 164
HDF (Hierarchical Data Format) 2
Helmert matrix 67
Hermitian matrix 53
Hessenberg matrix 129
hidden bit 7
Hierarchical Data Format (HDF) 2
Hilbert matrix 155
Horner's method 40
HotBot (Web search engine) 201
Householder transformation (reflection) 96, 97, 130
{\tt html} 200
hyperplane 51
hypothesis testing 169
idempotent matrix 66
identity matrix 60
IDL (software) 148
IEEE standards 4, 11, 16, 22
ill-conditioned (problem or data) 30
ill-conditioned data 30, 75, 172
image data 3
IMSL Libraries 143
induced matrix norm 72
infinity, floating-point representation 11, 12, 23
infix operator 24
inner product 50, 92
inner product space 51
integer representation 5
Interface Symposium 197, 199
International Association of Statistical Computing (IASC) 197, 199
Internet 199, 200
interval arithmetic 25, 26
inverse of a matrix 61
ISO (standards) 13
isometric matrix 73
iterative method 37
iterative method, linear systems 87, 103, 124
iterative refinement 109
Jacobi method for eigenvalues 124, 126
Jacobi transformation (rotation) 100
Jacobian 80
Java (software system) 200
{\sl Journal of Computational and Graphical Statistics} 199
{\sl Journal of Statistical Computation and Simulation} 199
Kalman filter 32
kind (for data types) 14
Kronecker multiplication 57
Krylov space 106
Lagrange multiplier 166, 180
Lanczos method 106
LAPACK 48, 91, 116, 142
latent root, {\it see} eigenvalue
$LDU$ factorization 92
least squares 95, 111
length of a vector 48, 51, 71
leverage 164, 181
linear convergence 38
linear independence 49
linear regression 162
LINPACK 48, 91, 116, 142
Lisp-Stat (software) 148
little endian 17
little o (order) 29
log order 33
Longley data 180
loop unrolling 140
$LU$ factorization 91, 92
$L_p$ norm 71, 72
MACHAR 11, 42
machine epsilon 9
Manhattan norm 71
manifold of a matrix 53
Maple (software) 25, 137
{\sl Mathematical Reviews} 198
{\tt mathml} 200
Matlab (software) 148
matrix 52
matrix derivative 79
matrix factorization 68, 92
matrix inverse 61
matrix multiplication 56, 59, 82
matrix of type 2 53
matrix polynomial 85
matrix storage mode 143
max norm 71
message passing 142
Message Passing Library 142
MIL-STD-1753 standard 15
Minkowski inequality 71
missing value 12
mobile Jacobi scheme 129
modified Cholesky decomposition 179
modified Gram-Schmidt procedure 102
Moore-Penrose inverse 63, 95, 96, 113
Mosaic (Web browser software) 200
MPI (message passing interface) 142
MPL (Message Passing Library) 142
MR classification system 198
multigrid method 107
multiple precision 25
NAG Libraries 143
NaN (``not-a-number'') 12, 23
{\tt netlib} x, 198, 201
nonnegative definite matrix 66, 67
nonsingular matrix 60
norm, vector and matrix 70, 71, 72
normal equations 95, 111, 163
normal (normalized) vector 60, 64
normalized floating-point numbers 7
normalized vector 64
not-a-number (``NaN'') 12
NP-complete problem 34
null space 62, 64
Octave (software) 150
1-norm, {\it see} $L_p$ norm
operator matrix 88
order of computations 33
order of convergence 28
order of error 28
order of a vector 48
order of a vector space 49
orthogonal complement 64
orthogonal matrix 64
orthogonal transformation 74
orthogonal vector spaces 64
orthogonal vectors 64, 84
orthogonality 64
orthogonalization, Gram-Schmidt 102
orthogonalization transformation 74
orthogonally similar 70, 73, 74, 78
orthonormal vectors 64
outer product 58, 92
overdetermined linear system 62, 94, 111
overflow, in computer operations 20, 23
overloading 16, 24, 49, 72
$p$-norm, {\it see} $L_p$ norm
paging 139
parallel processing 83, 84, 153
partial pivoting 90
partitioned matrix 58
partitioned matrix, inverse 61, 64
PBLAS (parallel BLAS) 142
permutation matrix 65, 88
pivoting 90, 96
PLAPACK 142
polynomial, evaluation of 40
polynomial order 33
portability 17, 137
positive definite matrix 66, 67, 93, 176
positive semidefinite matrix 66
power method for eigenvalues 124
precision, double 11, 16
precision, extended 11
precision, multiple 25
precision, single 11, 16
principal submatrix 58, 93
probabilistic error bound 28
{\sl Proceedings of the Statistical Computing Section} 199
projection matrix 66, 164
proper value, {\it see} eigenvalue
pseudoinverse 63
\\{\it also see} Moore-Penrose inverse
PV-Wave (software) 52, 148
$QR$ factorization 95
$QR$ method for eigenvalues 103, 124, 129
quadratic convergence 38
quadratic form 58, 71
R (software) 153
radix 6
range of a matrix 53
rank, linear independence 60, 115
rank, number of dimensions 52
rank of a matrix 60, 96, 115, 173
rank of an array 52
rank revealing $QR$ 96, 115, 173
rank$(\cdot)$ 60
rank-one update 97, 110
rate constant 38
rate of convergence 38
real numbers 5
recursion 39
reflection 97
register, in computer processor 21
regression 162
regular matrix 68
relative error 20, 26, 81
relative spacing 9
{\sl Reliable Computing} 26
residue arithmetic 23
Richardson extrapolation 39
ridge regression 179
robustness (algorithm or software) 30
root of a function 22
root-free Cholesky 93
Rosser test matrix 156
rotation 99, 100
rounding error 23, 26
row rank 60
row-major 81, 137, 138
row-sum norm 72
S, S-Plus (software) 151
sample variance, computing 31
{\tt saxpy} 49
ScaLAPACK 142
scaling of an algorithm 33
scaling of a vector or matrix 79
Schur complement 61, 165
shape of matrix 52
Sherman-Morrison formula 110, 167
{\sl SIAM Journal on Scientific Computing} 199
side effect 141
${\rm sign}(\cdot)$ 66
sign bit 4
significand 6
similar canonical form 68
similar matrices 69
similarity transformation 69, 126, 130
simple matrix 68
single precision 11, 16
singular matrix 60
singular value 69, 102
\\{\it also see} eigenvalue
singular value decomposition 69, 102, 131
software testing 155
SOR (method) 104
${\rm span}(\cdot)$ 50
spanning set 50
sparse matrix 82, 107, 137, 144, 154
spectral decomposition 69
spectral norm 73
spectral radius 73, 104
spectrum of a matrix 67
splitting extrapolation 39
square root, matrix 93
stability 30, 90
standard deviation, computing 31
Statistical Computing Section of the American Statistical Association 197, 199
{\sl Statistics and Computing} 199
{\tt statlib} x, 198, 200
stiff data 32
stochastic matrix 175
stopping criterion 37
storage mode, for matrices 143
storage unit 3, 6, 16
Strassen algorithm 83
stride 82, 137, 141
string, character 3
successive overrelaxation 104
sum of vector spaces 50
superlinear convergence 38
SVD {\it see} singular value decomposition
sweep operator 165
symmetric matrix 53
testbed 155
testing software 155
Toeplitz matrix 54, 159, 176
total least squares 162
trace$(\cdot)$ 55
trace of a matrix 55
truncation error 29
2-norm, {\it see} $L_p$ norm
twos-complement representation 4, 19
type 2 matrix 53
ulp (``unit in the last place'') 10
underdetermined linear system 63
underflow, in computer operations 8, 23
Unicode 2
unit in the last place 10
unit roundoff 9
unit vector 60
unitary matrix 64
unrolling do-loop 140
updating a solution 109, 114, 167
upper Hessenberg form 129
URL 200
variance, computing 31
vec$(\cdot)$ 54
vech$(\cdot)$ 55
vector 48
vector processing 153
vector space 49, 53, 62, 64
W3C (World Wide Web Consortium) 200
Ware's law 37
Web browser 200
weighted least squares 166
weighted least squares with equality constraints 180
Wilkinson matrix 156
wmf (windows meta file) 3
Woodbury formula 110, 167
word, computer 3, 6, 16
World Wide Web (WWW) 200
World Wide Web Consortium 200
{\tt xml} 200
{\tt Xnetlib} 201
Yahoo (Web search engine) 202
Yule-Walker equation 176