Basis Functions
The unit vectors,
e1 = (1, 0, 0),
e2 = (0, 1, 0), &
e3 = (0, 0, 1),
are a set of basis vectors for 3-dimensional Euclidean space because any
3-dimensional vector of real numbers can be expressed as a linear combination of
e1,
e2, and
e3.
E.g.,
(2, -11.5, 3.7) =
2e1
- 11.5e2
+ 3.7e3.
If the space of interest to us is a space of functions, then we may be able to identify and work with a
convenient set of
basis functions. E.g., the space of all cubic polynomials is spanned by
h0(x) = 1,
h1(x) = x,
h2(x) = x2, &
h3(x) = x3
--- these four functions are a set of basis functions for the space since any cubic polynomial can be expressed in
the form
a0h0(x) +
a1h1(x) +
a2h2(x) +
a3h3(x).
If we are trying to find an approximation of
E( Y |
(x1,
x2) )
which is a real-valued function of
x1 and
x2,
we might consider the set of basis functions
h0(x1, x2) = 1,
h1(x1, x2) = x1,
h2(x1, x2) = x2,
h3(x1, x2) = x12,
h4(x1, x2) = x22,
& h5(x1, x2) = x1x2,
and use OLS regression to arrive at an approximation of the form
b0h0(x1, x2) +
b1h1(x1, x2) +
b2h2(x1, x2) +
b3h3(x1, x2) +
b4h4(x1, x2) +
b5h5(x1, x2)
= b0 +
b1x1 +
b2x2 +
b3x12 +
b4x22 +
b5x1x2.
While polynomials are commonly used to approximate
E(Y | x),
one can consider other basis sets as well.