Welcome to CSI 972 / STAT 972 and CSI 973 / STAT 973
Mathematical Statistics I and II
Instructor:
James Gentle
If you send email to the instructor,
please put "CSI 972" or "CSI 973" in the subject line.
This two-course sequence covers topics in statistical theory
essential for advanced work in statistics.
Course Objectives:
At the end of this two-course sequence the student
should be very familiar with the concepts of mathematical statistics, and
should have the ability to read the advanced literature in the area.
The student should learn a set of tools for doctoral research and should
have the confidence to embark on such research.
The prerequisites for the first course include a course in mathematical statistics
at the advanced calculus level, for example, at George Mason, CSI 672 / STAT 652,
"Statistical Inference", and a measure-theory-based course in probability, for example,
at George Mason, CSI 971 / STAT 971, "Probability Theory".
The first course begins with a brief overview of concepts and
results in measure-theoretic probability theory that are useful in statistics.
This is followed by discussion of some fundamental concepts
in statistical decision theory and inference.
The basic approaches and principles of estimation are explored,
including minimum risk methods with various restrictions such as
unbiasedness or equivariance, maximum likelihood,
and functional methods such as the method of moments and other plug-in
methods. Bayesian decision rules are then considered in some detail.
The methods of minimum variance unbiased estimation are covered in detail.
Topics include sufficiency and completeness of statistics, Fisher
information, bounds on variances of estimators, asymptotic
properties, and statistical decision theory,
including minimax and Bayesian decision rules.
The second course begins where the first course ends. The second course
covers the principles of hypothesis testing and confidence sets in more
detail. We consider
characterization of the decision process, the Neyman-Pearson lemma and
uniformly most powerful tests, confidence sets, and unbiasedness
in inference procedures.
Additional topics include equivariance, robustness, and estimation of functions.
In addition to the classical results in mathematical statistics, the
theory underlying
Markov chain Monte Carlo, quasi-likelihood, empirical likelihood,
statistical functionals, generalized estimating equations, the jackknife,
and the bootstrap are addressed.
The use of computer software for symbolic computations and for Monte Carlo simulations
is encouraged throughout the courses.
I have put together a set of
notes
to supplement the material in the text and the lectures. These notes
are in the form of a book. It has
a subject index that should be useful. (I am continually working on
these notes, so they may change from week to week.)
The main ingredient for success in a course in mathematical statistics
is the ability to work problems. (It's harder to identify the "main
ingredient" for success in the field of statistics, but
even for that, the ability to work problems is an important component.)
The only way to enhance one's ability to work problems is to
work problems.
It is not sufficient to read, to watch, or to hear solutions to problems.
One of the most serious mistakes students make in courses in
mathematical statistics is to work through a solution that somebody else
has done and to think they have worked the problem.
Some problems, proofs, counterexamples, and derivations should become "easy pieces"
(see my other comments).
An easy piece is something that is important in its own right, but also
may serve as a model or template for many other problems. A student should
attempt to accumulate a large bag of easy pieces. If development of this
bag involves some memorization, that is OK, but things should just naturally
get into the bag in the process of working problems and observing similarities
among problems --- and by seeing the same problem over and over.
Student work in each course will consist of
a number of homework assignments
a midterm exam consisting of an in-class component and, possibly, a
take-home component
a final exam consisting of an in-class component and, possibly, a
take-home component
Instantiations
Fall, 2012:
CSI 972;
Fall, 2011:
CSI 972;
Spring, 2012:
CSI 973.
Fall, 2010:
CSI 972;
Spring, 2011:
CSI 973.
Fall, 2009:
CSI 972;
Spring, 2010:
CSI 973.
Fall, 2008:
CSI 972.
Fall, 2007:
CSI 972;
Spring, 2008:
CSI 973.
Fall, 2005:
CSI 972;
Spring, 2006:
CSI 973.
Fall, 2003:
CSI 972;
Spring, 2004:
CSI 973.
Fall, 2001:
CSI 972;
Spring, 2002:
CSI 973.
Some Useful References for Mathematical Statistics
Texts on general mathematical statistics at the level of this course, more-or-less
- Lehmann, E. L., and George Casella (1998), Theory of Point Estimation,
second edition, Springer.
- Lehmann, E. L., and Joeph P. Romano (2005), Testing Statistical Hypotheses,
third edition, Springer.
There is a useful companion
book called Testing Statistical Hypotheses: Worked Solutions
by some people at CWI in Amsterdam that has solutions to the exercises
in the first edition. (Most of these are also in the third edition.)
- Schervish, Mark J. (1995), Theory of Statistics,
Springer.
This rigorous and quite comprehensive text has a Bayesian orientation.
- Shao, Jun (2003), Mathematical Statistics, second edition, Springer.
Comprehensive and rigorous; better than the first edition.
- Shao, Jun (2005), Mathematical Statistics:
Exercises and Solutions,
Springer.
Solutions (or partial solutions) to some exercises in Shao (2003), plus some
additional exercises and solutions.
Texts in probability and measure theory and linear spaces
roughly at the level of this course
- Ash, Robert B., and Catherine A. Doleans-Dade (1999),
Probability & Measure Theory, second edition, Academic Press.
Accessible and wide-ranging text; also covers stochastic calculus.
- Athreya, Krishna B., and Soumen N. Lahiri (2006),
Measure Theory and Probability Theory, Springer.
A very solid book, but beware of typos in the first printing.
- Billingsley, Patrick (1995), Probability and Measure,
third edition, John Wiley & Sons.
This is one of the best books on probability and measure theory for probability,
in terms of coverage and rigor.
No explicit coverage of linear spaces.
- Breiman, Leo (1968), Probability,
Addison-Wesley.
This is a classic book on measure-theoretic-based probability theory.
No explicit coverage of measure theory or linear spaces.
The book (with corrections) is available in the SIAM Classics in Allied Mathematics
series (1992).
- Dudley, R. M. (2002),
Real Analysis and Probability, second edition, Cambridge University Press.
Accessible and comprehensive.
Texts that provide good background for this course
- Berger, James O. (1985), Statistical Decision Theory and Bayesian
Analysis, second edition, Springer.
- Bickel, Peter, and Kjell A. Doksum (2001), Mathematical Statistics:
Basic Ideas and Selected Topics, Volume I, second edition,
Prentice Hall.
This book covers material from Chapters 1-6 and Chapter 10 of the first edition,
but with more emphasis on nonparametric and semiparametric models and on
function-valued parameters. It also includes more Bayesian perspectives. The second
volume will not appear for a couple of years. In the meantime, the first edition
remains a very useful text.
- Casella, George, and Roger L. Berger (2001), Statistical Inference,
second edition, Duxbury Press.
- Robert, Christian P. (1995), The Bayesian Choice,
Springer.
This is a carefully-written book with a somewhat odd title. This book
is at a slightly higher level than the others in this grouping.
- Hogg, Robert V.; and Allen T. Craig (1994),
Introduction to Mathematical Statistics (5th Edition), Prentice-Hall.
This old standard
is at a slightly lower level than the others in this grouping.
Interesting monographs
- Barndorff-Nielson, O. E., and D. R. Cox (1994), Inference and Asymptotics,
Chapman and Hall.
- Brown, Lawrence D. (1986), Fundamentals of Statistical Exponential
Families with Applications in Statistical Decision Theory,
Institute of Mathematical Statistics.
- Lehmann, E. L. (1999), Elements of Large-Sample Theory,
Springer.
- Serfling, Robert J. (1980), Approximation Theorems
of Mathematical Statistics, John Wiley & Sons.
Interesting compendia of counterexamples
An interesting kind of book is one with
the word ``counterexamples'' in its title. Counterexamples provide useful
limits on mathematical facts.
As Gelbaum and Olmsted observed in the preface to their 1964 book, which was the
first in this genre, ``At the risk of oversimplification, we might say that (aside
from definitions, statements, and hard work), mathematics consists of two classes ---
proof and counterexamples, and that mathematical discovery is directed toward
two major goals --- the formulation of proofs and the construction of
counterexamples.''
- Gelbaum, Bernard R., and John M. H. Olmsted (1990), Theorems and
Counterexamples in Mathematics, Springer.
- Gelbaum, Bernard R., and John M. H. Olmsted (2003), Counterexamples in
Analysis,
(originally published in 1964; corrected reprint of the second printing published by
Holden-Day, Inc., San Francisco, 1965), Dover Publications, Inc., Mineola, New York.
- Romano, Joseph P., and Andrew F. Siegel (1986), Counterexamples in
Probability and Statistics, Chapman and Hall.
In the field of mathematical statistics, this is the most useful
of the ``counterexamples'' books.
It has been rumored that
course instructors get problems from this book. I can neither confirm
nor deny this rumor. I can report that I have the book.
- Stoyanov, Jordan M. (1987), Counterexamples in Probability,
John Wiley & Sons.
- Wise, Gary L., and Eric B. Hall (1993), Counterexamples in Probability
and Real Aanalysis, The Clarendon Press, Oxford University Press.
Interesting set of essays
- Various authors (2002), Chapter 4, Theory and Methods of Statistics, in
Statistics in the 21st Century, edited by Adrian E. Raftery, Martin A.
Tanner, and Martin T. Wells, Chapman and Hall.
The "golden age" of mathematical statistics was the middle third of the
twentieth century, and the content of the books in the first grouping above
cover the developments of this period very well. The set of
essays in Chapter 4 reviews some of the more recent and ongoing work.
Good compendium on standard probability distributions
- Evans, Merran; Nicholas Hastings; and Brian Peacock (2000), Statistical
Distributions, third edition, John Wiley & Sons.
There is also a multi-volume/multi-edition set of books by Norman Johnson and
Sam Kotz and co-authors, published by Wiley.
The books have titles like "Discrete Multivariate Distributions".
(The series began with four volumes in the 70's by Johnson and Kotz.
I have those, but over
the years they have been revised, co-authors have been added, and volumes
have been subdivided.
I am not sure what comprises the current set, but any or all of the books
are useful.)
Software for symbolic computation
There is, of course, no substitute for the ability to understand and
work through mathematical derivations and proofs, but just as data
analysis software aids in understanding statistical methods, software for
symbolic manipulation can aid in working through mathematical arguments.
In addition to the role played by data analysis
software in understanding applied statistics,
the software is a major tool of professionals who do data analysis.
Likewise, software for symbolic manipulation is becoming a major tool for
professionals working in mathematical statistics.
Some steps in work in "higher mathematics" depend on recognition of an
expression as a particular form of some well-known object.
This recognition is essentially a data-retrieval problem. The data
may be stored in one's brain, in a table of integrals, or in some other
place.
It is a
stretch to think of the recognition problem as one
that requires a "higher intelligence", although
certainly the ability to do it easily is an important component of
general mathematical ability. Software packages for symbolic computation
can sometimes help in the mechanical processes of solving
mathematical problems.
The main software packages for symbolic computation are Mathematica,
Maple, Macsyma, and Reduce. There are relatively inexpensive
student versions of all of these. Some University computer labs have
one or more of the packages installed. The SCS science cluster has
Mathematica.
Most of the books listed below
provide introductions to the software using relatively low-level
applications for illustration.
-
Abell, Martha L.; James P. Braselton; and John A. Rafter (1998),
Statistics With Mathematica , Academic Press.
Mostly devoted to elementary data analysis.
-
Andrews, D. F.; and J. E. H. Stafford (2000),
Symbolic Computation for Statistical Inference,
Oxford University Press.
Covers mathematical statistics (as opposed to data analysis).
Emphasizes Mathematica.
- Hastings, Kevin J. (2000),
Introduction to Probability with Mathematica ,
Lewis Publishers, Inc.
- Rafter, John A.; James P. Braselton; and Martha L. Abell (2002),
Statistics with Maple , Academic Press.
Mostly devoted to elementary data analysis.
- Rose, Colin; and Murray D. Smith (2002),
Mathematical Statistics with MATHEMATICA,
Springer.
Covers mathematical statistics (as opposed to data analysis). This book
includes a crippled version of a commercial product called
MathStatica .