Welcome to CSI 771 / STAT 751

Computational Statistics

Fall, 2013

Instructor: James Gentle

Lectures: Thursday, 4:30-7:10pm; Robinson Hall A349

Some of the lectures will be based on notes posted on this website. Some lectures will be accompanied only by notes written on the board. This course is about modern, computationally-intensive methods in statistics. It emphasizes the role of computation as a fundamental tool of discovery in data analysis, of statistical inference, and for development of statistical theory and methods.

Topics

The general description of the course is available at mason.gmu.edu/~jgentle/csi771/

Prerequisites:

  • a course in applied statistics such as STAT 554
  • a course in statistical inference such as CSI 672 / STAT 652.

    Text: Elements of Computational Statistics ISBN 978-1441930248.

    List of probability distributions.

    Computational Software:

    The main computational software that I use is R.

    R is open source and is free. It is installed on some GMU computers, but there are various binary executables available at the main R website, and it is best to load it on your own computer.

    A good way to learn R is just to use it for progressively more complicated problems. While there are many books on R, the various PDF manuals that come with the installation (use "Help" on the GUI) should be sufficient.

    Document Development Software:

    The main document development software that I use is TeX.

    TeX is owned by the American Mathematical Society. It is free. There are various implementations, and it is installed on some GMU computers. One version is MiKTeX. It is available at miktex.org, and it is best to load it on your own computer.

    There are many books on TeX, but a good way to learn TeX is just to use it for progressively more complicated writing projects.

    Email Communication

    The primary means of communication outside of class will be by email.

    Students must use their Mason email accounts to receive important University information, including messages related to this class. (You may, of course, forward email from your Mason email account to one that you check regularly.)

    If you send email to the instructor, please put "CSI 771" or "STAT 751" in the subject line.

    Grading

    Student work in the course (and the relative weighting of this work in the overall grade) will consist of

    Homework

    Each homework will be graded based on 100 points, and 5 points will be deducted for each day that the homework is late.
    Start each problem on a new sheet of paper and label it clearly.
    Homework will not be accepted as computer files; it must be submitted on paper.

    Project

    The course requires each student to complete a project that involves a Monte Carlo study of a statistical method.
    Project will be graded on

    Collaboration and Academic Integrity


    Each student enrolled in this course must assume the responsibilities of an active participant in GMU's scholarly community in which everyone's academic work and behavior are held to the highest standards of honesty. The GMU policy on academic conduct will be followed in this course.

    Collaborative work

    Students are free to discuss homework problems or other topics with each other or anyone else, and are free to use any reference sources. Group work and discussion outside of class is encouraged, but of course explicit copying of homework solutions should not be done.

    Make sure that work that is supposed to be yours is indeed your own.

    With cut-and-paste capabilities on webpages, it is easy to plagiarize.
    Sometimes it is even accidental, because it results from legitimate note-taking; nevertheless, it is plagiarism and it is illegal.

    Although the likelihood of "getting caught" should not influence your ethical standards, you should be aware of the fact that web searches can often identify plagiarism, and that there is even specialized software to facilitate such searches. Whenever I encounter phrases in a student's work that seem to be inconsistent with the usual language that the student uses, I routinely search the web for documents containing those phrases.

    Some good guidelines are here:
    http://ori.dhhs.gov/education/products/plagiarism/
    See especially the entry "26 Guidelines at a Glance".

    Self-Plagiarism

    The definition of ``plagiarism'' applies to the ``work of others'', so copying your own work does not fall within the scope of the crime of plagiarism. Generally, of course, you are free to copy what you've written. I do this all the time with class notes, for example. Whenever you reuse any material, except for relatively brief background or supporting material, you should reference your original source. In the case of my class notes that have not appeared in formal publications, I do not reference my earlier work.

    Representing a rehash or restatement of earlier work as original work is wrong. Such self-plagiarism becomes a breech of academic honor, for example, when a paper submitted for credit in one instance is subsequently submitted for credit in another instance.

    Students with disabilities

    Certification of a disability that requires accommodations must be be made by the Office of Disability Services (ODS). If you are a student with a disability and desire academic accommodations, please contact ODS and inform me during the first two week of classes.

    All academic accommodations must be arranged through the ODS.

    Lectures / assignments / exams schedule

    Week 1, August 29

    Course overview.
    Monte Carlo methods in statistics.
    Random number generation.

    Brief introduction to R.
    R functions.
    Random number generation in R.
    Saving graphics files in R.

    I will rarely post lecture slides (you're expected to attend class and make notes as you see fit!), but since this lecture included many little picky things about R, here they are.

    Assignments:
    (1) Read Appendix A in text (pages 337-350).
    (2) Choose two articles in JASA from 2010 to the present that report Monte Carlo studies and write brief descriptions of them (about a page for each), telling specifically what questions were studied by Monte Carlo.

    Week 2, September 5

    Brief discussion of Monte Carlo methods in statistics and project.
    Objectives and methods of computational statistics.
    Use of the ECDF.
    Statistical methods as optimization problems.

    Assignments: Read revised Chapter 1.
    Work problems 1.3, 1.4, 1.11, 1.17, and 1.22 in revised Chapter 1 to turn in (as hardcopies). (These problem numbers correspond to the current version of the revised Chapter 1. Although the numbers have changed, the problems themselves are the same as the ones originally assigned.)

    Week 3, September 12

    Optimization methods; EM. Lecture slides.
    Matrix calculus.
    Chapter 3.
    Assignment to turn in (as hardcopies). This assignment may be turned in on Sep 26.

    Week 4, September 19


    Monte Carlo methods for statistical inference.
    Simulation of probability distributions.
    Markov chains.

    Assignments: Read Chapter 2 in text. Work problems 2.1, 2.2, 2.4, 2.13, and 2.14 (here) to turn in (as hardcopies). (For the ``test'' in exercise 2.4, you may just use a q-q plot (which just provides a visual assessment, see pages 158ff), a KS test, or any other test you know of. There are functions in R for q-q plots, qqnorm, the KS test, ks.test, the Shapiro-Wilk test shapiro.test, and so on.)

    Week 5, September 26

    Data partitioning: cross validation.
    Monte Carlo methods, tests, use in partitioning (Exercise 2.11).
    Data partitioning: jackknife.
    Data resampling: bootstrap.

    Assignments: Read Chapters 3, 4.
    Work problems 2.9, 3.3, 3.4, 4.1, and 4.2.

    Week 6, October 3

    Review; discussion of homework and miscellaneous problems.

    Assignment, due October 17: Prepare and write up your plan for your project.
    This includes a brief description of the Monte Carlo study in the paper. (What are the statistical methods being evaluated? What scenarios were studied? What are the ``treatments'' (that is, methods) you will study? What scenarios (that is, blocks in your experiment) will you study?)
    I expect this write-up should be about 5 to 10 pages long.

    Week 7, October 10

    Midterm exam. Closed book, closed notes, and closed computers except for one sheet (front and back) of prewritten notes.

    Week 8, October 17

    Vector spaces; representation, transformations, etc.
    Linear structure.
    Here is the lecture I would have given.
    Assignments: Read Chapter 5.
    Work problems 5.1, 5.2, 5.5, and 5.8 in text, plus this one. Due October 31.

    Week 9, October 24

    Structure in data; measures of similarity (Chapters 5 and 10).
    Expansions and approximations.

    Assignments:Read Chapter 10.
    Work problem 5.7 to turn in October 31.

    Week 10, October 31

    Identifying structure in data.
    Principal components (Chapter 10).
    Clustering and classification (Chapter 10).

    Assignment:
    Work problems 10.1, 10.2, , 10.4, 10.5, 10.12 to turn in November 7.

    Week 11, November 7

    Identifying structure in data (Chapter 10).
    Methods of approximating and estimating functions (Chapter 6).

    Assignments: Read Chapter 6.
    Work problems 6.6, 6.7, 6.9, 6.10 to turn in November 14.

    Week 12, November 14

    Nonparametric probability density function estimation. (Chapter 9).

    Assignments: Read Chapter 9.
    Work problems 9.1, 9.2, 9.9 to turn in November 21.

    Week 13, November 21

    Project due
    This should be a hardcopy document that identifies the article you used, describes the problem studied, describes the design of your Monte Carlo study and how it compares with the one in the article and what you did to extend the study, summarizes the results of your study and how they compare with those in the article, and states the conclusions of your study.

    Nonparametric probability density function estimation. (Chapter 9).

    Assignment:
    Work problem 9.12 to turn in December 5.

    November 28

    Class does not meet this week

    Week 14, December 5

    Ten-Minute Presentations of Projects.

    Each student will make a brief presentation on the project.

  • Bibliographic info.
  • Problem studied.
  • How Monte Carlo methods were used in the original research.
  • Results of your Monte Carlo study.
  • What you did to extend the study.
  • Conclusions, what you learned, etc. etc. etc.

    Computer slides are preferred for the presentation.

    December 12

    4:30pm - 7:15pm Final Exam.