Comments Related to Chapter 1

  1. [last 2 sentences of 1st paragraph on p. 1] I don't like the broader interpretation of the word parameter. To me a parameter is an unspecified constant appearing in the specification of a probability distribution (of a parametric family). For example, if one is working with a normal distribution model, assuming i.i.d. random variables having a normal distribution with unknown mean μ and unknown standard deviation σ, then μ is certainly a parameter. But if we are just assuming that the observations come from an unspecified continuous distribution having mean μ, then I don't consider μ to be a parameter. (I refer to it as a distribution measure, since the distribution mean is a summary measure for the unspecified distribution). To me, parameters are associated with parametric distributions (but not everyone agrees with this purist interpretation of the word parameter).

  2. [the 2 complete paragraphs on p. 2 & top half of p. 5] The authors describe some aspects of classical parametric statistical inference. If one can assume a specific parametric model for the distribution underlying the observed data, then typically STAT 652 methods can be used to arrive at an optimal (in some sense), or at least a very good, inference procedure to apply. If the assumptions are perfectly met, there is little to worry about with regard to the choice of procedure. (The data of a particular sample can still result in a misleading inference, but the method chosen is one which should do, on average, about as good as can be expected.) But quite often it is questionable that the assumptions are met (or one might even know that the assumptions aren't perfectly met), and then the robustness of the derived parametric procedure comes into play. In some cases, based on an asymptotic analysis, it can be argued that the paramteric procedure ought to still perform decently if the assumptions are are only mildly violated and the sample size isn't too small. But with a small sample size, not only do we not have the large sample size that would make us more comfortable in relying on an asymptotic justification for the choice of procedure, but we are also unable to adequately check whether or not the assumptions appear to be only mildly violated.

  3. [bottom third of p. 2 and top half of p. 3] The authors begin to address what is meant by the terms distribution-free and nonparamteric.
    • Distribution-free is the easy one. It just means that the pertinent sampling distribution of a statistic (usually the null hypothesis distribution of a test statistic) is the same no matter what the parent distribution (aka underlying distribution) of the data is (although in some cases there can be stipulations, such as the distribution has to be continuous, or the distribution has to be symmetric). For example, when doing a test of the null hypothesis that the distribution median is 0 against the alternative that the median is not equal to 0, no matter what continuous distribution underlies the data, the null sampling distribution of the sign test statistic is a binomial (n,0.5) distribution (where n is the sample size). We don't have to know what the parent distribution of the data is in order to know what the null sampling distribution of the test statistic is. So, as the book states, "assumptions regarding the underlying population are not necessary."
    • Nonparametric is a bit harder to nail down, and not everyone uses this term the same way. The authors claim that "the term nonparametric test implies a test for a hypothesis which is not a statement about parameter values." Using the authors' broader interpretation of the term parameter, the sign test (a test about the median) is a distribution-free test, but not a nonparametric test ... although lots of people think of the sign test as a nonparametric test. But if we use my more restricted interpretation of the term parameter, there is no dilemma; when no parametric model is assumed, the median is not a parameter, and so the hypotheses do not involve any parameters. Even with the authors' broader interpretation of the term parameter, we'll see that most of the distribution-free tests covered in the book are nonparametric tests. (E.g., the common distribution-free tests based on two independent samples are fundamentally tests of the null hypothesis that the two parent distributions are identical against the general alternative that the two distributions aren't the same. Also, the tests in Ch. 11 which are in a sense distribution-free analogs of the t test of the null hypothesis that ρ equals 0 against the alternative that ρ is not equal to 0 (where ρ is Pearson's correlation coefficient) are fundamentally tests of the null hypothesis that two variables are independent against the alternative that the two variables are not independent. In both of these classes of distribution-free tests, the hypotheses do not involve parameters, and so they are nonparametric tests even if the broader interpretation of the term parameter is used.) Using my narrower interpretation of parameter seems to make things simpler: if you don't assume a parametric model, there can't be parameters, and so any test done when a parametric model has not been assumed is a nonparametric test. Using my narrower interpretation of the term parameter broadens the class of tests which can be called nonparametric tests. (Note: Oddly, while lines 12-14 on p. 3 suggest that parametric methods are those which are based on specific parametric models, but the authors don't simply define nonparametric methods as those which are not derived by assuming a specific parametric model.) (E.g., not only is the sign test about the median a nonparametric test, but so is Johnson's modified t test about the mean of a skewed distribution, even though many consider it to belong to the class of robust tests. (I think it's fine to allow some tests to be considered to be both robust tests and also nonparametric tests.) But many robust tests, for example tests based on M-estimators about the parameters of a regression model, are not nonparametric tests. (In the case of the test about regression model parameters, obviously we're dealing with parameters. The regression model has parameters even if we don't assume a particular form for the error term distribution and have a fully-specified parametric model.) It's interesting that even though we can view Johnson's modified t test as a nonparametric test, it's not a distribution-free test in a strict sense. Even though one always uses the T distribution with n-1 degrees of freedom as the null hypothesis reference distribution, the test statistic does not have this distribution exactly. The exact null sampling distribution of the test statistic is unknown, and it does depend on the parent distribution underlying the data. So Johnson's modified t is always an approximate test, with the approximation being pretty good in some cases and not so good in others.) At the risk of making things even more confusing for some of you, I'll point out that some goodness-of-fit tests can be considered to be nonparametric tests even though the hypotheses are about a specific distribution. For example, we can use the Kolmogorov-Smirnov test to test the null hypothesis that the data are observations of i.i.d. exponential random variables having mean 100 against the general alternative that this is not the case. So here we have a nonparametric test that relates to a specific distribution, but if you think about it, it's not a test about a specific parameter. The null hypothesis is about a completely specified distibution; it's not about the unknown value of a parameter. While it may seem a little fuzzy about whether or not the Kolmogorov-Smirnov test is a nonparametric test, there is no question about whether or not it's a distribution-free test. Whether we use the K-S test to do a test about a specific exponential distribution, a specific Pareto distribution, or a specific normal distribution, the null hypothesis distribution of the test statistic is always the same, so it is a distribution-free test.
    • As a final tidbit about what is meant by nonparametric, it's interesting to note that the first sentence of the first page of the well-known book by Hollander and Wolfe states
      a nonparametric procedure is a statistical procedure that has certain desirable properties that hold under relatively mild assumptions regarding the underlying populations from which the data are obtained.
      (E.g., the desirable property could be that the null sampling distribution of the test statistic is the same whatever the underlying distribution of the data is, and the mild assumption could be that we can view the data as being observations of iid continuous random variables.) I think this is a decent way to define what a nonparametric statistical procedure is. I think a key point is whether or not a specific parametric model has been assumed and not whether or not the hypotheses of a test are stated in terms of parameters.