Comments about Ch. 1 of Applied Logistic Regression, 2nd Ed.

  1. (pp. 5-6) Note that in the case of a single predictor, x, there is no reason why (1.1) on p. 6 is necessarily of the correct form --- the relationship between Y and x need not be such that E(Y|x) has the S shape corresponding to the logistic distribution cdf. Only in the case of x being dichotomous are there necessarily choices for the parameters to make (1.1) correct. This is what makes logistic regression modeling a bit of a challenge at times: one often needs to use transformations of the predictors in order to get a good fit. (The book doesn't get into this in Ch. 1, but does in later chapters.)
  2. (p. 8) Can you quickly show why in the case of the usual regression model, the least squares estimates are also the maximum likelihood estimates?
  3. (p. 9) Can you derive (1.5) and (1.6)?
  4. (p. 9) In the case of x being binary, can you use (1.5) and (1.6) to obtain that the maximum likelihood estimates of pi(0) and pi(1) are just the sample proportions? (In the case of x being binary, does it really make sense to use a logistic regression model, since we just have observations from two Bernoulli distributions?)
  5. (pp. 13-14) Do other books define deviance like (1.8) on p. 13, or is it more common to use (1.9) on p. 13? H&L introduce deviance at this point to make analogies to ordinary regression, but to give the test statistic there is no real need for deviance --- the form of the test statistic indicated in (1.10) on p. 14 is just that of the generalized likelihood ratio test.
  6. (pp. 14-15) In general, with GLR tests, for the null sampling distribution to be approximately a chi-square distribution, in addition to a large enough sample size, one also needs for the assumed parametric model to be correct.
  7. (p. 14) Note that in the case of x being dichotomous, testing using (1.11) is not equivalent to any of the most common tests for the "two-sample proportion problem" (with the most common tests being Fisher's exact test, the usual chi-square test (or equivalently, the usual z test), or the chi-square approximation of Fisher's exact test using Yates' continuity correction). With the data corresponding to a 2 by 2 table, which is what we have if x is dichotomous, I really don't see any point in using logisitc regression. (Comment: I've told some of you before that in a lot of cases where the data corresponds to a 2 by 2 table, I'm in favor of using an exact unconditional test, such as Barnard's test, since it tends to be more powerful than Fisher's exact test. A talk at the 2003 Virginia Academy of Science meeting has made me feel even stronger about this. (Among other things, the presentation showed that the commonly used chi-square test (or equivalently, the commonly used normal z test) can be rather anticonservative for even large sample sizes (for certain ratios of sample sizes), and the speaker, Roger Berger, also claims that unconditional tests are generally more powerful than conditional tests, such as Fisher's exact test.) This web page provides you with an easy way to perform an exact unconditional test (although it wasn't working when I tried it about 5:20 AM on June 6, 2003 (but Roger Berger assured me it worked most of the time)). (I believe the precise test being used is similar to, but not exactly, Barnard's test.))
  8. (p. 16) Note that (about 60% down page) "failing to reject the null hypothesis when the coefficient was significant" is not a good phrase. (If you don't reject, how can you say the coefficient is significant?)
  9. (p. 17) While the z-scores of 5.41, 4.61, and 5.14 indicate about the same thing, in a practical sense, the corresponding approximate p-values differ quite a bit, in relative magnitude. It'll be interesting to see (when we get farther along in the book) how well the best approximate procedure is when compared to an exact test (as can be done using LogXact).
  10. (p. 19) The reference to (1.8) near the middle of the page refers to the (1.8) on p. 10 as opposed to the (1.8) on p. 13. (Yes, there are two expressions labeled (1.8). Has anyone found an errata web page for this book?) Note: People having a later printing than the one I have may find that some of the errors that I identify have been corrected.
  11. (p. 20) Can you justify the confidence interval given by (1.21)? That is, why is it okay to apply the +/- to both occurences of g?
  12. (p. 21) I don't like that H&L indicate that, in statistics, classification is referred to as discriminant analysis.
  13. (p. 22) Do you follow most of p. 22?
  14. (p. 23) On the 10th line, I guess it should be (1.23) instead of (1.15).
  15. (p. 24) Note that variables 16 through 20 are dichotomous variables obtained by threshholding continuous phenomena.
  16. (p. 25 (two places, and also p. 26 and p. 28)) I wonder how the data has been modified, and why it's really necessary to alter the actual observed values.