MTB > # ACD, ACD+A data from Beyond ANOVA
MTB > # The 12 differences (of the form ACD+A - ACD) are in c3.
MTB > dotplot c3
: : . .. . : . .
-------+---------+---------+---------+---------+---------diff
-30 -20 -10 0 10 20
MTB > desc c3
N MEAN MEDIAN TRMEAN STDEV SEMEAN
diff 12 -2.24 2.30 -0.81 16.07 4.64
MIN MAX Q1 Q3
diff -34.20 15.40 -4.10 7.38
MTB > plot c2 c1
-
- *
60+
- *
ACD+A - *
- *
- * *
45+
- * *
-
-
- *
30+ *
- *
-
- *
-
------+---------+---------+---------+---------+---------+ACD
24 36 48 60 72 84
MTB > ttest 0 c3
TEST OF MU = 0.00 VS MU N.E. 0.00
N MEAN STDEV SE MEAN T P VALUE
diff 12 -2.24 16.07 4.64 -0.48 0.64
MTB > wtest 0 c3
TEST OF MEDIAN = 0.000000 VERSUS MEDIAN N.E. 0.000000
N FOR WILCOXON ESTIMATED
N TEST STATISTIC P-VALUE MEDIAN
diff 12 12 44.0 0.724 1.575
MTB > stest 0 c3
SIGN TEST OF MEDIAN = 0.00000 VERSUS N.E. 0.00000
N BELOW EQUAL ABOVE P-VALUE MEDIAN
diff 12 5 0 7 0.7744 2.300
MTB > # Now for the approximate version of the normal scores test.
MTB > # First I'll obtain the 12 scores to be used.
MTB > set c4
DATA> 1:12
DATA> end
MTB > let c4 = (13 + c4)/26
MTB > invcdf c4 c5;
SUBC> norm 0 1.
MTB > name c5 'scores'
MTB > print c5
scores
0.09656 0.19403 0.29338 0.39573 0.50240 0.61514 0.73632
0.86942 1.02008 1.19838 1.42608 1.76882
MTB > # Now I'll obtain the signed scores.
MTB > sort c3 c13
MTB > name c13 'sorted'
MTB > print c13
sorted
-34.2 -34.0 -4.2 -3.8 -2.5 1.7 2.9 4.5 7.3 7.4 12.6
15.4
MTB > # The 2nd, 4th, 5th, 11th, and 12th scores need sign changes (because -2.5 is
MTB > # the 2nd smallest in magnitude, -3.8 is the 4th smallest in magnitude, -4.2
MTB > # is the 5th smallest in magnitude, and so on).
MTB > let c5(2) = -c5(2)
MTB > let c5(4) = -c5(4)
MTB > let c5(5) = -c5(5)
MTB > let c5(11) = -c5(11)
MTB > let c5(12) = -c5(12)
MTB > # Now I'll put the value of the test statistic in k1 and the p-value in k2.
MTB > let k1 = sum(c5)/sqrt( sum( c5*c5 ) )
MTB > name k1 'teststat' k2 'p-value'
MTB > print k1
teststat 0.172698
MTB > # Since the statistic is positive, need to double an upper-tail probability.
MTB > cdf k1 k2;
SUBC> norm 0 1.
MTB > let k2 = 2*(1 - k2)
MTB > print k2
p-value 0.862888
MTB > # Now I'll do the approximate version of the permutation test.
MTB > let k1 = sum(c3)/sqrt( sum( c3*c3 ) )
MTB > print k1
teststat -0.499564
MTB > # Since the statistic is negative, need to double a lower-tail probability.
MTB > cdf k1 k2;
SUBC> norm 0 1.
MTB > let k2 = 2*k2
MTB > print k2
p-value 0.617382
MTB > save 'ACDA'
Saving worksheet in file: ACDA.MTW
Summary of p-values from Minitab
________________________________
--------------------------------
test p-value
----------------- ---------
t test (exact*) 0.64
sign test (exact) 0.77
s-r test (approx) 0.72
n-s test (approx) 0.86
perm test (approx) 0.62
(Notice that the permutation test p-value is close to the t test p-value.)
* I labeled the t test as exact because the test was done in a manner which would
produce an exact p-value if the underlying dist'n was normal. But given that with
"real world" phenomena we never anticipate having data from a perfectly normal
distribution, the p-value which results from a t test should be viewed as being
approximate (and it should be kept in mind that with data from a skewed distribution
the t test can be very inaccurate).
-------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------
StatXact Information
To do these tests using StatXact, I can start by putting the 12 differences into the
first column (Var1) of the CaseData editor.
Sign Test
Use
Nonparametrics > Paired Samples > Sign...
When the Sign Test box comes up, if the little Use Differences box isn't already
checked, click to check it. Then click Var1 into the Population Difference box
(by making sure Var1 is highlighted in the Variables box and then clicking the arrow).
Finally, click the Exact button, and then click OK.
The exact 2-sided p-value is 0.7744 (matching Minitab's p-value).
The asymptotic p-value of 0.5637 comes from using a normal approximation without
a continuity correction.
(Note: All of the tests considered here are found under Paired Samples. This
is because the tests are often used with differences obtained from matched pairs.
In our case the data are due to matched pairs, but when using StatXact we're
starting from the differences instead of two columns of values. But all of the
tests can be used with a single sample of values from iid r.v's having a symmetric
distribution, and we can interpret the p-values as pertaining to tests about the
distribution median/mean. Except for the sign test, which is valid as a test about
the median even if the distribution underlying the data is skewed, when using these
tests as a test about the median/mean, the key thing is to feel comfortable with the
assumption of symmetry. With matched pairs data, we can always interpret a small
p-value as being evidence of some sort of a treatment difference.)
Signed-Rank Test
Use
Nonparametrics > Paired Samples > Wilcoxon Signed Rank...
Doing the same steps as are given for the Sign Test above, one obtains an exact
2-sided test p-value of 0.7334. This is in close agreement with what we get when
doubling the appropriate upper-tailed value from p. 579 of G&C. (One gets 0.734
from the table in G&C, instead of StatXact's 0.733. The G&C result is a bit off
because the tabulated values are rounded, and then we're doubling a rounded value
which will increase the rounding error.)
One can note that StatXact's asymptotic p-value is about 0.695. It results from the
standard normal approximation *without* using a continuity correction. Minitab's
p-value of 0.724 comes from using a normal approximation *with* a continuity correction.
(Note that Minitab's value is closer to the exact p-value than is StatXact's asymptotic
p-value.)
Permutation Test
Use
Nonparametrics > Paired Samples > Permutation...
Doing the same steps as are given for the Sign Test and Signed Rank Test above, one obtains
an exact p-value of 0.6221. The asymptotic p-value of 0.6174 matches what I obtained using
Minitab.
Normal Scores Test
Although the two-sample normal scores test can be found in StatXact's menus, the one-sample
(or paired-samples) version is not amongst the choices. However, we can use StatXact's
paired-samples permutation test routine to do the normal scores test.
Click on DataEditor to bring the CaseData editor to the forefront, and then type the appropriate
signed normal scores into Var2. (Note: You can perhaps compute them using some other software
and then paste them into a column of StatXact's CaseData editor. If you type in the values,
be sure to use at least 5 or 6 places after the decimal.)
Note: I don't think there is an easy way to do this using
DataEditor > Transform Variable...
because the choices of functions are just too limited. But here's a somewhat complex way to
compute the normal scores within StatXact.
(1) Use
DataEditor > Transform Variable...
Put Var4 (or another variable you haven't used yet) as the Target Variable.
Click on ABS(var) under Function to make it so
ABS(?)
appears in the big box to the right of the Target Variable box.
Delete the ? and with the cursor between ( and ) (after ABS), click on Var1 (or whatever
the variable containing the matched pairs differences is) under the Variables list to
now make it so
ABS(Var1)
is in the big box.
Click OK.
(2) Use
DataEditor > Compute Scores...
Put Var5 (or another variable you haven't used yet) in the Target Variable box.
Click Var4 (or whatever the variable containing the absolute values of the matched pairs
differences is) into the Response box.
Make sure the Wilcoxon (Mid-Rank) button is selected under the Score heading.
Click OK.
(3) Use
DataEditor > Transform Variable...
Put Var6 as the Target Variable.
Click on NORMINV(var) under Function to make it so
NORMINV(?)
appears in the big box to the right of the Target Variable box.
Replace the ? by (13 + Var5)/26 (or in general it would be
(n + 1 + var)/(2n + 2)
where n is the sample size and var is the variable containing the ranks of the absolute
values of the matched pairs differences).
*** Note: The values don't exactly match what I got using Minitab and R ... I think
StatXact's NORMINV function isn't highly accurate. So it'll be better for
you to produce the normal scores using some other software, and then paste
the values into StatXact.
(4) At this point the normal scores in Var6 are all positive. Add appropriate signs to them
by typing - signs in front of the normal score values in Var 6 of the CaseData editor and
your normal scores will now be ready to use with StatXact's paired-sample permutation test.
Because the procedure described above to create the appropriate signed normal scores within
StatXact is messy and the values that result from StatXact seem to be slightly in error, I
think it's best to create the desired scores outside of StatXact and paste them into a
column, say Var 2, of StatXact's CaseData editor. (Note: After I created the 12 normal
scores in Minitab, and then changed the signs on 5 of them, I could have written the
signed scores to a file, viewed the file, and then copied and pasted the signed scores into
StatXact's CaseData editor. Or you can use some other way to get the signed scores in there.)
(Note: The signed scores don't have to be in an order which matches the data values in Var1.
One just needs the correct set of scores, correctly signed.)
Once the signed score are in Var2, use
Nonparametrics > Paired Samples > Permutation...
and proceed like above, only this time click Var2 into the Population Difference box.
One gets an exact 2-sided test p-value of 0.8701. The asymptotic p-value of 0.8629
matches what I got with Minitab. (The asymptotic value is 0.8623 if the scores
produced using StatXact are used.) It's interesting to note that the approximate
p-values obtained with Minitab are pretty close to StatXact's exact p-values, even
though the sample size is only 12. (However, for smaller p-values, say close to 0.01,
the relative differences between exact and approximate p-values can be much more
appreciable.)
T Test
StatXact can even do a ordinary t test. But unlike the nonparametric tests used above,
for which the data could be two columns of matched pairs observations, or a single
column of the differences, for the t test StatXact needs two columns. If you already
have the differences in the CaseData editor, but not the original matched pairs
observations (or if you have a single sample and you want to do a test about the mean),
then you can "trick" StatXact by putting in a new column of values (say in Var3) which are
all 0 (or whatever the desired mu_0 value is). Then use
Basic_Statistics > t-test > Paired...
and click the appropriate two columns of values in for Population 1 and Population 2,
and then click OK. (The result should match what Minitab gave.)