PSY-304B: Quantitative Methods & Experimental Design

For all problems, unless otherwise indicated, assume a Type 1 error rate = .05.

1. An experiment is conducted that compares the effects of three different levels of stress on cortisol secretion: high, medium, and low stress. Below are the data:

   High Stress	77	63	80	45	86	72
   Medium Stress	34	72	60	44	39	48
   Low Stress	51	44	29	48	33	32

   1. Using R, conduct a one-way ANOVA on these data.

      The R code is in homework_2.R to produce:

      Response: Cortisol_Secretion
                   Df  Sum Sq Mean Sq F value   Pr(>F)   
      Stress_Level  2 3004.00 1502.00    8.96 0.002753 **
      Residuals    15 2514.50  167.63

      To double check, the SAS program homework_2.sas gives similar output:

                             Sum of
       Source           DF  Squares  Mean Square  F Value  Pr > F
       Model             2  3004.00  1502.000000     8.96  0.0028
       Error            15  2514.50   167.633333
       Corrected Total  17  5518.50

   2. Indicate whether you reject the null hypothesis.

      $$\begin{array}{ccc}{}_{0.00275}F_{2,15}& \approxeq & 8.96\end{array}$$

      There is only a 0.275% chance that this data came from a central F-distribution. This is less than the 5% cutoff required to reject, so H₀ is rejected.

   3. Compute an estimate of the population value for Cohen's f.

      $$\begin{array}{ccccccc}f& =& \frac{{\sigma }_{\mu }}{{\sigma }_y}& =& \frac{\sqrt{\frac{\sum {\left({\mu }_j-\stackrel{\_}{\mu }\right)}^2}{J}}}{{\sigma }_y}& =& \sqrt{\frac{\sum {\left({\mu }_j-\stackrel{\_}{\mu }\right)}^2}{J{\sigma }_y^2}}\end{array}$$

      Cohen's magnitudes for effect size are:

      Size	Cohen's d	Cohen's f
      Small	0.2	0.1
      Medium	0.5	0.25
      Large	0.8	0.4

      No unbiased estimator of f is known. An only slightly biased estimator for equal sized groups is:

      The R code in homework_2.R produces $\hat{f}\approxeq 0.94045$ which is large.

      Another suggested slightly biased estimator is:

      $$\begin{array}{ccc}\hat{f}& =& \sqrt{\frac{{\tilde{R}}^2}{1-{\tilde{R}}^2}}\end{array}$$

      The R code in homework_2.R produces $\hat{f}\approxeq 0.96772$.

   4. Is the R² value listed in the SAS output an optimal measure of the proportion of variance in cortisol values accounted for by stress? Explain.

      This proportion varies between 0 (all variability is within groups) and 1 (all variability is between groups).

      Typically, R² will overestimate r². The amount of this bias is proportional to the ratio between the degrees of freedom. A "shrunken" value can take this into account and will give a less biased estimate:

      R gives both values:

      Multiple R.Squared: 0.5444, Adjusted R.squared: 0.4836

      SAS gives just R²:

      R-Square     Coeff Var      Root MSE    cortisol Mean
      0.544351      24.35234      12.94733         53.16667

   5. Compute two alternative measures of proportion of variance.

      One measure is ${\tilde{R}}^2$ listed above. The other is ${\omega }^2$:

      $$\begin{array}{ccc}{\sigma }_Y^2& =& {\sigma }_{\alpha }^2+{\sigma }_{\epsilon }^2\\ {\sigma }_{\alpha }^2& =& \frac{\sum _j{\left({\mu }_j-\mu \right)}^2}{J}& =& \frac{\sum _j{\alpha }_j^2}{J}\\ {\sigma }_{\epsilon }^2=& \frac{\sum _{i,j}{\left(Y_{i,j}-{\mu }_j\right)}^2}{N}\end{array}$$

      The R code in homework_2.R produces ${\hat{\omega }}^2\approxeq 0.46934$ which is less than ${\tilde{R}}^2$.

2. Suppose that you are planning a study to compare the effects of four recall-cue conditions (denoted A, B, C, and D) on memory for spatial relations. The primary dependent measure will be the number of spatial relations accurately recalled.

   1. You assume that the within-population standard deviations all equal 9. You set the Type 1 error rate at α = .05. You presume that the population means will have the following values: u_A = 17.5, u_B = 19, u_C = 25, and u_D = 20.5. You intend to run 80 subjects in all, with equal n's across all 4 groups. You plan on conducting a one-way ANOVA. Using GLMPOWER, compute your power to reject the null hypothesis under these conditions.

      From R using power.anova.test, I get:

      Balanced one-way analysis of variance power calculation
           groups = 4
                n = 20
      between.var = 10.5
       within.var = 81
        sig.level = 0.05
            power = 0.615523

      From SAS, I get:

      Power
      0.616

   2. You have the same Type 1 error rate and make the same assumptions about the population standard deviation and the population means as in part a. You still have 80 subjects in all but now you want to know how power might change by running 10 subjects in groups A, B, and D and 50 subjects in group C. Using GLMPOWER, determine the power under this subject allocation scheme. How would it compare to the power under the scheme noted in part a? Why do you think your power differs in the two cases?

      From SAS, I get:

      Power
      0.680

      I don't know how to do this calculation automatically in R. So, calculating the answer requires returning to the theoretical basis.

      The conceptual structure of the F-distribution is built as a combination of other distributions:

      1. The central limit theorem says that any population meeting certain broad criteria, if randomly sampled for long enough will see the distribution of sample values converge on a normal distribution modeled by the probability distribution function: $$\begin{array}{ccc}N\left(x;\mu ,\sigma \right)& =& \frac{1}{\sqrt{2\pi }\sigma }{e}^{-\frac{1}{2}{\left(\frac{x-\mu }{\sigma }\right)}^2}\end{array}$$
      2. If those random samples are divided by the deviation and shifted by the mean, they will become z-scores: $$\begin{array}{ccc}z& =& \frac{x-\mu }{\sigma }\end{array}$$ These scores will be distributed as: $$\begin{array}{ccc}z& ~& N\left(\mu =0,\sigma =1\right)\end{array}$$
      3. If the values of these z-scores are then squared, they will be distributed as a ${\chi }^2$-distribution. Each independent sample that is taken adds a degree of freedom, $r$.
      4. The F-distribution is two ${\chi }^2$-distributions each divided by their degrees of freedom: $$\begin{array}{ccc}\frac{\frac{{\chi }_{r_1}^2}{r_1}}{\frac{{\chi }_{r_2}^2}{r_2}}& =& F_{r_1,r_2}\end{array}$$
      5. The ${\chi }^2$-distributions that form the basis for this F-distribution are known as "central" distributions because the mean for all the independent samples is 0. Consider instead a ${\chi }^2$-distribution which samples from $N\left({\mu }_i,\sigma =1\right)$ distributions. This is known as a non-central ${\chi }^2$ (unless ${\mu }_i=0\forall i$), and the distribution is modified by an additional parameter, $\lambda$. $$\begin{array}{ccc}\lambda & =& \sum {\left(\frac{{\mu }_i}{{\sigma }_i}\right)}^2\end{array}$$ The distribution is then written with two parameters: $${\chi }_{r,\gamma }^2$$
      6. A non-central F-distribution is the combination of a non-central and central ${\chi }^2$: $$\begin{array}{ccc}\frac{\frac{{\chi }_{r_1,{\gamma }_1}^2}{r_1}}{\frac{{\chi }_{r_2,{\gamma }_2=0}^2}{r_2}}& =& F_{r_1,r_2,\gamma }\end{array}$$ $$\begin{array}{ccc}\gamma & =& \frac{\sum n_j{\left({\mu }_j-\stackrel{\_}{\mu }\right)}^2}{{\sigma }_Y^2}\end{array}$$
      7. In general, the ratio of the Mean Square Between and Mean Square Within will have a non-central F-distribution. If all the means are actually equal then γ = 0, otherwise the distribution is shifted to the right. Power in this situation is computed by finding the 95% cutoff point in a central F and finding the probability that the non-central F will produce that value.

      My R code comes up with the same value as SAS: 0.680. Power increases slightly because the non-centrality parameter increased from 7.778 to 8.889. It is important to recall that power is the likelihood of a correct rejection. As the real distribution moves to the right (with an increase in λ), it increases the likelihood that the sampled value will be greater than the critical value for the central F.

   3. Again, you assume that the within-population standard deviations all equal 9. You set the Type 1 error rate at α = .05. You are assuming that you want to conduct the study with equal sample sizes in the groups. You do not think that you know enough to be able to accurately estimate the specific values of the population means. However, you do know that you want to have at least 80% power to detect a medium effect size (f = .25) Using GLMPOWER, determine what the group sample sizes should be.

      HINT: It really doesn't matter what the precise pattern of 4 population means is, as long as it results in an f = .25. So, you might consider:

      1. setting the population standard deviation at 9
      2. setting the grand mean to a reasonable value (say 20.5 based on the values noted in part a)
      3. setting two of the group means equal to the value of the grand mean
      4. then figuring out the value of the remaining two means which would result in f = .25. You can do your computations figuring out appropriate values of the means outside SAS if you want. Just let me know somehow what those values are (they will have to show up in your SAS program somewhere).

      R uses the between and within group variances to compute power rather than a set of means. This means that the issue is finding a relationship between those characteristics rather than coming up with an appropriate set of means. Cohen's f is simply the ratio of the standard deviation between populations to the deviation within with populations. This means that given f and σ_y:

      $$\begin{array}{ccccccccccc}f& =& \frac{{\sigma }_{\mu }}{{\sigma }_y}& \Rightarrow & {\sigma }_{\mu }& =& f{\sigma }_y& \Rightarrow & {\sigma }_{\mu }^2& =& {\left(f{\sigma }_y\right)}^2\end{array}$$

      R computes:

      Balanced one.way analysis of variance power calculation
           groups = 4
                n = 59.13301
      between.var = 5.0625
       within.var = 81
        sig.level = 0.05
            power = 0.8

3. Let's say that a researcher conducts an experiment assessing whether males and females differ in verbal learning. Subjects are forced to perform a demanding secondary task when being exposed to verbal stimuli. Then number of verbal stimuli recalled is assessed. 40 married couples are randomly sampled (leading to 40 male and 40 female subjects and 80 subjects in all). The experimenter analyzes the data as a between-groups t test comparing the male and female groups. She finds that t(76) = 1.83, p < .07, and concludes that there is not sufficient evidence to reject the null hypothesis. Comment.

   One of the assumptions of the t-test is the independence of sample populations. Husbands and wives are very likely positively correlated in verbal capacity. The violation of the assumption of independence reduces the power of the test. The experimenter should choose a test statistic that takes into account the correlation such as a paired t-test.

4. Two effect size measures that we have talked about are Cohen's d for the two group t-test situation and Cohen's f for the one-way ANOVA. The formulas for these measures are as follows:

   $$\begin{array}{ccc}d& =& \frac{{\mu }_1-{\mu }_2}{{\sigma }_y}\end{array}$$ $$\begin{array}{ccc}f& =& \frac{\sqrt{\frac{\sum {\left({\mu }_j-{\mu }_{••}\right)}^2}{J}}}{{\sigma }_y}\end{array}$$

   It is well known that if you have two groups in the context of a between-groups design, you can do either a t-test or a one-way ANOVA. Correspondingly, one would expect that there would be a predictable mathematical relation between Cohen's d and f in the case of a two-group design. In fact, consideration of Cohen's recommendations for small, medium, and large effect sizes indicates such a relation. He has described small, medium, and large effect size d values of .2, .5, and .8 respectively, and f values of .1, .25 and .4, respectively. Using the formulas for d and f shown above, algebraically derive the relation between d and f.

   $$\begin{array}{ccccccc}d& =& \frac{{\mu }_1-{\mu }_2}{{\sigma }_y}& \iff & {\sigma }_y& =& \frac{{\mu }_1-{\mu }_2}{d}\end{array}$$ $$\begin{array}{ccccccc}{\left.f\right|}_{J=2}& =& \frac{\sqrt{\frac{\sum _{j=1}^2{\left({\mu }_j-\stackrel{\_}{\mu }\right)}^2}{2}}}{{\sigma }_y}& =& \frac{\sqrt{\frac{{\left({\mu }_1-\frac{{\mu }_1+{\mu }_2}{2}\right)}^2+{\left({\mu }_2-\frac{{\mu }_1+{\mu }_2}{2}\right)}^2}{2}}}{{\sigma }_y}& =& \frac{\sqrt{\frac{{\left(\frac{2{\mu }_1-{\mu }_1-{\mu }_2}{2}\right)}^2+{\left(\frac{2{\mu }_2-{\mu }_1-{\mu }_2}{2}\right)}^2}{2}}}{{\sigma }_y}\\ & =& \frac{\sqrt{\frac{{\left(\frac{{\mu }_1-{\mu }_2}{2}\right)}^2+{\left(\frac{{\mu }_2-{\mu }_1}{2}\right)}^2}{2}}}{{\sigma }_y}& =& \frac{\sqrt{\frac{2{\left({\mu }_1-{\mu }_2\right)}^2}{8}}}{{\sigma }_y}& =& \frac{{\mu }_1-{\mu }_2}{2{\sigma }_y}\\ & =& \frac{{\mu }_1-{\mu }_2}{2\left(\frac{{\mu }_1-{\mu }_2}{d}\right)}& =& \frac{d}{2}\end{array}$$