PSY-304B: Quantitative Methods & Experimental Design

For all problems, unless otherwise indicated, assume a Type 1 error rate = .05.

1. A researcher is interested in assessing whether those individuals who respond to antidepressant medication differ in levels of the hormone prolactin. Two groups of depressed patients are formed from those who participated in a lengthy treatment study: responders and non-responders. The researcher also wants to assess whether a personality disorder (PD) can affect prolactin responses. For this reason, patients are also divided into those with a Personality Disorder and those without a Personality Disorder. Thus, the final design is a 2 (Responder/No Responder) × 2 (Personality Disorder/No Personality Disorder).

   In these problems the effects are named as:

   • dep_gp — the first effect (responder to the anti-depressant medication)
   • pd_gp — the second effect (having a personality disorder)
   • prl — the dependent variable of prolactin level

   The data can be loaded into R using the following commands:

   RespPers = c(34, 49, 38, 30)
   RespNoPers = c(23, 29, 19, 44, 22, 30, 27, 30, 29, 27, 34)
   NoRespPers = c(54, 49, 24, 25, 47, 50, 34, 39, 44)
   NoRespNoPers = c(40, 36, 39)
   prolactin.frame = data.frame(prolactin_level = c(RespPers, RespNoPers, NoRespPers, NoRespNoPers)
                                responder = c(rep(True, length(RespPers) + length(RespNoPers)),
                                              rep(False, length(NoRespPers) + length(NoRespNoPers))),
                                disorder = c(rep(True, length(RespPers)),
                                             rep(False, length(RespNoPers)),
                                             rep(True, length(NoRespPers)),
                                             rep(False, length(NoRespNoPers))))

   In the questions below, when cell or marginal sample sizes are involved in the answer (e.g. as contrast coefficients, terms in null hypotheses), answer using the actual n’s involved in the experiment rather than abstract terms like n_1,1.

   There are two tables that will be useful in answering these questions. The first is a frequency table that shows the number of elements in each cell:

   		Disorder
   		True	False
   Responder	True	4	11
   	False	9	3

   The second is the cell means for each of the factor levels:

   		Disorder
   		True	False
   Responder	True	37.75	28.55
   	False	40.66	38.33

   To deal with boolean conditions in this document the following notation will be used:

   $$\begin{array}{ccccc}\overline{\mathrm{Responder}}& =& \neg <mi>Responder</mi>& =& \text{The set of people who were unresponsive to the anti-depressant}\end{array}$$

   1. Consider the main effect for dep_gp. Express the null hypothesis for the weighted mean/Type 1 SS test of this effect. Your expression should be in terms of cell population means that are expressed symbolically (i.e., 'mu's). You can either express sample sizes symbolically (i.e., n's) or use the actual sample sizes in this study. Maxwell and Delaney's Table 7.18 (p. 328) exemplifies expressions of null hypotheses in terms of cell means.

      For the balanced ANOVA, the null hypothesis for the main effect of responder is a responsiveness to the antidepressant mediation has no effect on prolactin level.

      $$\begin{array}{ccccc}{H}_0& :& {\mu }_{\mathrm{Responder}}& =& {\mu }_{\overline{\mathrm{Responder}}}\end{array}$$

      For the unbalanced ANOVA using weighted cell means the null hypothesis is that responsiveness to the antidepressant medication had an effect whose proportion is a function of the number of subjects used in this particular experiment: (for brevity, Responser is abbreviated 'R' and Disorder 'D')

      $$\begin{array}{ccccc}{H}_0& :& \frac{n_{R,D}{\mu }_{R,D}+n_{R,\overline{D}}{\mu }_{R,\overline{D}}}{n_R}& =& \frac{n_{\overline{R},D}{\mu }_{\overline{R},D}+n_{\overline{R},\overline{D}}{\mu }_{\overline{R},\overline{D}}}{n_{\overline{R}}}\end{array}$$

      There are eight unknowns in this equation. There is no simple statement about the data that can be made through an examination of this statement other than another experiment will need the same proportional distribution of subjects in order to verify this experiment.

   2. Consider again the main effect for dep_gp. Express the null hypothesis for the unweighted mean/Type 3 SS test of this effect. Your expression should again be in terms of cell population means that are expressed symbolically.

      For the unbalanced ANOVA using an unweighted mean, the null hypothesis is the same as that as the balanced ANOVA: responsiveness to the antidepressant mediation has no effect on prolactin level.

      $$\begin{array}{ccccccccccccc}{H}_0& :& \frac{{\mu }_{R,D}+{\mu }_{R,\overline{D}}}{2}& =& \frac{{\mu }_{\overline{R},D}+{\mu }_{\overline{R},\overline{D}}}{2}& \iff & {\mu }_{R,D}+{\mu }_{R,\overline{D}}& =& {\mu }_{\overline{R},D}+{\mu }_{\overline{R},\overline{D}}& \iff & {\mu }_R& =& {\mu }_{\overline{R}}\end{array}$$

   3. Consider the main effect for pd_gp. Let's say that you wanted to test this hypothesis using weighted means. What is the contrast that tests this main effect? What we're looking for here is a linear combination of the cell means in this sample of data (i.e., the linear combination that constitutes ψ^ — show coefficient values and relevant sample means).

      $$\begin{array}{ccccc}{H}_0& :& \frac{n_{R,D}{\mu }_{R,D}+n_{\overline{R},D}{\mu }_{\overline{R},D}}{n_D}& =& \frac{n_{R,\overline{D}}{\mu }_{R,\overline{D}}+n_{\overline{R},\overline{D}}{\mu }_{\overline{R},\overline{D}}}{n_{\overline{D}}}\end{array}$$ $$\begin{array}{cccc}\iff & \frac{n_{R,D}}{n_D}{\mu }_{R,D}+\frac{n_{\overline{R},D}}{n_D}{\mu }_{\overline{R},D}-\left(\frac{n_{R,\overline{D}}}{n_{\overline{D}}}{\mu }_{R,\overline{D}}+\frac{n_{\overline{R},\overline{D}}}{n_{\overline{D}}}{\mu }_{\overline{R},\overline{D}}\right)& =& 0\end{array}$$ $$\begin{array}{cccc}\iff & \frac{n_{R,D}}{n_D}{\mu }_{R,D}+\frac{n_{\overline{R},D}}{n_D}{\mu }_{\overline{R},D}-\frac{n_{R,\overline{D}}}{n_{\overline{D}}}{\mu }_{R,\overline{D}}-\frac{n_{\overline{R},\overline{D}}}{n_{\overline{D}}}{\mu }_{\overline{R},\overline{D}}& =& 0\end{array}$$

      As a table these values are:

      		Disorder
      		True	False
      Responder	True	$$\frac{n_{R,D}}{n_D}$$	$$-\frac{n_{R,\overline{D}}}{n_{\overline{D}}}$$
      	False	$$\frac{n_{\overline{R},D}}{n_D}$$	$$-\frac{n_{\overline{R},\overline{D}}}{n_{\overline{D}}}$$

      For this particular dataset, those values are:

      		Disorder
      		True	False
      Responder	True	$\frac{4}{13}$	$-\frac{11}{14}$
      	False	$\frac{9}{13}$	$-\frac{3}{14}$

      Note that these are admissible coefficients, but that they specify a specific combination of cell means that is not generally applicable assuming that the populations being sampled are potentially infinite.

      $$\begin{array}{ccc}{}^I\hat{\psi }& \approxeq & 9.12\end{array}$$

   4. Consider again the main effect for pd_gp. Now let's say that you wanted to test this hypothesis using unweighted means. What is the contrast that tests this main effect?

      $$\begin{array}{ccccc}{H}_0& :& \frac{{\mu }_{R,D}+{\mu }_{\overline{R},D}}{2}& =& \frac{{\mu }_{R,\overline{D}}+{\mu }_{\overline{R},\overline{D}}}{2}\end{array}$$ $$\begin{array}{cccc}\iff & {\mu }_{R,D}+{\mu }_{\overline{R},D}-\left({\mu }_{R,\overline{D}}+{\mu }_{\overline{R},\overline{D}}\right)& =& 0\end{array}$$ $$\begin{array}{cccc}\iff & {\mu }_{R,D}+{\mu }_{\overline{R},D}-{\mu }_{R,\overline{D}}-{\mu }_{\overline{R},\overline{D}}& =& 0\end{array}$$

      		Disorder
      		True	False
      Responder	True	1	-1
      	False	1	-1

      $$\begin{array}{ccc}{}^{\mathrm{III}}\hat{\psi }& \approxeq & 11.53\end{array}$$

   5. For these data, express the weighted mean contrast for the main effect of dep_gp in terms of effect parameters. Discuss the implications of your results for interpretation of the results of tests of weighted marginal means.

      Consider the general formula for a particular cell in terms of effect parameters:

      • ${\mu }_{j,k}$ — the population mean of a given combination of factors
      • $\overline{{\mu }_{•,•}}$ — the mean of the entire population
      • ${\alpha }_j=\overline{{\mu }_{j,•}}-\overline{{\mu }_{•,•}}$ — the main effect for a particular level of factor j
      • ${\beta }_k=\overline{{\mu }_{•,k}}-\overline{{\mu }_{•,•}}$ — the main effect for a particular level of factor k
      • ${\left(\alpha \beta \right)}_{j,k}={\mu }_{j,k}-\left(\overline{{\mu }_{•,•}}+{\alpha }_j+{\beta }_k\right)$ — the interaction effect for a specific combination of the two factors j and k

      $$\begin{array}{ccc}{\mu }_{j,k}& =& \overline{{\mu }_{•,•}}+{\alpha }_j+{\beta }_k+{\left(\alpha \beta \right)}_{j,k}\end{array}$$

      When attempting to determine if there is a significant main effect the hypothesis being tested is:

      $$\begin{array}{ccccc}{\alpha }_i& =& {\alpha }_j& \forall & \left(i,j\right)\end{array}$$

      When examining dep_gp, this is equivalent to:

      $$\begin{array}{ccc}{\alpha }_R& =& {\alpha }_{\overline{R}}\\ {\mu }_{R,•}-{\mu }_{•,•}& =& {\mu }_{\overline{R},•}-{\mu }_{•,•}\\ {\mu }_{R,•}& =& {\mu }_{\overline{R},•}\end{array}$$

      To this point the comparison of weighted and unweighted means is the same. They diverge is the value for $\overline{{\mu }_{j,•}}$. For weighted means these values are:

      $$\begin{array}{ccccccc}\stackrel{\stackrel{w}{¯}}{{\mu }_{R,•}}& =& \frac{n_{R,D}{\mu }_{R,D}+n_{R,\overline{D}}{\mu }_{R,\overline{D}}}{n_R}& =& \frac{n_{\overline{R},D}{\mu }_{\overline{R},D}+n_{\overline{R},\overline{D}}{\mu }_{\overline{R},\overline{D}}}{n_{\overline{R}}}& =& \stackrel{\stackrel{w}{¯}}{{\mu }_{\overline{R},•}}\\ & & \frac{n_{R,D}}{n_R}{\mu }_{R,D}+\frac{n_{R,\overline{D}}}{n_R}{\mu }_{R,\overline{D}}& =& \frac{n_{\overline{R},D}}{n_{\overline{R}}}{\mu }_{\overline{R},D}+\frac{n_{\overline{R},\overline{D}}}{n_{\overline{R}}}{\mu }_{\overline{R},\overline{D}}& & \end{array}$$

      Going from means back to effect parameters complicates the equation:

      $$\begin{array}{cc}\multicolumn{2}{c}{\frac{n_{R,D}}{n_R}\left(\overline{{\mu }_{•,•}}+{\alpha }_R+{\beta }_D+{\left(\alpha \beta \right)}_{R,D}\right)+\frac{n_{R,\overline{D}}}{n_R}\left(\overline{{\mu }_{•,•}}+{\alpha }_R+{\beta }_{\overline{D}}+{\left(\alpha \beta \right)}_{R,\overline{D}}\right)}\\ =& \frac{n_{\overline{R},D}}{n_{\overline{R}}}\left(\overline{{\mu }_{•,•}}+{\alpha }_{\overline{R}}+{\beta }_D+{\left(\alpha \beta \right)}_{\overline{R},D}\right)+\frac{n_{\overline{R},\overline{D}}}{n_{\overline{R}}}\left(\overline{{\mu }_{•,•}}+{\alpha }_{\overline{R}}+{\beta }_{\overline{D}}+{\left(\alpha \beta \right)}_{\overline{R},\overline{D}}\right)\end{array}$$

      The grand mean will factor out of everything:

      $$\begin{array}{cc}\multicolumn{2}{c}{\frac{n_{R,D}}{n_R}\left({\alpha }_R+{\beta }_D+{\left(\alpha \beta \right)}_{R,D}\right)+\frac{n_{R,\overline{D}}}{n_R}\left({\alpha }_R+{\beta }_{\overline{D}}+{\left(\alpha \beta \right)}_{R,\overline{D}}\right)}\\ =& \frac{n_{\overline{R},D}}{n_{\overline{R}}}\left({\alpha }_{\overline{R}}+{\beta }_D+{\left(\alpha \beta \right)}_{\overline{R},D}\right)+\frac{n_{\overline{R},\overline{D}}}{n_{\overline{R}}}\left({\alpha }_{\overline{R}}+{\beta }_{\overline{D}}+{\left(\alpha \beta \right)}_{\overline{R},\overline{D}}\right)\end{array}$$

      The weights for α_j will combine:

      $$\begin{array}{cc}\multicolumn{2}{c}{\left(\frac{n_{R,D}}{n_R}+\frac{n_{R,\overline{D}}}{n_R}\right){\alpha }_R+\frac{n_{R,D}}{n_R}\left({\beta }_D+{\left(\alpha \beta \right)}_{R,D}\right)+\frac{n_{R,\overline{D}}}{n_R}\left({\beta }_{\overline{D}}+{\left(\alpha \beta \right)}_{R,\overline{D}}\right)}\\ =& \left(\frac{n_{\overline{R},D}}{n_{\overline{R}}}+\frac{n_{\overline{R},\overline{D}}}{n_{\overline{R}}}\right){\alpha }_{\overline{R}}+\frac{n_{\overline{R},D}}{n_{\overline{R}}}\left({\beta }_D+{\left(\alpha \beta \right)}_{\overline{R},D}\right)+\frac{n_{\overline{R},\overline{D}}}{n_{\overline{R}}}\left({\beta }_{\overline{D}}+{\left(\alpha \beta \right)}_{\overline{R},\overline{D}}\right)\end{array}$$ $$\begin{array}{ccc}{\alpha }_R+\frac{n_{R,D}}{n_R}\left({\beta }_D+{\left(\alpha \beta \right)}_{R,D}\right)+\frac{n_{R,\overline{D}}}{n_R}\left({\beta }_{\overline{D}}+{\left(\alpha \beta \right)}_{R,\overline{D}}\right)& =& {\alpha }_{\overline{R}}+\frac{n_{\overline{R},D}}{n_{\overline{R}}}\left({\beta }_D+{\left(\alpha \beta \right)}_{\overline{R},D}\right)+\frac{n_{\overline{R},\overline{D}}}{n_{\overline{R}}}\left({\beta }_{\overline{D}}+{\left(\alpha \beta \right)}_{\overline{R},\overline{D}}\right)\end{array}$$

      So, for this data, the null hypothesis is:

      $$\begin{array}{ccc}{\alpha }_R-{\alpha }_{\overline{R}}& =& \frac{n_{R,D}}{n_R}\left({\beta }_D+{\left(\alpha \beta \right)}_{R,D}\right)+\frac{n_{R,\overline{D}}}{n_R}\left({\beta }_{\overline{D}}+{\left(\alpha \beta \right)}_{R,\overline{D}}\right)-\frac{n_{\overline{R},D}}{n_{\overline{R}}}\left({\beta }_D+{\left(\alpha \beta \right)}_{\overline{R},D}\right)-\frac{n_{\overline{R},\overline{D}}}{n_{\overline{R}}}\left({\beta }_{\overline{D}}+{\left(\alpha \beta \right)}_{\overline{R},\overline{D}}\right)\\ & =& \frac{4}{15}\left({\beta }_D+{\left(\alpha \beta \right)}_{R,D}\right)+\frac{11}{15}\left({\beta }_{\overline{D}}+{\left(\alpha \beta \right)}_{R,\overline{D}}\right)-\frac{9}{12}\left({\beta }_D+{\left(\alpha \beta \right)}_{\overline{R},D}\right)-\frac{3}{12}\left({\beta }_{\overline{D}}+{\left(\alpha \beta \right)}_{\overline{R},\overline{D}}\right)\end{array}$$

   6. For these data, express the unweighted mean contrast for the main effect of dep_gp in terms of effect parameters. Discuss the implications of your results for interpretation of the results of tests of unweighted marginal means.

      The initial logic is identical to the problem above. It deviates in the meaning of $\overline{{\mu }_{j,•}}$. For unweighted means these values are:

      $$\begin{array}{ccccccc}\stackrel{\stackrel{u}{¯}}{{\mu }_{R,•}}& =& \frac{{\mu }_{R,D}+{\mu }_{R,\overline{D}}}{2}& =& \frac{{\mu }_{\overline{R},D}+{\mu }_{\overline{R},\overline{D}}}{2}& =& \stackrel{\stackrel{u}{¯}}{{\mu }_{\overline{R},•}}\\ \multicolumn{3}{c}{{\mu }_{R,D}+{\mu }_{R,\overline{D}}}& =& \multicolumn{3}{c}{{\mu }_{\overline{R},D}+{\mu }_{\overline{R},\overline{D}}}\\ \multicolumn{3}{c}{\left(\overline{{\mu }_{•,•}}+{\alpha }_R+{\beta }_D+{\left(\alpha \beta \right)}_{R,D}\right)+\left(\overline{{\mu }_{•,•}}+{\alpha }_R+{\beta }_{\overline{D}}+{\left(\alpha \beta \right)}_{R,\overline{D}}\right)}& =& \multicolumn{3}{c}{\left(\overline{{\mu }_{•,•}}+{\alpha }_{\overline{R}}+{\beta }_D+{\left(\alpha \beta \right)}_{\overline{R},D}\right)+\left(\overline{{\mu }_{•,•}}+{\alpha }_{\overline{R}}+{\beta }_{\overline{D}}+{\left(\alpha \beta \right)}_{\overline{R},\overline{D}}\right)}\\ \multicolumn{3}{c}{\left({\alpha }_R+{\beta }_D+{\left(\alpha \beta \right)}_{R,D}\right)+\left({\alpha }_R+{\beta }_{\overline{D}}+{\left(\alpha \beta \right)}_{R,\overline{D}}\right)}& =& \multicolumn{3}{c}{\left({\alpha }_{\overline{R}}+{\beta }_D+{\left(\alpha \beta \right)}_{\overline{R},D}\right)+\left({\alpha }_{\overline{R}}+{\beta }_{\overline{D}}+{\left(\alpha \beta \right)}_{\overline{R},\overline{D}}\right)}\\ \multicolumn{3}{c}{2{\alpha }_R+{\beta }_D+{\beta }_{\overline{D}}+\left({\left(\alpha \beta \right)}_{R,D}+{\left(\alpha \beta \right)}_{R,\overline{D}}\right)}& =& \multicolumn{3}{c}{2{\alpha }_{\overline{R}}+{\beta }_D+{\beta }_{\overline{D}}+\left({\left(\alpha \beta \right)}_{\overline{R},D}+{\left(\alpha \beta \right)}_{\overline{R},\overline{D}}\right)}\\ \multicolumn{3}{c}{2{\alpha }_R+\left({\left(\alpha \beta \right)}_{R,D}+{\left(\alpha \beta \right)}_{R,\overline{D}}\right)}& =& \multicolumn{3}{c}{2{\alpha }_{\overline{R}}+\left({\left(\alpha \beta \right)}_{\overline{R},D}+{\left(\alpha \beta \right)}_{\overline{R},\overline{D}}\right)}\end{array}$$

      Recall that the interaction coefficients sum to zero along each row and column:

      $$\begin{array}{ccc}2{\alpha }_R+\left(0\right)& =& 2{\alpha }_{\overline{R}}+\left(0\right)\\ {\alpha }_R& =& {\alpha }_{\overline{R}}\end{array}$$

      So, when using unweighted means (from the type III sum of squares), the null hypothesis tested is the same as for balanced ANOVA.

   7. Using R, conduct a 2-way ANOVA of these data that generates both the Type 1 (weighted mean) and Type 3 (unweighted mean) SS solutions. Write the code in such a way that the dep_gp factor is the first one tested by the Type 1 SS.

      > # Type I Sum of Squares
      > prolactin.aov = aov(prolactin_level ~ responder * disorder,
      +                     data = prolactin.frame)
      > summary(prolactin.aov)
      
                         Df  Sum Sq Mean Sq F value  Pr(>F)
      responder           1  550.05  550.05  7.8669 0.01006
      disorder            1  200.66  200.66  2.8698 0.10376
      responder:disorder  1   60.12   60.12  0.8598 0.36342
      Residuals          23 1608.14   69.92
      
      > # Type III Sum of Squares
      > drop1(prolactin.aov, .~., test = "F")
      
      Single term deletions
      
      Model:
      prolactin_level ~ responder * disorder
                         Df Sum of Sq     RSS     AIC F value   Pr(F)
      <none>                          1608.14  118.35
      responder           1     23.56 1631.70  116.74  0.3369 0.56725
      disorder            1    248.52 1856.67  120.23  3.5544 0.07208
      responder:disorder  1     60.12 1668.26  117.34  0.8598 0.36342

   8. Consider the results for the main effect for dep_gp yielded by the Type 1 and Type 3 omnibus analyses. Explain why these two sets of results lead to discrepant conclusions about the effects of dep_gp.

      There is a significant main effect for reponse to the antidepressant when using the type I sum of squares, but not for the type III sum of squares.

      The issue is that the cells with the maxima and minima, $\overline{Y_{•,\overline{R},D}}$ and $\overline{Y_{•,R,\overline{D}}}$, have at nearly three times the number of subjects that the cells with median values. Because each level of the main effect for response contains one of these cells the weighted means are pulled away from each other. This has the ultimate effect of inflating the F value.

      It is important to note that the statistical result isn't incorrect. It represents a valid and interpretable result, but that result is not one that can be easily abstracted to make generalizable statements about the overall population.

      The type III sum of squares tests a hypothesis that is more generally interesting, and it finds no statistically significant trends in the data.