9 Within-Subjects t-test

within-subjects t-test: Compare two conditions where the participants are the same in both conditions (or more rarely are different participants that have been highly matched on a number of demographics such as IQ, reading ability, etc - must be matched on a number of demographics).

9.1 The Worked Example

Let's say that this is our starting data:

Participants	PreTest	PostTest
1	60	68
2	64	75
3	56	62
4	82	85
5	74	73
6	79	85
7	63	64
8	59	59
9	72	73
10	66	70

The first thing we need to do is calculate the difference between the PostTest and the PreTest for each participant, based on $D = PostTest - PreTest$. So for example:

• Participant 1 would be: 68 - 60 = 8
• Participant 2 would be: 75 - 64 = 11
• etc

And if we do that for each Participant and added a column of the differences ($D$) then we would see:

Participants	PreTest	PostTest	D
1	60	68	8
2	64	75	11
3	56	62	6
4	82	85	3
5	74	73	-1
6	79	85	6
7	63	64	1
8	59	59	0
9	72	73	1
10	66	70	4

Now, the within-subjects t-test formula is:

$$t = \frac{\bar{D}}{\frac{SD_{D}}{\sqrt{N}}}$$

We can see that $N = 10$, but we need to calculate $\bar{D}$ (called D-Bar, the mean of the $D$ column) and $SD_{D}$.

Calculating D-bar

So the $\bar{D}$ formula is the same as the mean formula:

$$\bar{D} = \frac{\sum{D}}{N}$$

Where $D$ is $PostTest - PreTest$ for each Participant.

Then:

$$\bar{D} = \frac{(68 - 60) + (75 - 64) + (62 - 56) + (85 - 82) + (73 - 74) + \\ (85- 79) + (64 - 63) + (59 - 59) + (73 - 72) + (70 - 66)}{10}$$

Which if we resolve all the brackets becomes:

$$\bar{D} = \frac{8 + 11 + 6 + 3 + -1 + 6 + 1 + 0 + 1 + 4}{10}$$

And if we sum the top half together

$$\bar{D} = \frac{39}{10}$$

Leaving us with:

$$\bar{D} = 3.9$$

So we find that $\bar{D}$ = 3.9, which is the mean difference between the Post test and Pre test values.

The Standard Deviation of D

The standard deviation formula is:

$$SD = \sqrt\frac{\sum(X - \bar{X})^2}{N-1}$$

Which if we translate to using D, becomes:

$$SD_{D} = \sqrt\frac{\sum(D - \bar{D})^2}{N-1}$$

Then:

$$SD_{D} =\sqrt\frac{(8 - 3.9)^2 + (11 - 3.9)^2 + (6 - 3.9)^2 + (3 - 3.9)^2 + (-1 - 3.9)^2 + \\ (6 - 3.9)^2 + (1 - 3.9)^2 + (0 - 3.9)^2 + (1 - 3.9)^2 + (4 - 3.9)^2}{10 - 1}$$

And if we start stepping through this analysis by dealing with the brackets:

$$SD_{D} =\sqrt\frac{(4.1)^2 + (7.1)^2 + (2.1)^2 + (-0.9)^2 + (-4.9)^2 + \\ (2.1)^2 + (-2.9)^2 + (-3.9)^2 + (-2.9)^2 + (0.1)^2}{10 - 1}$$

And then we square those brackets

$$SD_{D} =\sqrt\frac{16.81 + 50.41 + 4.41 + 0.81 + 24.01 + 4.41 + 8.41 + 15.21 + 8.41 + 0.01}{10 - 1}$$

And sum up all the values on the top half:

$$SD_{D} =\sqrt\frac{132.9}{10 - 1}$$

And then we need to sort out the bottom half

$$SD_{D} =\sqrt\frac{132.9}{9}$$

Which by dividing the top half by the bottom half reduces down to:

$$SD_{D} =\sqrt{14.7666667}$$

And then we finally take the square root, which leaves us with:

$$SD_{D} =3.8427421$$

And so we find that the $SD_{D}$ = 3.8427421 which to two decimal places would be, $SD_{D} = 3.84$

Calculating the t-value

And finally the t-test formula is:

$$t = \frac{\bar{D}}{\frac{SD_{D}}{\sqrt{N}}}$$

Then if we start filling in the values we know from above we see:

$$t = \frac{3.9}{\frac{3.8427421}{\sqrt{10}}}$$

And if we deal with the square root first:

$$t = \frac{3.9}{\frac{3.8427421}{3.1622777}}$$

And divide $SD_{D}$ by $\sqrt{N}$ - tidying up the bottom of the formula:

$$t = \frac{3.9}{1.2151817}$$

And then solve for $t$ by dividing the top half by the bottom half gives us:

$$t = 3.2093965$$

And so, rounding to two decimal places, we find that $t = 3.21$

Degrees of Freedom

Great! Now we just need the degrees of freedom where the formula is:

$$df = N - 1$$

And we already know that $N= 10$ and putting them into the equation we get:

$$df = 10 - 1$$

Which reduces to:

$$df = 9$$

Meaning that we find a $df = 9$

Effect Size - Cohen's d

And finally Cohen's d, the effect size. One of the common formulas based on knowing the t-value and the N is:

$$d = \frac{t}{\sqrt{N}}$$

Which, based on the info above, we know:

• $t = 3.21$
• $N = 10$

And putting those into the formula we get:

$$d = \frac{3.21}{\sqrt{10}}$$

Which gives us:

$$d = \frac{3.21}{3.1622777}$$

And so:

$$d = 1.0150911$$

Meaning that the effect size, to two decimal places, is d = 1.01.

Determining Significance

If we were to look at a critical values look-up table for $df = 9$ and $\alpha = .05$ (two-tailed), we would see that the critical value is $t_{crit} = 2.262$. Given that our t-value, ignoring polarity and just looking at the absolute value, so $t = 3.21$, is equal to or larger than our $t_{crit}$ then we can say our result is significant, and as such would be written up as t(9) = 3.21, p < .05, d = 1.01.

Remember: If you were writing this up as a report, and analysis the data in R, then you would see the p-value was actually p = 0.011, and would be written up as p = 0.011

9.2 Test Yourself

9.2.1 DataSet 1

Let's say that this is our starting data:

Participants	PreTest	PostTest
1	71	64
2	56	54
3	75	70
4	50	46
5	51	52
6	61	58
7	62	61
8	72	74
9	66	60
10	78	78

The first thing we need to do is calculate the difference between the PostTest and the PreTest for each participant, based on $D = PostTest - PreTest$. So for example:

• Participant 1 would be: 64 - 71 = -7
• Participant 2 would be: 54 - 56 = -2
• etc

And if we do that for each Participant and added a column of the differences ($D$) then we would see:

Participants	PreTest	PostTest	D
1	71	64	-7
2	56	54	-2
3	75	70	-5
4	50	46	-4
5	51	52	1
6	61	58	-3
7	62	61	-1
8	72	74	2
9	66	60	-6
10	78	78	0

Now, the within-subjects t-test formula is:

$$t = \frac{\bar{D}}{\frac{SD_{D}}{\sqrt{N}}}$$

We can see that $N = 10$, but we need to calculate $\bar{D}$ (called D-Bar, the mean of the $D$ column) and $SD_{D}$.

Calculating D-bar

So the $\bar{D}$ formula is the same as the mean formula:

$$\bar{D} = \frac{\sum{D}}{N}$$

Where $D$ is $PostTest - PreTest$ for each Participant.

Then:

$$\bar{D} = \frac{(64 - 71) + (54 - 56) + (70 - 75) + (46 - 50) + (52 - 51) + \\ (58- 61) + (61 - 62) + (74 - 72) + (60 - 66) + (78 - 78)}{10}$$

Which if we resolve all the brackets becomes:

$$\bar{D} = \frac{-7 + -2 + -5 + -4 + 1 + -3 + -1 + 2 + -6 + 0}{10}$$

And if we sum the top half together

$$\bar{D} = \frac{-25}{10}$$

Leaving us with:

$$\bar{D} = -2.5$$

So we find that $\bar{D}$ = -2.5, which is the mean difference between the Post test and Pre test values.

The Standard Deviation of D

The standard deviation formula is:

$$SD = \sqrt\frac{\sum(X - \bar{X})^2}{N-1}$$

Which if we translate to using D, becomes:

$$SD_{D} = \sqrt\frac{\sum(D - \bar{D})^2}{N-1}$$

Then:

$$SD_{D} =\sqrt\frac{(-7 - -2.5)^2 + (-2 - -2.5)^2 + (-5 - -2.5)^2 + (-4 - -2.5)^2 + \\ (1 - -2.5)^2 + (-3 - -2.5)^2 + (-1 - -2.5)^2 + (2 - -2.5)^2 + \\ (-6 - -2.5)^2 + (0 - -2.5)^2}{10 - 1}$$

And if we start stepping through this analysis by dealing with the brackets:

$$SD_{D} =\sqrt\frac{(-4.5)^2 + (0.5)^2 + (-2.5)^2 + (-1.5)^2 + (3.5)^2 + \\ (-0.5)^2 + (1.5)^2 + (4.5)^2 + (-3.5)^2 + (2.5)^2}{10 - 1}$$

And then we square those brackets

$$SD_{D} =\sqrt\frac{20.25 + 0.25 + 6.25 + 2.25 + 12.25 + \\ 0.25 + 2.25 + 20.25 + 12.25 + 6.25}{10 - 1}$$

And sum up all the values on the top half:

$$SD_{D} =\sqrt\frac{82.5}{10 - 1}$$

And then we need to sort out the bottom half

$$SD_{D} =\sqrt\frac{82.5}{9}$$

Which by dividing the top half by the bottom half reduces down to:

$$SD_{D} =\sqrt{9.1666667}$$

And then we finally take the square root, which leaves us with:

$$SD_{D} =3.0276504$$

And so we find that the $SD_{D}$ = 3.0276504 which to two decimal places would be, $SD_{D} = 3.03$

Calculating the t-value

And finally the t-test formula is:

$$t = \frac{\bar{D}}{\frac{SD_{D}}{\sqrt{N}}}$$

Then if we start filling in the values we know from above we see:

$$t = \frac{-2.5}{\frac{3.0276504}{\sqrt{10}}}$$

And if we deal with the square root first:

$$t = \frac{-2.5}{\frac{3.0276504}{3.1622777}}$$

And divide $SD_{D}$ by $\sqrt{N}$ - tidying up the bottom of the formula:

$$t = \frac{-2.5}{0.9574271}$$

And then solve for $t$ by dividing the top half by the bottom half gives us:

$$t = -2.6111648$$

And so, rounding to two decimal places, we find that $t = -2.61$

Degrees of Freedom

Great! Now we just need the degrees of freedom where the formula is:

$$df = N - 1$$

And we already know that $N= 10$ and putting them into the equation we get:

$$df = 10 - 1$$

Which reduces to:

$$df = 9$$

Meaning that we find a $df = 9$

Effect Size - Cohen's d

And finally Cohen's d, the effect size. One of the common formulas based on knowing the t-value and the N is:

$$d = \frac{t}{\sqrt{N}}$$

Which, based on the info above, we know:

• $t = -2.61$
• $N = 10$

And putting those into the formula we get:

$$d = \frac{-2.61}{\sqrt{10}}$$

Which gives us:

$$d = \frac{-2.61}{3.1622777}$$

And so:

$$d = -0.8253545$$

Meaning that the effect size, to two decimal places, is d = -0.83.

Determining Significance

If we were to look at a critical values look-up table for $df = 9$ and $\alpha = .05$ (two-tailed), we would see that the critical value is $t_{crit} = 2.262$. Given that our t-value, ignoring polarity and just looking at the absolute value, so $t = 2.61$, is equal to or larger than our $t_{crit}$ then we can say our result is not significant, and as such would be written up as t(9) = 2.61, p < .05, d = 0.83.

Remember: If you were writing this up as a report, and analysis the data in R, then you would see the p-value was actually p = 0.028, and would be written up as p = 0.028

9.2.2 DataSet 2

Let's say that this is our starting data:

Participants	PreTest	PostTest
1	73	72
2	60	54
3	53	52
4	74	74
5	58	54
6	55	52
7	54	49
8	57	58
9	59	60
10	72	72

The first thing we need to do is calculate the difference between the PostTest and the PreTest for each participant, based on $D = PostTest - PreTest$. So for example:

• Participant 1 would be: 72 - 73 = -1
• Participant 2 would be: 54 - 60 = -6
• etc

And if we do that for each Participant and added a column of the differences ($D$) then we would see:

Participants	PreTest	PostTest	D
1	73	72	-1
2	60	54	-6
3	53	52	-1
4	74	74	0
5	58	54	-4
6	55	52	-3
7	54	49	-5
8	57	58	1
9	59	60	1
10	72	72	0

Now, the within-subjects t-test formula is:

$$t = \frac{\bar{D}}{\frac{SD_{D}}{\sqrt{N}}}$$

We can see that $N = 10$, but we need to calculate $\bar{D}$ (called D-Bar, the mean of the $D$ column) and $SD_{D}$.

Calculating D-bar

So the $\bar{D}$ formula is the same as the mean formula:

$$\bar{D} = \frac{\sum{D}}{N}$$

Where $D$ is $PostTest - PreTest$ for each Participant.

Then:

$$\bar{D} = \frac{(72 - 73) + (54 - 60) + (52 - 53) + (74 - 74) + (54 - 58) + \\ (52- 55) + (49 - 54) + (58 - 57) + (60 - 59) + (72 - 72)}{10}$$

Which if we resolve all the brackets becomes:

$$\bar{D} = \frac{-1 + -6 + -1 + 0 + -4 + -3 + -5 + 1 + 1 + 0}{10}$$

And if we sum the top half together

$$\bar{D} = \frac{-18}{10}$$

Leaving us with:

$$\bar{D} = -1.8$$

So we find that $\bar{D}$ = -1.8, which is the mean difference between the Post test and Pre test values.

The Standard Deviation of D

The standard deviation formula is:

$$SD = \sqrt\frac{\sum(X - \bar{X})^2}{N-1}$$

Which if we translate to using D, becomes:

$$SD_{D} = \sqrt\frac{\sum(D - \bar{D})^2}{N-1}$$

Then:

$$SD_{D} =\sqrt\frac{(-1 - -1.8)^2 + (-6 - -1.8)^2 + (-1 - -1.8)^2 + (0 - -1.8)^2 + \\ (-4 - -1.8)^2 + (-3 - -1.8)^2 + (-5 - -1.8)^2 + (1 - -1.8)^2 + \\ (1 - -1.8)^2 + (0 - -1.8)^2}{10 - 1}$$

And if we start stepping through this analysis by dealing with the brackets:

$$SD_{D} =\sqrt\frac{(0.8)^2 + (-4.2)^2 + (0.8)^2 + (1.8)^2 + (-2.2)^2 + \\ (-1.2)^2 + (-3.2)^2 + (2.8)^2 + (2.8)^2 + (1.8)^2}{10 - 1}$$

And then we square those brackets

$$SD_{D} =\sqrt\frac{0.64 + 17.64 + 0.64 + 3.24 + 4.84 + \\ 1.44 + 10.24 + 7.84 + 7.84 + 3.24}{10 - 1}$$

And sum up all the values on the top half:

$$SD_{D} =\sqrt\frac{57.6}{10 - 1}$$

And then we need to sort out the bottom half

$$SD_{D} =\sqrt\frac{57.6}{9}$$

Which by dividing the top half by the bottom half reduces down to:

$$SD_{D} =\sqrt{6.4}$$

And then we finally take the square root, which leaves us with:

$$SD_{D} =2.5298221$$

And so we find that the $SD_{D}$ = 2.5298221 which to two decimal places would be, $SD_{D} = 2.53$

Calculating the t-value

And finally the t-test formula is:

$$t = \frac{\bar{D}}{\frac{SD_{D}}{\sqrt{N}}}$$

Then if we start filling in the values we know from above we see:

$$t = \frac{-1.8}{\frac{2.5298221}{\sqrt{10}}}$$

And if we deal with the square root first:

$$t = \frac{-1.8}{\frac{2.5298221}{3.1622777}}$$

And divide $SD_{D}$ by $\sqrt{N}$ - tidying up the bottom of the formula:

$$t = \frac{-1.8}{0.8}$$

And then solve for $t$ by dividing the top half by the bottom half gives us:

$$t = -2.25$$

And so, rounding to two decimal places, we find that $t = -2.25$

Degrees of Freedom

Great! Now we just need the degrees of freedom where the formula is:

$$df = N - 1$$

And we already know that $N= 10$ and putting them into the equation we get:

$$df = 10 - 1$$

Which reduces to:

$$df = 9$$

Meaning that we find a $df = 9$

Effect Size - Cohen's d

And finally Cohen's d, the effect size. One of the common formulas based on knowing the t-value and the N is:

$$d = \frac{t}{\sqrt{N}}$$

Which, based on the info above, we know:

• $t = -2.25$
• $N = 10$

And putting those into the formula we get:

$$d = \frac{-2.25}{\sqrt{10}}$$

Which gives us:

$$d = \frac{-2.25}{3.1622777}$$

And so:

$$d = -0.7115125$$

Meaning that the effect size, to two decimal places, is d = -0.71.

Determining Significance

If we were to look at a critical values look-up table for $df = 9$ and $\alpha = .05$ (two-tailed), we would see that the critical value is $t_{crit} = 2.262$. Given that our t-value, ignoring polarity and just looking at the absolute value, so $t = 2.25$, is smaller than our $t_{crit}$ then we can say our result is not significant, and as such would be written up as t(9) = 2.25, p > .05, d = 0.71.

Remember: If you were writing this up as a report, and analysis the data in R, then you would see the p-value was actually p = 0.051, and would be written up as p = 0.051

9.2.3 DataSet 3

Let's say that this is our starting data:

Participants	PreTest	PostTest
1	62	56
2	75	72
3	56	55
4	80	79
5	55	55
6	70	69
7	79	79
8	50	52
9	51	49
10	78	77

The first thing we need to do is calculate the difference between the PostTest and the PreTest for each participant, based on $D = PostTest - PreTest$. So for example:

• Participant 1 would be: 56 - 62 = -6
• Participant 2 would be: 72 - 75 = -3
• etc

And if we do that for each Participant and added a column of the differences ($D$) then we would see:

Participants	PreTest	PostTest	D
1	62	56	-6
2	75	72	-3
3	56	55	-1
4	80	79	-1
5	55	55	0
6	70	69	-1
7	79	79	0
8	50	52	2
9	51	49	-2
10	78	77	-1

Now, the within-subjects t-test formula is:

$$t = \frac{\bar{D}}{\frac{SD_{D}}{\sqrt{N}}}$$

We can see that $N = 10$, but we need to calculate $\bar{D}$ (called D-Bar, the mean of the $D$ column) and $SD_{D}$.

Calculating D-bar

So the $\bar{D}$ formula is the same as the mean formula:

$$\bar{D} = \frac{\sum{D}}{N}$$

Where $D$ is $PostTest - PreTest$ for each Participant.

Then:

$$\bar{D} = \frac{(56 - 62) + (72 - 75) + (55 - 56) + (79 - 80) + (55 - 55) + \\ (69- 70) + (79 - 79) + (52 - 50) + (49 - 51) + (77 - 78)}{10}$$

Which if we resolve all the brackets becomes:

$$\bar{D} = \frac{-6 + -3 + -1 + -1 + 0 + -1 + 0 + 2 + -2 + -1}{10}$$

And if we sum the top half together

$$\bar{D} = \frac{-13}{10}$$

Leaving us with:

$$\bar{D} = -1.3$$

So we find that $\bar{D}$ = -1.3, which is the mean difference between the Post test and Pre test values.

The Standard Deviation of D

The standard deviation formula is:

$$SD = \sqrt\frac{\sum(X - \bar{X})^2}{N-1}$$

Which if we translate to using D, becomes:

$$SD_{D} = \sqrt\frac{\sum(D - \bar{D})^2}{N-1}$$

Then:

$$SD_{D} =\sqrt\frac{(-6 - -1.3)^2 + (-3 - -1.3)^2 + (-1 - -1.3)^2 + (-1 - -1.3)^2 + \\ (0 - -1.3)^2 + (-1 - -1.3)^2 + (0 - -1.3)^2 + (2 - -1.3)^2 + \\ (-2 - -1.3)^2 + (-1 - -1.3)^2}{10 - 1}$$

And if we start stepping through this analysis by dealing with the brackets:

$$SD_{D} =\sqrt\frac{(-4.7)^2 + (-1.7)^2 + (0.3)^2 + (0.3)^2 + (1.3)^2 + \\ (0.3)^2 + (1.3)^2 + (3.3)^2 + (-0.7)^2 + (0.3)^2}{10 - 1}$$

And then we square those brackets

$$SD_{D} =\sqrt\frac{22.09 + 2.89 + 0.09 + 0.09 + 1.69 + \\ 0.09 + 1.69 + 10.89 + 0.49 + 0.09}{10 - 1}$$

And sum up all the values on the top half:

$$SD_{D} =\sqrt\frac{40.1}{10 - 1}$$

And then we need to sort out the bottom half

$$SD_{D} =\sqrt\frac{40.1}{9}$$

Which by dividing the top half by the bottom half reduces down to:

$$SD_{D} =\sqrt{4.4555556}$$

And then we finally take the square root, which leaves us with:

$$SD_{D} =2.1108187$$

And so we find that the $SD_{D}$ = 2.1108187 which to two decimal places would be, $SD_{D} = 2.11$

Calculating the t-value

And finally the t-test formula is:

$$t = \frac{\bar{D}}{\frac{SD_{D}}{\sqrt{N}}}$$

Then if we start filling in the values we know from above we see:

$$t = \frac{-1.3}{\frac{2.1108187}{\sqrt{10}}}$$

And if we deal with the square root first:

$$t = \frac{-1.3}{\frac{2.1108187}{3.1622777}}$$

And divide $SD_{D}$ by $\sqrt{N}$ - tidying up the bottom of the formula:

$$t = \frac{-1.3}{0.6674995}$$

And then solve for $t$ by dividing the top half by the bottom half gives us:

$$t = -1.9475671$$

And so, rounding to two decimal places, we find that $t = -1.95$

Degrees of Freedom

Great! Now we just need the degrees of freedom where the formula is:

$$df = N - 1$$

And we already know that $N= 10$ and putting them into the equation we get:

$$df = 10 - 1$$

Which reduces to:

$$df = 9$$

Meaning that we find a $df = 9$

Effect Size - Cohen's d

And finally Cohen's d, the effect size. One of the common formulas based on knowing the t-value and the N is:

$$d = \frac{t}{\sqrt{N}}$$

Which, based on the info above, we know:

• $t = -1.95$
• $N = 10$

And putting those into the formula we get:

$$d = \frac{-1.95}{\sqrt{10}}$$

Which gives us:

$$d = \frac{-1.95}{3.1622777}$$

And so:

$$d = -0.6166441$$

Meaning that the effect size, to two decimal places, is d = -0.62.

Determining Significance

If we were to look at a critical values look-up table for $df = 9$ and $\alpha = .05$ (two-tailed), we would see that the critical value is $t_{crit} = 2.262$. Given that our t-value, ignoring polarity and just looking at the absolute value, so $t = 1.95$, is smaller than our $t_{crit}$ then we can say our result is not significant, and as such would be written up as t(9) = 1.95, p > .05, d = 0.62.

Remember: If you were writing this up as a report, and analysis the data in R, then you would see the p-value was actually p = 0.083, and would be written up as p = 0.083

9.3 Look-Up table

Remembering that the $t_{crit}$ value is the smallest t-value you need to find a significant effect, find the $t_{crit}$ for your df, assuming $\alpha = .05$. If the $t$ value you calculated is equal to or larger than $t_{crit}$ then your test is significant.

df	$\alpha = .05$
1	12.706
2	4.303
3	3.182
4	2.776
5	2.571
6	2.447
7	2.365
8	2.306
9	2.262
10	2.228
15	2.131
20	2.086
30	2.042
40	2.021
50	2.009
60	2
70	1.994
80	1.99
90	1.987
100	1.984