## RCT, power analysis, and mediation factor

Do typical RCTs of Education Interventions Have sufficient Statistical Power for Linking Impacts on Teacher Practice and Student Achievement Outcomes?

October 2009

Peter Z. Schochet

## How to determine sample size for a binary variable

If you need to determine sample size for your survey when a variable of interest is a binary outcome, you can use power analysis and decide how many subjects you need to reach out for survey participation. You should also adjust the sample size by expected response missing rate. I wrote an Excel file for sample size calculation:

https://drive.google.com/file/d/0B7AoA5fyqX_sMkZJOUZxN3JvbUk/view?usp=sharing

Reference:

http://www.surveysystem.com/sscalc.htm

Thanks: Mr. George Ohashi for showing me the function that adjusts sample sizes by expected missing rate.

## Variance Almanac of Academic Achievement

Variance Almanac

https://arc.uchicago.edu/reese/variance-almanac-academic-achievement

Hedges and Hedberg (2007) paper:

http://jrre.vmhost.psu.edu/wp-content/uploads/2014/02/22-10.pdf

## Optimal Design Software Manual

## PowerUp! manual (pdf)

## Statistical power analysis in education research

## Arc center

## Bonferroni correction for power analysis with more than two groups

I'm trying to find a textbook reference for the following procedure written explicitly in the context of power analysis. Please let me know if you know (k u e k a w a AT gmail com).

When there are more than two conditions in the experiment design, the alpha level, one of the parameters that go into power analysis, can be divided by the number of contrasts. If there are three groups (control, treatment 1, treatment 2), there are three contrast points:

C vs T1, C vs T2, and T1 vs T2.

The typical alpha level is 0.5, so you can do:

0.5 / 3 = 0.16

and use that in the power analysis software.

If only two contrasts are important for your purpose:

0.5 / 2 = 0.25

Reference:

page 24 of

https://medschool.vanderbilt.edu/cqs/files/cqs/media/2010Ayumi.pdf

Bonferroni correction:

## SAS Proc POWER examples

Comparison of two independent groups:

proc power;

twosamplemeans test=diff

groupmeans = 0 | .2

stddev = 1

npergroup = .

power = .8;

run;

Comparison of dependent data (paired)

proc power;

pairedmeans test=diff

meandiff = .2

corr = 0.5

stddev = 1

npairs = .

power = .8;

run;

Comparison of proportions

proc power;

twosamplefreq test=pchi

groupproportions = (.65 .70)

nullproportiondiff = 0

power = .80

npergroup =.;

run;