## RCT, power analysis, and mediation factor

Do typical RCTs of Education Interventions Have sufficient Statistical Power for Linking Impacts on Teacher Practice and Student Achievement Outcomes?

October 2009

Peter Z. Schochet

http://ies.ed.gov/ncee/pdf/20094065.pdf

## How to determine sample size for a binary variable

If you need to determine sample size for your survey when a variable of interest is a binary outcome, you can use power analysis and decide how many subjects you need before collecting data.

You can ignore this one, if too confusing=> You should adjust the sample size by expected response missing rate (e.g., you aim to collect data from 100 people; you expect only 95 will reply; then you use 95 as the sample size for power evaluation).

I wrote an Excel file for sample size calculation, but let me write a bit more here about what I did in there:

If you have expectation as to what %s you will be looking at after your experiment, use those %s (% for the treatment group and % for the control/comparison group) and decide the sample size you will need to evaluate the two %s with confidence.

Often we don't have such expectations -- probably because no one has done a similar study like yours. You can just assume  the two percentages are close to 50%, which will give you the most conservative power analysis results. So if you want to see if the group difference of 5% will give you sufficient statistical confidence, given a certain sample size, you can set the two %s to be 47.5% and 52.5% (the difference being 5%).

If you want to see if the group difference of 10% will give you a good enough statistical confidence, given a certain sample size, you can set the two %s to be 45% and 55% (the difference being 5%).

I wish I could write this more tightly.

Reference:

http://www.surveysystem.com/sscalc.htm

Thanks: Mr. George Ohashi for showing me the function that adjusts sample sizes by expected missing rate.

Variance Almanac