## QC Comparison of WWC effect size code using R and SAS

I wrote this entry when I wanted to QC my WWC effect size calculation using R and SAS.

WWC standard doc

https://ies.ed.gov/ncee/wwc/Docs/referenceresources/wwc_procedures_v2_1_standards_handbook.pdf

Page 37.

In SAS:

data test;

N_Yes=4300;
Mean_Yes=400;
StdDev_Yes=200;

N_No=4000;
Mean_NO=300;
StdDev_NO=200;

/*create statistics*/
mean_dif=(Mean_Yes-Mean_NO);
/*Standardized effects*/

g1=((N_Yes-1)*(StdDev_Yes*StdDev_Yes)) +((N_No-1)*(StdDev_No*StdDev_No));
g2=N_Yes + N_No -2;
g3=sqrt(g1/g2);
WWC_effect=mean_dif/g3;

run;

In R:

FGC_l_N_YES=4300
FGC_l_Mean_YES=400
FGC_l_StdDev_YES=200

REG_l_N_YES=4000
REG_l_Mean_YES=300
REG_l_StdDev_YES=200

simple_gap=FGC_l_Mean_YES-REG_l_Mean_YES

g1<- ((FGC_l_N_YES-1)*(FGC_l_StdDev_YES*FGC_l_StdDev_YES))+((REG_l_N_YES-1)*(REG_l_StdDev_YES*REG_l_StdDev_YES))
g2= FGC_l_N_YES + REG_l_N_YES -2
g3= sqrt(g1/g2)
simple_gap_std= simple_gap/g3
simple_gap_std

g1
g2
g3
simple_gap_std

My web calculator

https://www.estat.us/file/calc_t_test1b.php

Treatment N:4300
Treatment mean:400
Treatment SD:200

Comparison N:4000
Comparison mean:300
Comparison SD:200
The group mean difference:100

[RESULTS FOR CONTINUOUS OUTCOME]

Probability: Under Development (Still working on this)
T-score is: 22.761
Significant at alpha 0.05 (two tail test;I used a z-test and ignored degree freedom; threshold 1.96)

T-test (the same test as above but with three thresholds)T 1.96, 2.576, 3.291, each for p=0.05, 0.01, 0.001
Sig at p=.001***

Hedges d 0.5

## SAS t-test for proportions

data kaz;

N_yes=1162;
N_no=381;

mean_yes=0.129088;
mean_no=0.170604;

DEG_FD=N_Yes + N_No -2;
/*QC’ed
tValue=2.228;
DEG_FD=10;
*/

tValue=(Mean_NO-Mean_Yes)/(SQRT((Mean_NO*(1-Mean_NO) /
N_No )+Mean_Yes*(1-Mean_Yes)/ N_Yes));

/*2 tail test*/
P_value=(1-probt(abs(tValue),DEG_FD))*2;
run;

<?php

function compute()
{

\$roundunit=3;

\$Tmean = \$_POST['Tmean'];
\$Cmean = \$_POST['Cmean'];
\$TSD = \$_POST['TSD'];
\$CSD = \$_POST['CSD'];
\$TN = \$_POST['TN'];
\$CN = \$_POST['CN'];

\$mean_dif=\$Tmean-\$Cmean;
\$SE=sqrt(
((\$TSD*\$TSD) / \$TN)+((\$CSD*\$CSD) / \$CN)
);

\$T=\$mean_dif/\$SE;
\$DF=\$TN+\$CN-2;

\$P="Under Development (Still working on this)";

/*I will get T for binary variable comparison*/
\$N_SUCCESS_T=\$TN*\$Tmean;
\$N_SUCCESS_C=\$CN*\$Cmean;
\$P_=(\$N_SUCCESS_T+\$N_SUCCESS_C)/(\$TN+\$CN);
\$Z_numerator=\$Tmean-\$Cmean-0;
\$Z_denom=SQRT((\$P_*(1-\$P_))*((1/\$TN)+(1/\$CN)));
\$Z_bin=abs(\$Z_numerator/\$Z_denom);
\$Z_bin_abs=abs(\$Z_numerator/\$Z_denom);

/*\$P=stats_dens_normal(\$T, 0,1);*/
/*\$P=stats_dens_gamma(float \$X, float \$shape, float \$scale);*/
/*\$P= \$T / 100 ;*/

/*Hedges g*/
/*g numerator*/
\$g_numerator=(\$Tmean-\$Cmean)*(1-3/((4*(\$TN+\$CN))-9));
/*g demnominator*/
\$g_denominator=SQRT((((\$TN-1)* (\$TSD**2) )+((\$CN-1)* (\$CSD**2) ))/(\$TN+\$CN-2));
\$hedges_d=\$g_numerator/\$g_denominator;
\$hedges_d_abs=abs(\$hedges_d);

/*if binary variabels*/
\$T_Odds=\$Tmean/(1-\$Tmean);
\$C_Odds=\$Cmean/(1-\$Cmean);
\$Odds_ratio=\$T_Odds/\$C_Odds;
\$Tstep1=log(\$T_Odds);
\$Cstep1=log(\$C_Odds);
\$step2=\$Tstep1-\$Cstep1;
\$WWC_binary_effect=\$step2/1.65;

/*
if (\$hedges_d >= 0.2) echo "Small Effect (Cohen)";
if (\$hedges_d >= 0.5) echo "Medium Effect (Cohen)";
if (\$hedges_d >= 0.8) echo "Large Effect (Cohen)";
*/

echo "<br>";
echo "WWC group comparison of continuous and binary variables";
echo "<br>";
echo "<br>";

echo "Treatment N:" .\$TN;
echo "<br>";

echo "Treatment mean:" .\$Tmean;
echo "<br>";
echo "Treatment SD:" .\$TSD;

echo "<br>";
echo "<br>";

echo "Comparison N:" .\$CN;
echo "<br>";
echo "Comparison mean:" .\$Cmean;
echo "<br>";
echo "Comparison SD:" .\$CSD;

echo "<br>";
echo "The group mean difference:".round(\$mean_dif,\$roundunit);
echo "<br>";
echo "<br>";

echo "[RESULTS FOR CONTINUOUS OUTCOME]";
echo "<br><br>";
/*echo "Probability " .round(\$P,2);*/
echo "Probability: " .\$P;
echo "<br>";

\$abs_T=abs(\$T);

echo "T-score is: " .round(\$T,\$roundunit);
echo "<br>";

if(\$abs_T < 1.96 ) {
echo "Not significant at alpha 0.05 (two tail test;I used a z-test and ignored degree of freedom; threshold 1.96)";
}elseif(\$abs_T >=1.96){
echo "Significant at alpha 0.05 (two tail test;I used a z-test and ignored degree freedom; threshold 1.96)";
}
echo "<br>";
echo "<br>";
echo "T-test (the same test as above but with three thresholds)";
echo "T 1.96, 2.576, 3.291, each for p=0.05, 0.01, 0.001";
echo "<br>";
if(\$abs_T < 1.96) {
echo "Not sig. at alpha 0.05";
}elseif(\$abs_T>=1.960 and \$abs_T < 2.576 ){
echo "Sig at p=.05*";

}elseif(\$abs_T>=2.576 and \$abs_T < 3.291 ){
echo "Sig at p=.01**";

}elseif(\$abs_T>=3.291 ){
echo "Sig at p=.001***";
}else {
echo "N/A";
}

echo "<br>";
echo "<br>";

echo "Hedges d " .round(\$hedges_d,\$roundunit);

echo "<br>";
/*cohen's rule of thumb*/
echo "Cohen's rule of thumb for effect size interpretation";
echo "<br>";
if(\$hedges_d_abs < 0.2) {
echo "Close to zero and Not even small Effect (Cohen)";
}elseif(\$hedges_d_abs>=0.2 and \$hedges_d_abs < 0.5){
echo "Small effect (Cohen)";
}elseif(\$hedges_d_abs>=0.5 and \$hedges_d_abs < 0.8){
echo "Medium effect (Cohen)";
}elseif(\$hedges_d_abs>=0.8){
echo "Large effect (Cohen)";
}else {
echo "others";
}
echo "<br>";

echo "Baseline equivalence test";
echo "<br>";
if(\$hedges_d_abs <= 0.05) {
echo "Satisfies the baseline equivalence requirement";
}elseif(\$hedges_d_abs>0.05 and \$hedges_d_abs <= 0.25){
echo "Requires statistical adjustment to satisfy the baseline equivalence requirement";
}elseif(\$hedges_d_abs>0.25){
echo "Does not satisfy the baseline equivalence requirement";
}elseif(\$hedges_d_abs>=10){
echo "Something Strange happened";
}else {
echo "N/A";
}

echo "<br>";
echo "<br>";

echo "[RESULTS FOR BINARY OUTCOME]";

echo "<br>";
echo "If outcomes were binary variables (range 0 to 1), the WWC effect size would be ";
echo "" .round(\$WWC_binary_effect,\$roundunit);

echo "<br>";
echo "T-score for the binary outcome is: " .round(\$Z_bin,\$roundunit);
echo "<br>";

echo "<br>";
echo "T-test for binary outcomes";
echo "<br>";
if(\$Z_bin_abs < 1.96) {
echo "Not sig. at alpha 0.05";
}elseif(\$Z_bin_abs>=1.960 and \$Z_bin_abs < 2.576 ){
echo "Sig at p=.05*";

}elseif(\$Z_bin_abs>=2.576 and \$Z_bin_abs < 3.291 ){
echo "Sig at p=.01**";

}elseif(\$Z_bin_abs>=3.291 ){
echo "Sig at p=.001***";
}else {
echo "N/A";
}

echo "<br>";
echo "<br>";

}

/*echo "The result is: " . compute();*/
compute();

?>

<br>

REFERENCE

<br>
Cohen's rule of thumb about effect sizes:
<br>
<li>If greater than 02, Small Effect
<br>
<li>If greater than 0.5, Medium Effect
<br>
<li>If greater 0.8 then Large Effect
<br>
Cohen, J. Statistical power for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum (1988).
<br>
<a href="https://wmich.edu/sites/default/files/attachments/u58/2015/Effect_Size_Substantive_Interpretation_Guidelines.pdf">
Effect Size Substantive Interpretation Guidelines: Issues in the Interpretation of Effect Sizes Jeff Valentine and Harris Cooper, Duke University(see page. 5)</a>
<br>
<br>
WWC related info:
<br>
<a href="https://ies.ed.gov/ncee/wwc/Docs/ReferenceResources/wwc_procedures_handbook_v4_draft.pdf">WWC procedures handbook (see page. 14)</a>
<br>
<a href="https://ies.ed.gov/ncee/wwc/Docs/OnlineTraining/wwc_training_m3.pdf">WWC standards slides (Definition of small sample size correction, slide 14)</a>
<br>
WWC considers the effect size greater than .25 substnatively important.
<a href="https://ies.ed.gov/ncee/wwc/Docs/referenceresources/wwc_procedures_handbook_v4.pdf">P.22 of WWC standards</a>

<br>
T-Table
<br>
<a href="https://www.sjsu.edu/faculty/gerstman/StatPrimer/t-table.pdf">T-Table</a>

<br>
<a href="calc_t_test1.php">Back to the calculcator </a>
<br>

<a href="https://www.estat.us">My website</a>
<br>
<br>

## When the laptop's fan start making a loud noise

When the laptop fan becomes too loud.

I plug off everything form the laptop and press the power button for 15 seconds and see what happens.

## i Bond

https://www.treasurydirect.gov/indiv/products/prod_ibonds_glance.htm