This article covers power calculations, parameter grids, and an end-to-end planning workflow. See Getting started for the core sizing and precision workflow, repeated surveys for overlap definitions, and multistage planning for cluster design inputs.
Power analysis
Precision analysis answers “how precisely can we estimate a level?” Power analysis answers a different question: “can we detect a change or difference between two groups or time points?”
Sample size for a detectable change
A survey measured vaccination coverage at 70%. How many households are needed in a follow-up to detect a 5 percentage point increase with 80% power?
power_prop(p1 = 0.70, p2 = 0.75, power = 0.80)
#> Power analysis for proportions (solved for sample size)
#> n = 1248 (per group), power = 0.800, effect = 0.0500
#> (p1 = 0.700, p2 = 0.750, alpha = 0.05, deff = 1)With a design effect:
power_prop(p1 = 0.70, p2 = 0.75, power = 0.80, deff = 2.0)
#> Power analysis for proportions (solved for sample size)
#> n = 2496 (per group), power = 0.800, effect = 0.0500
#> (p1 = 0.700, p2 = 0.750, alpha = 0.05, deff = 2.00)Three solve modes
power_prop() and power_mean() each solve
for whichever of n, power, or the effect size
is left unspecified:
# Solve for n (default when p2 and power given)
power_prop(p1 = 0.70, p2 = 0.75, power = 0.80, deff = 2.0)
#> Power analysis for proportions (solved for sample size)
#> n = 2496 (per group), power = 0.800, effect = 0.0500
#> (p1 = 0.700, p2 = 0.750, alpha = 0.05, deff = 2.00)
# Solve for power (set power = NULL)
power_prop(p1 = 0.70, p2 = 0.75, n = 1500, power = NULL, deff = 2.0)
#> Power analysis for proportions (solved for power)
#> n = 1500 (per group), power = 0.584, effect = 0.0500
#> (p1 = 0.700, p2 = 0.750, alpha = 0.05, deff = 2.00)
# Solve for MDE (omit p2)
power_prop(p1 = 0.70, n = 1500, deff = 2.0)
#> Power analysis for proportions (solved for minimum detectable effect)
#> n = 1500 (per group), power = 0.800, effect = 0.0639
#> (p1 = 0.700, p2 = 0.764, alpha = 0.05, deff = 2.00)Panel surveys
The power family takes the same overlap and
overlap_cor as n_change(), and
design_overlap() supplies the first of them from a
design_rotation(). If the follow-up resamples a fraction of
the original households and the between-round correlation is known, the
required sample size drops:
power_prop(p1 = 0.70, p2 = 0.75, power = 0.80, deff = 2.0,
overlap = 0.50, overlap_cor = 0.6)
#> Power analysis for proportions (solved for sample size)
#> n = 1749 (per group), power = 0.800, effect = 0.0500
#> (p1 = 0.700, p2 = 0.750, alpha = 0.05, deff = 2.00, overlap = 0.50, overlap_cor = 0.60)Continuous outcomes
The same three solve modes work for means:
# Sample size to detect a difference of 5 (within-group var = 200)
power_mean(200, effect = 5)
#> Power analysis for means (solved for sample size)
#> n = 126 (per group), power = 0.800, effect = 5.0000
#> (alpha = 0.05, deff = 1)
# MDE with n = 400 per group
power_mean(200, n = 400)
#> Power analysis for means (solved for minimum detectable effect)
#> n = 400 (per group), power = 0.800, effect = 2.8016
#> (alpha = 0.05, deff = 1)Difference-in-differences
power_did() handles two-group, two-period DiD designs.
Specify treated and control group outcomes as
c(baseline, endline) vectors:
# Proportion outcome: treated improves, control stable
power_did(treat = c(0.50, 0.55), control = c(0.50, 0.48),
outcome = "prop", effect = 0.07)
#> Power analysis for DiD proportions (solved for sample size)
#> n = 1598 (per group), power = 0.800, effect = 0.0700
#> (treat = (0.500, 0.550), control = (0.500, 0.480), alpha = 0.05, deff = 1)
# Mean outcome with panel overlap
power_did(treat = c(50, 55), control = c(50, 52),
outcome = "mean", var = 100, effect = 3,
overlap = 0.5, overlap_cor = 0.6)
#> Power analysis for DiD means (solved for sample size)
#> n = 245 (per group), power = 0.800, effect = 3.0000
#> (treat = (50.000, 55.000), control = (50.000, 52.000), alpha = 0.05, deff = 1, var = (100.00, 100.00, 100.00, 100.00), overlap = 0.50, overlap_cor = 0.60)Unequal groups and alternative methods
Both power_prop() and power_mean() support
unequal group sizes via ratio or explicit
n = c(n1, n2):
# 2:1 allocation ratio
power_prop(p1 = 0.30, p2 = 0.35, ratio = 2)
#> Power analysis for proportions (solved for sample size)
#> n1 = 2088, n2 = 1044 (total = 3132), power = 0.800, effect = 0.0500
#> (p1 = 0.300, p2 = 0.350, alpha = 0.05, deff = 1, ratio = 2)
# Unequal group variances
power_mean(c(80, 120), effect = 5)
#> Power analysis for means (solved for sample size)
#> n = 63 (per group), power = 0.800, effect = 5.0000
#> (alpha = 0.05, deff = 1)Near a boundary, the arcsine and log-odds calculations provide transformed-scale alternatives to the untransformed Wald calculation. Their operating characteristics still depend on the design and planning values:
power_prop(p1 = 0.15, p2 = 0.18, alternative = "one.sided",
method = "arcsine")
#> Power analysis for proportions (solved for sample size)
#> n = 1890 (per group), power = 0.800, effect = 0.0300
#> (p1 = 0.150, p2 = 0.180, alpha = 0.05, deff = 1, one-sided, method = arcsine)
power_prop(p1 = 0.15, p2 = 0.18, alternative = "one.sided",
method = "logodds")
#> Power analysis for proportions (solved for sample size)
#> n = 1889 (per group), power = 0.800, effect = 0.0300
#> (p1 = 0.150, p2 = 0.180, alpha = 0.05, deff = 1, one-sided, method = logodds)Power curve
plot() draws the power-vs-sample-size curve with
reference lines at the solved point:
pw <- power_prop(p1 = 0.70, p2 = 0.75, power = 0.80, deff = 2.0)
plot(pw)
Sensitivity analysis
How sensitive is a sample size to assumptions about the design
effect, response rate, or prevalence? The predict() method
evaluates any svyplan result at new parameter
combinations.
Varying design parameters
x <- n_prop(p = 0.3, moe = 0.05, deff = 1.5)
predict(x, expand.grid(
deff = c(1.0, 1.5, 2.0, 2.5),
resp_rate = c(0.8, 0.9, 1.0)
))
#> deff resp_rate n se moe cv rmoe
#> 1 1.0 0.8 403.3532 0.02551067 0.05 0.08503558 0.1666667
#> 2 1.5 0.8 605.0298 0.02551067 0.05 0.08503558 0.1666667
#> 3 2.0 0.8 806.7064 0.02551067 0.05 0.08503558 0.1666667
#> 4 2.5 0.8 1008.3829 0.02551067 0.05 0.08503558 0.1666667
#> 5 1.0 0.9 358.5362 0.02551067 0.05 0.08503558 0.1666667
#> 6 1.5 0.9 537.8042 0.02551067 0.05 0.08503558 0.1666667
#> 7 2.0 0.9 717.0723 0.02551067 0.05 0.08503558 0.1666667
#> 8 2.5 0.9 896.3404 0.02551067 0.05 0.08503558 0.1666667
#> 9 1.0 1.0 322.6825 0.02551067 0.05 0.08503558 0.1666667
#> 10 1.5 1.0 484.0238 0.02551067 0.05 0.08503558 0.1666667
#> 11 2.0 1.0 645.3651 0.02551067 0.05 0.08503558 0.1666667
#> 12 2.5 1.0 806.7064 0.02551067 0.05 0.08503558 0.1666667Cluster budget scenarios
cl <- n_cluster(stage_cost = c(500, 50), icc = 0.05, budget = 100000)
predict(cl, data.frame(budget = c(50000, 100000, 150000, 200000)))
#> budget n_psu n_per_psu total_n cv cost
#> 1 50000 42.04499 13.78405 579.5501 0.05318275 50000
#> 2 100000 84.08997 13.78405 1159.1003 0.03760588 100000
#> 3 150000 126.13496 13.78405 1738.6504 0.03070507 150000
#> 4 200000 168.17994 13.78405 2318.2006 0.02659137 200000Power curves
pw <- power_prop(p1 = 0.30, p2 = 0.35, n = 500, power = NULL)
predict(pw, data.frame(n = seq(200, 1000, 200)))
#> n power effect
#> 1 200 0.1877131 0.05
#> 2 400 0.3272968 0.05
#> 3 600 0.4569385 0.05
#> 4 800 0.5707088 0.05
#> 5 1000 0.6665884 0.05Precision by sample size
pr <- prec_prop(p = 0.3, n = 400)
predict(pr, data.frame(n = c(100, 200, 400, 800, 1600)))
#> n se moe cv rmoe
#> 1 100 0.04582576 0.08981683 0.15275252 0.29938944
#> 2 200 0.03240370 0.06351009 0.10801234 0.21170031
#> 3 400 0.02291288 0.04490842 0.07637626 0.14969472
#> 4 800 0.01620185 0.03175505 0.05400617 0.10585015
#> 5 1600 0.01145644 0.02245421 0.03818813 0.07484736Putting it all together
A typical planning sequence for a national household survey ties
several svyplan functions together. A
svyplan() profile captures the shared design
assumptions:
- Set shared design parameters once with
svyplan() - Define indicators and precision targets per domain
- Compute sample sizes with
n_multi() - Evaluate achieved precision with
prec_multi() - Run sensitivity analysis with
predict() - If frame data is available, estimate variance components with
varcomp() - Optimize the multistage allocation with
n_cluster() - Determine strata boundaries with
strata_bound() - Check that the design has adequate power with
power_prop()/power_mean()
# 1: shared design assumptions
design <- svyplan(deff = 2.0, resp_rate = 0.85)
# 2-3: multi-indicator sample size
targets <- data.frame(
name = c("stunting", "vaccination", "anemia"),
p = c(0.25, 0.70, 0.12),
moe = c(0.05, 0.05, 0.03),
deff = c(2.0, 1.5, 2.5)
)
result <- n_multi(targets)
result
#> Multi-indicator sample size
#> n = 1127 (binding: anemia)
#>
#> name .n .cv_target .cv_achieved .binding
#> stunting 577 0.10204269 0.07297042
#> vaccination 485 0.03644382 0.02388518
#> anemia 1127 0.12755336 0.12755336 *
# 4: achieved precision for each indicator
prec_multi(result)
#> Multi-indicator sampling precision
#>
#> name .se .moe .rmoe .cv
#> stunting 0.02551067 0.05 0.20000000 0.10204269
#> vaccination 0.02551067 0.05 0.07142857 0.03644382
#> anemia 0.01530640 0.03 0.25000000 0.12755336
# 5: how sensitive is the binding indicator to the response rate?
predict(
n_prop(p = 0.25, moe = 0.05, plan = design),
data.frame(resp_rate = c(0.7, 0.8, 0.9, 1.0))
)
#> resp_rate n se moe cv rmoe
#> 1 0.7 823.1697 0.02551067 0.05 0.1020427 0.2
#> 2 0.8 720.2735 0.02551067 0.05 0.1020427 0.2
#> 3 0.9 640.2431 0.02551067 0.05 0.1020427 0.2
#> 4 1.0 576.2188 0.02551067 0.05 0.1020427 0.2
# 9: can we detect a 5pp decline in stunting with this sample?
power_prop(
p1 = 0.25, p2 = 0.20,
n = as.integer(result), power = NULL,
plan = design
)
#> Power analysis for proportions (solved for power)
#> n = 1127 (net: 958, per group), power = 0.459, effect = 0.0500
#> (p1 = 0.250, p2 = 0.200, alpha = 0.05, deff = 2.00, resp_rate = 0.85)Sample-size, cluster, power, and strata-boundary results support
as.integer() and as.double(). For sample-size
results, these return the rounded-up and exact values of n,
respectively, which makes it straightforward to pass results between
functions or into other packages.