Power analysis for proportions

Compute sample size, power, or minimum detectable effect (MDE) for a two-sample test of proportions. Leave exactly one of n, power, or p2 as NULL to solve for that quantity.

Usage

power_prop(p1, ...)

# Default S3 method
power_prop(
  p1,
  ...,
  p2 = NULL,
  n = NULL,
  power = 0.8,
  alpha = 0.05,
  N = Inf,
  deff = 1,
  resp_rate = 1,
  alternative = c("two.sided", "one.sided"),
  ratio = 1,
  overlap = 0,
  overlap_cor = 0,
  method = c("wald", "arcsine", "logodds"),
  plan = NULL
)

Arguments

p1: Baseline proportion, in (0, 1).
...: Additional arguments passed to methods. Unused arguments are rejected.
p2: Alternative proportion, in (0, 1). Leave NULL to solve for the minimum detectable effect (MDE). The solver searches both above and below p1 and returns the alternative closest to p1 that achieves the target power. When p1 is near 0 or 1, the MDE may only be detectable in one direction.
n: Per-group sample size, measured as gross units drawn and bounded by the corresponding finite N. Scalar (equal groups) or length-2 vector c(n1, n2) for unequal groups. Leave NULL to solve for sample size.
power: Target power, in (0, 1). Leave NULL to solve for power.
alpha: Significance level, default 0.05.
N: Population size for finite-population correction. A scalar applies to both groups. A length-2 vector c(N1, N2) sets group-specific population sizes. Inf disables FPC for the corresponding group.
deff: Design effect multiplier (> 0). Values < 1 are valid for efficient designs (e.g., stratified sampling with Neyman allocation).
resp_rate: Expected response rate, in (0, 1]. Default 1 (no adjustment). The required sample size is inflated by 1 / resp_rate.
alternative: Character: "two.sided" (default) or "one.sided". Use "two.sided" when either an increase or decrease matters (the usual default). Use "one.sided" when only one direction is of interest (e.g. "has the intervention reduced stunting?"), which requires a smaller sample for the same power.
ratio: Allocation ratio n1/n2 (default 1). Only used when solving for n (n = NULL). For example, ratio = 2 means group 1 gets twice the sample of group 2.
overlap: Panel overlap fraction in [0, 1], for repeated surveys. Defined as the fraction of group 1 that also appears in group 2 (overlap = n12 / n1). Only supported with method = "wald".
overlap_cor: Correlation between occasions in [0, 1].
method: Variance method: "wald" (default), "arcsine", or "logodds". Arcsine and log-odds transforms are variance-stabilizing and perform better for rare or extreme proportions (Valliant, 2018, sections 4.3.4–4.3.5).
plan: Optional svyplan() object providing design defaults.

Value

A svyplan_power object with components:

n: Per-group sample size (scalar or length-2 for unequal groups).
power: Achieved power.
effect: Difference in proportions (abs(p2 - p1)).
solved: Which quantity was solved for ("n", "power", or "mde").
params: List of input parameters.

Details

Choosing a method

"wald" (default): Standard Wald test. Appropriate for proportions in the 0.2–0.8 range. The only method that supports panel overlap.
"arcsine": Arcsine-transformed test. The arcsine variance $1/(4n)$ is approximately constant in $p$, making it more accurate when either proportion is below 0.15 or above 0.85.
"logodds": Log-odds (logit) test with separate null and alternative variances. Also suitable for rare or extreme proportions. Uses Valliant Eq (4.23)/(4.24).

For proportions in the 0.2–0.8 range, all three methods give similar results. For rare or extreme proportions, "arcsine" or "logodds" are more reliable.

Null variance convention

method = "wald" uses the unpooled variance $p_1 q_1 / n_1 + p_2 q_2 / n_2$ for both the critical value and the power shift. stats::power.prop.test() instead evaluates the critical value under the null, using the pooled variance $\bar{p} \bar{q} (1/n_1 + 1/n_2)$ with $\bar{p} = (p_1 + p_2) / 2$. Both conventions are standard, and they differ by a few tenths of a percent in the resulting size:

power_prop(p1 = 0.3, p2 = 0.4, power = 0.8)$n   # 353.2
power.prop.test(p1 = 0.3, p2 = 0.4, power = 0.8)$n   # 355.9

The unpooled form is used here because it is the variance the package reports everywhere else: it is the same expression that deff, the finite population correction, and resp_rate act on in n_prop() and prec_prop(), so a power calculation and a precision calculation for the same design stay on one scale. The pooled form has no finite-population analogue that keeps that correspondence.

method = "logodds" does carry a pooled null, because the statistic it powers has one. It refers the log-odds difference to a critical value computed under the null and a power shift computed under the alternative, rejecting when

$$|\mathrm{logit}(\hat p_1) - \mathrm{logit}(\hat p_2)| > z_{\alpha} \sqrt{V_0},$$

with the two variances

$$V_0 = d \left( \frac{f_1}{n_1 \bar{p} \bar{q}} + \frac{f_2}{n_2 \bar{p} \bar{q}} \right), \qquad V_A = d \left( \frac{f_1}{n_1 p_1 q_1} + \frac{f_2}{n_2 p_2 q_2} \right),$$

where $d$ is deff and $f_i$ the finite population correction of group $i$. Under the null the two groups share one proportion, and the estimator of it is the pooled one, so $\bar{p}$ is the sample-size-weighted mean

$$\bar{p} = \frac{n_1 p_1 + n_2 p_2}{n_1 + n_2} = \frac{r p_1 + p_2}{r + 1},$$

with $r$ the allocation ratio. At ratio = 1 this is $(p_1 + p_2) / 2$; away from it the weighted and unweighted nulls give materially different sizes, so the weighting is not a refinement. Sizing p1 = 0.1 against p2 = 0.2 at ratio = 4 needs 523 and 131, against 457 and 115 for an unweighted null: 14 percent more fieldwork, because the larger group is the one with the smaller proportion and pulls the pooled null toward it. Both the size-solving and the power-computing path use this $\bar{p}$, so power_prop() inverts itself under any allocation.

Normal approximation

All three methods compute critical values and power from the standard normal, with no degrees-of-freedom correction and the variance treated as known. This is the convention for survey-scale samples and is what lets deff, a finite N, and resp_rate enter the variance directly. The choice of method addresses accuracy in $p$, not in $n$: "arcsine" and "logodds" improve the normal approximation for an extreme proportion, but none of the three is exact at small n.

The df argument that n_prop(), n_mean() and n_alloc() accept has no counterpart here, and its absence is a decision rather than an omission. There the quantile is the half-width of a confidence interval and a t quantile substitutes for a normal one directly; here it is a normal deviate for an alternative, and a t-based power calculation is a different procedure. Passing df is an error that says so.

References

Valliant, R., Dever, J. A., & Kreuter, F. (2018). Practical Tools for Designing and Weighting Survey Samples (2nd ed.). Springer. Chapter 4.

Cochran, W. G. (1977). Sampling Techniques (3rd ed.). Wiley.

Examples

# Sample size to detect a 5pp change from 30%
power_prop(p1 = 0.30, p2 = 0.35)
#> Power analysis for proportions (solved for sample size)
#> n = 1374 (per group), power = 0.800, effect = 0.0500
#> (p1 = 0.300, p2 = 0.350, alpha = 0.05, deff = 1)

# Power given n = 500
power_prop(p1 = 0.30, p2 = 0.35, n = 500, power = NULL)
#> Power analysis for proportions (solved for power)
#> n = 500 (per group), power = 0.394, effect = 0.0500
#> (p1 = 0.300, p2 = 0.350, alpha = 0.05, deff = 1)

# MDE with n = 1000
power_prop(p1 = 0.30, n = 1000)
#> Power analysis for proportions (solved for minimum detectable effect)
#> n = 1000 (per group), power = 0.800, effect = 0.0557
#> (p1 = 0.300, p2 = 0.244, alpha = 0.05, deff = 1)

# Arcsine transform for rare proportions
power_prop(p1 = 0.15, p2 = 0.18, alternative = "one.sided",
           method = "arcsine")
#> Power analysis for proportions (solved for sample size)
#> n = 1890 (per group), power = 0.800, effect = 0.0300
#> (p1 = 0.150, p2 = 0.180, alpha = 0.05, deff = 1, one-sided, method = arcsine)

# Log-odds transform
power_prop(p1 = 0.15, p2 = 0.18, alternative = "one.sided",
           method = "logodds")
#> Power analysis for proportions (solved for sample size)
#> n = 1889 (per group), power = 0.800, effect = 0.0300
#> (p1 = 0.150, p2 = 0.180, alpha = 0.05, deff = 1, one-sided, method = logodds)

# Allocation ratio 2:1
power_prop(p1 = 0.30, p2 = 0.35, ratio = 2)
#> Power analysis for proportions (solved for sample size)
#> n1 = 2088, n2 = 1044 (total = 3132), power = 0.800, effect = 0.0500
#> (p1 = 0.300, p2 = 0.350, alpha = 0.05, deff = 1, ratio = 2)