Power analysis for means

Compute sample size, power, or minimum detectable effect (MDE) for a two-sample test of means. Leave exactly one of n, power, or effect as NULL to solve for that quantity.

Usage

power_mean(var = NULL, ...)

# Default S3 method
power_mean(
  var = NULL,
  ...,
  sd = NULL,
  effect = NULL,
  n = NULL,
  power = 0.8,
  alpha = 0.05,
  N = Inf,
  deff = 1,
  resp_rate = 1,
  alternative = c("two.sided", "one.sided"),
  ratio = 1,
  overlap = 0,
  overlap_cor = 0,
  plan = NULL
)

Arguments

var: Within-group variance. Scalar (equal variances in both groups) or length-2 vector c(var1, var2) for unequal group variances.
...: Additional arguments passed to methods. Unused arguments are rejected.
sd: Population standard deviation, an alternative spelling of var. Supply exactly one of var or sd. Stratum frames and published survey reports usually quote standard deviations.
effect: Absolute difference in means (effect-size magnitude, positive). Leave NULL to solve for MDE.
n: Per-group sample size. Scalar (equal groups) or length-2 vector c(n1, n2) for unequal groups. Leave NULL to solve for sample size.
power: Target power, in (0, 1). Leave NULL to solve for power.
alpha: Significance level, default 0.05.
N: Population size for finite-population correction. A scalar applies to both groups. A length-2 vector c(N1, N2) sets group-specific population sizes. Inf disables FPC for the corresponding group.
deff: Design effect multiplier (> 0). Values < 1 are valid for efficient designs (e.g., stratified sampling with Neyman allocation).
resp_rate: Expected response rate, in (0, 1]. Default 1 (no adjustment). The required sample size is inflated by 1 / resp_rate.
alternative: Character: "two.sided" (default) or "one.sided". Use "two.sided" when either an increase or decrease matters (the usual default). Use "one.sided" when only one direction is of interest, which requires a smaller sample for the same power.
ratio: Allocation ratio n1/n2 (default 1). Only used when solving for n (n = NULL). For example, ratio = 2 means group 1 gets twice the sample of group 2.
overlap: Panel overlap fraction in [0, 1], for repeated surveys. Defined as the fraction of group 1 that also appears in group 2 (overlap = n12 / n1). A positive value describes a coordinated design with that many units deliberately held in common, which with a finite N requires both occasions to sample one population. The default 0 is the ordinary two-group comparison, where the groups are independent and may be different populations of different sizes; it is not the same model as a deliberately disjoint pair, which is why the two need not agree in the limit when N is small.
overlap_cor: Correlation between occasions in [0, 1].
plan: Optional svyplan() object providing design defaults.

Value

A svyplan_power object with components:

n: Per-group sample size (scalar or length-2 for unequal groups).
power: Achieved power.
effect: Effect size (difference in means).
solved: Which quantity was solved for ("n", "power", or "mde").
params: List of input parameters.

Details

The var argument is the within-group population variance. Estimate it from a pilot study, a previous survey, or published data for a similar population. When uncertain, use a conservative (larger) estimate. This inflates the sample size and reduces the risk of an underpowered design.

To specify the effect in terms of Cohen's d (standardized effect size), convert via effect = d * sqrt(mean(var)), where d follows Cohen's conventions: 0.2 (small), 0.5 (medium), 0.8 (large).

When var is a length-2 vector, the variance of the difference is:

$$V = \sigma^2_1 / r + \sigma^2_2 - 2 \cdot \text{overlap} \cdot \rho \cdot \sigma_1 \sigma_2$$

where r is the allocation ratio n1/n2 (default 1). When var is scalar and ratio = 1, this simplifies to the familiar V = 2 * var * (1 - overlap * overlap_cor).

With a finite N the correction applies to the marginal terms but not to the overlap covariance, which carries a single $1/N$: for two SRSWOR samples sharing $k = overlap \cdot n_1$ units, $Cov(\bar y_1, \bar y_2) = \rho S_1S_2\{k/(n_1n_2) - 1/N\}$. At overlap_cor = 1 with equal sizes and variances the population terms cancel exactly and the difference variance is $2S^2(1 - overlap)/n$, free of N.

Normal approximation

Critical values and power are computed from the standard normal, not from a t distribution with an estimated denominator: no degrees of freedom enter, and the variance is treated as known. This is the convention for survey-scale samples, where the two agree closely, and it is what makes deff and a finite N insertable directly into the variance. At small n the sizes are correspondingly smaller than stats::power.t.test(), which uses a noncentral t; use that function instead when the sample is small enough for the difference to matter.

The df argument that n_prop(), n_mean() and n_alloc() accept has no counterpart here, and its absence is a decision rather than an omission. There the quantile is the half-width of a confidence interval and a t quantile substitutes for a normal one directly; here it is a normal deviate for an alternative, and a t-based power calculation is a different procedure. Passing df is an error that says so.

References

Valliant, R., Dever, J. A., & Kreuter, F. (2018). Practical Tools for Designing and Weighting Survey Samples (2nd ed.). Springer. Chapter 4.

Cochran, W. G. (1977). Sampling Techniques (3rd ed.). Wiley.

Examples

# Sample size to detect a difference of 5 with variance 100
power_mean(100, effect = 5)
#> Power analysis for means (solved for sample size)
#> n = 63 (per group), power = 0.800, effect = 5.0000
#> (alpha = 0.05, deff = 1)

# Power given n = 200
power_mean(100, effect = 5, n = 200, power = NULL)
#> Power analysis for means (solved for power)
#> n = 200 (per group), power = 0.999, effect = 5.0000
#> (alpha = 0.05, deff = 1)

# MDE with n = 500
power_mean(100, n = 500)
#> Power analysis for means (solved for minimum detectable effect)
#> n = 500 (per group), power = 0.800, effect = 1.7719
#> (alpha = 0.05, deff = 1)

# With design effect
power_mean(100, effect = 5, deff = 1.5)
#> Power analysis for means (solved for sample size)
#> n = 95 (per group), power = 0.800, effect = 5.0000
#> (alpha = 0.05, deff = 1.50)

# Unequal group variances
power_mean(c(80, 120), effect = 5)
#> Power analysis for means (solved for sample size)
#> n = 63 (per group), power = 0.800, effect = 5.0000
#> (alpha = 0.05, deff = 1)

# Allocation ratio 2:1
power_mean(100, effect = 5, ratio = 2)
#> Power analysis for means (solved for sample size)
#> n1 = 95, n2 = 48 (total = 143), power = 0.800, effect = 5.0000
#> (alpha = 0.05, deff = 1, ratio = 2)