Precision targets for proportions, means, and ratios

This article helps you choose the quantity to estimate and the way to state its precision. Begin with Getting started for the basic sample-size and precision workflow.

Four ways to state precision

svyplan reports four related quantities:

	Absolute	Relative
Sampling uncertainty	standard error (se)	coefficient of variation (cv)
Interval half-width	margin of error (moe)	relative margin of error (rmoe)

For a proportion, moe = 0.05 means 5 percentage points. It does not mean 5% of the proportion. The latter is a relative margin of error. At p = 0.70, rmoe = 0.05 is an absolute margin of error of 0.035.

n_prop(p = 0.70, moe = 0.035)
#> Sample size for proportion (wald)
#> n = 659 (p = 0.70, moe = 0.035, deff = 1)
#> expected cases = 461.0
n_prop(p = 0.70, rmoe = 0.05)
#> Sample size for proportion (wald)
#> n = 659 (p = 0.70, rmoe = 0.050, deff = 1)
#> expected cases = 461.0
n_prop(p = 0.70, cv = 0.05)
#> Sample size for proportion (wald)
#> n = 172 (p = 0.70, cv = 0.050, deff = 1)
#> expected cases = 120.0

The coefficient of variation describes the sampling variance. The two margin of error targets also depend on the confidence level and interval method. For a two-sided Wald interval, rmoe is the interval quantile times cv.

Proportions and interval methods

The default Wald calculation is the familiar normal approximation:

n_prop(p = 0.30, moe = 0.05)
#> Sample size for proportion (wald)
#> n = 323 (p = 0.30, moe = 0.050, deff = 1)
#> expected cases = 96.8

Four interval methods are available:

Method	Planning interpretation
"wald"	Symmetric normal approximation (the default).
"wilson"	Wilson score interval, bounded to the parameter space.
"logodds"	Interval formed on the log-odds scale and transformed back.
"beta"	Korn–Graubard interval with optional design degrees of freedom.

Their differences are most visible near 0 or 1 or with few sampled clusters:

sapply(c("wald", "wilson", "logodds", "beta"), function(method) {
  result <- prec_prop(
    p = 0.02,
    n = 900,
    deff = 2,
    df = 25,
    method = method
  )
  c(se = result$se, cv = result$cv, moe = result$moe, rmoe = result$rmoe)
})
#>             wald      wilson     logodds        beta
#> se   0.006599663 0.006599663 0.006599663 0.006599663
#> cv   0.329983165 0.329983165 0.329983165 0.329983165
#> moe  0.013592261 0.014251833 0.014565337 0.014997984
#> rmoe 0.679613049 0.712591665 0.728266856 0.749899196

The sampling standard error and CV do not change across columns. The interval half-widths do because each method constructs a different interval. Choose the method that matches the intended reporting procedure. Use df = design_df(...) when the planned variance estimator has few degrees of freedom.

The log-odds limits bound the proportion, or equivalently its odds. They do not bound an odds ratio, and the method does not size a logistic-regression coefficient.

Means and totals

For a mean, supply a population variance or standard deviation estimated from a previous survey, pilot, or comparable population:

n_mean(var = 40000, moe = 30)
#> Sample size for mean
#> n = 171 (var = 40000.00, moe = 30.000, deff = 1)
n_mean(sd = 200, mu = 800, cv = 0.05)
#> Sample size for mean
#> n = 25 (var = 40000.00, cv = 0.050, deff = 1)

A target CV or relative margin of error also needs the anticipated mean because both are relative to its magnitude.

A total is a mean multiplied by a known population size. It therefore does not need a separate sizing function. A CV target is unchanged and an absolute margin of error for a total must be divided by N before sizing the mean:

population <- 10000
total_moe <- 500000

n_mean(
  var = 2500,
  moe = total_moe / population,
  N = population
)
#> Sample size for mean
#> n = 4 (var = 2500.00, moe = 50.000, deff = 1)

This conversion assumes that the population size used to define the total is known for planning.

Ratios of two totals

n_ratio() plans for

R = T_y / T_x,

where the numerator and denominator are observed on the same sampled units. Examples include consumption per person and yield per hectare. Supply the anticipated ratio, the coefficient of variation of each component, and their unit-level correlation:

n_ratio(
  r = 420,
  cv_num = 1.20,
  cv_den = 0.45,
  component_cor = 0.65,
  cv = 0.05
)
#> Sample size for ratio (linearization)
#> n = 377 (r = 420, unit_relvar = 0.941, cv = 0.050, deff = 1)

This estimand is different from the mean of unit ratios y_i/x_i, which is planned with n_mean(). A coverage rate among an eligible group is usually a domain proportion. First calculate the sample required within the eligible domain, then use its expected share to calculate the overall sample. The multiple-indicator and domain guide demonstrates that step.

The ratio calculation uses the relative variance of the linearized variable e = y - Rx. A stronger positive association between the components can make the ratio more stable:

ratio_plan <- n_ratio(
  r = 420,
  cv_num = 1.20,
  cv_den = 0.45,
  component_cor = 0.65,
  cv = 0.05
)

predict(
  ratio_plan,
  data.frame(component_cor = c(0.30, 0.65, 0.90))
)
#>   component_cor     n se      moe   cv      rmoe
#> 1          0.30 527.4 21 41.15924 0.05 0.0979982
#> 2          0.65 376.2 21 41.15924 0.05 0.0979982
#> 3          0.90 268.2 21 41.15924 0.05 0.0979982

Margin-of-error targets are stated on the ratio’s own scale. Here moe = 20 asks for an interval of plus or minus 20 currency units per person, and rmoe would state the same target relative to r. Design effect and response rate enter as for a mean:

n_ratio(
  r = 420,
  cv_num = 1.20,
  cv_den = 0.45,
  component_cor = 0.65,
  moe = 20,
  deff = 1.3,
  resp_rate = 0.85
)
#> Sample size for ratio (linearization)
#> n = 2437 gross (net: 2072) (r = 420, unit_relvar = 0.941, moe = 20.000, deff = 1.30, resp_rate = 0.85)

prec_ratio() evaluates a ratio design at a fixed sample size. Passing it the sizing result carries the same assumptions and returns the target CV:

prec_ratio(ratio_plan)
#> Sampling precision for ratio (linearization)
#> n = 377
#> se = 21.0000, moe = 41.1592, cv = 0.0500, rmoe = 0.0980

When the two component CVs are nearly equal and their correlation is near 1, the assumed ratio is nearly constant and the required size falls toward zero. That size depends on a correlation known to several decimals, and n_ratio() warns about it:

n_ratio(
  r = 420,
  cv_num = 1.20,
  cv_den = 1.19,
  component_cor = 0.9999,
  cv = 0.05
)
#> Warning: the ratio's unit relative variance (0.000386) is far below its
#> components': the plan rests on 'component_cor' = 0.9999 being known to that
#> precision. Vary it with predict() before committing to this size
#> Sample size for ratio (linearization)
#> n = 1 (r = 420, unit_relvar = 0.000386, cv = 0.050, deff = 1)

Using a known auxiliary total

A ratio can also be used to estimate a different quantity. If the auxiliary total T_x is known from the frame, the estimator \widehat{T}_y = T_x\widehat{R} targets the outcome total. For example, known total cultivated area can turn estimated yield per hectare into an estimated harvest total. Multiplication by a known constant preserves relative precision, while an absolute margin of error must be multiplied by T_x.

The first-order variance uses residuals e_i = y_i - R x_i and the planned sample design (Cochran 1977). A generic regression variance reduction such as 1-\rho^2 is not automatically the variance reduction of this ratio estimator, because that reduction depends on the assumed relation between the two variables. The calculation also omits higher-order ratio bias. Document those approximations when using n_ratio() for a known-total auxiliary estimator.

The same residuals give the planning input when the ratio estimator sharpens the estimate of the mean of y. Fit the ratio to pilot data, take the residual variance, and pass it to n_mean(var = ) with the mean of the analysis variable:

set.seed(2)
area <- rlnorm(200, 3, 0.5)
harvest <- 2.5 * area + rnorm(200, 0, 15)
pilot_ratio <- mean(harvest) / mean(area)
residual <- harvest - pilot_ratio * area

n_mean(var = var(harvest), mu = mean(harvest), cv = 0.02)
#> Sample size for mean
#> n = 839 (var = 1260.27, cv = 0.020, deff = 1)
n_mean(var = var(residual), mu = mean(harvest), cv = 0.02)
#> Sample size for mean
#> n = 147 (var = 220.22, cv = 0.020, deff = 1)
n_ratio(
  r = pilot_ratio,
  cv_num = sd(harvest) / mean(harvest),
  cv_den = sd(area) / mean(area),
  component_cor = cor(area, harvest),
  cv = 0.02
)
#> Sample size for ratio (linearization)
#> n = 147 (r = 2.648, unit_relvar = 0.0586, cv = 0.020, deff = 1)

The first call ignores the auxiliary variable. The second sizes the ratio estimator of the mean. With a CV target it gives the same size as n_ratio() on the pilot’s component CVs and correlation, because the two estimators have the same relative variance.

If the auxiliary total is estimated in an earlier sampling phase, it is no longer a fixed multiplier. Its uncertainty and covariance must be included. See the two-phase method boundary.

Ratios in a cluster design

For a multistage survey, estimate the variance components of the linearized variable. The relevant homogeneity is not generally the homogeneity of the numerator or denominator:

set.seed(1)
psu <- rep(1:30, each = 20)
x <- rlnorm(600, 1, 0.4)
y <- 3 * x + rnorm(600, 0, 0.8)

ratio <- mean(y) / mean(x)
linearized <- y - ratio * x
components <- suppressWarnings(varcomp(linearized, stage_id = list(psu)))

n_cluster(
  stage_cost = c(500, 50),
  icc = components$icc,
  unit_relvar = var(linearized) / mean(y)^2,
  var_ratio = components$var_ratio,
  cv = 0.01
)
#> Optimal 2-stage allocation
#> field design: n_psu = 10 | n_per_psu = 15 -> total n = 150
#> cv = 0.0099, cost = 12500
#> continuous optimum: n_psu = 9.49938 | n_per_psu = 15.88247 (cv = 0.0100, cost = 12293)
#> design df = 9

The linearized variable has mean zero by construction, so varcomp() warns that its relative variances are infinite. Its icc and var_ratio remain usable because they do not depend on the mean and unit_relvar is supplied separately on the ratio estimator’s scale.

Precision across sample sizes

prec_prop() and the other prec_*() functions evaluate a fixed sample size. predict() over n shows how precision changes as the sample grows:

fixed_n <- prec_prop(p = 0.30, n = 400)
predict(fixed_n, data.frame(n = c(100, 200, 400, 800, 1600)))
#>      n         se        moe         cv       rmoe
#> 1  100 0.04582576 0.08981683 0.15275252 0.29938944
#> 2  200 0.03240370 0.06351009 0.10801234 0.21170031
#> 3  400 0.02291288 0.04490842 0.07637626 0.14969472
#> 4  800 0.01620185 0.03175505 0.05400617 0.10585015
#> 5 1600 0.01145644 0.02245421 0.03818813 0.07484736

Each fourfold increase in n halves the standard error and the margin of error.

Solve for a reportable level

prec_prop() can solve for the smallest proportion that a fixed sample can estimate at a target relative precision:

floor <- prec_prop(n = 1500, cv = 0.10, N = 2e6)
floor
#> Sampling precision for proportion (wald, solved for p)
#> n = 1500
#> p = 0.06246
#> se = 0.0062, moe = 0.0122, cv = 0.1000, rmoe = 0.1960
#> expected cases = 93.7
floor$params$p
#> [1] 0.06245608

Every larger proportion has a smaller CV under the same design, and every smaller one is too rare to report at that precision. This makes the result a prospective reporting threshold. The printed expected_cases is the expected number of positive cases at that proportion, so the precision threshold can be read beside the min_cases floor of n_prop().

predict() traces the threshold across sample sizes, with the target CV held fixed:

predict(floor, expand.grid(n = c(500, 1500, 5000)))
#>      n          p          se         moe  cv      rmoe
#> 1  500 0.16663201 0.016663201 0.032659274 0.1 0.1959964
#> 2 1500 0.06245608 0.006245608 0.012241167 0.1 0.1959964
#> 3 5000 0.01955979 0.001955979 0.003833649 0.1 0.1959964

A cv target fixes the sampling variance and gives the same threshold under every interval method. An rmoe target fixes the relative half-width of the interval the survey will report, so its threshold moves with method:

prec_prop(n = 1500, rmoe = 0.20, method = "wald", N = 2e6)$params$p
#> [1] 0.06012947
prec_prop(n = 1500, rmoe = 0.20, method = "beta", N = 2e6)$params$p
#> [1] 0.06327848

Two properties bound the rmoe search. The Wilson and Korn–Graubard half-widths do not shrink to zero as p approaches 1, so their relative margin of error has a positive floor. A target below that floor is refused, and the error names the floor:

prec_prop(n = 1500, rmoe = 0.001, method = "wilson", N = 2e6)
#> Error:
#> ! 'rmoe' = 0.001 is unattainable: at n = 1500 the 'wilson' interval's relative margin of error never falls below 0.001276

The back-transformed log-odds half-width turns upward once the logit spread exceeds logit(p). The turn lies near p = 0.999 at n = 1500 and near p = 0.97 at n = 30. Under "logodds" the search stops at that turn and returns the lower root.

prec_mean() takes cv the same way and returns the magnitude of the mean. The standard error of a mean does not involve the mean, so this inversion is a division rather than a search:

prec_mean(var = 25, n = 1500, cv = 0.10)$params$mu
#> [1] 1.290994

Expected confidence intervals

confint() extracts the expected interval from supported size and precision results:

confint(n_prop(p = 0.30, moe = 0.05))
#>  2.5 % 97.5 %
#>   0.25   0.35
confint(n_mean(var = 100, mu = 50, moe = 2))
#>  2.5 % 97.5 %
#>     48     52
confint(ratio_plan)
#>     2.5 %   97.5 %
#>  378.8408 461.1592

The level argument sets the confidence level, and a precision result gives its interval the same way:

confint(fixed_n)
#>      2.5 %    97.5 %
#>  0.2550916 0.3449084
confint(fixed_n, level = 0.99)
#>      0.5 %    99.5 %
#>  0.2409803 0.3590197

A repeated-survey result has an interval too. For the average level pooled across two occasions:

confint(prec_pooled(var = 100, n = 300, occasions = 2, mu = 50))
#>     2.5 %   97.5 %
#>  49.19985 50.80015

The center for a mean must be supplied because variance and margin of error do not determine it. For a proportion, Wald endpoints are truncated to [0, 1] when needed. The other methods stay within the parameter space by construction. Cluster, allocation, power, and multi-indicator results have no single interval and are not supported by confint().

These calculations follow standard survey-sampling variance approximations (Cochran 1977; Valliant, Dever, and Kreuter 2018). Document the source and uncertainty of every planning value, then compare plausible alternatives with predict().

References

Cochran, William G. 1977. Sampling Techniques. 3rd ed. New York: Wiley.

Valliant, Richard, Jill A. Dever, and Frauke Kreuter. 2018. Practical Tools for Designing and Weighting Survey Samples. 2nd ed. Cham: Springer. https://doi.org/10.1007/978-3-319-93632-1.