Planning design effect

Build the design effect you expect a complex design to produce, before any data are collected, by combining the design features you are planning: clustering, unequal weighting, and stratification. The result is a multiplier you pass as deff to n_prop(), n_mean(), n_alloc(), or any other sizing or precision function.

Usage

design_effect(x = NULL, ...)

# Default S3 method
design_effect(
  x = NULL,
  ...,
  icc = NULL,
  n_per_psu = NULL,
  n_per_ssu = NULL,
  var_ratio = 1,
  weights = NULL,
  strata = NULL
)

# S3 method for class 'numeric'
design_effect(x, ...)

# S3 method for class 'svyplan_cluster'
design_effect(x, ..., weights = NULL, strata = NULL)

# S3 method for class 'svyplan_prec'
design_effect(x, ..., weights = NULL, strata = NULL)

# S3 method for class 'svyplan_varcomp'
design_effect(
  x,
  ...,
  n_per_psu = NULL,
  n_per_ssu = NULL,
  weights = NULL,
  strata = NULL
)

# S3 method for class 'svyplan_n'
design_effect(x, ..., weights = NULL)

Arguments

x: A svyplan result to read design features from: a n_cluster() or prec_cluster() allocation, a varcomp() estimate, or an n_alloc() allocation. NULL (default) builds the design effect from the component arguments below.
...: Additional arguments passed to methods. Unused arguments are rejected.
icc: Homogeneity of units within a cluster, in $[0, 1]$. A scalar for a two-stage design, or c(icc_psu, icc_ssu) for a three-stage design. This is the survey-planning measure returned by varcomp(), not a generic mixed-model ICC.
n_per_psu: Units taken per PSU in a two-stage design, or SSUs taken per PSU in a three-stage design. Required with icc.
n_per_ssu: Units taken per SSU. Supply only for a three-stage design, where icc has length 2.
var_ratio: Ratio of the components' unit variance to the analysis variable's, default 1. A scalar names var_ratio_psu and, for a three-stage icc, derives var_ratio_ssu = var_ratio_psu * (1 - icc_psu). Supply a length-2 vector only to override that identity. See Details.
weights: For an n_alloc() result only: an additional anticipated weighting adjustment charged on top of the allocation, such as a nonresponse correction or a calibration step. Do not pass the allocation's own weights. The stratum takes already carry those, and supplying them again counts the same disproportionality twice. The allocation component is exact for the plan, while this one is Kish's approximation, so with weights supplied the overall result is approximate too and prints as such.
strata: Stratum-level data frame describing a planned stratified allocation, using the n_alloc() column names. N with n gives the weighting component. N with sd and mean gives the stratification component. Supply all four for both.

Value

A svyplan_deff object: a numeric scalar carrying the component decomposition. Use it directly wherever a deff argument is expected, as.double() to strip it to a plain number, and as.data.frame() to export the components.

Details

The design effect (DEFF) is the ratio of the variance under the planned design to the variance of a simple random sample of the same size. DEFF = 2 means you need twice the sample for the same precision, whereas DEFF below 1 means the design is more efficient than simple random sampling.

design_effect() is a planning tool. It anticipates a design effect from design parameters you choose or estimate beforehand. It does not estimate a design effect from collected data: once you have a realized sample, use survey::svymean(..., deff = TRUE) and survey::deff(), which compute it from the actual weights, strata, and clusters.

Components

Components are selected by which arguments you supply, and multiply together:

$$DEFF = DEFF_{cluster} \times DEFF_{weight} \times DEFF_{strata}.$$

Clustering (icc, n_per_psu, n_per_ssu, var_ratio) uses the same variance model as n_cluster() and prec_cluster(), so the two always agree. For a two-stage design with n_per_psu units taken per PSU, $$DEFF_{cluster} = k(1 + \delta(m - 1)),$$ and for a three-stage design taking n_per_psu SSUs per PSU and n_per_ssu units per SSU, $$DEFF_{cluster} = k_1\delta_1 mq + k_2(1 + \delta_2(q - 1)).$$ Estimate icc (and var_ratio) from a previous round or pilot with varcomp().

In the three-stage form $k_1$ rescales the components' unit variance to the analysis variable and $k_2$ does the same for the within-PSU part, which is $1-\delta_1$ of it. The two are therefore linked, $$k_2 = k_1(1 - \delta_1),$$ and that identity is what makes the design effect collapse to $k_1$ at $m=q=1$, where one unit is taken per SSU and one SSU per PSU so no clustering remains. A scalar var_ratio supplies $k_1$ and derives $k_2$ from it. Supplying both explicitly overrides the identity, which is meaningful only when the two ratios come from different decompositions; note that varcomp() estimates $k_2$ from its own SSU-level decomposition rather than imposing the identity, so its value can differ by several percent on small clusters.

Written on components referenced to the total unit variance, $\delta_1=\sigma_1^2/S^2$ and $\delta_2^{tot}=\sigma_2^2/S^2$, the same quantity is the familiar $1+\delta_1(mq-1)+\delta_2^{tot}(q-1)$. The package's icc_ssu is referenced to the within-PSU variance instead, so $\delta_2=\delta_2^{tot}/(1-\delta_1)$.

Unequal weighting (weights, or N and n in strata) is Kish's weighting loss. From a vector of planned weights, $$DEFF_{weight} = n\sum_i w_i^2 / (\sum_i w_i)^2,$$ and from a planned stratified allocation with stratum sizes $N_h$ and takes $n_h$ the same quantity is $$DEFF_{weight} = n\sum_h N_h^2/n_h / (\sum_h N_h)^2, \quad n = \sum_h n_h.$$ Equal weights give exactly 1. This is the cost of a disproportionate allocation, of weighting classes, or of an anticipated nonresponse adjustment, and it is never below 1.

Stratification (N, sd, and mean in strata) is the gain from stratifying, the only component that can fall below 1. With $W_h = N_h/N$, $$DEFF_{strata} = \sum_h W_hS_h^2 / \left(\sum_h W_hS_h^2 + \sum_h W_h(\bar y_h - \bar y)^2\right).$$ It measures the gain under proportional allocation. Any departure from proportional is already charged to the weighting component, so the two compose without double counting.

Because the clustering component already prices the clustering, do not additionally inflate deff for it elsewhere. Components you leave unspecified are simply absent, which is equivalent to setting them to 1.

What a multiplier can and cannot say

Multiplying the components you supply by hand is Kish's approximation. It is exact when the stratum standard deviations are equal, whatever the stratum sizes, means and takes, and not otherwise. The gap runs in either direction, so it is not a bound: the weighting component charges a disproportionate allocation as a loss without knowing which strata were favored, so a Neyman allocation over strata that differ sharply in sd is reported well above its true variance ratio, while a constraint that forces units into a low-variance stratum is reported well below it. The printed result marks a multi-component product as approximate for that reason.

Given an n_alloc() result there is no need to approximate at all. Every quantity the ratio needs is already there, so design_effect(alloc) returns the allocation's own variance ratio, $$D = n \sum_h W_h^2S_h^2d_hk_h(1+\delta_h(m_h-1))/n_h \Big/ \left(\sum_h W_hS_h^2+\sum_h W_h(\bar y_h-\bar y)^2\right),$$ with the per-stratum cluster factors entering stratum by stratum rather than averaged, and the deff the allocation was built under counted once. A per-stratum deff weights its own stratum's contribution and a scalar factors straight back out. A generalized allocation optimizing several measures has no single such ratio and is refused. Take the measure-specific numbers from prec_alloc().

This ratio is computed without finite population corrections, on the same planning scale as the component arguments. prec_alloc() applies each stratum's FPC, so a ratio derived from its variance need not equal design_effect(alloc) once sampling fractions are material. Use prec_alloc() when the design variance itself is what you want.

Methods (by class)

design_effect(default): Build the design effect from planning components. Supply the components you are designing for and leave the rest NULL.
design_effect(numeric): Reject a bare numeric vector, which says nothing about which design component it describes.
design_effect(svyplan_cluster): Read icc, var_ratio, and the stage takes from a n_cluster() or prec_cluster() allocation.
design_effect(svyplan_prec): Read the design features off a prec_cluster() result, the same way as for n_cluster(). Other svyplan_prec types carry the deff you supplied rather than one to be derived, and are rejected.
design_effect(svyplan_varcomp): Read icc and var_ratio from a varcomp() estimate. The stage takes are your design choice, so n_per_psu (and n_per_ssu for three stages) must be supplied.
design_effect(svyplan_n): Read the planned allocation from an n_alloc() result. This returns the allocation's own variance ratio, computed from the stratum sizes, takes, standard deviations, means and any per-stratum cluster factors, rather than a product of separate weighting and stratification approximations. Supply weights only for an additional anticipated adjustment. See the weights argument.

References

Kish, L. (1965). Survey Sampling. Wiley.

Valliant, R., Dever, J. A., and Kreuter, F. (2018). Practical Tools for Designing and Weighting Survey Samples (2nd ed.). Springer. Ch. 9.

Examples

# Clustering: 25 households per cluster, homogeneity 0.05
design_effect(icc = 0.05, n_per_psu = 25)
#> Planning design effect: 2.2000

# Three stages: 10 SSUs per PSU, 4 units per SSU
design_effect(icc = c(0.01, 0.05), n_per_psu = 10, n_per_ssu = 4)
#> Planning design effect: 1.5385

# Weighting loss from a planned set of weights
design_effect(weights = rep(c(1, 4), c(300, 100)))
#> Planning design effect: 1.5510

# A planned stratified allocation: weighting loss and stratification gain
frame <- data.frame(
  N    = c(50000, 120000),
  n    = c(600, 400),
  sd   = c(12, 20),
  mean = c(55, 48)
)
design_effect(strata = frame)
#> Planning design effect: 1.3476

# Clustering and weighting together
deff <- design_effect(icc = 0.05, n_per_psu = 25, strata = frame)
deff
#> Planning design effect: 2.9648
n_prop(p = 0.3, moe = 0.05, deff = deff)
#> Sample size for proportion (wald)
#> n = 957 (p = 0.30, moe = 0.050, deff = 2.96)
#> expected cases = 287.0

# Read the design features off an existing plan
plan <- n_cluster(stage_cost = c(500, 50), icc = 0.05, cv = 0.05)
design_effect(plan)
#> Planning design effect: 1.6392