Optimal multistage cluster allocation

Compute optimal per-stage sample sizes for a multistage cluster design, minimizing cost for a given precision or minimizing variance for a given budget.

Usage

n_cluster(stage_cost = NULL, ...)

# Default S3 method
n_cluster(
  stage_cost = NULL,
  ...,
  icc = NULL,
  unit_relvar = NULL,
  var_ratio = NULL,
  cv = NULL,
  budget = NULL,
  n_psu = NULL,
  n_per_psu = NULL,
  n_per_ssu = NULL,
  resp_rate_psu = 1,
  resp_rate_ssu = 1,
  resp_rate = 1,
  fixed_cost = 0,
  indicators = NULL,
  domains = NULL,
  allocation = c("separate", "joint"),
  domain_sampling = c("separate", "natural"),
  min_n_domain = NULL,
  plan = NULL
)

# S3 method for class 'svyplan_prec'
n_cluster(stage_cost, ..., cv = NULL, budget = NULL)

Arguments

stage_cost: For the default method: numeric vector of per-stage costs. Length determines the number of stages (2 or 3). Named vectors are accepted with stage names cost_psu, cost_ssu, cost_tsu (cost_tsu aliases cost_ssu in 2-stage). For svyplan_prec objects: a precision result from prec_cluster().
...: Additional arguments passed to methods. Unused arguments are rejected.
icc: Numeric vector of homogeneity measures (length = stages - 1), or a svyplan_varcomp object. The ICC quantifies how similar units within the same cluster are: 0 means no similarity (clusters are as variable as the whole population), 1 means perfect similarity (all units in a cluster are identical). Typical values in household surveys range from 0.01 to 0.10. Higher icc means more clusters are needed for the same precision. Use varcomp() to estimate icc from a previous survey or pilot data.
unit_relvar: Unit relvariance (default 1). For most applications, the default of 1 is appropriate. Non-unit values arise when working with variance components from varcomp() that separate the total variance into stage-specific pieces.
var_ratio: Ratio of the stage components' unit variance to the analysis variable's, default 1. A scalar names var_ratio_psu and, for a three-stage design, derives var_ratio_ssu = var_ratio_psu * (1 - icc_psu), the identity the variance decomposition imposes. Supply a length-2 vector only to override it, which is meaningful when the two stages' ratios come from different decompositions. See design_effect() for the identity and why the design effect would not reduce to var_ratio_psu without it. The default of 1 is appropriate for most designs.
cv: Target coefficient of variation (relative standard error). For example, cv = 0.05 means the standard error of the estimate should be at most 5 percent of the estimate itself. Specify exactly one of cv or budget.
budget: Total budget. Specify exactly one of cv or budget.
n_psu: Fixed number of PSUs (stage-1 sample size). NULL (default) means optimize. For 2-stage, at most one of n_psu or n_per_psu may be specified. For 3-stage, up to two of n_psu, n_per_psu, n_per_ssu may be fixed.
n_per_psu: Fixed cluster size (stage-2 sample size per PSU). NULL (default) means optimize. This is the typical MICS/DHS parameterization where the number of households per cluster is fixed.
n_per_ssu: Fixed SSU take size (stage-3 sample size per SSU). NULL (default) means optimize. Only valid for 3-stage designs.
resp_rate_psu: Expected PSU-level response rate, in (0, 1]. Default 1 (no adjustment). It is the share of selected clusters that can be worked at all, and the stage-1 sample size is inflated by 1 / resp_rate_psu to cover the loss. Nonresponse among the ultimate units inside a cluster is a different quantity that acts at a different stage, and the plain resp_rate carries that meaning elsewhere in the package.
resp_rate_ssu: Expected SSU-level response rate, in (0, 1]. Three-stage designs only, default 1. It is the share of selected second-stage units, typically households, that can be interviewed at all. A 2-stage design has no such stage, since the units inside a PSU are the ultimate ones, and supplying it there is an error.
resp_rate: Expected ultimate-unit response rate, in (0, 1]. Default 1. It is the share of selected final-stage units, typically persons, that respond. This is the plain resp_rate the rest of the package uses. See the nonresponse section of svyplan-package for what this adjustment does and does not claim.
fixed_cost: Fixed overhead cost (C0). Default 0. The total cost model becomes C = C0 + c1*n_psu + c2*n_psu*n_per_psu [+ c3*n_psu*n_per_psu*n_per_ssu]. In budget mode, only budget - fixed_cost is available for variable costs. In CV mode, fixed_cost is added to the variable cost.
indicators: Optional data frame with one row per indicator, which switches the function to the several-indicators mode of Details. Each row carries its own homogeneity and its own precision target, and the allocation returned satisfies every row. Leave it NULL for the single indicator that icc, unit_relvar, var_ratio and cv describe.
domains: Optional character vector naming domain columns in indicators. Each domain is solved on its own unless allocation = "joint" in budget mode. Domains are sized as separate quotas, so $total_n is their sum and $n carries no aggregate stage vector. The fieldable per-domain stage sizes are in $domains, and domain_sampling covers why the quota reading is the only one offered for a multistage design.
allocation: How a budget is split across domains, either "separate" (default, each domain sized on its own) or "joint" (one budget split across domains to minimize the worst precision ratio). "joint" applies only when indicators, domains and budget are all supplied.
domain_sampling: How the per-domain requirements combine into the one overall size. Only "separate" (the default) is available for a cluster design, so the domains are quotas fielded in their own right and $total_n is the sum of the per-domain totals. "natural", which n_multi() offers for a single-stage design, is refused, because a domain's expected yield then depends on how its members sit inside PSUs and SSUs rather than on its share of the population alone, and that model is not in the package.
min_n_domain: Optional positive minimum total sample size per domain. In joint budget mode it is a constraint. With domains solved separately, domains below the floor produce a warning.
plan: Optional svyplan() object providing design defaults (including stage_cost, icc, unit_relvar, var_ratio, resp_rate_psu, fixed_cost). In the several-indicators mode a profile supplies only the defaults that mode accepts, so icc, unit_relvar and var_ratio are left to the indicator columns.

Value

A svyplan_cluster object with components:

n: Named numeric vector of continuous per-stage sample sizes (e.g. c(n_psu = 84.1, n_per_psu = 13.8)), the mathematical optimum.
stages: Number of stages (2 or 3).
total_n: Continuous total sample size (prod(n)).
cv: Coefficient of variation of the continuous optimum.
cost: Cost of the continuous optimum.
operational: The whole-unit field design, found by a discrete search: n (named integer stage sizes), total_n, cost, and cv, all recomputed from the integer design. In budget mode its cost never exceeds the budget, whereas in cv mode it meets the target at the lowest cost among the designs searched. as.integer() returns operational$n, and as.double() returns the continuous n.
params: List of input parameters.

With indicators, the same class carries $indicators, the per-row $detail and, where domains were named, $domains. The class does not depend on which optional arguments were supplied.

Details

Getting started

A typical 2-stage household survey workflow:

Decide what you know. You need the cost per cluster visit (stage_cost[1], e.g. travel + logistics) and the cost per interview (stage_cost[2]), plus an estimate of within-cluster homogeneity (icc). Estimate icc from a pilot or previous survey with varcomp(), or use a plausible range (0.01–0.10 for most household indicators).
Choose a mode. If you have a target precision, set cv. If you have a fixed budget, set budget. Never set both.
Fix stages or let the optimizer decide. In MICS/DHS-style designs, the number of households per cluster is fixed by fieldwork logistics (e.g. n_per_psu = 20). The optimizer then solves for how many clusters (n_psu) to visit. If no stage is fixed, both are optimized jointly.

How it works

Stage count is determined by length(stage_cost):

2-stage (e.g. clusters then households): stage_cost has 2 elements, icc is a scalar.
3-stage (e.g. districts, clusters, households): stage_cost has 3 elements, icc is length 2.

Two solving modes:

CV mode: minimize total cost subject to achieving the target CV.
Budget mode: minimize the CV (maximize precision) within the available budget.

Fixing stage sizes

One or more stage sizes can be fixed, leaving the remaining stage(s) to be optimized or derived from the constraint. For 2-stage designs, at most one stage may be fixed. For 3-stage designs, up to two stages may be fixed. The remaining free stage is derived from the budget or CV constraint.

If icc is a svyplan_varcomp object, icc, unit_relvar, and var_ratio are extracted automatically.

Several indicators at once

indicators sizes one design against several requirements at once. It is a data frame with one row per indicator, and the design returned satisfies every row. The scalar arguments icc, unit_relvar, var_ratio and cv are refused in this mode, because the frame carries each of them per row.

Each row needs p, var, or the ratio quartet (r, cv_num, cv_den, component_cor), a cv, moe or rmoe target, and icc_psu. A three-stage design also needs icc_ssu. Optional var_ratio_psu defaults to 1, and three-stage var_ratio_ssu is derived as var_ratio_psu * (1 - icc_psu) when absent, the value the variance decomposition implies (see design_effect()). On a ratio row icc_psu, icc_ssu and the stage variance ratios describe the linearized variable e = y - r * x, not either component. The remaining columns follow n_multi().

Nonresponse is still named for the stage it acts on. resp_rate_psu, resp_rate_ssu and resp_rate are the defaults used where a row's own column is absent or NA. A column naming a stage the design does not have is an error rather than a column carried along and ignored, since a silently dropped response rate plans a design with none.

Margin-of-error rows are converted to CV before optimization, respecting the row's prop_method. The multistage model is driven by a relative standard error, and only the Wald interval has half-width z * se, so a proportion row's moe is restated as the sampling CV at the effective sample size its own method needs to close the interval to that margin. A stricter interval therefore asks for a larger design, in the same order it does in n_prop(). Under "wald" the restatement is moe / (z * p) exactly. A mean row converts as moe / (z * |mu|), on the magnitude so that a negative mean yields a positive target. prec_cluster() inverts the same way, so a design sized from a moe target reports that moe back.

For each candidate allocation the required stage-1 size is the maximum across rows. The solver minimizes total cost against the precision targets, or the worst precision ratio under a fixed budget. domains sizes each domain on its own, allocation = "joint" splits one budget across them to minimize the worst precision ratio, and min_n_domain sets a floor.

How strong the several-indicators optimum is

With a stage size fixed, the remaining problem is solved from the closed-form cluster optimum. With all stage sizes free, the objective is a maximum over indicator requirements, which is not smooth, and it is minimized by a bounded quasi-Newton search. That search warns when it fails to converge or lands on a bound, but it carries no global-optimality or KKT certificate, so a successful return means the best design this search found, not a proven minimum-cost one. The $operational allocation can always be checked against its own precision or budget constraint, which is a separate and exact statement. n_alloc() gives the stronger guarantee where it applies, returning feasibility and KKT diagnostics for its convex continuous problem.

Boundary icc values

Boundary and near-boundary homogeneity values are not supported by the analytical optimum used here. When icc is near 0, most variability is within PSUs, so the closed-form optimum collapses toward taking many units in very few PSUs. When icc is near 1, most variability is between PSUs, so the optimum collapses toward taking very few units in many PSUs. In both cases the analytical allocation becomes degenerate, so n_cluster() rejects values numerically too close to 0 or 1.

Nonresponse acts at whichever stage it happens

A cluster design can lose units at more than one stage, and the losses are not interchangeable, so each has its own argument named for the stage it acts on. Writing the three-stage CV out with realized takes shows why:

$$cv^2 = \frac{V k_1 \delta_1}{a\,r_{psu}} + \frac{V k_2 (1 + \delta_2 (q r - 1))}{a\,r_{psu}\, m\,r_{ssu}\, q\,r}$$

resp_rate_psu divides both terms, so losing whole PSUs is a pure sample-size loss. resp_rate_ssu divides only the second. resp_rate divides the second and enters the $\delta_2$ bracket, because it shrinks the realized final-stage take and so changes the clustering penalty itself. None is recoverable from the others.

It follows that the later-stage rates move the cost-optimal design while the PSU rate does not. For two stages the optimal take becomes $$b^* = \sqrt{\frac{C_1 (1 - \mathrm{icc})}{C_2\,\mathrm{icc}\,r}},$$ with r the ultimate-unit rate. resp_rate_psu is absent because it scales cost without moving that trade-off. Sizes and costs stay gross: $n counts units to issue and $cost pays for them, while the variance reads what they realize.

This is a deterministic expected-take approximation. It substitutes the expected realized stage counts into the planning variance and assumes response is noninformative for it. Below one expected respondent per unit the substitution stops describing the design, and that is an error naming the approximation rather than the design.

These functions assume sampling fractions are negligible at each stage (equivalent to sampling with replacement). No finite population correction is applied. This is standard for multistage planning when cluster populations are large relative to the sample.

n_alloc() differs here. Its cluster mode carries the ultimate-unit correction 1 - n / N, so the two agree only where the ultimate-unit sampling fraction is negligible. They are the same variance model otherwise, and the clustering bracket, the stage response rates and the cost-optimal take are shared exactly. Size a design through both at an appreciable sampling fraction and n_alloc() returns the smaller answer, by that correction and nothing else.

References

Valliant, R., Dever, J. A., and Kreuter, F. (2018). Practical Tools for Designing and Weighting Survey Samples (2nd ed.). Springer. Ch. 9.

Examples

# 2-stage, budget mode
n_cluster(stage_cost = c(500, 50), icc = 0.05, budget = 100000)
#> Optimal 2-stage allocation
#> field design: n_psu = 80 | n_per_psu = 15 -> total n = 1200
#> cv = 0.0376, cost = 100000
#> continuous optimum: n_psu = 84.08997 | n_per_psu = 13.78405 (cv = 0.0376, cost = 100000)
#> design df = 79

# 2-stage, CV mode
n_cluster(stage_cost = c(500, 50), icc = 0.05, cv = 0.05)
#> Optimal 2-stage allocation
#> field design: n_psu = 52 | n_per_psu = 12 -> total n = 624
#> cv = 0.0498, cost = 57200
#> continuous optimum: n_psu = 47.5681 | n_per_psu = 13.78405 (cv = 0.0500, cost = 56568)
#> design df = 51

# 2-stage, fixed n_psu
n_cluster(stage_cost = c(500, 50), icc = 0.05, budget = 100000, n_psu = 40)
#> Optimal 2-stage allocation
#> field design: n_psu = 40 | n_per_psu = 40 -> total n = 1600
#> cv = 0.0429, cost = 100000
#> continuous optimum: n_psu = 40 | n_per_psu = 40 (cv = 0.0429, cost = 100000)
#> design df = 39

# 2-stage, fixed n_per_psu (MICS/DHS style: 20 households per cluster)
n_cluster(stage_cost = c(500, 50), icc = 0.05, budget = 100000, n_per_psu = 20)
#> Optimal 2-stage allocation
#> field design: n_psu = 66 | n_per_psu = 20 -> total n = 1320
#> cv = 0.0384, cost = 99000
#> continuous optimum: n_psu = 66.66667 | n_per_psu = 20 (cv = 0.0382, cost = 100000)
#> design df = 65

# 3-stage
n_cluster(stage_cost = c(500, 100, 50), icc = c(0.01, 0.05), cv = 0.05)
#> Optimal 3-stage allocation
#> field design: n_psu = 20 | n_per_psu = 6 | n_per_ssu = 5 -> total n = 600
#> cv = 0.0498, cost = 52000
#> continuous optimum: n_psu = 20.24698 | n_per_psu = 4.974937 | n_per_ssu = 6.164414 (cv = 0.0500, cost = 51243)
#> design df = 19

# 3-stage, fixed n_psu + n_per_ssu (solve for n_per_psu)
n_cluster(
  stage_cost = c(500, 100, 50), icc = c(0.01, 0.05),
  budget = 500000, n_psu = 50, n_per_ssu = 8
)
#> Optimal 3-stage allocation
#> field design: n_psu = 50 | n_per_psu = 19 | n_per_ssu = 8 -> total n = 7600
#> cv = 0.0194, cost = 500000
#> continuous optimum: n_psu = 50 | n_per_psu = 19 | n_per_ssu = 8 (cv = 0.0194, cost = 500000)
#> design df = 49

# With fixed overhead cost
n_cluster(stage_cost = c(500, 50), icc = 0.05, budget = 100000, fixed_cost = 5000)
#> Optimal 2-stage allocation
#> field design: n_psu = 76 | n_per_psu = 15 -> total n = 1140
#> cv = 0.0386, cost = 100000 (fixed: 5000)
#> continuous optimum: n_psu = 79.88547 | n_per_psu = 13.78405 (cv = 0.0386, cost = 100000)
#> design df = 75

# Several indicators, CV mode
indicators <- data.frame(
  name    = c("stunting", "anemia"),
  p       = c(0.30, 0.10),
  cv      = c(0.10, 0.15),
  icc_psu = c(0.02, 0.05)
)
n_cluster(indicators = indicators, stage_cost = c(500, 50))
#> Multi-indicator optimal allocation (2-stage)
#> field design: n_psu = 52 | n_per_psu = 12 -> total n = 624
#> worst cv = 0.1495, cost = 57200 (binding: anemia)
#> continuous optimum: n_psu = 47.5681 | n_per_psu = 13.78405 (cv = 0.1500, cost = 56568)
#> 
#>  name     .n       .cv_target .cv_achieved .binding
#>  stunting 292.9922 0.10       0.0668               
#>  anemia   655.6810 0.15       0.1500       *       

# Several indicators with MOE targets, converted to CV internally
by_moe <- data.frame(
  name    = c("stunting", "anemia"),
  p       = c(0.30, 0.10),
  moe     = c(0.05, 0.03),
  icc_psu = c(0.02, 0.05)
)
n_cluster(indicators = by_moe, stage_cost = c(500, 50))
#> Multi-indicator optimal allocation (2-stage)
#> field design: n_psu = 50 | n_per_psu = 12 -> total n = 600
#> worst cv = 0.1525, cost = 55000 (binding: anemia)
#> continuous optimum: n_psu = 45.68272 | n_per_psu = 13.78405 (cv = 0.1531, cost = 54326)
#> 
#>  name     .n       .cv_target .cv_achieved .binding
#>  stunting 405.1863 0.08503558 0.0682               
#>  anemia   629.6929 0.15306404 0.1531       *       

# One budget split across domains
by_domain <- data.frame(
  name    = rep(c("stunting", "anemia"), each = 2),
  p       = c(0.30, 0.25, 0.10, 0.15),
  cv      = c(0.10, 0.10, 0.15, 0.15),
  icc_psu = c(0.02, 0.03, 0.05, 0.04),
  region  = rep(c("Urban", "Rural"), 2)
)
n_cluster(
  indicators = by_domain,
  stage_cost = c(500, 50),
  domains = "region",
  budget = 100000,
  allocation = "joint"
)
#> Multi-indicator optimal allocation (2-stage, 2 domains, joint)
#> Total n = 1232 (unrounded: 1209)
#> 
#>  region n_psu n_per_psu .total_n .cv    .cost .binding
#>  Urban  52    14        728      0.1437 61621 anemia  
#>  Rural  28    18        504      0.0958 38379 stunting