Convert a tbl_sample to a survey design object

Creates a survey::svydesign() object from a tbl_sample, using the sampling design metadata (strata, clusters, weights, and finite population corrections) captured during execute().

Usage

as_svydesign(x, ...)

# S3 method for class 'tbl_sample'
as_svydesign(
  x,
  ...,
  nest = TRUE,
  method = NULL,
  systematic_variance = c("warn", "approximate", "error")
)

Arguments

x: A tbl_sample object produced by execute().
...: Additional arguments passed to survey::svydesign(), or to survey::twophase() for a sample exported as two phases. In particular, you can pass pps = survey::ppsmat(joint_matrix) to supply exact joint inclusion probabilities instead of the default Brewer approximation (see Details). Every argument must be named, and its name must be one the receiving function accepts: nest and method follow the ... and so are matched exactly, and a near miss such as nes is reported rather than forwarded. The design arguments themselves (ids, strata, weights, probs, fpc, data, and subset for a two-phase sample) are derived from the sample and cannot be given here.
nest: If TRUE (the default), relabel cluster ids to enforce nesting within strata, which suits most complex survey designs. Passed to survey::svydesign(). It has no effect on a sample exported with survey::twophase(), and giving it there raises a warning.
method: For two-phase samples, the variance method passed to survey::twophase(). One of "full", "approx", or "simple". This argument is only accepted for two-phase samples. A materialized wave is exported by ultimate clusters unless method is given, which asks for its two-phase export instead.
systematic_variance: What to do about the variance approximation used for systematic stages: simple random sampling for systematic, Brewer's for pps_systematic. "warn" (default) applies it and warns once per call, naming every affected stage and its approximation. "approximate" applies it silently, for a caller who has acknowledged it, and "error" refuses. Census stages are exempt, since a stage that took everything within reach contributes no variance, and so is a pps_systematic first stage exported with a caller-supplied pps object. The choice, the affected stages and each stage's approximation are recorded in the "samplyr_systematic_variance" attribute of the result. as_svrepdesign() takes the same argument for its own approximation of these stages.

Value

A survey.design2 object for single-phase and multi-stage samples and for materialized waves, or a twophase/twophase2 object for two-phase samples and for waves exported with method.

Details

Every executed stage contributes one ids term, one fpc term and, when stratified, one strata term, so survey::svydesign() represents the multi-stage structure in its linearization (Sarndal et al. 1992, ch. 4.3):

Cluster ids (ids): the cluster_by() variable of a clustered stage. A stage clustered by several variables, which execution treats as one cluster id, gets one synthesized interaction column, because survey::svydesign() reads each formula term as a separate stage. A final unclustered stage gets a synthesized row-identity column so that its sampling variance is represented. A WR or PMR stage uses its .draw_k column, and survey treats each occurrence as independent for Hansen-Hurwitz variance estimation. This is exact for WR and the documented approximation for PMR.
Strata (strata): aligned with ids. A stage stratified by several variables exports their cross-classification as one synthesized column, since survey silently ignores extra variables within a stage's term. Trailing unstratified stages are omitted, and unstratified stages before a stratified stage get a constant placeholder column.
Weights (weights): the .weight column, the product of the per-stage weights described in sample_columns. They are design weights. A sample whose .weight was edited is refused, and weights adjusted for nonresponse or calibrated are computed on the exported design, with survey::calibrate() or survey::postStratify().
FPC (fpc): aligned with ids. survey::svydesign() requires every FPC term on the same scale, so one of two encodings is used:
- Count scale (designs without unequal-probability WOR stages): .fpc_k, the stratum population count \(N_h\), for equal-probability WOR stages, and a synthetic Inf column (no correction, Hansen-Hurwitz variance) for WR/PMR stages.
- Fraction scale (multi-stage designs with a PPS WOR, balanced, or custom WOR stage): every WOR stage passes its per-unit stage sampling fraction \(1 / w_k = \pi_k\), and WR/PMR stages pass 0 (no correction). A multi-stage design with a random-size Poisson stage is refused by linearization export.
A single-stage PPS WOR design passes \(\pi_i\) directly, which survey interprets as inclusion probabilities.

For a two-stage stratified-cluster design with a final element stage, the exported call is equivalent to:

survey::svydesign(
  ids     = ~ ea_id + .id_2,       # stage-1 clusters, stage-2 elements
  strata  = ~ region,              # stage-1 strata
  weights = ~ .weight,             # product of per-stage weights
  fpc     = ~ .fpc_pi_1 + .fpc_f_2,  # per-stage sampling fractions
  data    = sample,
  nest    = TRUE
)

Multi-stage designs

Exactness depends on the variance treatment available for each method (variance_estimation). A design whose first stage is a census of PSUs correctly attributes all variance to the later stages. Executing stages separately, as stage1 <- execute(design, psu_frame, stages = 1) then sample <- execute(stage1, listing_frame), still gives one multi-stage design, exported with survey::svydesign().

An unclustered element-sampling stage followed by further stages is not nested cluster sampling, because the later selections are conditional on the realized element sample, which is phase sampling. execute() refuses it (samplyr_error_stage_parent_id). Express it as a two-phase sample instead: execute the element stage under its first-phase design, then a new second-phase design with that sample as its frame.

Two-phase samples

A new phase-2 sampling_design executed with the phase-1 tbl_sample as its frame, as in phase2 <- execute(design2, phase1), records a previous-phase link, and as_svydesign() calls survey::twophase().

A materialized wave exported with method is the other two-phase case. Its second phase is the panel activation: within a frozen block the master took a simple random sample without replacement of the realized quota, so the blocks are the phase-2 strata and their sizes the phase-2 population counts. The master is retained as the first phase and supplies the rows the wave did not keep. Columns added to the wave for analysis are carried into the exported design. Wave or phase-2 analysis columns replace same-named first-phase columns and are missing on unsampled rows. Columns absent from the wave remain available from the master, so dropping a wave column does not erase the master's measurements. Design identifiers, strata and internal sampling columns keep their recorded meanings. Keep earlier measurements under separate names if both are needed.

When phase 2 keeps unequal shares of the phase-1 strata, survey 4.5 gives each phase-1 stratum another stratum's sample size in the phase-1 part of this variance. The variance of a total then depends on the outcome's level. In simulations of a wave it averaged a quarter of the true variance or less and was often negative. The export rebuilds that part by survey's own formula with the sizes in place, which changes nothing where survey's result was already right. A phase 2 stratified across the phase-1 strata can still give a negative variance in some samples, from the instability of the Horvitz-Thompson form, and method = "approx" gives a finite standard error there.

A clustered first phase needs its second phase drawn inside each of its units, with stratify_by(<unit>), or by taking whole phase-1 units before sampling inside them. In a simulation both gave 0.93 to 1.03 of the true variance and never a negative one. When phase 2 draws smaller units across the phase-1 clusters, the estimator is right over repeated samples but not in one (variance_estimation has the measured error), and the export warns (samplyr_warning_twophase_across_units). The weights are exact either way.

One shape is not exported as two phases. Phase 1's final stage draws elements by srswor or systematic, and phase 2 draws the same way, without clusters, with one stratum per final-stage pool of phase 1, as in stratify_by(ea_id) when the final stage draws households within EAs. A simple random subsample of a simple random sample is a simple random sample, so the two phases are phase 1's multistage design with a smaller final take. as_svydesign() and as_svrepdesign() export it that way, with phase 1's strata, clusters and finite population corrections and phase 2's weights, whatever phase 1's earlier stages drew. Half of each EA's households kept for a second questionnaire is this shape. method does not apply to it, and no row identifier is needed to link the phases. When phase 2 is systematic, the final take is a systematic one, and systematic_variance applies to it, unless phase 2 took every unit of every pool. A phase 2 stratified more coarsely or more finely than the pools, clustered, or drawn with unequal probabilities keeps the two-phase route.

survey::twophase() takes no pps for the first phase, so any other two-phase sample, or wave exported with method, whose phase 1 was drawn with unequal probabilities without replacement is refused (samplyr_error_twophase_phase1_pps, samplyr_error_wave_phase1_pps). So are cube, first-stage pps_poisson and the spatial methods, which are unsupported at either phase. Equal-probability first phases export, stratified, clustered, multistage and with-replacement alike. No two-phase sample has a replicate export, except the subsample within pools above, which is exported as one phase. The weights of a refused sample are exact, so totals and means are right. For a variance, an ultimate-cluster approximation treats phase 1's units as drawn with replacement and carries everything below in them. It is conservative, by the margins variance_estimation reports, and much more so when phase 2 was drawn across the phase-1 units:

survey::svydesign(ids = ~ea_id, strata = ~region, weights = ~.weight,
                  data = as.data.frame(phase2))

Waves

A materialized wave, execute(master, wave = t), is exported as one phase by ultimate clusters. The master's stage-1 units are the clusters, its stage-1 stratification variables the strata, and the wave's weights the weights, with no finite population correction. Every later stage and the activation are carried in the unit totals, so the master may have been drawn by any method, with probabilities proportional to size included. It is the variance rotating household surveys publish for each quarter or month. Zones and variance pairs are not used, since activation leaves most zones with one unit or none. A stage-1 unit taken with certainty, with a later stage drawn inside it, represents itself. It is a stratum of its own, whose clusters are its units of the next stage, since only they vary between samples. In a simulation of one certainty EA among twelve, that brought the estimated variance of a wave from 1.7 to 1.1 times the true one, and of a two-wave window, where the EA otherwise stood alone among the units active at both waves and survey stopped on it, from 5.7 under survey.lonely.psu = "adjust" to 1.6. A unit taken with certainty and nothing drawn inside it, a stage-1 cluster with no later stage or an element, is the same in every sample. It is a stratum of its own taken as a census, the one stratum with a finite population correction, so it adds no variance, as in the master's export. A large sampling fraction in a stratum still errs toward too large a variance. In a simulation with 8 EAs active per stratum, its standard error was 0.89 of the true one with EAs drawn by simple random sampling, where the two-phase export gave 0.88, and 1.06 with EAs drawn by size, which has no two-phase export.

A window of several waves, execute(master, wave = 1:3), is exported the same way. A unit's rows over the window are one cluster, and the strata are crossed with the number of the window's waves a unit is active, which the schedule fixes and which sets the unit's share of the window's weight. In a simulation of windows over masters of 16 to 18 EAs per stratum, the standard error was 0.95 to 1.12 of the true one, and 95% intervals covered 92% to 97%, with EAs drawn by simple random sampling or by size. method is refused for a window.

A wave of a master whose later stages are still drawn one period or tenure at a time is exported the same way, from its own rows. The two-phase export reads the whole master and refuses it (samplyr_error_wave_provisional).

as_svrepdesign() replicates the same design, and as_survey_design() follows as_svydesign(). pps is refused (samplyr_error_survey_argument), and so is a wave holding a single primary unit (samplyr_error_wave_one_unit). A wave that activates every unit is the master's own design and is exported as such.

Modified samples and domain analysis

as_svydesign() raises an error on a tbl_sample whose row set changed after execute() (rows removed by dplyr::filter() or [, added, or duplicated by a join) or whose internal design columns (.weight, .weight_k, .fpc_k, ...) were overwritten, dropped, or renamed. The sample is verified against an integrity record stored at execution (row count and a hash of the weights, design metadata, and strata and cluster columns), so changes the dplyr hooks cannot see (base assignment, rbind(), vctrs operations, third-party verbs) are also caught, and an overwrite that left every value identical passes. Row reordering, one-to-one joins, and adding ordinary data columns do not mark the sample. One complete replicate extracted from a replicated execution (filter(.replicate == r)) is verified against the execution metadata and remains supported. A later stage drawn from a modified sample is refused too, since its weights carry the change, and so is every stage drawn after it.

Dropping out-of-domain rows before conversion is not domain estimation. The point estimate can agree, but its variance estimate is generally wrong, often too small, because the domain sample size is random under the design. Convert the full sample first and then subset the design:

svy <- as_svydesign(sample)
survey::svymean(~y, subset(svy, domain))
# or with srvyr:
as_survey_design(sample) |> filter(domain) |> summarise(...)

Variance by selection method

Which estimator each selection method reaches, how exact it is, and which designs are refused are in variance_estimation.

pps is read as follows. "brewer" is the default treatment and keeps every stage. FALSE states that no stage is PPS, so it is refused on a design with one. "overton", survey::HR() and the matrix objects (survey::ppsmat(), survey::poisson_sampling()) are single-stage in survey, so a multi-stage sample is exported at stage 1 with a warning. A matrix object is indexed by row, so it also needs one row per stage-1 unit, and a sample with several is refused (samplyr_error_pps_rows_per_psu). Any other value is refused (samplyr_error_pps_argument). A two-phase export takes no pps, because survey::twophase() does not apply it to the phase-2 variance (samplyr_error_twophase_phase2_pps). A phase 2 of one stage drawn with pps_sampford, pps_cps, pps_brewer, pps_sps or pps_pareto is exported with its joint inclusion probabilities, computed on the phase-1 sample, and needs method = "full". Any other unequal-probability, balanced or spatial phase 2 is refused with the same class.

Ultimate-cluster variance

Many survey programs publish sampling errors computed as if the primary units were drawn with replacement within their strata, without a finite population correction. The variance of every later stage is then carried by the primary-unit totals, which is why it is called the ultimate-cluster variance. It is the variance to reproduce when a new design is compared with what an earlier round published. as_svydesign() builds it for a materialized wave and for a settled reserve only. For other samples it derives ids, strata and fpc from the design, and survey builds the ultimate-cluster design from the sample's own columns:

survey::svydesign(
  ids = ~ea_id,
  strata = ~interaction(region, urban_rural, drop = TRUE),
  weights = ~.weight,
  nest = TRUE,
  data = as.data.frame(sample)
)

ids names the first stage's cluster variable and strata its stratification variables. The approximation is conservative. It leaves out the finite population correction, so it overstates the variance most where a stratum selects a large share of its primary units, and it treats certainty units as sampled. A stratum with a single primary unit stops survey, as it does for the export, unless options(survey.lonely.psu = ) says how to treat it.

A settled reserve

reserve_lists describes reserve lists and their settlement. A sample whose reserve is not settled is refused (samplyr_error_reserve_unsettled), since its weights describe the whole list. Once settled with execute(sample, response_status = col), it is exported by ultimate clusters. The settled stage's respondents are a random subset of a respondent count the field never observed, so that stage's finite population correction is unknown, and every stage is carried in the primary-unit totals. The share of each pool's eligible units that responded is treated as fixed. A two-phase sample, shared weights, and the "mrbbootstrap" and "rwyb" replicates of as_svrepdesign() would build the stages one by one, and refuse a settled reserve (samplyr_error_reserve_full_export). A wave holds no settlement of its own, so a wave of a master that draws a reserve is refused with the same class, settled or not, and the settled master exports as a whole.

With the reserve at the first stage, each pool's eligible units are estimated from the units the field worked. The export adds one row for each unit found ineligible among those the estimate counts, at the pool's size over the number of units counted. The pool's weights then add up to its size, and the variance treats the eligible count as estimated. These rows hold NA in every column except the strata, the weight and, while the sample still holds it, the response status column, which carries the ineligible code in the column's own type (4, "ineligible", or a factor level of that name, added if absent). Estimate for the eligible population as a domain:

svy <- as_svydesign(settled)
survey::svytotal(~income, svy, na.rm = TRUE)
survey::svymean(~income, subset(svy, result != "ineligible"))

na.rm = TRUE and subset() keep those rows in the variance, and removing them before the export understates it. A primary unit of a later reserve stage whose units all proved ineligible keeps one such row, as a zero total.

A primary unit taken with certainty, with a later stage drawn inside it, represents itself as it does in a wave. It is a stratum of its own whose clusters are its units of the next stage. With the reserve at that stage, its pools are exported as first-stage pools are, times the weight of the stage above, so its ineligible units add rows and a placeholder is a unit of its own. In a simulation of one certainty EA among twelve, this brought the estimated variance from 1.36 to 1.26 times the true one. The rest comes from the first stage's finite population correction, which the export leaves out.

A pool whose eligible units include no respondent still has an estimated eligible count, from the units the field worked. With every worked unit of unknown eligibility, it takes the resolved rate of the pools that share its strata, or counts every unit as eligible when none of them resolved one. Both are assumptions about the pool, and summary() counts the pools that rest on them. A pool estimated to hold no eligible unit, such as one whose resolved units all proved ineligible, is exported as ineligible. The export adds one placeholder row per other such pool, at the weight of the stages above times that count, coded as an eligible non-interview (2, "noninterview", or a factor level of that name). Its other columns hold NA, and below the first stage it sits in the primary unit of its pool. The export warns (samplyr_warning_reserve_no_respondent) and names the pools. Until a nonresponse adjustment moves that weight to respondents, estimates leave it out. With replicate weights, svrep::redistribute_weights() makes the adjustment in every replicate:

rep <- as_svrepdesign(settled)
rep <- svrep::redistribute_weights(
  rep,
  reduce_if = result == "noninterview",
  increase_if = result == "interview",
  by = "region"
)
survey::svytotal(~income, rep, na.rm = TRUE)

The adjustment moves the placeholder's weight but leaves its row, with NA in every analysis variable. An estimate therefore needs na.rm = TRUE, as for the ineligible rows. Without it a linearized total is NA, and a replicate design fails with "All replicates contained NAs".

With linearization, survey::calibrate() to known totals plays that role. After a stage below the reserve is drawn, the units of such a pool have no weight at that stage, so its export is refused (samplyr_error_reserve_no_respondent). Export before drawing it, or combine the stratum with a neighbor in the design.

The rows the export adds hold NA in the variables a calibration would use, so survey::calibrate(), survey::postStratify() and survey::rake() stop on them. Calibrate the respondents instead, through indicators that are 0 on those rows, and without an intercept, which would count them:

svy <- update(
  svy,
  female = as.numeric(sex %in% "female"),
  male = as.numeric(sex %in% "male")
)
svy <- survey::calibrate(
  svy, ~ 0 + female + male, c(female = 5200, male = 4800)
)

A row whose indicators are all 0 keeps its weight, under a linear calibration as under calfun = "raking". The ineligible rows still estimate the ineligible count, and the respondents alone carry the known totals. The same call works on a replicate design, before or after svrep::redistribute_weights().

A shared estimation weight

A sample from share_weights() carries weights for a population other than the one that was selected, so it is exported as its source-target contributions: one row per link, weighted by the recorded coefficient times the source unit's design weight. This gives the generalized weight share total exactly for any link structure. The result has more rows than the transformation returned, but no estimate changes: a total sums the same terms, and a mean's denominator is the estimated target population size either way.

A selected source unit linked to no target, such as a dwelling with nobody in the target population, has no contribution. Where no other unit of its primary unit contributes, it must still count in the variance between primary units, so the export keeps it as a zero row, as it keeps a unit found empty: weight 0, its source values, and NA in the target variables, so estimates need na.rm = TRUE. Inside a primary unit that has other contributions, the primary unit's total is right without it, and it has no row. The export then warns (samplyr_warning_export_empty_parent) that survey computes the variance within that primary unit without it, which matters at large first-stage sampling fractions: with 4 of 6 blocks drawn per stratum, a test gave a standard error 20 percent short. as_svrepdesign() replicates the complete source design, so its variance holds every unit.

An unequal-probability or random-size source design is also refused here, because its variance comes from a structure indexed by the source sample's rows, which the contribution rows no longer are. Use as_svrepdesign(), which replicates the source design and applies the sharing inside every replicate.

The survey package is required but not imported. It must be installed to use this function.

References

Sarndal, C.-E., Swensson, B. and Wretman, J. (1992). Model Assisted Survey Sampling. Springer.

Examples

# Stratified sample -> survey design
sample <- sampling_design() |>
  stratify_by(region, alloc = "proportional") |>
  draw(n = 300) |>
  execute(bfa_eas, seed = 42)

svy <- as_svydesign(sample)
survey::svymean(~households, svy)
#>              mean     SE
#> households 71.668 3.6922

# Two-stage sample: PPS selection of EAs, then households from a listing
selected <- sampling_design() |>
  add_stage() |>
    stratify_by(region) |>
    cluster_by(ea_id) |>
    draw(n = 5, method = "pps_brewer", mos = households) |>
  add_stage() |>
    draw(n = 8) |>
  execute(bfa_eas, stages = 1, seed = 2025)
listing <- selected |>
  as.data.frame() |>
  dplyr::reframe(hh_id = seq_len(households), .by = ea_id)
sample <- execute(selected, listing, seed = 2026)

# Brewer's variance approximation at the PPS stage
svy <- as_svydesign(sample)

# Half of each EA's households, kept as a second phase. It is exported as
# the two-stage design with a take of 4.
half <- sampling_design() |>
  stratify_by(ea_id) |>
  draw(n = 4) |>
  execute(sample, seed = 2027)
svy_half <- as_svydesign(half)

# A joint-probability matrix instead, for a single-stage Sampford sample,
# whose matrix is exact. A matrix needs one row per stage-1 unit.
sampford <- sampling_design() |>
  stratify_by(region) |>
  draw(n = 5, method = "pps_sampford", mos = households) |>
  execute(bfa_eas, seed = 2025)
jip <- joint_expectation(sampford, bfa_eas)
svy_joint <- as_svydesign(sampford, pps = survey::ppsmat(jip[[1]]))
survey::svytotal(~population, svy_joint)
#>               total     SE
#> population 20854071 297547