Execute a sampling design

execute() runs a sampling design against one or more data frames, producing a sampled dataset with appropriate weights and metadata.

Usage

execute(
  .data,
  ...,
  stages = NULL,
  seed = NULL,
  panels = NULL,
  panel_stage = NULL,
  small_pool = NULL,
  reps = NULL,
  wave = NULL,
  frame_digest = c("summary", "full", "none")
)

Arguments

.data

A sampling_design object to start a new execution, or a partially executed tbl_sample to continue the remaining stages of its stored design.

...

Data frame(s) to sample from. For single-stage designs, provide one frame. For multi-stage designs with separate frames, provide frames in stage order. Passing a tbl_sample here while .data is a new sampling_design starts a new sampling phase. It does not continue the stages stored in that sample. Ordinary input frames must have unique names and must not use columns reserved for execution output, such as .weight, .sample_id, .stage, .weight_k, or .fpc_k, where k is the stage number (.weight_1, .fpc_1, and so on). Frames are matched positionally, so any name given here is a label. A label resembling one of the arguments below (seedd, or the singular stage, rep, panel) is refused rather than read as a frame, because those arguments follow ... and are matched exactly.

A single unnamed list of data frames is read as those frames in that order, which is the same call written with a value rather than one argument per frame. Write them out when you are typing the call. The list is for code that already holds them, such as a loop over registers or replay_design(). Mixing the two is refused. Names inside the list are frame labels, so a misspelled argument must be written outside it to be reported as one.

stages

Integer vector specifying which stage(s) to execute. From a sampling_design, the vector must start at stage 1 and this is how an operational workflow stops after its first contiguous batch of stages. From a partial tbl_sample, it must start at the next unexecuted stage. Default (NULL) executes all remaining stages.

seed

Integer random seed for reproducibility, between -.Machine$integer.max and .Machine$integer.max.

panels

Rotation groups (panels) to partition the sample into for rotation or workload management, as an integer count, a rotation schedule, or an svyplan_schedule from svyplan::design_schedule(). Assignment is a randomized fixed-quota partition within the assignment stage's selection strata: every unit carries each panel with probability 1 / panels. It is not an additional probability-sampling phase. The output includes a .panel column with values 1 through the panel count. Default NULL means no panel partitioning. Cannot be used together with reps, and cannot be redeclared on a sample that already carries an assignment.

A schedule is a data frame with an integer panel column, an integer wave column and an optional logical active column. A combination left out is inactive. It declares the panel count and, through the fewest panels any wave activates, the assignment block size. Only a sample drawn with a schedule can be materialized by wave.

For an svyplan_schedule, execute() extracts the startup activity and checks its panel parameters before assignment. A gradual launch has one whole startup cohort and needs no partition. Permanent activation is not part of this automatic route.

panel_stage

Stage whose selected units are assigned to panels, as a single stage number, or NULL (the default) for the first executed stage. Accepted only alongside panels, and the stage must be one the execution completes. Every later stage inherits its ancestor's panel, so assigning at stage 1 rotates whole primary units while assigning lower down rotates units inside parents that stay in the survey: the address-panel design, in which selected areas are retained and households rotate within them.

Pools are then the assignment stage's strata inside each realized parent, so they are smaller than first-stage pools and small_pool matters more. Selection certainty counts only at the assignment stage, in both directions: a certainty primary unit does not make its households permanent, and a certainty selection below the assignment stage does not keep its household in every wave. A stage that selects with replacement assigns realized draw occurrences, so one population unit selected twice may take two different panels. A parent selected twice likewise gives two separate populations of households, and a household reached under both hits is assigned once for each.

Assigning below the first stage is not a default and should not be treated as one. Holding the parent fixed while its members rotate can bias cross-sectional estimates over time, and a unit that cannot move between parents is balanced for net change but not for gross change.

small_pool

What to do when a rotation schedule would leave a pool with no unit to activate, as "error" (the default, reached by NULL) or "permanent". A pool of m units leaves panels - m panels empty, so a wave activating r of them selects nothing from that pool when m <= panels - r. Those units then have inclusion probability zero in that wave rather than a small weight, so the wave's estimator is biased. "error" refuses such an assignment, before any panel is drawn, and names the pools. "permanent" instead activates them at every wave with probability one, warning that it has done so, which is exact but changes the operational design: wave sizes, overlap and repeated interviewing all increase. Meaningful only with a schedule, since a panel count declares no wave to protect. This governs positivity only: a pool with a positive but single active unit is still assigned, and is still marked as carrying no within-block variance estimate. An svyplan_schedule refuses "permanent" because its overlap describes a fully rotating life.

reps

Integer number of independent replicate samples to draw (>= 2), or NULL (default) for a single sample. When specified, execute() draws reps independent samples from the same frame under the same design and returns a single stacked tbl_sample with a .replicate column (integer 1 through reps). Replicate r uses seed seed + r - 1. The complete sequence must remain within R supported integer range. Cannot be combined with panels or with stages that use permanent random numbers.

This is repeated sample realization (drawing multiple independent samples), not replicate-weight variance estimation. For the latter, see as_svrepdesign().

wave

Integer wave of a scheduled master to materialize, or NULL (default). execute(master, wave = t) activates the panels the stored schedule declares active at t and compounds the exact activation factor into .weight. It takes no other execution input: no frame, no seed, stages, panels, small_pool or reps, since a wave selects units the master already assigned, under the policy the master froze.

frame_digest

Controls the frame digest, a compact execution manifest recorded with the sample and read by frame_summary(). "summary" (default) records anonymous population structure: selection pools, resolved chances (exact for cluster stages, constant or quantile-compressed for element stages), and the selected-unit trace. "full" keeps exact per-unit chances for element stages too. "none" records no digest and skips trace construction for minimum execution overhead. When one universe frame feeds every stage, later stages also record the pools their realization never reached, with chances resolved deterministically from the design (chance_status = "design_resolved"), this gives frame_summary() and downstream digest consumers complete universe denominators without the frame. The digest never affects selection, weights, or estimation. Design executions record a digest, replicated executions share the population structure across replicates with replicate-specific traces, and a stage continuation extends the digest carried by its input sample (an input without a valid digest yields no digest). In a replicated multi-stage execution, later-stage pools depend on each replicate's realized parents, so the digest keeps the stage prefix shared by all replicates and reports status "partial". Replicated multi-phase and replicated-continuation executions do not record one yet.

Value

A tbl_sample: a data frame subclass carrying the selected rows, the design that produced them, and generated columns recording the selection. Those are .sample_id, .weight, the per-stage .weight_k, .fpc_k, .draw_k and .certainty_k, and .replicate or .panel when reps or panels is used. sample-columns documents what each one holds.

Details

Every pattern below has a worked example under Examples.

Multi-stage with a single frame

For hierarchical data where all stages are in one frame, pass that one frame. It must contain all clustering variables and represent the stage hierarchy correctly. Lower-stage IDs may repeat across different parents. samplyr resolves them using the full ancestry from earlier stages.

Multi-stage with one frame per stage

When each stage has its own register, pass one frame per stage. Frames map to stages by position, and the number of frames is what schedules the stages: one frame is a shared hierarchy covering all of them, and one frame per stage gives each its own. Any other count is samplyr_error_frame_count.

Registers are supplied whole. Each is restricted to the units its parent stage selected, and the variables earlier stages introduced are carried onto it, so a lower register neither has to be pre-filtered nor to duplicate the upper stages' stratification columns. A register that legitimately omits a carried stratum is fine. One whose own copy of it disagrees is an error.

The frames may also be held as a list, which is the same call. Inside that list a name is a diagnostic label for the frame, not an argument name. Mixing the two spellings in one call is refused.

This form, the single-hierarchy form, and the stage continuation below run the same stage transition. Under one shared RNG stream, and with stages given on every intermediate call, all three draw the same sample.

Partial execution (operational sampling)

stages executes only the stages named, returning a partial tbl_sample. Fieldwork then produces the next stage's frame, and passing the partial sample back as .data continues the same design. Omitting stages on the continuation executes every remaining stage. Where more than one stage remains and one frame is supplied, that frame could be the next stage's register or a hierarchy covering the rest, and the two draw different samples. execute() refuses to guess and asks for stages (samplyr_error_ambiguous_continuation). validate_frame() applies the same rule, so a frame it approves is one this call accepts.

When the listing frame is derived from a tbl_sample (e.g. via tidyr::uncount() or dplyr::slice()), it may carry internal columns (.weight, .fpc_1, etc.) from the earlier stage. These are automatically stripped before sampling so they do not collide with the metadata carried by the stage-1 result. Pass the unmodified stage-1 result as .data and the expanded listing as the frame, as above. Passing the original design as .data instead starts a new execution at stage 1 and treats a tbl_sample frame as a previous sampling phase. It is not a stage continuation. When an intact frame is a strict partial result of that same design, execute() warns about this ambiguity but permits it because it is a valid new-phase operation and will export through survey::twophase(). If a class-dropping operation such as tidyr::uncount() leaves a plain listing with sampling attributes or generated columns, execute() refuses to use it as an ordinary frame for a fresh design execution. It remains a valid listing frame when the unmodified partial sample is .data. To use such rows as a genuinely unrelated ordinary frame, remove both the sampling attributes and the generated sample columns explicitly.

Multi-phase sampling

To start a new phase, use the new phase's design as .data and pass the previous phase's tbl_sample as its frame. This is distinct from stage continuation: phase1 is a frame for a new design, rather than .data carrying unexecuted stages of the same design. Weights compound automatically in multi-phase designs, and as_svydesign() exports this path through survey::twophase().

Weight calculation

The .weight column is the inverse of the selection chance that samplyr resolves for the unit based on the first-order inclusion probability for without-replacement methods, or the expected number of selections for with-replacement methods. The per-stage weight is $w_i^{(k)} = 1 / \pi_i^{(k)}$:

SRS: $w_i = N / n$, constant for all units.
Stratified SRS: $w_i = N_h / n_h$ within stratum $h$.
PPS WOR: $w_i = 1 / \pi_i$ where $\pi_i$ is computed from the measure of size by sondage::inclusion_prob(). Varies across units.
WR / PMR: $w_i = 1 / E(n_i)$ where $E(n_i) = n \cdot p_i$ is the expected number of selections. Each draw is one row. A unit selected $k$ times appears $k$ times, each with the same weight.

For every built-in method except "pps_sps" and "pps_pareto", the resolved chance equals the design's true first-order inclusion probability (or expected hits), so .weight is the inverse of that true probability. The order-sampling pair, and registered methods declared probabilities = "approximate", honor the resolved chance only to a documented approximation: .weight is then the inverse of the target probability, not of the design's true first-order inclusion probability. Weighting by the inverse target is standard practice for these methods, and the deviation is typically small, but it is a bias of the method, not of samplyr. The tier is recorded per stage in the frame digest, reported by frame_summary() as the probabilities column, and flagged by summary().

Multi-stage weight compounding

In a $K$-stage design, the overall weight for unit $i$ is the product of per-stage weights: $$w_i = \prod_{k=1}^{K} w_i^{(k)} = \prod_{k=1}^{K} \frac{1}{\pi_i^{(k \mid S^{(k-1)})}}$$ where $\pi_i^{(k \mid S^{(k-1)})}$ is the conditional inclusion probability at stage $k$, given the set of clusters selected at all prior stages. For example, in a two-stage design where 5 of 30 EAs are selected in a region (stage 1) and 12 of 50 households are listed within each selected EA (stage 2): $$w_i = \frac{30}{5} \times \frac{50}{12} = 6 \times 4.17 = 25$$ The .weight column always equals the product of .weight_1, .weight_2, etc. Per-stage weights are preserved for diagnostics and for survey export.

Multi-phase weight compounding

When a new phase's design is executed with a previous-phase tbl_sample as its frame, the phase-1 inclusion probability is already reflected in the input weights. The final .weight is the product of phase-1 and phase-2 weights: $$w_i = w_i^{(\text{phase 1})} \times w_i^{(\text{phase 2} \mid \text{phase 1})}$$ This ensures the Horvitz-Thompson estimator $\hat{Y} = \sum_S w_i \, y_i$ is unbiased for the population total.

Panel partitioning

When panels is specified, the sample is partitioned into non-overlapping groups for rotation or workload management.

Assignment is randomized with fixed quotas. Within each selection stratum of the assignment stage the assignment units are ordered, cut into consecutive blocks of 2 * panels, given a fixed quota per panel inside each block, and permuted within their block. Every unit therefore carries each panel with probability 1 / panels, and panel sizes within a pool differ by at most one.

Blocking is what preserves order. Units adjacent in the control order of the assignment stage's draw() fall in the same block, so every panel inherits the same spread over that order. A pool holding fewer than 2 * panels units is a single block: still assigned, simply with no block-level order structure left to preserve.

For multi-stage designs, panels are assigned at stage 1 by default, and every unit below inherits its ancestor's panel. panel_stage moves the assignment to another stage, which is what expresses a design that retains its primary units and rotates the units inside them. A pool is then the assignment stage's strata inside each realized parent and never crosses a parent. Under a with-replacement assignment stage the assignment unit is the realized draw, the unit the estimator uses, so one population cluster drawn twice may carry two different panels.

Certainty units are labelled from their own pools and consume no rotating quota. A certainty unit is in the sample at every occasion, so a schedule that rotated it out would sample the very units the certainty stratum exists to enumerate. Certainty counts only at the assignment stage, in both directions: a certainty primary unit does not make the units below it permanent when panel_stage names a lower stage, and a certainty selection at a stage below the assignment stage does not keep its assignment unit in every wave. A unit selected with probability one inside a rotating parent is absent from the waves that parent sits out.

Panels are assigned once. .panel is carried forward by a stage continuation, and redeclaring panels on a sample that already carries an assignment is an error.

Rotation schedules and waves

Passing a schedule to panels instead of a count declares which panels are active at which occasion, and lets execute(master, wave = t) materialize one of them:

schedule <- data.frame(
  panel  = rep(1:4, times = 3),
  wave   = rep(1:3, each = 4),
  active = c(TRUE, TRUE, FALSE, FALSE,
             FALSE, TRUE, TRUE, FALSE,
             FALSE, FALSE, TRUE, TRUE)
)
master <- execute(design, frame, seed = 1, panels = schedule)
wave_2 <- execute(master, wave = 2)

The schedule is read at the master draw, not only at materialization, because the block size follows from it. A schedule whose leanest wave activates r of the k panels blocks at k * ceiling(2 / r) rather than at the worst case 2k, which keeps more of the assignment order while still leaving two units per block in the take.

Materializing wave t selects the panels declared active at t and multiplies .weight by the inverse of the activation probability. That probability is the block's frozen quota for the active panels over the block size, so it is exact rather than nominal, and it is generally not k / r. Permanent certainty units are activated at every wave with probability one and their weights are untouched.

A materialized wave is a sample in its own right, with its own integrity record: it is not a filtered master, and frame_summary() and the weight diagnostics work on it. It retains the master as its first phase, so as_svydesign() exports it through survey::twophase() with the activation as the second phase. What it cannot do is replay, because a wave is derived from a recorded execution rather than being one. That refuses rather than answering approximately.

The schedule states which groups are live when. It does not replenish the sample: every panel comes from the frame vintage the master was drawn from. Steady-state replenishment is a fresh execute() against a later frame. See vignette("rotating-panels").

Weights are not adjusted for panel membership. They reflect the full-sample inclusion probability and are valid for the combined sample. Taking a subset of the panels is a simple random subsample without replacement within each block, but its conditional probability is the block's realized quota over the block size, not 1 / panels, so multiplying one panel's weights by panels is not generally valid for population inference. The block sizes and realized quotas are recorded with the sample and written to the design file by write_design(), because they, not 1 / panels, are what such a subset has to be computed against.

When a design cannot be realized as written

A pool holding fewer units than the stage asks for is selected whole, which makes the design non-self-weighting. execute() reports this, and four related outcomes, as classed conditions carrying a payload.

frame_digest defaults to "summary", so the ordinary way to read capping after the fact is the capped column of frame_summary(sample, detail = "pool"). See execution-conditions for the five classes, their payload fields, and how to capture one when no digest is kept.

Examples

# Basic SRS execution
sample <- sampling_design() |>
  draw(n = 100) |>
  execute(bfa_eas, seed = 1234)
sample
#> # A tbl_sample: 100 × 18
#> # Sampling:     1 stage | 100/44,570 units
#> # Weights:      445.7 [445.7, 445.7]
#>    ea_id region      province commune urban_rural population households area_km2
#>  * <int> <fct>       <fct>    <fct>   <fct>            <int>      <int>    <dbl>
#>  1  6473 Sahel       Soum     Koutou… Rural              133         23    19.4 
#>  2  6543 Sahel       Soum     Koutou… Rural              183         32     6.15
#>  3 10735 Sahel       Yagha    Tankou… Rural               36          5    16.7 
#>  4 36151 Centre-Nord Namente… Boulsa  Rural              526         75     3.75
#>  5 10510 Nord        Loroum   Sollé   Rural              239         28     5.45
#>  6 23624 Nord        Yatenga  Senguè… Rural              555         59     8.84
#>  7 33774 Centre-Nord Sanmate… Pissila Rural              857        110     4.94
#>  8  1118 Centre-Nord Namente… Boala   Rural              830        137    10.2 
#>  9 29691 Centre-Oue… Sissili  Bieha   Rural              190         28     8.83
#> 10 10559 Boucle du … Kossi    Sônô    Rural               69         10     8.89
#> # ℹ 90 more rows
#> # ℹ 10 more variables: pop_density <dbl>, longitude <dbl>, latitude <dbl>,
#> #   remoteness <fct>, fieldwork_cost <int>, .weight <dbl>, .sample_id <int>,
#> #   .stage <int>, .weight_1 <dbl>, .fpc_1 <int>

# Stratified execution with proportional allocation
sample <- sampling_design() |>
  stratify_by(region, alloc = "proportional") |>
  draw(n = 300) |>
  execute(bfa_eas, seed = 5789)
table(sample$region)
#> 
#> Boucle du Mouhoun          Cascades            Centre        Centre-Est 
#>                34                17                26                20 
#>       Centre-Nord      Centre-Ouest        Centre-Sud               Est 
#>                23                25                11                37 
#>     Hauts-Bassins              Nord   Plateau-Central             Sahel 
#>                32                20                11                28 
#>         Sud-Ouest 
#>                16 

# Two-stage cluster sample execution
zwe_frame <- zwe_eas |>
  dplyr::mutate(district_hh = sum(households), .by = district)

sample <- sampling_design() |>
  add_stage(label = "Districts") |>
    cluster_by(district) |>
    draw(n = 20, method = "pps_brewer", mos = district_hh) |>
  add_stage(label = "EAs") |>
    draw(n = 10) |>
  execute(zwe_frame, seed = 3)
length(unique(sample$district))  # 20 districts selected
#> [1] 20

# Partial execution: stage 1 only
design <- sampling_design() |>
  add_stage(label = "EAs") |>
    stratify_by(region) |>
    cluster_by(ea_id) |>
    draw(n = 5, method = "pps_brewer", mos = households) |>
  add_stage(label = "Households") |>
    draw(n = 12)

# Execute only stage 1 to get selected EAs
selected_eas <- execute(design, bfa_eas, stages = 1, seed = 2)
nrow(selected_eas)  # Number of selected EAs
#> [1] 65

# Continuation: the listing produced by fieldwork becomes the next frame,
# and the partial sample is passed back as `.data`
listing <- selected_eas |>
  dplyr::slice(rep(seq_len(dplyr::n()), each = 20)) |>
  dplyr::mutate(hh_id = dplyr::row_number())
sample <- selected_eas |> execute(listing, seed = 43)
nrow(sample)  # 12 households in each selected EA
#> [1] 780

# One frame per stage ----------------------------------------
# Frames map to stages by position: a district register, then an EA register
districts <- dplyr::distinct(zwe_eas, province, district)
two_stage <- sampling_design() |>
  add_stage(label = "Districts") |>
    cluster_by(district) |>
    draw(n = 8) |>
  add_stage(label = "EAs") |>
    draw(n = 3)
sample <- two_stage |> execute(districts, zwe_eas, seed = 424)
length(unique(sample$district))  # 8 districts, 3 EAs each
#> [1] 8

# The same call with the frames held as a list
registers <- list(districts, zwe_eas)
same <- two_stage |> execute(registers, seed = 424)
identical(sample$.sample_id, same$.sample_id)
#> [1] TRUE

# Multi-phase --------------------------------------------------
# The previous phase's sample is the new phase's frame
phase1 <- sampling_design() |>
  draw(n = 200) |>
  execute(bfa_eas, seed = 42)
phase2 <- sampling_design() |>
  draw(n = 50) |>
  execute(phase1, seed = 123)
nrow(phase2)  # weights compound across both phases
#> [1] 50

# Replicated sampling: 5 independent draws
sample <- sampling_design() |>
  draw(n = 100) |>
  execute(bfa_eas, seed = 42, reps = 5)
table(sample$.replicate)  # 100 per replicate
#> 
#>   1   2   3   4   5 
#> 100 100 100 100 100 

# Rotating panel: 4 rotation groups
sample <- sampling_design() |>
  stratify_by(region) |>
  draw(n = 200) |>
  execute(bfa_eas, seed = 1, panels = 4)
table(sample$.panel)  # ~50 per panel
#> 
#>   1   2   3   4 
#> 650 650 650 650