Summarize the population structure recorded with a sample

An executed sample can carry a frame digest, a compact record of the frames, selection pools, resolved selection chances, and allocation that each stage encountered at execution time. frame_summary() turns that record into tibbles, so the structure of the selection stays readable after the original frame has gone away.

Usage

frame_summary(
  x,
  frame = NULL,
  ...,
  stages = NULL,
  scope = c("eligible", "universe"),
  detail = c("stage", "pool", "unit")
)

Arguments

x: A tbl_sample produced by execute() that carries a frame digest, or a sampling_design restored by read_design() from a file written from an executed sample. That file's execution receipt carries the digest, so it supports next-wave planning without the frame or the sample. With frame, any complete sampling_design will do.
frame: Optional frame on which to preview what the design would do, in a form execute() accepts: one data frame for a shared hierarchy, or an ordered list of stage registers. No sampling is performed, no random numbers are drawn, and .Random.seed is left as it was. See "Previewing a design" below.
...: These dots are for future extensions and must be empty. The arguments after them must be named exactly, so the singular stage is reported rather than prefix-matched.
stages: An integer vector of stage numbers to report, or NULL (default) for all recorded stages.
scope: Denominator basis for population sizes and take rates. "eligible" (default) counts the units eligible for the recorded execution. "universe" reports denominators only where the digest covers the full population hierarchy. A denominator the recorded scope cannot support is NA, never invented.
detail: Resolution of the report: "stage" (one row per stage), "pool" (one row per selection pool and realization), or "unit" (one row per population unit, only for stages that retained a unit-level representation).

Value

A tibble. Its scope column states how complete the underlying representation is ("eligible", "universe", "conditional", "partial", or "unknown").

For detail = "stage", one row per stage with stage, unit_level, scope, chance_kind, probabilities, storage, n_pools, n_capped, N, n_target, n_expected, n_moved, n_tied, n_realized, and take_rate. n_capped counts the stage's pools whose capped is TRUE at detail = "pool", and is NA when any of them is NA. Like n_pools, it counts pools without weighting, so in a preview it counts candidate pools. n_moved is the number of units an allocation method moved away from strata it saturated, which n_target no longer shows. A preview reports it, rolled up like n_target. It is NA for a stage without an allocation method and on a recorded digest, which does not keep it. n_tied counts the pools marked tied at detail = "pool", without weighting, and is NA in the same cases.

For detail = "pool", one row per selection pool and realization with stage, pool_id, replicate, parent_unit, pool_key, any stratum label columns, N, n_target, n_expected, n_realized, scope, chance_status, chance (the constant per-unit chance where one applies, NA otherwise), take_rate, capped, and tied. replicate is 1 for an ordinary execution. pool_key names the pool in the frame's values, its parent path then its stratum, as the pool_keys field of a condition from execute() does. Only a preview can name pools, so it is NA on a recorded digest, and on a first-stage pool without strata. In a replicated execution, pool_id identifies the shared structural pool and the key is stage, pool_id, and replicate. Stratum columns are NA for stages that do not use them.

capped marks a pool that could not supply the executable target it was given. It is always FALSE on a random-size stage, and NA for a shortfall when no design is available to record which kind of stage it is. A stratum that an allocation method had already reduced to its population is FALSE, because n_target records the post-redistribution target. execute() reports that event as samplyr_message_allocation_capped or, when the stage takes everything within reach, samplyr_warning_census (see execution_conditions).

tied marks a pool whose allocated take was decided by a tie in the rounding: its remainder equals that of a stratum on the other side of the cut, and the rule in stratify_by() chose which got the extra unit. A preview reports it under an allocation method, and it is NA otherwise and on a recorded digest.

For detail = "unit", one row per population unit of each stage that stored units, with stage, pool_id, unit_id, unit_order, chance, is_certainty, n_descendants, is_selected, and n_hits. Stages that stored only a constant or a chance distribution contribute no rows, and requesting such a stage explicitly is an error.

Details

take_rate is n_realized / N, not an inclusion probability, and the two differ for unequal-probability designs. chance_kind states what the recorded chances mean: first-order inclusion probabilities for without-replacement stages, expected hits for with-replacement stages. probabilities is the stage's probability tier (see sample_columns): "exact" when the recorded chances equal the design's true first-order chances, "approximate" when the method achieves them only to a documented approximation, and NA for digests recorded before the field existed.

The allocation columns distinguish three quantities:

n_target is the nominal count requested by the design, before execution-time population limits, probability capping, or certainty overflow. For a random-size design specified with frac, it is N * frac and may be fractional.
n_expected is the sum of the final resolved inclusion probabilities or expected hits. Probability capping or other feasibility adjustments can make it differ from n_target.
n_realized is the number of selected units or occurrences.

The three often coincide for a feasible fixed-size design. Under Bernoulli and Poisson sampling, n_realized varies around n_expected and n_target keeps the nominal expectation. n_target is NA only when no nominal count can be recovered, as from an older digest or a method with unspecified target semantics.

In a replicated execution, stage detail reports the realized allocation when every replicate shares it, and NA for n_realized and take_rate otherwise. Pool detail has one row per pool and replicate even when the replicates agree, with population and design columns repeated, so select one replicate before using it as a next-wave planning frame. Unit detail spans the stacked replicates: is_selected marks units selected in at least one and n_hits counts occurrences across all, so a without-replacement unit can show n_hits > 1. The per-replicate trace is the replicate column of the digest's selected units. A replicated multi-stage execution records only the stage prefix every replicate shares, because later-stage pools hang off each replicate's own selected parents, and frame_summary() says so.

A digest whose sample was modified after execution (rows, weights, or design columns changed) is reported as invalidated and refused.

Previewing a design

With frame, every selection pool is enumerated and every chance resolved, but nothing is selected. An allocation can be inspected before it is committed to, and a design restored from a file can be checked against next wave's frame. n_realized and take_rate are NA, as are is_selected and n_hits at detail = "unit". Every other column, and the shape of the table, matches a recorded digest, so the two are directly comparable.

At detail = "pool", a stage below a clustered stage reports one pool per candidate parent, and its n_target is what that stage would take given that parent is selected, such as how many households to list in a selected cluster. At detail = "stage", those pools are rolled up weighted by the probability each parent is selected, so the row reports the expected stage size rather than the total over every candidate. A design taking 10 of 100 clusters and 5 units in each reports n_target = 50 at stage 2, not 500. With fixed stage sizes this is also the exact size. Where per-parent takes vary (frac over unequal clusters, for instance) the realized size varies around it.

Not every design can be previewed. A stage below a with-replacement stage is refused, because the number of times each parent is hit is random, and so is a design whose non-final stage selects elements, which cannot be executed either (samplyr_error_exante_unsupported, samplyr_error_stage_parent_id).

Examples

frame <- data.frame(
  id = 1:20,
  stratum = rep(c("A", "B"), each = 10)
)

# Fixed-size allocation: target, expectation, and realization agree.
fixed <- sampling_design() |>
  stratify_by(stratum) |>
  draw(n = 3) |>
  execute(frame, seed = 42)
frame_summary(fixed)
#> # A tibble: 1 × 15
#>   stage unit_level scope    chance_kind   probabilities storage n_pools n_capped
#>   <int> <chr>      <chr>    <chr>         <chr>         <chr>     <int>    <int>
#> 1     1 element    universe inclusion_pr… exact         consta…       2        0
#> # ℹ 7 more variables: N <dbl>, n_target <dbl>, n_expected <dbl>, n_moved <dbl>,
#> #   n_tied <int>, n_realized <dbl>, take_rate <dbl>
allocation_columns <- c(
  "stage", "pool_id", "replicate", "stratum", "N",
  "n_target", "n_expected", "n_realized", "take_rate"
)
fixed_pools <- frame_summary(fixed, detail = "pool")
print(fixed_pools[allocation_columns], width = Inf)
#> # A tibble: 2 × 9
#>   stage pool_id replicate stratum     N n_target n_expected n_realized take_rate
#>   <int>   <int>     <int> <chr>   <dbl>    <dbl>      <dbl>      <dbl>     <dbl>
#> 1     1       1         1 A          10        3          3          3       0.3
#> 2     1       2         1 B          10        3          3          3       0.3

# The same report before anything is drawn: pass the frame instead.
# Nothing is selected and no random numbers are used, so an allocation
# can be checked before it is committed to.
planned <- sampling_design() |>
  stratify_by(stratum) |>
  draw(n = 3)
frame_summary(planned, frame)
#> # A tibble: 1 × 15
#>   stage unit_level scope    chance_kind   probabilities storage n_pools n_capped
#>   <int> <chr>      <chr>    <chr>         <chr>         <chr>     <int>    <int>
#> 1     1 element    universe inclusion_pr… exact         consta…       2        0
#> # ℹ 7 more variables: N <dbl>, n_target <dbl>, n_expected <dbl>, n_moved <dbl>,
#> #   n_tied <int>, n_realized <dbl>, take_rate <dbl>
print(
  frame_summary(planned, frame, detail = "pool")[allocation_columns],
  width = Inf
)
#> # A tibble: 2 × 9
#>   stage pool_id replicate stratum     N n_target n_expected n_realized take_rate
#>   <int>   <int>     <int> <chr>   <dbl>    <dbl>      <dbl>      <dbl>     <dbl>
#> 1     1       1         1 A          10        3          3         NA        NA
#> 2     1       2         1 B          10        3          3         NA        NA

# A short pool before drawing: one of three villages holds fewer
# households than the take of 4, and the preview names it.
villages <- data.frame(
  village = rep(c("v1", "v2", "v3"), c(6, 3, 8)),
  household = 1:17
)
two_stage <- sampling_design() |>
  add_stage() |>
  cluster_by(village) |>
  draw(n = 2) |>
  add_stage() |>
  draw(n = 4)
frame_summary(two_stage, villages)[c("stage", "n_pools", "n_capped")]
#> # A tibble: 2 × 3
#>   stage n_pools n_capped
#>   <int>   <int>    <int>
#> 1     1       1        0
#> 2     2       3        1
short <- frame_summary(two_stage, villages, detail = "pool")
short[short$capped, c("stage", "pool_key", "N", "n_target")]
#> # A tibble: 1 × 4
#>   stage pool_key     N n_target
#>   <int> <chr>    <dbl>    <dbl>
#> 1     2 v2           3        4

# Random-size allocation: one scalar row per pool and replicate.
random_sample <- sampling_design() |>
  stratify_by(stratum) |>
  draw(frac = 0.3, method = "bernoulli", on_empty = "silent") |>
  execute(frame, seed = 42, reps = 3)
random_pools <- frame_summary(random_sample, detail = "pool")
print(random_pools[allocation_columns], width = Inf)
#> # A tibble: 6 × 9
#>   stage pool_id replicate stratum     N n_target n_expected n_realized take_rate
#>   <int>   <int>     <int> <chr>   <dbl>    <dbl>      <dbl>      <dbl>     <dbl>
#> 1     1       1         1 A          10        3          3          3       0.3
#> 2     1       1         2 A          10        3          3          2       0.2
#> 3     1       1         3 A          10        3          3          2       0.2
#> 4     1       2         1 B          10        3          3          3       0.3
#> 5     1       2         2 B          10        3          3          4       0.4
#> 6     1       2         3 B          10        3          3          4       0.4

# Probability capping can make the resolved expectation smaller
# than the nominal target.
pps_frame <- data.frame(id = 1:20, mos = c(100, rep(1, 19)))
capped <- sampling_design() |>
  draw(n = 5, method = "pps_poisson", mos = mos, on_empty = "silent") |>
  execute(pps_frame, seed = 42)
#> Warning: Stage 1: PPS Poisson expected sample size fell short of the reachable target in
#> 1 pool.
#> ✖ Reachable 5 units, expected 1.8.
#> ✖ 1 unit has an inclusion probability clipped at 1.
#> ℹ Handle dominant units explicitly with `certainty_size` or `certainty_prop`.
#> ℹ See `?selection_methods` for the "pps_poisson" contract.
capped_pool <- frame_summary(capped, detail = "pool")
print(
  capped_pool[c("N", "n_target", "n_expected", "n_realized")],
  width = Inf
)
#> # A tibble: 1 × 4
#>       N n_target n_expected n_realized
#>   <dbl>    <dbl>      <dbl>      <dbl>
#> 1    20        5       1.80          1

# Full digests support anonymous unit-level inspection.
full <- sampling_design() |>
  draw(n = 5, method = "pps_brewer", mos = id) |>
  execute(frame, seed = 42, frame_digest = "full")
head(frame_summary(full, detail = "unit"))
#> # A tibble: 6 × 9
#>   stage pool_id unit_id unit_order chance is_certainty n_descendants is_selected
#>   <int>   <int>   <int>      <int>  <dbl> <lgl>                <int> <lgl>      
#> 1     1       1       1          1 0.0238 FALSE                   NA FALSE      
#> 2     1       1       2          2 0.0476 FALSE                   NA FALSE      
#> 3     1       1       3          3 0.0714 FALSE                   NA FALSE      
#> 4     1       1       4          4 0.0952 FALSE                   NA FALSE      
#> 5     1       1       5          5 0.119  FALSE                   NA FALSE      
#> 6     1       1       6          6 0.143  FALSE                   NA FALSE      
#> # ℹ 1 more variable: n_hits <int>