Validate a frame against a design

Checks if a data frame contains all required variables for a sampling design and reports any issues.

Usage

validate_frame(
  design,
  frame,
  ...,
  stages = NULL,
  fingerprint = c("inform", "warn", "ignore")
)

Arguments

design: A sampling_design object, or a tbl_sample. A fully executed sample validates its stored design and compares its frame digest (when present) against frame. A partial sample validates the frame that would continue it, against the units it selected.
frame: A data frame to validate, or an ordered list of stage frames: one per stage, in the order execute() would receive them. Each entry may itself be hierarchical. A data frame and a one-element list are the same input, as they are in execute(), and every check here is one execute() runs before it samples: a frame this accepts is a frame execution accepts.
...: These dots are for future extensions and must be empty. stages and the arguments after it follow ..., so each must be named exactly: the singular stage is reported rather than prefix-matched.
stages: Which stage(s) to validate against. Defaults to all stages, and for a partial tbl_sample to every remaining stage, matching execute(). Where one frame cannot say whether it is the next stage's register or a hierarchy covering the rest, stages is required, again as in execute().
fingerprint: How to report differences between frame and what was recorded earlier: the frame fingerprint stored in a design file (designs restored with read_design() when saved with frame =) and the frame digest recorded at execution (when design is a tbl_sample, or a restored design whose receipt carries one). One of "inform" (default, emits a message), "warn", or "ignore". Both comparisons are informational and never fail validation, because a design remains executable on any frame that passes the variable checks.

Value

Invisibly returns TRUE if validation passes. Throws an informative error if validation fails.

Details

Validation checks include:

Presence of required stratification variables
Presence of required clustering variables
Presence of measure of size (MOS) variables for PPS sampling
Non-empty frame
Positive values for MOS variables

A fixed-size stage drawn without replacement whose take exceeds the population of a candidate pool is not an error by default, because execute() takes every unit of that pool and warns. A stage that sets on_capped = "error" in draw() is refused here with samplyr_error_size_capped, as execute() refuses it. Otherwise validate_frame() names such pools in a message of class samplyr_message_capped_pools, with the stage and a payload holding pool_keys, n_capped and n_pools. A candidate pool is one the stage would draw from if its parent were selected, so the message does not depend on the seed. The labels are the pool_key column of frame_summary(design, frame, detail = "pool").

In the same way, a candidate parent that lacks a stratum an explicit n names is reported with samplyr_message_absent_strata, the message execute() raises for a selected parent (see execution_conditions).

For designs restored with read_design(), if the design file carries a frame fingerprint, validate_frame() also compares it against frame and reports what changed (rows, columns, column types, or content).

When a frame digest is available (an executed tbl_sample, or a design file written from one), the structural comparison goes further: the role-scoped fingerprint (analysis columns added later do not trigger it), the frame size, and per-pool population sizes recomputed from frame at every stage the digest can anchor (stage 1 over the universe and later stages under the recorded parents). The report says where the frame drifted, not merely that it did.

Ordered stage frames

A data frame means one hierarchy. A list means separately supplied stage frames, and validate_frame() then checks everything execute() checks before it draws: the frame count, the columns each stage selects on, cluster-level variables that must be constant within a unit, ancestry types that must be joinable, and ancestry values that name no parent.

Values are judged on every unit a stage could reach, not on the parents a draw happens to select: missing, NA or mistyped strata, clusters, measures of size and the other columns a stage reads, reported with class samplyr_error_frame_invalid. execute() runs the same check on every later stage before it draws, so whether a frame is accepted never depends on the seed. Both also refuse a size in n or frac for a stratum no reachable unit belongs to (samplyr_error_alloc_unknown_strata), which would otherwise be dropped without a word. Tables that only describe strata, such as variance, may hold extra rows, and so may a take table keyed by a parent's identifier, for the parents a continuation did not select.

It also checks candidate coverage, and is stricter there than execution: a unit reachable at one stage with no rows in the register the next stage samples from is samplyr_error_frame_incomplete_register. execute() only warns, because the sample it happens to draw may never need that unit. The two agree on realized parents, which fail in both.

Continuing a partial sample

When design is a tbl_sample with stages left to run, validate_frame() checks the supplied frames against the units that sample actually selected, through the same transition the continuation would use: complete ancestry, joinable key types, no missing parent identifiers, and a row for every selected parent. It accepts the same shapes execute() does, so one register per remaining stage works here as it does there. The recorded fingerprint and frame digest are not compared, because they describe the frame the executed stages drew from rather than the register the next stage needs.

Preparing a second phase

When frame is itself a tbl_sample (that is, the design is being prepared as phase 2 of a two-phase sample), validate_frame() also pre-flights the two-phase export requirements. The phases declare their sampling units independently, so the link is the compound of every identifier either phase declares with cluster_by() that both samples carry, which is the bridge as_svydesign() builds. It warns when neither phase declares an identifier the phase-1 sample carries, and when the identifiers together do not uniquely identify phase-1 rows. Problems are reported as warnings rather than errors, because selection and weighting work without linkage. Only as_svydesign() needs it.

A previous-phase sample also carries identifiers that later stages must keep. One that varies inside a sampling unit cannot be carried, and is refused here for the same reason execute() refuses it: a clustered stage keeps one representative row, so by the time anything downstream looks, the disagreement is gone.

Examples

# A design stratified by region, with PPS on household counts
design <- sampling_design() |>
  stratify_by(region) |>
  cluster_by(ea_id) |>
  draw(n = 10, method = "pps_brewer", mos = households)

# Validate against bfa_eas (should pass)
validate_frame(design, bfa_eas)

# Create a frame missing required variables (will fail)
bad_frame <- data.frame(id = 1:100, value = rnorm(100))
try(validate_frame(design, bad_frame))
#> Error in validate_frame(design, bad_frame) : 
#>   frame 1, used for stage 1, is missing region, ea_id, and households.
#> ✖ region is the stratification variable.
#> ✖ ea_id is the cluster variable.
#> ✖ households is the MOS variable.
#> ℹ stage 1 selects on region, ea_id, and households.
#> ℹ A column name held in a variable is written `.data[[v]]`, as in `mos =
#>   .data[[v]]`.

# Validate only specific stages of a multi-stage design
zwe_frame <- zwe_eas |>
  dplyr::mutate(district_hh = sum(households), .by = district)

multi_design <- sampling_design() |>
  add_stage(label = "Districts") |>
    cluster_by(district) |>
    draw(n = 20, method = "pps_brewer", mos = district_hh) |>
  add_stage(label = "EAs") |>
    draw(n = 10)

# Validate stage 1 only
validate_frame(multi_design, zwe_frame, stages = 1)

# One register per stage: a list, in the order execute() receives them
districts <- zwe_frame |>
  dplyr::distinct(district, district_hh)
validate_frame(multi_design, list(districts, zwe_eas))

# A partial sample validates the register that would continue it
stage1 <- execute(multi_design, districts, stages = 1, seed = 5)
validate_frame(stage1, zwe_eas)
#> ℹ 73778 of 107250 rows of the register for stage 2 "EAs" belong to units the
#>   sample did not select.
#>   `execute()` sets them aside and samples from the other 33472.

# Designs restored from a file also check the stored frame fingerprint
path <- tempfile(fileext = ".json")
write_design(design, path, frame = bfa_eas)
restored <- read_design(path)

# Same frame, no message
validate_frame(restored, bfa_eas)

# A modified frame passes validation with an informational message
validate_frame(restored, bfa_eas[-1, ])
#> Frame differs from what was recorded when the design was saved:
#> • 44569 rows instead of the 44570 recorded
#> ℹ This is informational. The design remains executable on any frame that passes
#>   the variable checks.
unlink(path)