Sampling precision for a change between two occasions

Compute the sampling error (SE, margin of error, CV) for the change in a mean or a proportion between two occasions of the same population, given the size of each occasion and how far the two samples overlap. This is the inverse of n_change().

Usage

prec_change(var = NULL, ...)

# Default S3 method
prec_change(
  var = NULL,
  n,
  ...,
  sd = NULL,
  p = NULL,
  change = NULL,
  alpha = 0.05,
  N = Inf,
  deff = 1,
  resp_rate = 1,
  eligibility_rate = 1,
  overlap = 0,
  overlap_cor = 0,
  overlap_draw = c("fixed", "independent"),
  df = NULL,
  plan = NULL
)

# S3 method for class 'svyplan_n'
prec_change(var, ...)

Arguments

var: For the default method: the population variance $S^2$ on each occasion, one value for both or one per occasion. Supply this or p, not both. For svyplan_n objects: a sample size result from n_change().
...: Additional arguments passed to methods. Unused arguments are rejected.
n: Sample size, gross units drawn, either one size for both occasions or one per occasion. Two sizes make overlap directional, since it counts the shared units against the first occasion.
sd: Population standard deviation on each occasion, an alternative spelling of var. Supply exactly one of var or sd.
p: The two occasion proportions, c(p1, p2), as an alternative to var. The occasion variances are then $Np(1-p)/(N-1)$, the same finite population variance prec_prop() uses, and the change is $p_2 - p_1$, so change is determined and must not be supplied.
change: Expected change, $\mu_2 - \mu_1$, on the var scale. It may be negative. Required only for the relative measures: cv and rmoe are defined against it and are NA without it. Determined by p on the proportion scale.
alpha: Significance level, default 0.05.
N: Population size. Inf (default) means no finite population correction. One value covers both occasions, and two are accepted only at overlap = 0, where the occasions are independent and may legitimately be different populations. A positive overlap requires a single N, since units can only be shared by samples drawn from one population.
deff: Design effect multiplier (> 0), applied to the variance of the change rather than to either occasion separately.
resp_rate: Expected response rate among eligible units, in (0, 1]. Default 1 (no adjustment). It applies to both occasions, and nets each size down to n * resp_rate before the variance is formed. It is a single round's response, not attrition across a panel. See the nonresponse section of svyplan-package for what the rate counts.
eligibility_rate: Share of issued units in scope, in (0, 1]. Default 1. It applies to both occasions, and with a finite N the population in scope is N * eligibility_rate. See the nonresponse section of svyplan-package.
overlap: Fraction of the first occasion's responding sample carried into the second, in $[0, 1]$, measured against the first occasion as n12 / n1, so it cannot exceed n[2] / n[1]. 0 (default) drops the covariance entirely and reproduces the two-independent-samples result. 1 means every responding unit of the first occasion is measured again, which is a full panel when the occasions are the same size and a subsample of the second when it is larger. See Details on what a response rate below 1 does to this reading.
overlap_cor: Correlation between the two occasions among the overlapping units, in $[0, 1]$. Default 0, which makes overlap worthless, since it is the product overlap * overlap_cor that buys precision on a change, so a full panel of uncorrelated measurements buys nothing. On the proportion scale two Bernoulli marginals bound the correlation they can have, and a value above that bound is rejected.
overlap_draw: How the shared units are drawn. "fixed" (default) draws them as a fixed count with the rest of each occasion's sample, as a rotating panel does. "independent" draws a retained panel once and, on each occasion, a fresh sample from the whole frame independently of it. The two differ only in the finite population terms. See Details.
df: Degrees of freedom of the variance estimator the planned design will have, typically sampled PSUs minus strata, and available from design_df(). It switches the interval quantile from normal to t. NULL (default) applies no adjustment.
plan: Optional svyplan() object providing design defaults.

Value

A svyplan_prec object with type = "change":

se: Standard error of the estimated change, computed on the net sizes n * resp_rate.
moe: Margin of error, q * se, with q defined in Details. The interval is symmetric about the change, so the limits are change - moe and change + moe.
cv: Standard error relative to the change, se / abs(change). NA when change is neither supplied nor determined by p. A change near zero makes it large by construction, which is a statement about the estimand and not about the design.
params: The validated inputs. Dispersion is always stored as var, a pair, including when you supplied sd or p. p is kept as well when the proportion scale was used, and it is what the n_change() round trip reads the scale back from.

Nothing here is rounded, so passing a continuous n back from n_change() reproduces its se, moe and cv exactly. The print shows n per occasion as evaluated, and for a fit it says that n is the continuous optimum. For a fit res, the whole sample to field on each occasion is ceiling(res$n), and prec_change(res, n = ceiling(res$n)) gives its precision.

Details

Level or change

A repeated survey publishes three things, and each is read by its own function. The level at one occasion is prec_mean() or prec_prop() and is a function of that occasion's size alone. The change between two occasions is this function, which overlap improves. The average of several occasions is prec_pooled(), which overlap works against, because the covariance it induces is subtracted in a difference and added in a sum.

Here q is qnorm(1 - alpha / 2) by default and qt(1 - alpha / 2, df) when df is supplied.

The estimand is the change in one population measured twice, not a difference between two populations. Writing $v_1, v_2$ for the occasion variances, $n_1, n_2$ for the net sizes and $k$ for the shared units, the expression is piecewise in overlap. At overlap = 0 the two occasions are treated as independent, which is what also lets them come from different populations:

$$V = \frac{v_1}{n_1}\Big(1 - \frac{n_1}{N_1}\Big) + \frac{v_2}{n_2}\Big(1 - \frac{n_2}{N_2}\Big).$$

Above zero the occasions share one population of size $N$ and the covariance enters:

$$V = \frac{v_1}{n_1} + \frac{v_2}{n_2} - \frac{2\rho\,\mathrm{overlap}\sqrt{v_1v_2}}{n_2} - \frac{v_1 + v_2 - 2\rho\sqrt{v_1v_2}}{N},$$

a per-unit part less a population part. The marginal terms carry their own finite population correction, whereas the overlap covariance does not, since $Cov(\bar y_1, \bar y_2) = \rho S_1 S_2 \{k/(n_1n_2) - 1/N\}$ enters the population once as $1/N$. The two expressions agree at $\rho = 0$ and differ otherwise, because the second keeps the population term the first drops along with the rest of the covariance. At overlap = 1 and overlap_cor = 1 with equal sizes and variances the population terms cancel exactly and the whole reduces to $2S^2(1 - \mathrm{overlap})/n$, which is zero, the same units being measured twice and the change is observed rather than estimated.

deff multiplies the assembled variance. It is the design effect of the change, which is not in general either occasion's design effect, and a clustered design measuring the same clusters twice will have a smaller one than a design that reclusters.

What the overlap covariance assumes

Positive overlap on a finite frame must also fit the population: $n_1 + n_2 - \mathrm{overlap}\,n_1 \le N$, where the sizes count respondents after resp_rate. An inconsistent design is rejected even when overlap_cor = 0. In particular, two censuses require full overlap. The special overlap = 0 convention continues to denote independent samples.

The covariance is a model form, a correlation overlap_cor between occasions among the units the two occasions share, and simple random sampling otherwise. It is not the design-based covariance of two waves of a complex design, which carries the pairwise inclusion probabilities of the master sample the waves were coordinated from. Under a clustered or unequal-probability master the two can differ materially, and this function plans from the schedule rather than measuring a realization. overlap and overlap_cor are asserted here rather than derived, and nothing checks them against a rotation pattern.

The sizes enter the covariance after resp_rate has netted them down, so overlap is the overlap between the two responding samples, not between the issued ones. The two coincide at resp_rate = 1. Below it, converting an issued-sample overlap into this one needs an assumption about how response at the second occasion depends on response at the first, which this function does not make. Under independent response the responding overlap is the lower quantity, about resp_rate times the issued one. Supply the figure you expect among respondents. design_overlap() computes the issued figure a rotation schedule gives and states the conversion.

How the shared units are drawn

Under overlap_draw = "fixed" the shared units are a fixed count, overlap * n1, drawn with the rest of the sample, as in a rotating panel. Under "independent" a retained panel of that size is drawn once, and each occasion adds a fresh sample drawn from the whole frame independently of it. A unit can then enter twice on one occasion, and the frame check above does not apply, since each part fits the frame on its own. Each occasion's mean pools all its units with equal weight, a unit drawn twice counting twice. Writing $w_1 = \mathrm{overlap}$ and $w_2 = k/n_2$ for the retained shares, the per-unit part of $V$ is unchanged and the population part becomes

$$-\frac{m_1v_1 + m_2v_2 - 2w_1w_2\rho\sqrt{v_1v_2}}{N}, \qquad m_t = w_t^2 + (1 - w_t)^2.$$

At overlap = 0 this is the independent-samples result, so under "independent" the zero-overlap rule is a limit and not a special case. At overlap = 1 with equal sizes both draws are the same full panel. In between, the gap is of order $1/N$ and grows with the sampling fraction. At equal sizes and variances, half the units retained and overlap_cor = 0.8, the change variance under "independent" is 0.8% lower than under "fixed" at a sampling fraction of 5% and 1.7% lower at 10%. prec_pooled() shows the larger gap of an average.

Round-trip with n_change

prec_change() is the inverse of n_change(): computing res <- n_change(var = 100, moe = 2, overlap = 0.5, overlap_cor = 0.6) and then prec_change(res) recovers moe = 2.

References

Kish, L. (1965). Survey Sampling. Wiley. Chapter 12.

Examples

# Two independent occasions of 500: the flat two-sample result
prec_change(var = 100, n = 500)
#> Sampling precision for change (mean scale)
#> n = 500 per occasion (var = 100, 100, deff = 1)
#> No between-occasion covariance (overlap x overlap_cor = 0)
#> se = 0.6325, moe = 1.24

# A half-overlapping panel, correlated 0.6, buys precision
prec_change(var = 100, n = 500, overlap = 0.5, overlap_cor = 0.6)
#> Sampling precision for change (mean scale)
#> n = 500 per occasion (var = 100, 100, deff = 1)
#> overlap = 0.5, overlap_cor = 0.6 (70.0% of the independent variance)
#> se = 0.5292, moe = 1.037

# A change in a proportion: variances and the change come from p
prec_change(p = c(0.30, 0.36), n = 1200, overlap = 0.75, overlap_cor = 0.5)
#> Sampling precision for change (proportion scale)
#> n = 1200 per occasion (p = 0.3 to 0.36, deff = 1)
#> overlap = 0.75, overlap_cor = 0.5 (62.5% of the independent variance)
#> se = 0.01515, moe = 0.02969, cv = 0.2525, rmoe = 0.4949

# Relative measures need the change on the var scale
prec_change(var = 100, n = 500, change = 5)$cv
#> [1] 0.1264911

# Unequal occasions: overlap counts against the first
prec_change(var = 100, n = c(800, 400), overlap = 0.5, overlap_cor = 0.6)
#> Sampling precision for change (mean scale)
#> n = 800 then 400 (var = 100, 100, deff = 1)
#> overlap = 0.5, overlap_cor = 0.6 (60.0% of the independent variance)
#> se = 0.4743, moe = 0.9297

# Round-trip from n_change
res <- n_change(var = 100, moe = 2, overlap = 0.5, overlap_cor = 0.6)
prec_change(res)
#> Sampling precision for change (mean scale, continuous optimum)
#> n = 134.4511 per occasion (var = 100, 100, deff = 1)
#> overlap = 0.5, overlap_cor = 0.6 (70.0% of the independent variance)
#> se = 1.02, moe = 2
prec_change(res, n = ceiling(res$n))
#> Sampling precision for change (mean scale)
#> n = 135 per occasion (var = 100, 100, deff = 1)
#> overlap = 0.5, overlap_cor = 0.6 (70.0% of the independent variance)
#> se = 1.018, moe = 1.996

# A small frame: a retained panel topped up by independent fresh draws
prec_change(var = c(100, 164), n = 40, N = 400, overlap = 0.5,
            overlap_cor = 0.78, overlap_draw = "independent")
#> Sampling precision for change (mean scale)
#> n = 40 per occasion (var = 100, 164, deff = 1)
#> overlap = 0.5, overlap_cor = 0.78 (62.2% of the independent variance), independent draws
#> se = 1.974, moe = 3.869