--- title: "Survey Scheduling" output: rmarkdown::html_vignette: highlight: null vignette: > %\VignetteIndexEntry{Survey Scheduling} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8} --- ```{r setup, include = FALSE} knitr::opts_chunk$set( collapse = TRUE, comment = "#>" ) ``` ## Overview A sampling schedule defines which days and periods represent the survey season. With a stratified random schedule, estimates of effort, catch, and CPUE can represent that season without treating sampled days as if they were the whole population. `generate_schedule()` builds the count-period calendar from season boundaries and a target sampling intensity. ## The Count Period Framework Every tidycreel design rests on a calendar of **count periods** — the discrete time slots during which field crews collect data. A count period is defined by: - **Date**: the survey date - **Period**: the time-of-day window (e.g., "Morning", "Afternoon") - **Stratum**: the day type (weekday vs. weekend) Interviews and angler counts are both collected during count periods. The schedule you generate defines exactly which days and periods will be sampled. ## Building a Basic Schedule The minimal inputs are season boundaries, the number of periods per day, and sampling intensity. The example below builds a schedule for a May–September creel season with morning and afternoon count periods, sampling 30% of weekdays and 50% of weekends. ```{r basic-schedule, message = FALSE} library(tidycreel) sched <- generate_schedule( start_date = "2024-05-01", end_date = "2024-09-30", n_periods = 2, period_labels = c("Morning", "Afternoon"), sampling_rate = c(weekday = 0.30, weekend = 0.50), seed = 42 ) head(sched, 8) ``` The result is a `creel_schedule` object with `date`, `day_type`, and `period_id` columns. Period labels replace integer IDs with human-readable names. ```{r schedule-summary} # Counts of scheduled days by stratum table(sched$day_type) ``` ## Visualising Sampled vs. Unsampled Days Pass `include_all = TRUE` to return every day of the season with a `sampled` indicator. This is useful for checking seasonal coverage and communicating the design to stakeholders. ```{r include-all, message = FALSE} sched_full <- generate_schedule( start_date = "2024-05-01", end_date = "2024-09-30", n_periods = 1, sampling_rate = c(weekday = 0.30, weekend = 0.50), include_all = TRUE, seed = 42 ) # Sampled proportion by stratum table(sched_full$day_type, sched_full$sampled) ``` The sampled days are drawn by stratified random sampling within each stratum, so weekday and weekend sampling fractions match the requested rates. ## Scheduling High-Use or Other Special Periods Some surveys need more than the default weekday/weekend split. Opener weekends, holidays, or derby periods are often sampled more heavily because they are expected in advance to have different effort and catch characteristics. In that case, the scheduling layer should assign those dates to their own prospective stratum rather than leaving them inside the ordinary weekend pool. `generate_schedule()` accepts a `special_periods` data frame with `start_date`, `end_date`, `label`, and optional `reason` columns. The periods are expanded to day-level assignments before sampling, so a boundary-crossing opener is split deterministically by civil date. ```{r high-use-schedule, message = FALSE} opener_periods <- data.frame( start_date = as.Date("2027-07-31"), end_date = as.Date("2027-08-01"), label = "high_use", reason = "opener" ) sched_high_use <- generate_schedule( start_date = "2027-07-17", end_date = "2027-08-08", n_periods = 1, sampling_rate = c(weekday = 0.5, weekend = 1, high_use = 1), include_all = TRUE, expand_periods = FALSE, seed = 42, special_periods = opener_periods ) opener_days <- sched_high_use$date >= as.Date("2027-07-30") & sched_high_use$date <= as.Date("2027-08-03") as.data.frame(sched_high_use)[ opener_days, c("date", "day_type", "final_stratum", "special_period_reason", "sampled") ] ``` The output keeps the baseline `day_type` classification and adds `final_stratum`, which is what the scheduler actually uses for sample allocation when special periods are present. Here the July 31 and August 1 dates are both assigned to `high_use`, even though one is in July and the other is in August. You can inspect the audit surfaces directly: ```{r high-use-audit} attr(sched_high_use, "special_period_audit") attr(sched_high_use, "special_period_allocation") ``` These attributes show which dates were reassigned to the special stratum and how many available days ended up in each final stratum after the calendar rules were applied. When a declaration is valid but leaves a very small special stratum, or consumes most of a baseline stratum, `generate_schedule()` warns. The returned schedule also carries a `special_period_diagnostics` attribute, and `print()` / `format()` show a compact summary. Here the opener is a single day, so the high-use stratum has one day in it, and the call warns that the schedule is fragile. A stratum with one sampled day gives no variance estimate later, when the effort is estimated. ```{r high-use-diagnostics, message = FALSE} sched_fragile <- generate_schedule( start_date = "2027-08-01", end_date = "2027-08-08", n_periods = 1, sampling_rate = c(weekday = 1, weekend = 1, high_use = 1), include_all = TRUE, expand_periods = FALSE, seed = 42, special_periods = data.frame( start_date = as.Date("2027-08-07"), end_date = as.Date("2027-08-07"), label = "high_use", reason = "opener" ) ) attr(sched_fragile, "special_period_diagnostics") head(format(sched_fragile), 6) ``` If a declaration rewrites the schedule so aggressively that a requested baseline stratum no longer exists in the final schedule, tidycreel aborts and asks you to update `n_days` or `sampling_rate` to match the final strata. Declared this way, a high-use period is a design stratum planned before sampling, and a period that crosses a month boundary is split by day rather than assigned wholesale to the month where it started. ### Special Strata vs. Post-Stratification Use `special_periods` when the unusual period is known **before sampling** and should be planned explicitly in the field calendar. Do **not** use it for retrospective corrections after the season. If a survey did not schedule a high-use or special period prospectively and you later need to correct weights or population totals, that is a post-stratification problem, not a scheduling problem. Those retrospective corrections belong in a separate post-stratification workflow. ## Three-Period Schedules Some surveys divide the day into three periods — morning, midday, and evening — to capture temporal patterns in angler activity (e.g., a morning and evening peak with low midday use). ```{r three-periods, message = FALSE} sched3 <- generate_schedule( start_date = "2024-06-01", end_date = "2024-08-31", n_periods = 3, period_labels = c("Morning", "Midday", "Evening"), sampling_rate = c(weekday = 0.40, weekend = 0.60), seed = 101 ) head(sched3, 9) ``` ## Reproducibility with `seed` The `seed` argument ensures that the same calendar is produced every time. This is important when the schedule is pre-printed for field use and must match what the estimators will use. ```{r seed} sched_a <- generate_schedule("2024-06-01", "2024-06-30", n_periods = 1, sampling_rate = 0.4, seed = 99 ) sched_b <- generate_schedule("2024-06-01", "2024-06-30", n_periods = 1, sampling_rate = 0.4, seed = 99 ) identical(sched_a, sched_b) ``` ## Saving and Sharing the Schedule `write_schedule()` exports to CSV (for any spreadsheet application) or xlsx (for direct field printing). `read_schedule()` reloads the file with correct column types, so the schedule round-trips without type coercion issues. ```{r io, eval = FALSE} # Save for field crews. The format is not read from the file name, so name it # for xlsx. write_schedule(sched, "count_schedule_2024.csv") write_schedule(sched, "count_schedule_2024.xlsx", format = "xlsx") # Reload — column types are preserved automatically sched_reload <- read_schedule("count_schedule_2024.csv") identical(sched, sched_reload) ``` ## Linking the Schedule to a Design Pass the schedule directly to `creel_design()`. The schedule's stratum structure informs the survey weights used by all downstream estimators. ```{r design, message = FALSE} design <- creel_design(sched, date = date, strata = day_type) design ``` When `special_periods` are present, the scheduling layer adds `final_stratum`, which is the resolved day-level stratum actually used for allocation. That is the column the analysis layer should follow. ```{r design-special, message = FALSE} sched_for_analysis <- sched_high_use[, c("date", "final_stratum")] names(sched_for_analysis)[2] <- "analysis_stratum" design_special <- creel_design( sched_for_analysis, date = date, strata = analysis_stratum ) design_special ``` If you want to collapse multiple baseline labels into one analysis bucket (for example, `treat all non-special days as "regular"`), do that in the calendar you pass to `creel_design()`. The key rule is that the analysis strata must be derived from the same resolved day-level assignments the scheduler used. Sparse special strata remain visible in analysis. If a special stratum has too few sampled PSUs for variance estimation, `estimate_effort()` or the product estimators abort with an explicit single-PSU diagnostic naming the stratum. If the stratum is merely small rather than impossible, tidycreel emits a warning about unstable variance instead of failing silently. ## Count Period Scheduling vs. Interview Scheduling In standard access-point creel surveys, **interviews happen during count periods** — the same field crew that counts anglers also approaches them for interviews. The count schedule therefore governs both collection activities. There is no separate "interview schedule." The set of available interviews is the set of interviews collected on sampled count-period days. ## Bus-Route Scheduling For bus-route designs, `generate_bus_schedule()` turns a site-level frame into a bus-route sampling frame with inclusion probabilities: $$\pi_i = p_{site} \times p_{period} \quad \text{where} \quad p_{period} = \frac{\text{crew}}{n_{circuits}}$$ $p_{period}$ here is the chance that a given circuit is the one a crew works. The function takes the schedule as an argument but does not use it in the calculation, and it requires `p_site` to sum to 1 within a circuit. The published bus-route estimator weights each site by its wait time over the whole circuit time, travel included, so its weights sum to less than 1; give those to `creel_design()` through `wait_time` and `circuit_time` (`vignette("bus-route-surveys")`). Bringing the planner in line is open as issue #541. ```{r bus-schedule, message = FALSE} # Site-level selection probabilities (must sum to 1.0) site_frame <- data.frame( site_id = c("North Bay", "South Cove", "Dock Area", "Main Channel"), p_site = c(0.35, 0.25, 0.25, 0.15) ) # Build bus-route sampling frame using the count schedule bus_frame <- generate_bus_schedule( schedule = sched, sampling_frame = site_frame, site = site_id, p_site = p_site, crew = 1 ) bus_frame ``` The `inclusion_prob` column is each site's probability of being sampled on a worked circuit, $p_{site} \times p_{period}$. It is not a probability over the season. ## Sampling Rate Guidance The right sampling intensity depends on the precision the survey needs and on how much daily effort varies within each stratum. `creel_n_effort()` turns a target coefficient of variation and pilot estimates of each stratum's mean and variance into the number of days to sample, and `creel_power()` gives the power to detect a change. ```{r sample-size, message = FALSE} # Effort sample size to achieve CV ≤ 0.15 # N_h: total days per stratum in a 184-day season (illustrative, not the schedule above) # ybar_h: pilot mean daily effort (angler-hours) per stratum # s2_h: pilot variance of daily effort per stratum creel_n_effort( cv_target = 0.15, N_h = c(weekday = 132, weekend = 52), ybar_h = c(weekday = 280, weekend = 550), s2_h = c(weekday = 14400, weekend = 32400) ) ``` ## Within-Day Count Time Scheduling An instantaneous count estimates effort without bias only if the moment of the count is chosen at random; always counting at the same time of day gives a bias that "is unpredictable and can be very large" (Hoenig et al. 1993). Pollock et al. (1994, Chapter 11) give two ways to choose the times: simple random sampling, and systematic random sampling, a random start followed by counts at a fixed interval. Both are unbiased; random times can fall close together by chance, which systematic times avoid. `generate_count_times()` implements both, and also places counts at times you fix yourself. ### Random Strategy With the random strategy, `n_windows` non-overlapping windows are drawn uniformly at random from the available survey hours. `window_size` sets the duration (minutes) of each window and `min_gap` enforces a minimum separation (minutes) between consecutive windows to prevent back-to-back counts. ```{r count-times-random, message = FALSE} library(tidycreel) ct_random <- generate_count_times( start_time = "06:00", end_time = "14:00", strategy = "random", n_windows = 4, window_size = 30, min_gap = 10, seed = 42 ) ct_random ``` `n_windows` is the total number of count windows to place in the day, `window_size` is how long each window lasts in minutes, and `min_gap` is the minimum idle time between the end of one window and the start of the next. ### Systematic Strategy The systematic strategy follows Pollock et al. (1994, Chapter 11): the day is divided into `n_windows` equal periods of length k, the first count falls at a random time within the first period, and the rest follow at t₁+k, t₁+2k, and so on. The counts are spread evenly across the day, which random placement does not guarantee; the cost, in their words, is "more complexity of the variance formulas". ```{r count-times-systematic, message = FALSE} ct_systematic <- generate_count_times( start_time = "06:00", end_time = "14:00", strategy = "systematic", n_windows = 4, window_size = 30, min_gap = 10, seed = 42 ) ct_systematic ``` ### Fixed Strategy The fixed strategy takes count times you supply, as a `data.frame` with `start_time` and `end_time` columns, and wraps them in a `creel_schedule` without any random placement. Fixed times are not a sampling design: counts made at the same times every day are the error Hoenig et al. (1993) warn against, and the estimators cannot correct the bias. Use it to print times that were themselves drawn at random, or when a protocol fixes the times and you accept that risk. ```{r count-times-fixed, message = FALSE} fw <- data.frame( start_time = c("07:00", "09:30", "12:00"), end_time = c("07:30", "10:00", "12:30"), stringsAsFactors = FALSE ) ct_fixed <- generate_count_times(strategy = "fixed", fixed_windows = fw) ct_fixed ``` ### Progressive Count Strategy Progressive surveys require a randomised circuit start time in addition to a day-level schedule. Use `generate_progressive_start()` to draw start times that avoid the \eqn{U[0, T - \tau]} mid-day bias common in hand-crafted schedules. ```{r count-times-progressive, message = FALSE} prog_starts <- generate_progressive_start( open_start = "06:00", open_end = "16:00", # T = 10 h circuit_time = 2, # τ = 2 h → k = 5 discrete start offsets strategy = "discrete", n = 10, seed = 99 ) prog_starts ``` Use `strategy = "wraparound"` when τ does not divide T evenly. The `direction` column (`"forward"` / `"reverse"`) must be recorded in the field protocol. ### Exporting Count Time Schedules `generate_count_times()` returns a `creel_schedule` object, the same class returned by `generate_schedule()`. It therefore passes directly to `write_schedule()` for field printing without any conversion step. ```{r count-times-export, eval = FALSE} write_schedule(ct_systematic, "count_times_2024.csv") ``` ## Combining the Daily Schedule with Count Time Windows `generate_schedule()` and `generate_count_times()` each return a separate `creel_schedule`. Before field dispatch, biologists need a single table — one row per (date x period x count window) — that tells each crew exactly when to count. `attach_count_times()` performs the cross-join in one step. ```{r attach-count-times, message = FALSE} sched <- generate_schedule( start_date = "2024-06-01", end_date = "2024-06-07", n_periods = 2, sampling_rate = c(weekday = 0.5, weekend = 0.8), seed = 42 ) ct <- generate_count_times( start_time = "06:00", end_time = "14:00", strategy = "systematic", n_windows = 3, window_size = 30, min_gap = 10, seed = 42 ) field_schedule <- attach_count_times(sched, ct) field_schedule ``` The result has `nrow(sched) * nrow(ct)` rows — every sampled day-period combination appears once per count window. Pass `field_schedule` directly to `write_schedule()` to produce a printable field dispatch sheet. ## Validating the Design Before the Season After building a schedule, `validate_design()` checks whether the proposed sampling intensity is sufficient to meet a target coefficient of variation (CV) — catching under-sampling problems before the season starts rather than discovering them in the post-season analysis. It uses the same pilot mean and variance inputs as `creel_n_effort()` and compares the proposed day counts against the minimum required sample size per stratum. ```{r validate-design, message = FALSE} report <- validate_design( N_h = c(weekday = 132, weekend = 52), ybar_h = c(weekday = 280, weekend = 550), s2_h = c(weekday = 14400, weekend = 32400), n_proposed = c(weekday = 40L, weekend = 26L), cv_target = 0.15 ) report ``` `report$results` shows the per-stratum status (pass / warn / fail), the proposed sample size, the required sample size, and the achieved CV at the proposed intensity. `report$passed` is `TRUE` only when all strata pass — use this as a go/no-go gate before printing field schedules. ```{r validate-design-results, message = FALSE} report$results ``` ## Checking Data Completeness After the Season `check_completeness()` runs post-season on a `creel_design` with survey data attached, flagging missing sampling days and strata with too few interviews to produce reliable estimates. It is the first diagnostic step before running estimators — resolving completeness issues early avoids propagating gaps into effort or CPUE estimates. ```{r check-completeness, message = FALSE} data(example_calendar) data(example_counts) data(example_interviews) design <- creel_design(example_calendar, date = date, strata = day_type) design <- add_counts(design, prep_counts_daily_effort(example_counts, date = date, strata = day_type, daily_effort = effort_hours )) design <- add_interviews(design, example_interviews, catch = catch_total, effort = hours_fished, n_anglers = n_anglers, trip_status = trip_status ) comp <- check_completeness(design) comp ``` `comp$passed` returns `TRUE` if no issues were found. When `FALSE`, `comp$missing_days` identifies dates that appear in the schedule but have no count data, and `comp$low_n_strata` identifies strata where the interview count is below the minimum threshold (`n_min`, default 10). ## Assembling the Season Summary `season_summary()` accepts a named list of `creel_estimates` objects — the outputs of `estimate_effort()`, `estimate_catch_rate()`, and related functions — and joins them into a single wide tibble for reporting or export. It performs no re-estimation; it is purely an assembly step that makes it easy to view all key metrics side by side. ```{r season-summary, eval = FALSE} # Run estimators first (see the tidycreel workflow vignette) effort <- estimate_effort(design) cpue <- estimate_catch_rate(design) # Assemble the season summary summary_tbl <- season_summary(list(effort = effort, cpue = cpue)) summary_tbl$table ``` The full estimation workflow — building the design, attaching counts and interviews, and running the estimators — is covered in the main tidycreel vignette. `season_summary()` is the final step that wraps pre-computed results for export or reporting. ```{r season-summary-export, eval = FALSE} write_schedule(summary_tbl$table, "season_2024_summary.xlsx", format = "xlsx") ``` `write_schedule()` writes any data frame or tibble, so the season summary table exports to CSV or xlsx with a single call. Name `format = "xlsx"` for a spreadsheet: the format is not taken from the file name. ## References - Hoenig, J. M., Robson, D. S., Jones, C. M., and Pollock, K. H. (1993). Scheduling counts in the instantaneous and progressive count methods for estimating sportfishing effort. *North American Journal of Fisheries Management*, 13, 723–736. - Pollock, K. H., Jones, C. M., and Brown, T. L. (1994). *Angler Survey Methods and Their Applications in Fisheries Management*. American Fisheries Society, Bethesda, MD.