version: "1.0.1" name: ukbsci-cohort description: > Build a UK Biobank disease cohort from a phenotype table extracted on the Research Analysis Platform (RAP), using the UKBAnalytica R package. Covers disease-definition construction and catalog lookup (get_predefined_diseases, get_disease_catalog, get_pomegranate_diseases, create_disease_definition, combine_disease_definitions), source parsers (parse_icd10_diagnoses, parse_icd9_diagnoses, parse_opcs4_procedures, parse_cancer_registry, parse_death_records, parse_self_reported_illnesses), multi-source case extraction (extract_cases_by_source, extract_disease_diagnosis, extract_disease_history, extract_disease_history_sensitivity, compare_data_sources), reusable follow-up time skeletons (ukb_time_skeleton), and the Cox-ready dataset builder build_survival_dataset() with prevalent vs incident separation and follow-up time computation, plus select_incident_by_years() for time-window stratification. Use this skill when the user asks to define a UKB disease phenotype, inspect curated/Pomegranate diagnostic codes, separate prevalent from incident cases, build a survival dataset, compare diagnostic sources, or compute follow-up time. Triggers: UKB cohort, UKB disease definition, prevalent vs incident, survival dataset, build_survival_dataset, ICD10 phenotyping, UKB 队列, 病例定义, 生存数据集, /ukbsci-cohort. Hard rule: local agents must not read or inspect real UKB RAP participant-level data; generate scripts for RAP execution and interpret aggregate outputs only.

ukbsci-cohort — UK Biobank disease cohort & survival dataset builder

0. RAP guardrails (must always hold)

Strict local-agent boundary: this skill is for script generation, workflow planning, package guidance, and interpretation of aggregate outputs. The agent must not read, inspect, summarize, or process real UK Biobank RAP participant-level data, including de-identified row-level tables, raw RAP fields, exact dates, per-row predictions, row-level SHAP matrices, screenshots, tracebacks, or logs containing row-level values. Generate scripts for the user to run inside RAP; only aggregate results or rendered figures may be shared back with the agent. See ../references/agent-privacy-boundary.md.

This skill assumes its input — the phenotype data.table — was extracted by `ukbsci-rap-extract` (or equivalent rap_* calls) and lives in RAP project storage. The same rules carry over:

• Inputs and outputs default to /mnt/project/.... Do not emit code that

writes the cohort table or per-disease case files to a local laptop path.

• Aggregate results (cohort flow tables, Table 1, regression coefficients)

may be shared with the local agent. Per-participant tables are not.

• If the user asks "can I just run this locally with a small sample?", answer

with a simulated toy dataset — never a sample of real participants.

1. When to load this skill

• Define a UKB disease phenotype from scratch (create_disease_definition)

or pick one of the pre-validated definitions (get_predefined_diseases).

• Combine multiple definitions into one composite endpoint

(combine_disease_definitions).

• Identify cases from one or several diagnostic sources (ICD-10, ICD-9, OPCS-4,

death registry, cancer registry, first-occurrence, self-report, UKB algorithmically-defined outcomes).

• Separate prevalent (baseline-or-earlier) from incident (post-baseline)

cases and compute follow-up time for Cox / competing-risks models.

• Run a sensitivity analysis where the definition source set varies.
• Stratify incident cases by years since enrollment (e.g. ≤5y vs >5y).

2. When NOT to load this skill

• The user has not extracted phenotype data yet → load `ukbsci-rap-extract`.
• The user wants to fit a regression / Cox model on the survival dataset →

load `ukbsci-regression` or `ukbsci-survival`.

• The user wants a Table 1 summary of the cohort → load `ukbsci-baseline`.
• The user wants propensity-score matching → load `ukbsci-propensity`.

3. Prerequisites

Required columns in the input data.table:

The skill must inspect dt and warn the user about any source they request that is not represented in the columns.

library(UKBAnalytica)
library(data.table)
 
# dt is the data.table produced by rap_extract_pheno() / rap_submit_extract()
stopifnot(data.table::is.data.table(dt))
stopifnot("eid" %in% colnames(dt))
stopifnot("p53_i0" %in% colnames(dt))   # baseline date

4. Pipeline (4 phases)

Phase 1 — Pick or build disease definitions

Start with the curated catalogue. In UKBAnalytica 1.0.0, the curated library contains 86 analysis-ready disease definitions. The source-aware catalog also exposes 313 Pomegranate-derived definitions. The expanded union of curated and Pomegranate definitions currently contains 331 usable disease keys; the conservative intersection contains 56 shared keys.

defs_all <- get_predefined_diseases()
names(defs_all)
#> "AA", "TAA", "AAA", "CVD", "MI", "HF", "Stroke", "Hypertension",
#> "Diabetes", "T1DM", "T2DM", "Atrial_Fibrillation", "Lung_Cancer", ...
 
# Subset to your study endpoints
diseases <- defs_all[c("AA", "Hypertension", "Diabetes")]

Inspect exact code evidence before committing to an endpoint:

# Code-level table: source, code_system, field_id, code, validation_status
get_disease_catalog(disease = "COPD")
get_disease_catalog(disease = "diabetes", code_system = "ICD-10")
 
# Pomegranate-derived disease definitions that current parsers can use
pom_defs <- get_predefined_diseases(source = "pomegranate")
 
# Matched curated + Pomegranate definitions.
# "intersection" is conservative; "union" expands matched code sets and
# retains unmatched definitions from both sources.
shared_defs <- get_predefined_diseases(
  source = "both",
  merge_type = "intersection"
)
expanded_defs <- get_predefined_diseases(
  source = "both",
  merge_type = "union"
)

Use source = "curated" for protocol-driven analyses where stable manual definitions are preferred. Use source = "both", merge_type = "union" for endpoint discovery or sensitivity analysis. When the exact source matters, write the selected source and merge strategy into the analysis script.

Define a custom phenotype:

copd_def <- create_disease_definition(
  name             = "COPD",
  icd10_pattern    = "^J4[0-4]",        # J40-J44
  icd9_pattern     = "^49[0-2]",        # 490-492
  sr_codes         = c(1112, 1113, 1472),
  death_icd10      = "^J4[0-4]",
  opcs4_pattern    = NULL,
  algo_date_field   = 42016,            # UKB algorithmic COPD date field
  algo_source_field = 42017             # Source: hospital / death / algorithm
)

Custom cancer phenotype using histology + behaviour codes:

lung_ca_def <- create_disease_definition(
  name                 = "LungCancer_strict",
  cancer_icd10_pattern = "^C34",
  cancer_histology     = c(8070, 8071, 8140, 8240),   # squamous, adenoca, etc.
  cancer_behaviour     = 3L                            # malignant only
)

Combine definitions when you need composite endpoints:

cvd_composite <- combine_disease_definitions(
  defs_all$MI, defs_all$Stroke, defs_all$HF,
  name = "MACE"
)

Phase 2 — Parse source tables (optional inspection)

You normally don't call the parsers directly — extract_cases_by_source() does it for you. But for QC / debugging it is useful to know the schemas:

icd10 <- parse_icd10_diagnoses(dt)
# → eid, icd10_code, diag_date (Date), source = "ICD10"
 
icd9  <- parse_icd9_diagnoses(dt)
opcs4 <- parse_opcs4_procedures(dt)
canc  <- parse_cancer_registry(dt)
# → cancer_icd10_code, cancer_histology, cancer_behaviour, diag_date
 
death <- parse_death_records(dt)
# → death_code, death_date, source = "Death", cause_type (primary/secondary)
 
dod   <- get_death_dates(dt)
# → eid, death_date  (earliest per participant — for censoring)
 
sr    <- parse_self_reported_illnesses(dt, baseline_col = "p53_i0")
# → sr_code, diag_date (month-precision, imputed to day 15), instance, array_idx

If a parser returns a 0-row table, the corresponding source columns were not present — warn the user before they request that source downstream.

Phase 3 — Extract cases (one disease, one source, or many of each)

3a. Single-disease diagnosis table — extract_disease_diagnosis()

Use this when the user wants a compact diagnosis table before building a full survival dataset.

mi_def <- get_predefined_diseases(
  source = "both",
  merge_type = "union",
  disease = "MI"
)
 
mi_dx <- extract_disease_diagnosis(
  dt                  = dt,
  disease             = "MI",
  disease_definitions = mi_def,
  sources             = c("ICD10", "ICD9", "Death"),
  censor_date         = as.Date("2023-10-31"),
  baseline_col        = "p53_i0",
  include_all         = TRUE
)
# Columns: eid, disease, diagnosed, prevalent_case, incident_case,
# earliest_date, diagnosis_source, status, surv_time

3b. The flexible primitive — extract_cases_by_source()

cases_long <- extract_cases_by_source(
  dt                  = dt,
  disease_definitions = diseases,
  sources             = c("ICD10", "ICD9", "Self-report", "Death",
                          "OPCS4", "CancerRegistry",
                          "FirstOccurrence", "Algorithm"),
  censor_date  = as.Date("2023-10-31"),
  baseline_col = "p53_i0"
)
str(cases_long)
#> eid, disease, status, surv_time, prevalent_case,
#> earliest_date, diagnosis_source

diagnosis_source is the winning source per participant–disease (whichever gave the earliest valid date). For "Algorithm", it is further refined to "Algorithm_<source-code>".

3c. The history-only helper — extract_disease_history()

For "did the participant ever have disease X (per the chosen sources)?":

history <- extract_disease_history(
  dt                  = dt,
  diseases            = c("Hypertension", "Diabetes"),
  disease_definitions = defs_all,
  sources             = c("ICD10", "ICD9", "Self-report"),
  baseline_col        = "p53_i0"
)
#> Adds columns: Hypertension_history (0/1), Diabetes_history (0/1)

3d. Sensitivity variants — extract_disease_history_sensitivity()

For pre-baked sensitivity strata (ICD-10-only, ICD-10+ICD-9, all sources):

sens <- extract_disease_history_sensitivity(
  dt                  = dt,
  diseases            = c("Diabetes"),
  disease_definitions = defs_all,
  baseline_col        = "p53_i0"
)
#> Adds: Diabetes_history_ICD10, Diabetes_history_hospital, Diabetes_history_all

3e. Source-by-source comparison — compare_data_sources()

QC tool. Returns a per-disease table of how many cases each source uncovers and how much they overlap:

qc <- compare_data_sources(
  dt                  = dt,
  disease_definitions = diseases,
  baseline_col        = "p53_i0"
)

Phase 4 — Build reusable time skeleton, then Cox-ready dataset

For prospective analyses, first standardize the time backbone once and reuse it across disease endpoints.

time_skel <- ukb_time_skeleton(
  data                 = dt,
  id_col               = "eid",
  baseline_col         = "p53_i0",
  birth_year_col       = "p34",
  birth_month_col      = "p52",
  age_col              = "p21022",
  death_date_cols      = "^(participant\\.)?p40000_i[0-9]+$",
  lost_to_followup_col = "p191",
  admin_censor_date    = as.Date("2023-10-31")
)

admin_censor_date remains the database-level administrative censoring date. followup_end_date is participant-specific and uses the earliest of death, loss-to-follow-up, and administrative censoring.

The main entry point. Most user requests should resolve here.

cohort <- build_survival_dataset(
  dt                  = dt,
  disease_definitions = diseases,
  prevalent_sources   = c("ICD10", "ICD9", "Self-report", "Death"),
  outcome_sources     = c("ICD10", "ICD9", "Death"),
  censor_date         = as.Date("2023-10-31"),
  baseline_col        = "p53_i0",
  time_skeleton       = time_skel,
  primary_disease     = "AA",          # one key, or a vector for independent endpoints
  output              = "wide",        # or "long"
  include_all         = TRUE,
  show_flow           = TRUE,
  dt_threads          = 8
)
 
# Inspect attrition
flow <- attr(cohort, "participant_flow")
if (!is.null(flow)) print(flow)
 
# Check created columns without printing participant rows
stopifnot(all(c("AA_history", "AA_incident",
                "outcome_status", "outcome_surv_time") %in% names(cohort)))
cohort[, .(
  n = .N,
  n_incident = sum(outcome_status == 1, na.rm = TRUE),
  n_censored = sum(outcome_status == 0, na.rm = TRUE),
  n_prevalent = sum(is.na(outcome_status))
)]

Column semantics added by build_survival_dataset():

Prevalent ≠ at risk. Participants with <primary>_history = 1 have outcome_status = NA and outcome_surv_time = NA so that downstream Cox fits drop them automatically. When primary_disease is a vector, use the corresponding disease-specific _status and _surv_time columns instead.

Phase 5 — Optional: stratify incident cases by years since enrollment

For lag analyses ("MI in the first 5 years vs after"):

strat <- select_incident_by_years(
  df             = cohort,
  n_years        = 5,
  time_col       = "outcome_surv_time",
  status_col     = "outcome_status",
  baseline_col   = "p53_i0",
  event_date_col = "earliest_date",
  group_col      = "incident_timing",
  output         = "split",          # or "combined"
  copy           = TRUE,
  verbose        = TRUE
)
str(strat)
#> List of 2: $ within_5_years, $ after_5_years

output = "combined" returns one data.table with a incident_timing column taking values "within_5_years" / "after_5_years".

5. Decision rules the agent should ask the user about

Pause and ask before generating code if any of these is ambiguous:

1. Which disease(s)? Check get_disease_catalog(disease = "<term>")

and names(get_predefined_diseases()) first. Only build custom definitions if the requested endpoint is absent or the protocol requires different source/code rules.

1. `primary_disease`? One key returns outcome_status /

outcome_surv_time; multiple keys return independent disease-specific status/time columns. Default to the first disease, but confirm.

1. `prevalent_sources` vs `outcome_sources`? Defaults are:

• prevalent: c("ICD10", "ICD9", "Self-report", "Death") — broad,

because we want to exclude people who already have the disease

• outcome: c("ICD10", "ICD9", "Death") — Self-report excluded because

its year-only dates corrupt follow-up time Confirm whether the user wants to add OPCS4, CancerRegistry, FirstOccurrence, or Algorithm to outcomes.

1. `censor_date`? Default 2023-10-31. Should match the UKB data release

the user has access to — confirm with them.

1. `output = "wide"` vs `"long"`? Wide is the right default; long is for

per-source / per-disease audits.

1. `show_flow = TRUE`? Default on; the attrition table is **safe to

export** because it contains only aggregate counts.

6. Key functions (cheat-sheet)

See `references/functions.md` for full argument tables and return-value details.

7. Common pitfalls

1. `primary_disease` not in `disease_definitions`. Function errors out.

Always pass a name from names(disease_definitions).

1. Sources requested but columns missing. If dt lacks p41270,

requesting "ICD10" returns 0 cases silently and the user gets a misleading "no events" cohort. Inspect parse_icd10_diagnoses(dt) first when in doubt.

1. Self-report in outcomes. Self-report has only fractional-year

precision; including it in outcome_sources corrupts follow-up time. Default leaves it out; only override if you fully understand the cost.

1. Cancer histology codes. UKB stores them in p40011_i*. When using

cancer_histology, codes must match exactly (UKB uses the same numeric codes as the WHO ICD-O-3 dictionary).

1. First Occurrence special dates. UKB encodes "before 1900", "1901-01-01",

"2037-07-07", etc. as sentinel values; the parser drops them automatically — do not re-add them.

1. Date-format chaos. UKB columns may be character, R Date, or compact

YYYYMMDD numerics. extract_cases_by_source() routes everything through .safe_as_date(). Trust the function but read its warnings.

1. Censoring at wrong date. Default 2023-10-31 corresponds to one

specific UKB release. If the user is on a different release, override censor_date.

1. Combining `Algorithm` with hospital sources. UKB's algorithmic

outcomes already incorporate hospital and death data. Adding ICD10 and Algorithm to outcome_sources is usually redundant; pick one. Use compare_data_sources() to quantify the overlap.

1. `show_flow = TRUE` but `output = "long"`. The attrition table only

makes sense for wide format. Confirm before mixing.

8. Related skills

9. References

• `references/functions.md` — every function

signature, argument table, return-value details

• `references/disease-catalog.md` — the

pre-validated phenotype catalogue from get_predefined_diseases()

• `references/source-matrix.md` — which

source needs which UKB field IDs; recommended combinations

• `references/examples.md` — three minimal

end-to-end examples