hinna0818/ukbanalytica_v2/ukbsci-ml

Claude Code and agent skill by hinna0818/ukbanalytica_v2

Trust tier 2
  • community
LLM review pending
Source check pending

Checking that the SKILL.md file is still in the repository.

Machine learning workflows (binary / multiclass classification and right-censored survival) for UK Biobank Research Analysis Platform (RAP) cohorts using UKBAnalytica. End-to-end pipelines (ukb_ml_workflow, ukb_ml_flow, ukb_ml_survival_workflow) cover frozen train / validation / test split, feature selection (Boruta, filter, glmnet), hyperparameter tuning (grid / random / Bayesian), final refit, test-set evaluation, model comparison, and model-agnostic interpretation via SHAP. Supports RF / XGBoost / glmnet / SVM / NNet / RPart / Naive Bayes / Logistic for classification, and RSF / GBM / coxnet for survival. Output evaluation: ROC, PR, calibration, DCA, KS, gain & lift, confusion matrix, threshold optimization, SHAP summary / dependence / force, plus plot_ml_* and plot_shap_* visualizations. Use this skill when the user asks for a UKB-based predictive model, ML pipeline, SHAP interpretation, C-index for a survival model, or comparison between several ML algorithms. Triggers: UKB ML, machine learning, XGBoost, random forest, SHAP, C-index, survival ML, calibration, decision curve, AUC, /ukbsci-ml. Hard rule: local agents must not read or inspect real UKB RAP participant-level data; generate scripts for RAP execution and interpret aggregate outputs only.

Install

npx vskill@latest install hinna0818/ukbanalytica_v2/ukbsci-ml

SKILL.md

ukbsci-ml — Machine learning workflows for UKB cohorts

0. RAP guardrails

Core rule: do not export UK Biobank participant-level raw data, direct identifiers (eid, exact dates, raw RAP fields), or row-level source tables that can be linked back to participants.

Strict local-agent boundary: this skill is for script generation, workflow planning, package guidance, and interpretation of aggregate outputs. The agent must not read, inspect, summarize, or process real UK Biobank RAP participant-level data, including de-identified row-level tables, raw RAP fields, exact dates, per-row predictions, row-level SHAP matrices, screenshots, tracebacks, or logs containing row-level values. Generate scripts for the user to run inside RAP; only aggregate results or rendered figures may be shared back with the agent. See ../references/agent-privacy-boundary.md.

OutputSharing rule
UK Biobank RAP raw participant-level dataDo not export.
Direct identifiers (eid, raw dates, RAP field values)Do not export.
Train / validation / test datasets with raw fields or eidDo not export.
Trained model .rdsKeep in RAP project workspace unless sharing is explicitly permitted by project governance.
Per-row predictions containing eid or re-identifiable row indexDo not export.
Per-row predictions without any identifierKeep in RAP; do not share with the local agent. Export only aggregate metrics or rendered figures.
Aggregate metrics table (AUC, C-index, calibration intercept, RMSE…)Shareable with the local agent.
ROC / PR / calibration / DCA / KS / gain plotsShareable as rendered figures.
SHAP global importance table (mean \SHAP\per feature)Shareable — aggregate only.
SHAP beeswarm / dependence / summary plotsShareable as rendered figures only, provided no row-level source data, row indices, or identifiers accompany the file.
SHAP force plot for a real participant rowDo not share with the local agent. Use a synthetic or representative prototype instead.

1. When to load

  • User wants a predictive model for a binary outcome (e.g. 10-year MACE

risk), a multiclass outcome (diabetes subtype), or a survival endpoint.

  • User wants to compare several algorithms on the same train / test split.
  • User wants SHAP-based interpretation.
  • User wants Decision Curve Analysis or threshold optimization.

2. When NOT to load

  • Effect estimation / inferential modeling → ukbsci-regression.
  • Mediation, propensity scoring → respective skills.
  • Just a Cox model without ML → ukbsci-regression.

3. Prerequisites

r
library(UKBAnalytica); library(data.table); library(survival)
 
# Suggests (install on RAP only what your chosen model needs):
# ranger xgboost glmnet e1071 nnet rpart randomForestSRC gbm MASS
# Boruta rBayesianOptimization fastshap pROC

Before using high-level workflow helpers, verify that the latest UKBAnalytica code is installed or loaded in the RAP session. If ukb_ml_flow() or ukb_ml_compare_flows() is not found, reinstall the current package version using the project-approved method or load the current source tree with pkgload::load_all().

After ML interface changes, run the shared skill smoke test:

r
Rscript inst/skills/UKBAnalytica_skills/evals/smoke_test_core.R

4. Recommended entry points

Use the right function for your task:

TaskRecommended function
Single model, most analysesukb_ml_flow()
Compare several algorithms or feature setsukb_ml_compare_flows()
Fully staged workflow with explicit feature selection + per-stage artefactsukb_ml_workflow()
Manual step-by-step controllower-level functions (ukb_ml_split_data, ukb_ml_tune, …)
Return-object cheat-sheet
FunctionKey slots in the returned object
ukb_ml_workflow()$split, $feature_result, $tune_result, $threshold_result, $final_model, $final_test_metrics, $final_test_predictions
ukb_ml_flow()$model, $metrics, $predictions, $roc, $shap (if compute_shap = TRUE)
ukb_ml_compare_flows()$flows, $metrics, $comparison, $predictions, $roc, $thresholds

Never mix components across workflow objects (e.g. workflow_obj$roc does not exist; use flow_obj$roc).

4a. ukb_ml_flow() — single model (recommended for most analyses)
r
flow <- ukb_ml_flow(
  formula      = outcome_status ~ age + sex + bmi + sbp + ldl_cholesterol +
                                   hba1c + crp + smoking_status,
  data         = cohort,
  model        = "xgboost",
  outcome_type = "binary",
  split_params = list(split = "train_valid_test",
                       train_ratio = 0.7,
                       validation_ratio = 0.1,
                       test_ratio = 0.2,
                       stratify_by = "outcome"),
  tune    = TRUE,
  tune_params = list(search = "random", n_iter = 30,
                      resampling = "validation",
                      metric = "auc", maximize = TRUE),
  threshold_method = "youden",
  compute_shap = TRUE,
  shap_params  = list(nsim = 100, sample_n = 2000),
  seed    = 1234,
  verbose = TRUE
)
 
flow$metrics                   # aggregate — shareable with the local agent
# flow$predictions              # keep on RAP (may contain row indices)
 
ggplot2::ggsave("/mnt/project/<area>/05-figs/Fig13-roc.pdf",
                plot_ml_roc(flow))
ggplot2::ggsave("/mnt/project/<area>/05-figs/Fig14-calib.pdf",
                plot_ml_calibration(flow))
ggplot2::ggsave("/mnt/project/<area>/05-figs/Fig15-dca.pdf",
                plot_ml_dca(flow))
4b. ukb_ml_workflow() — staged workflow with explicit feature selection

Use when you need per-stage artefacts (Boruta importance table, tuning grid, held-out validation metrics).

r
flow <- ukb_ml_workflow(
  formula        = outcome_status ~ .,
  data           = cohort,
  model          = "xgboost",
  outcome_type   = "binary",
  split_params   = list(split = "train_valid_test",
                         train_ratio = 0.7, validation_ratio = 0.1,
                         test_ratio = 0.2, stratify_by = "outcome"),
  feature_select = "boruta",
  tune           = TRUE,
  tune_params    = list(search = "random", n_iter = 30,
                         resampling = "validation",
                         metric = "auc", maximize = TRUE),
  threshold_method = "youden",
  fit_final      = TRUE,
  evaluate_test  = TRUE,
  seed           = 1234
)
 
flow$final_test_metrics                           # aggregate — shareable
# flow$final_test_predictions                     # keep on RAP
 
ggplot2::ggsave("/mnt/project/<area>/05-figs/Fig13-roc.pdf",
                plot_ml_roc(flow$final_model, newdata = flow$split$test))

Note on `use_validation_in_refit`: ukb_ml_fit_final() uses the use_validation_in_refit argument to control whether validation-set rows are folded into the final refit. This defaults to TRUE in ukb_ml_fit_final() itself. In high-level frozen-test workflows such as ukb_ml_flow() and ukb_ml_compare_flows(), check the function default and pass this argument explicitly when you need to include or exclude validation rows from the final model.

4b. Survival — ukb_ml_survival_workflow()
r
sflow <- ukb_ml_survival_workflow(
  formula = survival::Surv(outcome_surv_time, outcome_status) ~
              age + sex + bmi + sbp + ldl_cholesterol + hba1c,
  data    = cohort,
  model   = "rsf",                          # "rsf", "gbm_surv", "coxnet"
  split_params = list(split = "train_valid_test",
                       train_ratio = 0.7, validation_ratio = 0.1,
                       test_ratio = 0.2,
                       stratify_by = "event"),
  tune    = TRUE,
  tune_params = list(search = "grid", metric = "c_index"),
  fit_final     = TRUE,
  evaluate_test = TRUE,
  seed   = 1234
)
sflow$final_test_metrics      # c_index + (optional) time-dependent IBS
sflow$final_test_predictions  # per-row survival probabilities — RAP-only

Predict at specific horizons:

r
preds <- ukb_ml_survival_predict(
  sflow,
  newdata = sflow$split$test,
  times   = c(1, 3, 5, 10),
  type    = "survival"        # "survival", "risk", "chf"
)

5. Sub-steps (when you don't want the all-in-one workflow)

r
# 1. Frozen split
split <- ukb_ml_split_data(
  cohort, outcome = "outcome_status",
  outcome_type = "binary",
  split = "train_valid_test",
  train_ratio = 0.7, validation_ratio = 0.1, test_ratio = 0.2,
  stratify_by = "outcome", seed = 1234
)
 
# 2. Feature selection
fs <- ukb_ml_feature_select(split, formula = outcome_status ~ .,
                             method = "boruta", boruta_params = list(),
                             seed = 1234)
formula_fs <- fs$formula
 
# 3. Tuning
tune <- ukb_ml_tune(split, formula = formula_fs, model = "xgboost",
                     search = "random", n_iter = 30,
                     resampling = "validation", metric = "auc",
                     seed = 1234)
best <- tune$best_params
 
# 4. Final fit
final <- ukb_ml_fit_final(split, formula = formula_fs, model = "xgboost",
                           best_params = best, use_validation_in_refit = TRUE,
                           seed = 1234)
 
# 5. Test evaluation
eval <- ukb_ml_evaluate_test(final, split, threshold = NULL)
eval$metrics

Use this sub-stepped flow when the user needs per-stage artefacts (e.g. Boruta status table, tuning grid history) for the manuscript.

6. Comparison across models

r
cmp <- ukb_ml_compare_flows(
  formula   = outcome_status ~ .,
  data      = cohort,
  models    = c("rf","xgboost","glmnet","logistic"),
  outcome_type = "binary",
  split_params = list(split = "train_test", train_ratio = 0.7,
                       stratify_by = "outcome"),
  tune_params  = list(search = "random", n_iter = 30),
  feature_params = list(method = "none"),
  n_repeats = 1, seed = 1234
)
# Plot overlay
ggplot2::ggsave("/mnt/project/<area>/05-figs/Fig16-roc_compare.pdf",
                plot_ml_roc_compare(cmp))

Alternative: compare feature sets with one model:

r
cmp_fs <- ukb_ml_compare_feature_sets(
  split, formula = outcome_status ~ .,
  models = "xgboost",
  feature_selections = list(none = NULL, boruta = list(),
                             filter = list(top_k = 20))
)

7. SHAP interpretation

r
shap <- ukb_shap(
  flow,                       # ukb_ml_flow or ukb_ml_workflow object
  data    = flow$split$test,
  nsim    = 100,
  sample_n = 2000,            # random subsample for speed
  seed    = 1234,
  method  = "auto"            # "auto" picks native XGBoost SHAP for tree models
)

Sharing rules for SHAP outputs:

SHAP outputSharing rule
Importance table (mean \SHAP\per feature)Shareable — aggregate summary, no identifiers.
Summary / beeswarm / dependence plotsShareable as rendered figures only; do not attach the row-level SHAP source matrix or feature-value table.
Force plot for a real participant rowDo not share with the local agent. Use a synthetic or representative prototype — a row constructed from group medians, not an actual participant.
Raw shap$shap_values matrixKeep on RAP and out of the agent context; it has one row per observation, even if eid is absent.
r
# Summary table is aggregate; beeswarm is a rendered figure. Do not export
# the row-level SHAP matrix or feature-value table.
fwrite(ukb_shap_summary(shap, top_n = 20, plot = FALSE),
       "/mnt/project/<area>/04-results/08-ml_shap_summary.csv")
ggplot2::ggsave("/mnt/project/<area>/05-figs/Fig17-shap_summary.pdf",
                plot_shap_summary(shap, top_n = 15))
ggplot2::ggsave("/mnt/project/<area>/05-figs/Fig18-shap_beeswarm.pdf",
                plot_shap_beeswarm(shap, top_n = 15))

This is the start of the file. Read the full SKILL.md on GitHub.

Security

  • Security scanPassed, 100 of 100Pattern checks on the skill's files.
  • Model reviewNot runA model review is scheduled for the next rescan.

See the full security report

Quality evaluation

  • Task evaluationNot runNo evaluation yet. Run one in Skill Studio.

See evaluation results

Source

Popularity

Repository stars
35
7-day trend score
18

Stars and forks are for the whole GitHub repository, not this skill alone. Refreshed 29 Aug 2026, 2 months ago.

Works with

Any agent vskill installs to: 56 targets in all. These 39 have a page on this site.

  • Claude Code
  • Cursor
  • GitHub Copilot
  • Windsurf
  • Codex
  • Gemini CLI
  • Cline
  • Amp
  • Roo Code
  • Goose
  • Continue
  • Kilo Code
  • Junie
  • Trae
  • OpenHands
  • Qwen Code
  • AI Agent
  • Tabnine Chat
  • Sourcegraph Cody
  • Amazon Q
  • Firebase Genkit
  • Sweep
  • Devin
  • Phind
  • Mentat
  • Claude Desktop
  • Aider
  • GPT Pilot
  • Plandex
  • bolt.new
  • v0
  • Replit Agent
  • Anse
  • TypingMind
  • BetterChatGPT
  • LibreChat
  • Warp AI
  • Pieces
  • Fig AI
Embed badge
vskill VERIFIED badge
[![vskill VERIFIED](https://verified-skill.com/api/v1/skills/hinna0818/ukbanalytica_v2/ukbsci-ml/badge)](https://verified-skill.com/skills/hinna0818/ukbanalytica_v2/ukbsci-ml)

Something wrong with this skill or its page? Tell us.