mdhewei/bioinfor-claw/clustering-analysis

Clustering Analysis for Omics Data

Purpose

Partition samples (or features) into groups based on expression similarity. Enables unsupervised discovery of cell types, disease subtypes, co-expressed gene modules. Provides cluster quality metrics and consensus-based robustness assessment.

Use When

• Discovering cell types or disease subtypes in single-cell or bulk RNA-seq
• Finding co-expressed gene modules
• Validating cell population predictions
• Generating sample labels for downstream analysis
• Assessing clustering robustness via consensus methods
• Determining optimal number of clusters

Do Not Use When

• Sample count < 10 (unreliable clustering)
• Ground truth labels exist; use supervised classification instead
• Data is categorical (use appropriate clustering for categorical data)
• Real-time clustering needed (consensus clustering is slow)

Expected Inputs

Required

• --input: TSV numeric matrix (rows = features, columns = samples OR features to cluster). First column = feature/sample IDs. No missing values.

Optional

• --metadata: Sample metadata TSV for heatmap annotations
• --color-by: Metadata column for heatmap sidebar colors
• --consensus-n: Number of consensus iterations (default: 100; increase for more robust assessment)

Expected Outputs

• cluster_labels.tsv: sample_id, cluster_id, silhouette_score per sample
• cluster_summary.tsv: cluster_id, n_samples, centroid_distance_to_global_centroid
• silhouette_plot.png: Per-sample silhouette coefficient plot, sorted by cluster
• cluster_heatmap.png: Hierarchical clustered heatmap of expression, samples ordered by cluster
• dendrogram.png: Dendrogram for hierarchical method
• consensus_matrix.png: Heatmap of consensus co-clustering frequencies (if consensus method)
• elbow_plot.png: Silhouette scores for k in --k-range (if auto k-selection)

Procedure

1. Load and validate data: Read TSV, check for missing values. Optionally transpose (if --transpose flag).
2. Normalize: Apply scaling (standard = z-score, minmax = [0,1], none = no scaling).
3. Auto k-selection (if --n-clusters 0):

• Try each k in --k-range
• Compute mean silhouette score for each k
• Select k with highest silhouette score
• Plot elbow/silhouette curve

1. Clustering (apply selected method(s)):

• K-means: Lloyd's algorithm with k-means++ initialization, 100 iterations, 10 restarts, keep best inertia
• Hierarchical: Build distance matrix, apply linkage (ward/complete/average/single), cut dendrogram at --n-clusters
• DBSCAN: Compute pairwise distances, identify core points (eps-neighbors >= min-samples), expand clusters
• Consensus: Run hierarchical clustering --consensus-n times on subsampled data (--consensus-subsample fraction), build consensus matrix, cluster consensus matrix

1. Compute cluster quality metrics:

• Silhouette score per sample: s_i = (b_i - a_i) / max(a_i, b_i), where a_i = mean intra-cluster distance, b_i = min mean inter-cluster distance
• Mean silhouette per cluster
• Inertia (K-means only)
• Cophenetic correlation (hierarchical only)
• Consensus score (consensus clustering only)

1. Generate visualizations:

• Silhouette plot: horizontal bars per sample, sorted by cluster
• Heatmap: samples (columns) ordered by cluster assignment, rows = features, metadata sidebar
• Dendrogram: hierarchical clustering tree
• Elbow plot: silhouette vs k
• Consensus matrix: heatmap of co-clustering frequencies

Key Execution Patterns

# Hierarchical clustering, auto k-selection by silhouette
python clustering_analysis.py \
  --input expression_matrix.tsv \
  --method hierarchical \
  --n-clusters 0 \
  --k-range 2,10 \
  --outdir results/
 
# K-means with fixed k=5, colored heatmap by cell type
python clustering_analysis.py \
  --input expression_matrix.tsv \
  --method kmeans \
  --n-clusters 5 \
  --metadata sample_metadata.tsv \
  --color-by cell_type \
  --outdir results/
 
# Consensus clustering for robustness (100 iterations)
python clustering_analysis.py \
  --input expression_matrix.tsv \
  --method consensus \
  --n-clusters 5 \
  --consensus-n 100 \
  --consensus-subsample 0.8 \
  --outdir results/
 
# All methods for comparison
python clustering_analysis.py \
  --input expression_matrix.tsv \
  --method all \
  --n-clusters 4 \
  --linkage ward \
  --metadata metadata.tsv \
  --color-by treatment \
  --outdir results/
 
# Cluster genes (features) instead of samples
python clustering_analysis.py \
  --input expression_matrix.tsv \
  --method hierarchical \
  --transpose \
  --n-clusters 10 \
  --linkage complete \
  --outdir results/
 
# DBSCAN with custom epsilon
python clustering_analysis.py \
  --input expression_matrix.tsv \
  --method dbscan \
  --eps 0.5 \
  --min-samples 5 \
  --distance-metric euclidean \
  --outdir results/

Parameter Decision Guide

Failure Modes