version: "1.0.0" id: cell_segmentation_index name: Cell Segmentation Skills Index description: | Cell and nucleus segmentation tools for microscopy images. Covers Cellpose, SAM-based methods, StarDist, InstanSeg, and Mesmer. tags: [segmentation, cellpose, sam, stardist, instanseg, mesmer, nucleus, cell]

Cell & Nucleus Segmentation Skills

Instance segmentation tools for cells and nuclei in microscopy images. Use the tool selection guide below to choose the right method, then load the corresponding skill file for detailed usage.

Tool Selection Guide

[!TIP]

Start with Cellpose (default in v4.x) for most tasks. It has the best

generalization. Switch to InstanSeg if speed matters or you need

simultaneous nuclei + cell masks.

[!WARNING]

Environment isolation is important. These tools have conflicting

dependencies. Cellpose/InstanSeg use PyTorch; StarDist/Mesmer use TensorFlow;

SAM-based tools need Python 3.10+. Create separate virtual environments

for each tool family:

- venv-cellpose: Cellpose + InstanSeg (both PyTorch)

- venv-stardist: StarDist (TensorFlow, numpy<2)

- venv-deepcell: Mesmer/DeepCell (TensorFlow, strict numpy version)

- venv-sam: micro-sam / CellSAM (Python 3.10+)

Available Skills

Cellpose

General-purpose cell and nucleus segmentation using Cellpose v4.x (includes Cellpose-SAM with ViT-L backbone). Image restoration, fine-tuning, and 3D segmentation.

Skill file: cellpose.md

When to use: Default choice for most segmentation tasks.

InstanSeg

Fast cell and nucleus segmentation with dual output (nuclei + cells simultaneously). Supports multiplexed images via ChannelNet.

Skill file: instanseg.md

When to use: Speed-critical workflows, multiplexed images, QuPath integration.

StarDist

Nuclear segmentation using star-convex polygon prediction. Extremely fast but assumes round/convex nuclei.

Skill file: stardist.md

When to use: Round nuclei in fluorescence images where speed matters.

Mesmer / DeepCell

Whole-cell segmentation using both nuclear and membrane markers. TissueNet-trained PanopticNet architecture.

Skill file: mesmer.md

When to use: Tissue images with both nuclear and membrane/cytoplasm markers.

SAM-Based Cell Segmentation

Cell segmentation using SAM adaptations: CellSAM (automatic), micro-sam (interactive + 3D), SAMCell (label-free).

Skill file: sam_based.md

When to use: Interactive annotation, 3D/tracking, or label-free brightfield.