<< All versions
Skill v1.0.0
currentAutomated scan100/100wenmin-wu/ds-skills/adjacent-span-merging
──Details
PublishedOctober 20, 2026 at 02:19 PM
Content Hashsha256:efbbe36405ba4607...
Git SHA718b09722905
──Files
Files (1 file, 2.6 KB)
SKILL.md2.6 KBactive
SKILL.md · 70 lines · 2.6 KB
version: "1.0.0" name: nlp-adjacent-span-merging description: > Merges nearby predicted NER spans of the same class within a word-distance threshold into single coherent segments.
Adjacent Span Merging
Overview
Token-level NER models often fragment a single discourse element into multiple adjacent spans — a long "Evidence" paragraph gets split into two or three pieces. Adjacent span merging fixes this by joining same-class spans that are within a configurable word-distance gap. This is especially important for long-document NER where discourse elements span many sentences.
Quick Start
python
def merge_adjacent_spans(predictions, max_gap=1, merge_classes=None):"""Merge nearby same-class spans within max_gap words.Args:predictions: list of (doc_id, class, word_indices_str)max_gap: max word-index gap to merge acrossmerge_classes: set of classes to merge (None = all)Returns:list of merged (doc_id, class, word_indices_str)"""from collections import defaultdictby_doc_class = defaultdict(list)for doc_id, cls, pred_str in predictions:words = sorted(int(x) for x in pred_str.split())by_doc_class[(doc_id, cls)].append(words)merged = []for (doc_id, cls), spans in by_doc_class.items():if merge_classes and cls not in merge_classes:for s in spans:merged.append((doc_id, cls, " ".join(map(str, s))))continue# Sort spans by start positionall_words = sorted(set(w for s in spans for w in s))# Split into groups by gapgroups = [[all_words[0]]]for w in all_words[1:]:if w - groups[-1][-1] <= max_gap:groups[-1].append(w)else:groups.append([w])for g in groups:merged.append((doc_id, cls, " ".join(map(str, g))))return merged
Workflow
- Group predictions by (document_id, class)
- Sort word indices across all spans of the same class
- Walk through sorted indices; if gap between consecutive words <= max_gap, merge
- Otherwise start a new span
- Optionally limit merging to specific classes (e.g., only "Evidence")
Key Decisions
- max_gap: 1-2 for tight merging; higher values risk over-merging distinct spans
- Selective classes: Only merge classes known to fragment (long spans like Evidence)
- Max merged length: Optionally cap merged span length to prevent runaway merging
- Order: Apply merging before length/probability filtering