ndpvt-web/arxiv-claude-skills/alignagent-adaptive-learner-intelligence

ALIGNAgent: Adaptive Learner Intelligence for Gap Identification and Next-step Guidance

This skill enables Claude to build multi-agent educational systems that close the loop between assessment, diagnosis, and intervention. Based on the ALIGNAgent framework, the approach processes learner performance data (quiz scores, gradebook records, preferences) through a Skill Gap Agent that computes topic-level proficiency scores and performs concept-level diagnostic reasoning to identify specific misconceptions, then routes deficiencies to a Recommender Agent that retrieves preference-aware learning materials. The key innovation is the continuous feedback loop: interventions occur before advancing to subsequent topics, creating an iterative remediation cycle rather than a one-shot recommendation.

When to Use

• When building a personalized learning platform that needs to diagnose what a student doesn't know and why
• When the user wants to analyze quiz or assessment data to compute per-topic proficiency scores
• When creating a tutoring system that adapts resource recommendations to individual learning gaps
• When implementing a multi-agent pipeline where one agent diagnoses and another recommends
• When the user needs to classify learner proficiency (High/Medium/Low) from gradebook data and validate against exam performance
• When designing formative assessment cycles that loop diagnosis into intervention before progression

Key Technique

Two-Agent Architecture with Continuous Feedback Loop. ALIGNAgent separates concerns into a Skill Gap Agent and a Recommender Agent connected by a structured handoff. The Skill Gap Agent computes topic-level proficiency as rho_t = (1/|I_t|) * sum(g_i for i in I_t) where I_t is the set of assessment items for topic t and g_i is the normalized score on item i. It then applies a mastery threshold tau to produce a gap set G(s) = {(t, rho_t) | rho_t < tau}, sorted by ascending proficiency so the weakest areas get attention first. Beyond this quantitative layer, the agent performs concept-level diagnostic reasoning -- analyzing patterns in incorrect answers, distractor selection frequency, and cross-item reasoning to produce natural-language explanations of why a learner is struggling (e.g., "confuses AVL rotation direction" rather than just "low score on AVL trees").

Preference-Aware Resource Retrieval. The Recommender Agent constructs structured queries from the gap set and learner preferences (video vs. text, self-paced vs. structured), retrieves candidate resources, validates them against modality preferences and topic relevance, and returns up to K resources per gap. This is diagnostic-driven retrieval -- the query is shaped by the specific misconception, not just the broad topic.

Formative Loop, Not One-Shot. The critical design principle is that the pipeline re-executes after intervention. New assessment data updates the proficiency vector, and topics that remain below threshold trigger fresh recommendations. This makes ALIGNAgent a cycle, not a pipeline -- each iteration refines the learner model.

Step-by-Step Workflow

1. Define the domain schema. Create a topic taxonomy for your subject area as a structured map: {topic_id: {name, prerequisite_topics[], concept_tags[]}}. Each assessment item must be tagged with its topic and concept(s).

1. Ingest learner data. Parse three input streams into a unified learner profile: (a) gradebook records with per-item scores normalized to [0, 1], (b) quiz/assessment responses including selected answers (not just correctness), and (c) learner preferences (modality, pacing, prior stated goals).

1. Compute topic-level proficiency. For each topic t, calculate rho_t = mean(normalized_scores for items tagged with t). Assemble the full proficiency vector rho(s) = [rho_t1, rho_t2, ..., rho_tn].

1. Classify proficiency levels. Map each rho_t to a categorical label using thresholds: High (rho_t >= 0.8), Medium (0.5 <= rho_t < 0.8), Low (rho_t < 0.5). These thresholds should be configurable.

1. Identify skill gaps. Apply mastery threshold tau (default: 0.7) to produce the gap set G(s) = {(t, rho_t) | rho_t < tau}. Sort gaps by ascending rho_t so the weakest topics are prioritized.

1. Run concept-level diagnostic reasoning. For each topic in the gap set, analyze the learner's incorrect responses: which distractors were selected, which concepts those distractors test, and whether errors cluster around a specific misconception. Produce a natural-language diagnostic per gap (e.g., "Consistently confuses DFS with BFS traversal order; correctly identifies base cases but fails on recursive step for graph cycles").

1. Construct retrieval queries. For each diagnosed gap, build a structured query combining: topic name, specific misconception description, and learner modality preference. Example: {topic: "AVL Trees", misconception: "rotation direction after double rotation", preference: "video tutorial"}.

1. Retrieve and filter resources. Search for candidate resources matching each query. Filter by: (a) topic relevance to the specific misconception, (b) modality alignment with learner preferences, (c) content quality signals. Return up to K resources per gap, ranked by alignment score.

1. Generate learner summary. Produce a structured output containing: strengths (topics with High proficiency), gaps (topics below threshold with diagnostic explanations), recommended actions (ordered resource list per gap), and next assessment targets.

1. Close the feedback loop. After the learner engages with resources and completes a new assessment, re-run steps 3-9 with updated data. Track proficiency delta per topic to measure intervention effectiveness and adjust recommendations accordingly.

Concrete Examples

Example 1: Analyzing quiz performance in a Python programming course

User: "I have quiz data for 30 students across 5 Python topics. Help me build a system that identifies each student's weak areas and recommends resources."

Approach:

1. Parse the quiz CSV with columns: student_id, quiz_id, topic, question_id, score, max_score
2. Compute per-student, per-topic proficiency scores
3. Apply threshold to identify gaps
4. Generate diagnostic reasoning per gap
5. Match resources to gaps and preferences

Output structure:

{
  "student_id": "S042",
  "proficiency_vector": {
    "variables_types": 0.92,
    "control_flow": 0.78,
    "functions": 0.45,
    "data_structures": 0.33,
    "file_io": 0.60
  },
  "proficiency_labels": {
    "variables_types": "High",
    "control_flow": "Medium",
    "functions": "Low",
    "data_structures": "Low",
    "file_io": "Medium"
  },
  "skill_gaps": [
    {
      "topic": "data_structures",
      "proficiency": 0.33,
      "diagnostic": "Correctly uses lists but fails when indexing nested structures. Confuses dict key access with list indexing. Missed all questions involving dictionary comprehensions.",
      "resources": [
        {"title": "Python Data Structures - Nested Collections", "type": "video", "url": "..."},
        {"title": "Dictionary Comprehension Practice Problems", "type": "interactive", "url": "..."}
      ]
    },
    {
      "topic": "functions",
      "proficiency": 0.45,
      "diagnostic": "Understands function definition syntax but struggles with return vs print distinction. Errors cluster around scope-related questions -- treats local variables as global.",
      "resources": [
        {"title": "Python Scoping Rules Explained", "type": "article", "url": "..."},
        {"title": "Functions and Return Values - Step by Step", "type": "video", "url": "..."}
      ]
    }
  ],
  "strengths": ["variables_types"],
  "next_assessment": ["data_structures", "functions"]
}

Example 2: Building the multi-agent system in code

User: "Implement the ALIGNAgent two-agent pipeline as a Python module."

Approach:

1. Define data models for learner profiles, assessments, and proficiency vectors
2. Implement the Skill Gap Agent as a class with proficiency computation and LLM-based diagnostic reasoning
3. Implement the Recommender Agent with query construction and resource retrieval
4. Wire them together in a pipeline that accepts assessment data and returns a learner summary

from dataclasses import dataclass, field
from typing import Optional
import json
@dataclass
class AssessmentItem:
    question_id: str
    topic: str
    concepts: list[str]
    score: float          # normalized to [0, 1]
    selected_answer: str  # the actual answer chosen
    correct_answer: str
@dataclass
class LearnerProfile:
    student_id: str
    preferences: dict     # e.g., {"modality": "video", "pacing": "self-paced"}
    assessments: list[AssessmentItem] = field(default_factory=list)
@dataclass
class SkillGap:
    topic: str
    proficiency: float
    diagnostic: str       # natural-language explanation
    misconceptions: list[str]
class SkillGapAgent:
    def __init__(self, mastery_threshold: float = 0.7, llm_client=None):
        self.tau = mastery_threshold
        self.llm = llm_client
    def compute_proficiency(self, profile: LearnerProfile) -> dict[str, float]:
        topic_scores: dict[str, list[float]] = {}
        for item in profile.assessments:
            topic_scores.setdefault(item.topic, []).append(item.score)
        return {t: sum(s) / len(s) for t, s in topic_scores.items()}
    def identify_gaps(self, proficiency: dict[str, float]) -> list[tuple[str, float]]:
        gaps = [(t, rho) for t, rho in proficiency.items() if rho < self.tau]
        return sorted(gaps, key=lambda x: x[1])  # weakest first
    def diagnose(self, profile: LearnerProfile, gaps: list[tuple[str, float]]) -> list[SkillGap]:
        diagnosed = []
        for topic, rho in gaps:
            incorrect = [a for a in profile.assessments
                         if a.topic == topic and a.score < 1.0]
            # Use LLM for concept-level diagnostic reasoning
            prompt = self._build_diagnostic_prompt(topic, incorrect)
            reasoning = self.llm.generate(prompt, temperature=0)
            diagnosed.append(SkillGap(
                topic=topic, proficiency=rho,
                diagnostic=reasoning["explanation"],
                misconceptions=reasoning["misconceptions"]
            ))
        return diagnosed
    def _build_diagnostic_prompt(self, topic: str, incorrect_items: list) -> str:
        items_desc = json.dumps([{
            "question_id": i.question_id,
            "concepts": i.concepts,
            "selected": i.selected_answer,
            "correct": i.correct_answer
        } for i in incorrect_items], indent=2)
        return f"""Analyze these incorrect responses for topic "{topic}".
Identify specific misconceptions by examining which distractors were chosen and what conceptual errors they reveal.
Incorrect responses:
{items_desc}
Return JSON with:
- "explanation": one paragraph describing the pattern of errors
- "misconceptions": list of specific misconception strings"""
class RecommenderAgent:
    def __init__(self, max_resources_per_gap: int = 3, search_fn=None):
        self.k = max_resources_per_gap
        self.search = search_fn
    def recommend(self, gaps: list[SkillGap], preferences: dict) -> dict[str, list]:
        recommendations = {}
        for gap in gaps:
            query = self._build_query(gap, preferences)
            candidates = self.search(query)
            filtered = [r for r in candidates
                        if self._matches_preference(r, preferences)]
            recommendations[gap.topic] = filtered[:self.k]
        return recommendations
    def _build_query(self, gap: SkillGap, preferences: dict) -> str:
        modality = preferences.get("modality", "any")
        return f"{gap.topic} {gap.misconceptions[0]} {modality} tutorial"