yaalalabs/agent-kernel/ak-add-capabilities

Add Capabilities

Use this skill to enhance your Agent Kernel project with additional capabilities.

Instructions for the Agent

When the user wants to add a capability, follow this workflow:

Step 1: Identify the Project

Check for an existing Agent Kernel project with pyproject.toml and agent definition file.

Step 2: Ask Which Capability

Which capability would you like to add?

1. Guardrails — Content safety filters for input and/or output
2. Tracing — Observability and monitoring (Langfuse, OpenLLMetry, Pydantic Logfire, or AWS CloudWatch)
3. Session Persistence — Durable conversation state (Redis, DynamoDB, Cosmos DB, Firestore)
4. Knowledge Base — Durable cross-session knowledge tools (ChromaDB, Neo4j, Starburst, Open Knowledge Format markdown bundle, or custom backend)
5. MCP Server — Expose agents as Model Context Protocol tools
6. A2A Server — Agent-to-Agent communication protocol
7. Hooks — Custom pre/post processing (RAG, logging, prompt modification)
8. Multimodal — Image and file attachment support
9. Conversation Threads — Persistent, named conversation history keyed by session_id
10. Sandbox — Isolated code/command execution with pluggable providers, workload profiles, policy, and per-user identity
11. AG-UI Server — Stream any agent to an AG-UI-compliant frontend (text, tool calls, reasoning, shared state)
12. Scheduled Tasks — Deferred and recurring chat execution (a schedule block on a chat request, management routes, agent tools)
13. Secret Resolution — Read API keys through SecretManager (environment first, then AWS SSM Parameter Store, Kubernetes Secrets mounted into the pod, or a custom provider)
14. Human in the Loop — Pause a run for a person's approval or answer and resume from their decision (a gated tool or an interrupt, a resume block on the next request)
15. A2UI — Let an agent answer with an interface instead of prose: its JSON reply is labelled so the client receives a document, not a string

Step 3: Generate Changes

Guardrails

Ask: Input guardrails, output guardrails, or both? Which provider — OpenAI, AWS Bedrock, or Walled AI?

For OpenAI Guardrails:

1. Update pyproject.toml:

dependencies = [
    "agentkernel[openai,api]>=0.9.5",
    # OpenAI guardrails use the openai extra — already included if using OpenAI framework
]

1. Create guardrails_input.json:

[
  {
    "type": "moderation",
    "moderation_config": {
      "content_type": ["violence", "sexual", "harassment", "self-harm"],
      "threshold": 0.5
    }
  },
  {
    "type": "jailbreak",
    "jailbreak_config": {}
  }
]

1. Create guardrails_output.json:

[
  {
    "type": "pii_detection",
    "pii_config": {
      "output_handling": "block",
      "entities": ["email_address", "phone_number", "ssn"]
    }
  }
]

1. Update config.yaml:

guardrail:
  input:
    enabled: true
    type: openai
    model: gpt-4o-mini
    config_path: guardrails_input.json
  output:
    enabled: true
    type: openai
    config_path: guardrails_output.json

For AWS Bedrock Guardrails:

1. Update pyproject.toml:

dependencies = [
    "agentkernel[openai,api,aws]>=0.9.5",
]

1. Update config.yaml:

guardrail:
  input:
    enabled: true
    type: bedrock
    guardrail_id: "<your-bedrock-guardrail-id>"
    guardrail_version: "DRAFT"
  output:
    enabled: true
    type: bedrock
    guardrail_id: "<your-bedrock-guardrail-id>"
    guardrail_version: "DRAFT"

1. Prerequisites: Create a guardrail in AWS Bedrock Console and note the guardrail ID.

For Walled AI Guardrails:

1. Update pyproject.toml:

dependencies = [
    "agentkernel[openai,api,walledai]>=0.9.5",
]

1. Update config.yaml:

guardrail:
  input:
    enabled: true
    type: walledai
    pii: true   # enable PII redaction on input
  output:
    enabled: true
    type: walledai
    pii: true   # enable PII unmasking on output

1. Set the environment variable:

export WALLED_API_KEY="your-walledai-api-key"

1. How it works:

• Input: Text is checked for safety via WalledProtect. If unsafe, the request is blocked. If pii: true, text is redacted via WalledRedact and the PII mapping is stored in the session's non-volatile cache.
• Output: If pii: true, redacted placeholders in the agent's reply are replaced with the original values using the stored mapping.

Tracing (Observability)

Ask: Which tracing backend — Langfuse, OpenLLMetry (Traceloop), Pydantic Logfire, or AWS CloudWatch?

For Langfuse:

1. Update pyproject.toml:

dependencies = [
    "agentkernel[openai,api,langfuse]>=0.9.5",
]

1. Update config.yaml:

trace:
  enabled: true
  type: langfuse

1. Set environment variables:

export LANGFUSE_PUBLIC_KEY="pk-..."
export LANGFUSE_SECRET_KEY="sk-..."
export LANGFUSE_HOST="https://cloud.langfuse.com"   # or self-hosted URL

1. No code changes needed — tracing is automatically applied to all agent executions.

For OpenLLMetry (Traceloop):

1. Update pyproject.toml:

dependencies = [
    "agentkernel[openai,api,openllmetry]>=0.9.5",
]

1. Update config.yaml:

trace:
  enabled: true
  type: openllmetry

1. Set environment variables per the Traceloop documentation.

For Pydantic Logfire:

1. Update pyproject.toml:

dependencies = [
    "agentkernel[openai,api,logfire]>=0.9.5",
]

1. Update config.yaml:

trace:
  enabled: true
  type: logfire

1. Set the write token (optional — without it, Logfire runs locally and does not ship traces):

export LOGFIRE_TOKEN="your-write-token"

1. No code changes needed — tracing is automatically applied to all agent executions.

For AWS CloudWatch:

1. Update pyproject.toml:

dependencies = [
    "agentkernel[openai,api,cloudwatch]>=0.9.5",
]

1. Update config.yaml:

trace:
  enabled: true
  type: cloudwatch

1. Set the region; credentials come from the standard AWS chain (env, profile, SSO, or instance/task role):

export AWS_REGION="us-east-1"
# Optional: export through the CloudWatch agent or an OpenTelemetry collector instead of straight to X-Ray
# export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT="http://localhost:4318/v1/traces"

1. One-time AWS setup: enable CloudWatch Transaction Search in the account, and attach the

AWSXrayWriteOnlyAccess managed policy to the role that runs the agent.

1. No code changes needed — tracing is automatically applied to all agent executions.

Fetching spans back (any of the four backends): to read recorded spans into Python (e.g. to evaluate recent runs), call Trace.get().fetch_spans(...) with provider-neutral filters; results are FetchedSpans, oldest first (each has a status of UNSET, OK or ERROR):

from datetime import timedelta
from agentkernel.trace import SpanKind, Trace
tool_calls = Trace.get().fetch_spans(last=timedelta(hours=6), kinds=[SpanKind.TOOL])
spans = Trace.get().fetch_spans(session_id="session-123", limit=500)
tokens = sum(span.usage.total_tokens or 0 for span in spans if span.usage)  # span.usage: tokens and cost per model call
# Whole traces (spans grouped by trace, with the root, input/output and summed usage); skip a trace whose root is missing
for trace in Trace.get().fetch_traces(session_id="session-123"):
    if trace.root is not None:
        print(trace.user_input, trace.output, trace.usage)

• Langfuse and OpenLLMetry reuse the sending credentials.
• Logfire needs a separate read token: export LOGFIRE_READ_TOKEN="...". It and TRACELOOP_API_KEY are read

through SecretManager, so they can also come from AWS SSM (secret.provider.type: aws_ssm).

• CloudWatch reads aws/spans through Logs Insights, so the identity needs logs:StartQuery,

logs:GetQueryResults and logs:StopQuery (AWSXrayWriteOnlyAccess does not grant them).

• Full filter list and provider limits: Traceability guide.
• End-to-end example (fetch a session's traces, grade them with Opik judges, store the scores in DynamoDB):

`examples/aws-serverless/openai-langfuse-trace-evaluation`.

Session Persistence

Ask: Which backend — Redis, DynamoDB (AWS), Cosmos DB (Azure), or Firestore (GCP)?

For Redis:

1. Update pyproject.toml:

dependencies = [
    "agentkernel[openai,api,redis]>=0.9.5",
]

1. Update config.yaml:

session:
  type: redis
  cache: 256                    # LRU cache size (optional, improves performance)
  redis:
    prefix: "ak:<project>:"    # Key prefix for namespacing
    url: "redis://localhost:6379"
    ttl: 3600                   # Session TTL in seconds (optional)

For DynamoDB:

1. Update pyproject.toml:

dependencies = [
    "agentkernel[openai,api,aws]>=0.9.5",
]

1. Update config.yaml:

session:
  type: dynamodb
  cache: 256
  dynamodb:
    table_name: "<your-table-name>"
    region: "us-east-1"
    ttl: 3600

1. Create a DynamoDB table with partition key session_id (String) and sort key key (String). Enable TTL on expiry_time attribute.

For Cosmos DB:

1. Update pyproject.toml:

dependencies = [
    "agentkernel[openai,api,azure]>=0.9.5",
]

1. Update config.yaml:

session:
  type: cosmosdb
  cache: 256
  cosmosdb:
    endpoint: "https://<account>.table.cosmos.azure.com:443/"
    table_name: "<your-table-name>"
    ttl: 3600

1. Set AZURE_COSMOS_KEY environment variable.

For Firestore (GCP):

1. Update pyproject.toml:

dependencies = [
    "agentkernel[openai,api,gcp]>=0.9.5",
]

1. Update config.yaml:

session:
  type: firestore
  cache: 256
  firestore:
    collection_name: "ak_sessions"
    project_id: "<your-gcp-project-id>"   # optional, inferred from ADC if omitted
    ttl: 604800

1. Enable a TTL policy on the Firestore collection pointing to the expiry_time field for automatic document expiry.

Knowledge Base

Add durable knowledge tools that your agents can query and update across sessions.

Ask: Which backend do you want to use?

• ChromaDB (semantic/vector)
• Neo4j (graph/relationships)
• Starburst (read-only SQL via Trino)
• Open Knowledge Format bundle (a directory of markdown documents, on disk or in S3 — no database to run)
• Custom adapter (developer extension)

1. Update `pyproject.toml` dependencies based on backend:

dependencies = [
  "agentkernel[openai,api,chromadb]>=0.9.5",  # for Chroma
  # or "agentkernel[openai,api,neo4j]>=0.9.5"
  # or "agentkernel[openai,api,trino]>=0.9.5"
  # OKF needs NO extra - pyyaml is a core dependency:
  #    "agentkernel[openai,api]>=0.9.5"
  # ...unless the bundle is served from S3, which uses the aws extra:
  #    "agentkernel[openai,api,aws]>=0.9.5"
]

2. Configure backend + `KnowledgeBuilder` in your agent file (OpenAI example):

from agents import Agent
from agentkernel.cli import CLI
from agentkernel.knowledgebase.chroma import ChromaManager
from agentkernel.knowledgebase.knowledgebuilder import KnowledgeBuilder
from agentkernel.openai import OpenAIModule, OpenAIToolBuilder
backend = ChromaManager(name="ChromaDB").add_schema(
  {
    "description": "Semantic vector store for unstructured facts",
    "store_payload": {"text": "string", "source": "string"},
    "read_payload": {"query": "string", "limit": "int"},
  }
)