byaan-ai/byaan/cloudwatch-logs

CloudWatch Logs API

Use the AWS CloudWatch Logs API to query log groups, search log events, and run CloudWatch Logs Insights queries.

Setup Guide

This skill requires read-only access to AWS CloudWatch Logs. No write or delete permissions are needed — Byaan only reads your logs, it never modifies them.

Choose an authentication mode

• Instance IAM role (recommended when Byaan runs on AWS) — no keys to create, store, or rotate. Byaan uses the AWS default credential chain (EC2 instance profile / ECS task role / env credentials). Select "Instance IAM role (auto)" and set only the Region.
• Access keys — for Byaan deployments outside AWS. An IAM user with programmatic access (Access Key ID + Secret Access Key) and read-only CloudWatch Logs permissions. Create a dedicated IAM user for Byaan rather than reusing an admin account.

Option A: Instance IAM role

1. In the AWS Console → IAM → Roles, create (or reuse) a role for the EC2 instance / ECS task running Byaan
2. Attach the CloudWatchLogsReadOnlyAccess managed policy, or the custom read-only policy shown below
3. Attach the role to the instance (EC2 → Instance → Actions → Security → Modify IAM role) or the ECS task definition
4. If Byaan runs in Docker on EC2 (the standard self-hosted setup), the container must be able to reach instance metadata. IMDSv2's default hop limit of 1 blocks containers — raise it once:

aws ec2 modify-instance-metadata-options --instance-id i-xxxx --http-put-response-hop-limit 2 --http-tokens required

1. In Byaan, go to Settings > Skills > CloudWatch Logs, choose Instance IAM role (auto), and set the Region

Option B: Access keys — step-by-step

1. In the AWS Console → IAM → Users, create a new user (e.g., byaan-cloudwatch-reader)
2. Attach the AWS managed policy CloudWatchLogsReadOnlyAccess, or create a custom policy with these specific actions:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "logs:DescribeLogGroups",
        "logs:DescribeLogStreams",
        "logs:FilterLogEvents",
        "logs:GetLogEvents",
        "logs:GetLogRecord",
        "logs:StartQuery",
        "logs:GetQueryResults",
        "logs:StopQuery"
      ],
      "Resource": "*"
    }
  ]
}

You can restrict Resource to specific log groups if you want tighter scoping (e.g., "arn:aws:logs:us-east-1:123456789:log-group:/aws/lambda/*").

1. Under Security credentials, create an Access Key (select "Third-party service" as the use case)
2. Copy the Access Key ID and Secret Access Key
3. In Byaan, go to Settings > Skills > CloudWatch Logs and paste:

• Access Key ID — starts with AKIA...
• Secret Access Key — the paired secret
• Region — the AWS region where your logs live (e.g., us-east-1). CloudWatch Logs are regional, so pick the region matching your services

How Authentication Works

You do not need to handle authentication manually. When you call execute_skill_api(), the system automatically:

1. Retrieves the AWS credentials from the configured skill
2. Creates a boto3 CloudWatch Logs client with those credentials
3. Calls the specified action with the provided parameters

Just call the API with skill name, action name, and parameters — authentication is handled for you.

How to Call the API

CloudWatch Logs uses action-based invocation. Set endpoint_path to the action name (PascalCase) and body to a JSON string of parameters.

execute_skill_api(
  skill_name="cloudwatch_logs",
  endpoint_path="DescribeLogGroups",
  method="POST",
  body="{}"
)

Parameters:

• skill_name: Always "cloudwatch_logs"
• endpoint_path: The action name in PascalCase (e.g., "FilterLogEvents", "DescribeLogGroups")
• method: Always "POST"
• body: JSON string of parameters for the action

Key Operations

DescribeLogGroups — List log groups

execute_skill_api(
  skill_name="cloudwatch_logs",
  endpoint_path="DescribeLogGroups",
  method="POST",
  body="{}"
)

With prefix filter:

execute_skill_api(
  skill_name="cloudwatch_logs",
  endpoint_path="DescribeLogGroups",
  method="POST",
  body="{\"logGroupNamePrefix\": \"/aws/lambda/\"}"
)

DescribeLogStreams — List streams in a log group

execute_skill_api(
  skill_name="cloudwatch_logs",
  endpoint_path="DescribeLogStreams",
  method="POST",
  body="{\"logGroupName\": \"/aws/lambda/my-function\", \"orderBy\": \"LastEventTime\", \"descending\": true, \"limit\": 10}"
)

FilterLogEvents — Search logs by pattern

This is the primary way to search for log entries. Supports filter patterns.

execute_skill_api(
  skill_name="cloudwatch_logs",
  endpoint_path="FilterLogEvents",
  method="POST",
  body="{\"logGroupName\": \"/aws/lambda/my-function\", \"filterPattern\": \"ERROR\", \"startTime\": 1700000000000, \"endTime\": 1700086400000, \"limit\": 100}"
)

Important: startTime and endTime are epoch milliseconds (not seconds). Multiply Unix timestamps by 1000.

GetLogEvents — Read a specific stream

execute_skill_api(
  skill_name="cloudwatch_logs",
  endpoint_path="GetLogEvents",
  method="POST",
  body="{\"logGroupName\": \"/aws/lambda/my-function\", \"logStreamName\": \"2024/01/15/[$LATEST]abc123\", \"startFromHead\": true, \"limit\": 50}"
)

GetLogRecord — Get full log record by pointer

When Insights queries return results, each record includes a @ptr field. Use GetLogRecord to retrieve the complete log event from that pointer.

execute_skill_api(
  skill_name="cloudwatch_logs",
  endpoint_path="GetLogRecord",
  method="POST",
  body="{\"logRecordPointer\": \"CpMBCgpBY2...(ptr value from Insights)\"}"
)

This is useful for drill-down: run an Insights query to find interesting records, then use @ptr to get the full untruncated log event.

StartQuery + GetQueryResults — CloudWatch Logs Insights

Insights queries are asynchronous. You must start the query, then poll for results.

Step 1: Start the query

execute_skill_api(
  skill_name="cloudwatch_logs",
  endpoint_path="StartQuery",
  method="POST",
  body="{\"logGroupNames\": [\"/aws/lambda/my-function\"], \"startTime\": 1700000000, \"endTime\": 1700086400, \"queryString\": \"fields @timestamp, @message | filter @message like /ERROR/ | sort @timestamp desc | limit 50\"}"
)

Note: For StartQuery, startTime and endTime are epoch seconds (not milliseconds, unlike FilterLogEvents).

This returns a queryId.

Step 2: Get results (poll until status is "Complete")

execute_skill_api(
  skill_name="cloudwatch_logs",
  endpoint_path="GetQueryResults",
  method="POST",
  body="{\"queryId\": \"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx\"}"
)

The response includes a status field. Poll until it is "Complete". Possible values: Scheduled, Running, Complete, Failed, Cancelled, Timeout.

StopQuery — Cancel a running Insights query

If a query is taking too long or the user wants to abort, cancel it with StopQuery:

execute_skill_api(
  skill_name="cloudwatch_logs",
  endpoint_path="StopQuery",
  method="POST",
  body="{\"queryId\": \"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx\"}"
)

Returns true if the query was successfully stopped. Use this when GetQueryResults keeps returning Running after multiple polls.

CloudWatch Logs Insights Query Syntax

Insights uses a pipe-delimited query language:

fields @timestamp, @message, @logStream
| filter @message like /ERROR/
| sort @timestamp desc
| limit 100

Commands

• fields: Select fields to display — fields @timestamp, @message
• filter: Filter results — filter @message like /pattern/ or filter statusCode >= 400
• sort: Order results — sort @timestamp desc
• limit: Cap results — limit 100
• stats: Aggregation — stats count(*) by bin(1h) or stats avg(duration) by functionName
• parse: Extract fields from messages — parse @message "Duration: * ms" as duration
• display: Choose which fields appear in output (after stats)

Built-in Fields

• @timestamp — Event timestamp
• @message — Full log message
• @logStream — Log stream name
• @log — Log group identifier
• @ingestionTime — When CloudWatch received the event

Example Insights Queries

Count errors per hour:

fields @timestamp, @message
| filter @message like /ERROR/
| stats count(*) as errorCount by bin(1h)
| sort errorCount desc

Average Lambda duration:

filter @type = "REPORT"
| parse @message "Duration: * ms" as duration
| stats avg(duration) as avgDuration, max(duration) as maxDuration by bin(1h)

Top 10 most frequent error messages:

filter @message like /ERROR/
| parse @message "ERROR *" as errorMsg
| stats count(*) as cnt by errorMsg
| sort cnt desc
| limit 10

Request tracing by ID:

fields @timestamp, @message, @logStream
| filter @message like /request-id-here/
| sort @timestamp asc

P99 latency:

filter @type = "REPORT"
| parse @message "Duration: * ms" as duration
| stats pct(duration, 99) as p99, pct(duration, 95) as p95, avg(duration) as avg by bin(1h)

Log Analysis Patterns

Use these Insights query recipes for operational analysis. These replicate the kind of analysis that dedicated log analysis tools perform.

Error Pattern Detection

Identify distinct error types and their frequency:

filter @message like /(?i)(error|exception|fatal|fail)/
| parse @message /(?<errorType>[\w.]+Exception|[\w.]+Error|FATAL|FAIL\w*)/
| stats count(*) as occurrences by errorType
| sort occurrences desc
| limit 20

Spike Detection

Compare current error rate against the previous period to detect anomalies:

filter @message like /ERROR/
| stats count(*) as errors by bin(5m)
| sort @timestamp desc

After retrieving results, compare the most recent bins against the average of earlier bins. A bin with >2x the average signals a spike.

Log Volume Analysis

Detect unusual log volume changes (potential indicator of issues):

stats count(*) as logCount by bin(5m)
| sort @timestamp desc

Message Pattern Extraction

Extract structured data from unstructured log messages:

parse @message /\[(?<level>\w+)\]\s+(?<component>[\w.]+)\s+-\s+(?<msg>.+)/
| stats count(*) as cnt by level, component
| sort cnt desc

Lambda Cold Start Analysis

filter @type = "REPORT"
| parse @message "Init Duration: * ms" as initDuration
| filter ispresent(initDuration)
| stats count(*) as coldStarts, avg(initDuration) as avgInitMs, max(initDuration) as maxInitMs, pct(initDuration, 99) as p99InitMs by bin(1h)

API Gateway Latency Breakdown

filter @message like /requestId/
| parse @message "integrationLatency: *" as integrationLatency
| parse @message "latency: *" as totalLatency
| stats avg(totalLatency) as avgTotal, avg(integrationLatency) as avgBackend, pct(totalLatency, 99) as p99Total by bin(15m)

Error Rate as Percentage

stats count(*) as total, sum(@message like /ERROR/) as errors by bin(1h)
| sort @timestamp desc

Cross-Log-Group Analysis

Query multiple log groups simultaneously by passing an array to logGroupNames in StartQuery:

execute_skill_api(
  skill_name="cloudwatch_logs",
  endpoint_path="StartQuery",
  method="POST",
  body="{\"logGroupNames\": [\"/aws/lambda/auth-service\", \"/aws/lambda/api-gateway\", \"/aws/lambda/order-service\"], \"startTime\": 1700000000, \"endTime\": 1700086400, \"queryString\": \"filter @message like /ERROR/ | stats count(*) as errors by @logStream | sort errors desc | limit 20\"}"
)

Operational Troubleshooting Workflow

When a user reports an incident or asks to investigate an issue, follow this structured approach:

Step 1: Gather Context