Skill v1.0.2
currentAutomated scan100/100+1 new, ~1 modified
name: detecting-network-anomalies-with-zeek description: Deploy and configure Zeek (formerly Bro) to passively analyze network traffic, generate structured connection/DNS/HTTP/SSL/file logs, detect anomalous behavior, and write custom scripts for organization-specific threats. Use for passive monitoring at network choke points, feeding SIEM/threat hunting with protocol metadata, or retrospective log analysis during incident response; not a substitute for inline IDS/IPS or host agents. domain: cybersecurity subdomain: network-security tags:
- network-security
- zeek
- network-monitoring
- anomaly-detection
- threat-hunting
version: '1.0' author: mahipal license: Apache-2.0 nist_csf:
- PR.IR-01
- DE.CM-01
- ID.AM-03
- PR.DS-02
mitre_attack:
- T1046
- T1040
- T1557
- T1071
Detecting Network Anomalies with Zeek
When to Use
- Deploying passive network security monitoring at key network choke points for continuous visibility
- Generating structured connection, DNS, HTTP, SSL, and file transfer logs for SIEM ingestion and threat hunting
- Writing custom Zeek scripts to detect organization-specific threats, policy violations, or beaconing behavior
- Performing retrospective analysis on network metadata to investigate security incidents
- Complementing IDS solutions with protocol-level metadata analysis that signature-based tools may miss
Do not use as a replacement for inline IDS/IPS that can actively block traffic, for monitoring encrypted payloads without TLS inspection, or on endpoints where host-based agents are more appropriate.
Prerequisites
- Zeek 6.0+ installed from source or package manager (
zeek --version) - Network interface configured on a span port, network tap, or virtual switch mirror for passive capture
- Sufficient disk storage for log files (estimate 1-5 GB/day per 100 Mbps of monitored traffic)
- Familiarity with Zeek's scripting language for writing custom detections
- Log aggregation system (Splunk, Elastic, Graylog) for centralized analysis
Workflow
Step 1: Install and Configure Zeek
# Install Zeek on Ubuntu/Debiansudo apt install -y zeek# Or install from source for latest versiongit clone --recursive https://github.com/zeek/zeekcd zeek && ./configure --prefix=/opt/zeek && make -j$(nproc) && sudo make installexport PATH=/opt/zeek/bin:$PATH# Configure the monitoring interfacesudo vi /opt/zeek/etc/node.cfg
# /opt/zeek/etc/node.cfg[zeek]type=standalonehost=localhostinterface=eth1
# Configure local network definitionssudo vi /opt/zeek/etc/networks.cfg
# /opt/zeek/etc/networks.cfg10.0.0.0/8 Internal172.16.0.0/12 Internal192.168.0.0/16 Internal
# Disable NIC offloading for accurate packet capturesudo ethtool -K eth1 rx off tx off gro off lro off tso off gso off# Deploy Zeeksudo zeekctl deploy# Verify Zeek is runningsudo zeekctl status
Step 2: Understand and Navigate Zeek Logs
# Zeek generates structured log files in /opt/zeek/logs/current/ls /opt/zeek/logs/current/# Key log files:# conn.log - All network connections (TCP, UDP, ICMP)# dns.log - DNS queries and responses# http.log - HTTP requests and responses# ssl.log - SSL/TLS handshake details# files.log - File transfers observed on the network# notice.log - Alerts from Zeek detection scripts# weird.log - Protocol anomalies and errors# x509.log - X.509 certificate details# smtp.log - SMTP email transactions# ssh.log - SSH connection details# View connection log with zeek-cut for column selectioncat /opt/zeek/logs/current/conn.log | zeek-cut ts id.orig_h id.orig_p id.resp_h id.resp_p proto service duration orig_bytes resp_bytes# View DNS logcat /opt/zeek/logs/current/dns.log | zeek-cut ts id.orig_h query qtype_name answers# View HTTP logcat /opt/zeek/logs/current/http.log | zeek-cut ts id.orig_h host uri method status_code user_agent
Step 3: Write Custom Detection Scripts
# Create a custom detection script directorysudo mkdir -p /opt/zeek/share/zeek/site/custom-detections
Create a script for detecting DNS tunneling:
# /opt/zeek/share/zeek/site/custom-detections/dns-tunneling.zeek@load base/frameworks/noticemodule DNSTunneling;export {redef enum Notice::Type += {DNS_Tunneling_Detected,DNS_Long_Query};# Threshold: number of unique queries per source in time windowconst query_threshold: count = 200 &redef;const time_window: interval = 5min &redef;const max_query_length: count = 50 &redef;}# Track query counts per source IPglobal dns_query_counts: table[addr] of count &create_expire=5min &default=0;event dns_request(c: connection, msg: dns_msg, query: string, qtype: count, qclass: count){local src = c$id$orig_h;# Check for unusually long domain queries (base64-encoded data)if ( |query| > max_query_length ){NOTICE([$note=DNS_Long_Query,$msg=fmt("Unusually long DNS query from %s: %s (%d chars)", src, query, |query|),$src=src,$identifier=cat(src, query)]);}# Track query volume per sourcedns_query_counts[src] += 1;if ( dns_query_counts[src] == query_threshold ){NOTICE([$note=DNS_Tunneling_Detected,$msg=fmt("Possible DNS tunneling: %s sent %d queries in %s", src, query_threshold, time_window),$src=src,$identifier=cat(src)]);}}
Create a script for detecting beaconing:
# /opt/zeek/share/zeek/site/custom-detections/beacon-detection.zeek@load base/frameworks/notice@load base/frameworks/sumstatsmodule BeaconDetection;export {redef enum Notice::Type += {Possible_Beaconing};const beacon_threshold: count = 50 &redef;const observation_window: interval = 1hr &redef;}event zeek_init(){local r1 = SumStats::Reducer($stream="beacon.connections",$apply=set(SumStats::SUM));SumStats::create([$name="detect-beaconing",$epoch=observation_window,$reducers=set(r1),$threshold_val(key: SumStats::Key, result: SumStats::Result) = {return result["beacon.connections"]$sum;},$threshold=beacon_threshold + 0.0,$threshold_crossed(key: SumStats::Key, result: SumStats::Result) = {NOTICE([$note=Possible_Beaconing,$msg=fmt("Possible beaconing: %s made %d connections in %s",key$str, result["beacon.connections"]$sum, observation_window),$identifier=key$str]);}]);}event connection_state_remove(c: connection){if ( c$id$resp_h !in Site::local_nets ){local key = fmt("%s->%s:%d", c$id$orig_h, c$id$resp_h, c$id$resp_p);SumStats::observe("beacon.connections", [$str=key], [$num=1]);}}
Step 4: Load Custom Scripts and Deploy
# Add custom scripts to local.zeeksudo tee -a /opt/zeek/share/zeek/site/local.zeek << 'EOF'# Custom detection scripts@load custom-detections/dns-tunneling.zeek@load custom-detections/beacon-detection.zeek# Enable additional protocol analyzers@load protocols/ftp/software@load protocols/http/software@load protocols/smtp/software@load protocols/ssh/detect-bruteforcing@load protocols/ssl/validate-certs@load protocols/ssl/log-hostcerts-only@load protocols/dns/detect-external-names# Enable file extraction@load frameworks/files/extract-all-files# Enable Intel framework for threat intelligence@load frameworks/intel/seen@load frameworks/intel/do_noticeEOF# Reload Zeek configurationsudo zeekctl deploy# Verify scripts loaded without errorssudo zeekctl diag
Step 5: Threat Hunting Queries on Zeek Logs
# Find long-duration connections (possible C2)cat /opt/zeek/logs/current/conn.log | zeek-cut ts id.orig_h id.resp_h id.resp_p duration | \awk '$5 > 3600 {print $0}' | sort -t$'\t' -k5 -rn | head -20# Find connections with high data transfer volumescat /opt/zeek/logs/current/conn.log | zeek-cut ts id.orig_h id.resp_h orig_bytes resp_bytes | \awk '$4 > 100000000 || $5 > 100000000 {print $0}'# Identify rare user agents (potential malware)cat /opt/zeek/logs/current/http.log | zeek-cut user_agent | sort | uniq -c | sort -n | head -20# Find self-signed or expired certificatescat /opt/zeek/logs/current/ssl.log | zeek-cut ts id.orig_h id.resp_h server_name validation_status | \grep -v "ok"# Detect DNS queries to newly registered domains (DGA patterns)cat /opt/zeek/logs/current/dns.log | zeek-cut ts id.orig_h query | \awk -F'\t' '{n=split($3,a,"."); if(length(a[n-1]) > 10) print $0}'# Find SSH brute force attemptscat /opt/zeek/logs/current/ssh.log | zeek-cut ts id.orig_h id.resp_h auth_success | \grep "F" | awk '{print $2}' | sort | uniq -c | sort -rn | head -10# Identify unusual port usagecat /opt/zeek/logs/current/conn.log | zeek-cut id.resp_p proto service | \sort | uniq -c | sort -rn | head -50
Step 6: Integrate with SIEM and Set Up Alerting
# Configure JSON log output for SIEM ingestionsudo tee /opt/zeek/share/zeek/site/json-logs.zeek << 'EOF'@load policy/tuning/json-logs.zeekredef LogAscii::use_json = T;EOF# Forward logs to Elastic via Filebeat# /etc/filebeat/filebeat.ymlsudo tee /etc/filebeat/filebeat.yml << 'EOF'filebeat.inputs:- type: logenabled: truepaths:- /opt/zeek/logs/current/*.logjson.keys_under_root: truejson.add_error_key: truefields:source: zeekfields_under_root: trueoutput.elasticsearch:hosts: ["https://elastic-siem:9200"]index: "zeek-%{+yyyy.MM.dd}"username: "elastic"password: "${ES_PASSWORD}"EOFsudo systemctl enable --now filebeat# Set up log rotationsudo tee /etc/cron.d/zeek-logrotate << 'EOF'0 0 * * * root /opt/zeek/bin/zeekctl cronEOF# Monitor Zeek healthsudo zeekctl statussudo zeekctl netstats
Key Concepts
| Term | Definition | |
|---|---|---|
| Network Security Monitor | Passive analysis tool that observes network traffic and generates structured metadata logs without altering or blocking traffic flow | |
| Zeek Script | Event-driven scripts written in Zeek's domain-specific language that process network events and generate notices, logs, and metrics | |
| Connection Log (conn.log) | Core Zeek log recording every observed connection with source/destination IPs, ports, protocol, duration, and byte counts | |
| Notice Framework | Zeek subsystem for generating alerts when detection scripts identify suspicious activity, outputting to notice.log | |
| SumStats Framework | Statistical analysis framework in Zeek for tracking metrics over time windows, enabling threshold-based detection of anomalies | |
| Intel Framework | Zeek module for matching observed network indicators against threat intelligence feeds and generating alerts on matches |
Tools & Systems
- Zeek 6.0+: Open-source network security monitor generating comprehensive protocol-level logs from passive traffic analysis
- zeek-cut: Zeek utility for extracting specific columns from tab-separated Zeek log files for quick analysis
- zeekctl: Zeek management tool for deploying, monitoring, and managing Zeek instances across single or clustered deployments
- RITA (Real Intelligence Threat Analytics): Open-source tool that analyzes Zeek logs for beaconing, DNS tunneling, and other threat indicators
- Filebeat: Elastic agent for shipping Zeek JSON logs to Elasticsearch for centralized analysis and visualization
Common Scenarios
Scenario: Detecting Command-and-Control Beaconing in Enterprise Traffic
Context: A threat intelligence report indicates that a specific threat actor uses HTTPS beaconing with 60-second intervals to compromised hosts. The SOC team needs to analyze Zeek logs to identify any hosts exhibiting this pattern across the enterprise network carrying 2 Gbps of traffic.
Approach:
- Deploy Zeek on a network tap at the internet egress point with AF_PACKET for high-throughput capture
- Enable the custom beacon detection script with thresholds tuned for 60-second intervals over 1-hour observation windows
- Query conn.log for connections to external IPs with consistent duration and inter-connection timing: filter connections where the standard deviation of inter-arrival times is less than 5 seconds
- Cross-reference suspicious destination IPs against threat intelligence feeds loaded into Zeek's Intel framework
- Examine ssl.log for the associated TLS certificates -- check for self-signed certificates, unusual issuer names, or certificates with short validity periods
- Generate a notice for each identified beaconing source and feed into the SIEM for SOC triage
Pitfalls:
- Not tuning beacon detection thresholds for the environment, resulting in false positives from legitimate update services (Windows Update, AV updates)
- Failing to exclude CDN and cloud service provider IP ranges that naturally receive many repeat connections
- Running Zeek without sufficient CPU cores, causing packet drops on high-throughput links
- Not enabling JSON log output, making SIEM integration unnecessarily complex with custom parsers
Output Format
## Zeek Network Anomaly Detection Report**Sensor**: zeek-sensor-01 (10.10.1.250)**Monitoring Interface**: eth1 (span port from Core-SW1)**Analysis Period**: 2024-03-15 00:00 to 2024-03-16 00:00 UTC**Total Connections Logged**: 2,847,392### Anomalies Detected| Notice Type | Source | Destination | Details ||-------------|--------|-------------|---------|| DNS_Tunneling_Detected | 10.10.3.45 | 8.8.8.8 | 847 queries to suspect-domain.xyz in 5 min || Possible_Beaconing | 10.10.5.12 | 203.0.113.50:443 | 62 connections with 59.8s avg interval || SSL::Invalid_Server_Cert | 10.10.8.22 | 198.51.100.33:443 | Self-signed cert, CN=localhost || SSH::Password_Guessing | 45.33.32.156 | 10.10.20.11:22 | 487 failed attempts in 30 min |### Recommendations1. Isolate 10.10.3.45 and investigate for DNS tunneling malware2. Block 203.0.113.50 at firewall and forensically image 10.10.5.123. Investigate self-signed TLS certificate on 198.51.100.334. Block 45.33.32.156 and enforce SSH key-only authentication