Skill v1.0.0
currentAutomated scan100/100name: json-yaml category: data description: > Parse, query, transform, validate, and convert JSON and YAML data using jq and fy (fast-yaml). Use when the user asks to filter JSON arrays, extract nested fields, reshape data structures, convert between JSON and YAML formats, validate or lint YAML files, pretty-print or minify JSON, format YAML, or perform complex data transformations on structured data. license: MIT compatibility: Requires jq. Requires fy (fast-yaml) for YAML processing (cargo install fast-yaml-cli) metadata: author: zeph version: "1.0"
JSON & YAML Processing with jq and fy
Quick Reference
| Action | Command | |
|---|---|---|
| Pretty-print JSON | jq . file.json | |
| Minify JSON | jq -c . file.json | |
| Extract field | jq '.name' file.json | |
| Extract raw string | jq -r '.name' file.json | |
| Validate YAML | fy parse file.yaml | |
| Format YAML | fy format -i file.yaml | |
| Lint YAML | fy lint file.yaml | |
| YAML to JSON | fy convert json file.yaml | |
| JSON to YAML | fy convert yaml file.json | |
| Validate JSON | jq empty file.json |
jq Basics
Identity and Field Access
# Identity (pass-through, pretty-print)cat data.json | jq .# Access nested fieldjq '.user.address.city' data.json# Access array elementjq '.[0]' data.jsonjq '.items[2]' data.json# Access multiple fields (comma operator)jq '.name, .age' data.json# Optional field access (no error if missing)jq '.missing?' data.json
String Interpolation
# Build strings from fieldsjq -r '"Name: \(.name), Age: \(.age)"' data.json# Use in filtersjq -r '.[] | "User \(.id): \(.email)"' users.json
Raw Output
# -r removes quotes from string outputjq -r '.name' data.json# -R treats input as raw strings (not JSON)echo "hello" | jq -R .# --raw-output0 uses NUL separator (for xargs -0)jq -r0 '.[]' list.json | xargs -0 -I{} echo "{}"
jq Filters
select — Filter by Condition
# Select objects matching conditionjq '.[] | select(.age > 30)' users.json# Select by string matchjq '.[] | select(.name == "Alice")' users.json# Select with containsjq '.[] | select(.tags | contains(["admin"]))' users.json# Select with test (regex)jq '.[] | select(.email | test("@gmail\\.com$"))' users.json# Select non-null valuesjq '.[] | select(.address != null)' users.json# Multiple conditions (and / or)jq '.[] | select(.age > 25 and .status == "active")' users.jsonjq '.[] | select(.role == "admin" or .role == "superadmin")' users.json
map — Transform Each Element
# Extract field from each elementjq '[.[] | .name]' users.jsonjq 'map(.name)' users.json # equivalent shorthand# Transform each elementjq 'map({name: .name, upper: (.name | ascii_upcase)})' users.json# Map with select (filter then transform)jq 'map(select(.active) | .email)' users.json# map_values — transform values of an objectjq 'map_values(. + 1)' '{"a": 1, "b": 2}'
reduce — Accumulate a Result
# Sum all valuesjq 'reduce .[] as $x (0; . + $x)' numbers.json# Build an object from arrayjq 'reduce .[] as $item ({}; . + {($item.id | tostring): $item.name})' items.json# Concatenate stringsjq 'reduce .[] as $s (""; . + $s + " ")' strings.json# Count by categoryjq 'reduce .[] as $item ({}; .[$item.category] += 1)' items.json
group_by — Group Elements
# Group by field valuejq 'group_by(.department)' employees.json# Group and countjq 'group_by(.status) | map({status: .[0].status, count: length})' items.json# Group and collect namesjq 'group_by(.team) | map({team: .[0].team, members: map(.name)})' people.json
sort_by — Sort Elements
# Sort by fieldjq 'sort_by(.name)' users.json# Sort descending (reverse)jq 'sort_by(.age) | reverse' users.json# Sort by multiple fieldsjq 'sort_by(.department, .name)' employees.json# Sort by numeric valuejq 'sort_by(-.score)' scores.json# unique_by — deduplicatejq 'unique_by(.email)' users.json
jq Object Construction
# Build new objectsjq '.[] | {name, email}' users.json# Rename fieldsjq '.[] | {user_name: .name, user_email: .email}' users.json# Add computed fieldsjq '.[] | . + {full_name: "\(.first) \(.last)"}' users.json# Merge objectsjq '. as $a | input as $b | $a * $b' file1.json file2.json# Delete fieldsjq 'del(.password, .secret)' user.json# Convert object to key-value pairsjq 'to_entries' object.jsonjq 'to_entries | map({key, value})' object.json# Convert key-value pairs back to objectjq 'from_entries' entries.json
jq Array Operations
# Lengthjq 'length' array.jsonjq '.items | length' data.json# Flatten nested arraysjq 'flatten' nested.jsonjq 'flatten(1)' nested.json # one level only# Add (sum numbers, concat strings/arrays)jq 'add' numbers.json# Min / maxjq 'min' numbers.jsonjq 'max_by(.score)' items.jsonjq 'min_by(.price)' products.json# First / last / nthjq 'first' array.jsonjq 'last' array.jsonjq 'nth(3)' array.json# Limit / untiljq 'limit(5; .[])' array.json# indices / indexjq 'index("needle")' array.json# Cartesian product (combinations)jq '[.a, .b] | combinations' data.json
jq Conditionals
# If-then-elsejq '.[] | if .age >= 18 then "adult" else "minor" end' people.json# Alternative operator (default value)jq '.name // "unknown"' data.json# Try-catchjq 'try .foo.bar catch "not found"' data.json
jq Type Functions
# Type checkingjq '.[] | type' data.jsonjq '.[] | select(type == "string")' data.json# Type conversionjq '.count | tonumber' data.jsonjq '.id | tostring' data.json# Numeric operationsjq '.price | floor' item.jsonjq '.price | ceil' item.jsonjq '.price | round' item.jsonjq '.values | add / length' data.json # average
jq String Functions
# Length, case conversionjq '.name | length' data.jsonjq '.name | ascii_upcase' data.jsonjq '.name | ascii_downcase' data.json# Split and joinjq '.path | split("/")' data.jsonjq '.tags | join(", ")' data.json# Test (regex match, returns bool)jq '.email | test("^[a-z]+@")' data.json# Match (regex capture groups)jq '.line | match("(\\w+)=(\\w+)")' data.json# Capture (named groups)jq '.line | capture("(?<key>\\w+)=(?<val>\\w+)")' data.json# Replace (gsub / sub)jq '.text | gsub("foo"; "bar")' data.jsonjq '.text | sub("^prefix_"; "")' data.json# Starts/ends withjq '.name | startswith("Dr.")' data.jsonjq '.file | endswith(".json")' data.json# ltrimstr / rtrimstrjq '.path | ltrimstr("/api/v1")' data.json
jq with Pipes and Shell
# Pipe curl output to jqcurl -s "https://api.example.com/users" | jq '.data[].name'# Use jq output in shell variablesname=$(jq -r '.name' config.json)# Process line by linejq -r '.[] | .id' items.json | while read -r id; do echo "Processing $id"; done# Read from stdinecho '{"a":1}' | jq '.a'# Multiple files (slurp mode)jq -s '.[0] * .[1]' defaults.json overrides.json# Create JSON from shell variablesjq -n --arg name "$NAME" --arg email "$EMAIL" '{name: $name, email: $email}'# Pass argumentsjq --arg threshold "$THRESH" '.[] | select(.score > ($threshold | tonumber))' data.json
fy (fast-yaml) — YAML Processing
All YAML operations use fy (fast-yaml CLI). Install: cargo install fast-yaml-cli.
Validate YAML
# Validate syntax (reports errors with rustc-style diagnostics)fy parse config.yaml# Validate multiple filesfy parse config.yaml values.yaml# Validate a directory (recursive, respects .gitignore)fy parse src/
Format YAML
# Format and print to stdoutfy format config.yaml# Format in-placefy format -i config.yaml# Format entire directory in-placefy format -i src/# Format with glob patternfy format -i "**/*.yaml"# Parallel formatting (8 workers)fy format -i -j 8 project/
Lint YAML
# Lint a file (validation + style rules)fy lint config.yaml# Lint with exclusionsfy lint --exclude "tests/**" .# Lint a directoryfy lint src/
Convert Between Formats
# YAML to JSONfy convert json config.yaml# JSON to YAMLfy convert yaml data.json# Convert and redirectfy convert json config.yaml > config.jsonfy convert yaml data.json > data.yaml
Batch Operations
fy activates batch mode automatically for directories, glob patterns, or multiple files:
- Respects
.gitignoreexclusions - Supports include/exclude patterns
- Parallel worker configuration via
-jflag - 6-15x speedup on multi-file operations
# Format all YAML in a project with 4 workersfy format -i -j 4 .# Lint all YAML except test fixturesfy lint --exclude "tests/fixtures/**" .# Validate all YAML in multiple directoriesfy parse src/ config/ deploy/
Workflow: Edit and Validate
After editing any YAML file, always validate with fy:
# 1. Edit the file (manually or with sed/awk)# 2. Validatefy parse config.yaml# 3. Format (normalize style)fy format -i config.yaml
Important Notes
jquses single quotes for the filter expression to avoid shell expansion- Use
-rfor raw output (no quotes around strings) when piping to other commands - Use
-eto set exit code based on output (null/false = exit 1) - Use
-cfor compact (single-line) output, useful for line-by-line processing - Use
--slurpfileto load a JSON file into a variable:jq --slurpfile cfg config.json '...' - When constructing JSON with
jq -n, use--argfor strings and--argjsonfor numbers/booleans - Always use
fyfor YAML validation, formatting, and conversion — never edit YAML manually without validating withfyafterward fysupports YAML 1.2.2 specification