rubylane/rl_json/rl-json

Package Overview

The rl_json package provides high-performance JSON manipulation for Tcl. It defines a custom Tcl_Obj type for efficient parsing and manipulation, with performance similar to the dict command. The package uses a custom-built parser (older versions used yajl, but that dependency has been removed).

Loading the Package

package require rl_json
# Optionally import the json command into the current namespace:
if {[llength [info commands json]] == 0} {namespace import ::rl_json::json}

Core Concept: JSON as Strings

The json commands operate on string values in JSON format, similar to how dict operates on strings in dictionary format. Values have an internal cached representation for performance, but semantically behave as if manipulating JSON strings directly.

Critical distinction:

• `json get` - Converts JSON to native Tcl types (objects→dicts, arrays→lists, null→empty string)
• `json extract` - Returns JSON as-is, preserving the JSON format for further manipulation

Command Reference

Reading JSON Data

json get ?-default defaultValue? jsonValue ?key ...? : Extract a value, returning native Tcl type. Cannot be queried further with json get.

``tcl set data {{"name": "Alice", "age": 30, "tags": ["dev", "admin"]}} json get $data name ;# Returns: Alice (Tcl string) json get $data tags ;# Returns: {"dev" "admin"} (Tcl list) json get $data missing ;# Error json get -default {} $data missing ;# Returns: {} ``

json extract ?-default defaultValue? jsonValue ?key ...? : Extract a value, returning JSON format. Can be queried with subsequent json commands.

```tcl set item [json extract $data tags] ;# Returns: ["dev","admin"] (JSON) json get $item 0 ;# Returns: dev

# Performance note: Direct deep indexing is preferred json get $data tags 0 ;# Preferred - equivalent but clearer ```

json exists jsonValue ?key ...? : Test if path exists and is not null. Returns boolean.

``tcl json exists $data name ;# true json exists $data missing ;# false json exists {{"x": null}} x ;# false (null is treated as non-existent) ``

json type jsonValue ?key ...? : Get type: "object", "array", "string", "number", "boolean", or "null".

``tcl json type $data ;# object json type $data tags ;# array json type $data age ;# number ``

json isnull jsonValue ?key ...? : Test if value is JSON null.

json length jsonValue ?key ...? : Get length (arrays: element count, objects: key count, strings: character count).

json keys jsonValue ?key ...? : Get object keys as Tcl list.

Modifying JSON Data

json set jsonVariableName ?key ...? value : Modify JSON in-place (like dict set). Auto-creates nested objects if path doesn't exist. Performs automatic type coercion on value.

```tcl set doc {{}} json set doc name Alice ;# "Alice" - not valid JSON, becomes string json set doc age 30 ;# 30 - valid JSON number json set doc active true ;# true - valid JSON boolean json set doc flag yes ;# "yes" - not valid JSON, becomes string json set doc tags {[]} ;# [] - valid JSON array json set doc count "42" ;# 42 - string rep is valid JSON number json set doc data null ;# null - valid JSON null

# Explicit type control when string rep could be ambiguous: json set doc name [json string Alice] ;# Force string type json set doc count [json number 42] ;# Force number type json set doc flag [json boolean yes] ;# Force boolean (yes→true)

# IMPORTANT: json set modifies the variable directly, don't reassign: json set doc key value ;# ✅ Correct set doc [json set doc key value] ;# ❌ Wrong - redundant ```

Type coercion rule for `json set`:

If the string representation of value (what puts would display) is valid JSON, it is inserted as-is. Otherwise, it is wrapped as a JSON string.

Examples (Tcl script literals → JSON output):

These show what happens when you write these values literally in Tcl code:

• true → true (valid JSON boolean)
• yes → "yes" (Tcl boolean, but not valid JSON - wrapped as string)
• 123 → 123 (valid JSON number)
• hello → "hello" (not valid JSON - wrapped as string)
• null → null (valid JSON null)
• {} → "" (empty string in Tcl - wrapped as empty JSON string)
• {{}} → {} (Tcl string containing "{}" - valid JSON empty object)
• {[]} → [] (Tcl string containing "[]" - valid JSON empty array)

Examples (String values → JSON output):

These show how rl_json interprets string values after Tcl quoting is resolved:

• String true → true (valid JSON boolean)
• String yes → "yes" (not valid JSON - wrapped as string)
• String 123 → 123 (valid JSON number)
• String "123" (with quote chars) → "123" (valid JSON string)
• String hello → "hello" (not valid JSON - wrapped as string)
• String null → null (valid JSON null)
• Empty string → "" (empty JSON string)
• String {} → {} (valid JSON empty object)
• String [] → [] (valid JSON empty array)

json unset jsonVariableName ?key ...? : Remove a value. For objects: removes key. For arrays: removes element and shifts later elements.

``tcl json unset doc tags json unset doc items 0 ;# Remove first array element ``

Creating JSON Values

json string value : Create JSON string.

json number value : Create JSON number. Accepts any Tcl numeric format.

json boolean value : Create JSON boolean. Accepts any Tcl boolean format (yes/no, true/false, 1/0, etc.).

json object ?key value ?key value ...?? : Create JSON object. value is a 2-element list: {type value} where type is one of: string, number, boolean, null, object, array, json.

```tcl json object name {string Alice} age {number 30} # Returns: {"name":"Alice","age":30}

# Alternate packed syntax: json object {name {string Alice} age {number 30}} ```

json array ?elem ...? : Create JSON array. elem is a 2-element list: {type value}.

``tcl json array {string foo} {number 42} {boolean true} # Returns: ["foo",42,true] ``

Note: For complex structures, `json template` is strongly preferred over json object/json array.

JSON Templates

json template jsonValue ?dictionary? : Interpolate values into a JSON template. Templates are valid JSON with special string markers.

Template markers (must be complete string values):

• ~S:name - Insert as string
• ~N:name - Insert as number
• ~B:name - Insert as boolean
• ~J:name - Insert as JSON fragment (no additional parsing)
• ~T:name - Insert as template (recursively process)
• ~L:remainder - Literal (use remaining chars as-is, for escaping)

Rules:

1. Template markers must be the entire string value in the template
2. The template itself must be valid JSON
3. Values come from dictionary (if provided) OR current scope variables (never both)

• With dictionary: ALL values sourced from dictionary keys only
• Without dictionary: ALL values sourced from variables in current scope only

1. Missing variables/keys interpolate as null

# ✅ Correct - using variables from current scope (no dictionary argument):
set name "Alice"
set age 30
json template {
    {
        "name": "~S:name",
        "age": "~N:age",
        "active": "~B:active"
    }
}
# Returns: {"name":"Alice","age":30,"active":null} (active is null - not defined)
 
# ✅ Correct - using dictionary (all values from dictionary):
json template {
    {
        "name": "~S:name",
        "age": "~N:age",
        "tags": ["~S:tag1", "~S:tag2"]
    }
} {name Alice age 30 tag1 dev tag2 admin}
# Returns: {"name":"Alice","age":30,"tags":["dev","admin"]}
 
# ❌ Wrong - can't mix dictionary values and variables:
set name "Alice"
json template {{"name": "~S:name"}} {age 30}
# name comes from dictionary (returns null), NOT from variable
 
# ❌ Wrong - can't interpolate inside string:
json template {{"url": "https://example.com/~S:path"}}  ;# Error: not valid JSON
 
# ✅ Correct - build the value first:
set url "https://example.com/$path"
json template {{"url": "~S:url"}}
 
# ❌ Wrong - template not valid JSON:
json template {{":val": ~J:data}}  ;# Missing quotes around template marker
 
# ✅ Correct - template is valid JSON:
json template {{":val": "~J:data"}}

Real-world example:

set users {[]}
db eval {SELECT name, email, active FROM users} {
    json set users end+1 [json template {
        {
            "name":   "~S:name",
            "email":  "~S:email",
            "active": "~B:active"
        }
    }]
}

Iteration

json foreach varList1 jsonValue1 ?varList2 jsonValue2 ...? script : Iterate like foreach. For objects, varList must be 2 elements: {key value}.

```tcl # Arrays: json foreach item {["a","b","c"]} { puts $item ;# Each $item is a JSON value }

# Objects: json foreach {key val} {{"x":1,"y":2}} { puts "$key = $val" ;# key is Tcl string, val is JSON value } ```

json lmap varList1 jsonValue1 ?varList2 jsonValue2 ...? script : Collecting iteration returning Tcl list. Supports continue and break.

json amap varList1 jsonValue1 ?varList2 jsonValue2 ...? script : Collecting iteration returning JSON array. Script results converted to JSON.

json omap varList1 jsonValue1 ?varList2 jsonValue2 ...? script : Collecting iteration returning JSON object. Script must return dict/list of 2n elements.

Utility Commands

json normalize jsonValue : Remove optional whitespace, return minimal JSON.

json pretty ?-indent indent? jsonValue ?key ...? : Format JSON for readability. Default indent is 4 spaces.

``tcl puts [json pretty $data] puts [json pretty -indent "\t" $data] ``

json decode bytes ?encoding? : Decode binary data to JSON string per RFC standards. Handles BOM detection and encodings: utf-8 (default), utf-16le, utf-16be, utf-32le, utf-32be.

``tcl proc readjson file { set h [open $file rb] try { json decode [read $h] } finally { close $h } } ``

json valid ?-extensions extensionlist? ?-details detailsvar? jsonValue : Validate JSON. About 3× faster than parsing with try/trap.

```tcl if {![json valid $input]} { error "Invalid JSON" }

# With details: if {![json valid -details info $input]} { puts stderr [dict get $info errmsg] puts stderr "At offset: [dict get $info char_ofs]" }

# Extensions (default: comments allowed): json valid -extensions {} $input ;# Strict RFC, no comments json valid -extensions {comments} $input ;# Allow // and / / comments ```

Path Specifications

Commands like json get, json set, json exists, json unset accept path arguments:

• Object keys: Use key name
• Array indices: Use integer or "end", "end-1", etc.
• Mixed: Navigate through nested structures

Path behavior:

• `json get`/`json extract`/`json exists`: Out-of-bounds array access returns null/default
• `json set`: Creates nested objects if keys don't exist; extends arrays with nulls if needed
• `json unset`: Does nothing if path doesn't exist

set data {{"items": [{"name":"first"},{"name":"second"},{"name":"third"}]}}
 
json get $data items end-1 name      ;# "second"
json get $data items 1 name          ;# "second"
json get $data items 99 name         ;# Returns null (out of bounds)