malloydata/publisher/malloy-model

Building Malloy Models

Tool names are written bare here - get_context, execute_query, search_malloy_docs. The exact prefixed name depends on the host surface; match each against the tools you actually have.

Getting Started (New Projects)

If no .malloy files exist yet, do discovery and propose a structure first, then return here to build base source and joined source files. Keep proposals and the analysis behind them in the conversation.

File structure convention (a flat layout at the package root is the simplest default):

<package-name>/
  publisher.json              # Required for publishing (name, version, description)
  customers.malloy            # Base source: one per table
  products.malloy
  orders.malloy
  user_order_facts.malloy     # Computed source
  order_analysis.malloy       # Source: one per analytical domain
  customer_health.malloy

Versions for new packages should start at "0.0.1".

Before creating any files, check for an existing publisher.json in the target directory. If one exists for a different package, create a new subdirectory for your package, don't overwrite another package's config.

In Publisher an environment is a project, and Publisher is single-tenant, so there is no org/tenant layer to model around: one environment holds one set of packages.

Prior Art Dispatch

If your discovery turned up existing modeling patterns to mirror (a derived table, UNNEST joins, a review or curation pass), read the relevant reference before building.

Base Source Templates

Base Source (Simple Mode)

source: customers is my_conn.table('sales.customers')
extend {
  primary_key: customer_id
 
  dimension:
    // A dimension is the lighter way to give a column a cleaner name
    order_type is `Type`
    full_name is concat(first_name, ' ', last_name)
    segment is lifetime_value ?
      pick 'enterprise' when >= 100000
      pick 'mid-market' when >= 10000
      else 'SMB'
 
  measure:
    customer_count is count()
}

Every `dimension:` needs `name is expr`. A bare column name like dimension: species is a parse error (unexpected '}', or missing 'is' before 'measure:' when another declaration follows). Raw columns are already usable in group_by / select without any declaration, so only add a dimension: when deriving or renaming a field (e.g. revenue is price * quantity).

Base Source (Curated Mode with Access Modifiers)

##! experimental.access_modifiers
 
source: orders is my_conn.table('sales.orders')
include {
  public:
    #(doc) Order identifier
    order_id
 
    #(doc) Customer who placed the order
    user_id
 
    #(doc) Total sale price in USD
    sale_price
 
    #(doc) Order creation timestamp
    created_at
 
  internal:
    raw_payload_json  // Verified empty via index query + user confirmation
}
extend {
  primary_key: order_id
 
  dimension:
    #(doc) Date the order was placed
    order_date is created_at::date
 
  measure:
    #(doc) Total number of orders
    order_count is count()
 
    #(doc) Total revenue in USD
    # currency
    revenue is sum(sale_price)
}

Computed Source (from Query)

Wrap the query in parentheses and extend it. from(...) was removed from the language and no longer parses (unexpected 'from').

import "orders.malloy"
 
source: user_order_facts is (
  orders -> {
    group_by: customer_id
    aggregate:
      total_orders is count()
      total_revenue is sum(sale_price)
      first_order_date is min(created_at)
      last_order_date is max(created_at)
  }
) extend {
  primary_key: customer_id
 
  dimension:
    days_since_last_order is days(last_order_date to now)
    is_repeat_buyer is total_orders > 1
 
  measure:
    buyer_count is count()
    avg_customer_ltv is avg(total_revenue)
}

For advanced query-based source patterns (window functions, pipelines), see reference/query-sources.md.

Joined Source File Template

import "customers.malloy"
import "orders.malloy"
import "user_order_facts.malloy"
 
#(doc) Customer health analysis. Use for retention, segmentation, and churn risk.
source: customer_health is customers extend {
  join_one: user_order_facts with customer_id
  join_many: orders on customer_id = orders.customer_id
 
  dimension:
    is_at_risk is user_order_facts.days_since_last_order > 90
      and user_order_facts.total_orders > 1
 
  measure:
    revenue_per_customer is orders.sale_price.sum() / nullif(customer_count, 0)
    at_risk_count is count() { where: is_at_risk = true }
}

Base vs Joined Sources

The "no views" rule is schema-first only. A schema-first model is built before anyone has asked a question, so any view in it is a guess. It does not apply to the analysis-first workflow (skill:malloy-model-as-you-go), where every view is a question that was asked and verified - there, saving a view or a dashboard is the right call, in the model file next to the measures it uses. Analysis-first still models everything else properly: documented dimensions, measures, and joins.

Two more rules here are schema-first only. Analysis-first should skip access modifiers and curation - there is no discovery surface to curate when every field was paid for by a question - and skip one-file-per-table, keeping a single domain file until it genuinely gets unwieldy.

Key Rules

• Every `dimension:` needs `name is expr`: a bare dimension: species is a parse error. Raw columns are queryable directly in group_by / select; only declare a dimension: to derive or rename a field.
• Define joined tables before referencing them, use import statements in multi-file architecture
• Use `nullif(denominator, 0)` for all division
• Alias joined fields before using in `order_by`: group_by: yr is table.year
• Verify join paths exist before referencing a.b.field (each hop needs explicit join)
• Pick syntax: value BEFORE condition, pick 'Small' when size < 10
• `where:` vs `having:`: Use where: for row filters, having: for aggregate filters
• `rename:` composes with `include {}`, but only in one order: the extend { rename: } must come before the include {}, which then names the field by its new name. Reversed, it fails with Can't find field 'X' to set access modifier. For a cleaner column name without a rename, internal: + dimension: is still the lighter move (mark ` Type as internal, add dimension: order_type is Type ). See skill:malloy-gotchas-modeling` § Field Management
• Mark raw columns `internal` when a derived dimension replaces them
• Check for duplicate rows before building measures
• When both a combined table (all types) and filtered/split tables exist, prefer the split tables
• DRY: define measures/dimensions in base source files, not inline in views
• Lay out a new file the same way throughout: two-space indentation, no tabs, one blank line between top-level declarations (consecutive import lines stay together), no trailing whitespace, and a long line broken after a comma or before an operator, at whatever width the project keeps to. In a project whose files already use another layout (tabs, four spaces), a new file matches them.
• An edit keeps the file's own layout: change only the lines the request needs, and never reindent or rewrap a line you weren't asked to change, so the diff shows the change and nothing else.
• Never write a threshold, tier boundary, or bucket cutoff you chose yourself. Every boundary in a pick expression or filtered measure is user-supplied, distribution-derived (query min/p25/p50/p75/p95 first and show the evidence; Malloy has no percentile function, so use the two-stage query in skill:malloy-discover § Example Queries; see skill:malloy-define § Data-driven proposals), or explicitly flagged as an assumption in its #(doc). A hardcoded cutoff nobody confirmed is a business decision shipped as fact.

Parameterizing sources with given: (preferred)

Native Malloy `given:` parameters are runtime values a model reads in its own logic; callers supply them at query time. #(filter) is deprecated: never add one. Enable givens with ##! experimental.givens at the top of the model.

A given belongs in the model only when the data's meaning or access depends on it:

• a row-level gate: #(authorize), #(access_filter), a #(secure) given, $GROUPS;
• a value every query must supply (the no-default row in the table below);
• a parameter a measure, dimension or view is defined with;
• scoping the author means to apply to every consumer of the source: chat, MCP and every report.

A filter a reader picks on a dashboard or notebook is not modeling. Manufacturer, a date range, region, a "major recalls only" toggle: don't add it to a source. Leave the source unfiltered, and let the report declare the given and bind it on its tiles (skill:malloy-reports). A model-level where: applies to every read of the source, chat and every other report included, and a report cannot take it over, because binding is per declaration, not per name. The report builder shows such a filter but cannot remove or reconfigure it.

##! experimental.givens
 
# A value every query must supply: the table is partitioned by day.
given: EVENT_DATE :: date
 
source: events is duckdb.table('data/events.parquet') extend {
  where: event_date = $EVENT_DATE
  measure: event_count is count()
}

A given is declared bare but referenced with a `$` sigil in expressions ($EVENT_DATE), as above.

• An optional given in the model gets a neutral, match-all default (a filter<> given defaulting to f''), so an unsupplied value returns unfiltered rows. Because it bakes an always-on where: into the source, a non-neutral default (a date floor, say) applies to _every_ read of the source. Defaults must be Malloy literals.
• Givens don't auto-inject a `where:`. Unlike #(filter), you write the filter expression that references the given yourself (e.g. where: dimension ~ $given_name).
• Every `#(filter)` use has a `given:` form. Never add a #(filter) annotation to a model, not even for required, implicit, or a date/number range: