Language Fundamentals¶

Understanding delightql means internalizing several simple but fundamental choices that are both semantic and syntactic in nature (and often times both).

Logic variables: delightql borrows Prolog’s unification syntax and semantics, though with a subtle definition of a logic variable as being qualified
Null semantics: delightql opts to provide null-safe equality in contrast to SQL’s = operator
SQL semantics: delightql defines its primary semantics via a direct mapping to SQL
Evaluation: delightql evaluates left to right in a manner remeniscent of concatenative languages and eschewing parenthesization
Column Scoping: delightql scopes left to right with well-known scope barriers that delimit what sections of the code may access which logic variables
Entity Scoping: delightql has certain rules about what scope contexts may or may not shadow existing entities with the same name
Column access: delightql makes strong choices about how a programmer may access columns by always providing ordinal access, and forbidding named access to ambiguous or generated columns
Moding rules and currying with higher-order rules: delightql, like Prolog, has a mechanism to mark syntax as moded for input vs output

Unification and Logical Variables¶

Delightql inherits from Prolog a simple rule: identifiers unify when their names match exactly. To unify means to insist on equality, i.e. the two variables at different locations must refer to the same datum. Unlike Prolog, however, delightql’s identifiers are qualified by their table names.

Delightql also uses Prolog’s argumentative unification syntax, where the position of an argument in a functor unifies whatever is written in this place: a ground, or another LVar with the dimension accessed at that location.

How Names Are Introduced¶

The way you access a table determines the names of its columns in scope:

Access Pattern	Columns Introduced
`users(id, name, status, _)`	`id`, `name`, `status`
`users(*)`	`users.id`, `users.name`, `users.status`
`users(*) as u`	`u.id`, `u.name`, `u.status`

Columns introduced by access pattern

Argumentative access introduces unqualified names – bare identifiers. Wildcard access introduces qualified names – prefixed by table name or alias.

Unification Creates Joins¶

When the same name appears in multiple places, unification creates a join condition:

artist(artist_id, name), album(album_id, title, artist_id)

Both introduce artist_id. Unification produces:

SELECT artist.artist_id, artist.name, album.album_id, album.title
FROM artist, album
WHERE artist.artist_id = album.artist_id;

Wildcard Access and Qualification¶

artist(*), album(*)

This introduces artist.artist_id and album.artist_id – different names. No unification occurs; the result is a cross join.

To join with wildcard access, use explicit conditions:

artist(*), album(*), artist.artist_id = album.artist_id

Or use the USING operator .(cols):

artist(*), album(*.(artist_id))

Qualified References in Argumentative Access¶

Argumentative patterns can reference lvars from other tables:

artist(*) as a, album(album_id, title, a.artist_id)

The a.artist_id in positional access matches the a.artist_id from artist(*) as a, creating unification. This mixes styles: wildcard for one table, positional for another, with explicit cross-reference.

A more elaborate example:

playlist(*) as p,
track(*) as t,
playlist_track(p.playlist_id, t.track_id)

Here playlist_track unifies with playlist on p.playlist_id and with track on t.track_id – a three-way join through positional cross-references.

Literals and Constraints¶

Ground terms in positional access create WHERE conditions:

invoice(invoice_id, customer_id, invoice_date, _, _, _, "Canada", _, total)

The positional grounding filters rows where the seventh column equals "Canada". The column billing_country has been unified with the ground value "Canada".

SELECT invoice_id, customer_id, invoice_date, total FROM invoice WHERE billing_country IS NOT DISTINCT FROM 'Canada';

Self-Unification¶

The same name repeated in positional access forces equality:

invoice(invoice_id, customer_id, invoice_date, _, billing_city, billing_city, _, _, total)

Columns 5 and 6 both bind to billing_city. This filters to rows where those columns are equal:

SELECT invoice_id, customer_id, invoice_date, billing_city, total FROM invoice WHERE billing_city IS NOT DISTINCT FROM billing_state;

Anonymous Tables and Unification¶

Anonymous tables participate in unification through their header names:

invoice(invoice_id, customer_id, _, _, _, _, billing_country, _, total),
_(billing_country @ "Canada"; "France"; "Germany")

The anonymous table introduces billing_country. This matches billing_country from invoice, creating:

SELECT invoice_id, customer_id, billing_country, total
FROM invoice
WHERE billing_country IN ('Canada', 'France', 'Germany');

With wildcard access, qualification is required:

invoice(*) as i,
_(i.billing_country @ "Canada"; "France"; "Germany")

Without the i. prefix, no unification occurs – the anonymous table’s billing_country wouldn’t match i.billing_country.

Lvars as Data in Anonymous Tables¶

Anonymous tables can use lvars as data values, not just in headers.

Constraint: An lvar cannot appear both in a header and in the data rows of the same anonymous table.

Inverted IN Pattern¶

employee(*) as e,
_(2 @ e.employee_id; e.reports_to)

Find employees where 2 appears in any of these columns:

SELECT * FROM employee e
WHERE 2 IN (e.employee_id, e.reports_to);

Or equivalently:

SELECT * FROM employee e
WHERE e.employee_id = 2
   OR e.reports_to = 2;

The anonymous table’s header is a literal (2); the data rows are lvars from employee. This inverts the typical IN pattern.

EAV Transformation¶

employee(*) as e,
_(attribute, value @
  "name", e.last_name;
  "email", e.email;
  "title", e.title;
  "hired", e.hire_date)

This is the melt pattern.

Row-Wise Correspondence¶

customer(*) as c,
employee(*) as e,
_(c.country, e.employee_id @
  e.country, c.support_rep_id;
  e.country, 1;
  "USA", 2)

Each row in the anonymous table represents a valid combination. The result includes only rows where (c.country, e.employee_id) matches one of the specified pairs:

SELECT *
FROM customer c, employee e
WHERE (c.country = e.country AND e.employee_id = c.support_rep_id)
   OR (c.country = e.country AND e.employee_id = 1)
   OR (c.country = 'USA' AND e.employee_id = 2);

Summary of Unification Rules¶

Pattern	Names Introduced	Unifies With
`t(a, b, c)`	`a`, `b`, `c`	Any `a`, `b`, `c`
`t(*)`	`t.a`, `t.b`, `t.c`	Only `t.a`, `t.b`, `t.c`
`t(*) as x`	`x.a`, `x.b`, `x.c`	Only `x.a`, `x.b`, `x.c`
`t(x.a, b, _)`	`x.a`, `b`	`x.a` from alias `x`; any `b`
`_("lit" @ v1; v2)`	(none – header is literal)	Filters where `lit` matches `v1` or `v2`
`_(col @ "a"; "b")`	`col`	Any `col`

Summary of unification rules

Identifiers Mirror Call Sites¶

In many programming languages the identifier always matches a particular grammar wherein the identifier may stand for a scalar, a function, a data structure, etc.

Thus, for example, the identifier execute could be a variable that holds an entity from different abstraction levels. This is especially prevelant in functional programming languages.

Delightql makes the identifier structured. The main place in which this matters is in the parameter list of a higher-order rule.

Null Semantics¶

Equality expressions¶

When both sides of a delightql = expression come from two different relations, then the traditional SQL = is used during transpilation. This rule applies for regular joins and inner relations (EXISTS and scalar subqueries).

When the above condition is not met, then delightql transpiles = to SQL’s IS NOT DISTINCT FROM.

Unification equality¶

The rules of equality expressions are created to use null-safe equality () in all places where join semantics are at play. This applies to Prolog-style unification as well:

playlist(playlist_id, name), playlist_track(playlist_id, track_id)

SELECT playlist.playlist_id, playlist.name, playlist_track.track_id
FROM playlist, playlist_track
WHERE playlist.playlist_id = playlist_track.playlist_id;

employee(*) as e, employee(*) as m, e.reports_to = m.employee_id

SELECT *
FROM employee e, employee m
WHERE e.reports_to = m.employee_id;

Traditional SQL Equals¶

If you must transpile the regular SQL equals in non-join locations, you can use +sql_eq(l,r) which is included as part of std::prelude.

Operational semantics of Query Expressions¶

Delightql operates by transforming valid delightql queries into valid SQL queries and submitting it to the relevant SQL engine.

Delightql makes a non-modification guarantee to the results that are returned – in other words, it passes back the raw bytes produced by the target engine.

By abstaining from any transformation to the data returned to the user, delightql ties the meaning of its code (query expressions ) to the meaning of the SQL it produces – that is to say, meaning (i.e. semantics) is transitively mapped from a delightql string to a SQL string, and then to a resulting table.

Any questions about delightql’s semantics are ultimately answered by the combination of its transpilation choices and what the targeted SQL engine returns.

Delightql may choose to arrange SQL in any of various forms if it can guarantee that the target SQL engine will return the exact same table. This choice affords flexibility in what SQL to produce that is orthogonal to performance or formatting.

Operational semantics of Directives¶

Directives (i.e. effects) are the impure entities within the delightql language. They cannot have a semantic that is defined by a single query. In fact, with the lack of certain branching and looping primitives in the SQL standard for DDL, some primitives cannot have a semantic consisting of a sequence of DDL and DML.

Delightql therefore absolves directives as a category from the obligation to return back from the target whatever was returned. The delightql runtime must interpret and act accordingly on different conditions.

Entity Shadowing and Uniqueness¶

There are three axes for understanding the identity of an entity:

name: the string identifier of the table, view, function or rule
arity: the number of columns
kind: whether the entity was defined as rule-first, function-first, sigma-condition-first or mounted from a source system

Delightql chooses a conservative identification scheme whereby name alone uniquely identifies an entity. This scheme prohibits the following in a single namespace:

two entities with the same name but different arities
two entities with the same name but different types (function vs rule vs sigma rule)

This does not prohibit the definition of shadowing entities, which may have different arities and/or kinds. Shadowing occurs when an entity is defined in a context/scope with higher precedence.

Some entities are always uniquely identified by a fully-qualified name – the concatentation of the namespace with a name. Referencing that entity with its fully-qualified name is guaranteed to access the entity in a way that will never be ambiguous. CTEs don’t exist in a namespace and are always referenced without qualification. A relational alias is not an entity at all: it names one occurrence (see Aliases name occurrences).

Ambiguity can occur only at a tier that contains peer candidates, principally the enlisted-namespace tier. Enlistment is an ergonomic choice that may make two distinct entities available under one bare name. It is legal to construct that state; the ambiguous name refuses only when mentioned.

It is up to the programmer to remove that ambiguity via qualification, aliasing, or renaming.

Contexts¶

Contexts are the scopes into which an entity may be defined.

Entities can exist (be defined) in the following contexts:

inside a data namespace (made visible by a mount!)
inside a lib namespace (made visible by a consult!)
inside a shadow namespace (created by a temp_table!, temp_view!)
inside the primary context ( home for REPL usage, and the primary namespace for file-based consulting)
as a CTE
within the target scope

A deep namespace path creates its missing prefixes as structural nodes. For example, consulting into a::b::c creates a and a::b if needed, but does not give them lib or data backing or entities. A later explicit consult! may claim an unbacked prefix as a library, or mount! may claim it as data. Either claim preserves existing children and refuses collisions or replacement of an already backed namespace. Backing belongs to each node independently: an empty library is still a library, and a child does not inherit its parent’s kind. Enlisting a parent does not make its children’s entities bare. This structural-prefix rule is ratified for alpha; implementation is pending.

Removing the last child does not silently remove an implicit parent. The empty structural node keeps its identity and can be claimed later. An explicit unconsult! of its exact path removes that node alone if it has no children or published dependents; it does not prune its parent.

Context Shadowing¶

We will use the notation T[c] to discuss the partial order that rules shadowing. It should be read aloud as “the entity T defined in context c” – for example users[cte] means “the entity with the name users defined as a CTE”.

The context shadowing rule states that the following order always applies:

T[cte]
  >  T[primary-context]
  > { all other contexts }

Where the > indicates that the scope to the left supersedes and redefines all entity references to the right.

The “all other contexts” refers to the combination of separately enlisted namespaces, none of which has priority over another – which we denote with the || parallel symbol.

The first nonempty tier wins. A same-named entity in a lower tier is not a fallback if the selected entity has the wrong kind, arity, or heading.

{ all other contexts } =
  { T[shadow-NS1] > T[NS1] }
  ||
  { T[shadow-NS2] > T[NS2] }
  ||
  { T[NS3] }

As an example:

mount!("etl.sqlite","etl")(*)
enlist!("etl")(*)

partner_sale_2025_07_31(*), quantity > 0 |> temp_table!(etl.partner_sale_2025_07_31(*))(*)

(~~ddl partner_sale_2025_07_31(*) :- _(sale_id @ 4; 5; 6) ~~)

invoice(*), total > 20 : partner_sale_2025_07_31
media_type(*) as partner_sale_2025_07_31 |> (partner_sale_2025_07_31.name)

In the example above we assume the existence of a partner_sale_2025_07_31 entity within the namespace etl.

after line 2 an enlisted partner_sale_2025_07_31 is available (from etl)
after line 4 the temporary table partner_sale_2025_07_31, created for etl, shadows the original within the etl candidate. The mounted database shares the primary connection’s temp schema, so its exact identity is sys::shadow::main.partner_sale_2025_07_31; no extra enlist edge is created, and a bare temp_table!(partner_sale_2025_07_31(*))(*) would instead have created a session object for main, the default write target.
after line 6 the primary context definition of partner_sale_2025_07_31 shadows the temporary table
after line 8 where the CTE partner_sale_2025_07_31 is defined, the query’s use of partner_sale_2025_07_31 will shadow all others
at line 9 the alias as partner_sale_2025_07_31 names the media_type occurrence, so partner_sale_2025_07_31.name addresses that occurrence’s column. It does not rebind the name partner_sale_2025_07_31: a partner_sale_2025_07_31(*) read still selects the CTE.

Warning: an alias names an occurrence, not an entity. In
invoice(*), total > 20 : partner_sale_2025_07_31
media_type(*) as partner_sale_2025_07_31 |> (partner_sale_2025_07_31.name), partner_sale_2025_07_31(*)
the partner_sale_2025_07_31(*) at the end of the query refers to the CTE. The alias never answers a relation name, and its column addressing ended at the scope barrier |> (partner_sale_2025_07_31.name).

Aliases name occurrences¶

An alias (as a) names one read of a relation for column addressing. It creates no relation that a later mention can call:

employee(*) as a, employee(*) as b    // two fresh reads of Employee: a self-join
invoice(*) as a, a.total > 20         // a.Total addresses the first occurrence
album(*) as artist, artist(*) as c    // Artist(*) selects by the shadowing order above

While the occurrence is live, a qualifier addresses its columns. Qualifiers address occurrences, never entities, so an entity that happens to be spelled artist does not compete with the alias. In relation position, artist(*) is an ordinary mention. It selects an independently defined artist by the shadowing order, or it fails; it never reopens the occurrence. A read whose scope would share the live name artist refuses, because two live scopes cannot share a name.

The same holds for a read of a CTE, of a mounted table, or of an aliased table expression (R as u(a, b)). A pipe ends the occurrence’s addressing; to address a result further, name it (|> (…) as v).

Materialization Shadowing¶

Materialization shadowing is exactly the SQL DDL semantic available via CREATE TEMPORARY TABLE and CREATE TEMPORARY VIEW. It is the mechanism by which a session scoped entity may exist only for the duration of that session.

The order induced by the materialized shadow, inside a data namespace D, is:

T[session object recorded as overlaying D] > T[D]

The delightql directives temp_table! and temp_view! may be used to create these shadowing entities.

mount!("etl.sqlite","data::baz")(*)
enlist!("data::baz")(*)

partner_sale_2025_07_31(*) , quantity > 0 |> temp_table!(data::baz.partner_sale_2025_07_31(*))(*)

The target names data::baz as the durable namespace this session object overlays. On SQLite, the mounted database shares the primary connection’s temp schema, so the object’s exact shadow path is sys::shadow::main.partner_sale_2025_07_31, not sys::shadow::data::baz.partner_sale_2025_07_31. Its recorded durable owner is still data::baz. Bare selection applies that overlay to the data::baz candidate before comparing it with peer enlisted namespaces; it does not apply it to main merely because main appears in the shadow path. A second durable owner on the same connection cannot create another physical temp partner_sale_2025_07_31 at that path; the second creation refuses and names the holder.

mount!("etl.sqlite","data::baz")(*)
enlist!("data::baz")(*)
partner_sale_2025_07_31(*) , quantity > 0 |> temp_table!(data::baz.partner_sale_2025_07_31(*))(*)

partner_sale_2025_07_31(*)                   // resolves from the local shadow
data::baz.partner_sale_2025_07_31(*)         // exact durable entity
sys::shadow::main.partner_sale_2025_07_31(*) // exact session entity on this connection

The creation receipt reports both names: target = data::baz.partner_sale_2025_07_31 is the resolved data-namespace target, while created = sys::shadow::main.partner_sale_2025_07_31 is the exact session entity. For a durable table!, target and created coincide. A bare creation target uses the session’s default data-write target (main); a qualified creation target must be data-backed. A source that cannot run on that target’s connection is refused, not relocated. Without a database file, main is backed by the session’s in-memory primary: ordinary table! objects last for that database session, and temporary objects use its temp schema.

Addressing and enlistment¶

Qualification selects an exact route:

(~~ddl:"reports" daily(*) :- invoice(*) |> %(invoice_date ~> sum:(total) as total) ~~)

home::reports.daily(*)     // exact path
.::reports.daily(*)        // relative to the primary definition context
alias!("home::reports", "r")(*)
r.daily(*)                 // exact alias route

An unmarked qualifier is an exact top-level namespace or an explicit alias! shorthand. It does not search the children of enlisted namespaces. Enlisting a namespace makes that namespace’s own entities bare; it does not make child namespaces implicitly addressable and is not transitive.

The session begins with these four enlist edges:

main
std::prelude
std::predicates
sys::meta

home is different: it is the top-level primary definition context, not a fifth auto-enlist. That is why its entities are bare and why they outrank enlisted entities. sys::ns.enlisted_namespace(*) reports enlist edges only; it does not report home merely for being the primary context.

Namespace aliases are qualifier shorthands. They do not grant bare entity names and, like relational aliases, occupy no tier of the shadowing order.

Ambiguity in non-shadowing contexts¶

Any context that is not a CTE or the primary context may be ambiguous with other such contexts. This is inclusive of materialized shadows which still attach to their original namespace.

  { T[shadow-NS1] > T[NS1] }
  ||
  { T[shadow-NS2] > T[NS2] }
  ||
  { T[NS3] } }

Multiple enlisted namespaces may export the same entity name. This overlap does not itself cause an error. A bare reference is ambiguous only when more than one distinct entity remains after applying the shadowing rules. Fully qualified references remain available. The bare references that are shared from more than one enlisted namespaces and which have no shadows are ambiguous entities.

Using an ambiguous entity is an error and must be resolved by the programmer. Enlistment itself still succeeds. Its receipt may list names that have become ambiguous, but the list is advice, not a second resolution authority. The diagnostic at the mention lists the complete set of distinct candidate identities; enlist order and main do not break the tie.

Shadowing is by name before shape. A temporary foo may have a different kind, arity, or heading from the durable foo it shadows. If the selected temporary entity cannot serve a use, that use refuses; DelightQL does not silently fall through to the durable entity.

Binding follows lexical definition¶

Take a library lib::a, consulted from a.dql:

// a.dql
items(id, name
  ------------
  1, "pick"; 2, "tuner")

In the following example, the binding of items within the body of report is a reference to the un-qualified entity from lib::a.

consult!("a.dql", "lib::a")(*)
enlist!("lib::a")(*)

(~~ddl
report(*) :- items(*)
~~)

delist!("lib::a")(*)
report(*)

After a delist!, we retain the reference to items from its namespace because it was defined lexically.

The other option, dynamic scoping, where items is dynamically swapped in and out is not correct semantics.

The dependency retains its namespace identity, not a historical copy of its implementation. Thus, a successful reconsult! of lib::a changes what subsequent statements observe through that dependency.

unconsult! differs from reconsult!: it refuses if removing lib::a would leave a published definition such as report depending on it. The refusal names the blocking definition and namespace. A fully qualified reference may have been written before its target was consulted; once that target exists and the reference resolves, it too blocks removal. Change or remove the dependent definition first, or rebuild the scripted session. This alpha removal rule is ratified; implementation is pending.

When no published definition blocks removal, unconsult! also removes the prompt’s enlistments and namespace aliases pointing to the target. Its receipt names these removed edges. A scratch rule created with (~~ddl … ~~) is a definition, not one of those disposable prompt edges.

For an ordinary mounted data namespace, unmount! follows the same rule: a published definition reading its tables blocks removal, while a successful removal cleans prompt-owned enlistments and aliases and names them in the receipt. The special lifecycle of main remains a separate question.

Retracting a session definition¶

retract! removes one complete session-authored entity family:

(~~ddl foo(*) :- _(x @ 1) ~~)
retract!(foo(*))(*)

The argument is resolved by the ordinary rules. At the prompt, bare foo therefore selects home.foo before an enlisted foo; the explicit spelling is retract!(home.foo(*))(*). This is family removal, not Prolog’s one-matching-clause retraction.

Only mutable definitions authored in home or another scratch namespace may be retracted. Data tables, system entities, consulted-source definitions, and CTEs refuse; an alias is never selected, so it is never a target. A dependent definition or grounding also blocks removal. The receipt names the identity removed; a later bare mention is resolved afresh and may therefore reveal a lower-tier candidate.

Default Transpilation Strategy and Resolution¶

Delightql syntax makes a clear distinction between what is

a table – foo(*)
a higher-order table – foo(bar)(*),
a function – foo:(x,y), or
a sigma predicate – +foo(bar)(*).

These different categories of entity have different rules for both resolution and transpilation. Importantly, functions and higher-order tables among these are permitted to be unknown and transpiled directly as SQL fragments. Tables and sigma predicates must be known.

Table Resolution and Transpilation¶

Calls of the form foo(*) may either be delightql rules or target tables.

artist(*)

In either case, delightql’s resolution both ensures that the name (artist) exists and that its columns (dimensions) are named and in a certain order. For a data-backed table, the default transpilation rule places the entity after a FROM (e.g. FROM artist).

select * from artist;

Resolution for non-namespaced entities checks common-table expressions first followed by any in-scope entities. Only CTEs are allowed to shadow – no named entities sharing the same name are allowed to exist in the same scope.

Higher-order Resolution and Transpilation¶

foo(bar)(*)

Higher-order forms can be delightql rules or target table-value functions. If delightql finds the entity as a delightql-authored higher-order rules, it will use the rule as needed.

But if a delightql-authored higher-order rule is not found it will default to assuming that the entity is a table-value function and transpile as a function-call form after the FROM.

select * from foo(bar);

This default tranpsilation rule runs assuming imperfect knowledge as to whether the SQL target defines such an entity.

Function Resolution and Transpilation¶

Functions may be delightql-authored or functions hosted by the target SQL engine. If delightql finds an entity that is delightql-authored, it will use it first inclusive of shadowing.

Upon finding no delightql-authored function, delightql will emit a SQL fragment that transpiles the delightql function form into the SQL function form:

album(*) |> +( hex:(title) as title_hex)

select *, hex(title) as title_hex from album;

This default tranpsilation rules runs assuming imperfect knowledge as to whether the SQL target defines such an entity. That is to say, the hex in the above SQL may or may not exist as a function in the target SQL engine.

The SQLite that DelightQL ships is the same build everywhere it runs, the command line and the browser playground alike, so a target call answers or refuses the same way on both. That build includes SQLite’s math functions (sqrt, cos, floor, log, …), its JSON functions, soundex, FTS and R-Tree; a call the engine does not define refuses with the engine’s own message.

Sigma Predicate Resolution and Transpilation¶

All sigma predicates have no default transpilation rule. This is because SQL has no uniform syntax for row predication with such diverse syntax as LIKE, BETWEEN, in-fix operators, IN, and EXSISTS.

All delightql calls of the form

track(*), +like(name, "%Love%")

will resolve to a delightql-authored entity (either built-in or programmer authored).

Variable and Column Scoping¶

Within a valid delightql expression the scope of logic variables (columns) is determined by a combination of a left-to-right evaluation strategy and per-operator rules.

The concepts of relational continuations and current pending relation are interrelated with this left-to-right evaluation strategy. Further scoping rules are given to interior relations which provide

Relational Continuation¶

Relational continuations are defined as all strings that - may be appended to a valid delightql relational expression (RELEX) to form a new valid delightql relational expression - and are not themselves valid delightql RELEXs

customer(*), country = "Canada"
   |> +( :"{last_name}, {first_name}" as full_name)

Examples of continuations in the above valid delightql RELEX:

, country = "Canada"
|> +( :"{last_name}, {first_name}" as full_name)
, country = "Canada" |> +( :"{last_name}, {first_name}" as full_name)

The continuation concept allows syntactic analysis of semantic concepts regarding scope and transformation.

Continuation Anchor¶

A continuation anchor is any location in a string where the substring to the left of the anchor is a RELEX and where a CONTINUATION may replace the substring to the right. This is inclusive of a zero width empty string at the end of the string.

album(*)               //  ①
   , artist_id = 22     //  ②
   |> ( title )        //  ③

The above example has three continuation anchors at the end of each line. These are the only three continuation anchors in this string.

Current Pending Relation¶

The current pending relation (CPR) is the relational value of the relational expression (RELEX) to the left of a continuation anchor. It is the ordered set of logic variables (columns) that are in scope.

album(*) as a
   //  ①  CPR = [ a.album_id, a.Title, a.artist_id ]
   , artist_id = 22
   //  ②  CPR = [ a.album_id, a.Title, a.artist_id ]
   |> ( title )
   //  ③  CPR = [ Title ]

Left-to-Right Evaluation¶

Delightql uses a left-to-right evaluation strategy for determining both what variables are in scope and the state of the current pending relation (CPR).

Within a delightql relational expression, parentheses are not used at the relational level to force scope or evaluation – though they are permitted for grouping and binding domain expressions.

Delightql does have something like parentheses, but these are called interior relations and are accurately described as named contextual scopes.

Scope Introduction¶

Logic variables are brought into scope by ground relational expressions (GRELEXs) either directly or via joining or unioning. A GRELEX is inclusive of anonymous tables and literal references.

employee(*)

The employee(*) GRELEX literal reference introduces its logic variables (LVars) into scope.

The current pending relation (CPR) of each new continuation may grow or shrink or stay the same based on the category of relational operator that is applied in the continuation.

employee(*)  //  ①
  ,_(a@3;39) //  ②

The employee(*) GRELEX introduces logic variables into scope, followed by the ,_(a@3;39) JOIN continuation which introduces even more logic variables (just a) into scope.

artist(*)
  ,album(*.(artist_id))
  |> ( name, title) //  ③

The |> ( name, title) PROJECTION continuation removes logic variables and establishes a new scope barrier.

Scope Barrier¶

A scope barrier replaces the local relational interface: following continuations cannot recover its old logic variables unless republished. A pipe inside an interior relation does not also remove bindings from an enclosing scope. Those remain accessible under the ordinary naming rules, without automatically becoming columns of the interior’s output.

album(*) as a
   //①  CPR = [ a.album_id, a.Title, a.artist_id ]
   , artist_id = 22
   //②  CPR = [ a.album_id, a.Title, a.artist_id ]
   |> ( title )
   //③  CPR = [ Title ]
   ,title="Coda"
   //④  no access to a.*.  Only Title

After the third continuation which is a scope barrier, the fourth continuation does not have access to the logic variables a.album_id, a.title, or a.artist_id.

Scope barriers are most often post-pipe projection operators, but also include metaize ^ and witness +.

employee(*) ^ // removes Employee's logical variables from scope

Non-commutativity of usage versus introduction¶

Because delightql uses a left-to-right evaluation scheme, a logic variable cannot be used unless it has been brought into scope to the left. This is in contrast to other “more declarative” languages where the usage and the introduction may be swapped.

album(*), milliseconds < 20000, track(*)

The above is incorrect as the logic variable milliseconds has not yet been brought into scope via a left-to-right evaluation strategy.

Interior relations and scope¶

Interior relations create their own scope within the parentheses of a functor.

track( , milliseconds<60000 |> (name))

An interior relation is constructed by authoring a relational continuation immediately after the opening (left) parenthesis of a named predicate and ending the continuation with the closing (right) parenthesis.

The interior relations’ scope is delimited and private, but has access to the surrounding scope’s logic variables. The final current pending relation (CPR) of the interior relation is published under the name of the functor.

track(                   //  ①
  , milliseconds<60000   //  ②
  |> (name)              //  ③
  )
  //④  Track.Name
  // is the only logic variable

In the above example, a relational continuation is started at position one, and proceeds through to position three. The closing of the interior relation publishes the last CPR of the interior relation as the columns of the relation named track.

The term interior relation is used for multiple manifestations:

simple subqueries
lateral joins
scalar subqueries
EXISTS semi-joins

Nominalization¶

Delightql has strict rules about what columns may be addressed by name and how they may be addressed by name.

Delightql insists that a column may always be addressable by name or never addressable by name (minting).

All columns are always addressable by index ordinalization.

Functor rules¶

A delightql functor (independent of usage – i.e. table, function or semi-join) may be of three forms:

inchoate
argumentative
interior

Inchoate¶

An inchoate functor is the literal reference to a ground relation GRELEX having nothing internal to its parentheses.

employee()

It names the relation without asking for any of its columns, so it selects zero rows. Its columns are opaque: none can be addressed by name, only by position. employee(*) activates them.

Argumentative¶

Interior¶

Minting¶

Any column that results from a function application or from a column ambiguity is automatically minted so that the column name may never be used again.

Function Application Minting¶

invoice_line(*)
  |> +( unit_price * quantity)

The unit_price * quantity column has a name generated for it that may never be used again. This means that absent naming the column at creation with as the only means of accessing this column is by ordinal index.

invoice_line(*)
  |> +( unit_price * quantity)
  |> ( |-1| )

Column Ambiguity¶

track(*), genre(*.(genre_id))
  |> (*)
  // Track.Name and Genre.Name receive
  // minted names

If track.name and genre.name are in-scope in the above example, then both are minted as neither are more real than the other.

Solve this by removing or renaming one of the columns:

track(*), genre(*.(genre_id))
  |> -( genre.name)
  // Name is addressable now

track(*), genre(*.(genre_id))
  |> *( genre.name as genre_name)
  // Name is addressable now
  // genre_name is addressable now

The Post-Fix Operators¶

+
\+
^
*
.(identifier)
.*

Moding and currying¶

Moding is the activity of marking a column as input only, output only, or both. Another way of saying input and output is supplied and returned.

The term mode comes from Prolog where syntax exists to declare certain dimensions to be input and others to be output. In ISO this syntax is documentation only. Some Prolog implementations enforce these annotations.

:- mode concatenate(+,+,-).

We must be aware of this Prolog heritage as we discuss where delightql matches versus diverges from it.

Prolog syntax¶

In Prolog, all predicates obey the same syntax (except for infix) and declare their dimensions.

foo(A,B,C).

But each column could be used in queries as output only, input only, or mixed (and possibly functional). Respectively:

All output:

foo(A,B,C).

All input:

foo(1,"bar",42).

Some input, some output:

foo(A,"bar",C).

In a very important way, the below is also an example of input only:

foo(A,"bar",C),A=1,C=42.

The logic variables A and C in the example above are still being input, as they have been instantiated with ground values.

Delightql’s answer to moding¶

Delightql has semantic enforced definitions of which columns must be input only, versus those that can be mixed input or output.

Succinctly:

All ordinary function parameters are input-only.
Every position in an ordinary table or relational-rule schema is relational/bidirectional: it may be supplied or returned.
All sigma-predicate parameters are input-only.
A fact function with a _ default (TOTAL-CASE) arm:
positions left of -> are input-only
and positions right of -> are output-only.
A fact function without the default arm (PARTIAL-CASE) is table-like: all columns may be input or output when accessed like a table (e.g. foo(a,b,c))
A fact function without the default arm (PARTIAL-CASE) can also be accessed like a function, in which case all ouput columns are elided and only the leftmost input columns are written (e.g. foo:(a,b) where a,b -> c).
A fact-function call with exactly one output may stand directly in scalar/value composition; this implicitly selects its sole output. The output may still be selected explicitly by name.
A fact-function call with several outputs must name the exact output selected whenever a scalar value is required..
All parameters in the first parentheses of a higher-order/input-moded rule are input-only

All of the above semantics are parsable, that is to say, they are all different syntaxes that should be easily discernible.

Functions Column Modality¶

Any syntax of the form foo:(x,y) requires that its columns be existent and passed in:

 employee(*) |> ( upper:(last_name), upper:("literal") )

The above syntax might seem almost counter-intuitively obvious to anyone who has used a general-purpose programming.

But delightql allows other syntax that is still input-moded and which may be unfamiliar to those not used to Prolog:

album(_, title, 22), upper:(title)="CODA"

Nevertheless, the fact that a form foo:(a) is discernible at all means that this column must be input only.

Tables Column Modality¶

Any syntax of the form foo(x,y) means that the columns may be input (supplied) or output (returned).

This matches the syntax and semantics of Prolog.

genre(id,"Jazz")

In the above form, the id variable is output and the "Jazz" argument is input, producing a WHERE predication via grounding:

select genre_id as id from genre where name = 'Jazz';

It is important to understand that in Prolog-like argumentatitve join syntax both columns are input at the same time :

playlist_track(5,track),playlist_track(17,track)

In the above the variable track is input (or bidirectionally unified) to both predicates, creating a join.

The GLOB makes all columns into output columns:

genre(*)

without having to enumerate all dimension positions.

Sigma Predicates Column Modality¶

All calls of the form +foo(x,y) or \+ foo(x,y) are EXISTS/NOT-EXISTS sigma-predicate invocations.

track(*),
  +between(milliseconds,60000,180000),
  +like(name,"%Love%")

Delightql requires that all columns be instantiated as input.

Sigma predicates defined as re-usable rules, must only use forms in their body that pass the input moding to their implementation:

no_data("NA"; "N/A"; "UNKNOWN")

empty(column) :- null = column
empty(column) :- trim:(column) = ""
empty(column) :- +no_data(upper:(column))

Note that all built-in predicates, like = and < and built-in functions, like + and * are also input-only moded.

Observing the definition of a rule consist of calls to input-moded predicates is enough to distinguish this rule from table rules.

The invocation of this rule will always be discernible, by virtue of requiring a + or \+ EXISTS marker before the predicate name:

track(*),
  +empty(composer)

Function Facts Column Modality¶

Function facts are DDL for defining named case statements. They utilitize the -> (spoken F-AND) in their definitions to separate dimensions that are input from dimensions that are output:

department_kind(
  department     -> kind
  ------------------
  "engineering"  -> "tech";
  "data science" -> "tech";
  _              -> "other"
)

In some respects, the invocation of the function fact is indistinguishable from a regular function call:

employee(*) |> +( department_kind:(title) as kind)

but when function facts are not-total, i.e. they lack a default arm _ -> "other",

partial_department_kind(
  department     -> kind
  ------------------
  "engineering"  -> "tech";
  "data science" -> "tech"
)

then they may be called as a table:

partial_department_kind(*)

and their columns may be input or output or both:

partial_department_kind( "engineering", kind)
// or
partial_department_kind( e, k)

and still be available to be called as functions:

employee(*)
  |> +( partial_department_kind:(title) as kind)

Function fact forms that return tuples must have their return columns accessed by name to project out the element of the tuple:

shipping(zone, weight -> carrier, days
         ---------------------------
         1, 5 -> "ground", 3;
         1, 50 -> "freight", 7;
         2, 5 -> "air", 1;
         _ -> "unrouted", 0)

_(zone, weight
  ------------
  1, 5; 1, 50; 2, 5; 3, 20)
  |> (shipping:(zone, weight).carrier as carrier,
      shipping:(zone, weight).days as days)

This is the only place in delightql where the column is output-only and manifest.

Higher-Order Predicates Column Modality¶

The other name for higher-order predicates is input-moded predicates.

They are discernible in code by having multiple parentheses, or by being the target of a relational pipe:

customer(*)
  |> at_least(50)(*)

at_least(50, customer(*))(*)

The first parentheses (the ones to the left and closest to the name of the higher-order rule) contain only input-only columns. The columns in the second parentheses are treated like table columns which are input-or-output.

Again, a literal reference is not required to trigger input instantiation:

_(val @ 1;2;3;4), iota(val)(*)

In the above example, the anonymous table’s lone column val becomes the input to the iota parameter set.