Local-first SQL that syncs, streams and queries

Name: Dux
Author: linesofcode

Dux is an open source sync stack. Apps read and write SQLite on the device, changes merge into PostgreSQL, and every committed row version lands in DuckLake for dux sql. Built in Rust by linesofcode.

For humansFor agents

$npm create dux-app my-app

$curl https://dux.sh/llms.txt

llms.txt AGENTS.md

Currently v0.1.0 · pre-release, M1 in progress

compute/sensors.yaml

# compute/sensors.yaml
dag: sensors
trigger: { every: 1m, on: readings }        # one run per closed minute of event time
steps:
  normalize:
    sql: SELECT * FROM readings WHERE ts >= $from AND ts < $to AND celsius > -99
    writes: clean_readings                   # a stream table: appended
  baseline:
    needs: [normalize]                       # runs once normalize finished the same minute
    sql: SELECT site, avg(celsius) AS mean, $from AS ts FROM clean_readings WHERE ts >= $from AND ts < $to GROUP BY site
    writes: site_baseline                    # a lake table: the minute is replaced on a re-run
  alert:
    needs: [normalize, baseline]
    entry: steps.py:alert                    # Python, with requirements.txt dependencies
    reads: [clean_readings, site_baseline]
    writes: [{ table: alerts, key: [key] }]  # merged on key; a synced table upserts and syncs to devices
    secrets: [PAGERDUTY_KEY]

compute/hot.ts

// compute/hot.ts
import { defineStep } from "@dux/compute";
export const hot = defineStep({
  reads: "clean_readings", writes: { table: "hot", key: ["sensor_id"] }, needs: ["normalize"],
  async run(ctx) { for await (const row of ctx.read("clean_readings")) if (row.celsius > 60) ctx.write(row); },
});

1.88×Ingest against Timescale's loader, TSBS devops

27 of 27TSBS query types with identical rows

560,000Accepted rows/s over HTTP, p99 ack 11.2 ms

24Rust crates in the workspace

Deterministic by default

dux-core has no I/O, clock or randomness. A seed replays the same trace and the same state hash.

SQL all the way down

SQLite on the device, PostgreSQL on the server, DuckDB over the lake. No query language to learn.

Measured in public

Benchmarks ship with the command that produced them, including the queries where Dux is slower.

One client, every surface

The TypeScript client runs the Rust engine as WebAssembly. Reads and writes hit local SQLite; client.start() syncs them. The same project deploys steps in SQL, Python and TypeScript.

Visit documentation

Supports

postgresqlsqliteduckdbpythontypescriptrust+ any Postgres client

ClientStreamsPython stepWorkflow

app.tsx

import { DuxClient } from "@dux/client";
import { createWebEngine } from "@dux/client/web";
import { DuxProvider, useQuery, useWrite } from "@dux/client/react";

const engine = await createWebEngine({ registry, account: "acme/alice" });
const client = new DuxClient({ engine, url: "wss://sync.example.com/sync", token: getToken });
client.start();

outbox.ts

await client.stream("metrics").append({ device: "d1", ts: new Date().toISOString(), value: 21.5 });

compute/steps.py

# compute/steps.py
from dux_compute import step

@step(reads="clean_readings", writes="site_stats", needs=["normalize"])
def site_stats(ctx):
    for row in ctx.read("clean_readings"):      # this step's interval only
        ...
        ctx.write({...})
    return {"done": True}                       # the run's output

compute/escalate.yaml

# compute/escalate.yaml
workflow: escalate
trigger: { change: { table: alerts, on: insert, where: "excess > 20" } }
steps:
  notify: { http: { url: "https://hooks.example.com/x", body: { text: "${{ trigger.row.site }}" } } }
  wait_ack: { needs: [notify], wait_event: { name: ack, match: "${{ trigger.row.key }}", timeout: 15m } }
  ask: { needs: [wait_ack], when: "${{ steps.wait_ack.output.timed_out }}", approval: { message: "Shut down ${{ trigger.row.site }}?" } }
  each: { needs: [ask], foreach: "${{ trigger.row.devices }}", concurrency: 4, do: { entry: ops.py:reboot, with: { id: "${{ item }}" } } }
outputs: { approved: "${{ steps.ask.output.approved }}" }

Offline-first sync

Run dux dev in a project and it compiles dux.schema, starts PostgreSQL on a free port and serves sync. Devices write SQLite offline; buckets and write rules decide what each user sees and may change. M1 commands.

terminal

cargo run -- init /tmp/dux-app
cd /tmp/dux-app
/path/to/dux/target/debug/dux dev
# Compiles dux.schema, starts PostgreSQL in .dux/postgres on a free port (or uses
# DUX_DATABASE_URL), serves sync on 127.0.0.1:8080 (or a free port if taken) and
# writes .dux-dev.json (endpoint, token, inspector URL, database URL).
# Inspector: /dev/inspector. Extra users: GET /dev/token?user=NAME.
# Attachments: set DUX_S3_ENDPOINT, DUX_S3_ACCESS_KEY, DUX_S3_SECRET_KEY
# (optional DUX_S3_BUCKET, DUX_S3_REGION, DUX_BLOB_DIR) to mount /blob.
# Analytics lake (docs/p1.md): dux dev runs one in .dux/lake (SQLite catalog,
# 2 s flush); dux serve opts in with --lake sqlite|postgres|files.
# Stream tables (docs/p2.md): POST /ingest appends; rollups run after each
# flush (1 s in dev); retention every --retention-secs (60 s in dev), which also
# expires DuckLake snapshots older than --lake-snapshot-days. `dux sql` includes
# rows still in the stream log.
# Streams need `write <stream> insert: <predicate>`. `dux dev --no-postgres` serves streams, rollups and
# /query over the lake alone (no sync).
# Production: configure issuer, audience and HTTPS JWKS instead.
/path/to/dux/target/debug/dux sql "SELECT lane_id, count(*) FROM cards GROUP BY 1"
/path/to/dux/target/debug/dux lake ingest visits visits.csv
/path/to/dux/target/debug/dux serve --issuer https://issuer.example --audience dux --jwks-url https://issuer.example/jwks.json --cors-origin https://app.example
# Without --database-url, serve streams and /query only: add --lake files|sqlite.

Replay any failure

The simulator drives three clients through dropped acks, partitions, crashes and clock jumps. Pass --seed and --steps; the same pair reproduces the same trace and state hash.

$ cargo run -- simulate --seed 7 --steps 1000captured output

seed=7 steps=1000 accepted=23 rejected=159 events=1326 state_hash=4fbadd1bf0e4a38f converged=true

Streams without the wait

Append-only rows go to POST /ingest or the client outbox and are acknowledged after fsync. Watermarked rollups can sync back to devices. P2 evidence.

Producers × rows per batch	Accepted rows/s	Ack p50	Ack p99	Notes
2 × 500	148,000	5.8 ms	12.6 ms	flush keeps up (86k rows left at the end)
4 × 500	286,000	5.8 ms	12.9 ms	flush keeps up
16 × 1,000	560,000	7.0 ms	11.2 ms	flush-bound: 4,176 busy (503) replies, backlog held at 2M rows

Speaks Postgres

--pg-listen serves the Postgres wire protocol, so psql and Grafana connect without a plugin. Timescale-style hyperfunctions like time_bucket_gapfill work as-is. Grafana walkthrough.

panel.sql

SELECT time_bucket_gapfill('2 minutes', ts) AS time,
       locf(approx_percentile(0.95, percentile_agg(value))) AS p95
FROM metrics WHERE $__timeFilter(ts) GROUP BY 1 ORDER BY 1

Run code next to the data

Put DAGs and workflows in compute/ and ship them with dux deploy. Outputs land exactly once in stream, lake or synced tables. P9 status.

terminal

dux dev --no-postgres --lake sqlite        # compute is on; python3 / bun run steps locally
mkdir compute && $EDITOR compute/sensors.yaml
dux deploy                                  # bundles compute/, validates, activates
dux runs ls                                 # the ledger

Against Timescale

TSBS's own generator, queries and runner on both, in containers with equal limits, via cargo run --release -p dux-tsbs -- run. p99 in ms with Dux's result cache off. Selective point lookups are still slower, and the table says so. Full results.

1.88×Ingest against Timescale's loader, TSBS devops

27 of 27TSBS query types with identical rows

devops query	Timescale best	dux	ratio (first run)
double-groupby-1 / -5 / -all	118.7 / 210.9 / 213.6	26.4 / 31.7 / 47.0	0.22 / 0.15 / 0.22 (0.97 / 0.76 / 0.65)
cpu-max-all-8	18.2	11.0	0.61 (9.89)
cpu-max-all-1	8.7	6.9	0.79 (20.79)
single-groupby-1-8-1	3.9	7.2	1.86 (13.66)
single-groupby-5-1-12 / -1-1-12	3.0 / 3.3	6.5 / 9.3	2.15 / 2.79 (16.44 / 18.15)
high-cpu-all	102.5	240.1	2.34 (17.80)
high-cpu-1	2.9	7.6	2.64 (27.31)
lastpoint	3.0	8.7	2.95 (88.37)
single-groupby-5-8-1	3.2	10.0	3.15 (4.95)
single-groupby-5-1-1 / -1-1-1	1.7 / 1.5	10.5 / 11.5	6.20 / 7.72 (24.74 / 24.26)
groupby-orderby-limit	1.9	28.8	15.08 (102.70)

Where it stands

Version 0.1.0 is a pre-release. The stopping-point report lists every open item.

What holds 3

Same rows as Timescale on 27 of 27 TSBS query types
Acknowledged stream rows survive kill -9 exactly once
Seeded simulations replay byte for byte

What is a judgement 3

Ingest numbers are from one laptop, scale 250
Point lookups are 2–8× slower than Timescale's indexes
The sync performance targets are not measured yet

What is not here yet 3

Loro text and RPC features
Torn-write and power-loss durability tests
A published release and the LICENSE file

Start

Scaffold an app with npm create dux-app, or run the full gate on a checkout before you change anything.

New app

npm create dux-app my-app        # kanban template: offline, collaborative, attachments
cd my-app && npm install
dux dev                          # local Postgres + sync server (install `dux` first)
npm run dev

Verification gate

cargo fmt --all -- --check
cargo test --workspace --locked
cargo clippy --workspace --all-targets --locked -- -D warnings
cargo +1.85.1 test --workspace --locked
cargo check -p dux-core --target wasm32-unknown-unknown --locked

Build something local-first today

Open the console Documentation$npm create dux-app my-app