Dux

https://dux.sh

The PostgreSQL/SQLite sync stack is runnable, with authenticated transport, Rust/TypeScript/React Native clients and an analytics lake: every committed row version lands in DuckLake or Parquet and is queryable with DuckDB through dux sql or POST /query (P1). Stream tables take high-volume append-only rows through POST /ingest or the TypeScript outbox, with watermarked rollups that can sync to devices, with or without PostgreSQL (P2). P3 phase A (status) puts them on a CAS manifest (SQLite, Postgres or S3) with batch ids, range-set dedupe and mergeable rollup partials. Nodes gossip (--gossip), replicate quorum batches to each other, spread commits, compaction and rollups across the cluster through leases, and answer queries with every node's acknowledged rows. Phase B adds a Postgres wire endpoint (psql, Grafana: walkthrough), Timescale-style hyperfunctions (time_bucket_gapfill, locf, interpolate, stats_agg, percentile_agg, counter_agg, time_weight, first/last) that also roll up, a result cache, a <stream>__last view, natural-key dedupe (dedupe on (…) within 1h) and predicate deletes (POST /delete, or DELETE over pgwire) that reach every copy of a row and the rollups built from it. Phase C starts with TSBS against Timescale: the same data, queries and query runner on both, in containers with equal limits (cargo run --release -p dux-tsbs -- run, results). P9 (status) runs your code inside Dux: data DAGs of SQL, Python and TypeScript steps over event-time intervals (every 1m on readings, after a rollup window, needs between steps), and workflows with waits, approvals, fan-out and sub-workflows, started by cron, events, webhooks or row changes. dux deploy ships compute/; outputs land exactly once in stream, lake or synced tables; steps run as local processes, in Docker, or on Fly Machines; branches replay history into their own tables. M0 is complete; M1.5 (status) is closing the remaining M1 gaps, and attachments run end to end. Start with the stopping-point report, M1 commands, and roadmap. The specification review records correctness changes to spec.md.

cargo run -- simulate --seed 7 --steps 1000
cargo run -- simulate --seed 7 --steps 1000 --trace

Captured run (2026-10-06, this commit's build):

$ cargo run -- simulate --seed 7 --steps 1000
seed=7 steps=1000 accepted=23 rejected=159 events=1326 state_hash=4fbadd1bf0e4a38f converged=true

A successful simulation prints converged=true, accepted/rejected transaction counts and a canonical state hash. Repeating the same seed and step count with the same build reproduces the same trace and result. dux_sim::replay(seed, &report.events) replays the recorded events without RNG. The CLI supports up to 100,000 random steps; fault checks emphasize correctness rather than throughput.

Core API

dux-core has no I/O, clock access or randomness and compiles for WASM. It provides injected-time Hlc::{send,receive,restart,within_drift} with the 60 s MAX_DRIFT_MS bound, typed IDs and composite keys, canonical Value constructors/codecs, packed generation-aware column clocks, RowState::{new,tombstone,set,counter,join,apply,encode,decode,hash}, and a RowOp envelope with table/key/transaction/origin/sequence identity.

use std::collections::BTreeMap;
use dux_core::{hlc::Hlc, ids::{ColumnId, SiteId},
    merge::{ColumnType, RowState, ScalarType}, metadata::Version, value::Value};

let column = ColumnId(1);
let schema = BTreeMap::from([(column, ColumnType::Lww(ScalarType::String))]);
let mut clock = Hlc::new(1000, 0).unwrap();
let site = SiteId([1; 16]);
let mut row = RowState::new(schema,
    Version { generation: 1, clock: clock.send(1000).unwrap(), site },
    BTreeMap::from([(column, Value::String("hello".into()))])).unwrap();
row.set(column, Value::String("updated".into()),
    Version { generation: 1, clock: clock.send(1000).unwrap(), site }).unwrap();
assert_eq!(row.value(column).unwrap(), Value::String("updated".into()));

Row generations are nonzero: odd is live, even is deleted. Every live image contains every schema column. A higher generation discards lower-generation column clocks, PN totals and min/max state. Creation/resurrection requires a complete image; M0 operations carry full state snapshots so delayed edits can precede creation. Equal LWW versions with different values return ClockConflict; invalid updates and failed joins leave the receiver unchanged. Child sets are empty-schema rows keyed by (parent, element) and use observed resurrection. PN totals merge by per-site max; reading a total outside i64 returns CounterOverflow without discarding the state. No PN entries are retired.

LWW scalars and blob references, counters and homogeneous scalar min/max are supported. Floats normalize NaN and negative zero; positive canonical NaN sorts after positive infinity. Decimal strings use normalized plain notation; exponent decimal inputs are rejected. JSON preserves exact decimal numbers, explicitly sorts object keys and canonicalizes equivalent number spellings; exponents must fit checked i64 arithmetic. This avoids silent f64 rounding of PostgreSQL JSON numbers. Null is allowed for LWW/blob, and rejected for ordered extrema. Text reports UnsupportedText until Loro integration. M1 adds typed PostgreSQL adapters and schema authority/rules; delta transport and Loro text remain pending.

The version-1 format is documented in each codec module: canonical LEB128/zigzag integers, little-endian HLC/floats, sorted IDs, typed length-prefixed fields. Unknown versions, noncanonical bytes, duplicate entries, trailing data and malformed lengths are rejected. Encoded objects are limited to 16 MiB; composite keys have 1–32 parts. Blob references include SHA-256 hash, size, MIME and optional key ID; this crate does not compute/verify blob SHA-256. State hashes are xxh3-64; op checksums use its low 32 bits with wrapping u32 sums. These hashes are not authentication.

Fault model and boundaries

The three-client in-memory model exercises two-row transactions split across two bucket fragments, contiguous per-site uploads, exact accepted/rejected receipts, dropped requests/acks, duplicates/reordering, process crash/restart, partitions, clock jumps and checkpoint healing. Each event checks paired-row atomic visibility, HLC monotonicity, checksums and retained accepted journal/receipt payloads. After healing, canonical state hashes must match. Tests intentionally corrupt checksums, partial visibility, journals and clocks to prove the checker detects faults.

Durable model snapshots swap atomically; crash discards volatile checkpoint fragments. This does not test real fsync, torn writes, power loss, MVCC or SQL/network adapters. Accepted upload receipts immediately update the client's acknowledged base while downloaded checkpoints advance cursors; production must define their persistence/recovery lifecycle. Pending intents replay over that base after rejection; submitted payloads stay fixed for exact retries, and rejected intents remain available in model storage. This model does not establish a production read-your-writes guarantee under remote clock conflicts.

Verification

Rust 1.85.1 is the minimum supported version (the duckdb crate requires it); unsafe code is forbidden outside the FFI crate and clippy pedantic is enabled. DuckDB is linked as the official prebuilt library, downloaded at build time (.cargo/config.toml), never compiled. The lake's DuckDB extensions (ducklake, sqlite, postgres, httpfs) download on first use into ~/.duckdb/extensions (DUX_DUCKDB_EXTENSIONS overrides).

cargo fmt --all -- --check
cargo test --workspace --locked
cargo clippy --workspace --all-targets --locked -- -D warnings
cargo +1.85.1 test --workspace --locked
cargo check -p dux-core --target wasm32-unknown-unknown --locked

CI runs tests on stable and 1.85.1 against live PostgreSQL and RustFS, checks WASM, and runs a deterministic simulation. The specification's sync performance targets have not been measured; stream ingest has (cargo run --release -p dux-stream --example ingest_bench, see P2), and so has the time-series path against Timescale (TSBS). License: Elastic License 2.0 (decided; the LICENSE file is not added yet).