Dev stack

When to use: edit code and see results in seconds, no containers.

Every service runs directly on the host under overmind. Postgres and NATS are brew-installed. Data lives in ./data/ (gitignored). For prod-parity validation (Docker images, manager-spawned bots, bot lifecycle via docker-socket-proxy) use make e2e-up.

Prerequisites

• macOS with Homebrew
• Rust toolchain (rustup)
• pnpm
• Docker Desktop (for make e2e-up only — not required for dev stack itself)

First-run setup

1. Install brew dependencies

brew install postgresql@16 nats-server overmind pnpm
cargo install cargo-watch

postgresql@16 is keg-only — brew does not symlink it onto $PATH. dev-init finds it via brew --prefix postgresql@16. To add it to your shell permanently:

export PATH="/opt/homebrew/opt/postgresql@16/bin:$PATH"

2. Get Binance testnet keys

1. Sign in at https://testnet.binancefuture.com with GitHub.
2. Generate an API key (right column, "API Key" tab, "Create API Key").
3. Save the public key and secret — the secret is shown once.

Testnet keys cannot move real funds. Store with mode 600.

3. Run bootstrap

make dev-init

This is idempotent. On first run:

• Verifies every brew dep is on PATH.
• Copies .env.dev.example to .env.dev (mode 600). The script exits and tells you to fill in the testnet key and secret.
• Runs initdb into ./data/postgres/.
• Creates the gordon database.
• Runs gordon-migrate (creates schemas, roles, partitions).
• Pre-declares the gordon-bus NATS JetStream stream.
• Seeds the bot_configs row keyed by GORDON_BOT_ID.

Fill in testnet keys, then re-run make dev-init — it skips every step that is already done.

4. Launch the stack

make dev-up

Overmind tiles every process in one terminal. Expected output:

postgres | LOG:  database system is ready to accept connections
nats     | [INF] Listening for client connections on 0.0.0.0:4222
data     | INFO gordon-data started on 127.0.0.1:8081
risk     | INFO gordon-risk started on 127.0.0.1:8082
manager  | INFO gordon-manager started on 127.0.0.1:8083
bot      | INFO gordon-bot started on 127.0.0.1:8084
executor | INFO gordon-executor started on 127.0.0.1:8085
console  | ready - started server on 0.0.0.0:3000

Open http://localhost:3000.

5. Seed historical data

make dev-seed

First run: approximately 20 minutes for BTC+ETH 1y of klines, funding, OI, metrics, Fear and Greed, stablecoin supply, GEX snapshot, and FRED macro. Re-runs only fetch the gap (ON CONFLICT DO NOTHING).

Skip stages: SKIP_GEX=1 SKIP_MACRO=1 make dev-seed

Override window: FROM_DATE=2025-01-01 make dev-seed

Daily loop

make dev-up        # launch; postgres data + ./data/ persist from last session

# edit code, save
# changed Rust crate auto-rebuilds (~3s incremental), service auto-restarts
# console hot-reloads via Next.js HMR

# in another terminal:
overmind restart bot    # bounce one process
overmind connect data   # focus a process pane (Ctrl-b d to detach)
make dev-psql           # quick SQL session

make dev-down      # at end of day — keeps ./data/

Auth in dev mode

Kanidm is bypassed via GORDON_CONSOLE_E2E_INJECT_HEADERS=true. Sign in with the token in .env.dev (GORDON_CONSOLE_E2E_AUTH_TOKEN=dev-auth-token). Real Kanidm only fires when that flag is false.

Vault integration

When Binance testnet keys are vaulted in homelab/group_vars/all/vault.yml:

make dev-vault            # decrypt vault, patch .env.dev
make dev-vault FORCE=--force   # overwrite existing .env.dev keys

Until the vault has these entries, paste the keys directly into .env.dev.

Inner-loop performance — sccache + nextest

The workspace ships .cargo/config.toml with rustc-wrapper = "sccache". When sccache is on PATH, cargo wraps every rustc invocation — cross-session compile-unit cache, significant wins on cold builds across the 12-crate meta-workspace.

Install the full dev toolchain (sccache, nextest, cargo-watch, llvm-cov):

./scripts/dev-stack-install.sh           # idempotent — safe to re-run
./scripts/dev-stack-install.sh --upgrade # bump all four to latest

Run tests in parallel via nextest:

cargo nextest run --workspace

Coverage still uses cargo llvm-cov (not nextest). Each repo's make coverage uses the cargo llvm-cov ... | jq pattern.

Meta-workspace mechanics

A Cargo.toml at the workspace root binds every Rust gordon-* crate into one cargo workspace. [patch.<kellnr-url>] entries rewrite the dependency graph to sibling source trees so any cross-crate edit shows up immediately in every consumer.

CI and Docker do not see the meta-workspace — each service's git repo is its own clone. Release integrity is preserved; the patches only affect local builds inside the gordon-workspace tree.

Lockfile drift

The meta-workspace produces gordon-workspace/Cargo.lock shared across all members. Per-crate Cargo.lock files (committed in each gordon-* repo) stay independent and are what CI/Docker use. Do not commit cargo update-induced workspace-lock churn.

Bumping a per-crate Cargo.lock between releases

make sync-crate-lock CRATE=gordon-manager DEP=gordon-strategy VERSION=4.2.0
make sync-crate-lock CRATE=gordon-bot     DEP=gordon-strategy
make sync-crate-lock CRATE=gordon-executor                          # all kellnr deps

The script snapshots workspace/Cargo.toml, temporarily excludes the target crate from [workspace] members, runs cargo update inside the crate (standalone), then restores. EXIT/INT/TERM traps guarantee restore even on Ctrl-C. Never auto-commits.

Patch-dropout gate

The workspace pre-push hook (scripts/check-workspace-patches.sh) fails the push if any gordon-* crate resolved to the kellnr registry instead of the sibling path tree. Approximately 500ms warm. Failure prints the dropped crate, version, and three-option remediation.

E2E build — Docker BuildKit cache mounts

Every Rust service Dockerfile uses --mount=type=cache for /build/target plus shared cargo-registry and cargo-git mounts. Cache mounts are ID'd per-service for target/ so feature unification across services with different gordon-platform features cannot cross-contaminate.

make e2e-build runs sequentially. Parallel builds OOM'd the 8 GB Docker VM (2026-04-20 postmortem). Cache mounts make sequential builds fast on iteration.

Verify

After make dev-up, confirm services are healthy:

make dev-status
curl -fsS http://localhost:8081/healthz
curl -fsS http://localhost:8082/healthz
curl -fsS http://localhost:8083/healthz

Expected output: HTTP 200 on every endpoint.

Troubleshooting

port 5432 already in use

A second Postgres is running (Postgres.app, Docker postgres from make e2e-up, system service). Stop it or change PGPORT in .env.dev.

port 4222 already in use

A Docker NATS from make e2e-up is running:

make e2e-down
make dev-up

gordon-bot crash-loops with "missing GORDON_BOT_ID"

The bot expects a UUIDv7 in GORDON_BOT_ID and a matching row in trading.bot_configs. Both are seeded by dev-init. If you wiped the DB without re-running init, re-run it.

Service won't pick up an env change

Overmind reads .env.dev at start:

overmind restart <process>

Disk filling up

./data/postgres grows with klines and precomputed TFs. BTC+ETH 1y is approximately 500 MB.

make dev-reset   # nukes ./data/ entirely — re-init needed afterwards

Backfill stops with HTTP 451 / region-blocked

You are calling Binance from a blocked region. Use a VPN or rely on synthetic data from make e2e-up (canned fixtures via mock-binance).

Related

• E2E testing — Docker stack validation
• Make targets — full command reference
• Data backfill — historical data bootstrap