docs(omo-codex): batch 90 (12 files)
This commit is contained in:
YeonGyu-Kim
@@ -0,0 +1,317 @@
|
||||
# Rust Programmer
|
||||
|
||||
Production Rust in 2026. **Explicit allocation, compile-time proof, zero hidden cost.** Type-state-first, unsafe-banished-by-default, agent-proof.
|
||||
|
||||
## Identity — What Kind of Rust You Write
|
||||
|
||||
You write Rust that looks like a Zig programmer designed it and a Rust compiler enforces it. Every allocation is visible. Every cost is explicit. Every invariant is encoded in the type system. Every cleanup is deterministic. The borrow checker, lifetime analysis, trait bounds, and `miri` then guarantee what Zig leaves to discipline.
|
||||
|
||||
**Five pillars, every file, no exceptions:**
|
||||
|
||||
| Pillar | Default Behavior | Reference |
|
||||
|---|---|---|
|
||||
| **Explicit allocation** | Arena for hot paths, `&[T]`/`Cow` over `Vec`/`String` in signatures, `try_*` when allocation can fail | [zero-cost-safety.md §1](zero-cost-safety.md) |
|
||||
| **Compile-time proof** | `const fn` everything const-eligible, `const { assert!(...) }` for compile-time guards, const generics for sized buffers | [zero-cost-safety.md §2](zero-cost-safety.md) |
|
||||
| **Zero hidden cost** | Slice-based APIs where caller owns memory, no hidden `.clone()`/`.to_string()`, `Cow` to defer allocation | [zero-cost-safety.md §3](zero-cost-safety.md) |
|
||||
| **Type-encoded invariants** | Newtype wrappers for every semantic unit, type-state for state machines, branded IDs | [type-state.md](type-state.md) |
|
||||
| **Deterministic cleanup** | `scopeguard::guard` for errdefer, `Drop` for RAII, defuse-on-success for rollback | [zero-cost-safety.md §5](zero-cost-safety.md) |
|
||||
|
||||
The two highest-leverage tools Rust gives a coding agent:
|
||||
|
||||
1. **Bounded polymorphism** (traits). Real, machine-checked, composable constraints.
|
||||
2. **Newtype-as-coordinate-space.** `Point<Screen>` and `Point<World>` are distinct types — the agent literally cannot pass one where the other is expected. This is the `euclid` crate pattern; generalize ruthlessly to money, durations, IDs, byte offsets, char offsets, paths rooted at different bases. Full patterns → [type-state.md](type-state.md).
|
||||
|
||||
---
|
||||
|
||||
## Hard Rules (Every `.rs` File)
|
||||
|
||||
### 1. No `unwrap()`, No `expect()` Outside Tests
|
||||
|
||||
```rust
|
||||
// WRONG
|
||||
let val = map.get("key").unwrap();
|
||||
|
||||
// RIGHT — propagate or provide context
|
||||
let val = map.get("key").context("missing 'key' in config")?;
|
||||
```
|
||||
|
||||
Typed errors for libraries ([thiserror](https://docs.rs/thiserror)), ad-hoc errors for binaries ([anyhow](https://docs.rs/anyhow) / [color-eyre](https://docs.rs/color-eyre)). Full stack → [libraries.md](libraries.md).
|
||||
|
||||
### 2. No `unsafe` Without Miri Proof
|
||||
|
||||
If `unsafe` is unavoidable, you have miri. Run it. Always. **Load [`../rust-ub/README.md`](../rust-ub/README.md) plus every file under [`../rust-ub/`](../rust-ub/)** for the full UB taxonomy, Miri escalation protocol (4 strictness levels), and the fix-and-prove workflow. Every `unsafe` block needs the three components from [unsafe-discipline.md](unsafe-discipline.md): safe wrapper, `// SAFETY:` comment, miri test.
|
||||
|
||||
```bash
|
||||
cargo +nightly miri nextest run
|
||||
```
|
||||
|
||||
### 3. Explicit Allocation — Arena by Default in Hot Paths
|
||||
|
||||
**Do not scatter `Box::new()` / `Vec::new()` across hot loops.** Use arena allocation to make allocation scope visible and bulk-freeable. Full recipes → [zero-cost-safety.md §1](zero-cost-safety.md).
|
||||
|
||||
```rust
|
||||
use bumpalo::Bump;
|
||||
|
||||
fn parse_frame<'a>(arena: &'a Bump, raw: &[u8]) -> Frame<'a> {
|
||||
let header = arena.alloc(parse_header(raw));
|
||||
let payload = arena.alloc_slice_copy(&raw[HEADER_LEN..]);
|
||||
Frame { header, payload }
|
||||
}
|
||||
// Caller owns arena. Caller decides when memory dies. Zero individual frees.
|
||||
```
|
||||
|
||||
When arena is overkill (simple CLI, one-shot allocation), `Vec`/`String` are fine — but **function signatures still prefer borrows**:
|
||||
|
||||
```rust
|
||||
// WRONG — forces caller to allocate
|
||||
fn process(input: String) -> String { ... }
|
||||
|
||||
// RIGHT — caller chooses allocation strategy
|
||||
fn process(input: &str) -> Cow<'_, str> { ... }
|
||||
|
||||
// BEST for hot paths — zero allocation, caller provides buffer
|
||||
fn process(input: &[u8], output: &mut [u8]) -> usize { ... }
|
||||
```
|
||||
|
||||
### 4. Compile-Time First — const fn Everything Const-Eligible
|
||||
|
||||
If a function CAN be `const fn`, it MUST be `const fn`. Full recipes → [zero-cost-safety.md §2](zero-cost-safety.md).
|
||||
|
||||
```rust
|
||||
// Lookup tables computed at compile time — zero runtime cost
|
||||
const CRC_TABLE: [u32; 256] = {
|
||||
let mut table = [0u32; 256];
|
||||
let mut i = 0;
|
||||
while i < 256 {
|
||||
let mut crc = i as u32;
|
||||
let mut j = 0;
|
||||
while j < 8 {
|
||||
crc = if crc & 1 != 0 { (crc >> 1) ^ 0xEDB88320 } else { crc >> 1 };
|
||||
j += 1;
|
||||
}
|
||||
table[i] = crc;
|
||||
i += 1;
|
||||
}
|
||||
table
|
||||
};
|
||||
|
||||
// Compile-time assertions — catch violations at build time, not runtime
|
||||
const { assert!(std::mem::size_of::<Header>() == 12, "Header must be 12 bytes") };
|
||||
```
|
||||
|
||||
Use `const generics` for stack-allocated buffers with compile-time size:
|
||||
|
||||
```rust
|
||||
struct RingBuffer<T, const N: usize> {
|
||||
data: [MaybeUninit<T>; N],
|
||||
head: usize,
|
||||
len: usize,
|
||||
}
|
||||
```
|
||||
|
||||
### 5. Scope Guards — Deterministic Cleanup on Every Path
|
||||
|
||||
Zig's `errdefer` in Rust. Full recipes → [zero-cost-safety.md §5](zero-cost-safety.md).
|
||||
|
||||
```rust
|
||||
use scopeguard::guard;
|
||||
|
||||
fn deploy(artifact: &Path) -> Result<(), DeployError> {
|
||||
let backup = snapshot_current()?;
|
||||
// errdefer: restore on failure
|
||||
let rollback = guard(backup, |b| { let _ = restore(&b); });
|
||||
|
||||
upload(artifact)?;
|
||||
health_check()?;
|
||||
|
||||
// Success: defuse the guard
|
||||
scopeguard::ScopeGuard::into_inner(rollback);
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
### 6. Bit-Level Layout — zerocopy for Wire Formats
|
||||
|
||||
Never hand-write `transmute` or pointer casts for parsing binary data. Full recipes → [zero-cost-safety.md §4](zero-cost-safety.md).
|
||||
|
||||
```rust
|
||||
use zerocopy::{FromBytes, IntoBytes, KnownLayout, Immutable};
|
||||
|
||||
#[derive(FromBytes, IntoBytes, KnownLayout, Immutable)]
|
||||
#[repr(C)]
|
||||
struct PacketHeader {
|
||||
magic: [u8; 4],
|
||||
version: u8,
|
||||
flags: u8,
|
||||
length: [u8; 2], // use byte array for packed fields, decode via from_le_bytes
|
||||
}
|
||||
```
|
||||
|
||||
### 7. Exhaustive Match — No Wildcard on Enums You Control
|
||||
|
||||
```rust
|
||||
// WRONG — silently ignores new variants
|
||||
match status {
|
||||
Status::Ok => handle_ok(),
|
||||
_ => handle_error(),
|
||||
}
|
||||
|
||||
// RIGHT — compiler forces update when variants change
|
||||
match status {
|
||||
Status::Ok => handle_ok(),
|
||||
Status::NotFound => handle_not_found(),
|
||||
Status::Timeout => handle_timeout(),
|
||||
}
|
||||
```
|
||||
|
||||
For `#[non_exhaustive]` enums from external crates, the wildcard `_` is required — but add a `tracing::warn!` in the catch-all so you notice when new variants appear.
|
||||
|
||||
### 8. Type-State Over Runtime Checks
|
||||
|
||||
Never `if self.state == State::Validated`. Encode states as distinct types so the compiler refuses invalid transitions. Full patterns → [type-state.md](type-state.md).
|
||||
|
||||
```rust
|
||||
struct Order<S: OrderState> { data: OrderData, _state: PhantomData<S> }
|
||||
struct Draft;
|
||||
struct Validated;
|
||||
struct Paid;
|
||||
|
||||
impl Order<Draft> {
|
||||
fn validate(self) -> Result<Order<Validated>, ValidationError> { ... }
|
||||
}
|
||||
impl Order<Validated> {
|
||||
fn pay(self, payment: Payment) -> Result<Order<Paid>, PaymentError> { ... }
|
||||
}
|
||||
// Order<Draft> has no .pay() method. Compiler enforces the workflow.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Standard Library Defaults
|
||||
|
||||
Full decision tree with rationale and code snippets → [libraries.md](libraries.md).
|
||||
|
||||
| Category | Crate | Why |
|
||||
|---|---|---|
|
||||
| Async runtime | `tokio` | Ecosystem standard. Patterns → [async-tokio.md](async-tokio.md) |
|
||||
| HTTP server | `axum` + `tower` | Type-safe extractors, tower middleware. Stack → [axum-stack.md](axum-stack.md) |
|
||||
| CLI | `clap` derive + `color-eyre` | Typed args, beautiful errors. Stack → [clap-stack.md](clap-stack.md) |
|
||||
| Serialization | `serde` + `serde_json` | Non-negotiable for any boundary type |
|
||||
| Error (library) | `thiserror` | Derive `Error` with zero boilerplate |
|
||||
| Error (binary) | `anyhow` / `color-eyre` | Context-rich ad-hoc errors |
|
||||
| Database | `sqlx` (compile-time checked) | No runtime SQL surprises |
|
||||
| Arena alloc | `bumpalo` / `typed-arena` | Explicit allocation scope. Patterns → [zero-cost-safety.md §1](zero-cost-safety.md) |
|
||||
| Zero-copy parse | `zerocopy` | Safe binary parsing, no transmute. Patterns → [zero-cost-safety.md §4](zero-cost-safety.md) |
|
||||
| Scope guard | `scopeguard` | errdefer/defer. Patterns → [zero-cost-safety.md §5](zero-cost-safety.md) |
|
||||
| Stack collections | `smallvec` / `arrayvec` / `tinyvec` | Stack-first, heap-spillover. Patterns → [zero-cost-safety.md §3](zero-cost-safety.md) |
|
||||
| Bitfield | `bitfield` / `modular-bitfield` | Bit-packed flags. Patterns → [zero-cost-safety.md §4](zero-cost-safety.md) |
|
||||
| Testing | `proptest` + `insta` | Property + snapshot tests. Patterns → [proptest-insta.md](proptest-insta.md) |
|
||||
| Concurrency | `tokio::sync` / `parking_lot` | Channel-first, lock-second. Patterns → [concurrency.md](concurrency.md) |
|
||||
|
||||
---
|
||||
|
||||
## Cargo Strict Configuration
|
||||
|
||||
Every new project gets the strict lint config from [cargo-strict.md](cargo-strict.md). The non-negotiable CI gate:
|
||||
|
||||
```bash
|
||||
cargo fmt --all -- --check && \
|
||||
cargo clippy --all-targets --all-features -- -D warnings && \
|
||||
cargo nextest run && \
|
||||
cargo +nightly miri nextest run # when unsafe is involved
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Code Review Checklist (Post-Write, Every PR)
|
||||
|
||||
Run through this list after writing any Rust code. Every item links to its recipe.
|
||||
|
||||
| # | Check | Fix Reference |
|
||||
|---|---|---|
|
||||
| 1 | Every function signature prefers `&[T]`/`&str`/`Cow` over owned types | [zero-cost-safety.md §3](zero-cost-safety.md) |
|
||||
| 2 | Hot-path allocations use arena (`bumpalo`) not scattered `Box`/`Vec` | [zero-cost-safety.md §1](zero-cost-safety.md) |
|
||||
| 3 | Const-eligible functions are `const fn` | [zero-cost-safety.md §2](zero-cost-safety.md) |
|
||||
| 4 | Lookup tables / config constants computed at compile time | [zero-cost-safety.md §2](zero-cost-safety.md) |
|
||||
| 5 | Binary format parsing uses `zerocopy`, not `transmute` | [zero-cost-safety.md §4](zero-cost-safety.md) |
|
||||
| 6 | Cleanup logic uses `scopeguard` or `Drop`, never manual `if err` cleanup | [zero-cost-safety.md §5](zero-cost-safety.md) |
|
||||
| 7 | Distinct semantic units are newtypes, not primitive aliases | [type-state.md](type-state.md) |
|
||||
| 8 | State machines use type-state, not runtime `if state ==` | [type-state.md](type-state.md) |
|
||||
| 9 | No `unwrap()`/`expect()` outside `#[cfg(test)]` | [libraries.md](libraries.md) |
|
||||
| 10 | Every `unsafe` has SAFETY comment + miri test | [unsafe-discipline.md](unsafe-discipline.md), [../rust-ub/](../rust-ub/) |
|
||||
| 11 | Match on owned enums is exhaustive (no `_ =>`) | This file §7 |
|
||||
| 12 | Clippy pedantic passes with zero warnings | [cargo-strict.md](cargo-strict.md) |
|
||||
| 13 | Property tests exist for any function with a nontrivial domain | [proptest-insta.md](proptest-insta.md) |
|
||||
| 14 | Concurrency uses channels first, locks second, atomics last | [concurrency.md](concurrency.md) |
|
||||
| 15 | Async code uses `JoinSet` for structured concurrency | [async-tokio.md](async-tokio.md) |
|
||||
|
||||
---
|
||||
|
||||
## Default Cargo.toml Dependencies — Zero-Cost Safety Stack
|
||||
|
||||
Every new project starts with these alongside the standard deps from [cargo-strict.md](cargo-strict.md):
|
||||
|
||||
```toml
|
||||
# Zero-cost safety stack
|
||||
bumpalo = { version = "3", features = ["collections"] }
|
||||
scopeguard = "1"
|
||||
smallvec = { version = "1", features = ["union", "const_generics"] }
|
||||
zerocopy = { version = "0.8", features = ["derive"] }
|
||||
|
||||
# Add when needed:
|
||||
# typed-arena = "2" # homogeneous arena
|
||||
# arrayvec = "0.7" # fixed-capacity stack vec
|
||||
# tinyvec = { version = "1", features = ["alloc"] }
|
||||
# bitfield = "0.17" # bit-packed flags
|
||||
# modular-bitfield = "0.11" # richer bitfield API
|
||||
# bytemuck = { version = "1", features = ["derive"] }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Reference Index
|
||||
|
||||
| File | When to Load |
|
||||
|---|---|
|
||||
| [zero-cost-safety.md](zero-cost-safety.md) | Arena, allocator, const fn, comptime, zero-alloc, bitfield, repr, scopeguard, errdefer, Zig-like patterns |
|
||||
| [type-state.md](type-state.md) | Newtype wrappers, type-state machines, branded IDs, phantom types |
|
||||
| [unsafe-discipline.md](unsafe-discipline.md) | Any `unsafe` block — SAFETY comments, safe wrappers, miri proof |
|
||||
| [libraries.md](libraries.md) | Library selection, crate decision tree, dependency audit |
|
||||
| [cargo-strict.md](cargo-strict.md) | Project bootstrap, lint config, CI gate commands |
|
||||
| [async-tokio.md](async-tokio.md) | Async runtime, spawning, cancellation, `JoinSet`, `select!` |
|
||||
| [axum-stack.md](axum-stack.md) | HTTP services — axum + sqlx + tower + tracing |
|
||||
| [clap-stack.md](clap-stack.md) | CLI tools — clap derive + color-eyre + indicatif |
|
||||
| [concurrency.md](concurrency.md) | Locks, atomics, channels, loom model checker |
|
||||
| [proptest-insta.md](proptest-insta.md) | Property tests, snapshot tests, round-trip invariants |
|
||||
| [one-liners.md](one-liners.md) | `rust-script` one-liners, disposable scripts, inline deps |
|
||||
| [../rust-ub/README.md](../rust-ub/README.md) | UB hunting — miri escalation, sanitizers, fuzzing |
|
||||
| [../rust-ub/ub-taxonomy.md](../rust-ub/ub-taxonomy.md) | 14-category UB taxonomy with detection status |
|
||||
| [../rust-ub/miri-sanitizers-loom.md](../rust-ub/miri-sanitizers-loom.md) | Miri flags, ASAN/TSAN/MSAN, loom, cargo-fuzz |
|
||||
|
||||
---
|
||||
|
||||
## The Shape of Every Function
|
||||
|
||||
```rust
|
||||
/// One-line doc explaining WHAT, not HOW.
|
||||
///
|
||||
/// # Errors
|
||||
/// Returns `FooError::Bar` when the input is invalid.
|
||||
const fn frobnicate<'a>(
|
||||
arena: &'a Bump, // explicit allocator when arena is in play
|
||||
input: &[u8], // borrow, not owned
|
||||
output: &mut [u8], // caller-provided buffer
|
||||
) -> Result<&'a Frob, FrobError> {
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
**Why this shape:** the caller sees every cost. Allocation scope is the arena's lifetime. Input is borrowed. Output buffer is caller-owned. Error is typed. The compiler enforces all of it.
|
||||
|
||||
---
|
||||
|
||||
## Activation
|
||||
|
||||
This skill activates whenever you are writing or modifying any `.rs` file or `Cargo.toml`. One-off scripts get the strict treatment too — `rust-script` + the same lints, the same gates. Details → [one-liners.md](one-liners.md).
|
||||
|
||||
**The promise:** production hygiene with throwaway ergonomics. Explicit allocation, compile-time proof, zero hidden cost, and **agent-proof safety at any volume**.
|
||||
@@ -0,0 +1,299 @@
|
||||
# Async with Tokio
|
||||
|
||||
Structured concurrency, cancellation, blocking-work isolation, channel selection. The patterns the agent should reach for by default.
|
||||
|
||||
## Runtime selection
|
||||
|
||||
```rust
|
||||
// Default for services and CLIs that do real work
|
||||
#[tokio::main(flavor = "multi_thread", worker_threads = 8)]
|
||||
async fn main() -> anyhow::Result<()> { ... }
|
||||
|
||||
// For tiny CLIs or wasm where you measured single-thread is enough
|
||||
#[tokio::main(flavor = "current_thread")]
|
||||
async fn main() -> anyhow::Result<()> { ... }
|
||||
```
|
||||
|
||||
Pick worker count explicitly. The default (`num_cpus`) is fine for servers; for desktop tools you usually want 2-4.
|
||||
|
||||
## Spawning
|
||||
|
||||
`tokio::spawn` returns a `JoinHandle<T>`. The future runs to completion even if the handle is dropped (detached). To enforce structured concurrency, use `JoinSet`:
|
||||
|
||||
```rust
|
||||
use tokio::task::JoinSet;
|
||||
|
||||
let mut set = JoinSet::new();
|
||||
for url in urls {
|
||||
let client = client.clone();
|
||||
set.spawn(async move { fetch(&client, &url).await });
|
||||
}
|
||||
|
||||
let mut results = Vec::new();
|
||||
while let Some(joined) = set.join_next().await {
|
||||
match joined {
|
||||
Ok(Ok(body)) => results.push(body),
|
||||
Ok(Err(error)) => tracing::warn!(%error, "fetch failed"),
|
||||
Err(panicked) if panicked.is_panic() => {
|
||||
tracing::error!(?panicked, "worker panicked");
|
||||
// Choose: re-raise, or continue with degraded result set.
|
||||
}
|
||||
Err(other) => tracing::error!(?other, "worker join error"),
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`JoinSet`:
|
||||
- Knows when all spawned tasks finish.
|
||||
- Dropping the set aborts every still-running task.
|
||||
- Lets you handle failures one by one rather than all-or-nothing.
|
||||
|
||||
For wait-for-all semantics with one type, `join!`:
|
||||
|
||||
```rust
|
||||
let (a, b, c) = tokio::join!(load_a(), load_b(), load_c());
|
||||
let a = a?; let b = b?; let c = c?;
|
||||
```
|
||||
|
||||
For first-of-many, `select!`:
|
||||
|
||||
```rust
|
||||
tokio::select! {
|
||||
biased; // bias to top-to-bottom checking when ordering matters
|
||||
_ = shutdown.recv() => {
|
||||
tracing::info!("shutdown signal");
|
||||
return Ok(());
|
||||
}
|
||||
request = listener.accept() => {
|
||||
handle_request(request?).await?;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Without `biased`, branches are polled in random order each iteration (good for fairness). Use `biased` only when you need deterministic priority (shutdown signal first, etc).
|
||||
|
||||
## Cancellation
|
||||
|
||||
A future is cancelled when it is dropped (e.g., the `select!` arm wins another branch). **Always think: if this future is dropped mid-await, what state is left behind?**
|
||||
|
||||
Cancel-safe futures (you can drop without lasting effect):
|
||||
- `recv()` on channels
|
||||
- `accept()` on listeners
|
||||
- `wait_for` on `watch::Receiver`
|
||||
- `read_buf`/`write_all` on streams **only when buffers are owned by the future**, otherwise no
|
||||
|
||||
Cancel-unsafe futures (dropping mid-way leaves partial state):
|
||||
- Manual `read_exact` into an external buffer
|
||||
- Custom futures that perform partial side effects before suspending
|
||||
|
||||
If a function is cancel-unsafe, document it in a rustdoc `# Cancel Safety` section.
|
||||
|
||||
To explicitly opt out of cancellation, use `tokio_util::sync::CancellationToken`:
|
||||
|
||||
```rust
|
||||
use tokio_util::sync::CancellationToken;
|
||||
|
||||
let token = CancellationToken::new();
|
||||
let child = token.child_token();
|
||||
tokio::spawn(async move {
|
||||
tokio::select! {
|
||||
_ = child.cancelled() => { /* clean up */ }
|
||||
result = work() => { /* normal */ }
|
||||
}
|
||||
});
|
||||
// later
|
||||
token.cancel();
|
||||
```
|
||||
|
||||
Pass child tokens down the call tree so the whole tree can be cancelled together.
|
||||
|
||||
## Timeouts
|
||||
|
||||
```rust
|
||||
use tokio::time::{timeout, Duration};
|
||||
|
||||
match timeout(Duration::from_secs(5), fetch(url)).await {
|
||||
Ok(Ok(body)) => Ok(body),
|
||||
Ok(Err(error)) => Err(error.into()),
|
||||
Err(_elapsed) => Err(anyhow::anyhow!("timed out fetching {url}")),
|
||||
}
|
||||
```
|
||||
|
||||
Set timeouts on every external I/O boundary. Defaults of "wait forever" are bugs.
|
||||
|
||||
## Blocking work
|
||||
|
||||
NEVER block inside an async task. Symptoms: deadlock, every future stalled, latency cliffs.
|
||||
|
||||
Heavy CPU or sync I/O → `spawn_blocking`:
|
||||
|
||||
```rust
|
||||
let result = tokio::task::spawn_blocking(|| {
|
||||
// CPU-bound: parsing, hashing, image processing
|
||||
// Or sync I/O: rusqlite, OS APIs without async wrappers
|
||||
expensive_pure_computation()
|
||||
}).await?;
|
||||
```
|
||||
|
||||
Long-running blocking jobs (more than ~1 second of CPU) → use a dedicated thread pool (`rayon`), not tokio's blocking pool which is sized for short bursts.
|
||||
|
||||
## Channels
|
||||
|
||||
| Need | Use |
|
||||
|---|---|
|
||||
| 1-many producers → 1 consumer, async | `tokio::sync::mpsc::channel(cap)` |
|
||||
| Same as above, both sync + async | `flume::bounded(cap)` |
|
||||
| 1 → many fan-out, latest-value semantics | `tokio::sync::watch::channel(initial)` |
|
||||
| 1 → many fan-out, queued | `tokio::sync::broadcast::channel(cap)` |
|
||||
| One-shot reply | `tokio::sync::oneshot::channel()` |
|
||||
| Backpressure-driven stream of items | `tokio::sync::mpsc::Receiver` + `ReceiverStream` |
|
||||
|
||||
Mpsc pattern:
|
||||
|
||||
```rust
|
||||
let (tx, mut rx) = tokio::sync::mpsc::channel::<Job>(256);
|
||||
|
||||
tokio::spawn(async move {
|
||||
while let Some(job) = rx.recv().await {
|
||||
if let Err(error) = process(job).await {
|
||||
tracing::warn!(%error, "job failed");
|
||||
}
|
||||
}
|
||||
tracing::info!("queue closed, shutting down worker");
|
||||
});
|
||||
|
||||
tx.send(Job { ... }).await?; // blocks if full, applies backpressure
|
||||
```
|
||||
|
||||
Always bound channels. Unbounded channels are a memory leak waiting to happen.
|
||||
|
||||
## Streams
|
||||
|
||||
`futures::Stream` is the async analogue of `Iterator`. Use it for paginated fetches, long-poll responses, file lines.
|
||||
|
||||
```rust
|
||||
use futures::stream::{StreamExt, TryStreamExt};
|
||||
|
||||
let urls: Vec<String> = ...;
|
||||
let bodies: Vec<String> = futures::stream::iter(urls)
|
||||
.map(|url| async move { fetch(&url).await })
|
||||
.buffer_unordered(8) // up to 8 in flight
|
||||
.try_collect()
|
||||
.await?;
|
||||
```
|
||||
|
||||
`buffer_unordered(n)` is the throttle. Use it instead of spawning N tasks manually.
|
||||
|
||||
For producing a stream from a channel:
|
||||
|
||||
```rust
|
||||
use tokio_stream::wrappers::ReceiverStream;
|
||||
|
||||
let (tx, rx) = tokio::sync::mpsc::channel::<Event>(64);
|
||||
let stream = ReceiverStream::new(rx);
|
||||
serve_sse(stream).await
|
||||
```
|
||||
|
||||
## Graceful shutdown
|
||||
|
||||
```rust
|
||||
use tokio::signal;
|
||||
|
||||
async fn shutdown_signal() {
|
||||
let ctrl_c = async { signal::ctrl_c().await.expect("ctrl_c handler") };
|
||||
#[cfg(unix)]
|
||||
let terminate = async {
|
||||
signal::unix::signal(signal::unix::SignalKind::terminate())
|
||||
.expect("install signal handler")
|
||||
.recv()
|
||||
.await;
|
||||
};
|
||||
#[cfg(not(unix))]
|
||||
let terminate = std::future::pending::<()>();
|
||||
|
||||
tokio::select! {
|
||||
_ = ctrl_c => {},
|
||||
_ = terminate => {},
|
||||
}
|
||||
tracing::info!("shutdown signal received");
|
||||
}
|
||||
|
||||
#[tokio::main]
|
||||
async fn main() -> anyhow::Result<()> {
|
||||
let token = CancellationToken::new();
|
||||
let server = tokio::spawn(run_server(token.child_token()));
|
||||
shutdown_signal().await;
|
||||
token.cancel();
|
||||
let _ = tokio::time::timeout(Duration::from_secs(10), server).await;
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
Pattern: catch signal → cancel a token shared with the server → server's `select!` arms see the cancel and exit cleanly → wait with a timeout so a hung worker can't deadlock shutdown.
|
||||
|
||||
## Concurrency primitives
|
||||
|
||||
- `tokio::sync::Mutex` — async mutex. Use for state shared between async tasks. **Do not hold across `.await` without thinking** (you'll serialize the whole system).
|
||||
- `tokio::sync::RwLock` — async read-write lock. Same caveat.
|
||||
- `parking_lot::Mutex` — sync mutex, faster than `std::sync::Mutex`, no poisoning. Use when the lock is held briefly and you do not need to `.await` while holding it.
|
||||
- `tokio::sync::Semaphore` — bound concurrent operations. Perfect for "max 10 in-flight HTTP requests" or "max 3 DB writers".
|
||||
|
||||
```rust
|
||||
let sem = Arc::new(tokio::sync::Semaphore::new(10));
|
||||
for url in urls {
|
||||
let permit = sem.clone().acquire_owned().await?;
|
||||
tokio::spawn(async move {
|
||||
let _permit = permit; // released on task end
|
||||
fetch(&url).await
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
## Common mistakes
|
||||
|
||||
1. **Holding a sync mutex across `.await`.** Compiles and runs, deadlocks at scale. Solution: refactor to release before await, or use `tokio::sync::Mutex`.
|
||||
2. **Forgetting `?` on `JoinHandle`.** A panicked task returns `Err(JoinError)`; if you `.await` and ignore, panics are silently swallowed.
|
||||
3. **`tokio::spawn` instead of `JoinSet`.** Detached tasks survive past their parent, causing leaks. Default to `JoinSet` for structured concurrency.
|
||||
4. **Unbounded channels.** Always set a capacity.
|
||||
5. **`block_on` inside an async context.** Causes deadlock under `current_thread` runtime, performance cliff under `multi_thread`.
|
||||
6. **CPU-heavy work in async fn.** Move to `spawn_blocking` or `rayon`.
|
||||
7. **No timeout on external I/O.** Every `await` that touches the network or filesystem needs `tokio::time::timeout` wrapping.
|
||||
|
||||
## Testing async code
|
||||
|
||||
```rust
|
||||
#[tokio::test]
|
||||
async fn fetches_and_parses() {
|
||||
let server = wiremock::MockServer::start().await;
|
||||
wiremock::Mock::given(wiremock::matchers::method("GET"))
|
||||
.respond_with(wiremock::ResponseTemplate::new(200).set_body_string("{\"id\":1}"))
|
||||
.mount(&server)
|
||||
.await;
|
||||
|
||||
let result = my_client::fetch(&server.uri()).await.unwrap();
|
||||
assert_eq!(result.id, 1);
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
|
||||
async fn parallel_work() { ... }
|
||||
```
|
||||
|
||||
For time-sensitive tests, advance virtual time:
|
||||
|
||||
```rust
|
||||
#[tokio::test(start_paused = true)]
|
||||
async fn time_travel() {
|
||||
let start = tokio::time::Instant::now();
|
||||
tokio::time::sleep(Duration::from_secs(3600)).await;
|
||||
assert!(start.elapsed() >= Duration::from_secs(3600));
|
||||
// Real wallclock elapsed: ~0ms.
|
||||
}
|
||||
```
|
||||
|
||||
## When NOT to use async
|
||||
|
||||
- Single-threaded CPU-heavy code that does no I/O — plain `fn` + `rayon` is simpler and often faster.
|
||||
- Trivial scripts that do one HTTP call — `ureq` (sync) is simpler.
|
||||
- FFI heavy code where the FFI side is sync.
|
||||
|
||||
Async pays off when you have many concurrent I/O operations or need cancellation as a first-class primitive.
|
||||
@@ -0,0 +1,467 @@
|
||||
# axum + sqlx + tracing + tower — HTTP API Stack
|
||||
|
||||
The canonical production HTTP service in Rust 2026.
|
||||
|
||||
## Cargo.toml dependencies
|
||||
|
||||
```toml
|
||||
[dependencies]
|
||||
axum = { version = "0.8", features = ["macros", "tracing", "ws", "multipart"] }
|
||||
tokio = { version = "1", features = ["full"] }
|
||||
tower = "0.5"
|
||||
tower-http = { version = "0.6", features = [
|
||||
"trace", "compression-gzip", "compression-br",
|
||||
"timeout", "cors", "request-id", "sensitive-headers",
|
||||
"limit", "set-header",
|
||||
] }
|
||||
|
||||
# Errors / observability
|
||||
anyhow = "1"
|
||||
thiserror = "2"
|
||||
tracing = "0.1"
|
||||
tracing-subscriber = { version = "0.3", features = ["env-filter", "json"] }
|
||||
color-eyre = "0.6"
|
||||
|
||||
# Database
|
||||
sqlx = { version = "0.8", features = [
|
||||
"runtime-tokio-rustls", "postgres", "uuid", "macros",
|
||||
"migrate", "json",
|
||||
] }
|
||||
|
||||
# Serialization / validation
|
||||
serde = { version = "1", features = ["derive"] }
|
||||
serde_json = "1"
|
||||
validator = { version = "0.18", features = ["derive"] }
|
||||
|
||||
# Types
|
||||
uuid = { version = "1", features = ["v4", "v7", "serde"] }
|
||||
jiff = { version = "0.1", features = ["serde"] }
|
||||
|
||||
# Config
|
||||
config = { version = "0.14", features = ["toml", "yaml"] }
|
||||
secrecy = { version = "0.10", features = ["serde"] }
|
||||
|
||||
# OpenAPI (optional but recommended)
|
||||
utoipa = { version = "5", features = ["axum_extras", "uuid", "chrono"] }
|
||||
utoipa-axum = "0.1"
|
||||
utoipa-swagger-ui = { version = "8", features = ["axum"] }
|
||||
```
|
||||
|
||||
## Project structure
|
||||
|
||||
```
|
||||
src/
|
||||
main.rs # binary entry
|
||||
lib.rs # re-exports + app builder
|
||||
config.rs # Settings type + loader
|
||||
state.rs # AppState (shared via Arc)
|
||||
routes/
|
||||
mod.rs # Router::new() composition
|
||||
health.rs
|
||||
users.rs
|
||||
middleware/
|
||||
mod.rs
|
||||
auth.rs
|
||||
request_id.rs
|
||||
models/
|
||||
mod.rs
|
||||
user.rs
|
||||
error.rs # AppError + IntoResponse impl
|
||||
db/
|
||||
mod.rs
|
||||
migrations/
|
||||
migrations/ # sqlx migrations
|
||||
```
|
||||
|
||||
## Error type
|
||||
|
||||
```rust
|
||||
// src/error.rs
|
||||
use axum::{http::StatusCode, response::{IntoResponse, Response}, Json};
|
||||
use serde_json::json;
|
||||
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
pub enum AppError {
|
||||
#[error("not found")]
|
||||
NotFound,
|
||||
#[error("unauthorized")]
|
||||
Unauthorized,
|
||||
#[error("validation: {0}")]
|
||||
Validation(String),
|
||||
#[error("conflict: {0}")]
|
||||
Conflict(String),
|
||||
#[error("internal")]
|
||||
Internal(#[from] anyhow::Error),
|
||||
#[error("database")]
|
||||
Database(#[from] sqlx::Error),
|
||||
}
|
||||
|
||||
impl IntoResponse for AppError {
|
||||
fn into_response(self) -> Response {
|
||||
let (status, code, message) = match &self {
|
||||
AppError::NotFound => (StatusCode::NOT_FOUND, "not_found", self.to_string()),
|
||||
AppError::Unauthorized => (StatusCode::UNAUTHORIZED, "unauthorized", "unauthorized".into()),
|
||||
AppError::Validation(m) => (StatusCode::UNPROCESSABLE_ENTITY, "validation", m.clone()),
|
||||
AppError::Conflict(m) => (StatusCode::CONFLICT, "conflict", m.clone()),
|
||||
AppError::Database(e) => {
|
||||
tracing::error!(error = ?e, "database error");
|
||||
(StatusCode::INTERNAL_SERVER_ERROR, "database", "internal".into())
|
||||
}
|
||||
AppError::Internal(e) => {
|
||||
tracing::error!(error = ?e, "internal error");
|
||||
(StatusCode::INTERNAL_SERVER_ERROR, "internal", "internal".into())
|
||||
}
|
||||
};
|
||||
(status, Json(json!({"error": {"code": code, "message": message}}))).into_response()
|
||||
}
|
||||
}
|
||||
|
||||
pub type AppResult<T> = std::result::Result<T, AppError>;
|
||||
```
|
||||
|
||||
Pattern: business errors return `AppResult<T>`; the `IntoResponse` impl translates them to HTTP. `sqlx::Error` and `anyhow::Error` auto-convert via `From`. Internal-bucket errors are logged but never leak their `Debug` representation to clients.
|
||||
|
||||
## AppState
|
||||
|
||||
```rust
|
||||
// src/state.rs
|
||||
use std::sync::Arc;
|
||||
use sqlx::PgPool;
|
||||
|
||||
#[derive(Clone)]
|
||||
pub struct AppState {
|
||||
pub db: PgPool,
|
||||
pub config: Arc<crate::config::Settings>,
|
||||
pub http: reqwest::Client,
|
||||
}
|
||||
|
||||
impl AppState {
|
||||
pub async fn new(config: crate::config::Settings) -> anyhow::Result<Self> {
|
||||
let db = sqlx::postgres::PgPoolOptions::new()
|
||||
.max_connections(config.db.max_connections)
|
||||
.acquire_timeout(std::time::Duration::from_secs(3))
|
||||
.connect(config.db.url.expose_secret())
|
||||
.await?;
|
||||
sqlx::migrate!("./migrations").run(&db).await?;
|
||||
let http = reqwest::Client::builder()
|
||||
.timeout(std::time::Duration::from_secs(15))
|
||||
.user_agent(concat!(env!("CARGO_PKG_NAME"), "/", env!("CARGO_PKG_VERSION")))
|
||||
.build()?;
|
||||
Ok(Self { db, config: Arc::new(config), http })
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Route handler
|
||||
|
||||
```rust
|
||||
// src/routes/users.rs
|
||||
use axum::{
|
||||
extract::{Path, State},
|
||||
Json,
|
||||
};
|
||||
use serde::{Deserialize, Serialize};
|
||||
use uuid::Uuid;
|
||||
use validator::Validate;
|
||||
|
||||
use crate::{error::{AppError, AppResult}, state::AppState};
|
||||
|
||||
#[derive(Debug, Deserialize, Validate)]
|
||||
pub struct CreateUser {
|
||||
#[validate(email)]
|
||||
pub email: String,
|
||||
#[validate(length(min = 1, max = 100))]
|
||||
pub name: String,
|
||||
}
|
||||
|
||||
#[derive(Debug, Serialize, sqlx::FromRow)]
|
||||
pub struct User {
|
||||
pub id: Uuid,
|
||||
pub email: String,
|
||||
pub name: String,
|
||||
pub created_at: jiff::Timestamp,
|
||||
}
|
||||
|
||||
#[tracing::instrument(skip(state, body))]
|
||||
pub async fn create_user(
|
||||
State(state): State<AppState>,
|
||||
Json(body): Json<CreateUser>,
|
||||
) -> AppResult<(axum::http::StatusCode, Json<User>)> {
|
||||
body.validate().map_err(|e| AppError::Validation(e.to_string()))?;
|
||||
|
||||
let id = Uuid::now_v7();
|
||||
let user = sqlx::query_as!(
|
||||
User,
|
||||
r#"INSERT INTO users (id, email, name, created_at)
|
||||
VALUES ($1, $2, $3, NOW())
|
||||
RETURNING id, email, name, created_at as "created_at: jiff::Timestamp""#,
|
||||
id, body.email, body.name
|
||||
)
|
||||
.fetch_one(&state.db)
|
||||
.await
|
||||
.map_err(|e| match &e {
|
||||
sqlx::Error::Database(db) if db.code().as_deref() == Some("23505") =>
|
||||
AppError::Conflict("email already exists".into()),
|
||||
_ => AppError::Database(e),
|
||||
})?;
|
||||
|
||||
Ok((axum::http::StatusCode::CREATED, Json(user)))
|
||||
}
|
||||
|
||||
#[tracing::instrument(skip(state))]
|
||||
pub async fn get_user(
|
||||
State(state): State<AppState>,
|
||||
Path(id): Path<Uuid>,
|
||||
) -> AppResult<Json<User>> {
|
||||
sqlx::query_as!(
|
||||
User,
|
||||
r#"SELECT id, email, name, created_at as "created_at: jiff::Timestamp"
|
||||
FROM users WHERE id = $1"#,
|
||||
id
|
||||
)
|
||||
.fetch_optional(&state.db)
|
||||
.await?
|
||||
.map(Json)
|
||||
.ok_or(AppError::NotFound)
|
||||
}
|
||||
```
|
||||
|
||||
## Router assembly
|
||||
|
||||
```rust
|
||||
// src/routes/mod.rs
|
||||
use axum::{routing::{get, post}, Router};
|
||||
use tower_http::{
|
||||
compression::CompressionLayer,
|
||||
cors::CorsLayer,
|
||||
request_id::{MakeRequestUuid, PropagateRequestIdLayer, SetRequestIdLayer},
|
||||
sensitive_headers::SetSensitiveHeadersLayer,
|
||||
timeout::TimeoutLayer,
|
||||
trace::{DefaultMakeSpan, DefaultOnResponse, TraceLayer},
|
||||
};
|
||||
use std::time::Duration;
|
||||
use crate::state::AppState;
|
||||
|
||||
mod health;
|
||||
mod users;
|
||||
|
||||
pub fn router(state: AppState) -> Router {
|
||||
let api = Router::new()
|
||||
.route("/health", get(health::handler))
|
||||
.route("/users", post(users::create_user))
|
||||
.route("/users/:id", get(users::get_user))
|
||||
.with_state(state);
|
||||
|
||||
Router::new()
|
||||
.nest("/api/v1", api)
|
||||
.layer(
|
||||
tower::ServiceBuilder::new()
|
||||
.layer(SetSensitiveHeadersLayer::new([
|
||||
axum::http::header::AUTHORIZATION,
|
||||
axum::http::header::COOKIE,
|
||||
]))
|
||||
.layer(SetRequestIdLayer::x_request_id(MakeRequestUuid))
|
||||
.layer(
|
||||
TraceLayer::new_for_http()
|
||||
.make_span_with(DefaultMakeSpan::new().include_headers(false))
|
||||
.on_response(DefaultOnResponse::new().latency_unit(tower_http::LatencyUnit::Millis)),
|
||||
)
|
||||
.layer(PropagateRequestIdLayer::x_request_id())
|
||||
.layer(TimeoutLayer::new(Duration::from_secs(30)))
|
||||
.layer(CompressionLayer::new())
|
||||
.layer(CorsLayer::permissive()), // tighten in production
|
||||
)
|
||||
}
|
||||
```
|
||||
|
||||
Order matters: outermost layer wraps the request first. Trace before timeout so timeouts get logged. Compression after trace so trace sees the original body size.
|
||||
|
||||
## Main + graceful shutdown
|
||||
|
||||
```rust
|
||||
// src/main.rs
|
||||
use my_app::{config::Settings, routes, state::AppState};
|
||||
|
||||
#[tokio::main]
|
||||
async fn main() -> color_eyre::Result<()> {
|
||||
color_eyre::install()?;
|
||||
init_tracing();
|
||||
|
||||
let config = Settings::load()?;
|
||||
let state = AppState::new(config.clone()).await?;
|
||||
let app = routes::router(state);
|
||||
|
||||
let listener = tokio::net::TcpListener::bind(&config.bind).await?;
|
||||
tracing::info!(addr = %config.bind, "listening");
|
||||
|
||||
axum::serve(listener, app)
|
||||
.with_graceful_shutdown(shutdown_signal())
|
||||
.await?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn init_tracing() {
|
||||
use tracing_subscriber::{fmt, EnvFilter};
|
||||
let filter = EnvFilter::try_from_default_env()
|
||||
.unwrap_or_else(|_| EnvFilter::new("info,sqlx=warn,hyper=warn,tower_http=info"));
|
||||
fmt().with_env_filter(filter).with_target(false).json().init();
|
||||
}
|
||||
|
||||
async fn shutdown_signal() {
|
||||
let ctrl_c = async { tokio::signal::ctrl_c().await.expect("ctrl_c handler"); };
|
||||
#[cfg(unix)]
|
||||
let terminate = async {
|
||||
tokio::signal::unix::signal(tokio::signal::unix::SignalKind::terminate())
|
||||
.expect("signal handler").recv().await;
|
||||
};
|
||||
#[cfg(not(unix))]
|
||||
let terminate = std::future::pending::<()>();
|
||||
|
||||
tokio::select! { _ = ctrl_c => {}, _ = terminate => {} }
|
||||
tracing::info!("shutting down");
|
||||
}
|
||||
```
|
||||
|
||||
## Middleware: bearer auth example
|
||||
|
||||
```rust
|
||||
// src/middleware/auth.rs
|
||||
use axum::{
|
||||
extract::{Request, State},
|
||||
http::header::AUTHORIZATION,
|
||||
middleware::Next,
|
||||
response::Response,
|
||||
};
|
||||
use crate::{error::AppError, state::AppState};
|
||||
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct AuthUser { pub id: uuid::Uuid }
|
||||
|
||||
pub async fn require_auth(
|
||||
State(state): State<AppState>,
|
||||
mut request: Request,
|
||||
next: Next,
|
||||
) -> Result<Response, AppError> {
|
||||
let token = request.headers()
|
||||
.get(AUTHORIZATION)
|
||||
.and_then(|v| v.to_str().ok())
|
||||
.and_then(|s| s.strip_prefix("Bearer "))
|
||||
.ok_or(AppError::Unauthorized)?;
|
||||
|
||||
let claims = verify_jwt(token, &state.config.jwt_secret)?;
|
||||
request.extensions_mut().insert(AuthUser { id: claims.sub });
|
||||
Ok(next.run(request).await)
|
||||
}
|
||||
```
|
||||
|
||||
Apply with `.route_layer(middleware::from_fn_with_state(state.clone(), require_auth))` on the subroutes that need it.
|
||||
|
||||
## Testing handlers
|
||||
|
||||
```rust
|
||||
// tests/users.rs
|
||||
#[tokio::test]
|
||||
async fn creates_user() {
|
||||
let pool = test_db().await; // helper that spins up a transactional DB
|
||||
let state = AppState::new_test(pool).await.unwrap();
|
||||
let app = my_app::routes::router(state);
|
||||
|
||||
let request = axum::http::Request::builder()
|
||||
.uri("/api/v1/users")
|
||||
.method("POST")
|
||||
.header("content-type", "application/json")
|
||||
.body(axum::body::Body::from(
|
||||
serde_json::to_vec(&serde_json::json!({"email": "a@b.com", "name": "A"})).unwrap()
|
||||
)).unwrap();
|
||||
|
||||
let response = tower::ServiceExt::oneshot(app, request).await.unwrap();
|
||||
assert_eq!(response.status(), 201);
|
||||
let bytes = axum::body::to_bytes(response.into_body(), 1 << 20).await.unwrap();
|
||||
let user: serde_json::Value = serde_json::from_slice(&bytes).unwrap();
|
||||
assert_eq!(user["email"], "a@b.com");
|
||||
}
|
||||
```
|
||||
|
||||
`tower::ServiceExt::oneshot` calls the router directly without binding a socket. Tests run in parallel without port collisions.
|
||||
|
||||
## Config
|
||||
|
||||
```rust
|
||||
// src/config.rs
|
||||
use secrecy::{Secret, ExposeSecret};
|
||||
|
||||
#[derive(Debug, Clone, serde::Deserialize)]
|
||||
pub struct Settings {
|
||||
pub bind: String,
|
||||
pub db: Database,
|
||||
pub jwt_secret: Secret<String>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, serde::Deserialize)]
|
||||
pub struct Database {
|
||||
pub url: Secret<String>,
|
||||
pub max_connections: u32,
|
||||
}
|
||||
|
||||
impl Settings {
|
||||
pub fn load() -> anyhow::Result<Self> {
|
||||
let cfg = config::Config::builder()
|
||||
.add_source(config::File::with_name("config/default").required(false))
|
||||
.add_source(config::File::with_name(&format!(
|
||||
"config/{}", std::env::var("APP_ENV").unwrap_or_else(|_| "dev".into())
|
||||
)).required(false))
|
||||
.add_source(config::Environment::with_prefix("APP").separator("__"))
|
||||
.build()?;
|
||||
Ok(cfg.try_deserialize()?)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`Secret<T>` from the `secrecy` crate hides the value in `Debug`/`Display` to prevent accidental log leakage. Access via `.expose_secret()` only where needed.
|
||||
|
||||
## OpenAPI (optional)
|
||||
|
||||
Add `utoipa` derive macros on your DTOs and handlers, mount Swagger UI at `/swagger-ui`:
|
||||
|
||||
```rust
|
||||
use utoipa::OpenApi;
|
||||
use utoipa_axum::router::OpenApiRouter;
|
||||
use utoipa_swagger_ui::SwaggerUi;
|
||||
|
||||
#[derive(OpenApi)]
|
||||
#[openapi(
|
||||
paths(routes::users::create_user, routes::users::get_user),
|
||||
components(schemas(routes::users::User, routes::users::CreateUser))
|
||||
)]
|
||||
struct ApiDoc;
|
||||
|
||||
let (router, api) = OpenApiRouter::with_openapi(ApiDoc::openapi())
|
||||
.routes(utoipa_axum::routes!(routes::users::create_user, routes::users::get_user))
|
||||
.split_for_parts();
|
||||
|
||||
let app = router.merge(SwaggerUi::new("/swagger-ui").url("/api-docs/openapi.json", api));
|
||||
```
|
||||
|
||||
## Production checklist
|
||||
|
||||
- Bind to `0.0.0.0` in containers, `127.0.0.1` for local-only services.
|
||||
- Set `RUST_LOG=info,sqlx=warn` (or use `EnvFilter` defaults as shown).
|
||||
- Send logs to stdout in JSON. Ingest via Vector / Fluent Bit / Loki.
|
||||
- Run migrations on startup (`sqlx::migrate!` block). Fail fast on schema mismatch.
|
||||
- Health endpoint **must hit the DB** (so load balancers know if the pool is dead).
|
||||
- Add `tower::limit::RateLimitLayer` or token-bucket middleware for public endpoints.
|
||||
- Set `tower_http::limit::RequestBodyLimitLayer` to bound request size.
|
||||
- Compress with brotli + gzip via `CompressionLayer`.
|
||||
- Tighten CORS, do not ship `CorsLayer::permissive()` to production.
|
||||
- Strip sensitive headers from traces via `SetSensitiveHeadersLayer`.
|
||||
- Set up SIGTERM-driven `with_graceful_shutdown` so deploys roll without dropping requests.
|
||||
- Containerize with `cargo chef` for incremental Docker builds.
|
||||
|
||||
## Common mistakes
|
||||
|
||||
1. **Forgetting `error_for_status()?` on outbound `reqwest`** — 4xx silently succeeds.
|
||||
2. **Returning `Result<T, sqlx::Error>` from handlers** — leak DB details to clients. Always go through `AppError`.
|
||||
3. **`Json<T>` extractor before validation** — invalid JSON returns axum's default 422 with no body shape. Wrap in a `ValidatedJson<T>` extractor that runs `validator` and returns `AppError`.
|
||||
4. **Holding DB connections across `.await` on slow external calls** — exhausts the pool. Acquire late, release early.
|
||||
5. **Skipping `tracing::instrument`** on handlers — losing per-request span correlation.
|
||||
6. **No `RequestBodyLimitLayer`** — DoS surface. Default axum has no limit.
|
||||
@@ -0,0 +1,317 @@
|
||||
# Cargo Strict Configuration
|
||||
|
||||
The exact knobs every new Rust project gets. Drop these in unmodified.
|
||||
|
||||
## `rust-toolchain.toml`
|
||||
|
||||
```toml
|
||||
[toolchain]
|
||||
channel = "stable"
|
||||
components = ["rustfmt", "clippy", "rust-src"]
|
||||
profile = "default"
|
||||
```
|
||||
|
||||
Pin nightly separately when miri runs:
|
||||
|
||||
```bash
|
||||
rustup install nightly
|
||||
rustup component add miri rust-src --toolchain nightly
|
||||
```
|
||||
|
||||
## `Cargo.toml` — `[lints]` section
|
||||
|
||||
```toml
|
||||
[lints.rust]
|
||||
unsafe_op_in_unsafe_fn = "deny"
|
||||
missing_docs = "warn"
|
||||
missing_debug_implementations = "warn"
|
||||
unreachable_pub = "warn"
|
||||
unused_must_use = "deny"
|
||||
elided_lifetimes_in_paths = "warn"
|
||||
non_ascii_idents = "deny"
|
||||
trivial_numeric_casts = "warn"
|
||||
unused_lifetimes = "warn"
|
||||
single_use_lifetimes = "warn"
|
||||
|
||||
[lints.clippy]
|
||||
# Groups
|
||||
all = { level = "deny", priority = -1 }
|
||||
pedantic = { level = "warn", priority = -1 }
|
||||
nursery = { level = "warn", priority = -1 }
|
||||
cargo = { level = "warn", priority = -1 }
|
||||
|
||||
# Hard denies - turn warnings into errors for sharp tools
|
||||
undocumented_unsafe_blocks = "deny"
|
||||
multiple_unsafe_ops_per_block = "deny"
|
||||
unwrap_used = "deny"
|
||||
expect_used = "deny"
|
||||
panic = "deny"
|
||||
todo = "deny"
|
||||
unimplemented = "deny"
|
||||
unreachable = "deny"
|
||||
indexing_slicing = "deny"
|
||||
mem_forget = "deny"
|
||||
arithmetic_side_effects = "warn"
|
||||
cast_possible_truncation = "warn"
|
||||
cast_possible_wrap = "warn"
|
||||
cast_precision_loss = "warn"
|
||||
cast_sign_loss = "warn"
|
||||
as_underscore = "deny"
|
||||
as_ptr_cast_mut = "deny"
|
||||
ptr_as_ptr = "warn"
|
||||
borrow_as_ptr = "warn"
|
||||
fn_to_numeric_cast_any = "deny"
|
||||
clone_on_ref_ptr = "warn"
|
||||
mutex_atomic = "warn"
|
||||
rc_buffer = "warn"
|
||||
rc_mutex = "warn"
|
||||
exit = "warn"
|
||||
allow_attributes_without_reason = "warn"
|
||||
dbg_macro = "warn"
|
||||
print_stderr = "warn"
|
||||
print_stdout = "warn"
|
||||
use_debug = "warn"
|
||||
|
||||
# Stylistic relaxations (project-wide opinions only)
|
||||
module_name_repetitions = "allow"
|
||||
must_use_candidate = "allow"
|
||||
missing_errors_doc = "allow" # we use anyhow::Result with .context() everywhere; doc rule is noisy
|
||||
|
||||
# Restriction lints - opt-in soundness rails
|
||||
unreachable = "deny"
|
||||
mod_module_files = "warn" # prefer foo.rs over foo/mod.rs
|
||||
empty_drop = "warn"
|
||||
empty_structs_with_brackets = "warn"
|
||||
empty_enum = "warn"
|
||||
exhaustive_enums = "warn" # public enums should consider #[non_exhaustive]
|
||||
exhaustive_structs = "warn"
|
||||
```
|
||||
|
||||
The `priority = -1` trick: group-level levels are weak; specific lints below them win. This lets us deny `unwrap_used` while still allowing `pedantic` group warnings instead of denies.
|
||||
|
||||
## `Cargo.toml` — release profile
|
||||
|
||||
```toml
|
||||
[profile.release]
|
||||
opt-level = 3
|
||||
lto = "fat"
|
||||
codegen-units = 1
|
||||
strip = "symbols"
|
||||
panic = "abort" # smaller, faster - if you need unwinding (FFI catch), set "unwind"
|
||||
debug = "line-tables-only"
|
||||
|
||||
[profile.dev]
|
||||
opt-level = 0
|
||||
debug = true
|
||||
incremental = true
|
||||
codegen-units = 256
|
||||
split-debuginfo = "unpacked"
|
||||
|
||||
# A profile for miri - opt-level 1 keeps simulation bearable while still
|
||||
# exercising real codegen patterns. miri ignores most profile keys but reads
|
||||
# overflow-checks.
|
||||
[profile.miri]
|
||||
inherits = "test"
|
||||
opt-level = 1
|
||||
overflow-checks = true
|
||||
```
|
||||
|
||||
## `Cargo.toml` — workspace level
|
||||
|
||||
```toml
|
||||
[workspace]
|
||||
resolver = "3"
|
||||
|
||||
[workspace.package]
|
||||
edition = "2024"
|
||||
rust-version = "1.83" # bump only when a needed feature lands
|
||||
license = "Apache-2.0 OR MIT"
|
||||
|
||||
[workspace.lints]
|
||||
# Then in each member crate:
|
||||
# [lints]
|
||||
# workspace = true
|
||||
```
|
||||
|
||||
## `rustfmt.toml`
|
||||
|
||||
```toml
|
||||
edition = "2024"
|
||||
max_width = 100
|
||||
imports_granularity = "Module"
|
||||
group_imports = "StdExternalCrate"
|
||||
reorder_imports = true
|
||||
reorder_modules = true
|
||||
newline_style = "Unix"
|
||||
use_field_init_shorthand = true
|
||||
use_try_shorthand = true
|
||||
unstable_features = false
|
||||
```
|
||||
|
||||
Most options come from stable rustfmt. `imports_granularity` and `group_imports` are nightly-only but ignored cleanly on stable; CI runs `cargo +nightly fmt --check` for the import grouping.
|
||||
|
||||
## `clippy.toml`
|
||||
|
||||
```toml
|
||||
# Reduce cognitive load thresholds.
|
||||
cognitive-complexity-threshold = 25
|
||||
type-complexity-threshold = 250
|
||||
too-many-arguments-threshold = 6
|
||||
too-many-lines-threshold = 100
|
||||
|
||||
# msrv - keeps clippy from suggesting features past our MSRV
|
||||
msrv = "1.83"
|
||||
|
||||
# Avoid `panic` lint complaining about derived Debug impls calling unreachable_unchecked etc.
|
||||
allow-unwrap-in-tests = true
|
||||
allow-expect-in-tests = true
|
||||
allow-panic-in-tests = true
|
||||
allow-dbg-in-tests = true
|
||||
allow-print-in-tests = true
|
||||
|
||||
# Force named arguments above N params
|
||||
single-char-binding-names-threshold = 4
|
||||
```
|
||||
|
||||
## `deny.toml` (cargo-deny)
|
||||
|
||||
```toml
|
||||
[advisories]
|
||||
db-path = "~/.cargo/advisory-db"
|
||||
db-urls = ["https://github.com/rustsec/advisory-db"]
|
||||
yanked = "deny"
|
||||
ignore = []
|
||||
|
||||
[licenses]
|
||||
allow = [
|
||||
"MIT",
|
||||
"Apache-2.0",
|
||||
"Apache-2.0 WITH LLVM-exception",
|
||||
"BSD-2-Clause",
|
||||
"BSD-3-Clause",
|
||||
"ISC",
|
||||
"Unicode-DFS-2016",
|
||||
"Unicode-3.0",
|
||||
"Zlib",
|
||||
"MPL-2.0",
|
||||
"CC0-1.0",
|
||||
]
|
||||
confidence-threshold = 0.93
|
||||
exceptions = []
|
||||
|
||||
[bans]
|
||||
multiple-versions = "warn"
|
||||
wildcards = "deny"
|
||||
highlight = "all"
|
||||
deny = [
|
||||
# Pin out unmaintained alternatives
|
||||
{ name = "async-std", reason = "use tokio" },
|
||||
{ name = "actix-web", reason = "use axum" },
|
||||
{ name = "chrono", reason = "use jiff" },
|
||||
]
|
||||
|
||||
[sources]
|
||||
unknown-registry = "deny"
|
||||
unknown-git = "warn"
|
||||
allow-registry = ["https://github.com/rust-lang/crates.io-index"]
|
||||
```
|
||||
|
||||
## CI Workflow (`.github/workflows/ci.yml`)
|
||||
|
||||
```yaml
|
||||
name: ci
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
pull_request:
|
||||
|
||||
env:
|
||||
CARGO_TERM_COLOR: always
|
||||
RUST_BACKTRACE: 1
|
||||
|
||||
jobs:
|
||||
fmt:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: dtolnay/rust-toolchain@nightly
|
||||
with:
|
||||
components: rustfmt
|
||||
- run: cargo +nightly fmt --all -- --check
|
||||
|
||||
clippy:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: dtolnay/rust-toolchain@stable
|
||||
with:
|
||||
components: clippy
|
||||
- uses: Swatinem/rust-cache@v2
|
||||
- run: cargo clippy --all-targets --all-features --workspace -- -D warnings
|
||||
|
||||
test:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: dtolnay/rust-toolchain@stable
|
||||
- uses: Swatinem/rust-cache@v2
|
||||
- uses: taiki-e/install-action@nextest
|
||||
- run: cargo nextest run --all-targets --all-features --workspace
|
||||
|
||||
miri:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: dtolnay/rust-toolchain@nightly
|
||||
with:
|
||||
components: miri, rust-src
|
||||
- uses: Swatinem/rust-cache@v2
|
||||
- uses: taiki-e/install-action@nextest
|
||||
- env:
|
||||
MIRIFLAGS: "-Zmiri-strict-provenance -Zmiri-symbolic-alignment-check"
|
||||
run: cargo +nightly miri nextest run --all-features --workspace
|
||||
|
||||
machete:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: dtolnay/rust-toolchain@stable
|
||||
- uses: bnjbvr/cargo-machete@main
|
||||
|
||||
deny:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: EmbarkStudios/cargo-deny-action@v2
|
||||
with:
|
||||
command: check all
|
||||
|
||||
audit:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: rustsec/audit-check@v1
|
||||
with:
|
||||
token: ${{ secrets.GITHUB_TOKEN }}
|
||||
```
|
||||
|
||||
## Project bootstrap
|
||||
|
||||
```bash
|
||||
cargo new --bin my-app --edition 2024
|
||||
cd my-app
|
||||
cargo install cargo-nextest cargo-machete cargo-deny cargo-edit cargo-watch
|
||||
rustup install nightly
|
||||
rustup component add miri --toolchain nightly
|
||||
# drop the configs above
|
||||
git add . && git commit -m "chore: bootstrap strict toolchain"
|
||||
```
|
||||
|
||||
After every change:
|
||||
|
||||
```bash
|
||||
cargo fmt --all -- --check && \
|
||||
cargo clippy --all-targets --all-features -- -D warnings && \
|
||||
cargo nextest run && \
|
||||
cargo +nightly miri nextest run # only if unsafe is involved
|
||||
```
|
||||
@@ -0,0 +1,409 @@
|
||||
# CLI Stack — clap + color-eyre + tracing + indicatif + dialoguer
|
||||
|
||||
The default for any new CLI tool. Strict typing on arguments, beautiful errors, progress feedback, interactive prompts when needed.
|
||||
|
||||
## Cargo.toml
|
||||
|
||||
```toml
|
||||
[package]
|
||||
name = "mytool"
|
||||
version = "0.1.0"
|
||||
edition = "2024"
|
||||
|
||||
[dependencies]
|
||||
clap = { version = "4", features = ["derive", "env", "wrap_help", "color", "unicode"] }
|
||||
clap_complete = "4"
|
||||
color-eyre = "0.6"
|
||||
tracing = "0.1"
|
||||
tracing-subscriber = { version = "0.3", features = ["env-filter", "fmt"] }
|
||||
anyhow = "1"
|
||||
indicatif = { version = "0.17", features = ["tokio"] }
|
||||
dialoguer = { version = "0.11", features = ["fuzzy-select"] }
|
||||
console = "0.15"
|
||||
tokio = { version = "1", features = ["macros", "rt-multi-thread", "fs", "process", "signal"] }
|
||||
|
||||
[profile.release]
|
||||
opt-level = 3
|
||||
lto = "fat"
|
||||
codegen-units = 1
|
||||
strip = "symbols"
|
||||
panic = "abort"
|
||||
```
|
||||
|
||||
## Command structure
|
||||
|
||||
```rust
|
||||
// src/cli.rs
|
||||
use clap::{Parser, Subcommand, ValueEnum};
|
||||
use std::path::PathBuf;
|
||||
|
||||
#[derive(Debug, Parser)]
|
||||
#[command(
|
||||
name = "mytool",
|
||||
author,
|
||||
version,
|
||||
about = "A short description",
|
||||
long_about = "A longer description that appears in --help",
|
||||
arg_required_else_help = true,
|
||||
)]
|
||||
pub struct Cli {
|
||||
/// Configuration file path
|
||||
#[arg(short, long, env = "MYTOOL_CONFIG", default_value = "config.toml", global = true)]
|
||||
pub config: PathBuf,
|
||||
|
||||
/// Increase verbosity (-v info, -vv debug, -vvv trace)
|
||||
#[arg(short, long, action = clap::ArgAction::Count, global = true)]
|
||||
pub verbose: u8,
|
||||
|
||||
/// Suppress all non-error output
|
||||
#[arg(short, long, global = true, conflicts_with = "verbose")]
|
||||
pub quiet: bool,
|
||||
|
||||
/// Force colored output even when stdout is not a terminal
|
||||
#[arg(long, global = true, value_enum, default_value_t = ColorChoice::Auto)]
|
||||
pub color: ColorChoice,
|
||||
|
||||
/// Output format
|
||||
#[arg(short, long, global = true, value_enum, default_value_t = OutputFormat::Pretty)]
|
||||
pub format: OutputFormat,
|
||||
|
||||
#[command(subcommand)]
|
||||
pub command: Command,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, ValueEnum)]
|
||||
pub enum ColorChoice { Auto, Always, Never }
|
||||
|
||||
#[derive(Debug, Clone, ValueEnum)]
|
||||
pub enum OutputFormat { Pretty, Json, Plain }
|
||||
|
||||
#[derive(Debug, Subcommand)]
|
||||
pub enum Command {
|
||||
/// Build the thing
|
||||
Build(BuildArgs),
|
||||
/// Watch and rebuild
|
||||
Watch(WatchArgs),
|
||||
/// Generate shell completions
|
||||
Completions { #[arg(value_enum)] shell: clap_complete::Shell },
|
||||
}
|
||||
|
||||
#[derive(Debug, clap::Args)]
|
||||
pub struct BuildArgs {
|
||||
/// Target directory
|
||||
#[arg(short, long, default_value = "target")]
|
||||
pub target: PathBuf,
|
||||
/// Build mode
|
||||
#[arg(short, long, value_enum, default_value_t = Mode::Release)]
|
||||
pub mode: Mode,
|
||||
/// Specific files to build (default: all)
|
||||
pub files: Vec<PathBuf>,
|
||||
}
|
||||
|
||||
#[derive(Debug, clap::Args)]
|
||||
pub struct WatchArgs {
|
||||
/// Glob pattern to watch
|
||||
#[arg(short, long, default_value = "**/*.rs")]
|
||||
pub pattern: String,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, ValueEnum)]
|
||||
pub enum Mode { Debug, Release }
|
||||
```
|
||||
|
||||
Key clap derive patterns:
|
||||
|
||||
- `env = "VAR"` — falls back to env var if flag not given.
|
||||
- `global = true` — flag inherits to subcommands.
|
||||
- `arg_required_else_help = true` — running with no args prints help instead of erroring.
|
||||
- `value_enum` on an enum — case-insensitive parsing + auto-completion.
|
||||
- `action = clap::ArgAction::Count` — `-v` is 1, `-vv` is 2, etc.
|
||||
- `conflicts_with` — incompatible flags.
|
||||
|
||||
## Main + tracing init
|
||||
|
||||
```rust
|
||||
// src/main.rs
|
||||
use clap::Parser;
|
||||
use mytool::cli::{Cli, Command, ColorChoice};
|
||||
use tracing::Level;
|
||||
use tracing_subscriber::EnvFilter;
|
||||
|
||||
fn main() -> color_eyre::Result<()> {
|
||||
color_eyre::install()?;
|
||||
let cli = Cli::parse();
|
||||
init_tracing(&cli);
|
||||
|
||||
if matches!(cli.color, ColorChoice::Always) {
|
||||
console::set_colors_enabled(true);
|
||||
} else if matches!(cli.color, ColorChoice::Never) {
|
||||
console::set_colors_enabled(false);
|
||||
}
|
||||
|
||||
match cli.command {
|
||||
Command::Build(args) => mytool::commands::build::run(&cli, args),
|
||||
Command::Watch(args) => mytool::commands::watch::run(&cli, args),
|
||||
Command::Completions { shell } => {
|
||||
let mut cmd = <Cli as clap::CommandFactory>::command();
|
||||
clap_complete::generate(shell, &mut cmd, "mytool", &mut std::io::stdout());
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn init_tracing(cli: &Cli) {
|
||||
let level = if cli.quiet {
|
||||
Level::ERROR
|
||||
} else {
|
||||
match cli.verbose {
|
||||
0 => Level::WARN,
|
||||
1 => Level::INFO,
|
||||
2 => Level::DEBUG,
|
||||
_ => Level::TRACE,
|
||||
}
|
||||
};
|
||||
let filter = EnvFilter::try_from_default_env()
|
||||
.unwrap_or_else(|_| EnvFilter::new(format!("mytool={level}")));
|
||||
tracing_subscriber::fmt()
|
||||
.with_env_filter(filter)
|
||||
.with_target(false)
|
||||
.without_time()
|
||||
.compact()
|
||||
.with_writer(std::io::stderr)
|
||||
.init();
|
||||
}
|
||||
```
|
||||
|
||||
Tracing on a CLI:
|
||||
- **Write to stderr.** stdout is for the tool's actual output (which the user might pipe). Logs and progress bars go to stderr.
|
||||
- **Verbosity from `-v`, not from `RUST_LOG`.** Users expect `-v` on a CLI; `RUST_LOG` is a developer escape hatch (kept, but secondary).
|
||||
|
||||
## Progress bars — `indicatif`
|
||||
|
||||
```rust
|
||||
use indicatif::{MultiProgress, ProgressBar, ProgressStyle};
|
||||
use std::time::Duration;
|
||||
|
||||
let mp = MultiProgress::new();
|
||||
let pb = mp.add(ProgressBar::new(files.len() as u64));
|
||||
pb.set_style(
|
||||
ProgressStyle::with_template(
|
||||
"{spinner:.green} [{elapsed_precise}] [{wide_bar:.cyan/blue}] {pos}/{len} ({eta}) {msg}"
|
||||
)?
|
||||
.progress_chars("=>-")
|
||||
);
|
||||
|
||||
for file in files {
|
||||
pb.set_message(file.display().to_string());
|
||||
process(&file)?;
|
||||
pb.inc(1);
|
||||
}
|
||||
pb.finish_with_message("done");
|
||||
```
|
||||
|
||||
For unbounded operations:
|
||||
|
||||
```rust
|
||||
let spinner = ProgressBar::new_spinner();
|
||||
spinner.enable_steady_tick(Duration::from_millis(80));
|
||||
spinner.set_message("connecting…");
|
||||
let result = connect().await?;
|
||||
spinner.finish_and_clear();
|
||||
```
|
||||
|
||||
With multiple parallel tasks:
|
||||
|
||||
```rust
|
||||
let mp = MultiProgress::new();
|
||||
let bars: Vec<_> = (0..workers).map(|i| {
|
||||
let pb = mp.add(ProgressBar::new(unit));
|
||||
pb.set_style(ProgressStyle::with_template("worker {prefix}: {pos}/{len}")?);
|
||||
pb.set_prefix(i.to_string());
|
||||
pb
|
||||
}).collect();
|
||||
```
|
||||
|
||||
`MultiProgress` keeps bars stacked and redraws cleanly even with concurrent updates from multiple tasks.
|
||||
|
||||
When stdout is not a terminal, indicatif silently disables animation. Force on/off with `pb.set_draw_target(ProgressDrawTarget::stdout())` / `hidden()`.
|
||||
|
||||
## Interactive prompts — `dialoguer`
|
||||
|
||||
```rust
|
||||
use dialoguer::{theme::ColorfulTheme, Confirm, Input, Password, Select, FuzzySelect, MultiSelect};
|
||||
|
||||
let name: String = Input::with_theme(&ColorfulTheme::default())
|
||||
.with_prompt("Project name")
|
||||
.validate_with(|input: &String| -> Result<(), &str> {
|
||||
if input.chars().all(|c| c.is_alphanumeric() || c == '-' || c == '_') {
|
||||
Ok(())
|
||||
} else {
|
||||
Err("alphanumeric, dash, underscore only")
|
||||
}
|
||||
})
|
||||
.interact_text()?;
|
||||
|
||||
let secret = Password::with_theme(&ColorfulTheme::default())
|
||||
.with_prompt("API key")
|
||||
.with_confirmation("Repeat", "passwords don't match")
|
||||
.interact()?;
|
||||
|
||||
let go: bool = Confirm::with_theme(&ColorfulTheme::default())
|
||||
.with_prompt(format!("Delete {}? This cannot be undone.", path.display()))
|
||||
.default(false)
|
||||
.interact()?;
|
||||
if !go { return Ok(()); }
|
||||
|
||||
let items = ["yes", "no", "maybe"];
|
||||
let idx = Select::with_theme(&ColorfulTheme::default())
|
||||
.with_prompt("Pick one")
|
||||
.items(&items)
|
||||
.default(0)
|
||||
.interact()?;
|
||||
|
||||
let picks = MultiSelect::with_theme(&ColorfulTheme::default())
|
||||
.with_prompt("Toggle features")
|
||||
.items(&["alpha", "beta", "gamma"])
|
||||
.defaults(&[true, false, false])
|
||||
.interact()?;
|
||||
```
|
||||
|
||||
Detect non-TTY before prompting:
|
||||
|
||||
```rust
|
||||
if !console::user_attended() {
|
||||
return Err(anyhow::anyhow!("input required but stdin is not a terminal"));
|
||||
}
|
||||
```
|
||||
|
||||
For automated tests, expose a `--non-interactive` flag and gate all prompts behind it.
|
||||
|
||||
## Structured output
|
||||
|
||||
```rust
|
||||
match cli.format {
|
||||
OutputFormat::Json => {
|
||||
serde_json::to_writer(std::io::stdout().lock(), &result)?;
|
||||
println!();
|
||||
}
|
||||
OutputFormat::Plain => {
|
||||
for row in &result.rows {
|
||||
println!("{}\t{}\t{}", row.a, row.b, row.c);
|
||||
}
|
||||
}
|
||||
OutputFormat::Pretty => {
|
||||
use console::{style, Term};
|
||||
let term = Term::stdout();
|
||||
for row in &result.rows {
|
||||
term.write_line(&format!(
|
||||
"{} {} {}",
|
||||
style(&row.a).green(),
|
||||
style(&row.b).yellow(),
|
||||
style(&row.c).dim(),
|
||||
))?;
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Always offer `--format json` for piping into `jq`, scripts, and other tools.
|
||||
|
||||
## Shell completions
|
||||
|
||||
Already shown in the `Completions` subcommand above. Distribute completions by adding to the install script:
|
||||
|
||||
```bash
|
||||
mytool completions bash > /etc/bash_completion.d/mytool
|
||||
mytool completions fish > ~/.config/fish/completions/mytool.fish
|
||||
mytool completions zsh > "${fpath[1]}/_mytool"
|
||||
```
|
||||
|
||||
## Signal handling
|
||||
|
||||
```rust
|
||||
// In an async CLI command
|
||||
use tokio::signal::ctrl_c;
|
||||
|
||||
tokio::select! {
|
||||
_ = ctrl_c() => {
|
||||
tracing::warn!("interrupted, cleaning up");
|
||||
cleanup().await?;
|
||||
std::process::exit(130); // standard exit code for SIGINT
|
||||
}
|
||||
result = long_running_task() => {
|
||||
result
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
For sync CLIs, install a one-shot handler with `ctrlc` crate:
|
||||
|
||||
```rust
|
||||
let interrupted = std::sync::Arc::new(std::sync::atomic::AtomicBool::new(false));
|
||||
let i = interrupted.clone();
|
||||
ctrlc::set_handler(move || i.store(true, std::sync::atomic::Ordering::SeqCst))?;
|
||||
|
||||
while !interrupted.load(std::sync::atomic::Ordering::Relaxed) {
|
||||
do_step()?;
|
||||
}
|
||||
```
|
||||
|
||||
## Error reporting with color-eyre
|
||||
|
||||
```rust
|
||||
fn main() -> color_eyre::Result<()> {
|
||||
color_eyre::config::HookBuilder::default()
|
||||
.display_env_section(false) // hide SPANTRACE/BACKTRACE env hints by default
|
||||
.display_location_section(false) // hide file:line section
|
||||
.panic_section("If this is a bug, please report at https://github.com/me/mytool/issues")
|
||||
.install()?;
|
||||
real_main()
|
||||
}
|
||||
```
|
||||
|
||||
Errors with `.wrap_err("...")` from `eyre::WrapErr` (compatible with anyhow's `.context`) show as a numbered chain. `RUST_BACKTRACE=1` shows the full trace; `RUST_SPANTRACE=1` shows tracing spans where the error fired.
|
||||
|
||||
## Distribution
|
||||
|
||||
- Add `cargo dist init` for prebuilt binary release pipeline (cross-platform tarballs + installers).
|
||||
- Publish to Homebrew tap, AUR, scoop, Chocolatey via dist.
|
||||
- Sign Linux binaries with `cosign` if your audience is enterprise.
|
||||
- Build single static binary on Linux with `--target x86_64-unknown-linux-musl` (or `aarch64-unknown-linux-musl`).
|
||||
- For wasm-runnable CLIs (`wasi-cli`), add `--target wasm32-wasip1`.
|
||||
|
||||
## Common mistakes
|
||||
|
||||
1. **Mixing stdout and stderr.** Tool output goes to stdout; logs and progress go to stderr.
|
||||
2. **No `--non-interactive` flag.** Interactive prompts block automation.
|
||||
3. **Printing colored output unconditionally.** Honor `NO_COLOR` env var, detect TTY with `console::user_attended()`.
|
||||
4. **`println!` for errors.** Use `tracing::error!` so logs go to stderr automatically and respect verbosity.
|
||||
5. **`unwrap()` on `Cli::parse()`.** clap returns clean errors with `--help` text; `parse()` exits on its own.
|
||||
6. **Long subcommand handlers in `main.rs`.** Split into `src/commands/<name>.rs` per command.
|
||||
7. **Missing exit code semantics.** Use `std::process::exit(1)` (general error), `2` (usage), `130` (SIGINT) appropriately. Or return `Result` and let main map.
|
||||
|
||||
## Testing CLIs
|
||||
|
||||
```rust
|
||||
// tests/cli.rs
|
||||
use assert_cmd::Command;
|
||||
use predicates::prelude::*;
|
||||
|
||||
#[test]
|
||||
fn shows_help() {
|
||||
Command::cargo_bin("mytool").unwrap()
|
||||
.arg("--help")
|
||||
.assert()
|
||||
.success()
|
||||
.stdout(predicate::str::contains("Usage:"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rejects_unknown_subcommand() {
|
||||
Command::cargo_bin("mytool").unwrap()
|
||||
.arg("nope")
|
||||
.assert()
|
||||
.failure()
|
||||
.stderr(predicate::str::contains("unrecognized subcommand"));
|
||||
}
|
||||
```
|
||||
|
||||
`assert_cmd` builds the binary once per test run and gives a fluent assertion API.
|
||||
@@ -0,0 +1,375 @@
|
||||
# Concurrency Primitives
|
||||
|
||||
Locks, atomics, channels, and the loom model checker. The decision tree that keeps the agent out of soundness trouble.
|
||||
|
||||
## The pyramid
|
||||
|
||||
```
|
||||
Highest level tokio::sync::mpsc / broadcast / watch
|
||||
(message passing — default for new code)
|
||||
|
||||
Arc<Mutex<T>> / Arc<RwLock<T>>
|
||||
(shared mutable state — common, easy to get right)
|
||||
|
||||
parking_lot::{Mutex, RwLock, Condvar}
|
||||
(faster sync locks, no poisoning)
|
||||
|
||||
Atomics (AtomicUsize, AtomicBool, AtomicPtr)
|
||||
(single-word lock-free state)
|
||||
|
||||
Lowest level UnsafeCell + unsafe + loom + miri
|
||||
(custom lock-free / wait-free primitives)
|
||||
```
|
||||
|
||||
**Always start at the top.** Drop a level only when you have measured a real bottleneck.
|
||||
|
||||
## Decision tree
|
||||
|
||||
```
|
||||
Need to share state between tasks?
|
||||
├── State is configuration (read-only after start)
|
||||
│ └── Arc<Config> (no lock needed)
|
||||
├── State is a queue of work
|
||||
│ └── tokio::sync::mpsc::channel(cap)
|
||||
├── State is "latest value" published to many readers
|
||||
│ └── tokio::sync::watch::channel(initial)
|
||||
├── State is broadcast (every consumer sees every value)
|
||||
│ └── tokio::sync::broadcast::channel(cap)
|
||||
├── State is request-response within one task tree
|
||||
│ └── tokio::sync::oneshot::channel()
|
||||
├── State is a counter
|
||||
│ └── AtomicU64 (or AtomicUsize)
|
||||
├── State is a flag / set-once
|
||||
│ └── AtomicBool / OnceLock<T> / OnceCell<T>
|
||||
├── State needs mutation across many tasks/threads, cheap critical sections
|
||||
│ ├── async context → tokio::sync::Mutex<T>
|
||||
│ └── sync context (no .await held) → parking_lot::Mutex<T>
|
||||
├── State needs mutation, many readers, few writers
|
||||
│ ├── async context → tokio::sync::RwLock<T>
|
||||
│ └── sync context → parking_lot::RwLock<T>
|
||||
└── State is a custom lock-free primitive (channels, hazard pointers)
|
||||
└── UnsafeCell + atomics + loom-tested + miri-tested + a co-author
|
||||
```
|
||||
|
||||
## Atomics — when and how
|
||||
|
||||
Use atomics for:
|
||||
- Counters incremented from many threads (`AtomicU64`).
|
||||
- Single-shot flags (`AtomicBool`).
|
||||
- Pointer publication (`AtomicPtr<T>`).
|
||||
|
||||
### Memory orderings
|
||||
|
||||
```rust
|
||||
use std::sync::atomic::{AtomicUsize, Ordering};
|
||||
|
||||
let c = AtomicUsize::new(0);
|
||||
|
||||
// Just need a count, no synchronization with other data
|
||||
c.fetch_add(1, Ordering::Relaxed);
|
||||
|
||||
// Reading a counter that was incremented from elsewhere
|
||||
let n = c.load(Ordering::Relaxed);
|
||||
```
|
||||
|
||||
| Ordering | When |
|
||||
|---|---|
|
||||
| `Relaxed` | Standalone counters, no other memory needs to be synchronized. |
|
||||
| `Acquire` (loads) / `Release` (stores) | Publish/consume pattern: you write some data then release a flag, readers acquire the flag then read the data. |
|
||||
| `AcqRel` | RMW that both reads-and-publishes (e.g., `fetch_add` on a sequence number). |
|
||||
| `SeqCst` | Total ordering across all `SeqCst` ops. Strongest, slowest. Use when in doubt and switch to a weaker ordering after testing under loom. |
|
||||
|
||||
**Default to `SeqCst` if unsure.** Performance difference is usually negligible. Going weaker requires loom.
|
||||
|
||||
### Publish-then-load pattern
|
||||
|
||||
```rust
|
||||
static READY: AtomicBool = AtomicBool::new(false);
|
||||
static mut DATA: Option<Config> = None;
|
||||
|
||||
// Producer thread:
|
||||
unsafe { DATA = Some(load_config()); }
|
||||
READY.store(true, Ordering::Release);
|
||||
|
||||
// Consumer thread:
|
||||
if READY.load(Ordering::Acquire) {
|
||||
// SAFETY: producer's Release pairs with our Acquire; if we see READY=true,
|
||||
// we are guaranteed to also see the DATA write that happened-before it.
|
||||
let cfg = unsafe { DATA.as_ref().unwrap() };
|
||||
}
|
||||
```
|
||||
|
||||
This is the canonical Release/Acquire pattern. **Use `OnceLock<Config>` instead** in new code — it encapsulates exactly this with safe API.
|
||||
|
||||
## Std vs parking_lot vs tokio for locks
|
||||
|
||||
| | std::sync::Mutex | parking_lot::Mutex | tokio::sync::Mutex |
|
||||
|---|---|---|---|
|
||||
| Speed | Slowest (OS futex direct) | Fastest (smarter parking) | Slow (await-aware) |
|
||||
| Poisoning | Yes (`PoisonError`) | No | No |
|
||||
| Hold across `.await` | Dangerous (deadlock under current-thread runtime) | Dangerous | Safe |
|
||||
| Drop guard releases | Yes | Yes | Yes |
|
||||
| RAII | Yes (`MutexGuard`) | Yes | Yes |
|
||||
| Const constructor | Yes (since 1.63) | Yes | No |
|
||||
| Async | No | No | Yes |
|
||||
|
||||
**Rule of thumb:**
|
||||
|
||||
- Hot, short critical section, no await inside → `parking_lot::Mutex`.
|
||||
- Shared state held across `.await` → `tokio::sync::Mutex`.
|
||||
- Static init / app config → `OnceLock` or `LazyLock`.
|
||||
- Avoid `std::sync::Mutex` for new code; the poisoning behavior is more annoying than useful and `parking_lot` is strictly faster.
|
||||
|
||||
### Common deadlock — async + sync mutex
|
||||
|
||||
```rust
|
||||
let m = std::sync::Mutex::new(0u64);
|
||||
let guard = m.lock().unwrap();
|
||||
something_async().await; // ❌ guard is held across await
|
||||
*guard += 1;
|
||||
```
|
||||
|
||||
Under `current_thread` runtime this deadlocks (the future suspends while holding the lock; another future on the same thread tries to acquire, blocks the executor). Under `multi_thread` it works but serializes the system.
|
||||
|
||||
Fix:
|
||||
|
||||
```rust
|
||||
{
|
||||
let mut guard = m.lock().unwrap();
|
||||
*guard += 1;
|
||||
} // guard released
|
||||
something_async().await;
|
||||
```
|
||||
|
||||
Or switch to `tokio::sync::Mutex` whose guard is `Send` across awaits.
|
||||
|
||||
## Channels
|
||||
|
||||
### Mpsc — the workhorse
|
||||
|
||||
```rust
|
||||
let (tx, mut rx) = tokio::sync::mpsc::channel::<Job>(256);
|
||||
|
||||
tokio::spawn(async move {
|
||||
while let Some(job) = rx.recv().await {
|
||||
process(job).await;
|
||||
}
|
||||
});
|
||||
|
||||
tx.send(Job::new()).await?; // backpressure: awaits if full
|
||||
```
|
||||
|
||||
Capacity is the backpressure budget. **Never `unbounded_channel()`** unless you have a hard upper bound elsewhere; otherwise it is a slow-leak memory bomb.
|
||||
|
||||
### Watch — latest-value pubsub
|
||||
|
||||
```rust
|
||||
let (tx, mut rx) = tokio::sync::watch::channel(Config::default());
|
||||
|
||||
// Producer:
|
||||
tx.send(new_config)?;
|
||||
|
||||
// Consumer:
|
||||
loop {
|
||||
rx.changed().await?;
|
||||
let cfg = rx.borrow();
|
||||
apply(&cfg);
|
||||
}
|
||||
```
|
||||
|
||||
Receivers see only the latest value (older updates are dropped). Perfect for config reload, leadership changes, "current time" propagation.
|
||||
|
||||
### Broadcast — fanout queue
|
||||
|
||||
```rust
|
||||
let (tx, _) = tokio::sync::broadcast::channel::<Event>(1024);
|
||||
let mut rx1 = tx.subscribe();
|
||||
let mut rx2 = tx.subscribe();
|
||||
|
||||
while let Ok(event) = rx1.recv().await {
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
Each subscriber has its own buffer. If a subscriber falls behind by more than the buffer size, it gets `RecvError::Lagged(n)` and skips messages. Decide explicitly: log + continue, or drop the subscriber and reconnect.
|
||||
|
||||
### Oneshot — single value
|
||||
|
||||
```rust
|
||||
let (tx, rx) = tokio::sync::oneshot::channel::<Response>();
|
||||
worker.send(Request { reply: tx }).await?;
|
||||
let response = rx.await?;
|
||||
```
|
||||
|
||||
The standard request/response pattern over an actor.
|
||||
|
||||
## Semaphores
|
||||
|
||||
Bound concurrent operations:
|
||||
|
||||
```rust
|
||||
let sem = Arc::new(tokio::sync::Semaphore::new(10));
|
||||
|
||||
for task in tasks {
|
||||
let permit = sem.clone().acquire_owned().await?;
|
||||
tokio::spawn(async move {
|
||||
let _hold = permit; // released when task exits
|
||||
process(task).await
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
Use cases:
|
||||
- "Max 10 outbound HTTP requests in flight."
|
||||
- "Max 3 DB connections doing writes."
|
||||
- "Max N tokio tasks running heavy CPU."
|
||||
|
||||
A semaphore with `permits=1` is a mutex. Use the actual `Mutex` for that — clearer intent.
|
||||
|
||||
## Arc and Rc
|
||||
|
||||
`Arc<T>` for cross-thread shared ownership, `Rc<T>` for single-thread (never spans threads).
|
||||
|
||||
```rust
|
||||
let shared = Arc::new(BigData::new());
|
||||
for _ in 0..workers {
|
||||
let s = shared.clone();
|
||||
tokio::spawn(async move { use_data(&s).await });
|
||||
}
|
||||
```
|
||||
|
||||
`Arc::clone(&s)` is just a reference-count increment; the data is not copied.
|
||||
|
||||
**Do not clone in hot loops** if you can pass a reference. `&Arc<T>` is fine to pass; only call `Arc::clone` when you need to move ownership across a thread/task boundary.
|
||||
|
||||
`Weak<T>` for back-references in graphs / parent pointers to avoid cycles.
|
||||
|
||||
## Once-init primitives
|
||||
|
||||
```rust
|
||||
use std::sync::{OnceLock, LazyLock};
|
||||
|
||||
// Lazy initialization, computed on first read
|
||||
static CONFIG: LazyLock<Config> = LazyLock::new(|| Config::load_from_env().unwrap());
|
||||
|
||||
fn get_config() -> &'static Config {
|
||||
&CONFIG
|
||||
}
|
||||
|
||||
// One-shot publication, set explicitly
|
||||
static DB: OnceLock<sqlx::PgPool> = OnceLock::new();
|
||||
|
||||
#[tokio::main]
|
||||
async fn main() {
|
||||
let pool = sqlx::PgPool::connect(&env_url()).await.unwrap();
|
||||
DB.set(pool).expect("only set once");
|
||||
// Now everywhere: DB.get().unwrap()
|
||||
}
|
||||
```
|
||||
|
||||
`OnceLock` is `std::sync` and stable. `LazyLock` is in `std::sync` since 1.80. Avoid the older `once_cell` crate for new code.
|
||||
|
||||
## Loom — model-checking lock-free code
|
||||
|
||||
When `unsafe` participates in a concurrent algorithm, miri's single-thread model is insufficient. Loom exhaustively explores thread interleavings.
|
||||
|
||||
`Cargo.toml`:
|
||||
|
||||
```toml
|
||||
[target.'cfg(loom)'.dev-dependencies]
|
||||
loom = "0.7"
|
||||
```
|
||||
|
||||
In code, switch between real and loom primitives:
|
||||
|
||||
```rust
|
||||
#[cfg(loom)]
|
||||
use loom::sync::atomic::{AtomicUsize, Ordering};
|
||||
#[cfg(not(loom))]
|
||||
use std::sync::atomic::{AtomicUsize, Ordering};
|
||||
|
||||
#[cfg(loom)]
|
||||
use loom::sync::Arc;
|
||||
#[cfg(not(loom))]
|
||||
use std::sync::Arc;
|
||||
```
|
||||
|
||||
Write a test:
|
||||
|
||||
```rust
|
||||
#[cfg(loom)]
|
||||
mod loom_tests {
|
||||
use super::*;
|
||||
use loom::thread;
|
||||
|
||||
#[test]
|
||||
fn concurrent_push_pop_preserves_order() {
|
||||
loom::model(|| {
|
||||
let queue = Arc::new(MyQueue::new());
|
||||
let q1 = queue.clone();
|
||||
let q2 = queue.clone();
|
||||
let h1 = thread::spawn(move || q1.push(1));
|
||||
let h2 = thread::spawn(move || q2.pop());
|
||||
h1.join().unwrap();
|
||||
h2.join().unwrap();
|
||||
// Assert the invariant: queue is in a coherent state.
|
||||
});
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
RUSTFLAGS="--cfg loom" cargo test --release -- --test-threads 1
|
||||
```
|
||||
|
||||
Loom explores every legal scheduling of the threads, including those a real scheduler would rarely produce. If your code has a race, loom will find it deterministically.
|
||||
|
||||
### Loom's limits
|
||||
|
||||
- Slow. Each `loom::model` invocation explores many schedules; keep tests tiny (2-3 threads, a few operations each).
|
||||
- Single-machine only. Doesn't model distributed systems.
|
||||
- Doesn't catch UB inside `unsafe` blocks the way miri does. **Run both: miri for memory safety, loom for thread schedules.**
|
||||
- Doesn't handle `tokio` directly. Loom replaces stdlib's sync primitives; tokio's are independent.
|
||||
|
||||
## Send and Sync — what they mean
|
||||
|
||||
- `T: Send` — `T` can be moved between threads safely.
|
||||
- `T: Sync` — `&T` can be shared between threads safely.
|
||||
|
||||
These are auto-derived for composite types if all components implement them. Manual `unsafe impl Send/Sync` is required only for raw pointer types and FFI handles.
|
||||
|
||||
```rust
|
||||
struct MyHandle { raw: *mut FfiObject }
|
||||
|
||||
// SAFETY: FfiObject's documented contract states that move-between-threads
|
||||
// is safe as long as concurrent use is externally synchronized. We do not
|
||||
// implement Sync because the FFI object is single-threaded once obtained.
|
||||
unsafe impl Send for MyHandle {}
|
||||
// Do NOT impl Sync — the FFI is not thread-safe.
|
||||
```
|
||||
|
||||
When the compiler complains that "T: Send is not satisfied", the cause is usually a raw pointer, an `Rc` (not `Arc`), or a `RefCell` (use `Mutex`).
|
||||
|
||||
## Common mistakes
|
||||
|
||||
1. **Holding a `std::sync::Mutex` guard across `.await`.** Compiles, deadlocks at runtime under `current_thread`.
|
||||
2. **`Arc::clone` in a tight loop.** Refcount bump is cheap but not free; pass `&Arc<T>` when possible.
|
||||
3. **`Mutex<HashMap<K, V>>` for hot reads.** Switch to `RwLock` or `Arc<dashmap::DashMap>`.
|
||||
4. **Atomic operations with `Ordering::Relaxed` for happens-before publication.** You need `Release`/`Acquire`. Run under loom to be sure.
|
||||
5. **Unbounded channels.** Always set capacity. If you "know it won't backlog", you don't, and it will.
|
||||
6. **Spawning detached tokio tasks for fire-and-forget cleanup.** Use `JoinSet` so panics surface.
|
||||
7. **`std::mem::transmute` to fake `Send`/`Sync`.** Use `unsafe impl` with a SAFETY comment instead. Transmute breaks Stacked Borrows and miri.
|
||||
8. **Locking order inversion across two mutexes.** Always acquire in a globally consistent order. For more than three locks, switch to a single mutex around a struct.
|
||||
|
||||
## When to escape to lock-free
|
||||
|
||||
You should reach for atomics + `UnsafeCell` only when:
|
||||
1. The hot path is **measured** to be bottlenecked on lock contention.
|
||||
2. There is no existing library (crossbeam, atomic-queue, hazardous) that solves your problem.
|
||||
3. You can write loom tests that pass.
|
||||
4. You can write miri tests that pass.
|
||||
5. You have at least one other engineer who can review the algorithm.
|
||||
|
||||
Practically all "I want to write a lock-free queue" projects fail (3) or (4). When in doubt, take the lock and move on.
|
||||
@@ -0,0 +1,439 @@
|
||||
# Library Defaults — Full Decision Tree
|
||||
|
||||
The opinionated, audited-in-prod stack for 2026 Rust. Every entry has a one-line rationale and a canonical code snippet so the agent does not have to relearn each library's idioms.
|
||||
|
||||
## Async runtime — `tokio`
|
||||
|
||||
The default. Use `tokio` for new work. Multi-thread runtime unless you have a measured reason to go single-thread.
|
||||
|
||||
```rust
|
||||
#[tokio::main(flavor = "multi_thread", worker_threads = 8)]
|
||||
async fn main() -> anyhow::Result<()> {
|
||||
tracing_subscriber::fmt::init();
|
||||
run().await
|
||||
}
|
||||
```
|
||||
|
||||
Avoid:
|
||||
- `async-std` — unmaintained, last release ages ago. crates.io download counts are misleading because of historical inertia.
|
||||
- `smol` — fine for embedded-ish niches; outside that, the ecosystem is on tokio.
|
||||
- Mixing runtimes in one binary. Pick one and stay.
|
||||
|
||||
## Errors — `anyhow` (apps) + `thiserror` (libs)
|
||||
|
||||
Application boundaries get `anyhow::Error` with `.context("...")` at every layer that adds meaning. Libraries expose `#[derive(thiserror::Error)]` enums with `#[non_exhaustive]`.
|
||||
|
||||
```rust
|
||||
// Application code
|
||||
use anyhow::Context as _;
|
||||
|
||||
pub async fn load_config(path: &Path) -> anyhow::Result<Config> {
|
||||
let text = tokio::fs::read_to_string(path)
|
||||
.await
|
||||
.with_context(|| format!("reading config from {}", path.display()))?;
|
||||
let cfg: Config = toml::from_str(&text)
|
||||
.with_context(|| format!("parsing config at {}", path.display()))?;
|
||||
Ok(cfg)
|
||||
}
|
||||
|
||||
// Library code
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
#[non_exhaustive]
|
||||
pub enum ParseError {
|
||||
#[error("expected {expected}, found {found} at position {position}")]
|
||||
Mismatch { expected: &'static str, found: String, position: usize },
|
||||
#[error("unexpected end of input after {context}")]
|
||||
UnexpectedEof { context: &'static str },
|
||||
#[error(transparent)]
|
||||
Io(#[from] std::io::Error),
|
||||
}
|
||||
```
|
||||
|
||||
`#[non_exhaustive]` on enums prevents downstream `match` from breaking when you add variants. `#[error(transparent)]` on a wrapper variant forwards Display + cause to the inner error.
|
||||
|
||||
## CLI — `clap` with derive
|
||||
|
||||
```rust
|
||||
use clap::{Parser, Subcommand};
|
||||
|
||||
#[derive(Parser, Debug)]
|
||||
#[command(author, version, about, long_about = None)]
|
||||
struct Cli {
|
||||
/// Path to the config file
|
||||
#[arg(short, long, env = "MYAPP_CONFIG", default_value = "config.toml")]
|
||||
config: PathBuf,
|
||||
|
||||
/// Enable verbose output (-v, -vv, -vvv)
|
||||
#[arg(short, long, action = clap::ArgAction::Count)]
|
||||
verbose: u8,
|
||||
|
||||
#[command(subcommand)]
|
||||
command: Command,
|
||||
}
|
||||
|
||||
#[derive(Subcommand, Debug)]
|
||||
enum Command {
|
||||
/// Run the server
|
||||
Serve {
|
||||
#[arg(short, long, default_value_t = 8080)]
|
||||
port: u16,
|
||||
},
|
||||
/// Migrate the database
|
||||
Migrate {
|
||||
#[arg(long)]
|
||||
dry_run: bool,
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
Avoid `structopt` (deprecated, merged into clap), `argh` (less ergonomic), `pico-args` (only when binary size matters more than DX).
|
||||
|
||||
## Logging — `tracing` + `tracing-subscriber`
|
||||
|
||||
Not `log` + `env_logger`. `tracing` supports spans (structured context that follows async tasks) and structured fields - `log` cannot.
|
||||
|
||||
```rust
|
||||
use tracing::{info, instrument, warn, Level};
|
||||
use tracing_subscriber::{fmt, EnvFilter};
|
||||
|
||||
fn init_tracing() {
|
||||
let filter = EnvFilter::try_from_default_env()
|
||||
.unwrap_or_else(|_| EnvFilter::new("info,sqlx=warn,hyper=warn"));
|
||||
fmt()
|
||||
.with_env_filter(filter)
|
||||
.with_target(false)
|
||||
.with_thread_ids(true)
|
||||
.with_line_number(true)
|
||||
.compact()
|
||||
.init();
|
||||
}
|
||||
|
||||
#[instrument(skip(db), fields(user_id = %user.id))]
|
||||
async fn process_user(db: &Pool, user: &User) -> anyhow::Result<()> {
|
||||
info!("processing user");
|
||||
if user.is_banned() {
|
||||
warn!(reason = "banned", "skipping");
|
||||
return Ok(());
|
||||
}
|
||||
// ... body ...
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
Replace `println!` with `info!`/`warn!`/`error!`. Replace `eprintln!` with `tracing::error!`.
|
||||
|
||||
## Error reporting (binaries) — `color-eyre`
|
||||
|
||||
For binary `main()`, hook `color-eyre` to give pretty panics + nice `Result` printing:
|
||||
|
||||
```rust
|
||||
fn main() -> color_eyre::Result<()> {
|
||||
color_eyre::install()?;
|
||||
tracing_subscriber::fmt::init();
|
||||
real_main()
|
||||
}
|
||||
```
|
||||
|
||||
Library code stays on `anyhow`/`thiserror`. `color-eyre` is purely a display layer for the binary.
|
||||
|
||||
## Serialization — `serde` + `serde_json`
|
||||
|
||||
The default for any data crossing a process boundary (file, network, IPC, database column).
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
|
||||
#[serde(deny_unknown_fields, rename_all = "snake_case")]
|
||||
pub struct ApiResponse {
|
||||
pub user_id: UserId,
|
||||
pub created_at: jiff::Timestamp,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub display_name: Option<String>,
|
||||
#[serde(flatten)]
|
||||
pub extra: HashMap<String, serde_json::Value>,
|
||||
}
|
||||
```
|
||||
|
||||
`deny_unknown_fields` catches typos in inputs. `rename_all = "snake_case"` aligns with REST/JSON conventions while keeping idiomatic Rust field names. `#[serde(flatten)]` for forward-compatible extra fields.
|
||||
|
||||
Alternatives:
|
||||
- `serde_yaml` (YAML — note: YAML's "deserialize anything" surface is a security trap; prefer JSON/TOML where possible)
|
||||
- `toml` (config files)
|
||||
- `rmp-serde` (MessagePack — binary, fast)
|
||||
- `ciborium` (CBOR)
|
||||
- `bincode 2` (binary, smaller; no serde required in v2 but interop fine)
|
||||
|
||||
## HTTP client — `reqwest`
|
||||
|
||||
```rust
|
||||
let client = reqwest::Client::builder()
|
||||
.timeout(std::time::Duration::from_secs(30))
|
||||
.user_agent(concat!(env!("CARGO_PKG_NAME"), "/", env!("CARGO_PKG_VERSION")))
|
||||
.https_only(true)
|
||||
.pool_max_idle_per_host(8)
|
||||
.build()?;
|
||||
|
||||
#[derive(serde::Deserialize)]
|
||||
struct Repo { full_name: String, stargazers_count: u64 }
|
||||
|
||||
let repo: Repo = client
|
||||
.get("https://api.github.com/repos/rust-lang/rust")
|
||||
.send().await?
|
||||
.error_for_status()?
|
||||
.json().await?;
|
||||
```
|
||||
|
||||
`error_for_status()?` turns 4xx/5xx into `Err`. Always include a User-Agent. `https_only(true)` is a soundness toggle - prevents accidental http:// downgrade.
|
||||
|
||||
## Web framework — `axum`
|
||||
|
||||
```rust
|
||||
use axum::{Router, routing::get, extract::State, response::Json};
|
||||
use std::sync::Arc;
|
||||
|
||||
#[derive(Clone)]
|
||||
struct AppState { db: sqlx::PgPool }
|
||||
|
||||
async fn health(State(state): State<Arc<AppState>>) -> Json<serde_json::Value> {
|
||||
let ok = sqlx::query_scalar!("SELECT 1::int4").fetch_one(&state.db).await.is_ok();
|
||||
Json(serde_json::json!({ "ok": ok }))
|
||||
}
|
||||
|
||||
#[tokio::main]
|
||||
async fn main() -> anyhow::Result<()> {
|
||||
tracing_subscriber::fmt::init();
|
||||
let state = Arc::new(AppState { db: sqlx::PgPool::connect(&env_db()).await? });
|
||||
let app = Router::new()
|
||||
.route("/health", get(health))
|
||||
.with_state(state)
|
||||
.layer(tower_http::trace::TraceLayer::new_for_http())
|
||||
.layer(tower_http::compression::CompressionLayer::new());
|
||||
let listener = tokio::net::TcpListener::bind("0.0.0.0:8080").await?;
|
||||
axum::serve(listener, app).await?;
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
Avoid `actix-web` (legacy patterns, separate runtime model), `warp` (filter explosion in non-trivial apps), `rocket` (slow release cadence). Pair `axum` with `tower-http` for middleware (trace, compression, CORS, timeout, request-id).
|
||||
|
||||
## Database — `sqlx` (compile-time checked SQL)
|
||||
|
||||
```rust
|
||||
use sqlx::PgPool;
|
||||
|
||||
#[derive(Debug, sqlx::FromRow)]
|
||||
pub struct User { pub id: uuid::Uuid, pub email: String, pub created_at: jiff::Timestamp }
|
||||
|
||||
pub async fn find_user(pool: &PgPool, email: &str) -> Result<Option<User>, sqlx::Error> {
|
||||
sqlx::query_as!(
|
||||
User,
|
||||
r#"SELECT id, email, created_at as "created_at: jiff::Timestamp"
|
||||
FROM users WHERE email = $1"#,
|
||||
email
|
||||
)
|
||||
.fetch_optional(pool)
|
||||
.await
|
||||
}
|
||||
```
|
||||
|
||||
`query_as!` checks the SQL against the live database at compile time. To work without a live DB during builds, generate offline metadata: `cargo sqlx prepare`. Commit the resulting `.sqlx/` directory.
|
||||
|
||||
Avoid `diesel` (sync-first, heavy DSL), raw `tokio-postgres` (loses type checks), `sea-orm` (more magic, less control).
|
||||
|
||||
For migrations: `sqlx migrate add <name>` + `sqlx::migrate!("./migrations").run(&pool).await?`.
|
||||
|
||||
## Time — `jiff`
|
||||
|
||||
The 2025+ choice. Single crate, sane defaults, civil time / instant / span distinction.
|
||||
|
||||
```rust
|
||||
use jiff::{Timestamp, Span, ToSpan, Zoned};
|
||||
|
||||
let now: Timestamp = Timestamp::now();
|
||||
let in_one_hour = now.checked_add(1.hour())?;
|
||||
let local: Zoned = now.in_tz("Asia/Seoul")?;
|
||||
let span: Span = local - some_earlier.in_tz("Asia/Seoul")?;
|
||||
```
|
||||
|
||||
Avoid `chrono` (old API, generic-heavy, time zone story still painful), `time` crate (split ecosystem, weaker docs). `jiff` is the post-`chrono` consolidation.
|
||||
|
||||
## UUID — `uuid` with v7
|
||||
|
||||
```rust
|
||||
use uuid::Uuid;
|
||||
|
||||
// v7 for IDs (sortable, time-ordered, monotonic-ish, RFC 9562)
|
||||
let id = Uuid::now_v7();
|
||||
```
|
||||
|
||||
v4 is fine for nonces, v7 for primary keys (better index locality). Never v1 (leaks MAC). Cargo features: `uuid = { version = "1", features = ["v4", "v7", "serde"] }`.
|
||||
|
||||
## DataFrames / analytics — `polars`
|
||||
|
||||
For columnar data, joins, group-by, lazy plans:
|
||||
|
||||
```rust
|
||||
use polars::prelude::*;
|
||||
|
||||
let df = LazyCsvReader::new("events.csv")
|
||||
.finish()?
|
||||
.group_by([col("user_id")])
|
||||
.agg([col("amount").sum().alias("total")])
|
||||
.sort(["total"], Default::default())
|
||||
.collect()?;
|
||||
```
|
||||
|
||||
The Rust API mirrors the Python one. Use the lazy API by default; materialize with `.collect()` at the end.
|
||||
|
||||
## Channels
|
||||
|
||||
- Single-producer single-consumer or bounded MPSC → `tokio::sync::mpsc` (async) or `flume` (sync + async).
|
||||
- Broadcast → `tokio::sync::broadcast`.
|
||||
- Watch (latest-value pubsub) → `tokio::sync::watch`.
|
||||
- Oneshot → `tokio::sync::oneshot`.
|
||||
|
||||
Avoid raw `std::sync::mpsc` (sync only, fewer features), `crossbeam-channel` (good but heavier; use only if you need rendezvous semantics).
|
||||
|
||||
## Coordinate spaces / 2D math — `euclid`
|
||||
|
||||
```rust
|
||||
use euclid::{Point2D, Size2D, default::Box2D};
|
||||
struct ScreenSpace;
|
||||
struct WorldSpace;
|
||||
|
||||
type ScreenPoint = Point2D<f32, ScreenSpace>;
|
||||
type WorldPoint = Point2D<f32, WorldSpace>;
|
||||
|
||||
let cursor: ScreenPoint = Point2D::new(120.0, 240.0);
|
||||
let player: WorldPoint = Point2D::new(3.5, 1.2);
|
||||
|
||||
// let mistake = cursor + player; // ❌ type error
|
||||
```
|
||||
|
||||
Generalize the pattern to your own domains (see `references/type-state.md`).
|
||||
|
||||
## Property tests — `proptest`
|
||||
|
||||
```rust
|
||||
use proptest::prelude::*;
|
||||
|
||||
proptest! {
|
||||
#[test]
|
||||
fn parse_roundtrips(s in r"[a-zA-Z0-9_-]{1,50}") {
|
||||
let parsed = parse(&s).unwrap();
|
||||
let back = parsed.to_string();
|
||||
prop_assert_eq!(back, s);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Avoid `quickcheck` (older, less ergonomic). proptest gives shrinking + regression corpus + integration with `criterion`.
|
||||
|
||||
## Snapshot tests — `insta`
|
||||
|
||||
```rust
|
||||
#[test]
|
||||
fn renders_help() {
|
||||
let output = render(&example_input());
|
||||
insta::assert_snapshot!(output);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn serializes_well() {
|
||||
insta::assert_json_snapshot!(serializable_value());
|
||||
}
|
||||
```
|
||||
|
||||
`cargo insta review` (after `cargo install cargo-insta`) — interactive review of changed snapshots.
|
||||
|
||||
## Benchmarks — `criterion`
|
||||
|
||||
Stable Rust friendly (no nightly `#[bench]`).
|
||||
|
||||
```rust
|
||||
use criterion::{black_box, criterion_group, criterion_main, Criterion};
|
||||
|
||||
fn bench_parse(c: &mut Criterion) {
|
||||
let input = std::fs::read_to_string("samples/large.txt").unwrap();
|
||||
c.bench_function("parse_large", |b| b.iter(|| parse(black_box(&input))));
|
||||
}
|
||||
|
||||
criterion_group!(benches, bench_parse);
|
||||
criterion_main!(benches);
|
||||
```
|
||||
|
||||
Run with `cargo bench`. HTML reports under `target/criterion/`. Pair with `cargo bench -- --save-baseline main` then `--baseline main` for comparison.
|
||||
|
||||
## Concurrency model — `loom`
|
||||
|
||||
For lock-free or atomic-heavy code (channels, refcounts, hazard pointers). See `references/concurrency.md` for the full pattern.
|
||||
|
||||
## Arena allocator — `bumpalo`
|
||||
|
||||
```rust
|
||||
use bumpalo::Bump;
|
||||
|
||||
let bump = Bump::new();
|
||||
let node = bump.alloc(Node { value: 42, next: None });
|
||||
let s: &str = bump.alloc_str("hello");
|
||||
// All allocations freed at once when `bump` drops.
|
||||
```
|
||||
|
||||
For parser nodes, AST construction, per-request scratch. Outperforms heap allocation for short-lived owned data by an order of magnitude.
|
||||
|
||||
## Web client (browser, WASM-bound) — `gloo` ecosystem
|
||||
|
||||
If targeting WASM browser, use `gloo-net` for fetch and `gloo-storage` for localStorage; not `web-sys` directly unless you need DOM-level APIs.
|
||||
|
||||
## Lazy statics — `std::sync::LazyLock` (since 1.80)
|
||||
|
||||
```rust
|
||||
use std::sync::LazyLock;
|
||||
static CONFIG: LazyLock<Config> = LazyLock::new(|| Config::load_from_env().unwrap());
|
||||
```
|
||||
|
||||
Avoid `lazy_static!` (macro-heavy, predates std), `once_cell` (now in std as `LazyLock`/`OnceLock`).
|
||||
|
||||
## Hash maps — `std::collections::HashMap` + `ahash` for hot paths
|
||||
|
||||
```rust
|
||||
use std::collections::HashMap;
|
||||
use ahash::RandomState;
|
||||
|
||||
type FastMap<K, V> = HashMap<K, V, RandomState>;
|
||||
let mut counters: FastMap<String, u64> = FastMap::default();
|
||||
```
|
||||
|
||||
`HashMap` defaults to SipHash (DoS-resistant). For internal hot loops where you trust the keys, `ahash` is 2-5x faster.
|
||||
|
||||
For sorted iteration, use `BTreeMap`. For small keys with known small N, `Vec<(K, V)>` may beat both.
|
||||
|
||||
## File I/O — `tokio::fs` (async) or `std::fs` (sync utility)
|
||||
|
||||
```rust
|
||||
let contents = tokio::fs::read_to_string("data.json").await?;
|
||||
```
|
||||
|
||||
For large files: `tokio::fs::File` + `tokio::io::BufReader`. For random access, `memmap2` (with the unsafe-discipline wrappers).
|
||||
|
||||
## Decision tree
|
||||
|
||||
```
|
||||
Need to ship the thing?
|
||||
├── HTTP server → axum + sqlx + tracing + jiff + tokio
|
||||
├── HTTP client → reqwest (+ tokio)
|
||||
├── CLI → clap + color-eyre + tracing + indicatif (progress) + dialoguer (prompts)
|
||||
├── TUI → ratatui + crossterm
|
||||
├── Background worker → tokio + ETL → polars + duckdb
|
||||
├── Game / graphics → wgpu + winit + euclid (or bevy if you want the engine)
|
||||
├── WASM front-end → leptos (or dioxus / yew) + wasm-bindgen + gloo
|
||||
├── Embedded → embassy (async on bare metal)
|
||||
├── FFI to C / Python → cxx (C++) / pyo3 (Python) / cbindgen (header gen)
|
||||
└── Just a script → rust-script (see one-liners.md)
|
||||
```
|
||||
|
||||
When in doubt, search crates.io for the latest version, then check:
|
||||
1. Is it maintained? (`cargo deny check` will scream if it's yanked or unmaintained)
|
||||
2. Does it have `serde` feature? (boundary types should always serde)
|
||||
3. Does it have `tokio` integration? (avoid runtime mixing)
|
||||
4. Is it on `tokio::io::AsyncRead`/`AsyncWrite` (the std for async I/O)?
|
||||
5. Are there safety-critical `unsafe` regions? If yes, has the author shipped miri proofs?
|
||||
@@ -0,0 +1,291 @@
|
||||
# One-Liners and Disposable Scripts
|
||||
|
||||
Production hygiene with throwaway ergonomics. Rust scripts get the same strict lints, the same miri rule when `unsafe` is touched, the same type discipline. The difference is dependency declaration lives inline.
|
||||
|
||||
## `rust-script` — the recommended path
|
||||
|
||||
Install once:
|
||||
|
||||
```bash
|
||||
cargo install rust-script
|
||||
```
|
||||
|
||||
Write a script:
|
||||
|
||||
```rust
|
||||
#!/usr/bin/env rust-script
|
||||
//! Fetch a URL and print its body length.
|
||||
//!
|
||||
//! Usage:
|
||||
//! ./fetch.rs <url>
|
||||
//!
|
||||
//! ```cargo
|
||||
//! [dependencies]
|
||||
//! anyhow = "1"
|
||||
//! reqwest = { version = "0.12", features = ["blocking"] }
|
||||
//! ```
|
||||
|
||||
use std::env;
|
||||
|
||||
fn main() -> anyhow::Result<()> {
|
||||
let url = env::args().nth(1).context("usage: fetch.rs <url>")?;
|
||||
let body = reqwest::blocking::get(&url)?.error_for_status()?.text()?;
|
||||
println!("{} bytes", body.len());
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
Make executable: `chmod +x fetch.rs`. Run: `./fetch.rs https://example.com`.
|
||||
|
||||
The `//! \`\`\`cargo` block is parsed as inline `Cargo.toml`. Everything else is normal Rust.
|
||||
|
||||
## With async
|
||||
|
||||
```rust
|
||||
#!/usr/bin/env rust-script
|
||||
//! ```cargo
|
||||
//! [dependencies]
|
||||
//! anyhow = "1"
|
||||
//! tokio = { version = "1", features = ["full"] }
|
||||
//! reqwest = "0.12"
|
||||
//! ```
|
||||
|
||||
#[tokio::main]
|
||||
async fn main() -> anyhow::Result<()> {
|
||||
let urls = [
|
||||
"https://example.com",
|
||||
"https://example.org",
|
||||
];
|
||||
let client = reqwest::Client::new();
|
||||
let bodies = futures::future::join_all(urls.iter().map(|u| {
|
||||
let c = client.clone();
|
||||
async move { c.get(*u).send().await?.text().await }
|
||||
})).await;
|
||||
for (url, body) in urls.iter().zip(bodies) {
|
||||
match body {
|
||||
Ok(b) => println!("{url}: {} bytes", b.len()),
|
||||
Err(e) => eprintln!("{url}: error {e}"),
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
## With CLI parsing
|
||||
|
||||
```rust
|
||||
#!/usr/bin/env rust-script
|
||||
//! ```cargo
|
||||
//! [dependencies]
|
||||
//! anyhow = "1"
|
||||
//! clap = { version = "4", features = ["derive"] }
|
||||
//! ```
|
||||
|
||||
use clap::Parser;
|
||||
|
||||
#[derive(Parser, Debug)]
|
||||
#[command(version, about = "rename files by pattern")]
|
||||
struct Cli {
|
||||
/// Glob to match
|
||||
pattern: String,
|
||||
/// Replacement template (use {n} for sequence)
|
||||
template: String,
|
||||
/// Show what would happen without doing it
|
||||
#[arg(long)]
|
||||
dry_run: bool,
|
||||
}
|
||||
|
||||
fn main() -> anyhow::Result<()> {
|
||||
let cli = Cli::parse();
|
||||
let entries: Vec<_> = glob::glob(&cli.pattern)?.collect::<Result<_, _>>()?;
|
||||
for (n, entry) in entries.iter().enumerate() {
|
||||
let target = cli.template.replace("{n}", &n.to_string());
|
||||
if cli.dry_run {
|
||||
println!("{} -> {target}", entry.display());
|
||||
} else {
|
||||
std::fs::rename(entry, &target)?;
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
## Caching
|
||||
|
||||
`rust-script` caches the compiled binary in `~/.cache/rust-script/`. First run is slow (full compile), subsequent runs are instant.
|
||||
|
||||
To clear: `rust-script --clear-cache`.
|
||||
|
||||
Pin a script's compile target into the script directory for portability:
|
||||
|
||||
```bash
|
||||
rust-script --build-only --base-path . ./script.rs
|
||||
```
|
||||
|
||||
This drops a `target/` next to the script with the prebuilt binary.
|
||||
|
||||
## `cargo-script` (RFC 3424, stable since Rust 1.85)
|
||||
|
||||
The official replacement that landed in cargo proper. Same idea, slightly different syntax:
|
||||
|
||||
```rust
|
||||
#!/usr/bin/env -S cargo +nightly -Zscript
|
||||
---
|
||||
package:
|
||||
name = "fetch"
|
||||
edition = "2024"
|
||||
|
||||
dependencies:
|
||||
anyhow = "1"
|
||||
reqwest = { version = "0.12", features = ["blocking"] }
|
||||
---
|
||||
|
||||
fn main() -> anyhow::Result<()> {
|
||||
let url = std::env::args().nth(1).context("url required")?;
|
||||
println!("{}", reqwest::blocking::get(&url)?.text()?.len());
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
Status as of 2026-05: stabilization in progress. Use `rust-script` for production now, migrate when `cargo script` is stable everywhere your tools live.
|
||||
|
||||
## Strict mode for scripts
|
||||
|
||||
Add a lints block in the inline `Cargo.toml`:
|
||||
|
||||
```rust
|
||||
//! ```cargo
|
||||
//! [dependencies]
|
||||
//! anyhow = "1"
|
||||
//!
|
||||
//! [lints.rust]
|
||||
//! unsafe_code = "forbid"
|
||||
//!
|
||||
//! [lints.clippy]
|
||||
//! all = "deny"
|
||||
//! pedantic = "warn"
|
||||
//! unwrap_used = "deny"
|
||||
//! expect_used = "deny"
|
||||
//! panic = "deny"
|
||||
//! ```
|
||||
```
|
||||
|
||||
Now the script gets the same strictness as the main project. If you need a one-line `unwrap()` for prototype velocity, switch the lint to `warn` for that one script - never blanket `allow`.
|
||||
|
||||
Run with lints visible:
|
||||
|
||||
```bash
|
||||
RUSTFLAGS="-D warnings" rust-script ./script.rs
|
||||
```
|
||||
|
||||
## When NOT to use a script
|
||||
|
||||
- It is going to live longer than a week → make it a real crate with `cargo new --bin`.
|
||||
- It needs custom build scripts (`build.rs`) → real crate.
|
||||
- It needs binary distribution to other machines → real crate with `cargo dist`.
|
||||
- It needs to be tested → real crate (scripts can technically run `#[test]`s under `cargo test`, but the workflow is awkward).
|
||||
|
||||
A reasonable migration path: start as a script, when complexity grows past ~200 lines or you reach for a second `.rs` file, run `rust-script --emit ./script.rs` to dump a regular Cargo project skeleton and continue from there.
|
||||
|
||||
## Inline tests in a script
|
||||
|
||||
```rust
|
||||
#!/usr/bin/env rust-script
|
||||
//! ```cargo
|
||||
//! [dependencies]
|
||||
//! ```
|
||||
|
||||
fn double(x: i32) -> i32 { x * 2 }
|
||||
|
||||
fn main() {
|
||||
println!("{}", double(21));
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn doubles_ints() {
|
||||
assert_eq!(double(5), 10);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Run tests: `rust-script --test ./script.rs`.
|
||||
|
||||
## A useful "Rust as awk" pattern
|
||||
|
||||
For ad-hoc data processing on stdin:
|
||||
|
||||
```rust
|
||||
#!/usr/bin/env rust-script
|
||||
//! ```cargo
|
||||
//! [dependencies]
|
||||
//! serde_json = "1"
|
||||
//! ```
|
||||
|
||||
use std::io::{self, BufRead, Write};
|
||||
|
||||
fn main() -> Result<(), Box<dyn std::error::Error>> {
|
||||
let stdin = io::stdin();
|
||||
let stdout = io::stdout();
|
||||
let mut out = stdout.lock();
|
||||
for line in stdin.lock().lines() {
|
||||
let line = line?;
|
||||
let value: serde_json::Value = serde_json::from_str(&line)?;
|
||||
if let Some(s) = value.get("level").and_then(|v| v.as_str()) {
|
||||
if s == "error" {
|
||||
writeln!(out, "{line}")?;
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
`cat logs.jsonl | ./filter-errors.rs` — filter JSON logs by `level == "error"`. Faster than `jq` for big files, type-safe.
|
||||
|
||||
For numerics:
|
||||
|
||||
```rust
|
||||
#!/usr/bin/env rust-script
|
||||
//! sum a column of numbers from stdin
|
||||
use std::io::{self, BufRead};
|
||||
fn main() {
|
||||
let total: f64 = io::stdin().lock().lines()
|
||||
.filter_map(|l| l.ok())
|
||||
.filter_map(|l| l.trim().parse::<f64>().ok())
|
||||
.sum();
|
||||
println!("{total}");
|
||||
}
|
||||
```
|
||||
|
||||
## The `rust-script` shebang trick on macOS
|
||||
|
||||
macOS does not support multi-arg shebangs without `env -S`. Use:
|
||||
|
||||
```rust
|
||||
#!/usr/bin/env -S rust-script --
|
||||
```
|
||||
|
||||
The `--` lets clap-style argument parsers see the user's args, not the rust-script arguments.
|
||||
|
||||
## Editor support
|
||||
|
||||
VS Code / Helix / Vim with `rust-analyzer`: open the script file as if it were `src/main.rs` of an inferred crate. Most editors auto-detect the inline manifest. If not, hand-create a `Cargo.toml` next to the script with matching deps for the duration of editing, then delete it.
|
||||
|
||||
## When `rust-script` is too heavy
|
||||
|
||||
For absolutely throwaway "one expression on stdin" use cases, a Rust REPL like `evcxr_jupyter` (Jupyter kernel) or `irust` (terminal REPL) is more appropriate:
|
||||
|
||||
```bash
|
||||
cargo install irust
|
||||
irust
|
||||
```
|
||||
|
||||
But these are interactive playgrounds, not scriptable. For shell pipelines, stay with `rust-script`.
|
||||
|
||||
## The Promise
|
||||
|
||||
Same strict lints. Same `clippy::pedantic` enforcement. Same `unsafe`-requires-SAFETY rule. The agent does not get a free pass on a 30-line script. The whole point of strict scripts is that **production hygiene is cheap when the toolchain enforces it**.
|
||||
@@ -0,0 +1,429 @@
|
||||
# Property Tests (proptest) + Snapshot Tests (insta)
|
||||
|
||||
Two test types every Rust project should have alongside unit tests. Proptest hunts for inputs your unit tests forgot to try. Insta locks down output shapes you do not want to silently change.
|
||||
|
||||
## When to reach for each
|
||||
|
||||
| Want to test… | Use |
|
||||
|---|---|
|
||||
| One specific behavior with a known input | `#[test]` + `assert_eq!` |
|
||||
| All inputs of a certain shape work | `proptest!` |
|
||||
| Output structure stays stable across refactors | `insta::assert_*_snapshot!` |
|
||||
| Parser/serializer round-trips | `proptest!` (the round-trip property) |
|
||||
| CLI help text, JSON response shape, debug output | `insta::assert_snapshot!` |
|
||||
| Concurrency under all interleavings | `loom` (see `concurrency.md`) |
|
||||
|
||||
Use all three. They cover different bug classes.
|
||||
|
||||
## Proptest — setup
|
||||
|
||||
`Cargo.toml`:
|
||||
|
||||
```toml
|
||||
[dev-dependencies]
|
||||
proptest = "1"
|
||||
proptest-derive = "0.5" # for #[derive(Arbitrary)]
|
||||
```
|
||||
|
||||
`proptest.toml` at project root (optional, sane defaults):
|
||||
|
||||
```toml
|
||||
cases = 256 # number of random inputs per property
|
||||
max_local_rejects = 65536
|
||||
max_global_rejects = 1024
|
||||
max_shrink_iters = 1024
|
||||
max_shrink_time = 60_000 # ms
|
||||
failure_persistence = { source_file = "proptest-regressions/", file_name = "regressions.txt" }
|
||||
verbose = 0
|
||||
```
|
||||
|
||||
`failure_persistence` is the killer feature: every failure is written to a regression file. On the next run, those exact inputs are replayed first, so once a bug is found it never escapes again.
|
||||
|
||||
## Basic property test
|
||||
|
||||
```rust
|
||||
use proptest::prelude::*;
|
||||
|
||||
fn parse(s: &str) -> Result<Color, ParseError> { /* ... */ }
|
||||
fn render(c: &Color) -> String { /* ... */ }
|
||||
|
||||
proptest! {
|
||||
#[test]
|
||||
fn parse_render_roundtrips(red in 0u8..=255, green in 0u8..=255, blue in 0u8..=255) {
|
||||
let color = Color { red, green, blue };
|
||||
let rendered = render(&color);
|
||||
let parsed = parse(&rendered).expect("our render should always parse");
|
||||
prop_assert_eq!(parsed, color);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`proptest!` macro takes `(arg in strategy, ...)` pairs. Each strategy produces values; proptest runs the body with random samples, then shrinks failing cases to minimal forms.
|
||||
|
||||
## Strategies — the value-generation language
|
||||
|
||||
| Strategy | Produces |
|
||||
|---|---|
|
||||
| `any::<T>()` | Any value of `T` (if `T: Arbitrary`) |
|
||||
| `0u32..100` | Integer ranges |
|
||||
| `prop::sample::select(slice)` | Pick from a list |
|
||||
| `prop::collection::vec(elem, range)` | Vec of length in range |
|
||||
| `prop::collection::hash_map(k, v, n..m)` | HashMap |
|
||||
| `prop::option::of(strategy)` | Option |
|
||||
| `prop::result::maybe_ok(ok, err)` | Result |
|
||||
| `(s1, s2).prop_map(\|(a, b)\| ...)` | Combine, transform |
|
||||
| `s.prop_filter("reason", \|v\| pred)` | Reject values |
|
||||
| `s.prop_flat_map(\|v\| dependent)` | Sequential dependency |
|
||||
| `prop_oneof![strategy1, strategy2]` | Union of strategies |
|
||||
| `r"[a-z]{3,10}"` | Regex-generated string |
|
||||
| `"\\PC*"` | Any printable non-control string |
|
||||
|
||||
Example combining several:
|
||||
|
||||
```rust
|
||||
fn config_strategy() -> impl Strategy<Value = Config> {
|
||||
(
|
||||
prop::sample::select(vec!["dev", "staging", "prod"]),
|
||||
0u16..=65535,
|
||||
prop::collection::hash_map(
|
||||
r"[a-z_]{1,20}",
|
||||
any::<String>(),
|
||||
0..5,
|
||||
),
|
||||
).prop_map(|(env, port, vars)| Config {
|
||||
env: env.into(),
|
||||
port,
|
||||
env_vars: vars,
|
||||
})
|
||||
}
|
||||
|
||||
proptest! {
|
||||
#[test]
|
||||
fn config_validates(cfg in config_strategy()) {
|
||||
let result = validate(&cfg);
|
||||
if cfg.port == 0 {
|
||||
prop_assert!(result.is_err());
|
||||
} else {
|
||||
prop_assert!(result.is_ok());
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Properties to write for every parser
|
||||
|
||||
1. **Round-trip:** `parse(render(x)) == x` for all valid `x`.
|
||||
2. **No-panic:** `parse(arbitrary_string)` never panics, always returns `Result`.
|
||||
3. **Idempotent:** `parse(parse(x).unwrap().render()) == parse(x).unwrap()`.
|
||||
4. **Whitespace insensitivity:** `parse(x) == parse(strip_whitespace(x))` (if applicable).
|
||||
|
||||
For every serializer:
|
||||
|
||||
1. **Length bound:** `render(x).len() <= bound(x)`.
|
||||
2. **Charset:** `render(x).chars().all(|c| ALLOWED.contains(&c))`.
|
||||
|
||||
For every collection operation:
|
||||
|
||||
1. **Identity:** `op_identity(x) == x` (sort an already-sorted, dedupe a unique).
|
||||
2. **Idempotence:** `op(op(x)) == op(x)`.
|
||||
3. **Commutativity:** `op(a, b) == op(b, a)` (set union, etc).
|
||||
4. **Length:** `op(a, b).len() == known_relation(a.len(), b.len())`.
|
||||
|
||||
For every numeric op:
|
||||
|
||||
1. **Monotonicity:** `a <= b => f(a) <= f(b)`.
|
||||
2. **Identity element:** `f(x, identity) == x`.
|
||||
|
||||
Write these mechanically. The agent should reach for proptest the moment any of these properties is checkable.
|
||||
|
||||
## Derive `Arbitrary`
|
||||
|
||||
```rust
|
||||
use proptest_derive::Arbitrary;
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Arbitrary)]
|
||||
struct Vec3 {
|
||||
#[proptest(strategy = "-100.0..=100.0")]
|
||||
x: f32,
|
||||
#[proptest(strategy = "-100.0..=100.0")]
|
||||
y: f32,
|
||||
#[proptest(strategy = "-100.0..=100.0")]
|
||||
z: f32,
|
||||
}
|
||||
|
||||
proptest! {
|
||||
#[test]
|
||||
fn dot_product_is_commutative(a: Vec3, b: Vec3) {
|
||||
prop_assert!((dot(&a, &b) - dot(&b, &a)).abs() < 1e-5);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`#[derive(Arbitrary)]` auto-implements the strategy. Per-field `#[proptest(strategy = "...")]` overrides.
|
||||
|
||||
## Stateful / state machine tests
|
||||
|
||||
For data structures with operations (queues, maps, trees), use `proptest-state-machine`:
|
||||
|
||||
```rust
|
||||
use proptest_state_machine::{ReferenceStateMachine, StateMachineTest};
|
||||
|
||||
struct MyQueueRef { state: VecDeque<i32> }
|
||||
struct MyQueueSut { sut: MyQueue<i32> }
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
enum Op { Push(i32), Pop }
|
||||
|
||||
impl ReferenceStateMachine for MyQueueRef {
|
||||
type State = VecDeque<i32>;
|
||||
type Transition = Op;
|
||||
|
||||
fn init_state() -> BoxedStrategy<Self::State> {
|
||||
Just(VecDeque::new()).boxed()
|
||||
}
|
||||
fn transitions(_: &Self::State) -> BoxedStrategy<Self::Transition> {
|
||||
prop_oneof![
|
||||
any::<i32>().prop_map(Op::Push),
|
||||
Just(Op::Pop),
|
||||
].boxed()
|
||||
}
|
||||
fn apply(mut state: Self::State, transition: &Self::Transition) -> Self::State {
|
||||
match transition {
|
||||
Op::Push(x) => state.push_back(*x),
|
||||
Op::Pop => { state.pop_front(); }
|
||||
}
|
||||
state
|
||||
}
|
||||
}
|
||||
|
||||
impl StateMachineTest for MyQueueSut {
|
||||
type SystemUnderTest = MyQueue<i32>;
|
||||
type Reference = MyQueueRef;
|
||||
|
||||
fn init_test(_: &<Self::Reference as ReferenceStateMachine>::State) -> Self::SystemUnderTest {
|
||||
MyQueue::new()
|
||||
}
|
||||
fn apply(mut sut: Self::SystemUnderTest, _: &VecDeque<i32>, transition: Op) -> Self::SystemUnderTest {
|
||||
match transition {
|
||||
Op::Push(x) => sut.push(x),
|
||||
Op::Pop => { sut.pop(); }
|
||||
}
|
||||
sut
|
||||
}
|
||||
fn check_invariants(sut: &Self::SystemUnderTest, state: &VecDeque<i32>) {
|
||||
assert_eq!(sut.len(), state.len());
|
||||
// also check head/tail/iteration order...
|
||||
}
|
||||
}
|
||||
|
||||
proptest_state_machine::prop_state_machine! {
|
||||
#[test]
|
||||
fn queue_matches_vecdeque(sequential 1..50 => MyQueueSut);
|
||||
}
|
||||
```
|
||||
|
||||
You define a reference implementation (`VecDeque` here), proptest fuzzes operations against both, asserts invariants every step. This is the technique for finding bugs in lock-free or complex containers.
|
||||
|
||||
## Shrinking
|
||||
|
||||
When a property fails, proptest reduces the input to a minimal counter-example. For built-in strategies this is automatic. For custom strategies built with `prop_map`, shrinking goes through the underlying strategy. Avoid breaking shrinking with `prop_filter` (rejection sampling) over wide spaces; prefer `prop_flat_map` or directly-shaped strategies.
|
||||
|
||||
## Regression corpus
|
||||
|
||||
When a property test fails, proptest writes the failing input to `proptest-regressions/<test_name>.txt`. Commit this directory. Future runs replay these failing inputs first, so the bug stays fixed forever.
|
||||
|
||||
```
|
||||
proptest-regressions/
|
||||
└── parse_color.txt # commit this
|
||||
```
|
||||
|
||||
## Insta — setup
|
||||
|
||||
`Cargo.toml`:
|
||||
|
||||
```toml
|
||||
[dev-dependencies]
|
||||
insta = { version = "1", features = ["yaml", "json", "redactions", "filters"] }
|
||||
|
||||
[dependencies.serde_yaml]
|
||||
version = "0.9"
|
||||
optional = true
|
||||
```
|
||||
|
||||
Install the CLI:
|
||||
|
||||
```bash
|
||||
cargo install cargo-insta
|
||||
```
|
||||
|
||||
## Insta — basic snapshots
|
||||
|
||||
```rust
|
||||
#[test]
|
||||
fn renders_default_help() {
|
||||
let output = render_help();
|
||||
insta::assert_snapshot!(output);
|
||||
}
|
||||
```
|
||||
|
||||
First run: creates `src/snapshots/mycrate__renders_default_help.snap.new`. Run `cargo insta review`, press `a` to accept, the `.new` extension is dropped. Subsequent runs diff against the committed snapshot; mismatches fail the test.
|
||||
|
||||
## Insta — typed snapshots
|
||||
|
||||
```rust
|
||||
#[derive(Debug, serde::Serialize)]
|
||||
struct Result {
|
||||
status: String,
|
||||
user: User,
|
||||
duration_ms: u64,
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn json_response() {
|
||||
let value = compute();
|
||||
insta::assert_json_snapshot!(value);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn yaml_response() {
|
||||
insta::assert_yaml_snapshot!(value);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn debug_repr() {
|
||||
insta::assert_debug_snapshot!(value);
|
||||
}
|
||||
```
|
||||
|
||||
Choose:
|
||||
- `assert_snapshot!` for `String`/`Display` output (CLI help, error messages, generated code).
|
||||
- `assert_debug_snapshot!` for `{:?}` (Rust-internal data).
|
||||
- `assert_json_snapshot!` for structured data crossing process boundaries.
|
||||
- `assert_yaml_snapshot!` when YAML is easier to read in diffs.
|
||||
|
||||
## Insta — redactions and filters
|
||||
|
||||
For values that change every run (timestamps, UUIDs, paths):
|
||||
|
||||
```rust
|
||||
#[test]
|
||||
fn with_redactions() {
|
||||
let value = ApiResponse {
|
||||
id: uuid::Uuid::now_v7(),
|
||||
created_at: jiff::Timestamp::now(),
|
||||
body: "hello".into(),
|
||||
};
|
||||
insta::assert_json_snapshot!(value, {
|
||||
".id" => "[uuid]",
|
||||
".created_at" => "[timestamp]",
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
For regex filters applied to all snapshots in a test:
|
||||
|
||||
```rust
|
||||
#[test]
|
||||
fn with_filters() {
|
||||
let mut settings = insta::Settings::clone_current();
|
||||
settings.add_filter(r"/tmp/[a-z0-9-]+", "[TMP]");
|
||||
settings.add_filter(r"\d+\.\d+ms", "[TIMING]");
|
||||
settings.bind(|| {
|
||||
let output = run_command();
|
||||
insta::assert_snapshot!(output);
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
`Settings::bind` scopes filters to the closure.
|
||||
|
||||
## Insta workflow
|
||||
|
||||
1. Write the test, run it. First run creates `.snap.new`.
|
||||
2. `cargo insta review` → interactive UI. Show diff, accept/reject.
|
||||
3. Accepted snapshots commit to the repo.
|
||||
4. Refactor code. Tests run; mismatches show as diffs.
|
||||
5. If the new output is correct, `cargo insta accept` (or selective `review`). If wrong, fix the code.
|
||||
|
||||
Pair with CI to fail builds when uncommitted `.snap.new` files exist:
|
||||
|
||||
```bash
|
||||
cargo nextest run
|
||||
if find . -name "*.snap.new" | grep -q .; then
|
||||
echo "Pending snapshots, run 'cargo insta review'"
|
||||
exit 1
|
||||
fi
|
||||
```
|
||||
|
||||
## Inline snapshots
|
||||
|
||||
```rust
|
||||
#[test]
|
||||
fn small_output() {
|
||||
let value = compute();
|
||||
insta::assert_snapshot!(value, @"hello world");
|
||||
}
|
||||
```
|
||||
|
||||
The trailing `@"..."` string is the expected snapshot, stored in source. Useful when the value is short enough that pulling out a separate file is overkill. `cargo insta accept` updates them in-place.
|
||||
|
||||
## Inline JSON snapshots
|
||||
|
||||
```rust
|
||||
#[test]
|
||||
fn json_inline() {
|
||||
insta::assert_json_snapshot!(value, @r###"
|
||||
{
|
||||
"status": "ok",
|
||||
"count": 3
|
||||
}
|
||||
"###);
|
||||
}
|
||||
```
|
||||
|
||||
## Anti-patterns
|
||||
|
||||
1. **Snapshots of unstable output.** If `HashMap` iteration order changes per run, snapshots will fail. Switch to `BTreeMap` or sort before snapshotting.
|
||||
2. **Massive snapshots.** A 10KB JSON dump where you really care about 3 fields. Either narrow to the fields, or accept that any refactor will require re-reviewing 10KB.
|
||||
3. **Snapshots that bake in implementation details.** "function called 3 times" is not a snapshot - it's a behavior assertion. Use a real assertion.
|
||||
4. **Skipping `cargo insta review`.** Accepting blind via `cargo insta accept --all` defeats the purpose. Always review.
|
||||
|
||||
## Combining proptest + insta
|
||||
|
||||
```rust
|
||||
proptest! {
|
||||
#[test]
|
||||
fn random_inputs_render_consistently(input: ValidInput) {
|
||||
let mut settings = insta::Settings::clone_current();
|
||||
settings.set_snapshot_suffix(format!("{}", input.hash()));
|
||||
settings.bind(|| {
|
||||
insta::assert_snapshot!(render(&input));
|
||||
});
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
But honestly, this is rarely a fit. Proptest tests properties, insta tests output shape. Don't snapshot random inputs - that defeats both tools.
|
||||
|
||||
## CI matrix recommendation
|
||||
|
||||
```yaml
|
||||
- name: Tests
|
||||
run: cargo nextest run --all-features
|
||||
|
||||
- name: Property regressions (replay)
|
||||
run: |
|
||||
# The regression files in proptest-regressions/ replay first.
|
||||
# Failures here mean a previously-fixed bug came back.
|
||||
cargo nextest run --all-features --test-threads 1
|
||||
```
|
||||
|
||||
When a proptest finds a new failure, the regression file appears as a git diff - check it in.
|
||||
|
||||
## What proptest cannot do
|
||||
|
||||
- Find bugs that require multi-process / multi-network coordination → integration tests + fault injection.
|
||||
- Find concurrency bugs → use `loom` (see `concurrency.md`).
|
||||
- Find performance regressions → use `criterion`.
|
||||
|
||||
But for any function with a domain (inputs to outputs), proptest can find more bugs than your unit tests. **Write the property first, derive the unit test second.**
|
||||
@@ -0,0 +1,354 @@
|
||||
# Type-State and Newtype Patterns
|
||||
|
||||
The single highest-leverage thing Rust gives a coding agent: encode invariants in the type system so the compiler refuses incorrect code. The agent does not have to "remember" rules - the rules are physical.
|
||||
|
||||
## The Two Core Patterns
|
||||
|
||||
1. **Newtype wrappers for distinct semantic units.** Money, IDs, byte offsets, coordinate spaces - each gets its own tuple struct. The agent cannot pass meters where feet are expected, even though both are `f64` under the hood. This is the `euclid::Point<Screen>` vs `euclid::Point<World>` example Chris Allen called out.
|
||||
2. **Type-state for state machines.** Instead of a struct with a `status: enum { Draft, Validated, Persisted }` field and methods that check `if self.status == ...`, model each state as its own type. Transitions are method calls that consume `self` and return a new type. Illegal transitions become unrepresentable.
|
||||
|
||||
## Newtype Wrapper Cookbook
|
||||
|
||||
### Domain IDs
|
||||
|
||||
```rust
|
||||
use uuid::Uuid;
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, serde::Serialize, serde::Deserialize)]
|
||||
#[serde(transparent)]
|
||||
pub struct UserId(Uuid);
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, serde::Serialize, serde::Deserialize)]
|
||||
#[serde(transparent)]
|
||||
pub struct ProductId(Uuid);
|
||||
|
||||
impl UserId {
|
||||
pub fn new() -> Self { Self(Uuid::now_v7()) }
|
||||
pub fn as_uuid(&self) -> &Uuid { &self.0 }
|
||||
}
|
||||
|
||||
impl ProductId {
|
||||
pub fn new() -> Self { Self(Uuid::now_v7()) }
|
||||
pub fn as_uuid(&self) -> &Uuid { &self.0 }
|
||||
}
|
||||
|
||||
// `fn buy(user: UserId, product: ProductId)` cannot be called with arguments swapped.
|
||||
```
|
||||
|
||||
`#[serde(transparent)]` keeps JSON/SQL round-trips identical to a bare `Uuid` - the wrapper is purely a compile-time discipline.
|
||||
|
||||
### Quantities with Phantom Type Tags
|
||||
|
||||
```rust
|
||||
use core::marker::PhantomData;
|
||||
use core::ops::{Add, Sub, Mul};
|
||||
|
||||
#[derive(Debug, Clone, Copy)]
|
||||
pub struct Quantity<Unit> {
|
||||
raw: f64,
|
||||
_unit: PhantomData<Unit>,
|
||||
}
|
||||
|
||||
// Tag types - zero-sized, never instantiated.
|
||||
pub struct Meters;
|
||||
pub struct Feet;
|
||||
pub struct Seconds;
|
||||
|
||||
impl<U> Quantity<U> {
|
||||
pub const fn new(value: f64) -> Self {
|
||||
Self { raw: value, _unit: PhantomData }
|
||||
}
|
||||
pub fn raw(self) -> f64 { self.raw }
|
||||
}
|
||||
|
||||
// Adding same-unit quantities: allowed.
|
||||
impl<U> Add for Quantity<U> {
|
||||
type Output = Self;
|
||||
fn add(self, rhs: Self) -> Self { Self::new(self.raw + rhs.raw) }
|
||||
}
|
||||
|
||||
// Subtraction: allowed.
|
||||
impl<U> Sub for Quantity<U> {
|
||||
type Output = Self;
|
||||
fn sub(self, rhs: Self) -> Self { Self::new(self.raw - rhs.raw) }
|
||||
}
|
||||
|
||||
// Multiplying by scalar: allowed.
|
||||
impl<U> Mul<f64> for Quantity<U> {
|
||||
type Output = Self;
|
||||
fn mul(self, rhs: f64) -> Self { Self::new(self.raw * rhs) }
|
||||
}
|
||||
|
||||
// Conversions are explicit, named methods - never `From`/`Into` between units.
|
||||
impl Quantity<Meters> {
|
||||
pub fn to_feet(self) -> Quantity<Feet> {
|
||||
Quantity::new(self.raw * 3.280_84)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Now:
|
||||
|
||||
```rust
|
||||
let distance: Quantity<Meters> = Quantity::new(100.0);
|
||||
let height: Quantity<Feet> = Quantity::new(50.0);
|
||||
let combined = distance + height; // ❌ compile error
|
||||
let combined = distance + height.to_feet().to_meters_oops(); // ❌ no such method
|
||||
let combined = distance + distance; // ✅
|
||||
```
|
||||
|
||||
The agent cannot accidentally mix units. Refactors that change a quantity's underlying unit are caught at compile time everywhere the type flows.
|
||||
|
||||
### Byte Offsets vs Character Offsets
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)]
|
||||
pub struct ByteOffset(pub u32);
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)]
|
||||
pub struct CharOffset(pub u32);
|
||||
|
||||
impl ByteOffset {
|
||||
pub fn add(self, delta: u32) -> Self { Self(self.0 + delta) }
|
||||
}
|
||||
|
||||
// Converting between them is a function on the actual text.
|
||||
pub fn byte_to_char(text: &str, byte: ByteOffset) -> Option<CharOffset> {
|
||||
text.get(..byte.0 as usize).map(|prefix| CharOffset(prefix.chars().count() as u32))
|
||||
}
|
||||
```
|
||||
|
||||
A function signature `fn slice(text: &str, start: ByteOffset, end: ByteOffset)` cannot be called with character offsets. The UTF-8 boundary bug is now a compile error.
|
||||
|
||||
### Currency
|
||||
|
||||
```rust
|
||||
use rust_decimal::Decimal;
|
||||
|
||||
pub struct Krw;
|
||||
pub struct Usd;
|
||||
pub struct Jpy;
|
||||
|
||||
#[derive(Debug, Clone, Copy)]
|
||||
pub struct Money<Currency> {
|
||||
amount: Decimal,
|
||||
_ccy: PhantomData<Currency>,
|
||||
}
|
||||
|
||||
impl<C> Money<C> {
|
||||
pub const fn new(amount: Decimal) -> Self { Self { amount, _ccy: PhantomData } }
|
||||
}
|
||||
|
||||
impl<C> Add for Money<C> {
|
||||
type Output = Self;
|
||||
fn add(self, rhs: Self) -> Self { Self::new(self.amount + rhs.amount) }
|
||||
}
|
||||
|
||||
// No blanket From<Money<X>> for Money<Y> - conversions go through an explicit
|
||||
// FX rate function that takes a `Rate<From, To>` argument.
|
||||
|
||||
pub struct Rate<From, To> {
|
||||
factor: Decimal,
|
||||
_from: PhantomData<From>,
|
||||
_to: PhantomData<To>,
|
||||
}
|
||||
|
||||
impl<From, To> Money<From> {
|
||||
pub fn convert(self, rate: Rate<From, To>) -> Money<To> {
|
||||
Money::new(self.amount * rate.factor)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The agent cannot add KRW and USD by accident. They cannot convert without a rate. They cannot apply a USD→JPY rate to a KRW value.
|
||||
|
||||
### Paths Rooted at Different Bases
|
||||
|
||||
```rust
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
/// A path guaranteed to be relative to the project root.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct ProjectRel(PathBuf);
|
||||
|
||||
/// A path guaranteed to be relative to the user's home directory.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct HomeRel(PathBuf);
|
||||
|
||||
impl ProjectRel {
|
||||
pub fn new(path: impl AsRef<Path>) -> Result<Self, PathError> {
|
||||
let path = path.as_ref();
|
||||
if path.is_absolute() { return Err(PathError::NotRelative); }
|
||||
if path.components().any(|c| matches!(c, std::path::Component::ParentDir)) {
|
||||
return Err(PathError::EscapesRoot);
|
||||
}
|
||||
Ok(Self(path.to_path_buf()))
|
||||
}
|
||||
|
||||
pub fn resolve(&self, project_root: &Path) -> PathBuf {
|
||||
project_root.join(&self.0)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The agent's path-handling code now distinguishes between project-relative and home-relative paths at the type level. A function taking `ProjectRel` cannot be called with a `HomeRel`.
|
||||
|
||||
## Type-State State Machines
|
||||
|
||||
Encode the lifecycle of a value as a sequence of types. Each transition consumes the previous state and returns the next.
|
||||
|
||||
### HTTP Request Builder
|
||||
|
||||
```rust
|
||||
pub struct RequestBuilder<State> {
|
||||
url: String,
|
||||
method: Method,
|
||||
headers: HeaderMap,
|
||||
body: Option<Vec<u8>>,
|
||||
_state: PhantomData<State>,
|
||||
}
|
||||
|
||||
pub struct NeedsUrl;
|
||||
pub struct NeedsMethod;
|
||||
pub struct Ready;
|
||||
|
||||
impl RequestBuilder<NeedsUrl> {
|
||||
pub fn new() -> Self {
|
||||
Self {
|
||||
url: String::new(),
|
||||
method: Method::GET,
|
||||
headers: HeaderMap::new(),
|
||||
body: None,
|
||||
_state: PhantomData,
|
||||
}
|
||||
}
|
||||
pub fn url(mut self, url: impl Into<String>) -> RequestBuilder<NeedsMethod> {
|
||||
self.url = url.into();
|
||||
RequestBuilder { url: self.url, method: self.method, headers: self.headers, body: self.body, _state: PhantomData }
|
||||
}
|
||||
}
|
||||
|
||||
impl RequestBuilder<NeedsMethod> {
|
||||
pub fn method(mut self, method: Method) -> RequestBuilder<Ready> {
|
||||
self.method = method;
|
||||
RequestBuilder { url: self.url, method: self.method, headers: self.headers, body: self.body, _state: PhantomData }
|
||||
}
|
||||
}
|
||||
|
||||
// .header(), .body() available in any state that has at least URL.
|
||||
impl<S> RequestBuilder<S> {
|
||||
pub fn header(mut self, name: HeaderName, value: HeaderValue) -> Self {
|
||||
self.headers.insert(name, value);
|
||||
self
|
||||
}
|
||||
}
|
||||
|
||||
// .send() only available once URL and method are set.
|
||||
impl RequestBuilder<Ready> {
|
||||
pub async fn send(self, client: &Client) -> reqwest::Result<Response> { /* ... */ }
|
||||
}
|
||||
```
|
||||
|
||||
`client.send(RequestBuilder::new().send(...))` - compile error. The agent has to fill in the required steps. The IDE autocomplete also reflects only the legal next steps.
|
||||
|
||||
### File Handles
|
||||
|
||||
```rust
|
||||
pub struct File<State> {
|
||||
fd: RawFd,
|
||||
_state: PhantomData<State>,
|
||||
}
|
||||
|
||||
pub struct Open;
|
||||
pub struct Locked;
|
||||
pub struct Closed;
|
||||
|
||||
impl File<Open> {
|
||||
pub fn open(path: &Path) -> std::io::Result<Self> { /* ... */ }
|
||||
pub fn lock_exclusive(self) -> std::io::Result<File<Locked>> { /* flock */ }
|
||||
pub fn close(self) -> std::io::Result<File<Closed>> { /* ... */ }
|
||||
}
|
||||
|
||||
impl File<Locked> {
|
||||
pub fn read(&mut self, buf: &mut [u8]) -> std::io::Result<usize> { /* ... */ }
|
||||
pub fn write(&mut self, buf: &[u8]) -> std::io::Result<usize> { /* ... */ }
|
||||
pub fn unlock(self) -> std::io::Result<File<Open>> { /* ... */ }
|
||||
}
|
||||
|
||||
impl File<Closed> {
|
||||
// No methods. The type only exists to be dropped.
|
||||
}
|
||||
```
|
||||
|
||||
You cannot `read()` an unlocked file. You cannot `close()` while holding a lock without unlocking first. You cannot use a closed file at all - it has no methods.
|
||||
|
||||
## Sealed Traits
|
||||
|
||||
Sometimes you want a closed set of types implementing a trait, defined by the crate, not extensible by downstream. The sealed trait pattern:
|
||||
|
||||
```rust
|
||||
mod sealed {
|
||||
pub trait Sealed {}
|
||||
}
|
||||
|
||||
pub trait Renderer: sealed::Sealed {
|
||||
fn render(&self, frame: &mut Frame);
|
||||
}
|
||||
|
||||
pub struct OpenGl;
|
||||
pub struct Vulkan;
|
||||
pub struct Metal;
|
||||
|
||||
impl sealed::Sealed for OpenGl {}
|
||||
impl sealed::Sealed for Vulkan {}
|
||||
impl sealed::Sealed for Metal {}
|
||||
|
||||
impl Renderer for OpenGl { fn render(&self, f: &mut Frame) { /* ... */ } }
|
||||
impl Renderer for Vulkan { fn render(&self, f: &mut Frame) { /* ... */ } }
|
||||
impl Renderer for Metal { fn render(&self, f: &mut Frame) { /* ... */ } }
|
||||
```
|
||||
|
||||
Downstream code cannot add new `impl Renderer for Whatever` because they cannot implement `sealed::Sealed` (its module is private). Useful when you want trait dispatch but maintain the invariant that you control all implementations.
|
||||
|
||||
## NonEmpty Collections
|
||||
|
||||
```rust
|
||||
pub struct NonEmptyVec<T> {
|
||||
head: T,
|
||||
tail: Vec<T>,
|
||||
}
|
||||
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
#[error("vector was empty")]
|
||||
pub struct Empty;
|
||||
|
||||
impl<T> NonEmptyVec<T> {
|
||||
pub fn try_from_vec(mut v: Vec<T>) -> Result<Self, Empty> {
|
||||
if v.is_empty() { return Err(Empty); }
|
||||
let tail = v.split_off(1);
|
||||
let head = v.into_iter().next().expect("checked non-empty");
|
||||
Ok(Self { head, tail })
|
||||
}
|
||||
|
||||
pub fn first(&self) -> &T { &self.head }
|
||||
pub fn len(&self) -> usize { self.tail.len() + 1 }
|
||||
}
|
||||
```
|
||||
|
||||
Functions taking `NonEmptyVec<T>` cannot receive an empty vector. The `first()` method returns `&T`, not `Option<&T>`. The agent never has to write `match v.first() { Some(x) => ..., None => panic!(...) }` again.
|
||||
|
||||
## When NOT to Newtype
|
||||
|
||||
- For one-off internal computations where the unit lives in a single function and never crosses a boundary.
|
||||
- When the wrapper does not change behavior or invariants vs. the underlying type (e.g., a `struct Count(u32)` that is only ever used in one struct).
|
||||
- When `From`/`Into` conversions would be ergonomic but would defeat the purpose (if you find yourself wanting `impl From<UserId> for Uuid`, you do not want a newtype - you want a type alias).
|
||||
|
||||
The cost of a newtype is one tuple struct + the impls you need. The break-even is around three uses across different functions, or any use that crosses an API boundary.
|
||||
|
||||
## When NOT to Use Type-State
|
||||
|
||||
- When the state space is small and transitions are simple (`Option<T>` and `Result<T, E>` are state machines already).
|
||||
- When the type-state would force runtime branching upward (e.g., reading "should this run as Open or Locked?" from config means you store `Box<dyn FileLike>` anyway).
|
||||
- When the API is consumed by code that does not know the state at compile time (heterogeneous collections, dynamic dispatch boundaries).
|
||||
|
||||
In those cases, regular enum-tagged states are correct. The line is: **can the call site know the state statically?** If yes, type-state. If no, enum-with-tag.
|
||||
@@ -0,0 +1,250 @@
|
||||
# Unsafe Discipline
|
||||
|
||||
The reason Chris Allen's "implementing a persistent memory arena in Rust was not hard" works: the unsafe surface area is microscopic, it lives behind one newtype with one constructor, and every block has a SAFETY comment that names a specific invariant. Coding agents follow the pattern mechanically once the shape is established.
|
||||
|
||||
## The Three Required Components
|
||||
|
||||
Every `unsafe` block needs all three. No exceptions.
|
||||
|
||||
1. **Safe wrapper.** No `unsafe fn` or raw pointer types in the crate's public API. If a caller needs to construct an instance, the constructor either does the work safely or is `unsafe` with a documented contract.
|
||||
2. **SAFETY comment.** A `// SAFETY:` line within 5 lines above the `unsafe { ... }` block, stating which invariant is upheld and where it comes from. Generic phrases ("this is safe because we checked") fail review.
|
||||
3. **miri proof.** A test that exercises the unsafe path under `cargo +nightly miri nextest run`. If the path cannot be exercised under miri (FFI, syscalls), provide an alternate proof and gate behind a feature flag ending in `-skip-miri`.
|
||||
|
||||
## The Wrapper Pattern (`NonNull<T>` style)
|
||||
|
||||
Reference the screenshot Chris Allen quoted - `std::ptr::NonNull<T>`. Mirror this shape for every raw pointer, raw slice, raw transmute, or uninit memory operation in your own code.
|
||||
|
||||
```rust
|
||||
use core::marker::PhantomData;
|
||||
use core::ptr::NonNull;
|
||||
|
||||
/// A non-null, properly aligned, initialized pointer that does not alias.
|
||||
///
|
||||
/// All invariants are upheld by [`Self::new`] (checked) or [`Self::new_unchecked`]
|
||||
/// (delegated to the caller's contract). Once you hold an `InitPtr<T>`, every
|
||||
/// public method on it is safe to call.
|
||||
#[derive(Debug)]
|
||||
#[repr(transparent)]
|
||||
pub struct InitPtr<T> {
|
||||
inner: NonNull<T>,
|
||||
_marker: PhantomData<T>,
|
||||
}
|
||||
|
||||
// Send/Sync are NOT automatic for raw-pointer-bearing types. Decide deliberately.
|
||||
// SAFETY: `InitPtr<T>` owns no concurrency state of its own; whether it is
|
||||
// Send/Sync depends on `T`. The bounds below mirror `Box<T>`.
|
||||
unsafe impl<T: Send> Send for InitPtr<T> {}
|
||||
unsafe impl<T: Sync> Sync for InitPtr<T> {}
|
||||
|
||||
impl<T> InitPtr<T> {
|
||||
/// Wrap a raw pointer after checking alignment and non-null. The
|
||||
/// initialization invariant is not statically checkable here; callers must
|
||||
/// only feed pointers to memory that was written before this call.
|
||||
pub fn new(ptr: *mut T) -> Option<Self> {
|
||||
if !ptr.is_aligned() {
|
||||
return None;
|
||||
}
|
||||
// SAFETY: alignment checked above; `NonNull::new` filters null. The
|
||||
// caller is documented to provide an initialized location.
|
||||
NonNull::new(ptr).map(|inner| Self { inner, _marker: PhantomData })
|
||||
}
|
||||
|
||||
/// Wrap a raw pointer the caller asserts is valid.
|
||||
///
|
||||
/// # Safety
|
||||
///
|
||||
/// - `ptr` is non-null.
|
||||
/// - `ptr` is aligned to `align_of::<T>()`.
|
||||
/// - `*ptr` is initialized at the time of this call.
|
||||
/// - For the lifetime of the returned value, no other `&T` or `&mut T`
|
||||
/// aliases `*ptr`.
|
||||
pub unsafe fn new_unchecked(ptr: *mut T) -> Self {
|
||||
// SAFETY: caller upholds non-null per the function contract.
|
||||
Self { inner: unsafe { NonNull::new_unchecked(ptr) }, _marker: PhantomData }
|
||||
}
|
||||
|
||||
pub fn as_ref(&self) -> &T {
|
||||
// SAFETY: the constructor's invariants guarantee `inner` points at an
|
||||
// initialized, aligned, non-aliased `T`. Reborrowing through `&self`
|
||||
// ties the resulting lifetime to `self`, enforcing the rest via
|
||||
// standard borrow rules.
|
||||
unsafe { self.inner.as_ref() }
|
||||
}
|
||||
|
||||
pub fn as_mut(&mut self) -> &mut T {
|
||||
// SAFETY: `&mut self` proves no other reference can alias `inner` for
|
||||
// the lifetime of the returned `&mut T`; remaining invariants come
|
||||
// from construction.
|
||||
unsafe { self.inner.as_mut() }
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The list of features this single shape gives you:
|
||||
|
||||
- The agent cannot construct `InitPtr<T>` without going through a checked path or accepting the `unsafe` obligation explicitly.
|
||||
- The agent cannot leak the raw pointer; `as_ref` / `as_mut` return safe references with proper lifetimes.
|
||||
- The agent cannot accidentally Send/Sync where it shouldn't - the `unsafe impl` is explicit per-bound.
|
||||
- `#[repr(transparent)]` means the type is layout-compatible with `*mut T` for FFI, without exposing the raw pointer.
|
||||
|
||||
## SAFETY Comment Grammar
|
||||
|
||||
Every comment maps one-to-one to an invariant. Format:
|
||||
|
||||
```rust
|
||||
// SAFETY: <which invariant is upheld>: <how it is established here>.
|
||||
```
|
||||
|
||||
Anti-examples (do not pass review):
|
||||
|
||||
```rust
|
||||
// SAFETY: this is fine
|
||||
// SAFETY: we know what we're doing
|
||||
// SAFETY: the caller will not pass null
|
||||
// SAFETY: tested
|
||||
```
|
||||
|
||||
Good examples:
|
||||
|
||||
```rust
|
||||
// SAFETY: `len <= self.capacity` was checked at line 87 and `self.ptr` was
|
||||
// allocated by the same allocator we are reading through.
|
||||
|
||||
// SAFETY: `read_volatile` requires alignment and non-null; both hold because
|
||||
// `self.inner` is an `InitPtr<T>` whose constructor enforced them.
|
||||
|
||||
// SAFETY: We hold `&mut self`, so no concurrent reader exists. The slice
|
||||
// reference is dropped before the next `&self` borrow because we shrink the
|
||||
// returned scope manually.
|
||||
```
|
||||
|
||||
## Persistent Memory Arena Pattern (the Chris Allen example)
|
||||
|
||||
A persistent memory arena (PMA) is an `mmap`-backed bump allocator that survives process restarts. It is the classic "lots of unsafe under one safe surface" project.
|
||||
|
||||
Shape:
|
||||
|
||||
```rust
|
||||
pub struct Arena {
|
||||
map: Mmap, // wraps `mmap(2)` - safe wrapper from `memmap2` crate
|
||||
head: AtomicUsize, // current bump offset, atomic for multi-writer if needed
|
||||
}
|
||||
|
||||
impl Arena {
|
||||
pub fn open(path: &Path, capacity: usize) -> std::io::Result<Self> { /* mmap, init header */ }
|
||||
|
||||
pub fn alloc<T>(&self, value: T) -> Result<Handle<T>, ArenaFull> {
|
||||
let layout = Layout::new::<T>();
|
||||
let aligned = align_up(self.head.load(Acquire), layout.align());
|
||||
let next = aligned.checked_add(layout.size()).ok_or(ArenaFull)?;
|
||||
if next > self.map.len() { return Err(ArenaFull); }
|
||||
// CAS the head forward; retry on contention.
|
||||
// ... omitted for brevity ...
|
||||
|
||||
// SAFETY: `aligned + layout.size() <= self.map.len()` was just proven.
|
||||
// The mmap region is exclusively owned by this arena while we hold the
|
||||
// bump. `aligned` is aligned to `layout.align()` by `align_up`. No
|
||||
// other writer can have observed this offset because the CAS above
|
||||
// returned `Ok`.
|
||||
let ptr = unsafe { self.map.as_mut_ptr().add(aligned) as *mut T };
|
||||
// SAFETY: `ptr` is non-null (mmap base + offset), aligned (above),
|
||||
// exclusively owned (CAS), and we are about to initialize it.
|
||||
unsafe { ptr::write(ptr, value) };
|
||||
|
||||
// SAFETY: same invariants; we wrap the now-initialized pointer in the
|
||||
// safe handle which encapsulates further accesses.
|
||||
Ok(unsafe { Handle::new_unchecked(ptr, self) })
|
||||
}
|
||||
}
|
||||
|
||||
pub struct Handle<'a, T> {
|
||||
inner: InitPtr<T>,
|
||||
_arena: PhantomData<&'a Arena>,
|
||||
}
|
||||
```
|
||||
|
||||
Three `unsafe` blocks, three SAFETY comments, one safe handle type emerging on the other side. The agent now uses `Handle<T>` everywhere - never `*mut T`.
|
||||
|
||||
## Miri Invocation
|
||||
|
||||
```bash
|
||||
# install once
|
||||
rustup install nightly
|
||||
rustup component add miri rust-src --toolchain nightly
|
||||
|
||||
# run on every change that touches unsafe
|
||||
MIRIFLAGS="-Zmiri-strict-provenance -Zmiri-symbolic-alignment-check" \
|
||||
cargo +nightly miri nextest run --all-features
|
||||
```
|
||||
|
||||
What miri catches that the borrow checker cannot:
|
||||
|
||||
- Use-after-free
|
||||
- Double-free
|
||||
- Reads of uninitialized memory
|
||||
- Pointer-from-integer reconstruction that violates strict provenance
|
||||
- Alignment lies (transmuting unaligned data)
|
||||
- Stacked borrows / Tree borrows aliasing violations
|
||||
- Data races (single-threaded model, but catches concurrent access through `UnsafeCell` misuse)
|
||||
- Atomic ordering bugs in some patterns
|
||||
- Memory leaks (with `-Zmiri-track-pointer-tag`)
|
||||
|
||||
## When Miri Cannot Run
|
||||
|
||||
Certain paths are off-limits for miri: most syscalls beyond a curated allowlist, real network I/O, `std::process` calls, OS-specific FFI, hardware-dependent intrinsics on non-x86. Strategy:
|
||||
|
||||
1. **Isolate.** Put the un-mirifiable code in its own module behind `#[cfg(feature = "ffi-real")]` or similar.
|
||||
2. **Mock at the boundary.** For everything below the FFI boundary, write a safe Rust fake (a `Vec<u8>`-backed "disk", a fake clock, an in-memory socket pair). Expose it as a trait the production code consumes.
|
||||
3. **Test the fake under miri.** The fake implementation exercises the same logic minus the syscall. If the logic is unsafe (raw pointer manipulation in the fake "disk" buffer), miri catches the bug.
|
||||
4. **Test the real path under regular `cargo test`.** With `cargo nextest run --features ffi-real`. No miri, but the surface area is now just the syscall boundary.
|
||||
5. **Document.** A `# Safety` section in the rustdoc names the obligations the FFI puts on us, and a `# Testing` section explains the mock-vs-real split.
|
||||
|
||||
## Loom for Concurrency
|
||||
|
||||
When `unsafe` participates in a concurrent algorithm (lock-free queue, hazard pointers, custom Arc), miri's single-thread model is insufficient. Use `loom`:
|
||||
|
||||
```rust
|
||||
#[cfg(loom)]
|
||||
use loom::sync::atomic::{AtomicUsize, Ordering};
|
||||
#[cfg(not(loom))]
|
||||
use std::sync::atomic::{AtomicUsize, Ordering};
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
#[cfg(loom)]
|
||||
fn concurrent_push_pop() {
|
||||
loom::model(|| {
|
||||
let queue = std::sync::Arc::new(MyQueue::new());
|
||||
let q1 = queue.clone();
|
||||
let q2 = queue.clone();
|
||||
let h1 = loom::thread::spawn(move || q1.push(1));
|
||||
let h2 = loom::thread::spawn(move || q2.pop());
|
||||
h1.join().unwrap();
|
||||
h2.join().unwrap();
|
||||
});
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Run: `RUSTFLAGS="--cfg loom" cargo test --release`. Loom exhaustively explores thread interleavings for the test scope. Combined with miri on the single-thread paths, you have machine-checked soundness over the full state space.
|
||||
|
||||
## The Forbidden List
|
||||
|
||||
Reject in code review, automatic CI fail:
|
||||
|
||||
- `unsafe { ... }` with no SAFETY comment within 5 lines above.
|
||||
- `unsafe { unsafe_op_a(); unsafe_op_b(); }` (multiple unsafe ops in one block - split them, one SAFETY each). Clippy: `multiple_unsafe_ops_per_block`.
|
||||
- `unsafe fn` exposed publicly without a documented `# Safety` section in rustdoc.
|
||||
- `std::mem::transmute` for anything but lifetime extension on the same layout (and that should usually be `core::mem::transmute_copy` or `bytemuck::cast` if the relayout is well-defined).
|
||||
- `std::ptr::read_unaligned` / `write_unaligned` without a comment explaining why aligned access is impossible.
|
||||
- `from_raw_parts` / `from_raw_parts_mut` without proving the source pointer's provenance covers the entire slice.
|
||||
- `Arc::get_mut_unchecked`, `Box::leak` to bypass ownership, `MaybeUninit::assume_init` on partially-initialized data.
|
||||
- `unsafe impl Send`, `unsafe impl Sync` on types containing raw pointers, without a comment naming exactly which interior-mutability rule is upheld.
|
||||
- Any `unsafe` block whose justification depends on "in practice this never happens".
|
||||
|
||||
## The One-Line Summary
|
||||
|
||||
> Wrap once. Prove once. Test under miri. Never let `unsafe` escape.
|
||||
@@ -0,0 +1,527 @@
|
||||
# Zero-Cost Safety — Zig Ergonomics in Rust
|
||||
|
||||
Rust already owns memory safety. This reference adds the patterns that give you Zig's *ergonomic* safety — explicit allocation control, compile-time computation, zero-hidden-cost APIs, bit-level layout, and deterministic cleanup — without leaving the Rust toolchain.
|
||||
|
||||
**When to load this file:** arena, allocator, bumpalo, const fn, const generics, comptime, zero-alloc, no-alloc, slice-based API, `#[repr]`, packed struct, bitfield, scopeguard, errdefer, RAII cleanup, Zig-like patterns.
|
||||
|
||||
---
|
||||
|
||||
## 1. Explicit Allocators — Arena Pattern
|
||||
|
||||
Zig passes `allocator: Allocator` to every function. Rust's stable equivalent: arena crates that make allocation scope visible and bulk-freeable.
|
||||
|
||||
### bumpalo — The Default Arena
|
||||
|
||||
```rust
|
||||
use bumpalo::Bump;
|
||||
|
||||
fn parse_tokens<'a>(arena: &'a Bump, input: &[u8]) -> Vec<&'a str> {
|
||||
// All allocations go into `arena`. Caller controls lifetime.
|
||||
// When `arena` drops, everything frees in one shot.
|
||||
let token = arena.alloc_str("hello");
|
||||
let slice = arena.alloc_slice_copy(&[1u8, 2, 3]);
|
||||
vec![token] // Vec itself is on heap; contents point into arena
|
||||
}
|
||||
|
||||
// Usage: caller owns the arena, decides when memory dies.
|
||||
let arena = Bump::new();
|
||||
let tokens = parse_tokens(&arena, b"...");
|
||||
drop(arena); // all arena memory freed, zero individual deallocations
|
||||
```
|
||||
|
||||
**When to use:** parsers, compilers, game frame allocators, request-scoped web handlers, any hot loop where individual `Box`/`Vec` alloc+free overhead matters.
|
||||
|
||||
### typed-arena — Homogeneous Arena
|
||||
|
||||
```rust
|
||||
use typed_arena::Arena;
|
||||
|
||||
struct AstNode { kind: u8, children: Vec<&'static AstNode> } // simplified
|
||||
|
||||
let node_arena: Arena<AstNode> = Arena::new();
|
||||
let root = node_arena.alloc(AstNode { kind: 0, children: vec![] });
|
||||
// All nodes share arena lifetime. No individual free.
|
||||
```
|
||||
|
||||
**When to use:** tree/graph structures where all nodes have the same type and same lifetime.
|
||||
|
||||
### allocator_api (nightly) — Full Zig Parity
|
||||
|
||||
```rust
|
||||
#![feature(allocator_api)]
|
||||
use std::alloc::Global;
|
||||
|
||||
// Vec parameterized by allocator — exactly like Zig.
|
||||
let v: Vec<u8, &Bump> = Vec::new_in(&arena);
|
||||
|
||||
// Custom allocator for tracking, limiting, or redirecting allocation
|
||||
struct CountingAlloc { inner: Global, count: AtomicUsize }
|
||||
unsafe impl Allocator for CountingAlloc { /* ... */ }
|
||||
```
|
||||
|
||||
**When to use:** when you need allocator-generic data structures on nightly. For stable code, prefer `bumpalo` directly.
|
||||
|
||||
### Decision Tree
|
||||
|
||||
```
|
||||
Need arena allocation?
|
||||
├── All items same type, same lifetime → typed-arena
|
||||
├── Mixed types, same lifetime → bumpalo
|
||||
├── Need allocator-generic containers → allocator_api (nightly)
|
||||
└── Just need fewer allocations → SmallVec / ArrayVec / tinyvec (stack-first)
|
||||
```
|
||||
|
||||
### Cargo.toml
|
||||
|
||||
```toml
|
||||
bumpalo = { version = "3", features = ["collections"] }
|
||||
typed-arena = "2"
|
||||
smallvec = { version = "1", features = ["union", "const_generics"] }
|
||||
tinyvec = { version = "1", features = ["alloc"] }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. Compile-Time Computation — const fn, const generics, proc macros
|
||||
|
||||
Zig's `comptime` runs arbitrary code at compile time. Rust splits this across three mechanisms.
|
||||
|
||||
### const fn — Compile-Time Pure Functions
|
||||
|
||||
```rust
|
||||
const fn fibonacci(n: usize) -> usize {
|
||||
match n {
|
||||
0 => 0,
|
||||
1 => 1,
|
||||
_ => fibonacci(n - 1) + fibonacci(n - 2),
|
||||
}
|
||||
}
|
||||
|
||||
const FIB_20: usize = fibonacci(20); // computed at compile time: 6765
|
||||
|
||||
// Use in array sizes
|
||||
const LOOKUP: [u8; 256] = {
|
||||
let mut table = [0u8; 256];
|
||||
let mut i = 0;
|
||||
while i < 256 {
|
||||
table[i] = (i as u8).wrapping_mul(7);
|
||||
i += 1;
|
||||
}
|
||||
table
|
||||
};
|
||||
```
|
||||
|
||||
**Stable since Rust 1.82:** `const fn` supports `match`, loops, `if`, references, mutable locals — nearly full Rust. Use `const { }` blocks (Rust 1.79+) for inline compile-time assertions.
|
||||
|
||||
```rust
|
||||
fn process<const N: usize>(data: &[u8; N]) {
|
||||
const { assert!(N > 0, "N must be positive") }; // compile-time panic if N == 0
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
### const generics — Type-Level Values
|
||||
|
||||
```rust
|
||||
struct Buffer<const N: usize> {
|
||||
data: [u8; N],
|
||||
len: usize,
|
||||
}
|
||||
|
||||
impl<const N: usize> Buffer<N> {
|
||||
const fn new() -> Self {
|
||||
Self { data: [0; N], len: 0 }
|
||||
}
|
||||
|
||||
fn push(&mut self, byte: u8) -> Result<(), BufferFullError> {
|
||||
if self.len >= N { return Err(BufferFullError); }
|
||||
self.data[self.len] = byte;
|
||||
self.len += 1;
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
// Compiler enforces: Buffer<16> and Buffer<32> are distinct types.
|
||||
let small: Buffer<16> = Buffer::new();
|
||||
let large: Buffer<1024> = Buffer::new();
|
||||
```
|
||||
|
||||
### proc macros — Code Generation (Zig comptime type creation)
|
||||
|
||||
When `const fn` is not enough (generating struct fields, impl blocks, or derive logic), proc macros fill the gap.
|
||||
|
||||
```rust
|
||||
// In a proc-macro crate:
|
||||
use proc_macro::TokenStream;
|
||||
use quote::quote;
|
||||
use syn::{parse_macro_input, DeriveInput};
|
||||
|
||||
#[proc_macro_derive(Builder)]
|
||||
pub fn derive_builder(input: TokenStream) -> TokenStream {
|
||||
let input = parse_macro_input!(input as DeriveInput);
|
||||
let name = &input.ident;
|
||||
// ... generate builder struct and impl
|
||||
TokenStream::from(quote! {
|
||||
impl #name {
|
||||
pub fn builder() -> #name##Builder { /* ... */ }
|
||||
}
|
||||
})
|
||||
}
|
||||
```
|
||||
|
||||
**Decision tree:**
|
||||
|
||||
```
|
||||
Need compile-time value computation? → const fn
|
||||
Need type parameterized by value? → const generics
|
||||
Need to generate new types/impls? → proc macro (derive or attribute)
|
||||
Need compile-time string processing? → proc macro
|
||||
Need typenum-level arithmetic? → typenum / generic-array (rare)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. Zero-Allocation API Design — No Hidden Costs
|
||||
|
||||
Zig's philosophy: no operator overloading, no hidden allocation, every cost visible. Rust achieves this with discipline.
|
||||
|
||||
### Slice-Based APIs — Caller Owns Memory
|
||||
|
||||
```rust
|
||||
// BAD: hidden allocation in return type
|
||||
fn process(input: &str) -> String {
|
||||
input.to_uppercase() // allocates
|
||||
}
|
||||
|
||||
// GOOD: caller provides output buffer, zero allocation
|
||||
fn process(input: &[u8], output: &mut [u8]) -> usize {
|
||||
let len = input.len().min(output.len());
|
||||
for i in 0..len {
|
||||
output[i] = input[i].to_ascii_uppercase();
|
||||
}
|
||||
len // returns bytes written
|
||||
}
|
||||
|
||||
// GOOD: return borrowed data when possible
|
||||
fn find_token<'a>(input: &'a str) -> Option<&'a str> {
|
||||
input.split_whitespace().next() // no allocation — borrows from input
|
||||
}
|
||||
```
|
||||
|
||||
### try_* APIs — Fallible Allocation
|
||||
|
||||
```rust
|
||||
// Allocation can fail explicitly (like Zig's allocator returning error)
|
||||
let mut v = Vec::new();
|
||||
v.try_reserve(1_000_000)?; // returns Result, not panic
|
||||
|
||||
// For Box:
|
||||
let b = Box::try_new(42)?; // nightly, or use allocator_api
|
||||
```
|
||||
|
||||
### SmallVec / ArrayVec — Stack-First Collections
|
||||
|
||||
```rust
|
||||
use smallvec::SmallVec;
|
||||
use arrayvec::ArrayVec;
|
||||
|
||||
// SmallVec: stack for small counts, heap spillover for large
|
||||
let mut tags: SmallVec<[u8; 8]> = SmallVec::new();
|
||||
tags.push(1); // on stack if <= 8 elements
|
||||
|
||||
// ArrayVec: purely stack, fixed capacity, no heap ever
|
||||
let mut buf: ArrayVec<u8, 64> = ArrayVec::new();
|
||||
buf.try_push(42).map_err(|_| "full")?; // returns error instead of panic
|
||||
```
|
||||
|
||||
### Cow — Defer Allocation Until Mutation
|
||||
|
||||
```rust
|
||||
use std::borrow::Cow;
|
||||
|
||||
fn normalize(input: &str) -> Cow<'_, str> {
|
||||
if input.contains('\t') {
|
||||
Cow::Owned(input.replace('\t', " ")) // allocates only when needed
|
||||
} else {
|
||||
Cow::Borrowed(input) // zero-cost pass-through
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### The #![no_std] Discipline
|
||||
|
||||
For maximum allocation control, go `#![no_std]`:
|
||||
|
||||
```rust
|
||||
#![no_std]
|
||||
extern crate alloc; // opt-in to heap when needed
|
||||
|
||||
use alloc::vec::Vec; // explicit: I chose to allocate
|
||||
use alloc::string::String; // explicit: I chose to allocate
|
||||
```
|
||||
|
||||
Even in `std` code, the *mindset* applies: prefer `&[T]` over `Vec<T>` in function signatures, `&str` over `String`, `&Path` over `PathBuf`.
|
||||
|
||||
### Clippy Lints for Hidden Allocations
|
||||
|
||||
```toml
|
||||
# Cargo.toml — catch accidental allocations
|
||||
[lints.clippy]
|
||||
# These warn on patterns that allocate when a borrow would suffice:
|
||||
unnecessary_to_owned = "warn" # .to_string() / .to_vec() when borrow works
|
||||
redundant_clone = "warn" # .clone() that's immediately consumed
|
||||
large_stack_arrays = "warn" # accidental large stack usage
|
||||
vec_init_then_push = "warn" # Vec::new() + push instead of vec![]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. Bit-Level Layout — repr, Packed Structs, Bitfields
|
||||
|
||||
Zig: `packed struct` with bit-level field control. Rust matches with `#[repr]` attributes and bitfield crates.
|
||||
|
||||
### #[repr(C)] — Guaranteed C-Compatible Layout
|
||||
|
||||
```rust
|
||||
#[repr(C)]
|
||||
struct Header {
|
||||
magic: [u8; 4],
|
||||
version: u16,
|
||||
flags: u16,
|
||||
length: u32,
|
||||
}
|
||||
// Layout is C ABI: fields in declaration order, C padding rules.
|
||||
// Safe to transmute from/to byte arrays via zerocopy.
|
||||
```
|
||||
|
||||
### #[repr(C, packed)] — No Padding
|
||||
|
||||
```rust
|
||||
#[repr(C, packed)]
|
||||
struct WireHeader {
|
||||
tag: u8,
|
||||
length: u16, // NOT aligned to 2-byte boundary
|
||||
checksum: u32,
|
||||
}
|
||||
// Total size: exactly 7 bytes. No padding.
|
||||
// WARNING: taking &self.length is UB if unaligned. Use read_unaligned or zerocopy.
|
||||
```
|
||||
|
||||
**Safe access pattern:**
|
||||
|
||||
```rust
|
||||
use std::ptr;
|
||||
|
||||
impl WireHeader {
|
||||
fn length(&self) -> u16 {
|
||||
// SAFETY: packed field may be unaligned; ptr::read_unaligned handles this.
|
||||
unsafe { ptr::read_unaligned(ptr::addr_of!(self.length)) }
|
||||
}
|
||||
}
|
||||
|
||||
// Better: use zerocopy to avoid manual unsafe entirely
|
||||
use zerocopy::{FromBytes, IntoBytes, KnownLayout, Immutable};
|
||||
|
||||
#[derive(FromBytes, IntoBytes, KnownLayout, Immutable)]
|
||||
#[repr(C, packed)]
|
||||
struct WireHeader {
|
||||
tag: u8,
|
||||
length: [u8; 2], // manual byte array avoids alignment issues
|
||||
checksum: [u8; 4],
|
||||
}
|
||||
|
||||
impl WireHeader {
|
||||
fn length(&self) -> u16 { u16::from_le_bytes(self.length) }
|
||||
fn checksum(&self) -> u32 { u32::from_le_bytes(self.checksum) }
|
||||
}
|
||||
```
|
||||
|
||||
### bitfield — Bit-Level Flag Packing
|
||||
|
||||
```rust
|
||||
use bitfield::bitfield;
|
||||
|
||||
bitfield! {
|
||||
pub struct Permissions(u8);
|
||||
impl Debug;
|
||||
pub bool, readable, set_readable: 0;
|
||||
pub bool, writable, set_writable: 1;
|
||||
pub bool, executable, set_executable: 2;
|
||||
pub u8, level, set_level: 5, 3; // bits 3-5
|
||||
}
|
||||
|
||||
let mut p = Permissions(0);
|
||||
p.set_readable(true);
|
||||
p.set_level(5);
|
||||
assert!(p.readable());
|
||||
assert_eq!(p.level(), 5);
|
||||
```
|
||||
|
||||
### modular-bitfield — Richer Bitfield API
|
||||
|
||||
```rust
|
||||
use modular_bitfield::prelude::*;
|
||||
|
||||
#[bitfield(bits = 16)]
|
||||
#[derive(Debug)]
|
||||
pub struct StatusWord {
|
||||
ready: bool, // 1 bit
|
||||
error_code: B4, // 4 bits
|
||||
#[skip] __: B3, // 3 bits padding
|
||||
priority: B8, // 8 bits
|
||||
}
|
||||
```
|
||||
|
||||
### zerocopy — Safe Zero-Copy Parsing
|
||||
|
||||
```rust
|
||||
use zerocopy::{FromBytes, IntoBytes, KnownLayout, Immutable, Ref};
|
||||
|
||||
#[derive(FromBytes, IntoBytes, KnownLayout, Immutable)]
|
||||
#[repr(C)]
|
||||
struct Packet {
|
||||
header: [u8; 4],
|
||||
payload_len: u32,
|
||||
}
|
||||
|
||||
fn parse(bytes: &[u8]) -> Option<&Packet> {
|
||||
Ref::<_, Packet>::from_prefix(bytes).map(|(pkt, _rest)| pkt.into_ref()).ok()
|
||||
}
|
||||
// Zero-copy, zero-allocation, fully safe. No transmute, no pointer cast.
|
||||
```
|
||||
|
||||
### Cargo.toml
|
||||
|
||||
```toml
|
||||
zerocopy = { version = "0.8", features = ["derive"] }
|
||||
bitfield = "0.17"
|
||||
modular-bitfield = "0.11"
|
||||
bytemuck = { version = "1", features = ["derive"] } # alternative to zerocopy
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Scope Guards — errdefer / Deterministic Cleanup
|
||||
|
||||
Zig's `errdefer` runs cleanup only on error paths. Rust's `Drop` always runs, but `scopeguard` gives fine-grained control.
|
||||
|
||||
### scopeguard — The errdefer Equivalent
|
||||
|
||||
```rust
|
||||
use scopeguard::{defer, guard};
|
||||
use std::fs;
|
||||
|
||||
fn create_and_process(path: &str) -> std::io::Result<()> {
|
||||
let file = fs::File::create(path)?;
|
||||
// If anything below fails, clean up the file.
|
||||
// This is exactly Zig's errdefer.
|
||||
let _cleanup = guard((), |_| {
|
||||
let _ = fs::remove_file(path);
|
||||
});
|
||||
|
||||
write_data(&file)?;
|
||||
validate_data(path)?;
|
||||
|
||||
// Success: defuse the guard so it does NOT run cleanup.
|
||||
std::mem::forget(_cleanup);
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
### defer! — Always-Run Cleanup (like Zig's defer)
|
||||
|
||||
```rust
|
||||
use scopeguard::defer;
|
||||
|
||||
fn with_temp_dir() -> anyhow::Result<()> {
|
||||
let dir = tempfile::tempdir()?;
|
||||
defer! {
|
||||
// Runs when scope exits, success or failure.
|
||||
println!("Cleaning up {}", dir.path().display());
|
||||
// dir's Drop also cleans up, but this shows the pattern.
|
||||
}
|
||||
|
||||
do_work(dir.path())?;
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
### Drop as RAII Cleanup
|
||||
|
||||
```rust
|
||||
struct TempFile { path: std::path::PathBuf }
|
||||
|
||||
impl TempFile {
|
||||
fn new(path: impl Into<std::path::PathBuf>) -> std::io::Result<Self> {
|
||||
let path = path.into();
|
||||
std::fs::File::create(&path)?;
|
||||
Ok(Self { path })
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for TempFile {
|
||||
fn drop(&mut self) {
|
||||
let _ = std::fs::remove_file(&self.path);
|
||||
}
|
||||
}
|
||||
|
||||
// Usage: file is auto-cleaned when `tmp` goes out of scope.
|
||||
let tmp = TempFile::new("/tmp/scratch.dat")?;
|
||||
```
|
||||
|
||||
### The errdefer Pattern — Defuse on Success
|
||||
|
||||
The key insight from Zig's `errdefer`: you want cleanup on error but NOT on success. In Rust:
|
||||
|
||||
```rust
|
||||
use scopeguard::ScopeGuard;
|
||||
|
||||
fn deploy(artifact: &Path) -> Result<(), DeployError> {
|
||||
let backup = backup_current()?;
|
||||
|
||||
// errdefer: restore backup if anything fails
|
||||
let rollback = guard(backup.clone(), |b| {
|
||||
let _ = restore_from_backup(&b);
|
||||
});
|
||||
|
||||
upload(artifact)?;
|
||||
health_check()?;
|
||||
|
||||
// Success path: defuse the rollback guard
|
||||
ScopeGuard::into_inner(rollback);
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
### Cargo.toml
|
||||
|
||||
```toml
|
||||
scopeguard = "1"
|
||||
tempfile = "3" # idiomatic RAII temp files/dirs
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Summary: Zig Advantage → Rust Pattern
|
||||
|
||||
| Zig Feature | Rust Equivalent | Difficulty | Reference |
|
||||
|---|---|---|---|
|
||||
| Explicit allocator passing | `bumpalo` / `typed-arena` / `allocator_api` | Easy | §1 |
|
||||
| `comptime` value computation | `const fn` + `const { }` blocks | Easy | §2 |
|
||||
| `comptime` type generation | proc macros (derive / attribute) | Medium | §2 |
|
||||
| No hidden allocations | `#![no_std]` / slice-based APIs / `Cow` | Style choice | §3 |
|
||||
| `packed struct` / bitfields | `#[repr(C, packed)]` / `bitfield` / `zerocopy` | Easy | §4 |
|
||||
| `errdefer` | `scopeguard::guard` + defuse on success | Easy | §5 |
|
||||
| `defer` | `scopeguard::defer!` / `Drop` | Easy | §5 |
|
||||
|
||||
All achievable within Rust's single toolchain. You get Zig's explicitness **plus** the borrow checker, lifetime analysis, trait bounds, and `miri`. The combination is strictly more powerful than either alone.
|
||||
|
||||
## When NOT to Use These Patterns
|
||||
|
||||
- **Arena allocation** overkill for simple CLI tools that allocate once and exit.
|
||||
- **Zero-alloc APIs** hurt readability when the function naturally produces owned data. Don't force `&mut [u8]` output buffers on a function that logically returns `String`.
|
||||
- **`#[repr(packed)]`** only for wire formats and FFI. Never for regular domain types.
|
||||
- **Scope guards** unnecessary when `Drop` on the value itself handles cleanup (e.g., `tempfile::NamedTempFile` already does this).
|
||||
- **`const fn`** everything? No — only when the value is genuinely needed at compile time or the function is trivially const-eligible. Don't contort logic just to be const.
|
||||
|
||||
The goal is **visible costs and explicit control**, not asceticism. Use `String` and `Vec` freely when they're the right tool. Reach for these patterns when allocation behavior matters for correctness or performance.
|
||||
Reference in New Issue
Block a user