docs(omo-codex): batch 22 (3 files)
This commit is contained in:
YeonGyu-Kim
@@ -0,0 +1,289 @@
|
||||
|
||||
# Rust Undefined Behavior Exorcist
|
||||
|
||||
You are a UB hunter. Your job is to find, classify, prove, and eliminate every instance of undefined behavior in Rust code. **Miri is your primary weapon** — everything else supplements where Miri cannot reach.
|
||||
|
||||
## Core Philosophy
|
||||
|
||||
1. **Miri first, always.** Before reading a single line of `unsafe`, run Miri. Before proposing a fix, run Miri. After applying a fix, run Miri. Miri is the oracle.
|
||||
2. **Classify before fixing.** Every UB finding gets classified against the 14-category taxonomy (see [ub-taxonomy.md](ub-taxonomy.md)). This prevents misdiagnosis and ensures the fix targets the root cause, not a symptom.
|
||||
3. **Prove the fix.** A fix is not done until Miri passes with full paranoia flags. If Miri cannot run the test (FFI, I/O), the fix is not done until the appropriate sanitizer passes.
|
||||
4. **Bead handoff.** Each resolved UB instance is a "bead" — a discrete, documented, proven fix. Hand it off with: the UB category, the root cause, the fix, and the Miri proof.
|
||||
|
||||
## The UB Taxonomy
|
||||
|
||||
14 categories. The full reference is in [ub-taxonomy.md](ub-taxonomy.md). Memorize the categories; classify every finding:
|
||||
|
||||
| # | Category | Miri? |
|
||||
|---|----------|-------|
|
||||
| 1 | Aliasing violations (Stacked/Tree Borrows) | YES |
|
||||
| 2 | Data races | YES |
|
||||
| 3 | Use-after-free / dangling pointers | YES |
|
||||
| 4 | Uninitialized memory | YES |
|
||||
| 5 | Invalid values (type invariant violations) | YES |
|
||||
| 6 | Misaligned pointer access | YES |
|
||||
| 7 | Pin invariant violations | PARTIAL |
|
||||
| 8 | FFI boundary UB | LIMITED |
|
||||
| 9 | Incorrect Send/Sync implementations | YES (via race) |
|
||||
| 10 | Out-of-bounds memory access | YES |
|
||||
| 11 | Provenance violations | YES (strict mode) |
|
||||
| 12 | Double free / invalid free | YES |
|
||||
| 13 | Library / unsafe contract violations | PARTIAL |
|
||||
| 14 | Unwinding across extern "C" | PARTIAL |
|
||||
|
||||
## The Hunt Workflow
|
||||
|
||||
### Phase 1: Reconnaissance
|
||||
|
||||
1. **Find all `unsafe` blocks and `unsafe impl`s:**
|
||||
```bash
|
||||
rg 'unsafe\s*(fn|impl|{|\{)' --type rust -n
|
||||
```
|
||||
|
||||
2. **Find all `unsafe` trait implementations:**
|
||||
```bash
|
||||
rg 'unsafe\s+impl\s+(Send|Sync)' --type rust -n
|
||||
```
|
||||
|
||||
3. **Find transmute / pointer casts / raw pointer derefs:**
|
||||
```bash
|
||||
rg '(transmute|transmute_copy|from_raw|into_raw|as_ptr|as_mut_ptr|offset|add|sub|read|write|copy|ptr::null)' --type rust -n
|
||||
```
|
||||
|
||||
4. **Find FFI boundaries:**
|
||||
```bash
|
||||
rg 'extern\s+"C"' --type rust -n
|
||||
```
|
||||
|
||||
5. **Count and catalog.** Create a hit list: file, line, `unsafe` category, initial risk assessment (high/medium/low based on the UB taxonomy).
|
||||
|
||||
### Phase 2: Miri Sweep (THE CRITICAL PHASE)
|
||||
|
||||
Run Miri with escalating strictness. **Do not skip any level.**
|
||||
|
||||
**Level 1 — Default (Stacked Borrows):**
|
||||
```bash
|
||||
cargo +nightly miri test 2>&1
|
||||
```
|
||||
|
||||
**Level 2 — Strict Provenance + Symbolic Alignment:**
|
||||
```bash
|
||||
MIRIFLAGS="-Zmiri-strict-provenance -Zmiri-symbolic-alignment-check -Zmiri-backtrace=full" \
|
||||
cargo +nightly miri test 2>&1
|
||||
```
|
||||
|
||||
**Level 3 — Full Paranoia (the audit standard):**
|
||||
```bash
|
||||
MIRIFLAGS="\
|
||||
-Zmiri-strict-provenance \
|
||||
-Zmiri-symbolic-alignment-check \
|
||||
-Zmiri-preemption-rate=0.1 \
|
||||
-Zmiri-backtrace=full \
|
||||
-Zmiri-disable-isolation" \
|
||||
cargo +nightly miri test 2>&1
|
||||
```
|
||||
|
||||
**Level 4 — Tree Borrows (second model confirmation):**
|
||||
```bash
|
||||
MIRIFLAGS="\
|
||||
-Zmiri-tree-borrows \
|
||||
-Zmiri-strict-provenance \
|
||||
-Zmiri-symbolic-alignment-check \
|
||||
-Zmiri-preemption-rate=0.1 \
|
||||
-Zmiri-backtrace=full \
|
||||
-Zmiri-disable-isolation" \
|
||||
cargo +nightly miri test 2>&1
|
||||
```
|
||||
|
||||
**Interpret results:**
|
||||
- Fails at Level 1 → Definite UB. Fix immediately.
|
||||
- Passes Level 1, fails Level 2 → Provenance or alignment UB. Fix.
|
||||
- Passes Levels 1-3, fails Level 4 → Tree Borrows found something Stacked Borrows missed (unusual). Investigate — may be a Tree Borrows false positive, but usually indicates fragile aliasing.
|
||||
- Passes all 4 → Miri-clean. Proceed to supplementary tools.
|
||||
|
||||
### Phase 3: Supplementary Scans
|
||||
|
||||
For code Miri cannot fully cover:
|
||||
|
||||
**Concurrent code with custom atomics:**
|
||||
```bash
|
||||
RUSTFLAGS="--cfg loom" cargo test --lib --release -- loom_tests 2>&1
|
||||
```
|
||||
|
||||
**FFI-heavy code:**
|
||||
```bash
|
||||
RUSTFLAGS="-Zsanitizer=address" cargo +nightly test -Zbuild-std --target $(rustc -vV | rg host | awk '{print $2}') 2>&1
|
||||
```
|
||||
|
||||
**Untrusted input parsing:**
|
||||
```bash
|
||||
cargo +nightly fuzz run <target> -- -max_total_time=300 2>&1
|
||||
```
|
||||
|
||||
### Phase 4: Fix-and-Prove Loop
|
||||
|
||||
For each UB finding:
|
||||
|
||||
1. **Classify** against the 14-category taxonomy.
|
||||
2. **Write the SAFETY comment** explaining what is wrong and what the fix must achieve.
|
||||
3. **Apply the minimal fix.** Do not refactor — fix the UB and nothing else.
|
||||
4. **Run Miri (Level 3 minimum) on the specific test that triggered the UB.**
|
||||
5. **Run Miri (Level 3) on the full test suite** to check for regressions.
|
||||
6. **Document the bead:**
|
||||
```
|
||||
BEAD: [Category #] [Short description]
|
||||
FILE: [path:line]
|
||||
ROOT CAUSE: [one sentence]
|
||||
FIX: [one sentence]
|
||||
PROOF: Miri Level [N] pass — [command used]
|
||||
```
|
||||
|
||||
### Phase 5: Hardening (Post-Fix)
|
||||
|
||||
After all beads are resolved:
|
||||
|
||||
1. **Add Miri to CI** if not already present (see [miri-sanitizers-loom.md](miri-sanitizers-loom.md) for the GitHub Actions config).
|
||||
2. **Add `#[cfg(miri)]` regression tests** for each bead — these are the tests that originally caught the UB, locked in so it never returns.
|
||||
3. **Review SAFETY comments** on every remaining `unsafe` block. Each must name the specific invariant from the taxonomy.
|
||||
4. **Run the full paranoia sweep one final time** to confirm clean.
|
||||
|
||||
## Miri-First Decision Protocol
|
||||
|
||||
When the agent encounters `unsafe` code during ANY Rust task (not just audits):
|
||||
|
||||
```
|
||||
Is there unsafe code in the changeset?
|
||||
YES → Run Miri Level 1 before proceeding.
|
||||
│ Miri fails?
|
||||
│ YES → Stop. Classify. Fix. Prove. Then continue.
|
||||
│ NO → Run Miri Level 2 (strict provenance).
|
||||
│ Miri fails?
|
||||
│ YES → Stop. Classify. Fix. Prove. Then continue.
|
||||
│ NO → Proceed with the original task.
|
||||
NO → Proceed normally.
|
||||
```
|
||||
|
||||
This is not optional. **Every `unsafe` block gets Miri'd before it ships.**
|
||||
|
||||
## SAFETY Comment Standard
|
||||
|
||||
Every `unsafe` block requires a SAFETY comment within 5 lines above it. The comment must:
|
||||
|
||||
1. **Name the UB category** it could trigger (from the taxonomy).
|
||||
2. **State the invariant** that makes this safe.
|
||||
3. **Name who/what guarantees** the invariant (caller contract, type system, runtime check).
|
||||
|
||||
```rust
|
||||
// SAFETY: [Category 4 — Uninitialized Memory]
|
||||
// All N elements have been written to via `ptr::write` in the loop above.
|
||||
// The loop runs exactly `len` times, and `len` was validated against the
|
||||
// allocation size at line 42. MaybeUninit::assume_init is therefore sound.
|
||||
unsafe { buf.assume_init() }
|
||||
```
|
||||
|
||||
Bad SAFETY comments that must be rejected:
|
||||
- `// SAFETY: we know this is safe` — Says nothing.
|
||||
- `// SAFETY: this is fine because we tested it` — Testing does not prove absence of UB.
|
||||
- `// SAFETY: the caller ensures correctness` — Which invariant? What is the contract?
|
||||
- No SAFETY comment at all — Immediate failure.
|
||||
|
||||
## Audit Report Format
|
||||
|
||||
When completing a UB audit, produce a summary:
|
||||
|
||||
```markdown
|
||||
## UB Audit Report
|
||||
|
||||
**Scope:** [crate/module/file]
|
||||
**Miri version:** [output of `cargo +nightly miri --version`]
|
||||
**Date:** [date]
|
||||
|
||||
### Findings
|
||||
|
||||
| # | Category | File:Line | Severity | Status |
|
||||
|---|----------|-----------|----------|--------|
|
||||
| 1 | Aliasing | src/buf.rs:42 | High | Fixed (Bead #1) |
|
||||
| 2 | Uninit | src/ffi.rs:98 | High | Fixed (Bead #2) |
|
||||
|
||||
### Beads
|
||||
|
||||
#### Bead #1: Aliasing violation in buffer resize
|
||||
- **Root cause:** `&mut` created while `&` to same slice existed
|
||||
- **Fix:** Restructured to drop shared ref before taking mutable
|
||||
- **Proof:** `cargo +nightly miri test -- test_buffer_resize` passes Level 3
|
||||
|
||||
### Miri CI Status
|
||||
- [ ] Miri added to CI (Level 2 minimum)
|
||||
- [ ] All SAFETY comments reviewed
|
||||
- [ ] Regression tests added for each bead
|
||||
```
|
||||
|
||||
## Common Fix Patterns
|
||||
|
||||
### Aliasing → Use `UnsafeCell` or restructure borrows
|
||||
```rust
|
||||
// BEFORE (UB: &mut while & exists)
|
||||
let ptr = slice.as_ptr();
|
||||
let mut_ref = &mut slice[0]; // UB: ptr still usable
|
||||
|
||||
// AFTER
|
||||
let mut_ref = &mut slice[0];
|
||||
// ptr is never created / used across the mutable borrow
|
||||
```
|
||||
|
||||
### Uninitialized → Use `MaybeUninit::write` + `assume_init`
|
||||
```rust
|
||||
// BEFORE (UB: mem::uninitialized)
|
||||
let x: T = unsafe { std::mem::uninitialized() };
|
||||
|
||||
// AFTER
|
||||
let x: T = unsafe {
|
||||
let mut uninit = MaybeUninit::<T>::uninit();
|
||||
uninit.write(initial_value);
|
||||
uninit.assume_init()
|
||||
};
|
||||
```
|
||||
|
||||
### Provenance → Use `expose_provenance` / `with_exposed_provenance`
|
||||
```rust
|
||||
// BEFORE (UB: provenance lost)
|
||||
let addr = ptr as usize;
|
||||
let recovered = addr as *const T;
|
||||
|
||||
// AFTER
|
||||
let addr = ptr.expose_provenance();
|
||||
let recovered = std::ptr::with_exposed_provenance::<T>(addr);
|
||||
```
|
||||
|
||||
### Send/Sync → Remove manual impl, use PhantomData
|
||||
```rust
|
||||
// BEFORE (unsound)
|
||||
unsafe impl Send for MyType {}
|
||||
|
||||
// AFTER — if MyType truly needs Send, prove it:
|
||||
// SAFETY: [Category 9 — Send/Sync]
|
||||
// MyType's only non-Send field is `*mut Buffer`. Access to the buffer
|
||||
// is guarded by `self.lock: Mutex<()>`, which provides the
|
||||
// happens-before guarantee required by Send.
|
||||
unsafe impl Send for MyType {}
|
||||
```
|
||||
|
||||
### FFI → Validate at boundary
|
||||
```rust
|
||||
// BEFORE (UB: null pointer from C becomes &T)
|
||||
let result = unsafe { ffi_call() };
|
||||
|
||||
// AFTER
|
||||
let raw = unsafe { ffi_call() };
|
||||
let result = NonNull::new(raw).ok_or(Error::NullFromFfi)?;
|
||||
```
|
||||
|
||||
## Activation
|
||||
|
||||
This skill activates when:
|
||||
- The user requests a "UB audit", "miri sweep", "unsafe audit", "soundness check", "rustonomicon audit", "race hunt"
|
||||
- The agent encounters `unsafe` code during a Rust task and needs to verify it
|
||||
- Miri reports a failure and the agent needs to classify and fix it
|
||||
- The user asks "is this sound?" about Rust code
|
||||
|
||||
**Miri is not optional. Miri is the proof. Ship nothing `unsafe` without Miri's blessing.**
|
||||
+411
@@ -0,0 +1,411 @@
|
||||
# Miri, Sanitizers, Loom, and Fuzzing — The UB Detection Arsenal
|
||||
|
||||
Miri is the **primary weapon**. Everything else is supplementary for the gaps Miri cannot reach.
|
||||
|
||||
---
|
||||
|
||||
## Miri — The First and Last Line of Defense
|
||||
|
||||
### What Miri Is
|
||||
|
||||
Miri is an interpreter for Rust's MIR (Mid-level IR). It executes your test suite inside a virtual machine that tracks every byte of memory for validity, provenance, alignment, initialization, and aliasing. It is **deterministic** — same inputs, same result — and it can find UB that no amount of testing on real hardware will ever trigger.
|
||||
|
||||
### Why Miri Is Non-Negotiable
|
||||
|
||||
- Detects 12 of 14 UB categories (see `ub-taxonomy.md`).
|
||||
- Catches aliasing violations that compile and run correctly on every platform today but are UB that future compiler optimizations will exploit.
|
||||
- Catches data races under a configurable scheduling model.
|
||||
- Catches provenance violations that are impossible to observe on real hardware.
|
||||
- **Zero false positives** — if Miri says it is UB, it is UB. Period.
|
||||
|
||||
### Installation
|
||||
|
||||
```bash
|
||||
rustup install nightly
|
||||
rustup component add miri rust-src --toolchain nightly
|
||||
```
|
||||
|
||||
Verify:
|
||||
```bash
|
||||
cargo +nightly miri --version
|
||||
```
|
||||
|
||||
### Running Miri
|
||||
|
||||
**Default run (Stacked Borrows, standard checks):**
|
||||
```bash
|
||||
cargo +nightly miri test
|
||||
```
|
||||
|
||||
**With nextest (recommended for projects already using nextest):**
|
||||
```bash
|
||||
cargo +nightly miri nextest run
|
||||
```
|
||||
|
||||
**Specific test:**
|
||||
```bash
|
||||
cargo +nightly miri test -- test_name
|
||||
```
|
||||
|
||||
**Run a binary:**
|
||||
```bash
|
||||
cargo +nightly miri run
|
||||
```
|
||||
|
||||
### MIRIFLAGS — The Dial-Up Knobs
|
||||
|
||||
These flags are set via the `MIRIFLAGS` environment variable. The agent should use ALL of the strictness flags during a UB audit.
|
||||
|
||||
#### Aliasing Model
|
||||
|
||||
```bash
|
||||
# Default: Stacked Borrows (strict)
|
||||
cargo +nightly miri test
|
||||
|
||||
# Tree Borrows (newer, more permissive — use as a second pass)
|
||||
MIRIFLAGS="-Zmiri-tree-borrows" cargo +nightly miri test
|
||||
```
|
||||
|
||||
**Protocol:** Run Stacked Borrows first. If it fails, fix it. Then run Tree Borrows to confirm. Code that passes Stacked Borrows is sound under both models.
|
||||
|
||||
#### Strict Provenance
|
||||
|
||||
```bash
|
||||
MIRIFLAGS="-Zmiri-strict-provenance" cargo +nightly miri test
|
||||
```
|
||||
|
||||
Catches `ptr as usize as *const T` roundtrips where provenance is lost. **Should be ON for every audit.**
|
||||
|
||||
#### Symbolic Alignment Checks
|
||||
|
||||
```bash
|
||||
MIRIFLAGS="-Zmiri-symbolic-alignment-check" cargo +nightly miri test
|
||||
```
|
||||
|
||||
Catches alignment UB that happens to be aligned on your machine but is not guaranteed by the type system.
|
||||
|
||||
#### Data Race Detection Tuning
|
||||
|
||||
```bash
|
||||
# Increase preemption rate to stress-test race conditions
|
||||
MIRIFLAGS="-Zmiri-preemption-rate=0.5" cargo +nightly miri test
|
||||
|
||||
# Disable preemption (sequential scheduling — fewer races found but deterministic)
|
||||
MIRIFLAGS="-Zmiri-preemption-rate=0" cargo +nightly miri test
|
||||
```
|
||||
|
||||
#### The Full Paranoia Sweep (Use This for Audits)
|
||||
|
||||
```bash
|
||||
MIRIFLAGS="\
|
||||
-Zmiri-strict-provenance \
|
||||
-Zmiri-symbolic-alignment-check \
|
||||
-Zmiri-preemption-rate=0.1 \
|
||||
-Zmiri-backtrace=full \
|
||||
-Zmiri-disable-isolation" \
|
||||
cargo +nightly miri test
|
||||
```
|
||||
|
||||
Then a second pass with Tree Borrows:
|
||||
```bash
|
||||
MIRIFLAGS="\
|
||||
-Zmiri-tree-borrows \
|
||||
-Zmiri-strict-provenance \
|
||||
-Zmiri-symbolic-alignment-check \
|
||||
-Zmiri-preemption-rate=0.1 \
|
||||
-Zmiri-backtrace=full \
|
||||
-Zmiri-disable-isolation" \
|
||||
cargo +nightly miri test
|
||||
```
|
||||
|
||||
#### Isolation and I/O
|
||||
|
||||
Miri runs in isolation by default — no file I/O, no network, no system calls. If your tests need the filesystem:
|
||||
```bash
|
||||
MIRIFLAGS="-Zmiri-disable-isolation" cargo +nightly miri test
|
||||
```
|
||||
|
||||
Use sparingly — isolation is a feature, not a limitation. Tests that need I/O should have a separate `#[cfg(not(miri))]` path.
|
||||
|
||||
### Miri Limitations
|
||||
|
||||
| Cannot do | Workaround |
|
||||
|-----------|-----------|
|
||||
| Execute FFI / C code | ASAN, MSAN, Valgrind |
|
||||
| Run I/O-heavy tests (default) | `-Zmiri-disable-isolation` or `#[cfg(not(miri))]` |
|
||||
| Exhaustive interleaving exploration | loom |
|
||||
| Find performance bugs | criterion, flamegraph |
|
||||
| Run inline assembly | skip with `#[cfg(not(miri))]` |
|
||||
| Test OS-specific behavior | real hardware + sanitizers |
|
||||
|
||||
### Miri in CI
|
||||
|
||||
```yaml
|
||||
# GitHub Actions example
|
||||
miri:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: dtolnay/rust-toolchain@nightly
|
||||
with:
|
||||
components: miri, rust-src
|
||||
- name: Miri test (Stacked Borrows + strict provenance)
|
||||
run: |
|
||||
MIRIFLAGS="-Zmiri-strict-provenance -Zmiri-symbolic-alignment-check -Zmiri-backtrace=full" \
|
||||
cargo +nightly miri test
|
||||
- name: Miri test (Tree Borrows)
|
||||
run: |
|
||||
MIRIFLAGS="-Zmiri-tree-borrows -Zmiri-strict-provenance -Zmiri-symbolic-alignment-check -Zmiri-backtrace=full" \
|
||||
cargo +nightly miri test
|
||||
```
|
||||
|
||||
### Miri-Incompatible Test Gating
|
||||
|
||||
```rust
|
||||
#[test]
|
||||
#[cfg_attr(miri, ignore)] // Miri cannot run this (FFI, I/O, inline asm)
|
||||
fn test_requires_real_hardware() {
|
||||
// ...
|
||||
}
|
||||
|
||||
// Or conditionally compile the test body:
|
||||
#[test]
|
||||
fn test_with_miri_fallback() {
|
||||
#[cfg(miri)]
|
||||
{
|
||||
// Simplified version that avoids FFI
|
||||
}
|
||||
#[cfg(not(miri))]
|
||||
{
|
||||
// Full version with FFI
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Sanitizers — Where Miri Cannot Reach
|
||||
|
||||
Sanitizers are compiler instrumentation passes. They run your actual binary on real hardware with extra checks injected. Use them for FFI, I/O-heavy code, and integration tests.
|
||||
|
||||
### AddressSanitizer (ASAN)
|
||||
|
||||
Detects: use-after-free, buffer overflow, stack-use-after-return, double-free, memory leaks.
|
||||
|
||||
```bash
|
||||
RUSTFLAGS="-Zsanitizer=address" cargo +nightly test -Zbuild-std --target x86_64-unknown-linux-gnu
|
||||
```
|
||||
|
||||
On macOS:
|
||||
```bash
|
||||
RUSTFLAGS="-Zsanitizer=address" cargo +nightly test -Zbuild-std --target aarch64-apple-darwin
|
||||
```
|
||||
|
||||
### ThreadSanitizer (TSAN)
|
||||
|
||||
Detects: data races on non-atomic accesses across threads.
|
||||
|
||||
```bash
|
||||
RUSTFLAGS="-Zsanitizer=thread" cargo +nightly test -Zbuild-std --target x86_64-unknown-linux-gnu
|
||||
```
|
||||
|
||||
**When to use over Miri:** Integration tests involving real threads + real I/O + FFI. Miri's data-race detector is superior for pure-Rust code.
|
||||
|
||||
### MemorySanitizer (MSAN)
|
||||
|
||||
Detects: reads of uninitialized memory.
|
||||
|
||||
```bash
|
||||
RUSTFLAGS="-Zsanitizer=memory -Zsanitizer-memory-track-origins" cargo +nightly test -Zbuild-std --target x86_64-unknown-linux-gnu
|
||||
```
|
||||
|
||||
**When to use over Miri:** FFI code where C/C++ may return uninitialized memory into Rust.
|
||||
|
||||
### UndefinedBehaviorSanitizer (UBSAN)
|
||||
|
||||
Detects: integer overflow, misaligned access, null dereference, and other C/C++-style UB at the LLVM level.
|
||||
|
||||
```bash
|
||||
RUSTFLAGS="-Zsanitizer=undefined" cargo +nightly test -Zbuild-std --target x86_64-unknown-linux-gnu
|
||||
```
|
||||
|
||||
### Sanitizer Limitations
|
||||
|
||||
- Require nightly + `-Zbuild-std` (rebuilds the standard library with instrumentation).
|
||||
- MSAN requires ALL dependencies (including C libs) to be instrumented — practically hard.
|
||||
- Cannot catch aliasing violations (that is Miri's domain).
|
||||
- Significant runtime overhead (2-15x slower).
|
||||
- Linux has the best support; macOS works for ASAN; Windows support is minimal.
|
||||
|
||||
---
|
||||
|
||||
## Loom — Exhaustive Concurrency Testing
|
||||
|
||||
Loom explores all possible thread interleavings of a bounded concurrent program. It is mandatory for lock-free and wait-free primitives.
|
||||
|
||||
### When to Use Loom
|
||||
|
||||
- Any `unsafe` code involving atomics with ordering weaker than `SeqCst`.
|
||||
- Custom lock implementations.
|
||||
- Lock-free queues, stacks, or other concurrent data structures.
|
||||
- Any code where you chose `Relaxed`, `Acquire`, or `Release` ordering.
|
||||
|
||||
### When NOT to Use Loom
|
||||
|
||||
- Code using only `Mutex`/`RwLock` from std or `parking_lot` — the locks are sound, your usage is the question, and Miri + TSAN cover that.
|
||||
- Async code (loom does not model async runtimes — use `tokio::test` + Miri instead).
|
||||
|
||||
### Setup
|
||||
|
||||
```toml
|
||||
[dev-dependencies]
|
||||
loom = "0.7"
|
||||
```
|
||||
|
||||
### Loom Test Pattern
|
||||
|
||||
```rust
|
||||
#[cfg(loom)]
|
||||
mod loom_tests {
|
||||
use loom::sync::atomic::{AtomicUsize, Ordering};
|
||||
use loom::sync::Arc;
|
||||
use loom::thread;
|
||||
|
||||
#[test]
|
||||
fn concurrent_increment_is_sound() {
|
||||
loom::model(|| {
|
||||
let counter = Arc::new(AtomicUsize::new(0));
|
||||
|
||||
let threads: Vec<_> = (0..2).map(|_| {
|
||||
let c = counter.clone();
|
||||
thread::spawn(move || {
|
||||
c.fetch_add(1, Ordering::SeqCst);
|
||||
})
|
||||
}).collect();
|
||||
|
||||
for t in threads {
|
||||
t.join().unwrap();
|
||||
}
|
||||
|
||||
assert_eq!(counter.load(Ordering::SeqCst), 2);
|
||||
});
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Conditional Compilation for Loom
|
||||
|
||||
```rust
|
||||
#[cfg(loom)]
|
||||
use loom::sync::atomic::{AtomicUsize, Ordering};
|
||||
#[cfg(not(loom))]
|
||||
use std::sync::atomic::{AtomicUsize, Ordering};
|
||||
```
|
||||
|
||||
### Running Loom Tests
|
||||
|
||||
```bash
|
||||
# Loom tests only (use cfg flag)
|
||||
RUSTFLAGS="--cfg loom" cargo test --lib -- loom_tests
|
||||
|
||||
# With release optimizations (loom is slow)
|
||||
RUSTFLAGS="--cfg loom" cargo test --lib --release -- loom_tests
|
||||
```
|
||||
|
||||
### Loom + Miri Interaction
|
||||
|
||||
Loom and Miri solve different problems:
|
||||
- **Miri** checks a single execution for UB (aliasing, validity, provenance).
|
||||
- **Loom** checks all interleavings for correctness (ordering, atomicity).
|
||||
|
||||
Run BOTH on lock-free code:
|
||||
```bash
|
||||
# Step 1: loom for interleaving correctness
|
||||
RUSTFLAGS="--cfg loom" cargo test --lib --release -- loom_tests
|
||||
|
||||
# Step 2: Miri for UB in each path
|
||||
cargo +nightly miri test -- concurrent_tests
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Cargo-Fuzz — Property-Based UB Hunting
|
||||
|
||||
Fuzzing generates random inputs to maximize code coverage and find crashes, panics, and UB.
|
||||
|
||||
### Setup
|
||||
|
||||
```bash
|
||||
cargo install cargo-fuzz
|
||||
cargo fuzz init
|
||||
```
|
||||
|
||||
### Fuzz Target
|
||||
|
||||
```rust
|
||||
// fuzz/fuzz_targets/parse_input.rs
|
||||
#![no_main]
|
||||
use libfuzzer_sys::fuzz_target;
|
||||
|
||||
fuzz_target!(|data: &[u8]| {
|
||||
// Your parsing/deserialization/processing code here.
|
||||
// If it panics or triggers UB, the fuzzer catches it.
|
||||
let _ = my_crate::parse(data);
|
||||
});
|
||||
```
|
||||
|
||||
### Running
|
||||
|
||||
```bash
|
||||
# Run until interrupted
|
||||
cargo +nightly fuzz run parse_input
|
||||
|
||||
# Run with ASAN (catches memory bugs in unsafe code)
|
||||
cargo +nightly fuzz run parse_input -- -rss_limit_mb=4096
|
||||
|
||||
# Minimize a crashing input
|
||||
cargo +nightly fuzz tmin parse_input artifacts/parse_input/crash-xxxxx
|
||||
```
|
||||
|
||||
### Fuzz + Miri Pipeline
|
||||
|
||||
When the fuzzer finds a crashing input:
|
||||
1. Minimize it with `cargo fuzz tmin`.
|
||||
2. Add it as a regression test.
|
||||
3. Run the regression test under Miri to classify whether it is a panic (safe) or UB (must fix).
|
||||
|
||||
```bash
|
||||
# After adding the input as a test case:
|
||||
cargo +nightly miri test -- test_fuzz_regression_001
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Tool Selection Decision Tree
|
||||
|
||||
```
|
||||
Start
|
||||
│
|
||||
├── Is it pure Rust (no FFI, no I/O)?
|
||||
│ YES → Miri (full paranoia flags)
|
||||
│ │ └── Also: loom (if atomics/lock-free)
|
||||
│ │ └── Also: proptest (if parsing/serialization)
|
||||
│ │ └── Also: cargo-fuzz (if untrusted input)
|
||||
│ │
|
||||
│ NO → Does it involve FFI?
|
||||
│ YES → ASAN + MSAN on integration tests
|
||||
│ │ └── Miri on the Rust-side handling
|
||||
│ │ └── cbindgen in CI for layout verification
|
||||
│ │
|
||||
│ NO → Is it I/O-heavy?
|
||||
│ YES → TSAN for thread safety
|
||||
│ │ └── Miri with -Zmiri-disable-isolation where possible
|
||||
│ │
|
||||
│ NO → Miri (full paranoia flags)
|
||||
│
|
||||
└── Always: Miri is the default. Other tools supplement.
|
||||
```
|
||||
|
||||
## The One Rule
|
||||
|
||||
> **When in doubt, run Miri.** If Miri cannot run it, write a version it can run, and test that under Miri. Then test the real version under sanitizers. Never ship `unsafe` code that has not passed Miri.
|
||||
@@ -0,0 +1,269 @@
|
||||
# Rust Undefined Behavior Taxonomy
|
||||
|
||||
Every category of UB the Rust compiler, Miri, and the language specification recognize. The agent must know the full surface to hunt systematically. Each entry names the UB class, its root cause, canonical trigger, Miri detection status, and the canonical fix.
|
||||
|
||||
## 1. Aliasing Violations (Stacked Borrows / Tree Borrows)
|
||||
|
||||
**Root cause:** Two pointers access the same memory in ways that violate Rust's borrowing model — even through raw pointers inside `unsafe`.
|
||||
|
||||
**Canonical triggers:**
|
||||
- Creating a `&mut T` while another `&T` or `&mut T` to the same location exists.
|
||||
- Dereferencing a raw pointer derived from a reference after that reference was invalidated (e.g., `&mut` was retaken).
|
||||
- Calling `slice::from_raw_parts_mut` on overlapping regions.
|
||||
- Interior mutability through `UnsafeCell` without going through the `UnsafeCell` API.
|
||||
- Casting `&T` to `*mut T` and writing through it (even via FFI).
|
||||
|
||||
**Miri detection:** YES — Stacked Borrows is the default model. Tree Borrows (`-Zmiri-tree-borrows`) is the newer, more permissive model. Run both:
|
||||
```bash
|
||||
cargo +nightly miri test # Stacked Borrows (stricter)
|
||||
MIRIFLAGS="-Zmiri-tree-borrows" cargo +nightly miri test # Tree Borrows (relaxed)
|
||||
```
|
||||
If code passes Tree Borrows but fails Stacked Borrows, it is *likely* sound but *possibly* relying on unspecified behavior. Fix it anyway — Stacked Borrows is the conservative bet.
|
||||
|
||||
**Fix pattern:** Use `UnsafeCell` for all interior mutability. Never cast `&T` to `*mut T`. Derive mutable pointers from `*mut T` obtained via `UnsafeCell::get()` or `addr_of_mut!()`.
|
||||
|
||||
---
|
||||
|
||||
## 2. Data Races
|
||||
|
||||
**Root cause:** Two threads access the same non-atomic memory location, at least one is a write, and there is no happens-before ordering between them.
|
||||
|
||||
**Canonical triggers:**
|
||||
- `unsafe impl Send for T` on a type containing `*mut U` without synchronization.
|
||||
- `unsafe impl Sync for T` on a type containing `Cell<T>` or `UnsafeCell<T>` without a lock.
|
||||
- Using `std::ptr::write` from multiple threads to the same allocation.
|
||||
- Shared `&T` where `T` has interior mutability but no atomic/lock guard.
|
||||
|
||||
**Miri detection:** YES — Miri's data-race detector is on by default. It detects races on non-atomic accesses. For **preemptive scheduling** stress, use:
|
||||
```bash
|
||||
MIRIFLAGS="-Zmiri-preemption-rate=0.1" cargo +nightly miri test
|
||||
```
|
||||
|
||||
**Complementary tools:** `loom` for exhaustive interleaving exploration on lock-free algorithms. ThreadSanitizer (TSAN) for integration tests Miri cannot run (I/O, FFI).
|
||||
|
||||
**Fix pattern:** Wrap in `Mutex`/`RwLock`/`AtomicXxx`. Never `unsafe impl Sync` unless you can name the synchronization primitive guarding every mutable field.
|
||||
|
||||
---
|
||||
|
||||
## 3. Use After Free / Dangling Pointers
|
||||
|
||||
**Root cause:** A pointer or reference outlives the allocation it points to.
|
||||
|
||||
**Canonical triggers:**
|
||||
- Returning a reference to a local variable (compiler catches most, but raw pointers escape).
|
||||
- `Box::into_raw` → manual `Box::from_raw` with wrong lifetime.
|
||||
- `Vec` reallocation invalidating raw pointers obtained from `as_ptr()` / `as_mut_ptr()`.
|
||||
- `Pin<Box<T>>` unpinned and moved after self-referential pointers were set up.
|
||||
|
||||
**Miri detection:** YES — allocation tracking catches use-after-free on the exact operation.
|
||||
|
||||
**Fix pattern:** Borrow checker for references. For raw pointers: tie pointer validity to an explicit lifetime via a `PhantomData<&'a T>` in the wrapper, or use arena allocation (`bumpalo`) so all pointers share one lifetime.
|
||||
|
||||
---
|
||||
|
||||
## 4. Uninitialized Memory
|
||||
|
||||
**Root cause:** Reading a value from memory that was never written to.
|
||||
|
||||
**Canonical triggers:**
|
||||
- `MaybeUninit::assume_init()` before all bytes are written.
|
||||
- `mem::uninitialized()` (deprecated, still compiles).
|
||||
- `alloc::alloc(layout)` returns uninitialized memory — reading it before writing is UB.
|
||||
- Padding bytes in structs read via `transmute` or raw pointer casts.
|
||||
- `read_unaligned` on uninitialized memory.
|
||||
|
||||
**Miri detection:** YES — tracks initialization state per byte. Catches partial-init structs, padding reads, and premature `assume_init`.
|
||||
|
||||
**Fix pattern:** Use `MaybeUninit::zeroed()` when zero-init is acceptable. Write every field before calling `assume_init()`. Use `MaybeUninit::write()` instead of raw pointer writes. Never `transmute` structs with padding unless you zeroed the padding.
|
||||
|
||||
---
|
||||
|
||||
## 5. Invalid Values (Type Invariant Violations)
|
||||
|
||||
**Root cause:** Producing a value that violates the type's validity invariant.
|
||||
|
||||
**Canonical triggers:**
|
||||
- `bool` not 0 or 1.
|
||||
- `char` outside Unicode scalar range.
|
||||
- Enum discriminant not matching any variant.
|
||||
- `NonZeroU32` containing 0.
|
||||
- `&T` or `&mut T` that is null or dangling.
|
||||
- `str` containing non-UTF-8 bytes.
|
||||
- `fn` pointer that is null.
|
||||
|
||||
**Miri detection:** YES — validity checks are on by default. Extra strictness:
|
||||
```bash
|
||||
MIRIFLAGS="-Zmiri-strict-provenance" cargo +nightly miri test
|
||||
```
|
||||
|
||||
**Fix pattern:** Validate before transmuting. Use `TryFrom` at boundaries. Never `transmute` to enum types — use a checked conversion function.
|
||||
|
||||
---
|
||||
|
||||
## 6. Misaligned Pointer Access
|
||||
|
||||
**Root cause:** Dereferencing a pointer that is not aligned to the type's required alignment.
|
||||
|
||||
**Canonical triggers:**
|
||||
- Casting `*const u8` to `*const u64` and dereferencing (alignment goes from 1 to 8).
|
||||
- `#[repr(packed)]` struct field references (the compiler warns, but raw pointers bypass the warning).
|
||||
- Network buffer parsing where offsets are arbitrary.
|
||||
|
||||
**Miri detection:** YES — immediate trap on misaligned read/write.
|
||||
|
||||
**Fix pattern:** Use `read_unaligned` / `write_unaligned` for packed data. Use `bytemuck` or `zerocopy` for safe reinterpretation with alignment checks.
|
||||
|
||||
---
|
||||
|
||||
## 7. Violating `Pin` Invariants
|
||||
|
||||
**Root cause:** Moving a value that was pinned and relied on its address stability (self-referential types, intrusive linked lists).
|
||||
|
||||
**Canonical triggers:**
|
||||
- `mem::swap` on a `Pin<&mut T>` after `unsafe` deref.
|
||||
- Implementing `Unpin` for a type that contains self-referential pointers.
|
||||
- Manually calling `Pin::new_unchecked` on a movable allocation.
|
||||
|
||||
**Miri detection:** PARTIAL — Miri detects the resulting aliasing/use-after-free if the self-referential pointer is actually used. It does not detect "Pin contract violated but pointer was never dereferenced."
|
||||
|
||||
**Fix pattern:** Never `impl Unpin` for self-referential types. Use `pin_project` or `pin_project_lite` for safe pin projections. Review every `Pin::new_unchecked` call.
|
||||
|
||||
---
|
||||
|
||||
## 8. FFI Boundary UB
|
||||
|
||||
**Root cause:** Mismatch between Rust's ABI expectations and the foreign code's actual behavior.
|
||||
|
||||
**Canonical triggers:**
|
||||
- C function returning uninitialized memory into a Rust `&T`.
|
||||
- Wrong `#[repr(C)]` layout (padding differs between platforms).
|
||||
- Passing a Rust `enum` to C without `#[repr(C)]` or `#[repr(i32)]`.
|
||||
- Null pointer passed where C expects non-null (and Rust wraps it in `&T`).
|
||||
- C code writing to Rust-owned memory through a pointer Rust considers immutable.
|
||||
- Forgetting to mark FFI functions as `unsafe extern "C"`.
|
||||
- longjmp/setjmp across Rust frames (unwinding UB).
|
||||
|
||||
**Miri detection:** LIMITED — Miri cannot execute foreign code. It detects UB in the Rust-side handling of FFI return values.
|
||||
|
||||
**Complementary tools:** AddressSanitizer (ASAN), MemorySanitizer (MSAN) for detecting actual FFI-side corruption. Valgrind as a last resort.
|
||||
|
||||
**Fix pattern:** Validate every FFI return at the boundary. Use `Option<NonNull<T>>` for nullable pointers. Use `CStr`/`CString` for strings. Add `cbindgen` to CI to verify layout agreement. Wrap every FFI call in a safe Rust function that checks preconditions.
|
||||
|
||||
---
|
||||
|
||||
## 9. Incorrect `Send` / `Sync` Implementations
|
||||
|
||||
**Root cause:** Manually implementing `Send` or `Sync` for a type that does not actually uphold the required invariant.
|
||||
|
||||
**Canonical triggers:**
|
||||
- `unsafe impl Send for Wrapper(*mut T)` when `T` is not `Send`.
|
||||
- `unsafe impl Sync for Wrapper(UnsafeCell<T>)` without a lock, atomic, or other synchronization.
|
||||
- Types containing `Rc<T>` with a manual `Send` impl (Rc is explicitly !Send).
|
||||
|
||||
**Miri detection:** YES for the *resulting* data race if exercised. Miri's data-race detector will fire when two threads access the same location unsynchronized.
|
||||
|
||||
**Fix pattern:** Never manually implement `Send`/`Sync` unless you can write a SAFETY proof naming the synchronization mechanism. Use `PhantomData<*const ()>` to opt-out of auto-`Send`/`Sync` when in doubt.
|
||||
|
||||
---
|
||||
|
||||
## 10. Out-of-Bounds Memory Access
|
||||
|
||||
**Root cause:** Pointer arithmetic or indexing that escapes the allocation.
|
||||
|
||||
**Canonical triggers:**
|
||||
- `ptr.offset(n)` where `n` exceeds the allocation size.
|
||||
- `slice::from_raw_parts(ptr, len)` where `len` is too large.
|
||||
- Off-by-one in manual buffer management.
|
||||
- Integer overflow in size calculations leading to undersized allocation.
|
||||
|
||||
**Miri detection:** YES — allocation-precise bounds checking.
|
||||
|
||||
**Fix pattern:** Use checked arithmetic (`checked_add`, `checked_mul`) for size calculations. Use `slice::from_raw_parts` only with validated lengths. Prefer safe indexing (`get()`, iterators) over raw pointer arithmetic.
|
||||
|
||||
---
|
||||
|
||||
## 11. Provenance Violations
|
||||
|
||||
**Root cause:** Using a pointer whose provenance does not grant access to the target memory, even if the address is numerically correct.
|
||||
|
||||
**Canonical triggers:**
|
||||
- Casting an integer to a pointer and dereferencing it (`addr as *const T`).
|
||||
- Roundtripping a pointer through `usize` and back (`ptr as usize as *const T`) — the provenance is lost.
|
||||
- Using `ptr::from_exposed_addr` without a corresponding `ptr.expose_provenance()`.
|
||||
|
||||
**Miri detection:** YES with strict provenance:
|
||||
```bash
|
||||
MIRIFLAGS="-Zmiri-strict-provenance" cargo +nightly miri test
|
||||
```
|
||||
|
||||
**Fix pattern:** Use `ptr::with_exposed_provenance` / `ptr.expose_provenance()` for legitimate int-to-ptr roundtrips. Avoid `as usize as *const T` entirely. Use `sptr` crate for provenance-safe pointer manipulation on stable.
|
||||
|
||||
---
|
||||
|
||||
## 12. Double Free / Invalid Free
|
||||
|
||||
**Root cause:** Freeing the same allocation twice, or freeing memory not obtained from the allocator.
|
||||
|
||||
**Canonical triggers:**
|
||||
- `Box::from_raw` called twice on the same pointer.
|
||||
- Manual `dealloc` on a pointer already freed.
|
||||
- `ManuallyDrop` dropped explicitly then the outer type also drops it.
|
||||
|
||||
**Miri detection:** YES — immediate trap.
|
||||
|
||||
**Fix pattern:** Enforce single ownership via RAII. Use `ManuallyDrop` with extreme care — document who is responsible for the drop. Never clone a raw pointer and `Box::from_raw` both copies.
|
||||
|
||||
---
|
||||
|
||||
## 13. Library / Unsafe Contract Violations
|
||||
|
||||
**Root cause:** Violating the documented safety invariant of a safe or unsafe API, where the library author relied on the invariant for soundness.
|
||||
|
||||
**Canonical triggers:**
|
||||
- `Vec::set_len(n)` where the first `n` elements are not initialized.
|
||||
- `String::from_utf8_unchecked` on non-UTF-8 bytes.
|
||||
- `HashMap` key mutated after insertion (violates hash invariant — not UB per se, but unsound and Miri may detect downstream effects).
|
||||
- `BTreeMap` key with broken `Ord` impl (the standard library assumes a total order).
|
||||
|
||||
**Miri detection:** DEPENDS — Miri catches the downstream UB (e.g., reading uninitialized bytes from a `Vec` with inflated len). It does not catch "you violated the documented contract" if no memory-level UB results.
|
||||
|
||||
**Fix pattern:** Read the `# Safety` section of every `unsafe fn` you call. Document the invariant in your SAFETY comment. When in doubt, use the safe API and pay the cost.
|
||||
|
||||
---
|
||||
|
||||
## 14. Unwinding Across `extern "C"` Boundaries
|
||||
|
||||
**Root cause:** A Rust panic unwinding through a frame that uses the C calling convention.
|
||||
|
||||
**Canonical triggers:**
|
||||
- `panic!()` inside a `#[no_mangle] extern "C" fn` callback passed to C code.
|
||||
- `unwrap()` inside FFI callbacks.
|
||||
|
||||
**Miri detection:** PARTIAL — Miri does not model foreign unwinding, but it can detect the immediate UB if the panic reaches the FFI boundary.
|
||||
|
||||
**Fix pattern:** Use `std::panic::catch_unwind` at every FFI entry point. Mark FFI callbacks as `extern "C-unwind"` when panic propagation is intentional (nightly). Prefer returning `Result`-like error codes from FFI callbacks.
|
||||
|
||||
---
|
||||
|
||||
## Summary Table
|
||||
|
||||
| # | Category | Miri Detects? | Complementary Tool |
|
||||
|---|----------|--------------|-------------------|
|
||||
| 1 | Aliasing (Stacked/Tree Borrows) | YES | — |
|
||||
| 2 | Data races | YES | loom, TSAN |
|
||||
| 3 | Use-after-free / dangling | YES | ASAN |
|
||||
| 4 | Uninitialized memory | YES | MSAN |
|
||||
| 5 | Invalid values | YES | — |
|
||||
| 6 | Misaligned access | YES | UBSAN |
|
||||
| 7 | Pin invariant violation | PARTIAL | manual review |
|
||||
| 8 | FFI boundary UB | LIMITED | ASAN, MSAN, Valgrind |
|
||||
| 9 | Incorrect Send/Sync | YES (via race) | loom |
|
||||
| 10 | Out-of-bounds access | YES | ASAN |
|
||||
| 11 | Provenance violations | YES (strict mode) | — |
|
||||
| 12 | Double free | YES | ASAN |
|
||||
| 13 | Library contract violations | PARTIAL | proptest, fuzzing |
|
||||
| 14 | Unwinding across FFI | PARTIAL | — |
|
||||
|
||||
## Miri Coverage Assessment
|
||||
|
||||
Miri catches categories 1-6, 9-12 with high confidence. Categories 7, 8, 13, 14 require supplementary tools or manual audit. **Miri is the single highest-leverage tool** — it should run on every PR that touches `unsafe`, and ideally on the full test suite regularly.
|
||||
Reference in New Issue
Block a user