# Python Programmer

Modern Python. Type-strict, stack-first, async-correct.

## Philosophy

The type checker is your compiler. Make illegal states unrepresentable. Parse at boundaries. Own resources explicitly. Every function has a contract; the type system enforces it.

## Hard rules

These are deliberate project choices. Violations are always wrong, not "style preferences".

### Tooling

| Category | Use | Never |
|---|---|---|
| Package manager | `uv` | pip, poetry, conda, pipenv |
| Type checker | `basedpyright` (`typeCheckingMode = "all"`) | pyright, mypy |
| Linter + formatter | `ruff` (`select = ["ALL"]`) | flake8, black, isort, autopep8 |
| Async runtime | `anyio` | `import asyncio` |
| Data | `polars` + `duckdb` + `numpy` | pandas |
| Web framework | FastAPI + Pydantic v2 | Flask, Django REST |
| ORM | SQLAlchemy 2.x async | Django ORM, Tortoise |
| HTTP client | [`httpx2`](https://github.com/pydantic/httpx2) | requests, aiohttp, httpx |
| Testing | `pytest` | unittest |
| CLI | `typer` + `rich` | argparse, click, fire |

### The iron list

1. **Frozen by default** — `@dataclass(frozen=True, slots=True)`. Pydantic: `model_config = ConfigDict(frozen=True)`. Mutable only when mutation is the documented purpose.
2. **NewType for distinct IDs** — `UserId = NewType("UserId", int)`. Never pass raw `int` where a branded type exists.
3. **`match` only for variants, `if` only for booleans** — **NEVER** use `if/elif/else` to discriminate on type (`isinstance`), enum value, or literal variant. `match/case` is mandatory for these — non-negotiable. **ALWAYS** end with `case unreachable: assert_never(unreachable)` — bare `case _: pass` and `case _: raise ValueError` are banned (they silently swallow new variants). `if/else` is fine only for boolean expressions, range checks, and predicate calls that aren't variant discrimination. See "Why `if/elif` on variants is banned" below for examples.
4. **Protocol over ABC** — `typing.Protocol` for interfaces. ABC only when you need shared method implementation.
5. **No raw dicts in signatures** — params and returns use `TypedDict`, `dataclass`, or Pydantic model. Internal scratch dicts are fine.
6. **Parse, don't validate** — constructors produce typed objects or raise. Never pass unvalidated data deeper into the call stack.
7. **Typed errors** — error types are dataclasses or exceptions with typed fields. Never `raise ValueError("something")` with a bare string. Use union returns when the caller is within 1-2 call levels and must handle the outcome (repository → service). Use exceptions when the error should propagate up many layers to a boundary handler (service → HTTP handler).
8. **Final for constants** — module-level constants use `Final`. Mutable module globals are a code smell.
9. **Explicit None** — annotate `-> X | None`. Never return `None` from a function whose signature omits it.
10. **Context managers for resources** — files, DB connections, HTTP clients, locks. No manual `.close()`.
11. **No Any, no object** — both are banned as type annotations. `object` erases all structural information (zero callable attributes, zero narrowing). Use `Protocol` (structural typing), `TypeVar` (generic pass-through), explicit union (known variants), or `TypedDict` (dict shapes).
12. **No cast** — `cast()` is banned. Redesign the types.
13. **No type: ignore** — fix the type error. The checker is right; you are wrong.
14. **No broad except** — `except Exception` and `except BaseException` are banned. Catch the **specific** exception you expect. A broad catch swallows bugs you need to see — `KeyError`, `AttributeError`, `TypeError` all vanish silently. If you genuinely need a catch-all at a top-level boundary (CLI entry, HTTP handler), use `# noqa: BROAD_EXCEPT_OK` and log + re-raise.

### Typing and safety

- `basedpyright` in `typeCheckingMode = "all"`. Every public function has full annotations. Internal helpers: annotate return type; parameter types may be inferred.
- `ruff` with `select = ["ALL"]`. Override specific rules per project in `pyproject.toml`, never globally disable the strict baseline.
- Every new function must have a `docstring` unless its name + signature makes it completely obvious (e.g. `def full_name(first: str, last: str) -> str:`).
- Use `X | Y` union syntax (PEP 604), never `Union[X, Y]` or `Optional[X]`.

### Why `object` is banned

`object` pretends to be safe ("it's the top type!") but gives **zero** narrowing and **zero** attributes. Even `Any` is more honest — it admits the boundary is untyped.

```python
# BANNED
def process(data: object) -> object: ...
def store(items: list[object]) -> None: ...
results: dict[str, object] = {}

# GOOD — Protocol for structural typing
class Serializable(Protocol):
    def serialize(self) -> bytes: ...
def process(data: Serializable) -> ProcessResult: ...

# GOOD — TypeVar for generic pass-through
def identity[T](x: T) -> T: ...
def first[T](items: Sequence[T]) -> T: ...

# GOOD — explicit union for known variants
def parse(raw: str | bytes) -> Document: ...
```

### Why `if/elif` on variants is banned

`if/elif/else` chains on type, enum, or literal values lose compile-time exhaustiveness. When a new variant is added, nothing warns you. `match/case` + `assert_never` does.

```python
# BANNED — if/elif for type discrimination
if isinstance(event, Click):
    handle_click(event.x, event.y)
elif isinstance(event, Scroll):
    handle_scroll(event.delta)
else:
    raise ValueError(f"Unknown: {event}")  # runtime bomb

# BANNED — if/elif for enum discrimination
if status == Status.PENDING:
    start_review()
elif status == Status.ACTIVE:
    continue_processing()
elif status == Status.CLOSED:
    archive()

# BANNED — non-exhaustive match (swallows new variants)
match event:
    case Click(x, y): handle_click(x, y)
    case _: pass

# GOOD — exhaustive match with assert_never
match event:
    case Click(x=x, y=y):
        handle_click(x, y)
    case Scroll(delta=delta):
        handle_scroll(delta)
    case unreachable:
        assert_never(unreachable)

# GOOD — enum match
match status:
    case Status.PENDING: start_review()
    case Status.ACTIVE:  continue_processing()
    case Status.CLOSED:  archive()
    case unreachable:    assert_never(unreachable)
```

`if/else` is fine for boolean conditions and range checks — things that aren't variant discrimination:

```python
# FINE — boolean, not variant
if age >= 18:
    grant_access()
else:
    deny_access()
```

### Why broad `except` is banned

`except Exception` catches **every** non-system exception — `KeyError`, `TypeError`, `AttributeError`, `ValueError` all vanish. You lose the stack trace that would have told you exactly what went wrong. The fix is always to name the exception you expect.

```python
# BANNED — swallows bugs
try:
    result = api.fetch(url)
except Exception as e:
    logger.error(e)
    return None

# BANNED — catch-and-ignore
try:
    parse(data)
except Exception:
    pass

# GOOD — catch what you expect
try:
    result = api.fetch(url)
except httpx.HTTPStatusError as e:
    logger.error("API %d: %s", e.response.status_code, e.request.url)
    return None
except httpx.ConnectError:
    raise ServiceUnavailableError(service="api") from None

# GOOD — top-level boundary (only place broad catch is acceptable)
def main() -> int:  # noqa: BROAD_EXCEPT_OK
    try:
        return run()
    except Exception:
        logger.exception("unhandled error")
        return 1
```

### Async

- `import asyncio` is **BANNED**. Use `import anyio`.
- For background tasks, use `anyio.create_task_group`. Never fire-and-forget with `asyncio.create_task`.
- For concurrency gates, use `anyio.CapacityLimiter` (not `asyncio.Semaphore`).
- Load `async-anyio.md` when writing async code for the full pattern library.

### Data modeling — which container, when

All model fields carry type annotations. No `Any`, no untyped dicts in public APIs.
Use `polars` + `duckdb` for data. pandas is never the right answer in this stack.

| Situation | Use |
|---|---|
| User input, API request/response | `Pydantic BaseModel (frozen=True)` |
| Internal value object (no I/O) | `@dataclass(frozen=True, slots=True)` |
| Function with multiple outcomes | Union of frozen dataclasses + `match` |
| Dict shape for JSON compat / `**kwargs` | `TypedDict` |
| Fixed constants | `StrEnum` / `IntEnum` |
| Distinct primitive (UserId vs MovieId) | `NewType` |
| Contract / capability | `Protocol` |
| Contract + shared implementation | `ABC` |
| ORM model (SQLAlchemy) | `Mapped[]` — inherently mutable, `# noqa: MUTABLE_OK` |
| Config from env vars | `pydantic-settings BaseSettings` |

**The one rule**: data crosses trust boundary → Pydantic. Everything else → dataclass.

Load `data-modeling.md` for the full decision flowchart and comparison matrix.

### When frozen=True does not apply

- **ORM models** — SQLAlchemy `Mapped[]` requires mutation. Use `# noqa: MUTABLE_OK`.
- **Builder / accumulator** — object exists to be mutated (counter, buffer, state machine). Docstring must explain why.
- **Pydantic Settings** — tests override fields. Mutable is acceptable.

If you need `# noqa: MUTABLE_OK`, the class docstring must say why mutation is required.

### Libraries

Canonical defaults (override only if `pyproject.toml` explicitly picks something else):

| Domain | Library | Reason |
|---|---|---|
| CLI | `typer` | Type-annotated CLI from function sigs |
| Pretty output | `rich` | Tables, progress, tracebacks, markdown |
| HTTP client | [`httpx2`](https://github.com/pydantic/httpx2) | Next-gen HTTP client (Pydantic stewardship), HTTP/2, brotli+zstd. Always `httpx2[http2,brotli,zstd]`. See `httpx2-optimization.md` |
| Validation | `pydantic` v2 | Fast native validator, JSON Schema |
| Web API | `fastapi` | Async, Pydantic-native, OpenAPI |
| ORM | `sqlalchemy` 2.x async | `Mapped[]` types, async sessions |
| DB driver (Postgres) | `asyncpg` (via SQLAlchemy) | Fastest PG driver |
| AI agents | `pydantic-ai` | Typed deps, structured output |
| TUI | `textual` | Rich-based, CSS layout, widgets |
| Logging | `rich.logging.RichHandler` | Pretty; swap to `structlog` in prod |

## pyproject.toml — the one true config

Scaffold a new project with all strict defaults pre-configured:

```bash
uv run ../../scripts/python/new-project.py myproject
uv run ../../scripts/python/new-project.py myproject --path ./workspace
uv run ../../scripts/python/new-project.py myproject --lib   # publishable library
```

Creates via `uv init`, then injects basedpyright `typeCheckingMode = "all"` + ruff `select = ["ALL"]` + pytest strict. Cross-platform (macOS, Linux, Windows).

For manual setup: `uv init --app myproject`, then load `pyproject-strict.md`.

## PEP 723 — inline script metadata (mandatory for ALL scripts)

Every `.py` script — even throwaway — MUST use PEP 723 inline metadata with the `# ─── How to run ───` comment block. No venv, no `requirements.txt`. The script IS the environment spec. A script without the usage comment block is incomplete.

Scaffold with: `uv run ../../scripts/python/new-script.py <name> --deps "httpx2[http2,brotli,zstd]"` (writes to temp dir by default, `--output` for specific path).

Load `one-liners.md` for full patterns, examples, and anti-patterns.

## Reference loading

Load on demand — not all at once.

| Need | Load |
|---|---|
| Full pyproject.toml config | `pyproject-strict.md` |
| Type patterns (NewType, Final, enums, narrowing) | `type-patterns.md` |
| Data modeling (container choice, frozen, parse-don't-validate) | `data-modeling.md` |
| Error handling (typed errors, union returns, exhaustive match) | `error-handling.md` |
| Async patterns (anyio) | `async-anyio.md` |
| Data processing (polars / duckdb) | `data-processing.md` |
| FastAPI + SQLAlchemy stack | `fastapi-stack.md` |
| Library decision tree | `libraries.md` |
| **httpx2 optimization** (MUST load for any network code) | `httpx2-optimization.md` |
| **orjson** (when JSON is in the hot path; FastAPI/Pydantic v2 integration) | `orjson-stack.md` |
| One-liner scripts (PEP 723) | `one-liners.md` |
| PydanticAI agents | `pydantic-ai.md` |
| Textual TUI | `textual-tui.md` |

## httpx2 — mandatory for ALL network requests

Every outgoing HTTP call MUST use [`httpx2`](https://github.com/pydantic/httpx2) (`httpx2[http2,brotli,zstd]`). Never `requests`, never `aiohttp`, never the original `httpx`.

**ALL optimizations are ON by default — not optional, not progressive, not "nice to have".** A bare `httpx2.AsyncClient()` is a bug — treat it like a lint violation. The correct way is the factory pattern in `httpx2-optimization.md` with: HTTP/2 enabled, tuned connection pool (200/40/30s), split timeouts (5/30/10/10), transport retries (3), TCP_NODELAY, follow_redirects, and event hooks for observability.

When writing or reviewing ANY network code, **ALWAYS load `httpx2-optimization.md`** and use the factory pattern verbatim. No exceptions.

## No-excuse audit

Violations caught by `../../scripts/python/check-no-excuse-rules.py`. Run after every edit session.

| Rule ID | Catches | Opt-out |
|---|---|---|
| `cast-any` | `cast(Any, ...)` | None — redesign types |
| `type-ignore` | `# type: ignore` | None — fix the type |
| `pyright-ignore` | `# pyright: ignore` | None — fix the type |
| `bare-except` | `except:` with no class | None — name the exception |
| `silent-except` | `except X: pass` / `except X: ...` | None — handle or re-raise |
| `no-asyncio` | `import asyncio` | `# noqa: ANYIO_OK` |
| `no-pandas` | `import pandas` | `# noqa: PANDAS_OK` |
| `mutable-dataclass` | `@dataclass` without `frozen=True` | `# noqa: MUTABLE_OK` |
| `missing-slots` | `@dataclass` without `slots=True` | `# noqa: SLOTS_OK` |
| `raw-dict-return` | `-> dict` in function return type | `# noqa: DICT_OK` |
| `missing-assert-never` | `match` block without `assert_never` default | `# noqa: MATCH_OK` |
| `generic-exception` | `raise ValueError("...")` / `raise TypeError("...")` with bare string | `# noqa: GENERIC_ERR_OK` |
| `no-object` | `object` used as type annotation (param, return, generic arg) | `# noqa: OBJECT_OK` |
| `if-elif-on-variant` | `if isinstance()`/`if x == Enum.V` chain that should be `match/case` | `# noqa: IF_VARIANT_OK` |
| `oversized-module` | File exceeds 250 pure LOC (non-blank, non-comment) | `# noqa: SIZE_OK` |
| `broad-except` | `except Exception` / `except BaseException` (too broad) | `# noqa: BROAD_EXCEPT_OK` |

Fix every violation before declaring work done. basedpyright + ruff strict config catches the rest.

## In tests

Tests are strict too, with these exceptions (already configured in `pyproject.toml` per-file-ignores):

| In tests you may | Why |
|---|---|
| Use `assert` | That's how pytest works (`S101` ignored) |
| Use magic numbers | Test data (`PLR2004` ignored) |
| Access `_private` members | Testing internals (`SLF001` ignored) |
| Skip docstrings | Test names are the docs (`D` ignored) |
| Have unused function args | Fixtures (`ARG` ignored) |

Tests still follow the iron list — frozen dataclasses, typed errors, exhaustive match. If test fixtures need mutable state, use `# noqa: MUTABLE_OK` on the fixture class.

## Existing codebases

When editing an existing file that doesn't follow these rules: **write new code in strict style, don't refactor existing code in the same change.** Mixing feature work with style migration makes reviews harder and bugs likelier.

## Activation

This skill activates whenever you are writing or modifying any `.py` file. Even one-off scripts get the strict treatment — that is the whole point of PEP 723 + uv: production hygiene with throwaway ergonomics.