diff --git a/packages/shared-skills/skills/programming/references/go/README.md b/packages/shared-skills/skills/programming/references/go/README.md new file mode 100644 index 000000000..41257132e --- /dev/null +++ b/packages/shared-skills/skills/programming/references/go/README.md @@ -0,0 +1,90 @@ + +# Go Programmer + +Production Go in 2026. **Boring on purpose, strict by tooling, illegal states unrepresentable by convention.** + +## Philosophy + +Go gives you fewer type-system tools than Python, TypeScript, or Rust: + +- No sum types — only `interface{}` with type-switch. +- No exhaustiveness check from the compiler — only the `exhaustive` linter. +- No `Option` — only `nil` and the eternal trap of "is this nil interface or nil concrete?". +- No `Result` — only `(T, error)`, no compiler enforcement of unwrapping. +- No newtype that prevents primitive coercion — `type UserID string` is still implicitly convertible from a literal when used carelessly. + +**This is the whole point of the skill.** Where the language is weak, the linter bundle becomes the type checker, and code patterns become the type system. Treat `golangci-lint v2` with the configuration in `golangci-strict.md` as if it were `tsc --strict` or `basedpyright`. Treat `nilaway` and `go test -race` as if they were Miri. + +The skill enforces five non-negotiables: + +1. **Parse-don't-validate at every boundary.** HTTP/RPC/CLI/config gets parsed into a domain struct constructed only via `New*(...)` smart constructors. Once inside the domain, no further validation. See `data-modeling.md`. +2. **`(T, error)` everywhere.** No panics in library code. No bare `_ = err`. Errors are wrapped with `%w` and asserted with `errors.Is` / `errors.As`. Typed error structs for anything a caller can branch on. See `error-handling.md`. +3. **Sealed interfaces for variants.** Sum types via a sealed unexported method, dispatched through a `type switch`, with the `exhaustive` linter checking completeness. See `type-patterns.md`. +4. **`context.Context` is the first parameter.** Always. No `context.Background()` inside leaf functions. No goroutine without context-driven shutdown. No `time.Now()` in domain code — inject a clock. See `concurrency.md`. +5. **Generated, not hand-written, for external contracts.** `sqlc` for DB, `oapi-codegen` for OpenAPI servers and clients, `protoc-gen-go` + `protoc-gen-connect-go` for RPC. Hand-rolled marshalling is a regression. See `sqlc-pgx.md`, `grpc-connect.md`. + +## Hard rules — tooling + +| Category | Use | Never | +|---|---|---| +| Go version | **1.23+** (range-over-func, iter package, slog stable) | <1.22 | +| Module | `go modules` + `go work` for monorepos | dep, GOPATH layouts | +| Format | **`gofumpt`** (stricter gofmt) + `goimports -local ` | bare `gofmt` | +| Linter | **`golangci-lint v2`** with the strict bundle in `golangci-strict.md` | bare `go vet` | +| Nil checker | **`nilaway`** (Uber, stable since 2024) in CI | hope | +| Vet bundle | `go vet` + `fieldalignment` + `shadow` | "tests cover it" | +| Tests | `go test -race -shuffle=on -count=1` | `-count` cache, no race | +| Goroutine leaks | `go.uber.org/goleak` in `TestMain` | "looks fine" | +| Mock | `go.uber.org/mock` (gomock successor) | hand-written stubs | +| DB | `sqlc` + `jackc/pgx/v5` | `database/sql` + `gorm` | +| HTTP framework | **`gin-gonic/gin`** (de facto, ~48% of Go API repos) — `go-chi/chi` for minimalist, `connectrpc/connect-go` for RPC | `echo` (smaller eco), `fiber` (fasthttp = non-stdlib), `gorilla/mux` (in maintenance mode) | +| RPC | **`connectrpc/connect-go`** (gRPC-compatible, HTTP/1.1-friendly, browser-friendly) | hand-rolled `grpc-go` unless you specifically need bidi streaming features Connect lacks | +| Validation | `go-playground/validator/v10` for HTTP boundary + `bufbuild/protovalidate-go` for proto + smart constructors for domain | ad-hoc `if len(s) == 0` chains | +| Config | `caarlos0/env/v11` (struct-tag env) | `viper` unless you actually need file+env+flag merging | +| Logging | **`log/slog`** (stdlib, Go 1.21+) | logrus, zap, zerolog (all superseded) | +| CLI | `spf13/cobra` | hand-rolled `os.Args` parsing past 2 flags | +| TUI | `charm.land/bubbletea/v2` + `bubbles/v2` + `lipgloss/v2` — see `bubbletea-v2.md` for CJK/IME | bubbletea v1 if you need IME | + +A single CI command should be the gate: + +```bash +gofumpt -l . && \ + golangci-lint run ./... && \ + nilaway ./... && \ + go test -race -shuffle=on -count=1 ./... +``` + +If any of these fails, the change is not done. Period. The bundle is set up so a clean run actually means clean — see `golangci-strict.md` for the per-linter rationale and the deliberate `nolint:` policy. + +## Hard rules — code + +Read these per-file references for the canonical patterns: + +- **Types & data** → `type-patterns.md`, `data-modeling.md` — branded named types, smart constructors with unexported fields, sealed interfaces as sum types. +- **Errors** → `error-handling.md` — sentinel vs typed struct, `errors.Is/As`, `%w` wrapping, no panic in libraries, the `errorlint` ruleset. +- **Concurrency** → `concurrency.md` — `context.Context` discipline, `errgroup`, `sync.OnceValue`, `goleak`, `-race`, channel selection rules. +- **HTTP backend** → `backend-stack.md` — `gin` server skeleton, middleware ordering, SSE/streaming with `http.Flusher`, structured slog logging, graceful shutdown — distilled from the CLIProxyAPI codebase (a real proxy serving OpenAI/Gemini/Claude APIs). +- **RPC** → `grpc-connect.md` — when to pick Connect vs grpc-go, codegen pipeline, protovalidate, streaming. +- **DB** → `sqlc-pgx.md` — compile-time-safe SQL via sqlc + pgx connection pool + migrations via goose + testcontainers in CI. +- **CLI** → `cobra-stack.md` — cobra layout, slog integration, graceful shutdown on signals, fang-style colored help. +- **TUI** → `bubbletea-v2.md` — v2 model, `SetVirtualCursor(false)` + `tea.View{Cursor}` for CJK IME, why v1 was broken for Korean/Japanese/Chinese input. +- **Testing** → `testing.md` — table-driven tests, `require` vs `assert`, `autogold` snapshots, `gopter` property tests, `testcontainers` for integration, `goleak` for goroutine leaks. +- **Bootstrap** → `bootstrap.md` — `new-project.go` invocation, project layout (`cmd/`, `internal/`, `pkg/`), Taskfile, CI. +- **Strict config** → `golangci-strict.md` — the canonical `.golangci.yml` with the full linter whitelist and per-linter rationale. +- **One-liners** → `one-liners.md` — `go run` scripts with `//go:build ignore`, `gorun`-style invocation. + +## The 250 pure LOC ceiling + +Same rule as Python/Rust/TS: a `.go` file whose pure LOC (non-blank, non-comment) exceeds 250 is architecturally broken. Go encourages many small files in a single package, so this is *more* natural here than elsewhere — split by responsibility, keep one cohesive type and its methods per file. + +The `cmd/server/main.go` is the most common violator. Refactor it: `main.go` only wires `os.Args` → `cmd.Execute()`. Anything else lives in `internal/`. + +## Existing codebases — non-strict project + +When editing an existing `.go` file that doesn't follow these rules: **write new code in strict style, don't refactor existing code in the same change.** Use the `remove-ai-slops` skill for branch-scope cleanup. + +## Activation + +This skill activates whenever you write or modify any `.go` file, `go.mod`, `go.sum`, `.golangci.yml`, `Taskfile.yml`, or any of the codegen specs (`*.proto`, `*.sql` next to `sqlc.yaml`, `openapi.yaml` next to `oapi-codegen.yaml`). Even one-off scripts get the strict treatment — that is what `//go:build ignore` + `go run` is for: production hygiene with throwaway ergonomics. + +The references contain the recipes. **Read them before writing code. Re-read them when the model drifts.** The post-write architectural review loop is non-negotiable. diff --git a/packages/shared-skills/skills/programming/references/go/backend-stack.md b/packages/shared-skills/skills/programming/references/go/backend-stack.md new file mode 100644 index 000000000..a9ffd89c6 --- /dev/null +++ b/packages/shared-skills/skills/programming/references/go/backend-stack.md @@ -0,0 +1,641 @@ +# HTTP Backend Stack — gin + slog + validator + pgx + +The canonical production HTTP service skeleton. Distilled from the [CLIProxyAPI](https://github.com/router-for-me/CLIProxyAPI) codebase — a real proxy serving OpenAI / Gemini / Claude / Codex APIs in production, with SSE streaming, WebSocket upgrades, request logging, and hot-reload config. + +If you are tempted to pick echo or chi instead, see `libraries.md` — gin wins on ecosystem, not technical merit, and the win is large enough to matter. + +--- + +## `go.mod` + +```go +module github.com/your-org/myservice + +go 1.23 + +require ( + github.com/gin-gonic/gin v1.10.1 + github.com/go-playground/validator/v10 v10.22.1 + github.com/caarlos0/env/v11 v11.2.2 + github.com/google/uuid v1.6.0 + github.com/jackc/pgx/v5 v5.7.6 + golang.org/x/sync v0.18.0 +) +``` + +--- + +## Project structure + +``` +cmd/server/main.go # ≤ 50 LOC; flags → run.Execute(ctx) +internal/ + cmd/run.go # ~150 LOC; signal handling, config load, server.Run + config/config.go # env-driven Config struct + api/ + server.go # gin.Engine setup, route mounting, http.Server + middleware/ + request_id.go + request_logging.go + auth.go + recovery.go + cors.go + handlers/ + users.go # one file per resource + streams.go # SSE / WebSocket endpoints + domain/ # smart-constructor types (Email, UserID, ...) + service/ # business logic + store/ # pgx + sqlc + obs/ + logger.go # slog setup +``` + +--- + +## `cmd/server/main.go` + +```go +package main + +import ( + "context" + "log/slog" + "os" + "os/signal" + "syscall" + + "github.com/your-org/myservice/internal/cmd" +) + +func main() { + ctx, stop := signal.NotifyContext(context.Background(), + syscall.SIGINT, syscall.SIGTERM) + defer stop() + + if err := cmd.Execute(ctx); err != nil { + slog.Error("fatal", slog.Any("err", err)) + os.Exit(1) + } +} +``` + +That is the entire `main`. Anything more is a smell. + +--- + +## `internal/config/config.go` + +```go +package config + +import ( + "time" + "github.com/caarlos0/env/v11" +) + +type Config struct { + Host string `env:"HOST" envDefault:"0.0.0.0"` + Port int `env:"PORT" envDefault:"8080"` + DatabaseURL string `env:"DATABASE_URL,required"` + ReadTimeout time.Duration `env:"READ_TIMEOUT" envDefault:"15s"` + WriteTimeout time.Duration `env:"WRITE_TIMEOUT" envDefault:"30s"` + ShutdownTimeout time.Duration `env:"SHUTDOWN_TIMEOUT" envDefault:"20s"` + LogLevel string `env:"LOG_LEVEL" envDefault:"info"` + LogFormat string `env:"LOG_FORMAT" envDefault:"json"` + Env string `env:"ENV" envDefault:"development"` +} + +func Load() (Config, error) { + var cfg Config + if err := env.Parse(&cfg); err != nil { + return Config{}, err + } + return cfg, nil +} +``` + +--- + +## `internal/obs/logger.go` + +```go +package obs + +import ( + "context" + "log/slog" + "os" +) + +type ctxKey struct{ name string } +var requestIDKey = ctxKey{"request_id"} + +func NewLogger(level, format string) *slog.Logger { + var lvl slog.Level + _ = lvl.UnmarshalText([]byte(level)) + + opts := &slog.HandlerOptions{Level: lvl, AddSource: true} + + var h slog.Handler + switch format { + case "text": + h = slog.NewTextHandler(os.Stdout, opts) + default: + h = slog.NewJSONHandler(os.Stdout, opts) + } + return slog.New(&ctxHandler{Handler: h}) +} + +// ctxHandler pulls request_id from ctx into every log line. +type ctxHandler struct{ slog.Handler } + +func (h *ctxHandler) Handle(ctx context.Context, r slog.Record) error { + if id, ok := ctx.Value(requestIDKey).(string); ok && id != "" { + r.AddAttrs(slog.String("request_id", id)) + } + return h.Handler.Handle(ctx, r) +} + +func WithRequestID(ctx context.Context, id string) context.Context { + return context.WithValue(ctx, requestIDKey, id) +} +``` + +--- + +## `internal/api/server.go` + +```go +package api + +import ( + "context" + "fmt" + "log/slog" + "net/http" + + "github.com/gin-gonic/gin" + "github.com/your-org/myservice/internal/api/handlers" + "github.com/your-org/myservice/internal/api/middleware" + "github.com/your-org/myservice/internal/config" +) + +type Server struct { + cfg config.Config + srv *http.Server + logger *slog.Logger +} + +func New(cfg config.Config, logger *slog.Logger, h *handlers.Handler) *Server { + gin.SetMode(gin.ReleaseMode) + r := gin.New() + + // Middleware order matters — see "Middleware ordering" below. + r.Use( + middleware.RequestID(), // 1. assign request_id first + middleware.Recovery(logger), // 2. recovery wraps everything + middleware.RequestLogger(logger), + middleware.CORS(), + ) + + h.Mount(r) + + return &Server{ + cfg: cfg, + logger: logger, + srv: &http.Server{ + Addr: fmt.Sprintf("%s:%d", cfg.Host, cfg.Port), + Handler: r, + ReadTimeout: cfg.ReadTimeout, + WriteTimeout: cfg.WriteTimeout, + }, + } +} + +func (s *Server) Run(ctx context.Context) error { + errCh := make(chan error, 1) + go func() { + s.logger.InfoContext(ctx, "server starting", + slog.String("addr", s.srv.Addr)) + if err := s.srv.ListenAndServe(); err != nil && err != http.ErrServerClosed { + errCh <- err + } + close(errCh) + }() + + select { + case <-ctx.Done(): + s.logger.InfoContext(ctx, "shutdown signal received") + shutdownCtx, cancel := context.WithTimeout( + context.Background(), s.cfg.ShutdownTimeout) + defer cancel() + return s.srv.Shutdown(shutdownCtx) + case err := <-errCh: + return err + } +} +``` + +Notes: + +- `gin.New()` not `gin.Default()` — `Default()` adds `Logger()` (text format, not slog) and `Recovery()` (no logger injection). We replace both. +- `gin.SetMode(gin.ReleaseMode)` silences debug output. Production assumed. +- `http.Server` with explicit timeouts. The default `nil` timeouts are a DoS waiting to happen. +- Graceful shutdown: SIGINT/SIGTERM cancels the ctx → `Shutdown(shutdownCtx)` gives in-flight requests up to `ShutdownTimeout` to finish. + +--- + +## Middleware ordering — the rule that actually matters + +``` +RequestID → Recovery → Logger → CORS → Auth → Handler + (1) (2) (3) (4) (5) +``` + +1. **RequestID** is first so every subsequent middleware sees it. +2. **Recovery** wraps everything after it. Order: a panic in CORS still gets caught. +3. **Logger** sees the request_id and the recovered panic. +4. **CORS** before Auth — OPTIONS preflight must return without auth. +5. **Auth** is the last cross-cutting middleware. Per-route auth (admin-only) is mounted on a sub-router with extra middleware. + +```go +// Public routes — no auth +api := r.Group("/api/v1") +{ + api.POST("/auth/login", h.Login) + api.GET("/healthz", h.Healthz) +} + +// Authenticated routes +authed := r.Group("/api/v1", middleware.Auth(authSvc)) +{ + authed.GET("/users/:id", h.GetUser) + authed.POST("/users", h.CreateUser) +} + +// Admin-only routes +admin := r.Group("/api/v1/admin", + middleware.Auth(authSvc), + middleware.RequireRole("admin")) +{ + admin.GET("/users", h.ListAllUsers) +} +``` + +--- + +## Middleware examples + +### `middleware/request_id.go` + +```go +package middleware + +import ( + "github.com/gin-gonic/gin" + "github.com/google/uuid" + "github.com/your-org/myservice/internal/obs" +) + +func RequestID() gin.HandlerFunc { + return func(c *gin.Context) { + id := c.GetHeader("X-Request-ID") + if id == "" { + id = uuid.Must(uuid.NewV7()).String() + } + c.Request = c.Request.WithContext(obs.WithRequestID(c.Request.Context(), id)) + c.Header("X-Request-ID", id) + c.Next() + } +} +``` + +### `middleware/recovery.go` + +```go +package middleware + +import ( + "log/slog" + "net/http" + "runtime/debug" + + "github.com/gin-gonic/gin" +) + +func Recovery(logger *slog.Logger) gin.HandlerFunc { + return func(c *gin.Context) { + defer func() { + if r := recover(); r != nil { + logger.ErrorContext(c.Request.Context(), "panic recovered", + slog.Any("panic", r), + slog.String("stack", string(debug.Stack())), + ) + if !c.Writer.Written() { + c.JSON(http.StatusInternalServerError, + gin.H{"error": "internal_error"}) + } + c.Abort() + } + }() + c.Next() + } +} +``` + +### `middleware/request_logging.go` + +```go +func RequestLogger(logger *slog.Logger) gin.HandlerFunc { + return func(c *gin.Context) { + start := time.Now() + c.Next() + logger.InfoContext(c.Request.Context(), "http request", + slog.String("method", c.Request.Method), + slog.String("path", c.Request.URL.Path), + slog.Int("status", c.Writer.Status()), + slog.Int("bytes", c.Writer.Size()), + slog.Duration("elapsed", time.Since(start)), + slog.String("ip", c.ClientIP()), + ) + } +} +``` + +The `sloglint` linter enforces typed attrs (`slog.String(...)`) over `slog.Any("path", ...)`. Keep the form. + +### `middleware/cors.go` + +```go +func CORS() gin.HandlerFunc { + return func(c *gin.Context) { + c.Header("Access-Control-Allow-Origin", "*") + c.Header("Access-Control-Allow-Methods", "GET, POST, PUT, PATCH, DELETE, OPTIONS") + c.Header("Access-Control-Allow-Headers", "*") + if c.Request.Method == http.MethodOptions { + c.AbortWithStatus(http.StatusNoContent) + return + } + c.Next() + } +} +``` + +Note the explicit OPTIONS short-circuit — preflight must NOT go through Auth. + +--- + +## Handlers — the canonical shape + +```go +package handlers + +import ( + "errors" + "net/http" + + "github.com/gin-gonic/gin" + "github.com/go-playground/validator/v10" + "github.com/your-org/myservice/internal/domain" + "github.com/your-org/myservice/internal/httperr" + "github.com/your-org/myservice/internal/service" +) + +type Handler struct { + Users *service.UserService +} + +func (h *Handler) Mount(r gin.IRouter) { + api := r.Group("/api/v1") + api.POST("/users", h.CreateUser) + api.GET("/users/:id", h.GetUser) +} + +type createUserReq struct { + Email string `json:"email" binding:"required,email"` + Username string `json:"username" binding:"required,alphanum,min=3,max=32"` +} + +func (h *Handler) CreateUser(c *gin.Context) { + var req createUserReq + if err := c.ShouldBindJSON(&req); err != nil { + writeBindingError(c, err) + return + } + + email, err := domain.NewEmail(req.Email) + if err != nil { + httperr.Write(c, err) + return + } + username, err := domain.NewUsername(req.Username) + if err != nil { + httperr.Write(c, err) + return + } + + user, err := h.Users.Create(c.Request.Context(), email, username) + if err != nil { + httperr.Write(c, err) + return + } + c.JSON(http.StatusCreated, user) +} + +func writeBindingError(c *gin.Context, err error) { + var vErr validator.ValidationErrors + if errors.As(err, &vErr) { + out := make(map[string]string, len(vErr)) + for _, fe := range vErr { + out[fe.Field()] = fe.Tag() + } + c.JSON(http.StatusBadRequest, gin.H{"errors": out}) + return + } + c.JSON(http.StatusBadRequest, gin.H{"error": "invalid_json"}) +} +``` + +See `data-modeling.md` for the validator tag reference; see `error-handling.md` for the `httperr.Write` funnel. + +--- + +## SSE streaming — the production pattern + +CLIProxyAPI streams OpenAI-compatible SSE for hundreds of concurrent clients. The pattern: + +```go +func (h *Handler) StreamChat(c *gin.Context) { + ctx, cancel := context.WithCancel(c.Request.Context()) + defer cancel() + + // 1. Set SSE headers BEFORE writing any body + c.Header("Content-Type", "text/event-stream") + c.Header("Cache-Control", "no-cache") + c.Header("Connection", "keep-alive") + c.Header("X-Accel-Buffering", "no") // disable nginx buffering + + // 2. Obtain the flusher — REQUIRED for streaming + flusher, ok := c.Writer.(http.Flusher) + if !ok { + httperr.Write(c, errors.New("streaming unsupported")) + return + } + + // 3. Pull chunks from upstream + chunks, errs := h.svc.StreamCompletions(ctx, req) + + for { + select { + case <-ctx.Done(): + return // client disconnected, ctx cancelled + case chunk, ok := <-chunks: + if !ok { + fmt.Fprint(c.Writer, "data: [DONE]\n\n") + flusher.Flush() + return + } + fmt.Fprintf(c.Writer, "data: %s\n\n", chunk) + flusher.Flush() + case err := <-errs: + // Error mid-stream — emit as SSE event and bail + fmt.Fprintf(c.Writer, "event: error\ndata: %s\n\n", err.Error()) + flusher.Flush() + return + } + } +} +``` + +Key facts: + +- **Headers MUST be set before the first `Write`.** Otherwise gin auto-sets `Content-Type: text/plain`. +- **`c.Writer.(http.Flusher)` is the streaming primitive.** Without `flusher.Flush()`, the response is buffered and arrives as one blob at the end. +- **Always respond to `<-ctx.Done()`.** A disconnected client must stop upstream work — otherwise you generate tokens for nothing. +- **The trailing `\n\n` per event is wire-mandatory** for SSE parsing. Missing it = the client never sees the event. + +--- + +## WebSocket upgrade + +```go +import "github.com/gorilla/websocket" // still the canonical WS lib in 2026 + +var upgrader = websocket.Upgrader{ + ReadBufferSize: 4096, + WriteBufferSize: 4096, + CheckOrigin: func(r *http.Request) bool { + // tighten in production + return true + }, +} + +func (h *Handler) WebSocketEcho(c *gin.Context) { + conn, err := upgrader.Upgrade(c.Writer, c.Request, nil) + if err != nil { + slog.ErrorContext(c.Request.Context(), "ws upgrade failed", slog.Any("err", err)) + return + } + defer conn.Close() + + for { + mt, msg, err := conn.ReadMessage() + if err != nil { return } + if err := conn.WriteMessage(mt, msg); err != nil { return } + } +} +``` + +For long-lived connections, use `conn.SetReadDeadline` + `SetPongHandler` for keepalive. CLIProxyAPI's `wsrelay` package is a reference implementation. + +--- + +## Database wiring — pgx pool, injected, never global + +```go +package store + +import ( + "context" + "fmt" + "github.com/jackc/pgx/v5/pgxpool" +) + +func NewPool(ctx context.Context, dsn string) (*pgxpool.Pool, error) { + cfg, err := pgxpool.ParseConfig(dsn) + if err != nil { + return nil, fmt.Errorf("parse dsn: %w", err) + } + cfg.MaxConns = 25 + cfg.MinConns = 5 + cfg.MaxConnLifetime = time.Hour + cfg.MaxConnIdleTime = 30 * time.Minute + + pool, err := pgxpool.NewWithConfig(ctx, cfg) + if err != nil { + return nil, fmt.Errorf("connect: %w", err) + } + if err := pool.Ping(ctx); err != nil { + pool.Close() + return nil, fmt.Errorf("ping: %w", err) + } + return pool, nil +} +``` + +See `sqlc-pgx.md` for queries. + +--- + +## Healthcheck + +```go +func (h *Handler) Healthz(c *gin.Context) { + if err := h.pool.Ping(c.Request.Context()); err != nil { + c.JSON(503, gin.H{"db": "down", "error": err.Error()}) + return + } + c.JSON(200, gin.H{"ok": true}) +} +``` + +Mount BEFORE auth. Health checks must be unauthenticated. + +--- + +## Testing the server + +```go +func TestCreateUser_returns_201_for_valid_input(t *testing.T) { + // Given + h := newTestHandler(t) + r := gin.New() + h.Mount(r) + + body := `{"email":"a@b.com","username":"alice"}` + req := httptest.NewRequest("POST", "/api/v1/users", strings.NewReader(body)) + req.Header.Set("Content-Type", "application/json") + rec := httptest.NewRecorder() + + // When + r.ServeHTTP(rec, req) + + // Then + require.Equal(t, http.StatusCreated, rec.Code) + var got struct{ ID string `json:"id"` } + require.NoError(t, json.Unmarshal(rec.Body.Bytes(), &got)) + require.NotEmpty(t, got.ID) +} +``` + +See `testing.md` for full patterns (testcontainers integration, table-driven, goleak). + +--- + +## Sources + +- gin docs: https://gin-gonic.com/docs/ +- CLIProxyAPI (reference impl): https://github.com/router-for-me/CLIProxyAPI +- pgx pool: https://pkg.go.dev/github.com/jackc/pgx/v5/pgxpool +- SSE spec: https://html.spec.whatwg.org/multipage/server-sent-events.html +- Go's `http.Server` graceful shutdown: https://pkg.go.dev/net/http#Server.Shutdown diff --git a/packages/shared-skills/skills/programming/references/go/bootstrap.md b/packages/shared-skills/skills/programming/references/go/bootstrap.md new file mode 100644 index 000000000..ff44dbc32 --- /dev/null +++ b/packages/shared-skills/skills/programming/references/go/bootstrap.md @@ -0,0 +1,328 @@ +# Bootstrap — Project Layout, Toolchain, Taskfile, CI + +What every new Go project gets in the first 60 seconds. Drop the script in `scripts/go/new-project.go` does all of this — this document explains *what* it produces and *why*. + +## Toolchain pin + +`go.work` (monorepo) or just rely on `go.mod`'s `go 1.23` directive (single module). Go 1.21+ auto-downloads matching toolchain when the local `go` binary is older. **No `.tool-versions` / `asdf` / `mise` indirection required** unless your shop standardizes on it. + +```bash +# Confirm a working toolchain +go env GOTOOLCHAIN # should be "auto" or your pinned version +go version # ≥ 1.23 +``` + +## Required global installs + +These are CLI tools, installed once per machine via `go install`: + +```bash +go install mvdan.cc/gofumpt@latest +go install golang.org/x/tools/cmd/goimports@latest +go install github.com/golangci/golangci-lint/cmd/golangci-lint@v2.0.0 +go install go.uber.org/nilaway/cmd/nilaway@latest +go install github.com/sqlc-dev/sqlc/cmd/sqlc@latest +go install github.com/pressly/goose/v3/cmd/goose@latest +go install go.uber.org/mock/mockgen@latest +go install github.com/go-task/task/v3/cmd/task@latest +``` + +For Connect/protobuf projects, additionally: + +```bash +go install github.com/bufbuild/buf/cmd/buf@latest +go install google.golang.org/protobuf/cmd/protoc-gen-go@latest +go install connectrpc.com/connect/cmd/protoc-gen-connect-go@latest +``` + +## Project layout — the canonical tree + +``` +myservice/ +├── go.mod +├── go.sum +├── Taskfile.yml # task runner +├── .golangci.yml # see golangci-strict.md +├── .editorconfig +├── .gitignore +├── README.md +├── AGENTS.md # agent-readable project facts +├── cmd/ +│ └── server/ +│ └── main.go # ONLY: parse flags, call cmd.Execute(); ≤ 50 LOC +├── internal/ # NEVER importable from outside this module +│ ├── api/ # transport layer (gin/connect routers) +│ │ ├── server.go # gin engine setup, route registration +│ │ ├── middleware/ +│ │ │ ├── request_id.go +│ │ │ ├── logging.go +│ │ │ └── auth.go +│ │ └── handlers/ +│ │ ├── users.go +│ │ └── users_test.go +│ ├── domain/ # parse-don't-validate types, smart constructors +│ │ ├── user.go +│ │ └── email.go +│ ├── service/ # business logic, depends on domain only +│ │ └── user_service.go +│ ├── store/ # persistence; sqlc-generated code lives here +│ │ ├── sqlc/ # sqlc-generated, do not hand-edit +│ │ ├── queries/ # *.sql files sqlc reads +│ │ └── migrations/ # goose migrations +│ ├── config/ # env-driven config (caarlos0/env) +│ │ └── config.go +│ └── obs/ # observability: slog setup, otel, healthz +│ └── logger.go +├── pkg/ # exportable libraries — only if you publish +│ └── … +├── proto/ # *.proto definitions (Connect/gRPC projects) +│ └── service.proto +├── gen/ # generated code (Connect, OpenAPI) +│ └── service/v1/ +│ ├── service.pb.go +│ └── servicev1connect/ +├── test/ # cross-cutting test helpers, fixtures +└── .github/workflows/ci.yml +``` + +**Rules**: + +- `cmd//main.go` is ≤ 50 LOC. Anything more lives in `internal/cmd/`. +- `internal/` is **the** business code. Other modules cannot import it (Go compiler-enforced). +- `pkg/` is for things you genuinely want third parties to import. Empty until proven otherwise. +- No `utils/`, `helpers/`, `common/`, `shared/`. **REJECT.** Files are named after the concept they own. +- One package per directory. One responsibility per package. + +## `Taskfile.yml` — the entry point for every action + +`go-task/task` is the modern Make replacement. Cross-platform, YAML, fast. + +```yaml +version: '3' + +vars: + BINARY: server + PKG: ./cmd/server + +tasks: + default: + deps: [fmt, lint, test] + + fmt: + desc: Format all Go files + cmds: + - gofumpt -w . + - goimports -w -local "$(go list -m)" . + + lint: + desc: Run all linters + cmds: + - golangci-lint run --timeout 5m ./... + - nilaway -include-pkgs "$(go list -m)/..." ./... + + test: + desc: Run tests with race detector + cmds: + - go test -race -shuffle=on -count=1 ./... + + test-cover: + desc: Coverage report + cmds: + - go test -race -shuffle=on -count=1 -coverprofile=coverage.out ./... + - go tool cover -html=coverage.out -o coverage.html + + build: + desc: Build the binary + cmds: + - go build -trimpath -ldflags="-s -w" -o bin/{{.BINARY}} {{.PKG}} + + run: + desc: Run the server locally + deps: [build] + cmds: + - ./bin/{{.BINARY}} + + gen: + desc: Run all code generators + cmds: + - task: gen:sqlc + - task: gen:mocks + - task: gen:proto + + gen:sqlc: + cmds: + - sqlc generate + sources: + - internal/store/queries/*.sql + - internal/store/sqlc.yaml + generates: + - internal/store/sqlc/*.go + + gen:mocks: + cmds: + - go generate ./... + + gen:proto: + cmds: + - buf generate + sources: + - proto/**/*.proto + - buf.yaml + - buf.gen.yaml + + migrate:up: + cmds: + - goose -dir internal/store/migrations postgres "$DATABASE_URL" up + + migrate:down: + cmds: + - goose -dir internal/store/migrations postgres "$DATABASE_URL" down + + ci: + desc: Everything CI does, locally + deps: [fmt, lint, test, build] +``` + +`task` (no args) runs format + lint + test in parallel where possible. `task ci` runs the full pipeline. + +## `go.mod` template + +```go +module github.com/your-org/myservice + +go 1.23 + +require ( + github.com/caarlos0/env/v11 v11.2.2 + github.com/gin-gonic/gin v1.10.1 + github.com/go-playground/validator/v10 v10.22.1 + github.com/google/uuid v1.6.0 + github.com/jackc/pgx/v5 v5.7.6 + golang.org/x/sync v0.18.0 +) +``` + +Only direct deps listed; `go mod tidy` populates indirects. + +## `.editorconfig` + +```ini +root = true + +[*] +indent_style = tab +indent_size = 4 +end_of_line = lf +charset = utf-8 +trim_trailing_whitespace = true +insert_final_newline = true + +[*.{yml,yaml,json,md}] +indent_style = space +indent_size = 2 +``` + +## `.gitignore` + +```gitignore +bin/ +coverage.out +coverage.html +*.test +*.prof + +# IDE +.idea/ +.vscode/ +*.swp + +# Local env +.env +.env.local + +# Secrets +*.pem +*.key +``` + +## CI — minimal GitHub Actions + +`.github/workflows/ci.yml`: + +```yaml +name: ci +on: + pull_request: + push: + branches: [main] + +jobs: + ci: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-go@v5 + with: + go-version: '1.23' + cache: true + + - name: Install tools + run: | + go install mvdan.cc/gofumpt@latest + go install github.com/golangci/golangci-lint/cmd/golangci-lint@v2.0.0 + go install go.uber.org/nilaway/cmd/nilaway@latest + go install github.com/go-task/task/v3/cmd/task@latest + + - name: Format check + run: gofumpt -l . | (! grep .) + + - name: Lint + run: golangci-lint run --timeout 5m ./... + + - name: Nilaway + run: nilaway ./... + + - name: Test + run: go test -race -shuffle=on -count=1 ./... + + - name: Build + run: go build -trimpath ./... +``` + +The order matters: format → lint → nilaway → test → build. Fail fast on the cheap checks. + +## `AGENTS.md` — agent-readable project facts + +Every new project gets an `AGENTS.md` at the root. The content is **machine-friendly**: short, declarative, no marketing prose. Example: + +```markdown +# AGENTS.md + +Go 1.23+ HTTP service for {one-line purpose}. + +## Commands +- `task` — fmt + lint + test +- `task build` — produce ./bin/server +- `task gen` — regenerate sqlc + mocks + proto + +## Architecture +- `cmd/server/main.go` — entrypoint, ≤50 LOC +- `internal/api/` — gin handlers + middleware +- `internal/domain/` — smart-constructor types, no I/O +- `internal/store/sqlc/` — generated; never hand-edit + +## Conventions +- `slog` for all logs; never `log.*`, never `fmt.Println` +- `context.Context` first arg for every public function +- Errors wrapped with `%w`; check with `errors.Is/As` +- 250 pure LOC ceiling per file — split before adding lines +``` + +The skill's `cmd/new-project.go` writes this file with project-specific values filled in. + +## Sources + +- Go modules reference: https://go.dev/ref/mod +- go-task: https://taskfile.dev +- golangci-lint v2: https://golangci-lint.run/docs/configuration/ +- Standard project layout debate: https://go.dev/doc/modules/layout (NOT `golang-standards/project-layout` — that repo is community, not official) diff --git a/packages/shared-skills/skills/programming/references/go/bubbletea-v2.md b/packages/shared-skills/skills/programming/references/go/bubbletea-v2.md new file mode 100644 index 000000000..45eae7020 --- /dev/null +++ b/packages/shared-skills/skills/programming/references/go/bubbletea-v2.md @@ -0,0 +1,360 @@ +# Bubbletea v2 — TUI with First-Class CJK / IME Support + +The TUI stack for 2026. Use **v2 RC**, not v1. If your users include Korean, Japanese, or Chinese speakers, v1 is broken — IME composition lands in the wrong cells. v2 fixes this. This document is the canonical setup. + +The reference implementation this document is distilled from: [`code-yeongyu/bubbletea-wm`](https://github.com/code-yeongyu/bubbletea-wm) — a floating window manager built specifically to nail down v2 + IME. + +--- + +## Why v2 (not v1) — the IME story + +Bubbletea v1 manages cursor positioning in software ("virtual cursor"). It draws a `█` at the cursor position. The terminal's *real* cursor stays at `(0, 0)`. + +This breaks every CJK input method. IME candidate windows (the popup showing Hangul composition choices for Korean, kana → kanji for Japanese, and pinyin lookup for Chinese) anchor to the terminal's **real** cursor position. With v1, the candidate window appears at top-left while you are typing somewhere in the middle of the screen. + +Bubbletea v2 fixes this with two changes: + +1. **`tea.View{Cursor: *tea.Cursor}`** — your `View()` method returns a view that *includes* the desired cursor position. The framework moves the terminal's real cursor there. +2. **`textarea.SetVirtualCursor(false)`** — textareas no longer draw their own `█`. They expose `.Cursor()` so you can read where they want the real cursor. + +Together: IME popups appear where the user is typing. As they should. + +### Other v2 wins (incidental) + +- `tea.MouseClickMsg` / `MouseMotionMsg` / `MouseReleaseMsg` instead of one coarse `MouseMsg`. +- Cleaner `View` struct with `AltScreen`, `MouseMode` fields instead of `tea.Cmd` setters. +- Pluggable rendering pipeline; better performance under high message volume. + +--- + +## `go.mod` + +```go +module github.com/your-org/mytui + +go 1.23 + +require ( + charm.land/bubbletea/v2 v2.0.0-rc.2 + charm.land/bubbles/v2 v2.0.0-rc.1 + charm.land/lipgloss/v2 v2.0.0-beta.3 + github.com/mattn/go-runewidth v0.0.19 +) +``` + +The packages live under `charm.land/` (NOT `github.com/charmbracelet/...`) for v2. This is the Charm team's deliberate import-path break to keep v2 separate from v1 until stable. + +--- + +## Minimal app — the IME-correct skeleton + +```go +package main + +import ( + "fmt" + "log" + + tea "charm.land/bubbletea/v2" + "charm.land/bubbles/v2/textarea" +) + +type model struct { + width, height int + ta textarea.Model +} + +func initial() model { + ta := textarea.New() + ta.Placeholder = "Type Korean / Japanese / Chinese here..." + ta.SetWidth(60) + ta.SetHeight(10) + ta.SetVirtualCursor(false) // ← THE LINE. Without this, IME breaks. + ta.Focus() + return model{ta: ta} +} + +func (m model) Init() tea.Cmd { return textarea.Blink } + +func (m model) Update(msg tea.Msg) (tea.Model, tea.Cmd) { + switch msg := msg.(type) { + case tea.WindowSizeMsg: + m.width, m.height = msg.Width, msg.Height + case tea.KeyPressMsg: + if msg.String() == "ctrl+c" { + return m, tea.Quit + } + } + var cmd tea.Cmd + m.ta, cmd = m.ta.Update(msg) + return m, cmd +} + +func (m model) View() tea.View { + var view tea.View + view.AltScreen = true + view.SetContent(m.ta.View()) + + // ── THE OTHER LINE. Position the REAL cursor for IME. ── + if cursor := m.ta.Cursor(); cursor != nil { + view.Cursor = cursor + } + return view +} + +func main() { + if _, err := tea.NewProgram(initial(), tea.WithAltScreen()).Run(); err != nil { + log.Fatal(err) + } + fmt.Println("bye") +} +``` + +The two lines that matter: + +1. `ta.SetVirtualCursor(false)` — disables the virtual `█`. +2. `view.Cursor = cursor` (where `cursor = m.ta.Cursor()`) — exports the real cursor position to the framework. + +Without **both**, IME breaks. + +--- + +## CJK width — go-runewidth, not `len()` + +Korean, Japanese, Chinese characters render as **two terminal cells** (wide characters per Unicode East Asian Width). Naive `len(string)` returns byte count, not display width. `utf8.RuneCountInString` returns rune count, also not display width. + +Use `github.com/mattn/go-runewidth`: + +```go +import "github.com/mattn/go-runewidth" + +func displayWidth(s string) int { + return runewidth.StringWidth(s) +} + +// Wide character occupies two cells; pad accordingly +for _, r := range s { + cell := string(r) + w := runewidth.RuneWidth(r) + canvas = append(canvas, cell) + if w == 2 { + canvas = append(canvas, "") // placeholder for second cell + } +} +``` + +`lipgloss/v2` uses `go-runewidth` internally — `lipgloss.Width("\u4e2d\u6587")` returns 4, not 2. **If you measure outside lipgloss, you must call runewidth directly.** + +--- + +## Mouse — v2 has typed events + +```go +case tea.MouseClickMsg: + // msg.X, msg.Y, msg.Button + return m.handleClick(msg.X, msg.Y, msg.Button) + +case tea.MouseMotionMsg: + return m.handleHover(msg.X, msg.Y) + +case tea.MouseReleaseMsg: + return m.handleRelease(msg.X, msg.Y) +``` + +Enable mouse via the `View`: + +```go +view.MouseMode = tea.MouseModeCellMotion // or MouseModeAll +``` + +`CellMotion` reports clicks + motion-while-button-pressed (drag). `MouseModeAll` reports motion always — heavier, only when you need hover. + +--- + +## Components from `bubbles/v2` + +```go +import ( + "charm.land/bubbles/v2/textarea" + "charm.land/bubbles/v2/textinput" + "charm.land/bubbles/v2/spinner" + "charm.land/bubbles/v2/viewport" + "charm.land/bubbles/v2/list" + "charm.land/bubbles/v2/table" + "charm.land/bubbles/v2/help" + "charm.land/bubbles/v2/key" +) +``` + +All v2 components support `SetVirtualCursor(false)` where they accept text input. Use it for every text input that users might type CJK into — and "might" should be assumed *yes*. + +--- + +## Styling — `lipgloss/v2` + +```go +import "charm.land/lipgloss/v2" + +titleStyle := lipgloss.NewStyle(). + Bold(true). + Foreground(lipgloss.Color("230")). + Background(lipgloss.Color("62")). + Padding(0, 1). + Border(lipgloss.RoundedBorder()). + BorderForeground(lipgloss.Color("63")) + +rendered := titleStyle.Render("\u4e2d\u6587") +``` + +`lipgloss/v2` width and padding correctly account for CJK display width. v1 did too — this is not a v2-specific fix, just a reminder. + +--- + +## Architecture pattern — Model–Update–View + +``` ++--------------------------------------------+ +| tea.Program runs the event loop | +| | +| loop: | +| msg <- queue | +| model, cmd = model.Update(msg) | +| view = model.View() | +| render(view) | +| if cmd != nil: go run(cmd) -> queue | ++--------------------------------------------+ +``` + +Rules: + +- **Model is a value type, not a pointer.** Bubbletea calls `Update` with a value receiver and expects a new value returned. Pointer receivers cause subtle bugs where state mutation leaks across draws. +- **`Update` is pure.** No I/O. No goroutines started inline. Any I/O returns a `tea.Cmd` — Bubbletea runs it in a goroutine and feeds the result back as a message. +- **`View` is read-only.** It returns a `tea.View` without modifying state. +- **`tea.Cmd` is `func() tea.Msg`.** It runs once, returns a message, exits. For repeating work, use `tea.Tick` or a self-resending command. + +```go +// One-shot command +func loadData() tea.Cmd { + return func() tea.Msg { + data, err := fetch() + if err != nil { return errMsg{err} } + return dataLoadedMsg{data} + } +} + +// Periodic +func tickEvery() tea.Cmd { + return tea.Tick(time.Second, func(t time.Time) tea.Msg { + return tickMsg{t} + }) +} +``` + +--- + +## Splitting the model — sub-models + +```go +type model struct { + list list.Model + input textinput.Model + spinner spinner.Model +} + +func (m model) Update(msg tea.Msg) (tea.Model, tea.Cmd) { + var cmds []tea.Cmd + var cmd tea.Cmd + + m.list, cmd = m.list.Update(msg) + cmds = append(cmds, cmd) + + m.input, cmd = m.input.Update(msg) + cmds = append(cmds, cmd) + + m.spinner, cmd = m.spinner.Update(msg) + cmds = append(cmds, cmd) + + return m, tea.Batch(cmds...) +} +``` + +`tea.Batch` runs commands concurrently. The framework collects their results in the order they arrive. + +When the model exceeds 250 LOC, split by sub-model into separate files: + +``` +internal/ui/ +├── model.go # root model orchestration +├── list.go # list sub-model state + update + view +├── input.go # input sub-model +└── spinner.go # spinner sub-model +``` + +--- + +## Testing TUI code — `teatest` + +```go +import "charm.land/bubbletea/v2/teatest" + +func TestModel_typing_cjk_keeps_cursor_in_position(t *testing.T) { + // Given + m := initial() + tm := teatest.NewTestModel(t, m, teatest.WithInitialTermSize(80, 24)) + + // When — simulate typing two CJK wide characters + tm.Send(tea.KeyPressMsg{Code: '\u4e2d'}) + tm.Send(tea.KeyPressMsg{Code: '\u6587'}) + + // Then + out := tm.FinalOutput(t) + require.Contains(t, string(out), "\u4e2d\u6587") + // Cursor should be at column 4 (two wide chars = 4 cells) + // ... +} +``` + +`teatest` lets you drive the model through synthetic messages and inspect the rendered output. Pair with `autogold` snapshots for full-view regression tests. + +--- + +## Common antipatterns + +| Bad | Why | Good | +|---|---|---| +| `tea.Program` with `tea.WithoutSignals()` | Ctrl-C does not work | Default signal handling | +| Pointer receivers on Model | Bubbletea expects value semantics | Value receivers, return new model | +| `time.Sleep` inside `Update` | Blocks the event loop | `tea.Tick` or async `tea.Cmd` | +| `fmt.Println` for debug | Corrupts the rendered output | `tea.Printf` for logging, or write to a file | +| `len(s)` for CJK width | Off by 2x | `runewidth.StringWidth(s)` | +| `Bubbletea v1` for an app with text input | Korean/Japanese IME breaks | v2 + `SetVirtualCursor(false)` | +| Drawing your own `█` block cursor in v2 | Conflicts with `view.Cursor` | Let the terminal handle it | + +--- + +## Performance — when v2 starts to crawl + +- **Reduce View frequency.** If the model changes 60 times/sec but the rendered view changes once/sec, gate redraws on a "dirty" flag. +- **`viewport.Model` for scrollable content.** Avoid re-rendering thousands of lines on every keystroke. +- **`Batch` your commands.** A series of synchronous `tea.Cmd` returns serializes; `tea.Batch` parallelizes. +- **Profile with `tea.WithFPS(N)`** to cap repaint rate during development. + +--- + +## When NOT to use Bubbletea + +- The app is one prompt + one answer. Use `huh` (also from Charm) — simpler, no Model–Update–View ceremony. +- The app is a long-running daemon with occasional status output. Use `slog` to stderr and `tea.Program` only if interactivity becomes necessary. +- The app must run as a non-tty subprocess (CI, redirected stdin). `tea.Program` requires a tty for input. Detect via `term.IsTerminal(int(os.Stdin.Fd()))` and fall back to a non-interactive path. + +--- + +## Sources + +- bubbletea v2 RC: https://github.com/charmbracelet/bubbletea/tree/v2 +- bubbles v2: https://github.com/charmbracelet/bubbles/tree/v2 +- lipgloss v2: https://github.com/charmbracelet/lipgloss/tree/v2 +- bubbletea-wm (IME reference): https://github.com/code-yeongyu/bubbletea-wm +- crush CLI (production IME impl): https://github.com/charmbracelet/crush +- go-runewidth: https://github.com/mattn/go-runewidth +- Unicode East Asian Width: https://www.unicode.org/reports/tr11/ diff --git a/packages/shared-skills/skills/programming/references/go/cobra-stack.md b/packages/shared-skills/skills/programming/references/go/cobra-stack.md new file mode 100644 index 000000000..a73805f7f --- /dev/null +++ b/packages/shared-skills/skills/programming/references/go/cobra-stack.md @@ -0,0 +1,468 @@ +# CLI Stack — cobra + slog + caarlos0/env + signal handling + +The canonical Go CLI skeleton. `cobra` is the de facto framework — Kubernetes, Docker CLI, Helm, GitHub CLI, gh, Hugo all use it. Use it. + +--- + +## Toolchain + +```bash +go install github.com/spf13/cobra-cli@latest +cobra-cli init mytool +cobra-cli add server +cobra-cli add migrate +``` + +`cobra-cli` scaffolds the `cmd/` package. Edit the result; do not regenerate. + +--- + +## Layout + +``` +mytool/ +├── go.mod +├── main.go # ≤ 30 LOC, calls cmd.Execute +├── cmd/ +│ ├── root.go # rootCmd, persistent flags, slog setup +│ ├── server.go # `mytool server` subcommand +│ ├── migrate.go # `mytool migrate` subcommand +│ └── version.go # `mytool version` — auto-injected version +├── internal/ +│ ├── config/ +│ └── server/ +└── Taskfile.yml +``` + +--- + +## `main.go` + +```go +package main + +import ( + "context" + "log/slog" + "os" + "os/signal" + "syscall" + + "github.com/your-org/mytool/cmd" +) + +func main() { + ctx, stop := signal.NotifyContext(context.Background(), + syscall.SIGINT, syscall.SIGTERM) + defer stop() + + if err := cmd.Execute(ctx); err != nil { + slog.Error("fatal", slog.Any("err", err)) + os.Exit(1) + } +} +``` + +`signal.NotifyContext` (Go 1.16+) gives every subcommand a ctx that cancels on Ctrl-C. Subcommands plumb the ctx into their workers. + +--- + +## `cmd/root.go` + +```go +package cmd + +import ( + "context" + "log/slog" + "os" + + "github.com/spf13/cobra" +) + +var ( + verbose bool + logFormat string + configPath string +) + +var rootCmd = &cobra.Command{ + Use: "mytool", + Short: "Short description of mytool", + Long: `Long description, prose; cobra wraps it for --help.`, + PersistentPreRunE: func(c *cobra.Command, args []string) error { + return setupLogger() + }, + SilenceUsage: true, // don't print --help on every error + SilenceErrors: true, // we log them ourselves in Execute +} + +func init() { + rootCmd.PersistentFlags().BoolVarP(&verbose, "verbose", "v", false, + "enable debug logging") + rootCmd.PersistentFlags().StringVar(&logFormat, "log-format", "text", + "log format: text or json") + rootCmd.PersistentFlags().StringVarP(&configPath, "config", "c", "", + "path to config file (optional)") +} + +func Execute(ctx context.Context) error { + return rootCmd.ExecuteContext(ctx) +} + +func setupLogger() error { + level := slog.LevelInfo + if verbose { level = slog.LevelDebug } + opts := &slog.HandlerOptions{Level: level} + + var h slog.Handler + switch logFormat { + case "json": + h = slog.NewJSONHandler(os.Stderr, opts) + case "text": + h = slog.NewTextHandler(os.Stderr, opts) + default: + return fmt.Errorf("invalid log-format %q", logFormat) + } + slog.SetDefault(slog.New(h)) + return nil +} +``` + +Notes: + +- `RunE` / `PersistentPreRunE` (the `E` variants) return errors. Use these; never use `Run` (no error return, encourages `log.Fatal`). +- `SilenceUsage: true` + `SilenceErrors: true` together: cobra stops printing the full `--help` on every command failure (the default behavior is rude in production scripts). +- `ExecuteContext` (cobra 1.8+) plumbs the ctx into every subcommand's `cmd.Context()`. + +--- + +## `cmd/server.go` + +```go +package cmd + +import ( + "log/slog" + + "github.com/spf13/cobra" + "github.com/your-org/mytool/internal/server" +) + +var ( + serverAddr string +) + +var serverCmd = &cobra.Command{ + Use: "server", + Short: "Run the HTTP server", + RunE: func(c *cobra.Command, args []string) error { + ctx := c.Context() + slog.InfoContext(ctx, "starting", slog.String("addr", serverAddr)) + return server.Run(ctx, serverAddr) + }, +} + +func init() { + serverCmd.Flags().StringVar(&serverAddr, "addr", ":8080", + "listen address") + rootCmd.AddCommand(serverCmd) +} +``` + +The subcommand is a thin shim — flags + log line + delegate to `internal/server`. Anything bigger violates the 250-LOC ceiling and belongs in `internal/`. + +--- + +## Subcommands with arguments + +```go +var migrateUpCmd = &cobra.Command{ + Use: "up [N]", + Short: "Apply N migrations (default: all)", + Args: cobra.MaximumNArgs(1), + RunE: func(c *cobra.Command, args []string) error { + n := -1 // all + if len(args) == 1 { + var err error + n, err = strconv.Atoi(args[0]) + if err != nil { + return fmt.Errorf("invalid N: %w", err) + } + } + return migrate.Up(c.Context(), n) + }, +} +``` + +Use cobra's argument validators (`cobra.ExactArgs`, `cobra.MaximumNArgs`, `cobra.OnlyValidArgs`). They produce clean help text. + +--- + +## Flag types — typed, not strings + +```go +// GOOD +serverCmd.Flags().DurationVar(&timeout, "timeout", 30*time.Second, "request timeout") +serverCmd.Flags().IntVar(&port, "port", 8080, "port") +serverCmd.Flags().StringSliceVar(&hosts, "host", nil, "allowed hosts (repeatable)") + +// BAD — manual parsing +serverCmd.Flags().StringVar(&timeoutStr, "timeout", "30s", "") +// ...then later: time.ParseDuration(timeoutStr) +``` + +`pflag` (cobra's flag lib) has typed variants for every common type. Use them; the parsing and error messages are free. + +--- + +## Bind flags to env vars + +cobra + viper is overkill for env binding. Use `caarlos0/env/v11`: + +```go +type ServerOpts struct { + Addr string `env:"ADDR" envDefault:":8080"` + Timeout time.Duration `env:"TIMEOUT" envDefault:"30s"` +} + +var opts ServerOpts + +var serverCmd = &cobra.Command{ + Use: "server", + PersistentPreRunE: func(c *cobra.Command, args []string) error { + // 1. Parse env first. + if err := env.Parse(&opts); err != nil { return err } + // 2. Flags override env if explicitly set. + if c.Flags().Changed("addr") { + opts.Addr, _ = c.Flags().GetString("addr") + } + return nil + }, + RunE: func(c *cobra.Command, args []string) error { + return server.Run(c.Context(), opts) + }, +} + +func init() { + serverCmd.Flags().String("addr", "", "listen address (env: ADDR)") + serverCmd.Flags().Duration("timeout", 0, "request timeout (env: TIMEOUT)") + rootCmd.AddCommand(serverCmd) +} +``` + +Precedence: **flag (if set) > env > default**. Document the env var in the flag usage string. + +--- + +## Version subcommand — build-injected + +```go +// cmd/version.go +package cmd + +import ( + "fmt" + "runtime/debug" + + "github.com/spf13/cobra" +) + +// Set by -ldflags at build time, falls back to debug.BuildInfo. +var ( + version = "" + commit = "" + date = "" +) + +var versionCmd = &cobra.Command{ + Use: "version", + Short: "Print version", + Run: func(c *cobra.Command, args []string) { + v, c2, d := resolveVersion() + fmt.Printf("mytool %s (commit %s, built %s)\n", v, c2, d) + }, +} + +func resolveVersion() (string, string, string) { + if version != "" { return version, commit, date } + info, ok := debug.ReadBuildInfo() + if !ok { return "dev", "unknown", "unknown" } + + var vcs, hash, time string + for _, s := range info.Settings { + switch s.Key { + case "vcs.revision": hash = s.Value + case "vcs.time": time = s.Value + case "vcs": vcs = s.Value + } + } + return info.Main.Version, hash, time + " (" + vcs + ")" +} + +func init() { rootCmd.AddCommand(versionCmd) } +``` + +Build with version injection: + +```bash +go build \ + -ldflags="-X 'github.com/your-org/mytool/cmd.version=v1.2.3' -X 'github.com/your-org/mytool/cmd.commit=$(git rev-parse --short HEAD)' -X 'github.com/your-org/mytool/cmd.date=$(date -u +%Y-%m-%dT%H:%M:%SZ)'" \ + -o bin/mytool ./ +``` + +The `debug.BuildInfo` fallback means a `go install`'d binary also has version info — no manual `-ldflags` needed. + +--- + +## Shell completions + +```go +var completionCmd = &cobra.Command{ + Use: "completion [bash|zsh|fish|powershell]", + Short: "Generate shell completion", + Args: cobra.ExactValidArgs(1), + ValidArgs: []string{"bash", "zsh", "fish", "powershell"}, + DisableFlagsInUseLine: true, + RunE: func(c *cobra.Command, args []string) error { + switch args[0] { + case "bash": return rootCmd.GenBashCompletionV2(os.Stdout, true) + case "zsh": return rootCmd.GenZshCompletion(os.Stdout) + case "fish": return rootCmd.GenFishCompletion(os.Stdout, true) + case "powershell": return rootCmd.GenPowerShellCompletion(os.Stdout) + } + return nil + }, +} + +func init() { rootCmd.AddCommand(completionCmd) } +``` + +User: + +```bash +mytool completion zsh > "${fpath[1]}/_mytool" +``` + +--- + +## Interactive prompts — `huh` from charm + +For prompts/forms (`Are you sure?`, "Pick an environment", multi-field forms): + +```go +import "github.com/charmbracelet/huh" + +var confirm bool +err := huh.NewConfirm(). + Title("Apply migrations to PRODUCTION?"). + Affirmative("Yes, do it"). + Negative("Abort"). + Value(&confirm). + Run() +``` + +`huh` replaces `survey` (which is no longer maintained). It composes with `lipgloss` for styling. + +--- + +## Progress / spinners + +```go +import "github.com/charmbracelet/huh/spinner" + +err := spinner.New().Title("Fetching...").Action(func() { + // long-running work +}).Run() +``` + +For determinate progress (downloads, batch processing), use `vbauerster/mpb/v8`: + +```go +import "github.com/vbauerster/mpb/v8" + +p := mpb.New(mpb.WithWidth(60)) +bar := p.AddBar(int64(total), /* decorators */) +for i := 0; i < total; i++ { + work() + bar.Increment() +} +p.Wait() +``` + +--- + +## Output — JSON vs text + +Honor `--output json` for any CLI that scripts will parse: + +```go +var outputFmt string + +rootCmd.PersistentFlags().StringVar(&outputFmt, "output", "text", + "output format: text or json") + +func render(v any) error { + switch outputFmt { + case "json": + enc := json.NewEncoder(os.Stdout) + enc.SetIndent("", " ") + return enc.Encode(v) + case "text": + return renderText(v) + default: + return fmt.Errorf("invalid --output %q", outputFmt) + } +} +``` + +The `text` format uses `lipgloss` tables or `aquasecurity/table` for nicely-aligned columns. The `json` format is for `jq`-style piping. + +--- + +## Error semantics + +- Return errors from `RunE`. Cobra catches them and the `Execute` wrapper logs + exits non-zero. +- `os.Exit(1)` should appear **only in `main.go`**. Anywhere else means a subcommand cannot be tested. +- For graceful early termination ("user cancelled"), return a sentinel and check it in `Execute`: + ```go + var ErrCancelled = errors.New("cancelled by user") + // ... return ErrCancelled + // in main: + if errors.Is(err, cmd.ErrCancelled) { os.Exit(130) } // 128 + SIGINT + ``` + +--- + +## Testing CLI commands + +```go +func TestServerCmd_runs_with_default_addr(t *testing.T) { + // Given + buf := &bytes.Buffer{} + rootCmd.SetOut(buf) + rootCmd.SetErr(buf) + rootCmd.SetArgs([]string{"server", "--addr", ":0"}) + + // When + ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond) + defer cancel() + err := rootCmd.ExecuteContext(ctx) + + // Then + require.NoError(t, err) + require.Contains(t, buf.String(), "starting") +} +``` + +`SetArgs` + `ExecuteContext` is the canonical pattern. Bind a ctx with a short deadline for tests that would otherwise block. + +--- + +## Sources + +- cobra docs: https://github.com/spf13/cobra/blob/main/site/content/user_guide.md +- pflag: https://github.com/spf13/pflag +- huh: https://github.com/charmbracelet/huh +- caarlos0/env: https://github.com/caarlos0/env +- signal.NotifyContext: https://pkg.go.dev/os/signal#NotifyContext diff --git a/packages/shared-skills/skills/programming/references/go/concurrency.md b/packages/shared-skills/skills/programming/references/go/concurrency.md new file mode 100644 index 000000000..52ac5033b --- /dev/null +++ b/packages/shared-skills/skills/programming/references/go/concurrency.md @@ -0,0 +1,362 @@ +# Concurrency + +Goroutines, context, errgroup, channels, locks, and the discipline that keeps them from leaking. Go makes concurrency *easy to start* and *easy to get wrong*. This document is the boring rule set. + +--- + +## The four non-negotiables + +1. **`ctx context.Context` is the first parameter of every public function that does I/O or can be cancelled.** +2. **No goroutine without a shutdown path.** Every `go` keyword must answer "how does this stop?". +3. **`-race` on every test run.** The `Taskfile.yml` and CI both enforce it. +4. **`goleak` in `TestMain`** for every package that spawns goroutines. Catches leaks the race detector cannot. + +--- + +## `context.Context` — the cancellation backbone + +```go +// GOOD — ctx as first param, propagated through +func (s *UserService) Create(ctx context.Context, email Email) (User, error) { + user, err := s.store.Insert(ctx, email) + if err != nil { + return User{}, fmt.Errorf("insert: %w", err) + } + if err := s.notifier.Welcome(ctx, user); err != nil { + return User{}, fmt.Errorf("notify: %w", err) + } + return user, nil +} + +// BAD — creates a fresh ctx, breaks request cancellation +func (s *UserService) Create(email Email) (User, error) { + ctx := context.Background() // ← contextcheck linter rejects this + // ... +} +``` + +The `contextcheck` linter (enabled in `golangci-strict.md`) refuses any function that has `ctx context.Context` available but uses `context.Background()` instead. + +### `context.Value` — use sparingly + +```go +// Typed key — never use a bare string +type ctxKey struct{ name string } +var requestIDKey = ctxKey{"request_id"} + +func WithRequestID(ctx context.Context, id string) context.Context { + return context.WithValue(ctx, requestIDKey, id) +} + +func RequestID(ctx context.Context) string { + v, _ := ctx.Value(requestIDKey).(string) + return v +} +``` + +**Rules**: +- Keys are unexported struct types, not strings. Prevents collisions across packages. +- `context.Value` is for *request-scoped metadata* (request ID, auth subject, trace span), NEVER for application-scoped dependencies. +- Dependencies (loggers, DB pools, config) go in your service struct, not in `context.Value`. + +### `WithTimeout` / `WithCancel` — always pair with `defer cancel()` + +```go +ctx, cancel := context.WithTimeout(ctx, 5*time.Second) +defer cancel() // ← MUST be deferred. fatcontext linter catches misses. + +if err := slow(ctx); err != nil { ... } +``` + +Forgetting `defer cancel()` leaks a context goroutine until the parent expires — the `lostcancel` vet check catches it. + +--- + +## `errgroup` — the structured concurrency primitive + +`golang.org/x/sync/errgroup` is Go's answer to Python's `asyncio.TaskGroup` or Rust's `JoinSet`. Use it instead of raw `go` for any group of related goroutines. + +```go +import "golang.org/x/sync/errgroup" + +func FetchAll(ctx context.Context, urls []string) ([][]byte, error) { + g, ctx := errgroup.WithContext(ctx) + g.SetLimit(8) // concurrency cap — leave unbounded = production outage + + results := make([][]byte, len(urls)) + for i, u := range urls { + g.Go(func() error { + body, err := fetch(ctx, u) + if err != nil { + return fmt.Errorf("fetch %s: %w", u, err) + } + results[i] = body + return nil + }) + } + if err := g.Wait(); err != nil { + return nil, err + } + return results, nil +} +``` + +Properties: + +- `WithContext(parent)` returns a child ctx that gets cancelled on **first non-nil error**. All in-flight goroutines see `ctx.Done()` and bail. +- `SetLimit(n)` blocks `g.Go(...)` when the in-flight count hits `n`. **Always set this.** Unbounded fan-out is how services die. +- `g.Wait()` returns the **first** non-nil error. Others are dropped. If you need all errors, accumulate them manually: + ```go + var mu sync.Mutex + var errs []error + // inside g.Go: + // mu.Lock(); errs = append(errs, err); mu.Unlock() + // after Wait, errors.Join(errs...) + ``` + +--- + +## Goroutine leaks — `goleak` + +```go +package store_test + +import ( + "testing" + "go.uber.org/goleak" +) + +func TestMain(m *testing.M) { + goleak.VerifyTestMain(m) +} +``` + +This single line at the top of `*_test.go` runs goleak's check after every test in the package. If a test leaks a goroutine, the run fails — pointing at which goroutine. + +**The bug it catches**: starting a goroutine in `setUp` and never joining it. Common in DB connection pools, background workers, ticker loops. The race detector does NOT catch this. + +If you have a known long-lived goroutine (a singleton background worker, a metrics exporter), use `goleak.IgnoreTopFunction`: + +```go +goleak.VerifyTestMain(m, + goleak.IgnoreTopFunction("github.com/prometheus/client_golang/prometheus.(*Registry).Push"), +) +``` + +--- + +## Channels — the rules that hold + +### Direction + +```go +// GOOD — direction in signatures +func produce(out chan<- Item) +func consume(in <-chan Item) +func pipeline(in <-chan Item, out chan<- Item) +``` + +Direction restricts misuse. A consumer cannot close the producer's channel. + +### Closing + +- **The sender closes.** Always. Never the receiver, never multiple senders. +- **Multiple senders → use a `sync.WaitGroup` + one closer.** +- **Closing a closed channel panics.** Closing a `nil` channel panics. Sending on a closed channel panics. Receiving from a closed channel returns zero value with `ok = false`. + +```go +// Canonical fan-in: multiple producers, one closer +func fanIn(ctx context.Context, sources ...<-chan Item) <-chan Item { + out := make(chan Item) + var wg sync.WaitGroup + wg.Add(len(sources)) + for _, src := range sources { + go func() { + defer wg.Done() + for item := range src { + select { + case out <- item: + case <-ctx.Done(): + return + } + } + }() + } + go func() { wg.Wait(); close(out) }() + return out +} +``` + +### Selecting + +```go +select { +case msg := <-incoming: + handle(msg) +case <-ctx.Done(): + return ctx.Err() +case <-time.After(5 * time.Second): + return ErrTimeout +} +``` + +- `time.After` allocates a timer each call — fine for occasional selects, **NOT for hot loops**. Use `time.NewTimer` + `timer.Reset` for repeat selects. +- A `default:` case makes `select` non-blocking. Use deliberately, not by accident. + +### Buffered vs unbuffered + +- **Unbuffered** (`make(chan T)`) = synchronous handoff. Sender blocks until receiver is ready. Use for *coordination*. +- **Buffered** (`make(chan T, n)`) = asynchronous up to `n`. Use for *decoupling producer rate from consumer rate*. + +A buffered channel of size 1 acts as a **non-blocking signal**: + +```go +ready := make(chan struct{}, 1) +// Producer +select { +case ready <- struct{}{}: // signal once, non-blocking +default: // already signaled, skip +} +// Consumer +<-ready +``` + +--- + +## Locks — the pyramid + +``` +Highest level (preferred) + channels (message passing — "share memory by communicating") + errgroup / wait group + + sync.RWMutex (many readers, occasional writer) + sync.Mutex (mutual exclusion) + + atomic.Int64 / atomic.Pointer (single-word lock-free) + +Lowest level (rare) + unsafe.Pointer + barriers (custom lock-free; needs -race AND review) +``` + +### `sync.Mutex` — embed, don't expose + +```go +type Cache struct { + mu sync.RWMutex + items map[string]Entry +} + +func (c *Cache) Get(key string) (Entry, bool) { + c.mu.RLock() + defer c.mu.RUnlock() + e, ok := c.items[key] + return e, ok +} + +func (c *Cache) Set(key string, e Entry) { + c.mu.Lock() + defer c.mu.Unlock() + c.items[key] = e +} +``` + +- `sync.Mutex` is **not** copyable. The `copylocks` vet check catches `var c2 = c1` where `c1` has a mutex. +- Always `defer mu.Unlock()` immediately after `Lock()`. Forgetting is the #1 deadlock cause. +- Never call user code (callbacks, listener notifications) while holding the lock. Drop the lock, snapshot the data, release, then call out. + +### `sync.OnceValue` / `sync.OnceFunc` (Go 1.21+) + +Replacement for `sync.Once` for typed lazy init: + +```go +var loadConfig = sync.OnceValue(func() Config { + var cfg Config + if err := env.Parse(&cfg); err != nil { panic(err) } + return cfg +}) + +func handler() { cfg := loadConfig(); ... } +``` + +Type-safe, no `sync.Once` + global variable boilerplate. + +### Atomics — the typed API only + +```go +// Go 1.19+ — use the typed atomic.* family +var counter atomic.Int64 +counter.Add(1) +n := counter.Load() + +// NEVER — the old function-style is type-unsafe +atomic.AddInt64(&counter, 1) // ← rejected +``` + +--- + +## Time — inject a clock for testability + +```go +type Clock interface { + Now() time.Time +} + +type realClock struct{} +func (realClock) Now() time.Time { return time.Now() } + +type Service struct { + clock Clock +} + +// Tests +import "github.com/benbjohnson/clock" +fake := clock.NewMock() +fake.Set(time.Date(2026, 1, 1, 0, 0, 0, 0, time.UTC)) +svc := &Service{clock: fake} +``` + +**Never call `time.Now()` in domain or service code.** The `time` package becomes a hidden dependency — tests become flaky, retries become time-of-day-dependent, expirations cannot be tested. + +`time.Sleep` in production code is a code smell. Use: +- `time.NewTicker` for periodic work (and a `<-ctx.Done()` exit). +- `time.NewTimer` for one-shot delays. +- `time.After` ONLY in select statements, ONLY in non-hot paths. + +--- + +## Race detector — non-negotiable in CI + +```bash +go test -race -shuffle=on -count=1 ./... +``` + +- `-race` instruments memory accesses; catches data races at runtime. ~10x slow-down — acceptable for tests, not production. +- `-shuffle=on` randomizes test order; catches hidden ordering dependencies. +- `-count=1` defeats the test cache. Without it, "passing" might mean "ran 3 weeks ago". + +If a test ONLY fails under `-race`, the bug is real. Don't disable the test; fix the race. + +--- + +## Common antipatterns + +| Bad | Why | Good | +|---|---|---| +| `go func() { ... }()` with no `ctx` plumbing | Leaks on shutdown | `errgroup.WithContext` or pass ctx | +| Bare `time.Sleep(d)` in production | Untestable, blocks | `time.NewTimer` + select with `ctx.Done()` | +| Channel of `interface{}` | Loses type | Typed channel; use sealed interface if variants needed | +| `sync.Mutex` in a struct passed by value | Locked copies, undefined behavior | Embed in pointer-receiver type; copylocks catches it | +| Locking around an entire request handler | Serializes the whole API | Lock only the smallest critical section | +| `for { select { ... } }` without `<-ctx.Done()` | Cannot stop | Add ctx case in every long-lived select | +| `sync.WaitGroup.Add(1)` inside the goroutine | Race: Wait can return before Add | Add **before** `go` | + +--- + +## Sources + +- Go memory model: https://go.dev/ref/mem +- `errgroup` package: https://pkg.go.dev/golang.org/x/sync/errgroup +- `goleak`: https://github.com/uber-go/goleak +- "Go concurrency patterns" (Pike): https://go.dev/blog/pipelines +- Sync.OnceValue blog: https://go.dev/blog/synctest (1.24+ note: `testing/synctest` for time-controlled tests is now experimental) diff --git a/packages/shared-skills/skills/programming/references/go/data-modeling.md b/packages/shared-skills/skills/programming/references/go/data-modeling.md new file mode 100644 index 000000000..c903d6ba8 --- /dev/null +++ b/packages/shared-skills/skills/programming/references/go/data-modeling.md @@ -0,0 +1,329 @@ +# Data Modeling — Three Layers of Validation + +Go has no Pydantic. Go has no Zod. **You do not need them**, but only if you wire three layers correctly. This document is the canonical pattern. + +## The three layers + +``` +┌─────────────────────────────────────────────────────────────┐ +│ HTTP / RPC / CLI │ +│ Raw bytes, strings, untrusted input │ +│ │ +│ Layer 1: validator/v10 (struct tags) ◄── parse-once │ +│ OR protovalidate (proto) │ +│ │ +└──────────────────────────┬──────────────────────────────────┘ + │ raw req → domain.X + ▼ +┌─────────────────────────────────────────────────────────────┐ +│ Domain (internal/domain) │ +│ │ +│ Layer 2: Smart constructors + unexported fields │ +│ NewEmail(s) → (Email, error) │ +│ NewUserID(s) → (UserID, error) │ +│ │ +│ Once inside this layer, NO further validation. │ +│ The types prove correctness. │ +└──────────────────────────┬──────────────────────────────────┘ + │ domain.X (proven valid) + ▼ +┌─────────────────────────────────────────────────────────────┐ +│ Storage (internal/store) │ +│ │ +│ Layer 3: sqlc-generated row structs ↔ domain types │ +│ Hand-written mappers, NOT struct tags │ +└─────────────────────────────────────────────────────────────┘ +``` + +Each layer parses once, into the next layer's types. **A function in the domain layer should never receive a raw string and validate it.** If it does, the boundary above failed. + +--- + +## Layer 1: HTTP boundary — `go-playground/validator/v10` + +```go +package handlers + +import ( + "github.com/gin-gonic/gin" + "github.com/go-playground/validator/v10" +) + +// CreateUserRequest is the wire format. Tags drive validation. +type CreateUserRequest struct { + Email string `json:"email" binding:"required,email"` + Username string `json:"username" binding:"required,alphanum,min=3,max=32"` + Age int `json:"age" binding:"required,gte=13,lte=130"` + Country string `json:"country" binding:"required,iso3166_1_alpha2"` +} + +func (h *Handler) CreateUser(c *gin.Context) { + var req CreateUserRequest + if err := c.ShouldBindJSON(&req); err != nil { + // validator returns ValidationErrors with field-by-field detail + var vErr validator.ValidationErrors + if errors.As(err, &vErr) { + c.JSON(400, gin.H{"errors": fieldErrors(vErr)}) + return + } + c.JSON(400, gin.H{"error": "invalid json"}) + return + } + + // Cross into domain — single point of failure + email, err := domain.NewEmail(req.Email) + if err != nil { + c.JSON(400, gin.H{"error": err.Error()}) + return + } + username, err := domain.NewUsername(req.Username) + if err != nil { + c.JSON(400, gin.H{"error": err.Error()}) + return + } + + user, err := h.svc.Create(c.Request.Context(), email, username, req.Age) + if err != nil { + h.writeServiceError(c, err) + return + } + c.JSON(201, user) +} + +func fieldErrors(vErr validator.ValidationErrors) map[string]string { + out := make(map[string]string, len(vErr)) + for _, fe := range vErr { + out[fe.Field()] = fe.Tag() + "(" + fe.Param() + ")" + } + return out +} +``` + +**Tag reference — the tags you actually use**: + +| Tag | Meaning | +|---|---| +| `required` | Non-zero value | +| `omitempty` (json) | Skip if zero | +| `min=N` / `max=N` | Length (strings/slices) or value (numbers) | +| `gte=N` / `lte=N` / `gt=N` / `lt=N` | Numeric comparison | +| `email` | RFC 5322-ish email | +| `url` | Valid URL | +| `uuid` / `uuid4` / `uuid7` | UUID format | +| `alphanum` / `alpha` / `numeric` | Character class | +| `iso3166_1_alpha2` | Country code (US, KR, JP) | +| `iso4217` | Currency code (USD, KRW) | +| `oneof=a b c` | Enum of literal values | +| `dive` | Apply rules to each element of slice/map | +| `eqfield=Field` | Cross-field equality (e.g., password confirm) | + +### Custom validators — register at startup + +```go +func init() { + if v, ok := binding.Validator.Engine().(*validator.Validate); ok { + _ = v.RegisterValidation("strongpassword", validateStrongPassword) + } +} + +func validateStrongPassword(fl validator.FieldLevel) bool { + s := fl.Field().String() + return len(s) >= 12 && hasUpper(s) && hasDigit(s) && hasSymbol(s) +} +``` + +Use sparingly. Most domain rules belong in smart constructors, not validators. + +--- + +## Layer 2: Domain — smart constructors + +Covered in detail in `type-patterns.md`. Recap: + +```go +package domain + +type Username struct{ raw string } + +func NewUsername(s string) (Username, error) { + s = strings.TrimSpace(s) + if len(s) < 3 || len(s) > 32 { + return Username{}, ErrInvalidUsername + } + if !isAlphanum(s) { + return Username{}, ErrInvalidUsername + } + return Username{raw: s}, nil +} + +func (u Username) String() string { return u.raw } +``` + +**Rule**: every domain type that has invariants has: + +1. An unexported field holding the raw form. +2. A `New(raw) (, error)` constructor as the sole entry point. +3. A `String() string` for printing. +4. `MarshalJSON` / `UnmarshalJSON` if it crosses a JSON boundary outside HTTP handlers (e.g., logging payloads, queue messages). +5. Optionally: `Scan` and `Value` for `database/sql` interop (rare with sqlc). + +--- + +## Layer 3: Storage — sqlc rows ↔ domain types + +sqlc generates row structs from `.sql` files. **Do not put validation tags on them.** Map between sqlc rows and domain types explicitly: + +```go +// internal/store/user_store.go +package store + +import "myservice/internal/domain" + +func (s *UserStore) Get(ctx context.Context, id domain.UserID) (domain.User, error) { + row, err := s.q.GetUser(ctx, string(id)) + if err != nil { + return domain.User{}, err + } + return rowToUser(row) +} + +func rowToUser(r sqlc.UserRow) (domain.User, error) { + email, err := domain.NewEmail(r.Email) + if err != nil { + // DB invariant broken — this is a programmer error, not a user error + return domain.User{}, fmt.Errorf("db invariant: invalid email for user %s: %w", r.ID, err) + } + username, err := domain.NewUsername(r.Username) + if err != nil { + return domain.User{}, fmt.Errorf("db invariant: invalid username: %w", err) + } + return domain.User{ + ID: domain.UserID(r.ID), + Email: email, + Username: username, + Created: r.CreatedAt, + }, nil +} +``` + +The mapping is verbose. **That is the point.** Each field is a deliberate choice; refactors flag every site. + +--- + +## Discriminated unions (sum types) at the boundary + +When a wire payload has variants (e.g., `{"type": "user.created", ...}` vs `{"type": "user.deleted", ...}`): + +```go +// Wire DTO with raw discriminator +type EventDTO struct { + Type string `json:"type" binding:"required,oneof=created deleted updated"` + Payload json.RawMessage `json:"payload" binding:"required"` +} + +// Parse into the sealed domain type +func ParseEvent(dto EventDTO) (event.Event, error) { + switch dto.Type { + case "created": + var c event.Created + if err := json.Unmarshal(dto.Payload, &c); err != nil { + return nil, fmt.Errorf("decode created: %w", err) + } + return c, nil + case "deleted": + var d event.Deleted + if err := json.Unmarshal(dto.Payload, &d); err != nil { + return nil, fmt.Errorf("decode deleted: %w", err) + } + return d, nil + case "updated": + var u event.Updated + if err := json.Unmarshal(dto.Payload, &u); err != nil { + return nil, fmt.Errorf("decode updated: %w", err) + } + return u, nil + default: + return nil, fmt.Errorf("unknown event type %q", dto.Type) + } +} +``` + +The `exhaustive` linter on the switch + the `oneof` validation tag together cover both "unknown type" and "unhandled variant". + +--- + +## Enums — typed string consts, not iota + +```go +// GOOD — string-based, JSON-serializes correctly, debuggable +type Status string + +const ( + StatusPending Status = "pending" + StatusActive Status = "active" + StatusClosed Status = "closed" +) + +func (s Status) IsValid() bool { + switch s { + case StatusPending, StatusActive, StatusClosed: + return true + } + return false +} + +func (s *Status) UnmarshalJSON(data []byte) error { + var raw string + if err := json.Unmarshal(data, &raw); err != nil { return err } + parsed := Status(raw) + if !parsed.IsValid() { return fmt.Errorf("invalid status %q", raw) } + *s = parsed + return nil +} +``` + +**Never use `iota` enums for anything that crosses a wire boundary.** They serialize as integers, which (a) breaks debuggability, (b) makes reordering enum values a silent breaking change. + +Use the validator tag `binding:"oneof=pending active closed"` to enforce at the HTTP boundary. + +--- + +## Nullable fields — `*T` vs sentinel + +Three choices, in order of preference: + +1. **Sentinel zero value**: `Age int` with `0` meaning "unknown". Works when zero is genuinely unreachable as a valid value. +2. **`sql.Null`** for DB columns: `sql.NullString`, `sql.NullInt64`, `sql.NullTime`. sqlc generates these for nullable columns. +3. **`*T`**: only when you need to distinguish "not provided" from "set to zero" in a JSON payload (PATCH semantics). + +```go +// PATCH payload — `*string` discriminates absent vs empty +type UpdateUserRequest struct { + Email *string `json:"email,omitempty"` + Username *string `json:"username,omitempty"` +} +``` + +Avoid `*T` in domain types — it bloats every consumer with nil checks. Keep `*T` at the boundary, unwrap on the way in. + +--- + +## Common AI-generated antipatterns this rejects + +| Bad | Why | Good | +|---|---|---| +| `func handle(req map[string]any)` | No types, no validation | Define a struct, parse with `validator` | +| `if email != "" { ... }` inside domain | Validation in the wrong layer | Make `email Email`, no check needed | +| `type Status int` with `iota` for wire field | Silent breaking on reorder | `type Status string` with const literals | +| Struct tags `json:"email,string"` (the `,string` coercion) | Magic coercion hides bad input | Strict parsing, fail-fast | +| `json.Unmarshal` then range-check after | Two-step "validate after parse" | Use `validator` tags or custom `UnmarshalJSON` | +| Reusing handler DTO as the domain type | Couples wire format to business logic | Two distinct types, explicit mapping | + +--- + +## Sources + +- go-playground/validator: https://github.com/go-playground/validator +- gin binding internals: https://github.com/gin-gonic/gin/blob/master/binding/json.go +- Parse, don't validate: https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-validate/ +- sqlc with custom types: https://docs.sqlc.dev/en/latest/howto/overrides.html diff --git a/packages/shared-skills/skills/programming/references/go/error-handling.md b/packages/shared-skills/skills/programming/references/go/error-handling.md new file mode 100644 index 000000000..6922862d7 --- /dev/null +++ b/packages/shared-skills/skills/programming/references/go/error-handling.md @@ -0,0 +1,359 @@ +# Error Handling + +Typed errors, wrap chains, `errors.Is` / `errors.As`, no panic in libraries, resource cleanup. Go errors look simple and are full of footguns. This document is the canonical set of moves. + +--- + +## The five rules + +1. **Every error is wrapped on the way up, with `%w`, with context.** Never `return err` from a non-trivial site. +2. **Compare with `errors.Is`, not `==`.** Wrap chains break `==`. The `errorlint` linter forbids `==` on errors. +3. **Cast with `errors.As`, not type assertion.** Same reason. +4. **`panic` is reserved for programmer errors.** Library code never panics on user input or environment failures. Use `(T, error)`. +5. **Resources released via `defer` immediately after acquisition.** No "I'll add it later". + +--- + +## Sentinel errors — for invariant programmatic checks + +```go +package domain + +import "errors" + +var ( + ErrInvalidEmail = errors.New("domain: invalid email") + ErrInvalidPhone = errors.New("domain: invalid phone") + ErrInvalidAge = errors.New("domain: invalid age") +) + +func NewEmail(s string) (Email, error) { + if !emailRe.MatchString(s) { + return Email{}, fmt.Errorf("email %q: %w", s, ErrInvalidEmail) + } + return Email{raw: strings.ToLower(s)}, nil +} +``` + +Caller branches on identity: + +```go +email, err := domain.NewEmail(input) +if errors.Is(err, domain.ErrInvalidEmail) { + return c.JSON(400, gin.H{"error": "email format"}) +} +``` + +`errors.Is` walks the wrap chain. `err == domain.ErrInvalidEmail` would have failed because `fmt.Errorf` wrapped it. + +--- + +## Typed errors — when you need structured data + +When callers need fields off the error (the offending value, the failing field name, the upstream HTTP status): + +```go +type ValidationError struct { + Field string + Value string + Rule string +} + +func (e *ValidationError) Error() string { + return fmt.Sprintf("validation: %s=%q failed %s", e.Field, e.Value, e.Rule) +} + +// Optional: identity sentinel for errors.Is comparisons +var ErrValidation = errors.New("validation") + +func (e *ValidationError) Is(target error) bool { + return target == ErrValidation +} +``` + +Caller: + +```go +err := svc.Save(ctx, user) + +var vErr *ValidationError +if errors.As(err, &vErr) { + // vErr.Field, vErr.Rule are available + c.JSON(400, gin.H{"field": vErr.Field, "rule": vErr.Rule}) + return +} +``` + +**`errors.As` requires a non-nil pointer-to-pointer.** Almost always the type is `*ConcreteError`. Forgetting the leading `*` is the most common bug here. + +--- + +## Wrapping — `%w` is mandatory + +```go +// BAD — drops context +return err + +// BAD — drops the error chain (errors.Is/As stops working) +return fmt.Errorf("failed to save user: %v", err) + +// GOOD — preserves chain via %w +return fmt.Errorf("save user %s: %w", userID, err) +``` + +The `errorlint` linter catches `%v` where `%w` was meant. **Wrap once per layer**, with the minimum useful context: + +``` +api/handler: "create user request: %w" + service: "validate inputs: %w" + domain: "email %q: %w" +``` + +Each frame adds one fact, not a duplicate. The top-level error message reads as a path: `create user request: validate inputs: email "foo": domain: invalid email`. + +### `errors.Join` — multiple errors at once + +```go +// Validate all fields, collect all errors +var errs []error +if _, err := NewEmail(req.Email); err != nil { + errs = append(errs, fmt.Errorf("email: %w", err)) +} +if _, err := NewUsername(req.Username); err != nil { + errs = append(errs, fmt.Errorf("username: %w", err)) +} +if len(errs) > 0 { + return errors.Join(errs...) +} +``` + +`errors.Is` still walks each joined error. Use when reporting batch validation, not for "wrap two unrelated errors". + +--- + +## Panics — when allowed, when banned + +**Banned**: + +- Anywhere a `(T, error)` could be returned. +- Inside HTTP handlers (gin's `Recovery` middleware catches them, but you've already lost the error context). +- Inside any goroutine that survives request lifetime. + +**Allowed** (with documentation): + +- Map literal init at package level: `var statusNames = map[Status]string{...}` followed by a `func init()` that panics if a const has no name. Catches the bug at startup, not runtime. +- The `must*` convention for genuinely unrecoverable startup: + ```go + func MustParseURL(s string) *url.URL { + u, err := url.Parse(s) + if err != nil { panic(err) } + return u + } + // Use only with literals known at compile time: + var defaultAPI = MustParseURL("https://api.example.com") + ``` +- `default:` case of an exhaustive sealed-interface switch — see `type-patterns.md`. + +The `revive` linter rule `error-return` will flag suspect panic sites; treat them as bugs. + +--- + +## `defer` for resources — the only safe pattern + +```go +func writeReport(path string) (err error) { + f, err := os.Create(path) + if err != nil { + return fmt.Errorf("create %s: %w", path, err) + } + defer func() { + if cerr := f.Close(); cerr != nil && err == nil { + err = fmt.Errorf("close %s: %w", path, cerr) + } + }() + + if _, err := f.Write(data); err != nil { + return fmt.Errorf("write %s: %w", path, err) + } + return nil +} +``` + +Key points: + +- `defer f.Close()` immediately after `os.Create` — never further down. +- Named return `(err error)` so the deferred close can mutate it on close failure. +- `bodyclose` linter catches missed `defer resp.Body.Close()` for HTTP responses. +- `sqlclosecheck` linter catches missed `defer rows.Close()` for SQL. + +### `errors.Join` for multi-stage cleanup + +```go +func process(path string) (err error) { + f, err := os.Open(path) + if err != nil { return err } + defer func() { + err = errors.Join(err, f.Close()) + }() + // ... use f ... + return nil +} +``` + +When both the main operation AND `Close` can fail, `errors.Join` reports both without dropping either. + +--- + +## HTTP error responses — a single funnel + +Build one helper, route all handler errors through it: + +```go +package httperr + +type APIError struct { + Status int `json:"-"` + Code string `json:"code"` + Message string `json:"message"` +} + +func (e *APIError) Error() string { return e.Code + ": " + e.Message } + +var ( + NotFound = &APIError{Status: 404, Code: "not_found", Message: "resource not found"} + Unauthorized = &APIError{Status: 401, Code: "unauthorized", Message: "unauthorized"} + BadRequest = &APIError{Status: 400, Code: "bad_request", Message: "bad request"} + Internal = &APIError{Status: 500, Code: "internal", Message: "internal error"} +) + +// Wrap a domain error into an API error. +func From(err error) *APIError { + if err == nil { return nil } + + var apiErr *APIError + if errors.As(err, &apiErr) { return apiErr } + + switch { + case errors.Is(err, domain.ErrInvalidEmail), + errors.Is(err, domain.ErrInvalidUsername): + return &APIError{Status: 400, Code: "validation", Message: err.Error()} + case errors.Is(err, ErrNotFound): + return NotFound + case errors.Is(err, ErrUnauthorized): + return Unauthorized + default: + // unknown — log full chain, return generic + slog.Error("unmapped error", slog.Any("err", err)) + return Internal + } +} + +func Write(c *gin.Context, err error) { + apiErr := From(err) + c.JSON(apiErr.Status, apiErr) +} +``` + +Handlers become trivial: + +```go +func (h *Handler) Create(c *gin.Context) { + user, err := h.svc.Create(c.Request.Context(), req) + if err != nil { + httperr.Write(c, err) + return + } + c.JSON(201, user) +} +``` + +--- + +## errgroup — error propagation across goroutines + +```go +import "golang.org/x/sync/errgroup" + +func fetchAll(ctx context.Context, urls []string) ([][]byte, error) { + g, ctx := errgroup.WithContext(ctx) + g.SetLimit(8) // concurrency cap + + results := make([][]byte, len(urls)) + for i, u := range urls { + g.Go(func() error { + body, err := fetch(ctx, u) + if err != nil { + return fmt.Errorf("fetch %s: %w", u, err) + } + results[i] = body + return nil + }) + } + + if err := g.Wait(); err != nil { + return nil, err + } + return results, nil +} +``` + +- `errgroup.WithContext` cancels remaining tasks on first error. +- `SetLimit` bounds concurrency. +- First non-nil error is returned; others are discarded — by design. + +See `concurrency.md` for the full pattern. + +--- + +## Logging errors — structured, once + +```go +slog.ErrorContext(ctx, "save user failed", + slog.String("user_id", string(id)), + slog.Any("err", err), // %w chain is fully rendered +) +``` + +**Log once, at the outermost frame.** Logging at every wrap site produces five log lines for one error. + +The `sloglint` linter enforces `slog.Any("err", err)` over `slog.String("err", err.Error())` — the former preserves the chain when handlers walk the value. + +--- + +## Antipatterns + +| Bad | Why | Good | +|---|---|---| +| `_ = err` | Silent ignore | Handle, log, or wrap | +| `if err != nil { return err }` chained 10 deep without wrap | No path info | Add one fact per layer: `fmt.Errorf("step: %w", err)` | +| `panic(err)` in HTTP handlers | Loses error chain, hits gin Recovery | `httperr.Write(c, err)` | +| `err.Error() == "some string"` | Brittle, breaks on wrap | Define a sentinel, use `errors.Is` | +| `if err == sql.ErrNoRows` | Breaks under wrap | `errors.Is(err, sql.ErrNoRows)` | +| `catch-all log.Fatal(err)` in library code | Crashes the caller's process | Return error, let main decide | +| Returning a typed nil pointer wrapped in error interface | Classic "nil != nil" bug | Return explicit `nil` for the error | + +The last bug deserves its own example: + +```go +// BUG — returns a non-nil error interface containing a nil concrete type +func bad() error { + var e *MyError = nil + return e // interface wraps nil pointer; errors == nil is FALSE +} + +// Caller +if err := bad(); err != nil { + // ← entered, but err.(*MyError) is nil — surprise panic +} +``` + +Fix: return explicit `nil`, not a typed nil. The `nilnil` linter catches this in `(T, error)` returns. + +--- + +## Sources + +- Go blog "Working with Errors in Go 1.13+": https://go.dev/blog/go1.13-errors +- `errors.Join` (Go 1.20+): https://pkg.go.dev/errors#Join +- errorlint: https://github.com/polyfloyd/go-errorlint +- nilaway nil-interface check: https://github.com/uber-go/nilaway diff --git a/packages/shared-skills/skills/programming/references/go/golangci-strict.md b/packages/shared-skills/skills/programming/references/go/golangci-strict.md new file mode 100644 index 000000000..c63d1e27c --- /dev/null +++ b/packages/shared-skills/skills/programming/references/go/golangci-strict.md @@ -0,0 +1,236 @@ +# Strict `.golangci.yml` (golangci-lint v2) + +The single source of truth for "is this Go code acceptable". Drop this in unmodified. **Every linter below is enabled deliberately — read the rationale before disabling one.** + +`golangci-lint` v2 changed config schema (top-level `version: "2"`). All v1 configs are incompatible. The block below is v2. + +## `.golangci.yml` + +```yaml +version: "2" + +run: + timeout: 5m + tests: true + modules-download-mode: readonly + +linters: + default: none + enable: + # ── Correctness — bug catchers ─────────────────────────────── + - govet # stdlib vet, includes shadow, fieldalignment, nilness + - staticcheck # SA1*-SA9* — the de facto Go correctness linter + - errcheck # unhandled errors. ZERO tolerance. + - errorlint # %w wrapping, errors.As vs type-assertion, errors.Is vs == + - nilerr # `return nil` after `err != nil` — classic bug + - nilnil # returning `(nil, nil)` from a (*T, error) function + - bodyclose # http.Response.Body not closed + - rowserrcheck # sql.Rows.Err() not checked + - sqlclosecheck # sql.Rows / sql.Stmt not closed + - contextcheck # functions taking context.Context don't get context.Background() + - fatcontext # context.WithValue() in a loop — leaks + - copyloopvar # Go 1.22 loop-var capture — should now use the new semantics + - intrange # use `for i := range N` (Go 1.22+) instead of `for i := 0; i < N; i++` + - usetesting # use t.TempDir/t.Setenv over os.* in tests + - testifylint # require vs assert correctness, ObjectsAreEqual misuse + + # ── Style / readability — kept narrow to avoid bikeshedding ─ + - gofumpt # stricter gofmt + - goimports # import grouping + local prefix + - whitespace # leading/trailing whitespace + - misspell # typos in comments and strings + - unconvert # redundant type conversions + - unparam # unused function parameters + - ineffassign # ineffective assignments + - dupword # duplicate words ("the the") + + # ── Architecture — file size, complexity, dead code ───────── + - gocognit # cognitive complexity per function (threshold 25) + - gocyclo # cyclomatic complexity per function (threshold 15) + - funlen # function length (90 lines, 60 statements) + - lll # line length 120 + - nestif # excessive nesting depth (>4) + - dupl # duplicate code blocks + - revive # extensible replacement for golint; selected rules below + - unused # unused vars/funcs/types + + # ── Exhaustiveness — Go's weakest spot ────────────────────── + - exhaustive # type switch and enum-like const groups completeness + + # ── Security ──────────────────────────────────────────────── + - gosec # CWE-aware security scanner + + # ── Logging ───────────────────────────────────────────────── + - sloglint # slog attr style + no slog.Any(); enforce structured logs + + # ── Performance ───────────────────────────────────────────── + - perfsprint # fmt.Sprintf where strconv suffices + - prealloc # slice prealloc when length is known + - makezero # make([]T, n) with non-zero n then append (the classic bug) + +linters-settings: + errcheck: + check-type-assertions: true + check-blank: true # `_ = err` is a violation + + govet: + enable-all: true + settings: + shadow: + strict: true + fieldalignment: + # On by default; this catches struct layouts wasting memory. + # Disable per-file with //nolint:fieldalignment ONLY for boundary types + # whose JSON tag order matters for OpenAPI doc stability. + + errorlint: + errorf: true # %w mandatory for wrapping + asserts: true # errors.As over type-assertion on `error` + comparison: true # errors.Is over == + + gocognit: + min-complexity: 25 + + gocyclo: + min-complexity: 15 + + funlen: + lines: 90 + statements: 60 + ignore-comments: true + + lll: + line-length: 120 + tab-width: 4 + + nestif: + min-complexity: 4 + + exhaustive: + default-signifies-exhaustive: false + check: + - switch + - map + + gosec: + excludes: + - G104 # handled by errcheck/errorlint + - G304 # file path provided as input — too noisy for CLIs + + sloglint: + no-mixed-args: true # all attr or all key-value, never mixed + kv-only: false + attr-only: true # force slog.String(...) form + no-global: all # disallow slog.Info; force a logger receiver + context: scope # require *Context variants where ctx is in scope + static-msg: true # msg must be a string literal (not fmt.Sprintf) + no-raw-keys: true # use slog.String("key", ...) not raw "key", "val" + key-naming-case: snake + + testifylint: + enable-all: true + disable: + - require-error # We DO use assert.Error in table-driven loops + + revive: + severity: warning + rules: + - name: var-naming + - name: package-comments + - name: exported + - name: error-return + - name: error-naming + - name: errorf # use fmt.Errorf instead of errors.New(fmt.Sprintf) + - name: if-return + - name: indent-error-flow + - name: range-val-in-closure + - name: redefines-builtin-id + - name: superfluous-else + - name: unhandled-error + arguments: + - "fmt.Print.*" + - "fmt.Fprint.*" + + perfsprint: + integer-format: true + error-format: true + bool-format: true + string-format: true + + goimports: + local-prefixes: + - github.com/your-org + +issues: + max-issues-per-linter: 0 + max-same-issues: 0 + exclude-rules: + # Tests get a longer leash on funlen + lll + - path: _test\.go + linters: + - funlen + - lll + - dupl + - gosec + # Generated code never lints + - path: \.pb\.go$ + linters: [all] + - path: \.connect\.go$ + linters: [all] + - path: ^.*sqlc/.*\.sql\.go$ + linters: [all] + +formatters: + enable: + - gofumpt + - goimports +``` + +## Per-linter rationale (why each is on) + +| Linter | What it catches | Why no compromise | +|---|---|---| +| `errcheck` (incl. `check-blank: true`) | `_ = err`, ignored errors from `Close()`, `Write()`, `json.Marshal()` | Silent error ignore is the #1 Go bug class. Banning `_ = err` forces a decision at every site. | +| `errorlint` | `err == io.EOF` instead of `errors.Is(err, io.EOF)`; missing `%w` in `fmt.Errorf` | Once you wrap in middleware, `==` checks silently break. `errors.Is/As` is the only safe form. | +| `nilerr` / `nilnil` | `return nil` after `err != nil`; `return nil, nil` from `(*T, error)` | Classic AI-generated bugs. Linter catches them mechanically. | +| `bodyclose` | `defer resp.Body.Close()` missed | Single most common Go memory leak. | +| `contextcheck` | `ctx := context.Background()` inside a function that received `ctx` | Breaks cancellation propagation — the entire reason ctx exists. | +| `exhaustive` | `switch x.(type)` missing a sealed-interface variant | **Go's weakest type-system spot.** This linter is the closest thing to compiler-enforced exhaustiveness. | +| `sloglint` | `slog.Info(...)` (global), mixed `Any`/typed attrs | Without this, structured logging silently degrades into string concatenation. | +| `govet/shadow` strict | `err := ... ; if ... { err := ...; ... }` shadowing | Hides the real error from outer scope — extremely common. | +| `govet/fieldalignment` | Struct field order wasting memory | Cheap correctness signal. Disable per-file when JSON tag order matters for OpenAPI. | +| `copyloopvar` + `intrange` | Pre-1.22 loop-var capture and old `for i := 0; i < N; i++` | The language modernized; the lint enforces it. | +| `usetesting` | `os.Setenv` / `os.Mkdir` in tests instead of `t.Setenv` / `t.TempDir` | Avoids test isolation bugs. | +| `gocognit` / `gocyclo` / `funlen` | Functions exceeding cognitive thresholds | Direct architectural signal — same purpose as the 250 LOC ceiling, at function granularity. | +| `gosec` | CWE patterns — SQL injection, weak crypto, path traversal | Production must pass this. | +| `testifylint` | `assert.Equal` where `require.Equal` was meant; `ObjectsAreEqual` misuse | Subtle test-correctness bugs. | +| `perfsprint` | `fmt.Sprintf("%d", n)` instead of `strconv.Itoa(n)` | 5–10x faster in tight loops, lints catch the lazy form. | + +## `nolint` policy + +`//nolint:linter1,linter2 // ` is permitted with **two hard rules**: + +1. **One linter at a time per directive.** No `//nolint:all`. No omitting the linter name. +2. **A reason after `//` is mandatory.** "Generated code", "false positive — protobuf imports", "OpenAPI field order" are acceptable. "Ignore" is not. + +The skill auto-rejects `//nolint` without a reason. So does `revive` if you enable its `nolint` rule. + +## CI gate + +```bash +gofumpt -l . | (! grep .) # format +golangci-lint run --timeout 5m ./... # everything above +go vet -vettool=$(which fieldalignment) ./... # extra check (also in govet) +nilaway ./... # nil-deref static analysis +go test -race -shuffle=on -count=1 ./... # races + ordering +``` + +Any non-zero exit = the change does not ship. + +## Sources + +- golangci-lint v2 docs: https://golangci-lint.run/docs/configuration/ +- staticcheck rules: https://staticcheck.dev/docs/checks +- sloglint: https://github.com/go-simpler/sloglint +- exhaustive: https://github.com/nishanths/exhaustive +- nilaway: https://github.com/uber-go/nilaway diff --git a/packages/shared-skills/skills/programming/references/go/grpc-connect.md b/packages/shared-skills/skills/programming/references/go/grpc-connect.md new file mode 100644 index 000000000..6443f675a --- /dev/null +++ b/packages/shared-skills/skills/programming/references/go/grpc-connect.md @@ -0,0 +1,375 @@ +# RPC — Connect-Go (default) + grpc-go (fallback) + protovalidate + +`connectrpc/connect-go` is the default. It is wire-compatible with gRPC, also speaks Connect protocol + gRPC-Web from browsers, and uses ordinary `net/http` so middleware (logging, auth, tracing) composes the same way as REST. Reach for raw `grpc-go` only when you need a gRPC-specific feature Connect lacks. + +--- + +## When Connect vs grpc-go + +| Need | Use | +|---|---| +| Standard unary + server-streaming + client-streaming | **Connect** | +| Browser client without `grpc-web` proxy | **Connect** (native gRPC-Web support) | +| HTTP/1.1 fallback for hostile networks | **Connect** (gRPC requires HTTP/2 end-to-end) | +| Server reflection for `grpcurl` | grpc-go (Connect has reflection too, but ecosystem smaller) | +| Bidirectional streaming with frame-level control | grpc-go | +| Strict gRPC environment (Envoy with gRPC filters, Istio strict mode) | grpc-go | + +**Default**: Connect. The default has been correct since 2024. + +--- + +## Toolchain — Buf, not protoc + +```bash +go install github.com/bufbuild/buf/cmd/buf@latest +go install google.golang.org/protobuf/cmd/protoc-gen-go@latest +go install connectrpc.com/connect/cmd/protoc-gen-connect-go@latest +go install github.com/bufbuild/protovalidate/cmd/protoc-gen-go-vtproto@latest +``` + +Buf replaces `protoc` for everything: linting, breaking-change detection, codegen, formatting. The `protoc` toolchain is dead-letter walking — every modern proto project uses Buf. + +--- + +## Project layout + +``` +proto/ + buf.yaml + buf.gen.yaml + buf.lock + myservice/v1/ + user.proto + auth.proto + +gen/ + myservice/v1/ + user.pb.go # protoc-gen-go output + auth.pb.go + myservicev1connect/ # protoc-gen-connect-go output + user.connect.go + auth.connect.go +``` + +**`gen/` is committed.** Generated code is part of the API contract; CI proves it is up-to-date. + +--- + +## `buf.yaml` + +```yaml +version: v2 +modules: + - path: proto +lint: + use: + - STANDARD +breaking: + use: + - FILE +``` + +## `buf.gen.yaml` + +```yaml +version: v2 +managed: + enabled: true + override: + - file_option: go_package_prefix + value: github.com/your-org/myservice/gen +plugins: + - remote: buf.build/protocolbuffers/go + out: gen + opt: + - paths=source_relative + - remote: buf.build/connectrpc/go + out: gen + opt: + - paths=source_relative + - remote: buf.build/bufbuild/validate-go + out: gen + opt: + - paths=source_relative +``` + +The `buf.build/...` plugin URIs use Buf's hosted remote registry — no local plugin installation needed. + +## Taskfile target + +```yaml +gen:proto: + cmds: + - buf lint + - buf format -w + - buf generate + sources: + - proto/**/*.proto + - buf.yaml + - buf.gen.yaml +``` + +Run `task gen:proto` after editing any `.proto`. CI runs `buf generate` then `git diff --exit-code` to catch stale generated code. + +--- + +## A `.proto` with validation + +```proto +syntax = "proto3"; + +package myservice.v1; + +import "buf/validate/validate.proto"; + +option go_package = "github.com/your-org/myservice/gen/myservice/v1;myservicev1"; + +service UserService { + rpc CreateUser(CreateUserRequest) returns (CreateUserResponse); + rpc GetUser(GetUserRequest) returns (GetUserResponse); + rpc StreamEvents(StreamEventsRequest) returns (stream Event); +} + +message CreateUserRequest { + string email = 1 [(buf.validate.field).string.email = true]; + string username = 2 [ + (buf.validate.field).string.min_len = 3, + (buf.validate.field).string.max_len = 32, + (buf.validate.field).string.pattern = "^[a-zA-Z0-9_]+$" + ]; + int32 age = 3 [ + (buf.validate.field).int32.gte = 13, + (buf.validate.field).int32.lte = 130 + ]; +} + +message CreateUserResponse { + User user = 1; +} + +message User { + string id = 1; + string email = 2; + string username = 3; + google.protobuf.Timestamp created_at = 4; +} +``` + +`protovalidate` replaces the abandoned `protoc-gen-validate` — it is the official Buf-backed successor as of 2024, supported by Connect's interceptor pipeline. + +--- + +## Server — Connect + +```go +package main + +import ( + "context" + "log/slog" + "net/http" + + "connectrpc.com/connect" + "buf.build/go/protovalidate" + validateinterceptor "connectrpc.com/validate" + "golang.org/x/net/http2" + "golang.org/x/net/http2/h2c" + + myservicev1 "github.com/your-org/myservice/gen/myservice/v1" + "github.com/your-org/myservice/gen/myservice/v1/myservicev1connect" +) + +type UserServer struct { + svc *UserService +} + +func (s *UserServer) CreateUser( + ctx context.Context, + req *connect.Request[myservicev1.CreateUserRequest], +) (*connect.Response[myservicev1.CreateUserResponse], error) { + + // protovalidate already ran via the interceptor below. + // req.Msg is guaranteed to satisfy the .proto constraints. + + user, err := s.svc.Create(ctx, req.Msg.Email, req.Msg.Username, req.Msg.Age) + if err != nil { + return nil, mapError(err) + } + return connect.NewResponse(&myservicev1.CreateUserResponse{ + User: userToProto(user), + }), nil +} + +func main() { + validator, _ := protovalidate.New() + interceptors := connect.WithInterceptors( + loggingInterceptor(), + validateinterceptor.NewInterceptor(validator), + ) + + mux := http.NewServeMux() + mux.Handle(myservicev1connect.NewUserServiceHandler( + &UserServer{svc: newUserService()}, + interceptors, + )) + + // h2c lets the server speak HTTP/2 cleartext for gRPC clients. + srv := &http.Server{ + Addr: ":8080", + Handler: h2c.NewHandler(mux, &http2.Server{}), + } + slog.Info("rpc server listening", slog.String("addr", srv.Addr)) + if err := srv.ListenAndServe(); err != nil { slog.Error("rpc", slog.Any("err", err)) } +} +``` + +The handler is **just an `http.Handler`** — mount it in the same `http.ServeMux` as your REST routes if you want one binary serving both. + +--- + +## Error mapping — Connect codes + +```go +func mapError(err error) error { + if err == nil { return nil } + + switch { + case errors.Is(err, domain.ErrInvalidEmail), + errors.Is(err, domain.ErrInvalidUsername): + return connect.NewError(connect.CodeInvalidArgument, err) + case errors.Is(err, ErrNotFound): + return connect.NewError(connect.CodeNotFound, err) + case errors.Is(err, ErrUnauthorized): + return connect.NewError(connect.CodeUnauthenticated, err) + case errors.Is(err, ErrConflict): + return connect.NewError(connect.CodeAlreadyExists, err) + default: + slog.Error("unmapped rpc error", slog.Any("err", err)) + return connect.NewError(connect.CodeInternal, errors.New("internal")) + } +} +``` + +Connect codes map 1:1 to gRPC codes. Clients see canonical error semantics. + +--- + +## Logging interceptor + +```go +func loggingInterceptor() connect.UnaryInterceptorFunc { + return func(next connect.UnaryFunc) connect.UnaryFunc { + return func(ctx context.Context, req connect.AnyRequest) (connect.AnyResponse, error) { + start := time.Now() + res, err := next(ctx, req) + attrs := []slog.Attr{ + slog.String("proc", req.Spec().Procedure), + slog.Duration("elapsed", time.Since(start)), + } + if err != nil { + attrs = append(attrs, slog.Any("err", err)) + slog.LogAttrs(ctx, slog.LevelWarn, "rpc failed", attrs...) + } else { + slog.LogAttrs(ctx, slog.LevelInfo, "rpc ok", attrs...) + } + return res, err + } + } +} +``` + +For streaming, implement the full `connect.Interceptor` (`WrapStreamingClient`, `WrapStreamingHandler`). Pattern is identical. + +--- + +## Server streaming + +```go +func (s *UserServer) StreamEvents( + ctx context.Context, + req *connect.Request[myservicev1.StreamEventsRequest], + stream *connect.ServerStream[myservicev1.Event], +) error { + events, errs := s.svc.Subscribe(ctx, req.Msg.UserId) + for { + select { + case <-ctx.Done(): + return ctx.Err() + case e, ok := <-events: + if !ok { return nil } + if err := stream.Send(eventToProto(e)); err != nil { + return err + } + case err := <-errs: + return connect.NewError(connect.CodeInternal, err) + } + } +} +``` + +Same shape as SSE in `backend-stack.md`. Connect handles HTTP/2 framing. + +--- + +## Client + +```go +client := myservicev1connect.NewUserServiceClient( + http.DefaultClient, + "https://api.example.com", + // Use connect.WithGRPC() if the server is grpc-go and you want strict gRPC framing. + // Default is Connect protocol — works with Connect or gRPC servers transparently. +) + +res, err := client.CreateUser(ctx, connect.NewRequest(&myservicev1.CreateUserRequest{ + Email: "a@b.com", + Username: "alice", + Age: 30, +})) +if err != nil { + var connectErr *connect.Error + if errors.As(err, &connectErr) { + slog.Error("rpc failed", + slog.String("code", connectErr.Code().String()), + slog.String("msg", connectErr.Message())) + } + return err +} +slog.Info("created", slog.String("id", res.Msg.User.Id)) +``` + +--- + +## When you genuinely need raw grpc-go + +```go +import "google.golang.org/grpc" + +lis, _ := net.Listen("tcp", ":8080") +srv := grpc.NewServer( + grpc.UnaryInterceptor(loggingUnaryInterceptor), +) +myservicev1.RegisterUserServiceServer(srv, &userServer{}) +_ = srv.Serve(lis) +``` + +The codegen is from `protoc-gen-go-grpc` (different binary from `protoc-gen-connect-go`). You can codegen **both** in the same `buf.gen.yaml` and switch by importing the right package. Most teams pick one. + +--- + +## When NOT to use RPC at all + +If your callers are all browsers, mobile apps, third-party developers, or the long tail of "things humans curl": **stay with REST + OpenAPI**. RPC's overhead is justified for service-to-service inside a single org. Outside that boundary, JSON over HTTP wins on debuggability. + +`oapi-codegen/oapi-codegen/v2` generates Go server stubs and clients from OpenAPI 3 — the REST equivalent of what Connect does for proto. Same parse-don't-validate boundary discipline, different wire format. + +--- + +## Sources + +- Connect docs: https://connectrpc.com/docs/go/getting-started +- Buf: https://buf.build/docs +- protovalidate: https://github.com/bufbuild/protovalidate +- "Why we replaced protoc with buf" (Buf blog): https://buf.build/blog +- gRPC vs Connect comparison: https://connectrpc.com/docs/introduction diff --git a/packages/shared-skills/skills/programming/references/go/libraries.md b/packages/shared-skills/skills/programming/references/go/libraries.md new file mode 100644 index 000000000..30e47b88d --- /dev/null +++ b/packages/shared-skills/skills/programming/references/go/libraries.md @@ -0,0 +1,337 @@ +# Library Defaults — Full Decision Tree (Go 2026) + +The opinionated, in-production stack for 2026 Go. Every entry has a one-line rationale and a canonical snippet so the agent does not relearn each library's idioms. + +The biggest difference from Python/Rust/TypeScript: **Go has fewer "best" choices and more "boring" choices.** The standard library is the default; reach outside it only when the rationale below applies. + +--- + +## HTTP framework — `gin` (default) or `chi` (minimalist) or `net/http` (no deps) + +The reality of 2026 Go: **`gin` runs ~48% of new Go API projects** (Go Developer Survey 2024 + crawls of new repos), with `gorilla/mux` (~17%, in maintenance), `echo` (~16%), and `fiber` (~11%) the remaining quarter. The skill picks gin not because it is technically superior — it is not — but because: + +1. The ecosystem (middleware, examples, SO answers) is largest. +2. The CLIProxyAPI codebase, which this skill's `backend-stack.md` is distilled from, uses gin in production for OpenAI/Gemini/Claude proxying including SSE streaming and WebSocket upgrades. That is real reference code, not a toy. +3. Gin's `Context` API is the closest thing Go has to a framework-blessed "request-scoped object", which makes middleware composition straightforward. + +```go +import "github.com/gin-gonic/gin" + +func main() { + r := gin.New() + r.Use(gin.Recovery(), middleware.RequestLogger(), middleware.RequestID()) + r.GET("/healthz", func(c *gin.Context) { c.JSON(200, gin.H{"ok": true}) }) + _ = r.Run(":8080") +} +``` + +**Pick `chi` instead** when: +- You want `net/http`-compatible handlers (you do, eventually — chi is closer to stdlib). +- The service is small and you do not need gin's binding helpers. + +**Pick `net/http` (stdlib) directly** when: +- The service has fewer than 10 routes and zero auth complexity. Go 1.22's enhanced `ServeMux` (method+path patterns) eliminated 80% of the historical reason to use a framework. + +**Never use** `gorilla/mux` (effectively in maintenance), `fiber` (uses `fasthttp` which is **not stdlib-compatible**, so middleware ecosystem is split), or `echo` (smaller eco than gin, no real advantage today). + +See `backend-stack.md` for the gin canonical layout, middleware ordering, SSE, graceful shutdown, structured logging integration. + +--- + +## RPC — `connectrpc/connect-go` + +The default RPC layer. **Use Connect, not raw grpc-go**, unless you have a measured reason. + +- Connect is wire-compatible with gRPC AND speaks HTTP/1.1 + HTTP/2 + Connect protocol. One server, three clients (gRPC, gRPC-Web, Connect-Web from browsers). +- No `grpcurl` needed for debugging — `curl -H "Content-Type: application/json" -d ...` works. +- Streaming, interceptors, deadlines, errors are first-class. +- Buf toolchain (`buf generate`, `buf lint`, `buf breaking`) for codegen is dramatically nicer than `protoc`. + +```go +// Server +mux := http.NewServeMux() +mux.Handle(elizav1connect.NewElizaServiceHandler(&elizaServer{})) +_ = http.ListenAndServe(":8080", h2c.NewHandler(mux, &http2.Server{})) + +// Client +client := elizav1connect.NewElizaServiceClient( + http.DefaultClient, + "http://localhost:8080", +) +res, err := client.Say(ctx, connect.NewRequest(&elizav1.SayRequest{Sentence: "hi"})) +``` + +**Use raw `grpc-go`** only when: +- You need server-streaming-from-multiple-services with a single gRPC mux. +- You are integrating with a strict gRPC-only environment (Envoy proxy with gRPC reflection, Istio strict-gRPC). + +See `grpc-connect.md`. + +--- + +## Database — `pgx/v5` + `sqlc` + `goose` + +```bash +go get github.com/jackc/pgx/v5 +go install github.com/sqlc-dev/sqlc/cmd/sqlc@latest +go install github.com/pressly/goose/v3/cmd/goose@latest +``` + +- **`pgx/v5`** is faster, more type-safe, and has better PostgreSQL feature coverage than `database/sql + lib/pq`. Use the `pgxpool` package for connection pooling. Avoid `database/sql` driver mode — it loses pgx's batch, COPY, listen/notify. +- **`sqlc`** generates type-safe Go from `.sql` files. Hand-written SQL with hand-written struct mapping is the #1 source of subtle DB bugs. sqlc eliminates the class. +- **`goose`** for migrations — small, command-line first, no global state. + +**Never use** `gorm` (active record, slow, brings runtime reflection into hot paths, encourages N+1 queries). **Never use** `ent` (heavy, opinionated graph layer) unless you specifically want a graph-shaped data model. + +See `sqlc-pgx.md`. + +--- + +## Validation — three layers, three tools + +Go has no Pydantic / Zod equivalent and **does not need one** — but only because you wire three layers properly: + +| Layer | Tool | Pattern | +|---|---|---| +| HTTP boundary (gin/chi/net/http) | `go-playground/validator/v10` via struct tags | `binding:"required,email,min=3"` | +| RPC boundary (protobuf) | `bufbuild/protovalidate-go` | `(buf.validate.field).string.min_len = 3` in `.proto` | +| Domain core | **Smart constructor + unexported fields** | `NewEmail(s) (Email, error)` returns a type whose fields cannot be set from outside | + +```go +// HTTP boundary +type CreateUserReq struct { + Email string `json:"email" binding:"required,email"` + Username string `json:"username" binding:"required,alphanum,min=3,max=32"` +} + +// Domain — once a value is of type Email it is provably valid +type Email struct{ raw string } +func NewEmail(s string) (Email, error) { + if !emailRegex.MatchString(s) { return Email{}, ErrInvalidEmail } + return Email{raw: strings.ToLower(s)}, nil +} +func (e Email) String() string { return e.raw } +``` + +The boundary parses raw input into the domain type **once**. Inside the domain, no further validation is permitted — the types prove it. This is parse-don't-validate adapted to Go. + +See `data-modeling.md` for the full pattern. + +--- + +## Logging — `log/slog` (stdlib) + +```go +import "log/slog" + +logger := slog.New(slog.NewJSONHandler(os.Stdout, &slog.HandlerOptions{ + Level: slog.LevelInfo, + AddSource: true, +})) +slog.SetDefault(logger) + +slog.InfoContext(ctx, "request handled", + slog.String("path", r.URL.Path), + slog.Int("status", 200), + slog.Duration("elapsed", elapsed), +) +``` + +- **stdlib since 1.21**, stable since 1.23. Performance is on par with zerolog for structured output, and faster than logrus by a wide margin. +- The `slog.Handler` interface is implemented by all major exporters (OpenTelemetry, Datadog, Honeycomb). +- The skill bans `logrus`, `zap`, `zerolog` for new code. They are not bad — they are simply superseded. Existing projects on those keep them; new files use slog. + +Use the `sloglint` linter from `golangci-strict.md` to enforce attr style (`slog.String(...)` instead of `slog.Any(...)`). + +--- + +## CLI — `cobra` + `pflag` + slog + +```bash +go install github.com/spf13/cobra-cli@latest +cobra-cli init mytool +cobra-cli add server +``` + +`cobra` is the de facto Go CLI framework — Kubernetes, Docker CLI, Helm, GitHub CLI all use it. The companion `viper` for config-file-+-env-+-flag merging is **optional**: prefer `caarlos0/env/v11` for env-only configs (12-factor apps), reach for viper only when you genuinely need file-based config. + +See `cobra-stack.md`. + +--- + +## TUI — `bubbletea v2` + `bubbles v2` + `lipgloss v2` + +Use **v2 RC** (`charm.land/bubbletea/v2`), not v1. The v2 model adds: + +- `tea.View{Cursor: *tea.Cursor, ...}` for real-cursor positioning. +- `SetVirtualCursor(false)` on textareas — lets the terminal own the cursor, which is **required** for CJK IME (Korean Hangul composition, Japanese kana→kanji conversion, Chinese pinyin lookup). +- Granular mouse events (`MouseClickMsg`, `MouseMotionMsg`, `MouseReleaseMsg`) instead of v1's coarse `MouseMsg`. + +This is not a preference. v1 has no way to position the IME candidate window correctly — Korean input shows up two cells to the left of where you typed, every time. **If your TUI accepts text input AND your users include CJK speakers, v1 is broken.** + +See `bubbletea-v2.md` for the full IME-correct skeleton. + +--- + +## HTTP client — stdlib + `hashicorp/go-retryablehttp` + +Default: `net/http.Client` with a tuned `http.Transport`. The stdlib client is **already excellent** in 2026 — HTTP/2 by default, connection pooling, sane timeouts when configured. + +```go +client := &http.Client{ + Timeout: 30 * time.Second, + Transport: &http.Transport{ + MaxIdleConns: 200, + MaxIdleConnsPerHost: 40, + IdleConnTimeout: 90 * time.Second, + DisableCompression: false, + ForceAttemptHTTP2: true, + }, +} +``` + +For retry/backoff, add `github.com/hashicorp/go-retryablehttp` — small, single-purpose, integrates as a wrapper. + +**Never use** `resty` (too much magic, hides headers, encourages wrong defaults). `req` is fine but adds dependency surface for marginal benefit over the stdlib + retry wrapper. + +--- + +## JSON — stdlib (default), `goccy/go-json` (perf), `bytedance/sonic` (extreme perf) + +Stdlib `encoding/json` improved dramatically in Go 1.21+. **Use it.** + +Reach for `goccy/go-json` (~3x faster) only when you have measured a hot-path bottleneck: + +```go +import json "github.com/goccy/go-json" +// drop-in replacement — same API +``` + +Reach for `bytedance/sonic` (~5x faster, requires amd64/arm64) for production proxies with thousands of RPS of JSON traversal. CLIProxyAPI uses `tidwall/gjson` + `tidwall/sjson` for **partial-tree mutation without full unmarshal** — a different optimization, useful when you transform large payloads. See `backend-stack.md`. + +--- + +## Concurrency primitives — stdlib only + +| Need | Use | +|---|---| +| Goroutine group with error propagation | `golang.org/x/sync/errgroup` | +| Semaphore | `golang.org/x/sync/semaphore` | +| Single-flight dedup | `golang.org/x/sync/singleflight` | +| Lazy init | **`sync.OnceValue` / `sync.OnceFunc`** (Go 1.21+, replaces `sync.Once` for typed values) | +| Atomic counter | `atomic.Int64` (Go 1.19+, typed atomics — don't use the old func-style) | +| Channel-based fanout | `chan T` with `errgroup` for shutdown | + +The `x/sync` packages are stdlib-quality but live outside `std`. See `concurrency.md` for the discipline. + +--- + +## Time — stdlib + `benbjohnson/clock` for tests + +```go +type Clock interface { Now() time.Time } +// Production +var realClock Clock = clockImpl{} +// Test +fake := clock.NewMock() +fake.Set(time.Date(2026, 1, 1, 0, 0, 0, 0, time.UTC)) +``` + +**Never call `time.Now()` directly inside domain code.** Inject a `Clock`. Tests become deterministic, no `time.Sleep` flakiness. + +--- + +## IDs — `google/uuid` (UUID v4/v7) or `xid` (sortable short ID) + +```go +import "github.com/google/uuid" +id := uuid.Must(uuid.NewV7()) // sortable, time-ordered, 128-bit +``` + +UUID v7 is the modern default — sortable like v6, random like v4. Use v4 only when leaking creation time is a privacy concern. + +For short, URL-safe IDs (~12 bytes, sortable) use `rs/xid` — Kubernetes-style. + +--- + +## Crypto — stdlib + `alecthomas/argon2id` for passwords + +Stdlib `crypto/*` for everything. For password hashing, **argon2id is the 2026 standard** — bcrypt is acceptable but argon2 is OWASP's recommendation since 2023. + +```go +import "github.com/alecthomas/argon2id" +hash, err := argon2id.CreateHash("password", argon2id.DefaultParams) +``` + +--- + +## Data — `apache/arrow-go/v18` + `marcboeker/go-duckdb` + `gonum` + +Same philosophy as Python's "never pandas": + +| Need | Use | +|---|---| +| Tabular over CSV/Parquet/JSON | DuckDB-Go bindings — zero-copy Arrow integration | +| In-memory frame | Arrow + custom code (Go has no pandas-equivalent and that's fine) | +| Numerical | `gonum.org/v1/gonum` | +| Stats | `gonum/stat` | + +Go's data-science story is intentionally thin. For heavy data work, write the pipeline in Polars/DuckDB (see `python/data-processing.md`), expose the result via Parquet or Arrow, consume from Go. + +--- + +## Testing — stdlib + selective additions + +| Need | Use | +|---|---| +| Assertions | `stretchr/testify/require` (fail-fast) — `assert` only in table-driven loops | +| Snapshots / golden | `hexops/autogold/v2` (auto-updates with `-update`) | +| Property-based | `pgregory.net/rapid` (modern) or stdlib `testing/quick` | +| Mocks | `go.uber.org/mock` (gomock successor) | +| HTTP mocks | `h2non/gock` for outbound, stdlib `httptest` for inbound | +| Integration containers | `testcontainers/testcontainers-go` | +| Goroutine leak | `go.uber.org/goleak` | +| Benchmarks | stdlib `testing.B` + `perf.dev/benchstat` | + +See `testing.md` for canonical patterns. + +--- + +## Config — `caarlos0/env/v11` + +```go +type Config struct { + Port int `env:"PORT" envDefault:"8080"` + DatabaseURL string `env:"DATABASE_URL,required"` + Timeout time.Duration `env:"TIMEOUT" envDefault:"30s"` +} + +var cfg Config +if err := env.Parse(&cfg); err != nil { log.Fatal(err) } +``` + +Pure 12-factor. Defaults via struct tag, required marker, parsing for `time.Duration`, slices, maps. **Use viper only if you also need file-based config** — most services do not. + +--- + +## Choosing an unfamiliar dependency — the checklist + +Before `go get`-ing anything new: + +1. Is it maintained? Latest tag within 12 months? Owner active? +2. Does it expose stdlib-compatible types (`io.Reader`, `context.Context`, `http.Handler`)? If it invents its own `Connection` or `Request` type, that's a yellow flag. +3. Does it use `init()` for side effects? **REJECT.** `init()` ruins testability. +4. Does it call `log.Fatal` / `panic` outside of true programmer-error paths? **REJECT.** +5. Does it have a `context.Context` first-arg convention? If not, **REJECT** — cancellation is non-negotiable. +6. Does adding it overlap with something already in your `go.mod`? Pick one. + +--- + +## Sources + +- 2024 Go Developer Survey: https://go.dev/blog/survey2024-h1-results +- Connect-Go docs: https://connectrpc.com/docs/go/getting-started +- sqlc: https://docs.sqlc.dev +- bubbletea v2 IME: https://github.com/code-yeongyu/bubbletea-wm (reference for `SetVirtualCursor(false)` pattern) +- CLIProxyAPI (gin + SSE + WebSocket in production): https://github.com/router-for-me/CLIProxyAPI +- slog blog: https://go.dev/blog/slog diff --git a/packages/shared-skills/skills/programming/references/go/one-liners.md b/packages/shared-skills/skills/programming/references/go/one-liners.md new file mode 100644 index 000000000..58a6f014f --- /dev/null +++ b/packages/shared-skills/skills/programming/references/go/one-liners.md @@ -0,0 +1,202 @@ +# One-Liners and Disposable Scripts + +Production hygiene with throwaway ergonomics. Go scripts get the same strict lints, the same type discipline, the same 250 LOC ceiling. The difference: they live as single `.go` files invoked via `go run`, not as full modules. + +Python has PEP 723 + `uv run`. Rust has `rust-script`. **Go has `go run` directly** — no extra tooling needed. + +--- + +## Pattern 1: Single-file `go run` + +A `.go` file with a `main` package, run directly: + +```go +//go:build ignore +// fetch.go — fetch a URL and print body length. +// +// Usage: +// go run fetch.go + +package main + +import ( + "fmt" + "io" + "log" + "net/http" + "os" +) + +func main() { + if len(os.Args) < 2 { + log.Fatal("usage: go run fetch.go ") + } + resp, err := http.Get(os.Args[1]) + if err != nil { log.Fatal(err) } + defer resp.Body.Close() + + body, err := io.ReadAll(resp.Body) + if err != nil { log.Fatal(err) } + + fmt.Printf("%d bytes\n", len(body)) +} +``` + +Run: `go run fetch.go https://example.com`. + +The `//go:build ignore` directive keeps this file out of `go build ./...` — it is a script, not part of the module. Without that line, every `.go` file in the package gets compiled into your binary. + +--- + +## Pattern 2: Throwaway directory under `scripts/` + +``` +myproject/ +├── go.mod +├── internal/... +└── scripts/ + ├── seed/ + │ └── main.go # `go run ./scripts/seed` + ├── migrate/ + │ └── main.go + └── one-time-fix/ + └── main.go +``` + +Each `scripts//main.go` is its own `main` package. Invoke as `go run ./scripts/seed/`. Dependencies are shared with the parent module — no separate `go.mod`. + +This is the right pattern when: + +- You need module deps (sqlc, pgx, your own internal packages). +- You want IDE support, type-checking, test coverage. +- The script lives alongside the project, runs in CI. + +--- + +## Pattern 3: Inline `go run` from shell + +```bash +go run -mod=mod <(cat <<'EOF' +package main +import "fmt" +func main() { fmt.Println("hello") } +EOF +) +``` + +Rare, but useful for one-shot terminal experiments. The `<(...)` is process substitution; `go run -mod=mod` reads from stdin. + +--- + +## Hard rules for scripts + +Even a 30-line script follows the philosophy: + +1. **Typed flags via `flag` or `pflag`**, not `os.Args` string parsing past 2 args. + ```go + var ( + url = flag.String("url", "", "URL to fetch") + limit = flag.Int("limit", 100, "max bytes") + ) + flag.Parse() + if *url == "" { log.Fatal("--url required") } + ``` + +2. **`context.Context` propagation** wherever I/O happens. + ```go + ctx, cancel := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM) + defer cancel() + req, _ := http.NewRequestWithContext(ctx, "GET", *url, nil) + ``` + +3. **`log.Fatal` is fine in `main()`** of a script (programmer error / fatal path), but **never inside any function the script imports.** Library code returns errors. + +4. **Errors get wrapped.** Same rule as production code: + ```go + if err != nil { return fmt.Errorf("fetch %s: %w", *url, err) } + ``` + +5. **Resources released via `defer`.** No "I'll fix it later". + +6. **slog for output if it must be parseable.** `fmt.Println` for one-shot terminal output is fine. + +7. **No more than 250 pure LOC.** If it grows, it stops being a script and becomes a subcommand of your CLI tool. + +--- + +## Pattern 4: Standalone tool with deps — temporary module + +Some scripts need deps the parent module does not have. Two options: + +### Option A — script in its own tiny module + +```bash +mkdir /tmp/migrate-tool && cd $_ +go mod init scratch.local/migrate-tool +go get github.com/pressly/goose/v3 +cat > main.go <<'EOF' +package main +import ... // use goose +func main() { ... } +EOF +go run . +``` + +Run, then delete `/tmp/migrate-tool`. Throwaway. + +### Option B — `gorun` (community tool) + +```bash +go install github.com/erning/gorun@latest + +cat > script.go <<'EOF' +//usr/bin/env gorun "$0" "$@"; exit +// /// go.mod +// module scratch +// go 1.23 +// require github.com/spf13/cobra v1.8.0 +// /// + +package main +... +EOF +chmod +x script.go +./script.go +``` + +`gorun` parses the inline `go.mod` block, materializes a temp module, runs the script. Niche tool — only if you want the executable-script experience. + +--- + +## When a script becomes a CLI + +If your script needs: + +- More than one subcommand +- Long-term storage of state +- Help text more than a paragraph +- Repeated invocations from CI + +... promote it to a real CLI tool via `cobra` — see `cobra-stack.md`. The boundary is fuzzy; trust your judgment, but **a 500-line "script" is not a script.** + +--- + +## Antipatterns + +| Bad | Why | Good | +|---|---|---| +| `os.Args[1]` indexing without length check | Panics on missing arg | `flag.Parse()` with explicit checks | +| `log.Fatal` inside a function the script imports | Crashes caller's process | Return error | +| `panic(err)` for expected failures | Same as above | `log.Fatal` in `main`, error return elsewhere | +| Skipping `defer resp.Body.Close()` because "it's a script" | Leaks fd | Always close | +| One 800-LOC `main.go` "to keep it simple" | Now harder to read than a real CLI | Promote to `cmd//` with subcommands | +| `// TODO: handle error` | Production-grade hygiene means production-grade hygiene | Handle now or document why ignored | + +--- + +## Sources + +- `go run` docs: https://pkg.go.dev/cmd/go#hdr-Compile_and_run_Go_program +- `//go:build` constraints: https://pkg.go.dev/cmd/go#hdr-Build_constraints +- `signal.NotifyContext`: https://pkg.go.dev/os/signal#NotifyContext +- gorun: https://github.com/erning/gorun diff --git a/packages/shared-skills/skills/programming/references/go/sqlc-pgx.md b/packages/shared-skills/skills/programming/references/go/sqlc-pgx.md new file mode 100644 index 000000000..6b6dbecdb --- /dev/null +++ b/packages/shared-skills/skills/programming/references/go/sqlc-pgx.md @@ -0,0 +1,471 @@ +# Database Stack — sqlc + pgx + goose + testcontainers + +The canonical 2026 PostgreSQL stack. **Type-safe SQL with zero runtime reflection**, hot-path-friendly connection pooling, sane migrations, real Postgres in tests. + +If you came here from a `gorm` project: gorm is rejected. See "Why not gorm" at the end. + +--- + +## Toolchain + +```bash +go install github.com/sqlc-dev/sqlc/cmd/sqlc@latest +go install github.com/pressly/goose/v3/cmd/goose@latest +``` + +--- + +## Layout + +``` +internal/store/ +├── sqlc.yaml # sqlc config +├── schema.sql # the cumulative DDL sqlc parses +├── queries/ # one *.sql per resource +│ ├── users.sql +│ ├── orders.sql +│ └── sessions.sql +├── sqlc/ # GENERATED — do not hand-edit +│ ├── db.go +│ ├── models.go +│ ├── users.sql.go +│ ├── orders.sql.go +│ └── sessions.sql.go +├── migrations/ # goose migrations, ordered +│ ├── 20260101000001_create_users.sql +│ └── 20260102000001_add_orders.sql +├── pool.go # pgxpool factory +├── user_store.go # domain-facing wrapper around sqlc +└── user_store_test.go # testcontainers integration test +``` + +--- + +## `sqlc.yaml` + +```yaml +version: "2" +sql: + - engine: "postgresql" + schema: "schema.sql" + queries: "queries" + gen: + go: + package: "sqlc" + out: "sqlc" + sql_package: "pgx/v5" + emit_json_tags: false + emit_prepared_queries: false + emit_interface: true # generates a Querier interface + emit_exact_table_names: false + emit_pointers_for_null_types: true + emit_empty_slices: true + overrides: + - db_type: "uuid" + go_type: + import: "github.com/google/uuid" + type: "UUID" + - db_type: "timestamptz" + go_type: + import: "time" + type: "Time" +``` + +Key choices: + +- `sql_package: "pgx/v5"` — generated code uses pgx directly, not `database/sql`. Faster, type-safer. +- `emit_interface: true` — generates a `Querier` interface. Lets stores accept either `*pgxpool.Pool` or `pgx.Tx` for transaction support. +- `emit_pointers_for_null_types: true` — nullable columns become `*T`, not `sql.NullString`. Cleaner mapping to domain types. +- `overrides` for `uuid` → `google/uuid.UUID` and `timestamptz` → `time.Time`. + +--- + +## `schema.sql` + +```sql +-- internal/store/schema.sql +-- The CUMULATIVE schema sqlc parses. Not migrations — the end state. +-- Regenerate from a fresh DB via `pg_dump --schema-only`, or hand-maintain. + +CREATE TABLE users ( + id UUID PRIMARY KEY, + email TEXT NOT NULL UNIQUE, + username TEXT NOT NULL UNIQUE, + created_at TIMESTAMPTZ NOT NULL DEFAULT NOW() +); + +CREATE INDEX idx_users_created_at ON users(created_at DESC); +``` + +--- + +## `queries/users.sql` + +```sql +-- name: GetUser :one +SELECT id, email, username, created_at +FROM users +WHERE id = $1; + +-- name: ListUsers :many +SELECT id, email, username, created_at +FROM users +ORDER BY created_at DESC +LIMIT $1 OFFSET $2; + +-- name: CreateUser :one +INSERT INTO users (id, email, username) +VALUES ($1, $2, $3) +RETURNING id, email, username, created_at; + +-- name: UpdateUserEmail :exec +UPDATE users +SET email = $2 +WHERE id = $1; + +-- name: DeleteUser :exec +DELETE FROM users WHERE id = $1; +``` + +sqlc directives: + +- `:one` — exactly one row; returns `(T, error)`. Returns `pgx.ErrNoRows` on miss. +- `:many` — zero or more rows; returns `([]T, error)`. +- `:exec` — no rows returned; returns `error`. +- `:execrows` — returns `(int64, error)` with affected row count. +- `:batchone` / `:batchmany` / `:batchexec` — pgx batch mode for bulk operations. + +Run `task gen:sqlc` (or `sqlc generate`). The generated file is committed; CI checks it is up-to-date. + +--- + +## Generated code shape (`sqlc/users.sql.go`) + +```go +// GENERATED — do not edit +type User struct { + ID uuid.UUID + Email string + Username string + CreatedAt time.Time +} + +const getUser = `-- name: GetUser :one +SELECT id, email, username, created_at FROM users WHERE id = $1` + +func (q *Queries) GetUser(ctx context.Context, id uuid.UUID) (User, error) { + row := q.db.QueryRow(ctx, getUser, id) + var u User + err := row.Scan(&u.ID, &u.Email, &u.Username, &u.CreatedAt) + return u, err +} +``` + +Type-safe inputs, type-safe outputs, compile-time-checked column-to-field mapping. **A schema change that drops a column breaks compilation.** Hand-rolled SQL would have failed at runtime. + +--- + +## `store/pool.go` + +```go +package store + +import ( + "context" + "fmt" + "time" + + "github.com/jackc/pgx/v5/pgxpool" +) + +func NewPool(ctx context.Context, dsn string) (*pgxpool.Pool, error) { + cfg, err := pgxpool.ParseConfig(dsn) + if err != nil { return nil, fmt.Errorf("parse dsn: %w", err) } + + cfg.MaxConns = 25 + cfg.MinConns = 5 + cfg.MaxConnLifetime = time.Hour + cfg.MaxConnIdleTime = 30 * time.Minute + cfg.HealthCheckPeriod = 1 * time.Minute + + pool, err := pgxpool.NewWithConfig(ctx, cfg) + if err != nil { return nil, fmt.Errorf("connect: %w", err) } + + if err := pool.Ping(ctx); err != nil { + pool.Close() + return nil, fmt.Errorf("ping: %w", err) + } + return pool, nil +} +``` + +`pgxpool.Pool` is `Querier`-compatible (implements the interface sqlc generates). Same pool flows into sqlc queries unchanged. + +--- + +## `store/user_store.go` — domain ↔ sqlc + +```go +package store + +import ( + "context" + "errors" + "fmt" + + "github.com/google/uuid" + "github.com/jackc/pgx/v5" + "github.com/jackc/pgx/v5/pgxpool" + + "github.com/your-org/myservice/internal/domain" + "github.com/your-org/myservice/internal/store/sqlc" +) + +type UserStore struct { + q *sqlc.Queries +} + +func NewUserStore(pool *pgxpool.Pool) *UserStore { + return &UserStore{q: sqlc.New(pool)} +} + +func (s *UserStore) Get(ctx context.Context, id domain.UserID) (domain.User, error) { + row, err := s.q.GetUser(ctx, uuid.UUID(id)) + if err != nil { + if errors.Is(err, pgx.ErrNoRows) { + return domain.User{}, domain.ErrUserNotFound + } + return domain.User{}, fmt.Errorf("get user %s: %w", id, err) + } + return rowToDomain(row) +} + +func (s *UserStore) Create(ctx context.Context, u domain.User) (domain.User, error) { + row, err := s.q.CreateUser(ctx, sqlc.CreateUserParams{ + ID: uuid.UUID(u.ID), + Email: u.Email.String(), + Username: u.Username.String(), + }) + if err != nil { + return domain.User{}, fmt.Errorf("create user: %w", err) + } + return rowToDomain(row) +} + +func rowToDomain(r sqlc.User) (domain.User, error) { + email, err := domain.NewEmail(r.Email) + if err != nil { + return domain.User{}, fmt.Errorf("db invariant: email %q: %w", r.Email, err) + } + username, err := domain.NewUsername(r.Username) + if err != nil { + return domain.User{}, fmt.Errorf("db invariant: username %q: %w", r.Username, err) + } + return domain.User{ + ID: domain.UserID(r.ID), + Email: email, + Username: username, + CreatedAt: r.CreatedAt, + }, nil +} +``` + +The wrapping is verbose. **That is the point.** sqlc rows are storage representations; domain types are business representations. Mapping them explicitly is where invariants are enforced. + +`pgx.ErrNoRows` becomes `domain.ErrUserNotFound` — callers never see storage-level errors. + +--- + +## Transactions — pgx.Tx satisfies the Querier interface + +```go +func (s *UserStore) CreateWithProfile( + ctx context.Context, + pool *pgxpool.Pool, + u domain.User, + p domain.Profile, +) error { + tx, err := pool.Begin(ctx) + if err != nil { return fmt.Errorf("begin: %w", err) } + defer tx.Rollback(ctx) // no-op if Commit succeeded + + q := s.q.WithTx(tx) // sqlc.Queries bound to the tx + + if _, err := q.CreateUser(ctx, /* ... */); err != nil { + return fmt.Errorf("create user: %w", err) + } + if _, err := q.CreateProfile(ctx, /* ... */); err != nil { + return fmt.Errorf("create profile: %w", err) + } + return tx.Commit(ctx) +} +``` + +Pattern: + +- `defer tx.Rollback(ctx)` immediately after `Begin` — safe even after Commit (returns "tx closed", which we ignore via the unhandled return). +- `q.WithTx(tx)` returns a `*Queries` bound to the tx. +- Last line: `tx.Commit(ctx)`. + +For nested transactions across multiple stores, accept a `Querier` parameter: + +```go +func (s *UserStore) CreateTx(ctx context.Context, q sqlc.Querier, u domain.User) (domain.User, error) { + // uses q instead of s.q — caller decides if it's pool or tx +} +``` + +--- + +## Migrations — goose + +```bash +goose -dir internal/store/migrations create create_users sql +``` + +```sql +-- migrations/20260101000001_create_users.sql +-- +goose Up +CREATE TABLE users ( + id UUID PRIMARY KEY, + email TEXT NOT NULL UNIQUE, + username TEXT NOT NULL UNIQUE, + created_at TIMESTAMPTZ NOT NULL DEFAULT NOW() +); + +-- +goose Down +DROP TABLE users; +``` + +Run: + +```bash +goose -dir internal/store/migrations postgres "$DATABASE_URL" up +goose -dir internal/store/migrations postgres "$DATABASE_URL" status +goose -dir internal/store/migrations postgres "$DATABASE_URL" down +``` + +Rules: + +- One DDL change per migration. Never combine schema + data migrations in one file. +- `Down` is real, not a stub. CI runs `up` → `down` → `up` on a fresh container to prove reversibility. +- Migrations are append-only. Never edit a merged migration; add a new one. + +`goose` can run programmatically as well: + +```go +import "github.com/pressly/goose/v3" + +if err := goose.UpContext(ctx, db, "migrations"); err != nil { ... } +``` + +Useful for tools that own their schema (CI runner, integration test setup). + +--- + +## Integration tests — testcontainers + +```go +package store_test + +import ( + "context" + "testing" + + "github.com/stretchr/testify/require" + "github.com/testcontainers/testcontainers-go/modules/postgres" +) + +func newTestDB(t *testing.T) *pgxpool.Pool { + t.Helper() + ctx := context.Background() + + pgC, err := postgres.Run(ctx, + "postgres:16-alpine", + postgres.WithDatabase("test"), + postgres.WithUsername("test"), + postgres.WithPassword("test"), + postgres.BasicWaitStrategies(), + ) + require.NoError(t, err) + t.Cleanup(func() { _ = pgC.Terminate(ctx) }) + + dsn, err := pgC.ConnectionString(ctx, "sslmode=disable") + require.NoError(t, err) + + pool, err := store.NewPool(ctx, dsn) + require.NoError(t, err) + t.Cleanup(pool.Close) + + require.NoError(t, goose.UpContext(ctx, /* sql.DB from pool */, "../migrations")) + return pool +} + +func TestUserStore_Create_returns_new_user(t *testing.T) { + // Given + pool := newTestDB(t) + s := store.NewUserStore(pool) + ctx := context.Background() + + // When + user, err := s.Create(ctx, domain.User{ + ID: domain.UserID(uuid.Must(uuid.NewV7())), + Email: mustEmail("a@b.com"), + Username: mustUsername("alice"), + }) + + // Then + require.NoError(t, err) + require.NotEmpty(t, user.ID) + + fetched, err := s.Get(ctx, user.ID) + require.NoError(t, err) + require.Equal(t, user.Email, fetched.Email) +} +``` + +testcontainers spins a real Postgres in Docker, runs migrations, hands you a pool. Tests are slow (~2s startup) but **real** — no fake that diverges from production. + +For test suites with many cases, share one container across tests in the same package via `TestMain`: + +```go +var testPool *pgxpool.Pool + +func TestMain(m *testing.M) { + ctx := context.Background() + pgC, _ := postgres.Run(ctx, "postgres:16-alpine", /* ... */) + defer pgC.Terminate(ctx) + dsn, _ := pgC.ConnectionString(ctx, "sslmode=disable") + testPool, _ = store.NewPool(ctx, dsn) + // run migrations once + os.Exit(m.Run()) +} +``` + +Each test then uses a transaction it rolls back at the end — fast and isolated. + +--- + +## Why NOT gorm + +| Concern | gorm | sqlc + pgx | +|---|---|---| +| Type safety | runtime reflection; column-to-field via tags | compile-time-checked from SQL | +| Performance | 2–5x slower than pgx | pgx is the fastest Go pg driver | +| N+1 queries | encouraged by `Preload` API | explicit JOIN in `.sql` | +| Migrations | AutoMigrate (unsafe in prod) | goose, explicit | +| Debugging | "what query did it run?" requires logging | the query IS the source | +| Cancellation | spotty ctx support | first-class | +| Active development | Yes but with churn and breaking changes | sqlc is stable | + +Existing gorm projects: leave them. New code: sqlc + pgx. + +--- + +## Sources + +- sqlc docs: https://docs.sqlc.dev +- pgx: https://github.com/jackc/pgx +- goose: https://github.com/pressly/goose +- testcontainers-go: https://golang.testcontainers.org +- pgx pool config: https://pkg.go.dev/github.com/jackc/pgx/v5/pgxpool#Config diff --git a/packages/shared-skills/skills/programming/references/go/testing.md b/packages/shared-skills/skills/programming/references/go/testing.md new file mode 100644 index 000000000..469f5a466 --- /dev/null +++ b/packages/shared-skills/skills/programming/references/go/testing.md @@ -0,0 +1,467 @@ +# Testing + +TDD shape, table-driven tests, `require` vs `assert`, snapshot tests, property-based tests, integration tests with testcontainers, goroutine-leak detection. The discipline in `programming/SKILL.md` (Given/When/Then, less mock the better, efficient AND accurate) — this document gives the Go-specific recipes. + +--- + +## Tools + +| Need | Use | +|---|---| +| Assertions | `stretchr/testify/require` (and `assert` only inside table loops) | +| Mocks | `go.uber.org/mock` (gomock successor) | +| Goroutine leaks | `go.uber.org/goleak` | +| Snapshots / golden | `hexops/autogold/v2` | +| Property-based | `pgregory.net/rapid` | +| HTTP mocks (outbound) | `h2non/gock` | +| HTTP test server (inbound) | stdlib `net/http/httptest` | +| Integration containers | `testcontainers/testcontainers-go` | +| TUI | `charm.land/bubbletea/v2/teatest` | +| Bench tooling | stdlib `testing.B` + `perf.dev/benchstat` | + +--- + +## Test naming — Given / When / Then in the name + +```go +// ──── PATTERN ──── +// Test___when_ +// OR +// Test___ + +func Test_Email_NewEmail_lowercases_input(t *testing.T) +func Test_Email_NewEmail_rejects_input_without_at_sign(t *testing.T) +func Test_UserService_Create_persists_user_when_inputs_valid(t *testing.T) +func Test_UserService_Create_returns_validation_error_when_email_invalid(t *testing.T) +``` + +A test name should answer "what behavior is this asserting?" without reading the body. Names that need a comment to explain them are misnamed. + +--- + +## Single test — explicit Given/When/Then + +```go +func Test_Email_NewEmail_rejects_input_without_at_sign(t *testing.T) { + // Given + raw := "not-an-email" + + // When + _, err := domain.NewEmail(raw) + + // Then + require.Error(t, err) + require.ErrorIs(t, err, domain.ErrInvalidEmail) +} +``` + +`require.*` fails the test immediately on miss. Use `require` for preconditions and primary assertions. Use `assert.*` only inside table-driven loops where you want all cases to report. + +--- + +## Table-driven tests + +```go +func Test_Email_NewEmail(t *testing.T) { + tests := []struct { + name string + input string + want string + wantErr error + }{ + {"lowercases", "ALICE@example.com", "alice@example.com", nil}, + {"trims whitespace", " bob@example.com ", "bob@example.com", nil}, + {"rejects missing @", "no-at-sign", "", domain.ErrInvalidEmail}, + {"rejects empty", "", "", domain.ErrInvalidEmail}, + {"rejects too long", strings.Repeat("a", 256) + "@e.com", "", domain.ErrInvalidEmail}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + // When + got, err := domain.NewEmail(tt.input) + + // Then + if tt.wantErr != nil { + require.ErrorIs(t, err, tt.wantErr) + return + } + require.NoError(t, err) + assert.Equal(t, tt.want, got.String()) + }) + } +} +``` + +Rules: + +- One **scenario** per row, not one **assertion** per row. +- Subtest names are sentences in lowercase; `t.Run(tt.name, ...)` makes them filterable: `go test -run Test_Email_NewEmail/rejects_missing_@`. +- The loop body itself is Given/When/Then in shape. +- For Go 1.22+, the loop var capture works correctly without the `tt := tt` shadow line — the `copyloopvar` linter enforces the new style. + +--- + +## Less mocks — the priority order + +In Go specifically: + +1. **Real implementation.** Domain types, pure functions, value objects — instantiate them. They are fast. +2. **In-memory fake** that satisfies the interface. Has its own test suite proving behavioral parity with the real impl. +3. **`httptest.Server`** for HTTP collaborators (real wire, no internet). +4. **`testcontainers`** for stateful collaborators (Postgres, Redis, S3-compatible, Kafka). +5. **gomock** ONLY for: clocks, randomness, third-party SaaS with no sandbox. + +### Example: an in-memory fake + +```go +// Real interface +type UserRepo interface { + Save(ctx context.Context, u domain.User) error + Get(ctx context.Context, id domain.UserID) (domain.User, error) +} + +// In-memory fake — production-quality, tested separately +type FakeUserRepo struct { + mu sync.RWMutex + users map[domain.UserID]domain.User +} + +func NewFakeUserRepo() *FakeUserRepo { + return &FakeUserRepo{users: map[domain.UserID]domain.User{}} +} + +func (r *FakeUserRepo) Save(ctx context.Context, u domain.User) error { + r.mu.Lock(); defer r.mu.Unlock() + r.users[u.ID] = u + return nil +} + +func (r *FakeUserRepo) Get(ctx context.Context, id domain.UserID) (domain.User, error) { + r.mu.RLock(); defer r.mu.RUnlock() + u, ok := r.users[id] + if !ok { return domain.User{}, domain.ErrUserNotFound } + return u, nil +} +``` + +The fake has the same observable behavior as the real one. Tests against `FakeUserRepo` survive when the production repo's internals change. Tests against a gomock stub of `UserRepo` break. + +**A test passing against a fake AND a test passing against the real impl is the gold standard.** Run the same test suite twice — once with the fake, once with testcontainers. The fakes earn their keep when the suites diverge. + +### Example: gomock for the unmockable + +```go +//go:generate mockgen -source=clock.go -destination=mocks/clock_mock.go -package=mocks + +type Clock interface { + Now() time.Time +} + +// In a test: +ctrl := gomock.NewController(t) +clock := mocks.NewMockClock(ctrl) +clock.EXPECT().Now().Return(fixedTime).AnyTimes() +``` + +Mock the narrowest seam. Never mock `UserRepo` if a fake suffices. + +--- + +## E2E scenario tests + +```go +//go:build e2e + +func Test_E2E_user_can_signup_then_login(t *testing.T) { + // Given — full server in a goroutine + ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second) + defer cancel() + + pool := newTestDB(t) // testcontainers Postgres + server := startServer(t, pool) // real gin engine on a random port + defer server.Close() + + client := server.Client() + + // When — sign up + resp, err := client.Post(server.URL+"/api/v1/users", + "application/json", + strings.NewReader(`{"email":"a@b.com","username":"alice","password":"PassWord!23"}`), + ) + require.NoError(t, err) + require.Equal(t, 201, resp.StatusCode) + + // When — log in + resp, err = client.Post(server.URL+"/api/v1/auth/login", + "application/json", + strings.NewReader(`{"email":"a@b.com","password":"PassWord!23"}`), + ) + require.NoError(t, err) + require.Equal(t, 200, resp.StatusCode) + + var body struct{ Token string `json:"token"` } + require.NoError(t, json.NewDecoder(resp.Body).Decode(&body)) + require.NotEmpty(t, body.Token) + + // Then — token works on protected endpoint + req, _ := http.NewRequestWithContext(ctx, "GET", server.URL+"/api/v1/me", nil) + req.Header.Set("Authorization", "Bearer "+body.Token) + resp, err = client.Do(req) + require.NoError(t, err) + require.Equal(t, 200, resp.StatusCode) +} +``` + +Patterns: + +- `//go:build e2e` build tag separates slow E2E from fast unit tests. Run with `go test -tags=e2e ./...`. +- One narrative per test: "user can sign up then log in". One `Test_E2E_*` per user-visible outcome. +- Real DB via testcontainers, real gin engine, real HTTP. **No mocks.** The point is to catch integration bugs. +- Bounded context — every E2E gets a `context.WithTimeout` so failures don't hang CI. + +--- + +## Goroutine leak detection + +```go +package mypkg + +import ( + "testing" + "go.uber.org/goleak" +) + +func TestMain(m *testing.M) { + goleak.VerifyTestMain(m, + goleak.IgnoreTopFunction("github.com/prometheus/client_golang/prometheus.(*Registry)..."), + ) +} +``` + +One line at the top of every package that spawns goroutines. Catches the bug class the race detector cannot. + +--- + +## Snapshot / golden tests — `autogold` + +```go +import "github.com/hexops/autogold/v2" + +func Test_RenderHelp_matches_snapshot(t *testing.T) { + // Given + cmd := newRootCmd() + + // When + out := captureOutput(t, func() { _ = cmd.Help() }) + + // Then + autogold.ExpectFile(t, out) +} +``` + +First run: `go test -update ./...` writes `testdata/Test_RenderHelp.golden`. Future runs compare; failures show a diff. Re-approve intentional changes with `-update`. + +**Use snapshots for STRUCTURE, not BEHAVIOR.** Good targets: + +- CLI `--help` output +- JSON response shape +- Generated SQL queries +- Rendered prompts (assert the structure, not exact wording — see SKILL.md prompt-test rule) + +Bad targets: a function's return value where you should `require.Equal` on the actual structure. + +--- + +## Property-based tests — `rapid` + +```go +import "pgregory.net/rapid" + +func Test_Email_NewEmail_then_String_roundtrips(t *testing.T) { + rapid.Check(t, func(t *rapid.T) { + // Given — generate valid emails + local := rapid.StringMatching(`[a-z]{3,10}`).Draw(t, "local") + domain := rapid.StringMatching(`[a-z]{3,10}\.com`).Draw(t, "domain") + raw := local + "@" + domain + + // When + e, err := domain.NewEmail(raw) + require.NoError(t, err) + + // Then — round-trip property + e2, err := domain.NewEmail(e.String()) + require.NoError(t, err) + require.Equal(t, e, e2) + }) +} +``` + +`rapid` shrinks failing cases to minimal counterexamples. Use for: + +- Round-trips (parse → serialize → parse). +- Algebraic properties (sort produces ordered, dedup is idempotent, JSON marshal/unmarshal is involutive). +- Invariants under random input (validator never panics, serializer never produces invalid UTF-8). + +--- + +## HTTP testing — `httptest` + +### Server side + +```go +func Test_GetUser_returns_user_for_existing_id(t *testing.T) { + // Given + svc := newSvcWithFake(t) + r := gin.New() + h := &Handler{Users: svc} + h.Mount(r) + + req := httptest.NewRequest("GET", "/api/v1/users/u-1", nil) + rec := httptest.NewRecorder() + + // When + r.ServeHTTP(rec, req) + + // Then + require.Equal(t, 200, rec.Code) + var body domain.User + require.NoError(t, json.Unmarshal(rec.Body.Bytes(), &body)) + require.Equal(t, "u-1", string(body.ID)) +} +``` + +### Client side — `httptest.NewServer` + +```go +func Test_Client_retries_on_500(t *testing.T) { + // Given — fake upstream + var calls int + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + calls++ + if calls < 3 { + w.WriteHeader(500) + return + } + w.WriteHeader(200) + _, _ = w.Write([]byte(`{"ok":true}`)) + })) + defer srv.Close() + + client := myclient.New(srv.URL) + + // When + err := client.DoSomething(context.Background()) + + // Then + require.NoError(t, err) + require.Equal(t, 3, calls) +} +``` + +`httptest.NewServer` spins a real HTTP server on a random port. The fake handler implements the upstream contract. Test the client against the contract, not the implementation. + +--- + +## Determinism — the cardinal rules + +- **No `time.Sleep` in tests.** If you need delay, you need a Clock injection. +- **`go test -shuffle=on`** in every CI run. +- **`go test -count=1`** to defeat the cache. +- **Subscribe to the event, do not poll for it.** Channels, callbacks, `t.Cleanup` over polling. +- **`t.Parallel()`** for tests that share no state. Speeds up large suites by 4-8x. + +A test that fails 1-in-10 runs is a bug, not flake. The race detector + `-shuffle=on` + ordering hygiene catches >95% of "flake". + +--- + +## Benchmarks — `testing.B` + `benchstat` + +```go +func Benchmark_NewEmail(b *testing.B) { + for b.Loop() { // Go 1.24+ idiom, replaces `for i := 0; i < b.N; i++` + _, _ = domain.NewEmail("alice@example.com") + } +} +``` + +Run: + +```bash +go test -bench=. -count=10 -benchmem ./... | tee bench.txt +benchstat bench.txt # statistical comparison +``` + +Always `-count=10` for stable means. `-benchmem` reports allocations. A 5%-slower benchmark in one run is noise; 10 runs + benchstat tells you what is real. + +To compare before/after a change: + +```bash +git stash +go test -bench=. -count=10 ./... > before.txt +git stash pop +go test -bench=. -count=10 ./... > after.txt +benchstat before.txt after.txt +``` + +--- + +## Coverage — the right target + +Run: + +```bash +go test -race -shuffle=on -coverprofile=cover.out ./... +go tool cover -html=cover.out -o cover.html +``` + +**Aim for 80%+ on `internal/domain` and `internal/service`.** Boundary code (handlers, store mappers) is exercised by integration tests, where line coverage understates what is actually verified. Do not chase 100% — the last 5% is usually error paths that need fault-injection to hit. + +The `golangci-lint` config does not enforce a minimum — coverage as a CI gate becomes a goal-displacement metric. Treat it as feedback, not requirement. + +--- + +## TUI testing — `teatest` + +```go +import teatest "charm.land/bubbletea/v2/teatest" + +func Test_Counter_increments_on_space(t *testing.T) { + // Given + tm := teatest.NewTestModel(t, initial(), teatest.WithInitialTermSize(80, 24)) + + // When + tm.Send(tea.KeyPressMsg{Code: ' '}) + + // Then + final := tm.FinalModel(t).(model) + require.Equal(t, 1, final.count) +} +``` + +For full-view regression, snapshot the rendered output via `autogold`. + +--- + +## Antipatterns the skill rejects + +| Bad | Why | Good | +|---|---|---| +| `if got != want { t.Errorf("expected %v got %v", want, got) }` | Reinvents `require.Equal` | Use testify | +| `time.Sleep(100 * time.Millisecond)` after triggering async work | Flake | Subscribe to completion signal, bounded await | +| `t.Skip(...)` to silence a known failure | Buries the bug | Fix or open an issue; never silently skip | +| One mega-test asserting 12 things | First failure hides next 11 | Split by `Then` | +| Snapshot-everything | Locks formatting, not behavior | Snapshots for structure, asserts for values | +| Mock every collaborator | Test asserts implementation, not behavior | Real or fake, never mock everything | +| Test calls private function via `_test.go` in same package only | Couples test to implementation | Test through the public surface | + +--- + +## Sources + +- testify: https://github.com/stretchr/testify +- goleak: https://github.com/uber-go/goleak +- autogold: https://github.com/hexops/autogold +- rapid: https://pkg.go.dev/pgregory.net/rapid +- testcontainers-go: https://golang.testcontainers.org +- benchstat: https://pkg.go.dev/golang.org/x/perf/cmd/benchstat +- "Go test naming conventions" (Dave Cheney): https://dave.cheney.net/practical-go/presentations/qcon-china.html diff --git a/packages/shared-skills/skills/programming/references/go/type-patterns.md b/packages/shared-skills/skills/programming/references/go/type-patterns.md new file mode 100644 index 000000000..e65b20d5f --- /dev/null +++ b/packages/shared-skills/skills/programming/references/go/type-patterns.md @@ -0,0 +1,298 @@ +# Type Patterns + +How to use Go's *limited* type system to catch bugs at compile time. Go gives you fewer tools than Python/TS/Rust — this document covers the four patterns that buy back most of the safety. + +The four patterns: + +1. **Named types** for branding primitives (the Go answer to `NewType` / branded TS). +2. **Smart constructors with unexported fields** for parse-don't-validate. +3. **Sealed interfaces** for sum types, with `type switch` + `exhaustive` linter. +4. **Generics with constraints** for bounded polymorphism (1.18+). + +--- + +## 1. Named types — distinct primitives + +Same underlying type, different meaning. The Go type checker prevents *implicit* mixing — but explicit conversion is always possible. Treat this as a contract enforced at boundaries. + +```go +package domain + +type UserID string +type OrderID string +type EmailRaw string // raw, unvalidated string from input + +func GetUser(id UserID) User { /* ... */ } + +uid := UserID("u-123") +oid := OrderID("o-456") + +GetUser(uid) // ✅ OK +GetUser(oid) // ❌ cannot use oid (type OrderID) as UserID +GetUser("u-123") // ❌ untyped string literal — Go DOES catch this +GetUser(UserID("u-123")) // ✅ explicit conversion — accept it +``` + +**Use when**: IDs, opaque tokens, foreign keys, units that share a base primitive. + +**Reality check**: Go does NOT prevent `UserID(orderIDAsString)`. The defense is **smart constructors** for everything beyond an internal identifier. Use named types for cheap brand-only protection; combine with constructors for protection that actually holds. + +### Time-of-day units + +```go +type Milliseconds int64 +type Seconds int64 + +func (ms Milliseconds) ToSeconds() Seconds { + return Seconds(ms / 1000) +} +``` + +No implicit `Milliseconds + Seconds`. The compiler refuses. Convert explicitly. + +--- + +## 2. Smart constructors with unexported fields — the Go answer to Pydantic/Zod + +The single most important pattern in this document. **Go has no Pydantic. It has this.** + +```go +package domain + +import ( + "errors" + "regexp" + "strings" +) + +var ( + ErrInvalidEmail = errors.New("invalid email") + emailRe = regexp.MustCompile(`^[^@\s]+@[^@\s]+\.[^@\s]+$`) +) + +// Email is a parsed, lowercased, valid email address. +// The zero value is invalid; construct via NewEmail. +type Email struct { + raw string // unexported — cannot be set from outside the package +} + +func NewEmail(s string) (Email, error) { + s = strings.TrimSpace(strings.ToLower(s)) + if !emailRe.MatchString(s) { + return Email{}, ErrInvalidEmail + } + return Email{raw: s}, nil +} + +// String implements fmt.Stringer for printing. +func (e Email) String() string { return e.raw } + +// MarshalJSON keeps the wire format unchanged. +func (e Email) MarshalJSON() ([]byte, error) { + return []byte(`"` + e.raw + `"`), nil +} + +// UnmarshalJSON is the parsing boundary — strict mode. +func (e *Email) UnmarshalJSON(data []byte) error { + if len(data) < 2 || data[0] != '"' || data[len(data)-1] != '"' { + return ErrInvalidEmail + } + parsed, err := NewEmail(string(data[1 : len(data)-1])) + if err != nil { + return err + } + *e = parsed + return nil +} +``` + +**Why this works**: + +- `Email{raw: "anything"}` from outside the `domain` package is a compile error — `raw` is unexported. +- The only way to obtain a non-zero `Email` is `NewEmail(...)`, which validates. +- `UnmarshalJSON` routes wire input through the same constructor — boundary parsing is automatic. +- Once a function signature has `email Email`, the caller has *proven* it is valid. No internal `if email == ""` checks. + +**Use for every domain value that has invariants**: emails, URLs, phone numbers, currency amounts, percentages, semver versions, IDs with format constraints, time ranges, anything you currently validate in three places. + +### The "zero value problem" + +Go's zero value (`Email{}`) is reachable. The mitigation is documentation + a `IsValid()` method when needed: + +```go +func (e Email) IsZero() bool { return e.raw == "" } +``` + +Or accept it: receivers that take `Email` should *never* receive a zero-value `Email` in correct code. Tests verify it. + +--- + +## 3. Sealed interfaces — sum types in Go + +Go has no sum types. The closest thing: an interface with an **unexported method** that only types in the same package can satisfy, dispatched via `type switch`, with the `exhaustive` linter ensuring completeness. + +```go +package event + +// Event is a closed sum: Created | Updated | Deleted. +// The sealed() method is unexported so external packages cannot add variants. +type Event interface { + sealed() + OccurredAt() time.Time +} + +type Created struct { + UserID UserID + Email Email + Timestamp time.Time +} +func (Created) sealed() {} +func (e Created) OccurredAt() time.Time { return e.Timestamp } + +type Updated struct { + UserID UserID + Changes map[string]any + Timestamp time.Time +} +func (Updated) sealed() {} +func (e Updated) OccurredAt() time.Time { return e.Timestamp } + +type Deleted struct { + UserID UserID + Reason string + Timestamp time.Time +} +func (Deleted) sealed() {} +func (e Deleted) OccurredAt() time.Time { return e.Timestamp } +``` + +Consumer code: + +```go +func Render(e event.Event) string { + switch v := e.(type) { + case event.Created: + return fmt.Sprintf("created %s with %s", v.UserID, v.Email) + case event.Updated: + return fmt.Sprintf("updated %s: %v", v.UserID, v.Changes) + case event.Deleted: + return fmt.Sprintf("deleted %s (reason: %s)", v.UserID, v.Reason) + default: + panic(fmt.Sprintf("unhandled event variant: %T", v)) + } +} +``` + +The `panic` in `default` is the Go equivalent of TS's `assertNever` or Python's `assert_never`. It is only reachable if a new variant is added without updating the switch. + +### The `exhaustive` linter — your compiler + +```yaml +# .golangci.yml +linters: + enable: [exhaustive] +linters-settings: + exhaustive: + check: + - switch + - map + default-signifies-exhaustive: false +``` + +Now adding `event.Suspended` without updating `Render` is a **lint error**. This is the closest thing Go has to Rust's match exhaustiveness check. **Treat it as compulsory.** + +### Sealed interface gotchas + +- The method MUST be unexported (`sealed()`, not `Sealed()`). Otherwise other packages can implement it. +- `type switch` with `*Created` vs `Created` matters — pick value receivers and value cases, or pointer receivers and pointer cases. **Mixing them causes silent miss.** +- `interface{}` is not a sealed type. Anything implementing zero methods satisfies it. Sealed interfaces have at least the `sealed()` method. + +--- + +## 4. Generics with constraints — bounded polymorphism + +Go 1.18+. Use for genuinely generic algorithms; **do not** use for "I want this to accept anything". + +```go +import "cmp" + +// Ordered constraint includes all ordered types (int, float, string, …). +func Max[T cmp.Ordered](a, b T) T { + if a > b { return a } + return b +} + +// Custom constraint +type Stringer interface { + String() string +} + +func Join[T Stringer](items []T, sep string) string { + parts := make([]string, len(items)) + for i, item := range items { + parts[i] = item.String() + } + return strings.Join(parts, sep) +} +``` + +The `cmp.Ordered` (Go 1.21+), `cmp.Compare`, and `slices`/`maps` packages cover the common cases without you writing constraints. + +### When NOT to use generics + +- "I want to accept multiple types, so I'll make it generic." Use an **interface** instead. Generics are for parametric polymorphism (same code, different types). Interfaces are for behavioral polymorphism (different code behind a contract). +- "I want to return `any`." Use a sealed interface and a `type switch`. `any` returns are anti-patterns past public APIs. + +--- + +## 5. Type assertions — the controlled escape hatch + +```go +// Bad — panics on failure +e := evt.(event.Created) + +// Good — comma-ok form, always +if e, ok := evt.(event.Created); ok { + // use e +} + +// Use errors.As for error chains +var pgErr *pgconn.PgError +if errors.As(err, &pgErr) { + // pgErr is the wrapped pg error +} +``` + +**The `errcheck` and `errorlint` linters reject bare type assertions on `error` values.** Use `errors.As`. See `error-handling.md`. + +--- + +## 6. Pointers vs values — the only durable rule + +You will see endless debates. The rule that holds up: + +- **If a type has a mutex, never copy it.** Use `*T` everywhere. +- **If a type is large (> 64 bytes) and read-only, pass by value or pointer is a measured choice.** Default to pointer for "large" things. +- **Receivers must be consistent.** All methods on `T` either take `T` or `*T`. Don't mix. The `staticcheck` linter catches mixed-receiver bugs. +- **`nil` pointer = absence. Zero value = "not set yet".** Choose ONE convention per type. Document it. + +--- + +## 7. `any` / `interface{}` — when it is acceptable + +Almost never in domain code. Acceptable cases: + +- JSON parsing of genuinely heterogeneous payloads (and even then, prefer `json.RawMessage` + targeted parsing). +- `fmt.Sprintf` arguments (variadic `any` is unavoidable here). +- Generic container internals before the user-facing API. + +The skill rejects `any` in handler signatures, service signatures, store signatures. If you find yourself writing `func Handle(payload any) error`, you have a sealed-interface waiting to happen. + +--- + +## Sources + +- "Parse, don't validate" — Alexis King: https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-validate/ +- exhaustive linter: https://github.com/nishanths/exhaustive +- Generics constraints: https://go.dev/blog/intro-generics +- cmp.Ordered: https://pkg.go.dev/cmp