refactor(explore): make ast_grep_search discipline the core of Tool Strategy
The previous Tool Strategy was a neutral 5-bullet list that treated
ast_grep_search and grep as equals. LLMs read 'structural patterns
(function shapes, class structures)' and reach for ast_grep_search
first, then call it with regex ('foo|bar', '.*', '\\w') and silently
get zero results.
Rewrite so the default is clear - grep first, ast_grep_search only for
true AST shape matching - and enumerate the regex anti-patterns with
their corrective switches. Add an explicit rule: if ast_grep_search
returns zero matches and the printed hint says the pattern is regex-
shaped, switch to grep instead of retrying with another regex variant.
Preserves the existing absolute-path requirement, <results> block
format, and read-only / no-emoji constraints.
This commit is contained in:
@@ -0,0 +1,92 @@
|
||||
/// <reference types="bun-types" />
|
||||
|
||||
import { describe, expect, it } from "bun:test"
|
||||
import { createExploreAgent } from "./explore"
|
||||
|
||||
describe("explore agent tool strategy", () => {
|
||||
const model = "openai/gpt-5.4-mini-fast"
|
||||
|
||||
it("#given the prompt #when inspecting #then defaults to grep for most searches", () => {
|
||||
// given
|
||||
const agent = createExploreAgent(model)
|
||||
|
||||
// when
|
||||
const prompt = agent.prompt ?? ""
|
||||
|
||||
// then
|
||||
expect(prompt.toLowerCase()).toContain("default to `grep`")
|
||||
})
|
||||
|
||||
it("#given the prompt #when inspecting #then warns against regex in ast_grep_search", () => {
|
||||
// given
|
||||
const agent = createExploreAgent(model)
|
||||
|
||||
// when
|
||||
const prompt = agent.prompt ?? ""
|
||||
|
||||
// then
|
||||
expect(prompt).toContain("ast_grep_search")
|
||||
expect(prompt.toLowerCase()).toContain("not use regex")
|
||||
expect(prompt).toContain("|")
|
||||
expect(prompt).toContain(".*")
|
||||
expect(prompt).toContain("\\w")
|
||||
})
|
||||
|
||||
it("#given the prompt #when inspecting #then mandates falling back to grep on regex-shaped patterns", () => {
|
||||
// given
|
||||
const agent = createExploreAgent(model)
|
||||
|
||||
// when
|
||||
const prompt = agent.prompt ?? ""
|
||||
|
||||
// then
|
||||
expect(prompt.toLowerCase()).toContain("switch to grep")
|
||||
})
|
||||
|
||||
it("#given the prompt #when inspecting #then gives concrete AST pattern examples", () => {
|
||||
// given
|
||||
const agent = createExploreAgent(model)
|
||||
|
||||
// when
|
||||
const prompt = agent.prompt ?? ""
|
||||
|
||||
// then
|
||||
expect(prompt).toContain("$$$")
|
||||
expect(prompt).toContain("function $NAME")
|
||||
})
|
||||
|
||||
it("#given the prompt #when inspecting #then tells LLM to read the returned hint before retrying", () => {
|
||||
// given
|
||||
const agent = createExploreAgent(model)
|
||||
|
||||
// when
|
||||
const prompt = agent.prompt ?? ""
|
||||
|
||||
// then
|
||||
expect(prompt.toLowerCase()).toContain("read the hint")
|
||||
})
|
||||
|
||||
it("#given the prompt #when inspecting #then preserves the absolute-path requirement", () => {
|
||||
// given
|
||||
const agent = createExploreAgent(model)
|
||||
|
||||
// when
|
||||
const prompt = agent.prompt ?? ""
|
||||
|
||||
// then
|
||||
expect(prompt).toContain("absolute")
|
||||
expect(prompt).toContain("<results>")
|
||||
})
|
||||
|
||||
it("#given the prompt #when inspecting #then keeps the read-only and no-emoji constraints", () => {
|
||||
// given
|
||||
const agent = createExploreAgent(model)
|
||||
|
||||
// when
|
||||
const prompt = agent.prompt ?? ""
|
||||
|
||||
// then
|
||||
expect(prompt).toContain("Read-only")
|
||||
expect(prompt).toContain("No emojis")
|
||||
})
|
||||
})
|
||||
+22
-6
@@ -106,12 +106,28 @@ Your response has **FAILED** if:
|
||||
|
||||
## Tool Strategy
|
||||
|
||||
Use the right tool for the job:
|
||||
- **Semantic search** (definitions, references): LSP tools
|
||||
- **Structural patterns** (function shapes, class structures): ast_grep_search
|
||||
- **Text patterns** (strings, comments, logs): grep
|
||||
- **File patterns** (find by name/extension): glob
|
||||
- **History/evolution** (when added, who changed): git commands
|
||||
**Default to \`grep\`** for almost every search. Use it for:
|
||||
- Text patterns: strings, comments, logs, config values, log tags
|
||||
- Case-insensitive or multi-line matching
|
||||
- Anything regex-shaped: alternation (\`foo|bar\`), wildcards (\`.*\`), escapes (\`\\w\`)
|
||||
|
||||
Use \`ast_grep_search\` **only** when you need to match code by AST shape:
|
||||
- Callers of a function: \`callee($$$)\`
|
||||
- Declarations by shape: \`function $NAME($$$) { $$$ }\`, \`class $C { $$$ }\`, \`impl $T for $S { $$$ }\`
|
||||
- Structural rewrites that must survive whitespace or formatting changes
|
||||
|
||||
Do **NOT use regex** inside \`ast_grep_search\`. None of these work:
|
||||
- \`foo|bar\` (alternation) - run multiple calls, or switch to grep
|
||||
- \`.*\`, \`.+\` (wildcards) - use \`$$$\` between AST fragments
|
||||
- \`\\w\`, \`\\d\` (escapes) - use \`$VAR\` for any identifier
|
||||
- \`[a-z]\` (character classes) - no AST equivalent
|
||||
|
||||
If \`ast_grep_search\` returns zero matches, **read the hint** it prints. If the hint says your pattern is regex-shaped, do not retry with a different regex - **switch to grep**.
|
||||
|
||||
Other tools:
|
||||
- **LSP** - semantic navigation: definitions, references, workspace symbols
|
||||
- **glob** - files by name or extension
|
||||
- **git** - history, blame, who-changed-what
|
||||
|
||||
Flood with parallel calls. Cross-validate findings across multiple tools.`,
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user