# Hillclimbing

Hillclimbing improves an agent one measured change at a time. Hold the inputs fixed, change one
thing, and keep it only when the same run gets better without moving the behavior you froze.

A coding agent in your checkout can run this loop by following the `agent-sdk-hillclimb` skill.

## Run one measured round

First agree on the agent, one to three fixed inputs, the metric to improve, and the behavior that
must not change, such as the output shape, side effects, and existing evals.

Then run one round:

1. Run the agent the way a user would, on saved or local inputs, without deploying or posting live.
2. Read the trajectory and name the main failure in one sentence.
3. Change one lever.
4. Rerun the same fixtures.
5. Compare correctness, efficiency, and contract fit.
6. Keep, revert, or narrow the change.

Don't swap a fixture mid-round, or the comparison means nothing.

## Choose the smallest lever

Start with the earliest place that can remove the failure. Prepare evidence on the host before the
turn, then change the shape of that evidence, then the instructions or skills, then remove or gate
tools that invite wandering, and change the framework only when the agent can't express the fix.

Fixing the host or evidence often improves quality and latency together, because the model searches
less for ground truth.

## Match the fix to the failure

When the agent picks the wrong tool, such as a PR reviewer that calls `approve_pr` before
`inspect_pr`, pin one PR and tighten only the instruction that owns that decision, so you know which
edit fixed it.

```ts
t.calledTool("inspect_pr");
t.notCalledTool("approve_pr");
```

When a run wanders or runs long, it often repeats reads or makes the model find evidence the host
already has. Move that preparation ahead of the turn, and keep the change only if the answer still
finds the same issue.

```text
before: 14 tool calls, 92 seconds, 6 repeated reads
after:   5 tool calls, 31 seconds, 0 repeated reads
```

When the answer is weak, freeze the public shape, compare both runs against known-good findings, and
edit the instruction or skill that owns the missing judgment.

```text
fixture: saved checkout PR diff
success: report both gold findings with file paths
freeze:  findings JSON shape; no review post; existing smoke evals
```

## Lock every kept win

Each kept change needs an eval that would have failed before it, such as a tool-choice assertion, a
`maxToolCalls` bound, an output-shape check, or a quality score.

```bash
agent-sdk eval --dir . --tag smoke
```

Don't loosen an existing eval so that a new change passes. If an older eval starts failing, the
change broke something that used to work, and you should fix the change instead.
