Overview

A problem is what you hand hillclimb: a task for the coding agent, and a verifier that scores every solution. The easiest way to start is hillclimb problem new <problem>, which writes a small problem that already runs, for you to edit into your own:

hillclimb problem new my-problem          # problems/my-problem/, a working example
hillclimb verify my-problem --repeat 3    # scores its baseline; run it after every edit
hillclimb run my-problem --budget 10m     # then climb

problems/my-problem/
├── problem.yaml      # the metric and which way is better
├── description.md    # the task, as the coding agent reads it
├── contract.md       # the function solution.py must define
├── run.py            # calls the solution and writes what it returned
├── score.py          # checks that and writes the score
├── instances.py      # the inputs, shared by run.py and score.py
└── baseline.py       # the starting solution, scored first

The example splits lists of numbers into two halves with equal sums. Replace it piece by piece: the inputs in instances.py, the function in contract.md and run.py, the check and the score in score.py, and the words in description.md. You can also copy one of the pre-created problems with hillclimb problem get <problem> and change it. These pages take it one part at a time:

To define a problem in Python instead, from a scoring function, see Problem in the Python SDK.