walkthroughhillclimb

Walkthrough

The Heilbronn triangle problem end to end, with the figures the run produced.

Five steps, one problem: 13 points in the plane, scored by the smallest triangle they form relative to their convex hull. Every figure below is from the run itself — the same data hillclimb chart, hillclimb treeclimb and hillclimb graph draw in the terminal.

01 — Specify the problem

Start in an empty folder. hillclimb init makes it a hillclimb dir, where everything hillclimb reads and writes will live:

mkdir heilbronn && cd heilbronn
hillclimb init
ls -a
.gitignore    climbers/    hillclimb.yaml    problems/    runs/

hillclimb.yaml holds your settings, problems/ the problems, runs/ everything a search produces (and runs/config.yaml, how runs climb by default), and climbers/ your own climbers, empty until you hillclimb climber get one. Then copy the Heilbronn problem in:

hillclimb problem get heilbronn-convex-13
ls problems/heilbronn-convex-13
baseline.py    description.md    interface.py    problem.yamlsample_submission.csv    verifier.sh    verify.py

description.md is what the coding agent reads, and verifier.sh runs a solution and then verify.py, the scorer. hillclimb runs only verifier.sh; verify.py is how this problem happens to score inside it, and is where most of the problem is written. For the Heilbronn problem, the scorer looks something like the following:

problems/heilbronn-convex-13/verify.py (shortened)
# 13 points in the plane; score = min triangle area / convex hull area
N, N_TRIANGLES = 13, 286

def fail(reason):
    write_report(0.0, report_error=reason)   # invalid configurations score 0
    sys.exit(0)

df = pd.read_csv("submission.csv")   # id,x,y
if len(df) != N or sorted(df["id"].tolist()) != list(range(N)):
    fail(f"need exactly {N} rows with id 0..{N - 1}")
points = df.sort_values("id")[["x", "y"]].to_numpy(float)
if not np.all(np.isfinite(points)):
    fail("coordinates must be finite")

# translation and uniform scaling leave the score unchanged; normalize
# so very large or very small valid coordinates stay numerically safe
points = points - points[0]
points = points / np.max(np.abs(points))

hull_area = polygon_area(convex_hull(points))
if hull_area <= np.finfo(float).eps:
    fail("degenerate convex hull")

# the smallest of all C(13,3) = 286 triangles, relative to the hull
min_area = min(
    0.5 * abs(cross(points[i], points[j], points[k]))
    for i, j, k in combinations(range(N), 3)
)
write_report(min_area / hull_area)   # -> $HILLCLIMB_RESULT

Exiting 0 means the candidate is valid, and whatever the scorer writes to $HILLCLIMB_RESULT is its score: a bare number, or {"score": 0.0309}. The full file also reports the six smallest triangles and their areas, so an improve operator sees where the configuration is weakest, not just the final number. Tip: make the verifier hard to fool: whatever it fails to check, the search will eventually exploit.

A config file in the same folder problem.yaml names the metric, the winning direction, and any baseline scores the chart draws as reference lines:

problems/heilbronn-convex-13/problem.yaml
problem_id: heilbronn-convex-13
metric: normalized-min-triangle-area
higher_is_better: true
description: description.md
baseline: baseline.py
baseline_files: {submission.csv: sample_submission.csv}
chart_baselines:
  "OpenEvolve (GPT-5, 100 candidates)": 0.0267
  "AdaEvolve (GPT-5, 100 candidates)": 0.0290
  "AlphaEvolve": 0.030936889034895654
allow_internet_during_solution: false

The budget is wall-clock time: hillclimb keeps drafting and scoring candidates until it is spent, and that is the only stopping rule. --agent picks the coding agent that writes the candidates (claude-code, codex or pi) and --model the model it uses; both can be set once in runs/config.yaml and left off the command, and so can the budget. Everything a search produces, from every candidate's code to its logs and scores, lands under runs/.

hillclimb run heilbronn-convex-13 \
  --budget 30m \
  --parallel-searches 2 \
  --parallel-agents 3 \
  --agent claude-code \
  --model claude-sonnet-5

The command gives you the prompt back right away; the searches run in the background. A run folder has appeared under runs/, with one folder per search:

ls runs/*/
logs/    run.yaml    searches/    spec.yaml
ls runs/*/searches/
heilbronn-convex-13/    heilbronn-convex-13-2/

Agents now take turns as operators: draft a new approach, debug one that crashed, improve the best scorer so far, ensemble the survivors at the end. Every new candidate runs through your verifier, and the best solution.py is kept in the search's best/ folder. Each attempt gets a folder of its own under candidates/:

ls runs/*/searches/heilbronn-convex-13/
best/    candidates/    climber/    journal.jsonl    search.yaml    status.json
ls -1 runs/*/searches/heilbronn-convex-13/candidates/c001/
prompt.md            # what the coding agent was askedagent_stream.jsonl   # what it didnotes.md             # what it says it didsolution.py          # the code it wrotesubmission.csv       # what the code producedeval_result.json     # the score the verifier gave itexec_stdout.log      # the verifier's outputtrials/              # each scored run of that code

03 — Watch it climb

Now watch the agent work with:

hillclimb watch    # the tree, live
hillclimb chart    # the curve below

Every search of the problem, as hillclimb chart plots it: each dot is a scored candidate in landing order, the line is the best score so far across all searches, and the published scores from problem.yaml are the reference lines.

The same search as a tree beside its climb: one circle per candidate, filled by score and ringed by what the search did with it. Drag the slider or press play to watch both grow.

tree
climb

04 — Inspect the solution

What a search leaves behind is code:

ls runs/*/searches/heilbronn-convex-13/best/
params.json    solution.py    submission.csv

best/solution.py is the winning program and best/submission.csv is what it produced. When the solution declares tunable parameters, best/params.json holds the values of its best trial. This search's winner is the draft itself: a two-phase global search that polishes structured and random starts across hull sizes 8–13 with an SLSQP solve (maximize the smallest triangle area with the hull pinned to area 1), then perturbs the leader until time runs out. Both parallel searches converged on the same configuration.

runs/<run-id>/searches/heilbronn-convex-13/best/solution.py (shortened)
"""Multi-start global search over diverse hull/interior topologies for
the normalized Heilbronn problem (13 points), each candidate polished
with an epigraph SLSQP formulation (maximize t = min triangle area
subject to a fixed convex-hull area of 1), keeping the best
true-scored configuration found.
"""

def true_score(pts):
    area, hull_idx = hull_area_and_indices(pts)   # scipy ConvexHull
    tri = triangle_areas(pts)                     # all 286 of them
    return float(tri.min() / area)

def polish(pts0, maxiter=150):
    # pin every triangle's orientation, then push up the smallest
    # signed area as an epigraph variable t
    pts_g, hull_idx = gauge_normalize(pts0)       # hull area -> 1
    signs = np.sign(triangle_crosses(pts_g))

    def tri_cons(x):
        pts, t = unpack(x)
        return signs * triangle_crosses(pts) * 0.5 - t   # area_i >= t

    res = minimize(neg_t, x_init, jac=neg_t_grad, method="SLSQP",
                   constraints=[{"type": "eq", "fun": hull_eq},
                                {"type": "ineq", "fun": tri_cons}],
                   options={"maxiter": maxiter, "ftol": 1e-14})
    return unpack(res.x)[0]

def main():
    best_pts = polish_iterated(REF_PTS, rounds=3)  # known-good seed
    best_score = true_score(best_pts)

    # phase 1: broad exploration across hull/interior topologies
    for kind, hull_size in starters:               # hull sizes 8..13
        if time.time() > phase1_deadline:
            break
        pts = polish_iterated(make_start(kind, hull_size), rounds=2)
        if pts is not None and true_score(pts) > best_score:
            best_pts, best_score = pts, true_score(pts)

    # phase 2: basin-hopping local search around the current best
    while time.time() - start < TIME_BUDGET_S:
        pts = polish_iterated(perturb(best_pts), rounds=2)
        if pts is not None and true_score(pts) > best_score:
            best_pts, best_score = pts, true_score(pts)

    # final tight refinement of the winner, then write it out
    best_pts = polish_iterated(best_pts, rounds=4, maxiter=400)
    write_csv("submission.csv", best_pts)

Its submission.csv, drawn: 13 points, ten on the hull and three interior. The shaded triangle is the smallest of the 286; its area over the hull's is 0.0309372, where AlphaEvolve reported 0.0309369.

05 — See what it learns

Every search writes what it learned into a graph: claims, techniques, libraries, and the problems it ran on. Later searches read it back, so the next one does not start from nothing. Below is an example of a hillclimbrun solving the Heilbronn problem. Slide back through time and each search is a tick where new claims appear. Drag to rotate, shift-drag to pan, scroll to zoom, click a type to hide it, slide to scrub time.