Parallelism

A run uses your machine at four levels at once, and every level is a count of how much runs side by side. Three are hillclimb's. The fourth is inside the solution, and it is the one most easily forgotten: a solution.py that starts its own process pool runs on top of the other three.

A run can keep up to N × M × P × C cores busy (with P capped at the trial's replicate count). hillclimb run warns when that is more than the machine has.

hillclimb run heilbronn-convex-13 \
  --parallel-searches 2 \      # two search trees, each its own engine
  --parallel-agents 3 \        # three coding agents per search
  --parallel-replicates 0 \    # a trial's replicates all at once (the default)
  --solution-cpus 1            # one core per solution run (the default)

How many replicates a trial gets is not on this page. That is the problem's evaluation.n_replicates (R, in its problem.yaml), the sample size the trial's median is taken over — an evaluation setting, one it keeps whether the runs overlap or not. The parallelism knob is how many of those R execute at once.

Searches and coding agents

--parallel-searches starts independent searches on the same problem, each in its own engine process, all under one run. They share what they learn through memory as they go.

--parallel-agents is how many attempts one search has in flight: each is a coding agent writing a candidate, then a verifier run scoring it. Tune jobs, which re-run a candidate's own code with other params.json values, share these slots. A coding agent spends most of its time waiting on its model, so this level is cheap on the CPU until the verifier runs.

Across every search and every folder, the machine runs at most concurrency.machine_max_agents coding agents at once; the default is the number of cores minus two, at most eight. See Machine resources.

Replicates

With evaluation.n_replicates: R above 1 in the problem's problem.yaml, every trial runs R times with different seeds and takes the median, so seed variance is measured rather than climbed. --parallel-replicates P is how many of those R run at once: 0 (the default) means all of them, 1 means one after another, 2 means batches of two.

For a metric that measures the machine itself — time, throughput, memory — they must not overlap, since concurrent runs would measure each other:

evaluation:
  n_replicates: 3

concurrency:
  parallel_replicates: 1

Each replicate is one solution run, so it gets solution_cpus cores like any other; there is no core budget for "the replicates" as a group. A trial with P replicates in flight uses P × C cores while they overlap.

Solutions

One run of the solution is one run of the verifier: it starts solution.py and scores what it wrote. Each such run is given --solution-cpus cores, and three things follow from it:

• The run's environment carries HILLCLIMB_CPUS=C, and the math libraries' thread pools are capped to match (OMP_NUM_THREADS, OPENBLAS_NUM_THREADS, MKL_NUM_THREADS, …). The holdout run and the coding agent's own test runs of the solution get the same.
• The contract every coding agent reads says so: size any process or thread pool from HILLCLIMB_CPUS, never from os.cpu_count().
• Nothing stops a solution from starting more processes than that. A run whose CPU time outpaces its wall time by more than 1.5 × its allotment is marked in hillclimb watch beside the candidate — cpu 6.9/1 means seven cores busy on a one-core allotment — and the journal records each run's allotment as cpus.

Why it matters: a solution that searches until a deadline (while time.time() < t_end) finds less on a busy machine in the same seconds. Stack its own eight-process pool on three parallel coding agents and its score measures the load, not the code: the same solution.py scores differently from one evaluation to the next, and differently again on another machine.

With a problem that benefits from a wider solution, give it the cores and lower the parallelism to match:

hillclimb run my-problem --parallel-agents 2 --solution-cpus 4    # 2 × 4 = 8 cores