Stepping
climber.start opens a search and lets you run it one action at a time.
run gives the whole search to the climber. climber.start opens the same search but leaves
the loop with you. You can ask the policy what it would do next, run that or something else, look
at what the policy is looking at, and hand the rest back to the climber when you have seen enough.
from hillclimb import Action, Budget, Climber, Problem
from hillclimb.policies import Greedy
from hillclimb.selectors import Best
problem = Problem("fitness-landscape")
budget = Budget(evaluations=30)
climber = Climber(selector_policy=Best(num_drafts=3), operator_policy=Greedy())
climber.start(problem, budget=budget, learning=False)
climber.select() # the selector's pick: which node, or None for a root step
action = climber.propose() # the selector's pick, then the policy's operator on it. Nothing runs yet.
outcome = climber.run(action) # run it. The attempt is made, scored, committed, and the policy sees it.
outcome = climber.step() # ask and run in one call
climber.run(Action("improve", target_id="c002")) # your own move
climber.state # what the policy sees: the journal, what is in flight, the budget left
climber.candidates # every candidate so far
climber.best # the best scored one
climber.finish() # the climber's own loop runs the rest of the budget
result = climber.close() # settle the search. You get the same SearchOutcome run returns.start takes the same arguments as search. The search is a normal one, so you
can keep hillclimb watch open in another terminal while you step, and hillclimb status lists
its candidates.
The calls
| call | what it does |
|---|---|
climber.start(problem, **options) | open a search and write its floor (the baseline, and a seed if you gave one). Returns the Search underneath |
climber.select() | the selector's answer alone: a Selection naming the node(s) the next attempt would start from, or None for a root step |
climber.propose() | the whole decision as an Action: the selector's pick, then the policy's operator on it. None if the policy wants to wait or the search is closed |
climber.run(action) | run one action to the end and let the policy see the result. Returns an Outcome |
climber.step() | run(propose()). None if there was nothing to run |
climber.state | the SearchState the policy decides on. Its journal (with holdout scores hidden), what is in flight, the budget left, and which direction is better |
climber.candidates, climber.best | the candidates so far, as the policy sees them |
climber.finish() | hand the rest to the climber's own loop. Returns the SearchOutcome, and the result reads off the climber as after search |
climber.close() | settle the search where it is. Returns the SearchOutcome, and keeps returning it if you call it again |
climber.session | the open Search itself, with .ref, .is_open, .closed_reason, .source(candidate_id), .result (a live reading), .harness and .loop |
An Action names an operator and, for everything except draft, the candidate to start from.
Some examples: Action("improve", target_id="c002"), Action("ensemble", target_id="c002", inspiration_ids=("c004", "c007")), Action("tune", target_id="c002"). Action comes from hillclimb
or hillclimb.sdk.
An Outcome has a .kind (evaluated, tuned, rejected, agent_failed, crashed and a few
more), a .candidate (the one it made or changed, with holdout scores hidden) and
.ticket.action.
Good to know
The clock only runs during a step. The time you spend reading an outcome is yours. The
other limits of the Budget count as usual.
Bad actions do not raise. If you name a candidate that does not exist, an operator the climber
does not have, or the budget is already spent, run returns a rejected outcome and
ticket.rejected tells you why. A policy that keeps proposing impossible things ends its search
after three tries. A person trying things out does not.
How it ends. When a budget closes the search it is done, and gets everything a finished
search gets: holdout scoring, best/, the knowledge it leaves behind. A stop from outside
(hillclimb stop, or SIGTERM) shows up at your next propose as closed_reason, and close()
then makes the search stopped. A park, for example from a cost ceiling, makes it parked. If
you close it yourself with budget left, it is stopped. You can continue it later with
hillclimb resume, and it leaves no knowledge behind.
One at a time. Starting a second search while one is open in the same process is refused. Close the first one. (The status record carries the process id, so two searches in one process would look like one engine to every viewer.)
Policy climbers only. A loop: climber such as gepa runs its own loop. propose, run and
step raise for it, and finish() runs it.
What is underneath
climber.start calls hillclimb.api.start(problem, climber=climber, ...), which returns a
hillclimb.api.Search. That object holds everything the engine sets up around a harness: the
store, the journal, the status record, the agents, the evaluator and the climber's loop. It can be
stepped, and it is settled exactly once. The climber only forwards your calls to it, so starting
a search changes nothing about the climber. Its block and its identity stay as they were.