Similarity — hillclimb

Similarity scores: pluggable "how alike are these two solutions?" measures.

A score is a SimilarityScore subclass (base.py holds the contract: represent one solution, compare two representations). Scores are named like policies are:

a registry name: solution-card (LLM method card → embedding → cosine), api-calls (imports + library calls, rename-invariant), code-tokens (token-set Jaccard, the similarity map's structural distance);
a Python file, any name ending in .py (relative to the folder holding the hillclimb dir, like search.policy): the file sets SIMILARITY_SCORE = <class> or defines exactly one SimilarityScore subclass;
package.module:ClassName for scores shipped in an installed package;
or register_score(cls) from code that embeds hillclimb.

similarity.scores in config.yaml maps the names to their params; the hillclimb similarity scores command computes the pairwise matrices.

src/hillclimb/modules/similarity/base.py

class SimilarityScore:
    """Subclass, set `name`, implement `represent` (and `compare` when the
    representation is not a vector, dict or set)."""

    name: ClassVar[str] = ""
    description: ClassVar[str] = ""
    version: ClassVar[str] = "1"
    cache: ClassVar[bool] = False
    defaults: ClassVar[Mapping[str, Any]] = {}

    def represent(self, solution: Solution) -> Any:
        raise NotImplementedError(f"{type(self).__name__} must implement represent(solution)")

    def compare(self, a: Any, b: Any) -> float:
        return default_compare(a, b)

    def cache_key(self, solution: Solution) -> str | None:
        """What identifies a solution's representation besides the score
        and its params; None = never cache this one."""
        ...

    def explain(self, solution: Solution) -> str | None:
        """Optional human-readable view of a representation (e.g. the card
        an LLM wrote); None when the score has nothing to show."""
        return None

hillclimb similarity

Live 3D similarity views of a search's candidates: `map` (pairwise, default) and `reference`.