Running Discovery

Learn how to run SkyDiscover to evolve solutions using the CLI and Python API

Quick Start

Using the CLI

The simplest way to run SkyDiscover is with the skydiscover-run command:

skydiscover-run initial_program.py evaluator.py -c config.yaml -s adaevolve -i 100

Required arguments:

• initial_program.py - Starting solution (optional for from-scratch generation)
• evaluator.py - Scoring function that returns metrics

Common flags:

• -c, --config - Path to YAML configuration file
• -s, --search - Search algorithm (topk, adaevolve, beam_search, evox, gepa, openevolve)
• -i, --iterations - Maximum number of iterations (default: 100)
• -m, --model - Model name(s), comma-separated (e.g., gpt-5, gemini/gemini-3-pro)
• -o, --output - Output directory for results
• --agentic - Enable agentic mode for codebase-aware generation
• --api-base - Custom API endpoint URL
• -l, --log-level - Logging level (DEBUG, INFO, WARNING, ERROR)

Using the Python API

Basic Usage

from skydiscover import run_discovery

result = run_discovery(
    evaluator="examples/my_problem/eval.py",
    initial_program="examples/my_problem/init.py",
    model="gpt-5",
    iterations=50,
)

print(f"Best score: {result.best_score}")
print(f"Solution:\n{result.best_solution}")

from skydiscover import run_discovery
from skydiscover.config import Config

# Load custom config
config = Config.from_yaml("config.yaml")

result = run_discovery(
    evaluator="eval.py",
    initial_program="init.py",
    config=config,
    search="adaevolve",
    iterations=100,
    output_dir="./outputs/run_001",
    agentic=True,
    system_prompt="You are an expert in computational geometry.",
)

print(f"Improvement: {result.best_score / result.initial_score:.2%}")

from skydiscover import discover_solution

def my_evaluator(solution: str) -> dict:
    # Score the solution
    score = evaluate_somehow(solution)
    return {"combined_score": score, "accuracy": 0.95}

result = discover_solution(
    evaluator=my_evaluator,
    initial_solution="def solve(x): return x",
    iterations=50,
    model="gpt-5",
)

Search Algorithms

topk

Simple top-k selection. Good baseline for quick experiments.

-s topk

adaevolve

Adaptive multi-island evolution with dynamic search intensity.

-s adaevolve

beam_search

Beam search with diversity-weighted selection.

-s beam_search

evox

Co-evolution of solutions and search strategies.

-s evox

gepa

Guided evolution with acceptance gating and merging.

-s gepa

openevolve

MAP-Elites with island-based exploration (external package).

pip install openevolve
-s openevolve

CLI Examples

Basic Run

# Run with default config
skydiscover-run benchmarks/math/circle_packing/initial_program.py \
  benchmarks/math/circle_packing/evaluator.py

Specify Model and Search Algorithm

# Use GPT-5 with AdaEvolve search
skydiscover-run init.py eval.py \
  -m gpt-5 \
  -s adaevolve \
  -i 200

Multiple Models

# Use multiple models with automatic sampling
skydiscover-run init.py eval.py \
  -m "gpt-5,gemini/gemini-3-pro,claude-3-7-sonnet" \
  -i 100

Custom API Endpoint

# Use local vLLM server
skydiscover-run init.py eval.py \
  --api-base http://localhost:8000/v1 \
  -m qwen/Qwen2.5-Coder-32B-Instruct

Agentic Mode

# Enable codebase-aware generation
skydiscover-run benchmarks/math/circle_packing/initial_program.py \
  benchmarks/math/circle_packing/evaluator.py \
  --agentic \
  -m gpt-5

Resume from Checkpoint

# Resume from saved checkpoint
skydiscover-run init.py eval.py \
  --checkpoint outputs/adaevolve/problem_0305_1430/checkpoints/checkpoint_50

Understanding Results

Output Directory Structure

outputs/adaevolve/circle_packing_0305_1430/
├── best_program.py          # Best solution found
├── database.db             # SQLite database of all programs
├── logs/                   # Detailed run logs
├── checkpoints/            # Periodic checkpoints
│   ├── checkpoint_10/
│   ├── checkpoint_20/
│   └── checkpoint_50/
└── programs/               # All generated programs
    ├── program_0001.py
    ├── program_0002.py
    └── ...

DiscoveryResult Fields

best_program · Program

Full Program object with code, metrics, and lineage

best_score · float

The combined_score from the best program’s metrics

best_solution · str

Source code of the best solution

metrics · dict

All metrics returned by the evaluator for the best program

initial_score · float

Score of the initial program (if provided)

output_dir · str

Path to the output directory (None if cleanup=True)

Environment Variables

OPENAI_API_KEY · string · required

OpenAI API key (or set per-model in config)

GEMINI_API_KEY · string

Google Gemini API key (alias: GOOGLE_API_KEY)

ANTHROPIC_API_KEY · string

Anthropic Claude API key

DEEPSEEK_API_KEY · string

DeepSeek API key

OPENAI_API_BASE · string

Override the default API base URL (alias: OPENAI_BASE_URL)

From-Scratch Generation

You can omit the initial program to have the LLM generate solutions from scratch:

# CLI: pass evaluator only
skydiscover-run evaluator.py -m gpt-5 -i 50

# API: omit initial_program
result = run_discovery(
    evaluator="evaluator.py",
    initial_program=None,  # Generate from scratch
    model="gpt-5",
    iterations=50,
)

For from-scratch generation, provide a detailed system_message in your config describing the problem, input/output format, and constraints.

Performance Tips

Parallel Iterations

Speed up discovery by running multiple iterations concurrently:

max_parallel_iterations: 4  # Run 4 iterations in parallel

Works best with fast evaluators and sufficient API rate limits.

Cascade Evaluation

Exit early on low-scoring programs:

evaluator:
  cascade_evaluation: true
  cascade_thresholds: [0.3, 0.6]  # Quick check, then full eval

Implement evaluate_stage1() and evaluate_stage2() in your evaluator.

Checkpoint Regularly

Save progress and resume if interrupted:

checkpoint_interval: 10  # Save every 10 iterations

Next Steps

Writing Evaluators

Learn how to write effective scoring functions

Configuration

Deep dive into YAML configuration options

Model Providers

Set up OpenAI, Gemini, Anthropic, and local models

Monitoring

Watch your discovery run in real-time