LLM Configuration

Configure language models, API settings, and generation parameters for SkyDiscover

Overview

The llm section controls all language model settings including API endpoints, model selection, generation parameters, and retry behavior.

Basic Configuration

llm:
  models:
    - name: "gpt-5"
      weight: 1.0
  api_base: "https://api.openai.com/v1"
  temperature: 0.7
  top_p: 0.95
  max_tokens: 32000
  timeout: 600

Model Configuration

Single Model

Configure a single model for solution generation:

llm:
  models:
    - name: "gpt-5"
      weight: 1.0

Multiple Models

Use multiple models with weighted sampling:

llm:
  models:
    - name: "gpt-5"
      weight: 0.7
    - name: "claude-opus-3"
      weight: 0.3

SkyDiscover randomly samples models based on their weights. In this example, gpt-5 is selected 70% of the time.

Provider-Specific Models

Use the provider/model format to specify different providers:

llm:
  models:
    - name: "openai/gpt-5"
      weight: 0.5
    - name: "anthropic/claude-opus-3"
      weight: 0.3
    - name: "gemini/gemini-3-pro"
      weight: 0.2

Supported Providers

SkyDiscover supports multiple LLM providers with automatic API base URL and key resolution:

OpenAI

llm:
  models:
    - name: "gpt-5"
    - name: "gpt-5-mini"
    - name: "o1"
  api_base: "https://api.openai.com/v1"

Environment variables: OPENAI_API_KEY

Automatically detected for models starting with: gpt-, o1, o3, o4

Anthropic (Claude)

llm:
  models:
    - name: "anthropic/claude-opus-3"
    - name: "anthropic/claude-sonnet-5"
  # api_base auto-configured

Environment variables: ANTHROPIC_API_KEY

Automatically detected for models starting with: claude-

Google Gemini

llm:
  models:
    - name: "gemini/gemini-3-pro"
  api_base: "https://generativelanguage.googleapis.com/v1beta/openai/"

Environment variables: GEMINI_API_KEY, GOOGLE_API_KEY

Automatically detected for models starting with: gemini-

DeepSeek

llm:
  models:
    - name: "deepseek/deepseek-chat"
  api_base: "https://api.deepseek.com/v1"

Environment variables: DEEPSEEK_API_KEY

Automatically detected for models starting with: deepseek-

Mistral

llm:
  models:
    - name: "mistral/mistral-large"
  api_base: "https://api.mistral.ai/v1"

Environment variables: MISTRAL_API_KEY

Automatically detected for models starting with: mistral-

Ollama (Local)

llm:
  models:
    - name: "ollama/llama3"
      api_base: "http://localhost:11434/v1"
  # No API key required

No environment variables required. Specify api_base for your local Ollama instance.

vLLM (Local)

llm:
  models:
    - name: "vllm/custom-model"
      api_base: "http://localhost:8000/v1"
  # No API key required

No environment variables required. Specify api_base for your vLLM server.

LLMConfig Parameters

Defined in skydiscover/config.py:151-224

API Settings

api_base · str · default: "https://api.openai.com/v1"

Base URL for the LLM API endpoint

api_key · str · default: "None"

API key for authentication. Auto-resolved from environment if not specified

Model Selection

models · List[LLMModelConfig] · default: "[]"

List of models for solution generation. Each model can have individual settings

evaluator_models · List[LLMModelConfig] · default: "[]"

Models for evaluation. Defaults to same as models if not specified

guide_models · List[LLMModelConfig] · default: "[]"

Models for guide tasks (idea generation, paradigm breakthroughs). Defaults to models

Generation Parameters

temperature · float · default: "0.7"

Sampling temperature (0.0-2.0). Higher values increase randomness

top_p · float · default: "0.95"

Nucleus sampling threshold (0.0-1.0). Alternative to temperature

max_tokens · int · default: "32000"

Maximum tokens to generate in a single completion

system_message · str · default: "system_message"

System message for the model. Usually set in prompt section instead

Request Parameters

timeout · int · default: "600"

Request timeout in seconds

retries · int · default: "3"

Number of retry attempts on failure

retry_delay · int · default: "5"

Delay in seconds between retries

Reasoning Parameters

reasoning_effort · str · default: "None"

Reasoning effort level for o1-series models: low, medium, or high

LLMModelConfig Parameters

Per-model configuration (defined in skydiscover/config.py:121-148):

name · str · default: "None"

Model identifier (e.g., gpt-5, anthropic/claude-opus-3)

weight · float · default: "1.0"

Sampling weight for this model in the pool

api_base · str · default: "None"

Override API base URL for this specific model

api_key · str · default: "None"

Override API key for this specific model

init_client · Callable · default: "None"

Custom LLM client initialization function

All generation and request parameters can also be overridden per-model.

Advanced Examples

LLM-as-a-Judge

Use separate models for generation and evaluation:

configs/llm_judge.yaml

llm:
  # Fast model for generation
  models:
    - name: "gpt-4o-mini"
      weight: 1.0
  
  # Powerful model for evaluation
  evaluator_models:
    - name: "gpt-5"
      weight: 1.0
  
  temperature: 0.7
  max_tokens: 16000

Mixed Provider Setup

llm:
  models:
    - name: "gpt-5"
      weight: 0.5
      api_base: "https://api.openai.com/v1"
      api_key: ${OPENAI_API_KEY}
    
    - name: "claude-opus-3"
      weight: 0.3
      api_base: "https://api.anthropic.com/v1"
      api_key: ${ANTHROPIC_API_KEY}
    
    - name: "llama3"
      weight: 0.2
      api_base: "http://localhost:11434/v1"
      temperature: 0.8  # Per-model override
  
  temperature: 0.7
  max_tokens: 32000

Reasoning Models (o1-series)

llm:
  models:
    - name: "o1"
      weight: 1.0
  reasoning_effort: "high"  # low, medium, or high
  max_tokens: 100000

CLI Overrides

Override model settings from the command line:

# Override model
skydiscover-run program.py evaluator.py -m gpt-5-mini

# Multiple models
skydiscover-run program.py evaluator.py -m "gpt-5,claude-opus-3"

# Override API base
skydiscover-run program.py evaluator.py --api-base http://localhost:8000/v1

# Both
skydiscover-run program.py evaluator.py \
  -m gemini-3-pro \
  --api-base https://generativelanguage.googleapis.com/v1beta/openai/

See skydiscover/config.py:819-910 for override implementation.

Environment Variables

Provider-specific API keys are automatically resolved:

# OpenAI
export OPENAI_API_KEY="sk-..."

# Anthropic
export ANTHROPIC_API_KEY="sk-ant-..."

# Google Gemini
export GEMINI_API_KEY="..."
export GOOGLE_API_KEY="..."  # Alternative

# DeepSeek
export DEEPSEEK_API_KEY="..."

# Mistral
export MISTRAL_API_KEY="..."

# Custom API base
export OPENAI_API_BASE="http://localhost:8000/v1"
export OPENAI_BASE_URL="http://localhost:8000/v1"  # Alternative

Next Steps

Prompt Configuration

Configure system messages and prompt templates

Search Configuration

Configure evolutionary search algorithms