claude-engineering-plugin/plugins/compounding-engineering/skills/dspy-ruby/references/providers.md

DSPy.rb LLM Providers

Supported Providers

DSPy.rb provides unified support across multiple LLM providers through adapter gems that automatically load when installed.

Provider Overview

• OpenAI: GPT-4, GPT-4o, GPT-4o-mini, GPT-3.5-turbo
• Anthropic: Claude 3 family (Sonnet, Opus, Haiku), Claude 3.5 Sonnet
• Google Gemini: Gemini 1.5 Pro, Gemini 1.5 Flash, other versions
• Ollama: Local model support via OpenAI compatibility layer
• OpenRouter: Unified multi-provider API for 200+ models

Configuration

Basic Setup

require 'dspy'

DSPy.configure do |c|
  c.lm = DSPy::LM.new('provider/model-name', api_key: ENV['API_KEY'])
end

OpenAI Configuration

Required gem: dspy-openai

DSPy.configure do |c|
  # GPT-4o Mini (recommended for development)
  c.lm = DSPy::LM.new('openai/gpt-4o-mini', api_key: ENV['OPENAI_API_KEY'])

  # GPT-4o (more capable)
  c.lm = DSPy::LM.new('openai/gpt-4o', api_key: ENV['OPENAI_API_KEY'])

  # GPT-4 Turbo
  c.lm = DSPy::LM.new('openai/gpt-4-turbo', api_key: ENV['OPENAI_API_KEY'])
end

Environment variable: OPENAI_API_KEY

Anthropic Configuration

Required gem: dspy-anthropic

DSPy.configure do |c|
  # Claude 3.5 Sonnet (latest, most capable)
  c.lm = DSPy::LM.new('anthropic/claude-3-5-sonnet-20241022',
    api_key: ENV['ANTHROPIC_API_KEY'])

  # Claude 3 Opus (most capable in Claude 3 family)
  c.lm = DSPy::LM.new('anthropic/claude-3-opus-20240229',
    api_key: ENV['ANTHROPIC_API_KEY'])

  # Claude 3 Sonnet (balanced)
  c.lm = DSPy::LM.new('anthropic/claude-3-sonnet-20240229',
    api_key: ENV['ANTHROPIC_API_KEY'])

  # Claude 3 Haiku (fast, cost-effective)
  c.lm = DSPy::LM.new('anthropic/claude-3-haiku-20240307',
    api_key: ENV['ANTHROPIC_API_KEY'])
end

Environment variable: ANTHROPIC_API_KEY

Google Gemini Configuration

Required gem: dspy-gemini

DSPy.configure do |c|
  # Gemini 1.5 Pro (most capable)
  c.lm = DSPy::LM.new('gemini/gemini-1.5-pro',
    api_key: ENV['GOOGLE_API_KEY'])

  # Gemini 1.5 Flash (faster, cost-effective)
  c.lm = DSPy::LM.new('gemini/gemini-1.5-flash',
    api_key: ENV['GOOGLE_API_KEY'])
end

Environment variable: GOOGLE_API_KEY or GEMINI_API_KEY

Ollama Configuration

Required gem: None (uses OpenAI compatibility layer)

DSPy.configure do |c|
  # Local Ollama instance
  c.lm = DSPy::LM.new('ollama/llama3.1',
    base_url: 'http://localhost:11434')

  # Other Ollama models
  c.lm = DSPy::LM.new('ollama/mistral')
  c.lm = DSPy::LM.new('ollama/codellama')
end

Note: Ensure Ollama is running locally: ollama serve

OpenRouter Configuration

Required gem: dspy-openai (uses OpenAI adapter)

DSPy.configure do |c|
  # Access 200+ models through OpenRouter
  c.lm = DSPy::LM.new('openrouter/anthropic/claude-3.5-sonnet',
    api_key: ENV['OPENROUTER_API_KEY'],
    base_url: 'https://openrouter.ai/api/v1')

  # Other examples
  c.lm = DSPy::LM.new('openrouter/google/gemini-pro')
  c.lm = DSPy::LM.new('openrouter/meta-llama/llama-3.1-70b-instruct')
end

Environment variable: OPENROUTER_API_KEY

Provider Compatibility Matrix

Feature Support

Feature	OpenAI	Anthropic	Gemini	Ollama
Structured Output	✅	✅	✅	✅
Vision (Images)	✅	✅	✅	⚠️ Limited
Image URLs	✅	❌	❌	❌
Tool Calling	✅	✅	✅	Varies
Streaming	❌	❌	❌	❌
Function Calling	✅	✅	✅	Varies

Legend: ✅ Full support | ⚠️ Partial support | ❌ Not supported

Vision Capabilities

Image URLs: Only OpenAI supports direct URL references. For other providers, load images as base64 or from files.

# OpenAI - supports URLs
DSPy::Image.from_url("https://example.com/image.jpg")

# Anthropic, Gemini - use file or base64
DSPy::Image.from_file("path/to/image.jpg")
DSPy::Image.from_base64(base64_data, mime_type: "image/jpeg")

Ollama: Limited multimodal functionality. Check specific model capabilities.

Advanced Configuration

Custom Parameters

Pass provider-specific parameters during configuration:

DSPy.configure do |c|
  c.lm = DSPy::LM.new('openai/gpt-4o',
    api_key: ENV['OPENAI_API_KEY'],
    temperature: 0.7,
    max_tokens: 2000,
    top_p: 0.9
  )
end

Multiple Providers

Use different models for different tasks:

# Fast model for simple tasks
fast_lm = DSPy::LM.new('openai/gpt-4o-mini', api_key: ENV['OPENAI_API_KEY'])

# Powerful model for complex tasks
powerful_lm = DSPy::LM.new('anthropic/claude-3-5-sonnet-20241022',
  api_key: ENV['ANTHROPIC_API_KEY'])

# Use different models in different modules
class SimpleClassifier < DSPy::Module
  def initialize
    super
    DSPy.configure { |c| c.lm = fast_lm }
    @predictor = DSPy::Predict.new(SimpleSignature)
  end
end

class ComplexAnalyzer < DSPy::Module
  def initialize
    super
    DSPy.configure { |c| c.lm = powerful_lm }
    @predictor = DSPy::ChainOfThought.new(ComplexSignature)
  end
end

Per-Request Configuration

Override configuration for specific predictions:

predictor = DSPy::Predict.new(MySignature)

# Use default configuration
result1 = predictor.forward(input: "data")

# Override temperature for this request
result2 = predictor.forward(
  input: "data",
  config: { temperature: 0.2 }  # More deterministic
)

Cost Optimization

Model Selection Strategy

1. Development: Use cheaper, faster models (gpt-4o-mini, claude-3-haiku, gemini-1.5-flash)
2. Production Simple Tasks: Continue with cheaper models if quality is sufficient
3. Production Complex Tasks: Upgrade to more capable models (gpt-4o, claude-3.5-sonnet, gemini-1.5-pro)
4. Local Development: Use Ollama for privacy and zero API costs

Example Cost-Conscious Setup

# Development environment
if Rails.env.development?
  DSPy.configure do |c|
    c.lm = DSPy::LM.new('ollama/llama3.1')  # Free, local
  end
elsif Rails.env.test?
  DSPy.configure do |c|
    c.lm = DSPy::LM.new('openai/gpt-4o-mini',  # Cheap for testing
      api_key: ENV['OPENAI_API_KEY'])
  end
else  # production
  DSPy.configure do |c|
    c.lm = DSPy::LM.new('anthropic/claude-3-5-sonnet-20241022',
      api_key: ENV['ANTHROPIC_API_KEY'])
  end
end

Provider-Specific Best Practices

OpenAI

• Use gpt-4o-mini for development and simple tasks
• Use gpt-4o for production complex tasks
• Best vision support including URL loading
• Excellent function calling capabilities

Anthropic

• Claude 3.5 Sonnet is currently the most capable model
• Excellent for complex reasoning and analysis
• Strong safety features and helpful outputs
• Requires base64 for images (no URL support)

Google Gemini

• Gemini 1.5 Pro for complex tasks, Flash for speed
• Strong multimodal capabilities
• Good balance of cost and performance
• Requires base64 for images

Ollama

• Best for privacy-sensitive applications
• Zero API costs
• Requires local hardware resources
• Limited multimodal support depending on model
• Good for development and testing

Troubleshooting

API Key Issues

# Verify API key is set
if ENV['OPENAI_API_KEY'].nil?
  raise "OPENAI_API_KEY environment variable not set"
end

# Test connection
begin
  DSPy.configure { |c| c.lm = DSPy::LM.new('openai/gpt-4o-mini',
    api_key: ENV['OPENAI_API_KEY']) }
  predictor = DSPy::Predict.new(TestSignature)
  predictor.forward(test: "data")
  puts "✅ Connection successful"
rescue => e
  puts "❌ Connection failed: #{e.message}"
end

Rate Limiting

Handle rate limits gracefully:

def call_with_retry(predictor, input, max_retries: 3)
  retries = 0
  begin
    predictor.forward(input)
  rescue RateLimitError => e
    retries += 1
    if retries < max_retries
      sleep(2 ** retries)  # Exponential backoff
      retry
    else
      raise
    end
  end
end

Model Not Found

Ensure the correct gem is installed:

# For OpenAI
gem install dspy-openai

# For Anthropic
gem install dspy-anthropic

# For Gemini
gem install dspy-gemini