refactor(skills): remove 5 unused skills and clean references (#634)
This commit is contained in:
Trevin Chow
@@ -1,184 +0,0 @@
|
||||
---
|
||||
name: ce-andrew-kane-gem-writer
|
||||
description: This skill should be used when writing Ruby gems following Andrew Kane's proven patterns and philosophy. It applies when creating new Ruby gems, refactoring existing gems, designing gem APIs, or when clean, minimal, production-ready Ruby library code is needed. Triggers on requests like "create a gem", "write a Ruby library", "design a gem API", or mentions of Andrew Kane's style.
|
||||
---
|
||||
|
||||
# Andrew Kane Gem Writer
|
||||
|
||||
Write Ruby gems following Andrew Kane's battle-tested patterns from 100+ gems with 374M+ downloads (Searchkick, PgHero, Chartkick, Strong Migrations, Lockbox, Ahoy, Blazer, Groupdate, Neighbor, Blind Index).
|
||||
|
||||
## Core Philosophy
|
||||
|
||||
**Simplicity over cleverness.** Zero or minimal dependencies. Explicit code over metaprogramming. Rails integration without Rails coupling. Every pattern serves production use cases.
|
||||
|
||||
## Entry Point Structure
|
||||
|
||||
Every gem follows this exact pattern in `lib/gemname.rb`:
|
||||
|
||||
```ruby
|
||||
# 1. Dependencies (stdlib preferred)
|
||||
require "forwardable"
|
||||
|
||||
# 2. Internal modules
|
||||
require_relative "gemname/model"
|
||||
require_relative "gemname/version"
|
||||
|
||||
# 3. Conditional Rails (CRITICAL - never require Rails directly)
|
||||
require_relative "gemname/railtie" if defined?(Rails)
|
||||
|
||||
# 4. Module with config and errors
|
||||
module GemName
|
||||
class Error < StandardError; end
|
||||
class InvalidConfigError < Error; end
|
||||
|
||||
class << self
|
||||
attr_accessor :timeout, :logger
|
||||
attr_writer :client
|
||||
end
|
||||
|
||||
self.timeout = 10 # Defaults set immediately
|
||||
end
|
||||
```
|
||||
|
||||
## Class Macro DSL Pattern
|
||||
|
||||
The signature Kane pattern—single method call configures everything:
|
||||
|
||||
```ruby
|
||||
# Usage
|
||||
class Product < ApplicationRecord
|
||||
searchkick word_start: [:name]
|
||||
end
|
||||
|
||||
# Implementation
|
||||
module GemName
|
||||
module Model
|
||||
def gemname(**options)
|
||||
unknown = options.keys - KNOWN_KEYWORDS
|
||||
raise ArgumentError, "unknown keywords: #{unknown.join(", ")}" if unknown.any?
|
||||
|
||||
mod = Module.new
|
||||
mod.module_eval do
|
||||
define_method :some_method do
|
||||
# implementation
|
||||
end unless method_defined?(:some_method)
|
||||
end
|
||||
include mod
|
||||
|
||||
class_eval do
|
||||
cattr_reader :gemname_options, instance_reader: false
|
||||
class_variable_set :@@gemname_options, options.dup
|
||||
end
|
||||
end
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
## Rails Integration
|
||||
|
||||
**Always use `ActiveSupport.on_load`—never require Rails gems directly:**
|
||||
|
||||
```ruby
|
||||
# WRONG
|
||||
require "active_record"
|
||||
ActiveRecord::Base.include(MyGem::Model)
|
||||
|
||||
# CORRECT
|
||||
ActiveSupport.on_load(:active_record) do
|
||||
extend GemName::Model
|
||||
end
|
||||
|
||||
# Use prepend for behavior modification
|
||||
ActiveSupport.on_load(:active_record) do
|
||||
ActiveRecord::Migration.prepend(GemName::Migration)
|
||||
end
|
||||
```
|
||||
|
||||
## Configuration Pattern
|
||||
|
||||
Use `class << self` with `attr_accessor`, not Configuration objects:
|
||||
|
||||
```ruby
|
||||
module GemName
|
||||
class << self
|
||||
attr_accessor :timeout, :logger
|
||||
attr_writer :master_key
|
||||
end
|
||||
|
||||
def self.master_key
|
||||
@master_key ||= ENV["GEMNAME_MASTER_KEY"]
|
||||
end
|
||||
|
||||
self.timeout = 10
|
||||
self.logger = nil
|
||||
end
|
||||
```
|
||||
|
||||
## Error Handling
|
||||
|
||||
Simple hierarchy with informative messages:
|
||||
|
||||
```ruby
|
||||
module GemName
|
||||
class Error < StandardError; end
|
||||
class ConfigError < Error; end
|
||||
class ValidationError < Error; end
|
||||
end
|
||||
|
||||
# Validate early with ArgumentError
|
||||
def initialize(key:)
|
||||
raise ArgumentError, "Key must be 32 bytes" unless key&.bytesize == 32
|
||||
end
|
||||
```
|
||||
|
||||
## Testing (Minitest Only)
|
||||
|
||||
```ruby
|
||||
# test/test_helper.rb
|
||||
require "bundler/setup"
|
||||
Bundler.require(:default)
|
||||
require "minitest/autorun"
|
||||
require "minitest/pride"
|
||||
|
||||
# test/model_test.rb
|
||||
class ModelTest < Minitest::Test
|
||||
def test_basic_functionality
|
||||
assert_equal expected, actual
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
## Gemspec Pattern
|
||||
|
||||
Zero runtime dependencies when possible:
|
||||
|
||||
```ruby
|
||||
Gem::Specification.new do |spec|
|
||||
spec.name = "gemname"
|
||||
spec.version = GemName::VERSION
|
||||
spec.required_ruby_version = ">= 3.1"
|
||||
spec.files = Dir["*.{md,txt}", "{lib}/**/*"]
|
||||
spec.require_path = "lib"
|
||||
# NO add_dependency lines - dev deps go in Gemfile
|
||||
end
|
||||
```
|
||||
|
||||
## Anti-Patterns to Avoid
|
||||
|
||||
- `method_missing` (use `define_method` instead)
|
||||
- Configuration objects (use class accessors)
|
||||
- `@@class_variables` (use `class << self`)
|
||||
- Requiring Rails gems directly
|
||||
- Many runtime dependencies
|
||||
- Committing Gemfile.lock in gems
|
||||
- RSpec (use Minitest)
|
||||
- Heavy DSLs (prefer explicit Ruby)
|
||||
|
||||
## Reference Files
|
||||
|
||||
For deeper patterns, see:
|
||||
- `references/module-organization.md` - Directory layouts, method decomposition
|
||||
- `references/rails-integration.md` - Railtie, Engine, on_load patterns
|
||||
- `references/database-adapters.md` - Multi-database support patterns
|
||||
- `references/testing-patterns.md` - Multi-version testing, CI setup
|
||||
- `references/resources.md` - Links to Kane's repos and articles
|
||||
@@ -1,231 +0,0 @@
|
||||
# Database Adapter Patterns
|
||||
|
||||
## Abstract Base Class Pattern
|
||||
|
||||
```ruby
|
||||
# lib/strong_migrations/adapters/abstract_adapter.rb
|
||||
module StrongMigrations
|
||||
module Adapters
|
||||
class AbstractAdapter
|
||||
def initialize(checker)
|
||||
@checker = checker
|
||||
end
|
||||
|
||||
def min_version
|
||||
nil
|
||||
end
|
||||
|
||||
def set_statement_timeout(timeout)
|
||||
# no-op by default
|
||||
end
|
||||
|
||||
def check_lock_timeout
|
||||
# no-op by default
|
||||
end
|
||||
|
||||
private
|
||||
|
||||
def connection
|
||||
@checker.send(:connection)
|
||||
end
|
||||
|
||||
def quote(value)
|
||||
connection.quote(value)
|
||||
end
|
||||
end
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
## PostgreSQL Adapter
|
||||
|
||||
```ruby
|
||||
# lib/strong_migrations/adapters/postgresql_adapter.rb
|
||||
module StrongMigrations
|
||||
module Adapters
|
||||
class PostgreSQLAdapter < AbstractAdapter
|
||||
def min_version
|
||||
"12"
|
||||
end
|
||||
|
||||
def set_statement_timeout(timeout)
|
||||
select_all("SET statement_timeout = #{timeout.to_i * 1000}")
|
||||
end
|
||||
|
||||
def set_lock_timeout(timeout)
|
||||
select_all("SET lock_timeout = #{timeout.to_i * 1000}")
|
||||
end
|
||||
|
||||
def check_lock_timeout
|
||||
lock_timeout = connection.select_value("SHOW lock_timeout")
|
||||
lock_timeout_sec = timeout_to_sec(lock_timeout)
|
||||
# validation logic
|
||||
end
|
||||
|
||||
private
|
||||
|
||||
def select_all(sql)
|
||||
connection.select_all(sql)
|
||||
end
|
||||
|
||||
def timeout_to_sec(timeout)
|
||||
units = {"us" => 1e-6, "ms" => 1e-3, "s" => 1, "min" => 60}
|
||||
timeout.to_f * (units[timeout.gsub(/\d+/, "")] || 1e-3)
|
||||
end
|
||||
end
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
## MySQL Adapter
|
||||
|
||||
```ruby
|
||||
# lib/strong_migrations/adapters/mysql_adapter.rb
|
||||
module StrongMigrations
|
||||
module Adapters
|
||||
class MySQLAdapter < AbstractAdapter
|
||||
def min_version
|
||||
"8.0"
|
||||
end
|
||||
|
||||
def set_statement_timeout(timeout)
|
||||
select_all("SET max_execution_time = #{timeout.to_i * 1000}")
|
||||
end
|
||||
|
||||
def check_lock_timeout
|
||||
lock_timeout = connection.select_value("SELECT @@lock_wait_timeout")
|
||||
# validation logic
|
||||
end
|
||||
end
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
## MariaDB Adapter (MySQL variant)
|
||||
|
||||
```ruby
|
||||
# lib/strong_migrations/adapters/mariadb_adapter.rb
|
||||
module StrongMigrations
|
||||
module Adapters
|
||||
class MariaDBAdapter < MySQLAdapter
|
||||
def min_version
|
||||
"10.5"
|
||||
end
|
||||
|
||||
# Override MySQL-specific behavior
|
||||
def set_statement_timeout(timeout)
|
||||
select_all("SET max_statement_time = #{timeout.to_i}")
|
||||
end
|
||||
end
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
## Adapter Detection Pattern
|
||||
|
||||
Use regex matching on adapter name:
|
||||
|
||||
```ruby
|
||||
def adapter
|
||||
@adapter ||= case connection.adapter_name
|
||||
when /postg/i
|
||||
Adapters::PostgreSQLAdapter.new(self)
|
||||
when /mysql|trilogy/i
|
||||
if connection.try(:mariadb?)
|
||||
Adapters::MariaDBAdapter.new(self)
|
||||
else
|
||||
Adapters::MySQLAdapter.new(self)
|
||||
end
|
||||
when /sqlite/i
|
||||
Adapters::SQLiteAdapter.new(self)
|
||||
else
|
||||
Adapters::AbstractAdapter.new(self)
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
## Multi-Database Support (PgHero pattern)
|
||||
|
||||
```ruby
|
||||
module PgHero
|
||||
class << self
|
||||
attr_accessor :databases
|
||||
end
|
||||
|
||||
self.databases = {}
|
||||
|
||||
def self.primary_database
|
||||
databases.values.first
|
||||
end
|
||||
|
||||
def self.capture_query_stats(database: nil)
|
||||
db = database ? databases[database] : primary_database
|
||||
db.capture_query_stats
|
||||
end
|
||||
|
||||
class Database
|
||||
attr_reader :id, :config
|
||||
|
||||
def initialize(id, config)
|
||||
@id = id
|
||||
@config = config
|
||||
end
|
||||
|
||||
def connection_model
|
||||
@connection_model ||= begin
|
||||
Class.new(ActiveRecord::Base) do
|
||||
self.abstract_class = true
|
||||
end.tap do |model|
|
||||
model.establish_connection(config)
|
||||
end
|
||||
end
|
||||
end
|
||||
|
||||
def connection
|
||||
connection_model.connection
|
||||
end
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
## Connection Switching
|
||||
|
||||
```ruby
|
||||
def with_connection(database_name)
|
||||
db = databases[database_name.to_s]
|
||||
raise Error, "Unknown database: #{database_name}" unless db
|
||||
|
||||
yield db.connection
|
||||
end
|
||||
|
||||
# Usage
|
||||
PgHero.with_connection(:replica) do |conn|
|
||||
conn.execute("SELECT * FROM users")
|
||||
end
|
||||
```
|
||||
|
||||
## SQL Dialect Handling
|
||||
|
||||
```ruby
|
||||
def quote_column(column)
|
||||
case adapter_name
|
||||
when /postg/i
|
||||
%("#{column}")
|
||||
when /mysql/i
|
||||
"`#{column}`"
|
||||
else
|
||||
column
|
||||
end
|
||||
end
|
||||
|
||||
def boolean_value(value)
|
||||
case adapter_name
|
||||
when /postg/i
|
||||
value ? "true" : "false"
|
||||
when /mysql/i
|
||||
value ? "1" : "0"
|
||||
else
|
||||
value.to_s
|
||||
end
|
||||
end
|
||||
```
|
||||
@@ -1,121 +0,0 @@
|
||||
# Module Organization Patterns
|
||||
|
||||
## Simple Gem Layout
|
||||
|
||||
```
|
||||
lib/
|
||||
├── gemname.rb # Entry point, config, errors
|
||||
└── gemname/
|
||||
├── helper.rb # Core functionality
|
||||
├── engine.rb # Rails engine (if needed)
|
||||
└── version.rb # VERSION constant only
|
||||
```
|
||||
|
||||
## Complex Gem Layout (PgHero pattern)
|
||||
|
||||
```
|
||||
lib/
|
||||
├── pghero.rb
|
||||
└── pghero/
|
||||
├── database.rb # Main class
|
||||
├── engine.rb # Rails engine
|
||||
└── methods/ # Functional decomposition
|
||||
├── basic.rb
|
||||
├── connections.rb
|
||||
├── indexes.rb
|
||||
├── queries.rb
|
||||
└── replication.rb
|
||||
```
|
||||
|
||||
## Method Decomposition Pattern
|
||||
|
||||
Break large classes into includable modules by feature:
|
||||
|
||||
```ruby
|
||||
# lib/pghero/database.rb
|
||||
module PgHero
|
||||
class Database
|
||||
include Methods::Basic
|
||||
include Methods::Connections
|
||||
include Methods::Indexes
|
||||
include Methods::Queries
|
||||
end
|
||||
end
|
||||
|
||||
# lib/pghero/methods/indexes.rb
|
||||
module PgHero
|
||||
module Methods
|
||||
module Indexes
|
||||
def index_hit_rate
|
||||
# implementation
|
||||
end
|
||||
|
||||
def unused_indexes
|
||||
# implementation
|
||||
end
|
||||
end
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
## Version File Pattern
|
||||
|
||||
Keep version.rb minimal:
|
||||
|
||||
```ruby
|
||||
# lib/gemname/version.rb
|
||||
module GemName
|
||||
VERSION = "2.0.0"
|
||||
end
|
||||
```
|
||||
|
||||
## Require Order in Entry Point
|
||||
|
||||
```ruby
|
||||
# lib/searchkick.rb
|
||||
|
||||
# 1. Standard library
|
||||
require "forwardable"
|
||||
require "json"
|
||||
|
||||
# 2. External dependencies (minimal)
|
||||
require "active_support"
|
||||
|
||||
# 3. Internal files via require_relative
|
||||
require_relative "searchkick/index"
|
||||
require_relative "searchkick/model"
|
||||
require_relative "searchkick/query"
|
||||
require_relative "searchkick/version"
|
||||
|
||||
# 4. Conditional Rails loading (LAST)
|
||||
require_relative "searchkick/railtie" if defined?(Rails)
|
||||
```
|
||||
|
||||
## Autoload vs Require
|
||||
|
||||
Kane uses explicit `require_relative`, not autoload:
|
||||
|
||||
```ruby
|
||||
# CORRECT
|
||||
require_relative "gemname/model"
|
||||
require_relative "gemname/query"
|
||||
|
||||
# AVOID
|
||||
autoload :Model, "gemname/model"
|
||||
autoload :Query, "gemname/query"
|
||||
```
|
||||
|
||||
## Comments Style
|
||||
|
||||
Minimal section headers only:
|
||||
|
||||
```ruby
|
||||
# dependencies
|
||||
require "active_support"
|
||||
|
||||
# adapters
|
||||
require_relative "adapters/postgresql_adapter"
|
||||
|
||||
# modules
|
||||
require_relative "migration"
|
||||
```
|
||||
@@ -1,183 +0,0 @@
|
||||
# Rails Integration Patterns
|
||||
|
||||
## The Golden Rule
|
||||
|
||||
**Never require Rails gems directly.** This causes loading order issues.
|
||||
|
||||
```ruby
|
||||
# WRONG - causes premature loading
|
||||
require "active_record"
|
||||
ActiveRecord::Base.include(MyGem::Model)
|
||||
|
||||
# CORRECT - lazy loading
|
||||
ActiveSupport.on_load(:active_record) do
|
||||
extend MyGem::Model
|
||||
end
|
||||
```
|
||||
|
||||
## ActiveSupport.on_load Hooks
|
||||
|
||||
Common hooks and their uses:
|
||||
|
||||
```ruby
|
||||
# Models
|
||||
ActiveSupport.on_load(:active_record) do
|
||||
extend GemName::Model # Add class methods (searchkick, has_encrypted)
|
||||
include GemName::Callbacks # Add instance methods
|
||||
end
|
||||
|
||||
# Controllers
|
||||
ActiveSupport.on_load(:action_controller) do
|
||||
include Ahoy::Controller
|
||||
end
|
||||
|
||||
# Jobs
|
||||
ActiveSupport.on_load(:active_job) do
|
||||
include GemName::JobExtensions
|
||||
end
|
||||
|
||||
# Mailers
|
||||
ActiveSupport.on_load(:action_mailer) do
|
||||
include GemName::MailerExtensions
|
||||
end
|
||||
```
|
||||
|
||||
## Prepend for Behavior Modification
|
||||
|
||||
When overriding existing Rails methods:
|
||||
|
||||
```ruby
|
||||
ActiveSupport.on_load(:active_record) do
|
||||
ActiveRecord::Migration.prepend(StrongMigrations::Migration)
|
||||
ActiveRecord::Migrator.prepend(StrongMigrations::Migrator)
|
||||
end
|
||||
```
|
||||
|
||||
## Railtie Pattern
|
||||
|
||||
Minimal Railtie for non-mountable gems:
|
||||
|
||||
```ruby
|
||||
# lib/gemname/railtie.rb
|
||||
module GemName
|
||||
class Railtie < Rails::Railtie
|
||||
initializer "gemname.configure" do
|
||||
ActiveSupport.on_load(:active_record) do
|
||||
extend GemName::Model
|
||||
end
|
||||
end
|
||||
|
||||
# Optional: Add to controller runtime logging
|
||||
initializer "gemname.log_runtime" do
|
||||
require_relative "controller_runtime"
|
||||
ActiveSupport.on_load(:action_controller) do
|
||||
include GemName::ControllerRuntime
|
||||
end
|
||||
end
|
||||
|
||||
# Optional: Rake tasks
|
||||
rake_tasks do
|
||||
load "tasks/gemname.rake"
|
||||
end
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
## Engine Pattern (Mountable Gems)
|
||||
|
||||
For gems with web interfaces (PgHero, Blazer, Ahoy):
|
||||
|
||||
```ruby
|
||||
# lib/pghero/engine.rb
|
||||
module PgHero
|
||||
class Engine < ::Rails::Engine
|
||||
isolate_namespace PgHero
|
||||
|
||||
initializer "pghero.assets", group: :all do |app|
|
||||
if app.config.respond_to?(:assets) && defined?(Sprockets)
|
||||
app.config.assets.precompile << "pghero/application.js"
|
||||
app.config.assets.precompile << "pghero/application.css"
|
||||
end
|
||||
end
|
||||
|
||||
initializer "pghero.config" do
|
||||
PgHero.config = Rails.application.config_for(:pghero) rescue {}
|
||||
end
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
## Routes for Engines
|
||||
|
||||
```ruby
|
||||
# config/routes.rb (in engine)
|
||||
PgHero::Engine.routes.draw do
|
||||
root to: "home#index"
|
||||
resources :databases, only: [:show]
|
||||
end
|
||||
```
|
||||
|
||||
Mount in app:
|
||||
|
||||
```ruby
|
||||
# config/routes.rb (in app)
|
||||
mount PgHero::Engine, at: "pghero"
|
||||
```
|
||||
|
||||
## YAML Configuration with ERB
|
||||
|
||||
For complex gems needing config files:
|
||||
|
||||
```ruby
|
||||
def self.settings
|
||||
@settings ||= begin
|
||||
path = Rails.root.join("config", "blazer.yml")
|
||||
if path.exist?
|
||||
YAML.safe_load(ERB.new(File.read(path)).result, aliases: true)
|
||||
else
|
||||
{}
|
||||
end
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
## Generator Pattern
|
||||
|
||||
```ruby
|
||||
# lib/generators/gemname/install_generator.rb
|
||||
module GemName
|
||||
module Generators
|
||||
class InstallGenerator < Rails::Generators::Base
|
||||
source_root File.expand_path("templates", __dir__)
|
||||
|
||||
def copy_initializer
|
||||
template "initializer.rb", "config/initializers/gemname.rb"
|
||||
end
|
||||
|
||||
def copy_migration
|
||||
migration_template "migration.rb", "db/migrate/create_gemname_tables.rb"
|
||||
end
|
||||
end
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
## Conditional Feature Detection
|
||||
|
||||
```ruby
|
||||
# Check for specific Rails versions
|
||||
if ActiveRecord.version >= Gem::Version.new("7.0")
|
||||
# Rails 7+ specific code
|
||||
end
|
||||
|
||||
# Check for optional dependencies
|
||||
def self.client
|
||||
@client ||= if defined?(OpenSearch::Client)
|
||||
OpenSearch::Client.new
|
||||
elsif defined?(Elasticsearch::Client)
|
||||
Elasticsearch::Client.new
|
||||
else
|
||||
raise Error, "Install elasticsearch or opensearch-ruby"
|
||||
end
|
||||
end
|
||||
```
|
||||
@@ -1,119 +0,0 @@
|
||||
# Andrew Kane Resources
|
||||
|
||||
## Primary Documentation
|
||||
|
||||
- **Gem Patterns Article**: https://ankane.org/gem-patterns
|
||||
- Kane's own documentation of patterns used across his gems
|
||||
- Covers configuration, Rails integration, error handling
|
||||
|
||||
## Top Ruby Gems by Stars
|
||||
|
||||
### Search & Data
|
||||
|
||||
| Gem | Stars | Description | Source |
|
||||
|-----|-------|-------------|--------|
|
||||
| **Searchkick** | 6.6k+ | Intelligent search for Rails | https://github.com/ankane/searchkick |
|
||||
| **Chartkick** | 6.4k+ | Beautiful charts in Ruby | https://github.com/ankane/chartkick |
|
||||
| **Groupdate** | 3.8k+ | Group by day, week, month | https://github.com/ankane/groupdate |
|
||||
| **Blazer** | 4.6k+ | SQL dashboard for Rails | https://github.com/ankane/blazer |
|
||||
|
||||
### Database & Migrations
|
||||
|
||||
| Gem | Stars | Description | Source |
|
||||
|-----|-------|-------------|--------|
|
||||
| **PgHero** | 8.2k+ | PostgreSQL insights | https://github.com/ankane/pghero |
|
||||
| **Strong Migrations** | 4.1k+ | Safe migration checks | https://github.com/ankane/strong_migrations |
|
||||
| **Dexter** | 1.8k+ | Auto index advisor | https://github.com/ankane/dexter |
|
||||
| **PgSync** | 1.5k+ | Sync Postgres data | https://github.com/ankane/pgsync |
|
||||
|
||||
### Security & Encryption
|
||||
|
||||
| Gem | Stars | Description | Source |
|
||||
|-----|-------|-------------|--------|
|
||||
| **Lockbox** | 1.5k+ | Application-level encryption | https://github.com/ankane/lockbox |
|
||||
| **Blind Index** | 1.0k+ | Encrypted search | https://github.com/ankane/blind_index |
|
||||
| **Secure Headers** | — | Contributed patterns | Referenced in gems |
|
||||
|
||||
### Analytics & ML
|
||||
|
||||
| Gem | Stars | Description | Source |
|
||||
|-----|-------|-------------|--------|
|
||||
| **Ahoy** | 4.2k+ | Analytics for Rails | https://github.com/ankane/ahoy |
|
||||
| **Neighbor** | 1.1k+ | Vector search for Rails | https://github.com/ankane/neighbor |
|
||||
| **Rover** | 700+ | DataFrames for Ruby | https://github.com/ankane/rover |
|
||||
| **Tomoto** | 200+ | Topic modeling | https://github.com/ankane/tomoto-ruby |
|
||||
|
||||
### Utilities
|
||||
|
||||
| Gem | Stars | Description | Source |
|
||||
|-----|-------|-------------|--------|
|
||||
| **Pretender** | 2.0k+ | Login as another user | https://github.com/ankane/pretender |
|
||||
| **Authtrail** | 900+ | Login activity tracking | https://github.com/ankane/authtrail |
|
||||
| **Notable** | 200+ | Track notable requests | https://github.com/ankane/notable |
|
||||
| **Logstop** | 200+ | Filter sensitive logs | https://github.com/ankane/logstop |
|
||||
|
||||
## Key Source Files to Study
|
||||
|
||||
### Entry Point Patterns
|
||||
- https://github.com/ankane/searchkick/blob/master/lib/searchkick.rb
|
||||
- https://github.com/ankane/pghero/blob/master/lib/pghero.rb
|
||||
- https://github.com/ankane/strong_migrations/blob/master/lib/strong_migrations.rb
|
||||
- https://github.com/ankane/lockbox/blob/master/lib/lockbox.rb
|
||||
|
||||
### Class Macro Implementations
|
||||
- https://github.com/ankane/searchkick/blob/master/lib/searchkick/model.rb
|
||||
- https://github.com/ankane/lockbox/blob/master/lib/lockbox/model.rb
|
||||
- https://github.com/ankane/neighbor/blob/master/lib/neighbor/model.rb
|
||||
- https://github.com/ankane/blind_index/blob/master/lib/blind_index/model.rb
|
||||
|
||||
### Rails Integration (Railtie/Engine)
|
||||
- https://github.com/ankane/pghero/blob/master/lib/pghero/engine.rb
|
||||
- https://github.com/ankane/searchkick/blob/master/lib/searchkick/railtie.rb
|
||||
- https://github.com/ankane/ahoy/blob/master/lib/ahoy/engine.rb
|
||||
- https://github.com/ankane/blazer/blob/master/lib/blazer/engine.rb
|
||||
|
||||
### Database Adapters
|
||||
- https://github.com/ankane/strong_migrations/tree/master/lib/strong_migrations/adapters
|
||||
- https://github.com/ankane/groupdate/tree/master/lib/groupdate/adapters
|
||||
- https://github.com/ankane/neighbor/tree/master/lib/neighbor
|
||||
|
||||
### Error Messages (Template Pattern)
|
||||
- https://github.com/ankane/strong_migrations/blob/master/lib/strong_migrations/error_messages.rb
|
||||
|
||||
### Gemspec Examples
|
||||
- https://github.com/ankane/searchkick/blob/master/searchkick.gemspec
|
||||
- https://github.com/ankane/neighbor/blob/master/neighbor.gemspec
|
||||
- https://github.com/ankane/ahoy/blob/master/ahoy_matey.gemspec
|
||||
|
||||
### Test Setups
|
||||
- https://github.com/ankane/searchkick/tree/master/test
|
||||
- https://github.com/ankane/lockbox/tree/master/test
|
||||
- https://github.com/ankane/strong_migrations/tree/master/test
|
||||
|
||||
## GitHub Profile
|
||||
|
||||
- **Profile**: https://github.com/ankane
|
||||
- **All Ruby Repos**: https://github.com/ankane?tab=repositories&q=&type=&language=ruby&sort=stargazers
|
||||
- **RubyGems Profile**: https://rubygems.org/profiles/ankane
|
||||
|
||||
## Blog Posts & Articles
|
||||
|
||||
- **ankane.org**: https://ankane.org/
|
||||
- **Gem Patterns**: https://ankane.org/gem-patterns (essential reading)
|
||||
- **Postgres Performance**: https://ankane.org/introducing-pghero
|
||||
- **Search Tips**: https://ankane.org/search-rails
|
||||
|
||||
## Design Philosophy Summary
|
||||
|
||||
From studying 100+ gems, Kane's consistent principles:
|
||||
|
||||
1. **Zero dependencies when possible** - Each dep is a maintenance burden
|
||||
2. **ActiveSupport.on_load always** - Never require Rails gems directly
|
||||
3. **Class macro DSLs** - Single method configures everything
|
||||
4. **Explicit over magic** - No method_missing, define methods directly
|
||||
5. **Minitest only** - Simple, sufficient, no RSpec
|
||||
6. **Multi-version testing** - Support broad Rails/Ruby versions
|
||||
7. **Helpful errors** - Template-based messages with fix suggestions
|
||||
8. **Abstract adapters** - Clean multi-database support
|
||||
9. **Engine isolation** - isolate_namespace for mountable gems
|
||||
10. **Minimal documentation** - Code is self-documenting, README is examples
|
||||
@@ -1,261 +0,0 @@
|
||||
# Testing Patterns
|
||||
|
||||
## Minitest Setup
|
||||
|
||||
Kane exclusively uses Minitest—never RSpec.
|
||||
|
||||
```ruby
|
||||
# test/test_helper.rb
|
||||
require "bundler/setup"
|
||||
Bundler.require(:default)
|
||||
require "minitest/autorun"
|
||||
require "minitest/pride"
|
||||
|
||||
# Load the gem
|
||||
require "gemname"
|
||||
|
||||
# Test database setup (if needed)
|
||||
ActiveRecord::Base.establish_connection(
|
||||
adapter: "postgresql",
|
||||
database: "gemname_test"
|
||||
)
|
||||
|
||||
# Base test class
|
||||
class Minitest::Test
|
||||
def setup
|
||||
# Reset state before each test
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
## Test File Structure
|
||||
|
||||
```ruby
|
||||
# test/model_test.rb
|
||||
require_relative "test_helper"
|
||||
|
||||
class ModelTest < Minitest::Test
|
||||
def setup
|
||||
User.delete_all
|
||||
end
|
||||
|
||||
def test_basic_functionality
|
||||
user = User.create!(email: "test@example.org")
|
||||
assert_equal "test@example.org", user.email
|
||||
end
|
||||
|
||||
def test_with_invalid_input
|
||||
error = assert_raises(ArgumentError) do
|
||||
User.create!(email: nil)
|
||||
end
|
||||
assert_match /email/, error.message
|
||||
end
|
||||
|
||||
def test_class_method
|
||||
result = User.search("test")
|
||||
assert_kind_of Array, result
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
## Multi-Version Testing
|
||||
|
||||
Test against multiple Rails/Ruby versions using gemfiles:
|
||||
|
||||
```
|
||||
test/
|
||||
├── test_helper.rb
|
||||
└── gemfiles/
|
||||
├── activerecord70.gemfile
|
||||
├── activerecord71.gemfile
|
||||
└── activerecord72.gemfile
|
||||
```
|
||||
|
||||
```ruby
|
||||
# test/gemfiles/activerecord70.gemfile
|
||||
source "https://rubygems.org"
|
||||
gemspec path: "../../"
|
||||
|
||||
gem "activerecord", "~> 7.0.0"
|
||||
gem "sqlite3"
|
||||
```
|
||||
|
||||
```ruby
|
||||
# test/gemfiles/activerecord72.gemfile
|
||||
source "https://rubygems.org"
|
||||
gemspec path: "../../"
|
||||
|
||||
gem "activerecord", "~> 7.2.0"
|
||||
gem "sqlite3"
|
||||
```
|
||||
|
||||
Run with specific gemfile:
|
||||
|
||||
```bash
|
||||
BUNDLE_GEMFILE=test/gemfiles/activerecord70.gemfile bundle install
|
||||
BUNDLE_GEMFILE=test/gemfiles/activerecord70.gemfile bundle exec rake test
|
||||
```
|
||||
|
||||
## Rakefile
|
||||
|
||||
```ruby
|
||||
# Rakefile
|
||||
require "bundler/gem_tasks"
|
||||
require "rake/testtask"
|
||||
|
||||
Rake::TestTask.new(:test) do |t|
|
||||
t.libs << "test"
|
||||
t.pattern = "test/**/*_test.rb"
|
||||
end
|
||||
|
||||
task default: :test
|
||||
```
|
||||
|
||||
## GitHub Actions CI
|
||||
|
||||
```yaml
|
||||
# .github/workflows/build.yml
|
||||
name: build
|
||||
|
||||
on: [push, pull_request]
|
||||
|
||||
jobs:
|
||||
build:
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
include:
|
||||
- ruby: "3.2"
|
||||
gemfile: activerecord70
|
||||
- ruby: "3.3"
|
||||
gemfile: activerecord71
|
||||
- ruby: "3.3"
|
||||
gemfile: activerecord72
|
||||
|
||||
env:
|
||||
BUNDLE_GEMFILE: test/gemfiles/${{ matrix.gemfile }}.gemfile
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- uses: ruby/setup-ruby@v1
|
||||
with:
|
||||
ruby-version: ${{ matrix.ruby }}
|
||||
bundler-cache: true
|
||||
|
||||
- run: bundle exec rake test
|
||||
```
|
||||
|
||||
## Database-Specific Testing
|
||||
|
||||
```yaml
|
||||
# .github/workflows/build.yml (with services)
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:15
|
||||
env:
|
||||
POSTGRES_USER: postgres
|
||||
POSTGRES_PASSWORD: postgres
|
||||
ports:
|
||||
- 5432:5432
|
||||
options: >-
|
||||
--health-cmd pg_isready
|
||||
--health-interval 10s
|
||||
--health-timeout 5s
|
||||
--health-retries 5
|
||||
|
||||
env:
|
||||
DATABASE_URL: postgres://postgres:postgres@localhost/gemname_test
|
||||
```
|
||||
|
||||
## Test Database Setup
|
||||
|
||||
```ruby
|
||||
# test/test_helper.rb
|
||||
require "active_record"
|
||||
|
||||
# Connect to database
|
||||
ActiveRecord::Base.establish_connection(
|
||||
ENV["DATABASE_URL"] || {
|
||||
adapter: "postgresql",
|
||||
database: "gemname_test"
|
||||
}
|
||||
)
|
||||
|
||||
# Create tables
|
||||
ActiveRecord::Schema.define do
|
||||
create_table :users, force: true do |t|
|
||||
t.string :email
|
||||
t.text :encrypted_data
|
||||
t.timestamps
|
||||
end
|
||||
end
|
||||
|
||||
# Define models
|
||||
class User < ActiveRecord::Base
|
||||
gemname_feature :email
|
||||
end
|
||||
```
|
||||
|
||||
## Assertion Patterns
|
||||
|
||||
```ruby
|
||||
# Basic assertions
|
||||
assert result
|
||||
assert_equal expected, actual
|
||||
assert_nil value
|
||||
assert_empty array
|
||||
|
||||
# Exception testing
|
||||
assert_raises(ArgumentError) { bad_code }
|
||||
|
||||
error = assert_raises(GemName::Error) do
|
||||
risky_operation
|
||||
end
|
||||
assert_match /expected message/, error.message
|
||||
|
||||
# Refutations
|
||||
refute condition
|
||||
refute_equal unexpected, actual
|
||||
refute_nil value
|
||||
```
|
||||
|
||||
## Test Helpers
|
||||
|
||||
```ruby
|
||||
# test/test_helper.rb
|
||||
class Minitest::Test
|
||||
def with_options(options)
|
||||
original = GemName.options.dup
|
||||
GemName.options.merge!(options)
|
||||
yield
|
||||
ensure
|
||||
GemName.options = original
|
||||
end
|
||||
|
||||
def assert_queries(expected_count)
|
||||
queries = []
|
||||
callback = ->(*, payload) { queries << payload[:sql] }
|
||||
ActiveSupport::Notifications.subscribe("sql.active_record", callback)
|
||||
yield
|
||||
assert_equal expected_count, queries.size, "Expected #{expected_count} queries, got #{queries.size}"
|
||||
ensure
|
||||
ActiveSupport::Notifications.unsubscribe(callback)
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
## Skipping Tests
|
||||
|
||||
```ruby
|
||||
def test_postgresql_specific
|
||||
skip "PostgreSQL only" unless postgresql?
|
||||
# test code
|
||||
end
|
||||
|
||||
def postgresql?
|
||||
ActiveRecord::Base.connection.adapter_name =~ /postg/i
|
||||
end
|
||||
```
|
||||
@@ -1,138 +0,0 @@
|
||||
---
|
||||
name: ce-changelog
|
||||
description: Create engaging changelogs for recent merges to main branch
|
||||
argument-hint: "[optional: daily|weekly, or time period in days]"
|
||||
disable-model-invocation: true
|
||||
---
|
||||
|
||||
You are a witty and enthusiastic product marketer tasked with creating a fun, engaging change log for an internal development team. Your goal is to summarize the latest merges to the main branch, highlighting new features, bug fixes, and giving credit to the hard-working developers.
|
||||
|
||||
## Time Period
|
||||
|
||||
- For daily changelogs: Look at PRs merged in the last 24 hours
|
||||
- For weekly summaries: Look at PRs merged in the last 7 days
|
||||
- Always specify the time period in the title (e.g., "Daily" vs "Weekly")
|
||||
- Default: Get the latest changes from the last day from the main branch of the repository
|
||||
|
||||
## PR Analysis
|
||||
|
||||
Analyze the provided GitHub changes and related issues. Look for:
|
||||
|
||||
1. New features that have been added
|
||||
2. Bug fixes that have been implemented
|
||||
3. Any other significant changes or improvements
|
||||
4. References to specific issues and their details
|
||||
5. Names of contributors who made the changes
|
||||
6. Use gh cli to lookup the PRs as well and the description of the PRs
|
||||
7. Check PR labels to identify feature type (feature, bug, chore, etc.)
|
||||
8. Look for breaking changes and highlight them prominently
|
||||
9. Include PR numbers for traceability
|
||||
10. Check if PRs are linked to issues and include issue context
|
||||
|
||||
## Content Priorities
|
||||
|
||||
1. Breaking changes (if any) - MUST be at the top
|
||||
2. User-facing features
|
||||
3. Critical bug fixes
|
||||
4. Performance improvements
|
||||
5. Developer experience improvements
|
||||
6. Documentation updates
|
||||
|
||||
## Formatting Guidelines
|
||||
|
||||
Now, create a change log summary with the following guidelines:
|
||||
|
||||
1. Keep it concise and to the point
|
||||
2. Highlight the most important changes first
|
||||
3. Group similar changes together (e.g., all new features, all bug fixes)
|
||||
4. Include issue references where applicable
|
||||
5. Mention the names of contributors, giving them credit for their work
|
||||
6. Add a touch of humor or playfulness to make it engaging
|
||||
7. Use emojis sparingly to add visual interest
|
||||
8. Keep total message under 2000 characters for Discord
|
||||
9. Use consistent emoji for each section
|
||||
10. Format code/technical terms in backticks
|
||||
11. Include PR numbers in parentheses (e.g., "Fixed login bug (#123)")
|
||||
|
||||
## Deployment Notes
|
||||
|
||||
When relevant, include:
|
||||
|
||||
- Database migrations required
|
||||
- Environment variable updates needed
|
||||
- Manual intervention steps post-deploy
|
||||
- Dependencies that need updating
|
||||
|
||||
Your final output should be formatted as follows:
|
||||
|
||||
<change_log>
|
||||
|
||||
# 🚀 [Daily/Weekly] Change Log: [Current Date]
|
||||
|
||||
## 🚨 Breaking Changes (if any)
|
||||
|
||||
[List any breaking changes that require immediate attention]
|
||||
|
||||
## 🌟 New Features
|
||||
|
||||
[List new features here with PR numbers]
|
||||
|
||||
## 🐛 Bug Fixes
|
||||
|
||||
[List bug fixes here with PR numbers]
|
||||
|
||||
## 🛠️ Other Improvements
|
||||
|
||||
[List other significant changes or improvements]
|
||||
|
||||
## 🙌 Shoutouts
|
||||
|
||||
[Mention contributors and their contributions]
|
||||
|
||||
## 🎉 Fun Fact of the Day
|
||||
|
||||
[Include a brief, work-related fun fact or joke]
|
||||
|
||||
</change_log>
|
||||
|
||||
## Style Guide Review
|
||||
|
||||
Now review the changelog using the EVERY_WRITE_STYLE.md file and go one by one to make sure you are following the style guide. Use multiple agents, run in parallel to make it faster.
|
||||
|
||||
Remember, your final output should only include the content within the <change_log> tags. Do not include any of your thought process or the original data in the output.
|
||||
|
||||
## Discord Posting (Optional)
|
||||
|
||||
You can post changelogs to Discord by adding your own webhook URL:
|
||||
|
||||
```
|
||||
# Set your Discord webhook URL
|
||||
DISCORD_WEBHOOK_URL="https://discord.com/api/webhooks/YOUR_WEBHOOK_ID/YOUR_WEBHOOK_TOKEN"
|
||||
|
||||
# Post using curl
|
||||
curl -H "Content-Type: application/json" \
|
||||
-d "{\"content\": \"{{CHANGELOG}}\"}" \
|
||||
$DISCORD_WEBHOOK_URL
|
||||
```
|
||||
|
||||
To get a webhook URL, go to your Discord server → Server Settings → Integrations → Webhooks → New Webhook.
|
||||
|
||||
## Error Handling
|
||||
|
||||
- If no changes in the time period, post a "quiet day" message: "🌤️ Quiet day! No new changes merged."
|
||||
- If unable to fetch PR details, list the PR numbers for manual review
|
||||
- Always validate message length before posting to Discord (max 2000 chars)
|
||||
|
||||
## Schedule Recommendations
|
||||
|
||||
- Run daily at 6 AM NY time for previous day's changes
|
||||
- Run weekly summary on Mondays for the previous week
|
||||
- Special runs after major releases or deployments
|
||||
|
||||
## Audience Considerations
|
||||
|
||||
Adjust the tone and detail level based on the channel:
|
||||
|
||||
- **Dev team channels**: Include technical details, performance metrics, code snippets
|
||||
- **Product team channels**: Focus on user-facing changes and business impact
|
||||
- **Leadership channels**: Highlight progress on key initiatives and blockers
|
||||
@@ -1,112 +0,0 @@
|
||||
---
|
||||
name: ce-deploy-docs
|
||||
description: Validate and prepare documentation for GitHub Pages deployment
|
||||
disable-model-invocation: true
|
||||
---
|
||||
|
||||
# Deploy Documentation Command
|
||||
|
||||
Validate the documentation site and prepare it for GitHub Pages deployment.
|
||||
|
||||
## Step 1: Validate Documentation
|
||||
|
||||
Run these checks:
|
||||
|
||||
```bash
|
||||
# Count components
|
||||
echo "Agents: $(ls plugins/compound-engineering/agents/*.md | wc -l)"
|
||||
echo "Skills: $(ls -d plugins/compound-engineering/skills/*/ 2>/dev/null | wc -l)"
|
||||
|
||||
# Validate JSON
|
||||
cat .claude-plugin/marketplace.json | jq . > /dev/null && echo "✓ marketplace.json valid"
|
||||
cat plugins/compound-engineering/.claude-plugin/plugin.json | jq . > /dev/null && echo "✓ plugin.json valid"
|
||||
|
||||
# Check all HTML files exist
|
||||
for page in index agents commands skills mcp-servers changelog getting-started; do
|
||||
if [ -f "plugins/compound-engineering/docs/pages/${page}.html" ] || [ -f "plugins/compound-engineering/docs/${page}.html" ]; then
|
||||
echo "✓ ${page}.html exists"
|
||||
else
|
||||
echo "✗ ${page}.html MISSING"
|
||||
fi
|
||||
done
|
||||
```
|
||||
|
||||
## Step 2: Check for Uncommitted Changes
|
||||
|
||||
```bash
|
||||
git status --porcelain plugins/compound-engineering/docs/
|
||||
```
|
||||
|
||||
If there are uncommitted changes, warn the user to commit first.
|
||||
|
||||
## Step 3: Deployment Instructions
|
||||
|
||||
Since GitHub Pages deployment requires a workflow file with special permissions, provide these instructions:
|
||||
|
||||
### First-time Setup
|
||||
|
||||
1. Create `.github/workflows/deploy-docs.yml` with the GitHub Pages workflow
|
||||
2. Go to repository Settings > Pages
|
||||
3. Set Source to "GitHub Actions"
|
||||
|
||||
### Deploying
|
||||
|
||||
After merging to `main`, the docs will auto-deploy. Or:
|
||||
|
||||
1. Go to Actions tab
|
||||
2. Select "Deploy Documentation to GitHub Pages"
|
||||
3. Click "Run workflow"
|
||||
|
||||
### Workflow File Content
|
||||
|
||||
```yaml
|
||||
name: Deploy Documentation to GitHub Pages
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
paths:
|
||||
- 'plugins/compound-engineering/docs/**'
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
pages: write
|
||||
id-token: write
|
||||
|
||||
concurrency:
|
||||
group: "pages"
|
||||
cancel-in-progress: false
|
||||
|
||||
jobs:
|
||||
deploy:
|
||||
environment:
|
||||
name: github-pages
|
||||
url: ${{ steps.deployment.outputs.page_url }}
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/configure-pages@v4
|
||||
- uses: actions/upload-pages-artifact@v3
|
||||
with:
|
||||
path: 'plugins/compound-engineering/docs'
|
||||
- uses: actions/deploy-pages@v4
|
||||
```
|
||||
|
||||
## Step 4: Report Status
|
||||
|
||||
Provide a summary:
|
||||
|
||||
```
|
||||
## Deployment Readiness
|
||||
|
||||
✓ All HTML pages present
|
||||
✓ JSON files valid
|
||||
✓ Component counts match
|
||||
|
||||
### Next Steps
|
||||
- [ ] Commit any pending changes
|
||||
- [ ] Push to main branch
|
||||
- [ ] Verify GitHub Pages workflow exists
|
||||
- [ ] Check deployment at https://everyinc.github.io/compound-engineering-plugin/
|
||||
```
|
||||
@@ -1,737 +0,0 @@
|
||||
---
|
||||
name: ce-dspy-ruby
|
||||
description: Build type-safe LLM applications with DSPy.rb — Ruby's programmatic prompt framework with signatures, modules, agents, and optimization. Use when implementing predictable AI features, creating LLM signatures and modules, configuring language model providers, building agent systems with tools, optimizing prompts, or testing LLM-powered functionality in Ruby applications.
|
||||
---
|
||||
|
||||
# DSPy.rb
|
||||
|
||||
> Build LLM apps like you build software. Type-safe, modular, testable.
|
||||
|
||||
DSPy.rb brings software engineering best practices to LLM development. Instead of tweaking prompts, define what you want with Ruby types and let DSPy handle the rest.
|
||||
|
||||
## Overview
|
||||
|
||||
DSPy.rb is a Ruby framework for building language model applications with programmatic prompts. It provides:
|
||||
|
||||
- **Type-safe signatures** — Define inputs/outputs with Sorbet types
|
||||
- **Modular components** — Compose and reuse LLM logic
|
||||
- **Automatic optimization** — Use data to improve prompts, not guesswork
|
||||
- **Production-ready** — Built-in observability, testing, and error handling
|
||||
|
||||
## Core Concepts
|
||||
|
||||
### 1. Signatures
|
||||
|
||||
Define interfaces between your app and LLMs using Ruby types:
|
||||
|
||||
```ruby
|
||||
class EmailClassifier < DSPy::Signature
|
||||
description "Classify customer support emails by category and priority"
|
||||
|
||||
class Priority < T::Enum
|
||||
enums do
|
||||
Low = new('low')
|
||||
Medium = new('medium')
|
||||
High = new('high')
|
||||
Urgent = new('urgent')
|
||||
end
|
||||
end
|
||||
|
||||
input do
|
||||
const :email_content, String
|
||||
const :sender, String
|
||||
end
|
||||
|
||||
output do
|
||||
const :category, String
|
||||
const :priority, Priority # Type-safe enum with defined values
|
||||
const :confidence, Float
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
### 2. Modules
|
||||
|
||||
Build complex workflows from simple building blocks:
|
||||
|
||||
- **Predict** — Basic LLM calls with signatures
|
||||
- **ChainOfThought** — Step-by-step reasoning
|
||||
- **ReAct** — Tool-using agents
|
||||
- **CodeAct** — Dynamic code generation agents (install the `dspy-code_act` gem)
|
||||
|
||||
### 3. Tools & Toolsets
|
||||
|
||||
Create type-safe tools for agents with comprehensive Sorbet support:
|
||||
|
||||
```ruby
|
||||
# Enum-based tool with automatic type conversion
|
||||
class CalculatorTool < DSPy::Tools::Base
|
||||
tool_name 'calculator'
|
||||
tool_description 'Performs arithmetic operations with type-safe enum inputs'
|
||||
|
||||
class Operation < T::Enum
|
||||
enums do
|
||||
Add = new('add')
|
||||
Subtract = new('subtract')
|
||||
Multiply = new('multiply')
|
||||
Divide = new('divide')
|
||||
end
|
||||
end
|
||||
|
||||
sig { params(operation: Operation, num1: Float, num2: Float).returns(T.any(Float, String)) }
|
||||
def call(operation:, num1:, num2:)
|
||||
case operation
|
||||
when Operation::Add then num1 + num2
|
||||
when Operation::Subtract then num1 - num2
|
||||
when Operation::Multiply then num1 * num2
|
||||
when Operation::Divide
|
||||
return "Error: Division by zero" if num2 == 0
|
||||
num1 / num2
|
||||
end
|
||||
end
|
||||
end
|
||||
|
||||
# Multi-tool toolset with rich types
|
||||
class DataToolset < DSPy::Tools::Toolset
|
||||
toolset_name "data_processing"
|
||||
|
||||
class Format < T::Enum
|
||||
enums do
|
||||
JSON = new('json')
|
||||
CSV = new('csv')
|
||||
XML = new('xml')
|
||||
end
|
||||
end
|
||||
|
||||
tool :convert, description: "Convert data between formats"
|
||||
tool :validate, description: "Validate data structure"
|
||||
|
||||
sig { params(data: String, from: Format, to: Format).returns(String) }
|
||||
def convert(data:, from:, to:)
|
||||
"Converted from #{from.serialize} to #{to.serialize}"
|
||||
end
|
||||
|
||||
sig { params(data: String, format: Format).returns(T::Hash[String, T.any(String, Integer, T::Boolean)]) }
|
||||
def validate(data:, format:)
|
||||
{ valid: true, format: format.serialize, row_count: 42, message: "Data validation passed" }
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
### 4. Type System & Discriminators
|
||||
|
||||
DSPy.rb uses sophisticated type discrimination for complex data structures:
|
||||
|
||||
- **Automatic `_type` field injection** — DSPy adds discriminator fields to structs for type safety
|
||||
- **Union type support** — `T.any()` types automatically disambiguated by `_type`
|
||||
- **Reserved field name** — Avoid defining your own `_type` fields in structs
|
||||
- **Recursive filtering** — `_type` fields filtered during deserialization at all nesting levels
|
||||
|
||||
### 5. Optimization
|
||||
|
||||
Improve accuracy with real data:
|
||||
|
||||
- **MIPROv2** — Advanced multi-prompt optimization with bootstrap sampling and Bayesian optimization
|
||||
- **GEPA** — Genetic-Pareto Reflective Prompt Evolution with feedback maps, experiment tracking, and telemetry
|
||||
- **Evaluation** — Comprehensive framework with built-in and custom metrics, error handling, and batch processing
|
||||
|
||||
## Quick Start
|
||||
|
||||
```ruby
|
||||
# Install
|
||||
gem 'dspy'
|
||||
|
||||
# Configure
|
||||
DSPy.configure do |c|
|
||||
c.lm = DSPy::LM.new('openai/gpt-4o-mini', api_key: ENV['OPENAI_API_KEY'])
|
||||
end
|
||||
|
||||
# Define a task
|
||||
class SentimentAnalysis < DSPy::Signature
|
||||
description "Analyze sentiment of text"
|
||||
|
||||
input do
|
||||
const :text, String
|
||||
end
|
||||
|
||||
output do
|
||||
const :sentiment, String # positive, negative, neutral
|
||||
const :score, Float # 0.0 to 1.0
|
||||
end
|
||||
end
|
||||
|
||||
# Use it
|
||||
analyzer = DSPy::Predict.new(SentimentAnalysis)
|
||||
result = analyzer.call(text: "This product is amazing!")
|
||||
puts result.sentiment # => "positive"
|
||||
puts result.score # => 0.92
|
||||
```
|
||||
|
||||
## Provider Adapter Gems
|
||||
|
||||
Two strategies for connecting to LLM providers:
|
||||
|
||||
### Per-provider adapters (direct SDK access)
|
||||
|
||||
```ruby
|
||||
# Gemfile
|
||||
gem 'dspy'
|
||||
gem 'dspy-openai' # OpenAI, OpenRouter, Ollama
|
||||
gem 'dspy-anthropic' # Claude
|
||||
gem 'dspy-gemini' # Gemini
|
||||
```
|
||||
|
||||
Each adapter gem pulls in the official SDK (`openai`, `anthropic`, `gemini-ai`).
|
||||
|
||||
### Unified adapter via RubyLLM (recommended for multi-provider)
|
||||
|
||||
```ruby
|
||||
# Gemfile
|
||||
gem 'dspy'
|
||||
gem 'dspy-ruby_llm' # Routes to any provider via ruby_llm
|
||||
gem 'ruby_llm'
|
||||
```
|
||||
|
||||
RubyLLM handles provider routing based on the model name. Use the `ruby_llm/` prefix:
|
||||
|
||||
```ruby
|
||||
DSPy.configure do |c|
|
||||
c.lm = DSPy::LM.new('ruby_llm/gemini-2.5-flash', structured_outputs: true)
|
||||
# c.lm = DSPy::LM.new('ruby_llm/claude-sonnet-4-20250514', structured_outputs: true)
|
||||
# c.lm = DSPy::LM.new('ruby_llm/gpt-4o-mini', structured_outputs: true)
|
||||
end
|
||||
```
|
||||
|
||||
## Events System
|
||||
|
||||
DSPy.rb ships with a structured event bus for observing runtime behavior.
|
||||
|
||||
### Module-Scoped Subscriptions (preferred for agents)
|
||||
|
||||
```ruby
|
||||
class MyAgent < DSPy::Module
|
||||
subscribe 'lm.tokens', :track_tokens, scope: :descendants
|
||||
|
||||
def track_tokens(_event, attrs)
|
||||
@total_tokens += attrs.fetch(:total_tokens, 0)
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
### Global Subscriptions (for observability/integrations)
|
||||
|
||||
```ruby
|
||||
subscription_id = DSPy.events.subscribe('score.create') do |event, attrs|
|
||||
Langfuse.export_score(attrs)
|
||||
end
|
||||
|
||||
# Wildcards supported
|
||||
DSPy.events.subscribe('llm.*') { |name, attrs| puts "[#{name}] tokens=#{attrs[:total_tokens]}" }
|
||||
```
|
||||
|
||||
Event names use dot-separated namespaces (`llm.generate`, `react.iteration_complete`). Every event includes module metadata (`module_path`, `module_leaf`, `module_scope.ancestry_token`) for filtering.
|
||||
|
||||
## Lifecycle Callbacks
|
||||
|
||||
Rails-style lifecycle hooks ship with every `DSPy::Module`:
|
||||
|
||||
- **`before`** — Runs ahead of `forward` for setup (metrics, context loading)
|
||||
- **`around`** — Wraps `forward`, calls `yield`, and lets you pair setup/teardown logic
|
||||
- **`after`** — Fires after `forward` returns for cleanup or persistence
|
||||
|
||||
```ruby
|
||||
class InstrumentedModule < DSPy::Module
|
||||
before :setup_metrics
|
||||
around :manage_context
|
||||
after :log_metrics
|
||||
|
||||
def forward(question:)
|
||||
@predictor.call(question: question)
|
||||
end
|
||||
|
||||
private
|
||||
|
||||
def setup_metrics
|
||||
@start_time = Time.now
|
||||
end
|
||||
|
||||
def manage_context
|
||||
load_context
|
||||
result = yield
|
||||
save_context
|
||||
result
|
||||
end
|
||||
|
||||
def log_metrics
|
||||
duration = Time.now - @start_time
|
||||
Rails.logger.info "Prediction completed in #{duration}s"
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
Execution order: before → around (before yield) → forward → around (after yield) → after. Callbacks are inherited from parent classes and execute in registration order.
|
||||
|
||||
## Fiber-Local LM Context
|
||||
|
||||
Override the language model temporarily using fiber-local storage:
|
||||
|
||||
```ruby
|
||||
fast_model = DSPy::LM.new("openai/gpt-4o-mini", api_key: ENV['OPENAI_API_KEY'])
|
||||
|
||||
DSPy.with_lm(fast_model) do
|
||||
result = classifier.call(text: "test") # Uses fast_model inside this block
|
||||
end
|
||||
# Back to global LM outside the block
|
||||
```
|
||||
|
||||
**LM resolution hierarchy**: Instance-level LM → Fiber-local LM (`DSPy.with_lm`) → Global LM (`DSPy.configure`).
|
||||
|
||||
Use `configure_predictor` for fine-grained control over agent internals:
|
||||
|
||||
```ruby
|
||||
agent = DSPy::ReAct.new(MySignature, tools: tools)
|
||||
agent.configure { |c| c.lm = default_model }
|
||||
agent.configure_predictor('thought_generator') { |c| c.lm = powerful_model }
|
||||
```
|
||||
|
||||
## Evaluation Framework
|
||||
|
||||
Systematically test LLM application performance with `DSPy::Evals`:
|
||||
|
||||
```ruby
|
||||
metric = DSPy::Metrics.exact_match(field: :answer, case_sensitive: false)
|
||||
evaluator = DSPy::Evals.new(predictor, metric: metric)
|
||||
result = evaluator.evaluate(test_examples, display_table: true)
|
||||
puts "Pass Rate: #{(result.pass_rate * 100).round(1)}%"
|
||||
```
|
||||
|
||||
Built-in metrics: `exact_match`, `contains`, `numeric_difference`, `composite_and`. Custom metrics return `true`/`false` or a `DSPy::Prediction` with `score:` and `feedback:` fields.
|
||||
|
||||
Use `DSPy::Example` for typed test data and `export_scores: true` to push results to Langfuse.
|
||||
|
||||
## GEPA Optimization
|
||||
|
||||
GEPA (Genetic-Pareto Reflective Prompt Evolution) uses reflection-driven instruction rewrites:
|
||||
|
||||
```ruby
|
||||
gem 'dspy-gepa'
|
||||
|
||||
teleprompter = DSPy::Teleprompt::GEPA.new(
|
||||
metric: metric,
|
||||
reflection_lm: DSPy::ReflectionLM.new('openai/gpt-4o-mini', api_key: ENV['OPENAI_API_KEY']),
|
||||
feedback_map: feedback_map,
|
||||
config: { max_metric_calls: 600, minibatch_size: 6 }
|
||||
)
|
||||
|
||||
result = teleprompter.compile(program, trainset: train, valset: val)
|
||||
optimized_program = result.optimized_program
|
||||
```
|
||||
|
||||
The metric must return `DSPy::Prediction.new(score:, feedback:)` so the reflection model can reason about failures. Use `feedback_map` to target individual predictors in composite modules.
|
||||
|
||||
## Typed Context Pattern
|
||||
|
||||
Replace opaque string context blobs with `T::Struct` inputs. Each field gets its own `description:` annotation in the JSON schema the LLM sees:
|
||||
|
||||
```ruby
|
||||
class NavigationContext < T::Struct
|
||||
const :workflow_hint, T.nilable(String),
|
||||
description: "Current workflow phase guidance for the agent"
|
||||
const :action_log, T::Array[String], default: [],
|
||||
description: "Compact one-line-per-action history of research steps taken"
|
||||
const :iterations_remaining, Integer,
|
||||
description: "Budget remaining. Each tool call costs 1 iteration."
|
||||
end
|
||||
|
||||
class ToolSelectionSignature < DSPy::Signature
|
||||
input do
|
||||
const :query, String
|
||||
const :context, NavigationContext # Structured, not an opaque string
|
||||
end
|
||||
|
||||
output do
|
||||
const :tool_name, String
|
||||
const :tool_args, String, description: "JSON-encoded arguments"
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
Benefits: type safety at compile time, per-field descriptions in the LLM schema, easy to test as value objects, extensible by adding `const` declarations.
|
||||
|
||||
## Schema Formats (BAML / TOON)
|
||||
|
||||
Control how DSPy describes signature structure to the LLM:
|
||||
|
||||
- **JSON Schema** (default) — Standard format, works with `structured_outputs: true`
|
||||
- **BAML** (`schema_format: :baml`) — 84% token reduction for Enhanced Prompting mode. Requires `sorbet-baml` gem.
|
||||
- **TOON** (`schema_format: :toon, data_format: :toon`) — Table-oriented format for both schemas and data. Enhanced Prompting mode only.
|
||||
|
||||
BAML and TOON apply only when `structured_outputs: false`. With `structured_outputs: true`, the provider receives JSON Schema directly.
|
||||
|
||||
## Storage System
|
||||
|
||||
Persist and reload optimized programs with `DSPy::Storage::ProgramStorage`:
|
||||
|
||||
```ruby
|
||||
storage = DSPy::Storage::ProgramStorage.new(storage_path: "./dspy_storage")
|
||||
storage.save_program(result.optimized_program, result, metadata: { optimizer: 'MIPROv2' })
|
||||
```
|
||||
|
||||
Supports checkpoint management, optimization history tracking, and import/export between environments.
|
||||
|
||||
## Rails Integration
|
||||
|
||||
### Directory Structure
|
||||
|
||||
Organize DSPy components using Rails conventions:
|
||||
|
||||
```
|
||||
app/
|
||||
entities/ # T::Struct types shared across signatures
|
||||
signatures/ # DSPy::Signature definitions
|
||||
tools/ # DSPy::Tools::Base implementations
|
||||
concerns/ # Shared tool behaviors (error handling, etc.)
|
||||
modules/ # DSPy::Module orchestrators
|
||||
services/ # Plain Ruby services that compose DSPy modules
|
||||
config/
|
||||
initializers/
|
||||
dspy.rb # DSPy + provider configuration
|
||||
feature_flags.rb # Model selection per role
|
||||
spec/
|
||||
signatures/ # Schema validation tests
|
||||
tools/ # Tool unit tests
|
||||
modules/ # Integration tests with VCR
|
||||
vcr_cassettes/ # Recorded HTTP interactions
|
||||
```
|
||||
|
||||
### Initializer
|
||||
|
||||
```ruby
|
||||
# config/initializers/dspy.rb
|
||||
Rails.application.config.after_initialize do
|
||||
next if Rails.env.test? && ENV["DSPY_ENABLE_IN_TEST"].blank?
|
||||
|
||||
RubyLLM.configure do |config|
|
||||
config.gemini_api_key = ENV["GEMINI_API_KEY"] if ENV["GEMINI_API_KEY"].present?
|
||||
config.anthropic_api_key = ENV["ANTHROPIC_API_KEY"] if ENV["ANTHROPIC_API_KEY"].present?
|
||||
config.openai_api_key = ENV["OPENAI_API_KEY"] if ENV["OPENAI_API_KEY"].present?
|
||||
end
|
||||
|
||||
model = ENV.fetch("DSPY_MODEL", "ruby_llm/gemini-2.5-flash")
|
||||
DSPy.configure do |config|
|
||||
config.lm = DSPy::LM.new(model, structured_outputs: true)
|
||||
config.logger = Rails.logger
|
||||
end
|
||||
|
||||
# Langfuse observability (optional)
|
||||
if ENV["LANGFUSE_PUBLIC_KEY"].present? && ENV["LANGFUSE_SECRET_KEY"].present?
|
||||
DSPy::Observability.configure!
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
### Feature-Flagged Model Selection
|
||||
|
||||
Use different models for different roles (fast/cheap for classification, powerful for synthesis):
|
||||
|
||||
```ruby
|
||||
# config/initializers/feature_flags.rb
|
||||
module FeatureFlags
|
||||
SELECTOR_MODEL = ENV.fetch("DSPY_SELECTOR_MODEL", "ruby_llm/gemini-2.5-flash-lite")
|
||||
SYNTHESIZER_MODEL = ENV.fetch("DSPY_SYNTHESIZER_MODEL", "ruby_llm/gemini-2.5-flash")
|
||||
end
|
||||
```
|
||||
|
||||
Then override per-tool or per-predictor:
|
||||
|
||||
```ruby
|
||||
class ClassifyTool < DSPy::Tools::Base
|
||||
def call(query:)
|
||||
predictor = DSPy::Predict.new(ClassifyQuery)
|
||||
predictor.configure { |c| c.lm = DSPy::LM.new(FeatureFlags::SELECTOR_MODEL, structured_outputs: true) }
|
||||
predictor.call(query: query)
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
## Schema-Driven Signatures
|
||||
|
||||
**Prefer typed schemas over string descriptions.** Let the type system communicate structure to the LLM rather than prose in the signature description.
|
||||
|
||||
### Entities as Shared Types
|
||||
|
||||
Define reusable `T::Struct` and `T::Enum` types in `app/entities/` and reference them across signatures:
|
||||
|
||||
```ruby
|
||||
# app/entities/search_strategy.rb
|
||||
class SearchStrategy < T::Enum
|
||||
enums do
|
||||
SingleSearch = new("single_search")
|
||||
DateDecomposition = new("date_decomposition")
|
||||
end
|
||||
end
|
||||
|
||||
# app/entities/scored_item.rb
|
||||
class ScoredItem < T::Struct
|
||||
const :id, String
|
||||
const :score, Float, description: "Relevance score 0.0-1.0"
|
||||
const :verdict, String, description: "relevant, maybe, or irrelevant"
|
||||
const :reason, String, default: ""
|
||||
end
|
||||
```
|
||||
|
||||
### Schema vs Description: When to Use Each
|
||||
|
||||
**Use schemas (T::Struct/T::Enum)** for:
|
||||
- Multi-field outputs with specific types
|
||||
- Enums with defined values the LLM must pick from
|
||||
- Nested structures, arrays of typed objects
|
||||
- Outputs consumed by code (not displayed to users)
|
||||
|
||||
**Use string descriptions** for:
|
||||
- Simple single-field outputs where the type is `String`
|
||||
- Natural language generation (summaries, answers)
|
||||
- Fields where constraint guidance helps (e.g., `description: "YYYY-MM-DD format"`)
|
||||
|
||||
**Rule of thumb**: If you'd write a `case` statement on the output, it should be a `T::Enum`. If you'd call `.each` on it, it should be `T::Array[SomeStruct]`.
|
||||
|
||||
## Tool Patterns
|
||||
|
||||
### Tools That Wrap Predictions
|
||||
|
||||
A common pattern: tools encapsulate a DSPy prediction, adding error handling, model selection, and serialization:
|
||||
|
||||
```ruby
|
||||
class RerankTool < DSPy::Tools::Base
|
||||
tool_name "rerank"
|
||||
tool_description "Score and rank search results by relevance"
|
||||
|
||||
MAX_ITEMS = 200
|
||||
MIN_ITEMS_FOR_LLM = 5
|
||||
|
||||
sig { params(query: String, items: T::Array[T::Hash[Symbol, T.untyped]]).returns(T::Hash[Symbol, T.untyped]) }
|
||||
def call(query:, items: [])
|
||||
return { scored_items: items, reranked: false } if items.size < MIN_ITEMS_FOR_LLM
|
||||
|
||||
capped_items = items.first(MAX_ITEMS)
|
||||
predictor = DSPy::Predict.new(RerankSignature)
|
||||
predictor.configure { |c| c.lm = DSPy::LM.new(FeatureFlags::SYNTHESIZER_MODEL, structured_outputs: true) }
|
||||
|
||||
result = predictor.call(query: query, items: capped_items)
|
||||
{ scored_items: result.scored_items, reranked: true }
|
||||
rescue => e
|
||||
Rails.logger.warn "[RerankTool] LLM rerank failed: #{e.message}"
|
||||
{ error: "Rerank failed: #{e.message}", scored_items: items, reranked: false }
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
**Key patterns:**
|
||||
- Short-circuit LLM calls when unnecessary (small data, trivial cases)
|
||||
- Cap input size to prevent token overflow
|
||||
- Per-tool model selection via `configure`
|
||||
- Graceful error handling with fallback data
|
||||
|
||||
### Error Handling Concern
|
||||
|
||||
```ruby
|
||||
module ErrorHandling
|
||||
extend ActiveSupport::Concern
|
||||
|
||||
private
|
||||
|
||||
def safe_predict(signature_class, **inputs)
|
||||
predictor = DSPy::Predict.new(signature_class)
|
||||
yield predictor if block_given?
|
||||
predictor.call(**inputs)
|
||||
rescue Faraday::Error, Net::HTTPError => e
|
||||
Rails.logger.error "[#{self.class.name}] API error: #{e.message}"
|
||||
nil
|
||||
rescue JSON::ParserError => e
|
||||
Rails.logger.error "[#{self.class.name}] Invalid LLM output: #{e.message}"
|
||||
nil
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
## Observability
|
||||
|
||||
### Tracing with DSPy::Context
|
||||
|
||||
Wrap operations in spans for Langfuse/OpenTelemetry visibility:
|
||||
|
||||
```ruby
|
||||
result = DSPy::Context.with_span(
|
||||
operation: "tool_selector.select",
|
||||
"dspy.module" => "ToolSelector",
|
||||
"tool_selector.tools" => tool_names.join(",")
|
||||
) do
|
||||
@predictor.call(query: query, context: context, available_tools: schemas)
|
||||
end
|
||||
```
|
||||
|
||||
### Setup for Langfuse
|
||||
|
||||
```ruby
|
||||
# Gemfile
|
||||
gem 'dspy-o11y'
|
||||
gem 'dspy-o11y-langfuse'
|
||||
|
||||
# .env
|
||||
LANGFUSE_PUBLIC_KEY=pk-...
|
||||
LANGFUSE_SECRET_KEY=sk-...
|
||||
DSPY_TELEMETRY_BATCH_SIZE=5
|
||||
```
|
||||
|
||||
Every `DSPy::Predict`, `DSPy::ReAct`, and tool call is automatically traced when observability is configured.
|
||||
|
||||
### Score Reporting
|
||||
|
||||
Report evaluation scores to Langfuse:
|
||||
|
||||
```ruby
|
||||
DSPy.score(name: "relevance", value: 0.85, trace_id: current_trace_id)
|
||||
```
|
||||
|
||||
## Testing
|
||||
|
||||
### VCR Setup for Rails
|
||||
|
||||
```ruby
|
||||
VCR.configure do |config|
|
||||
config.cassette_library_dir = "spec/vcr_cassettes"
|
||||
config.hook_into :webmock
|
||||
config.configure_rspec_metadata!
|
||||
config.filter_sensitive_data('<GEMINI_API_KEY>') { ENV['GEMINI_API_KEY'] }
|
||||
config.filter_sensitive_data('<OPENAI_API_KEY>') { ENV['OPENAI_API_KEY'] }
|
||||
end
|
||||
```
|
||||
|
||||
### Signature Schema Tests
|
||||
|
||||
Test that signatures produce valid schemas without calling any LLM:
|
||||
|
||||
```ruby
|
||||
RSpec.describe ClassifyResearchQuery do
|
||||
it "has required input fields" do
|
||||
schema = described_class.input_json_schema
|
||||
expect(schema[:required]).to include("query")
|
||||
end
|
||||
|
||||
it "has typed output fields" do
|
||||
schema = described_class.output_json_schema
|
||||
expect(schema[:properties]).to have_key(:search_strategy)
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
### Tool Tests with Mocked Predictions
|
||||
|
||||
```ruby
|
||||
RSpec.describe RerankTool do
|
||||
let(:tool) { described_class.new }
|
||||
|
||||
it "skips LLM for small result sets" do
|
||||
expect(DSPy::Predict).not_to receive(:new)
|
||||
result = tool.call(query: "test", items: [{ id: "1" }])
|
||||
expect(result[:reranked]).to be false
|
||||
end
|
||||
|
||||
it "calls LLM for large result sets", :vcr do
|
||||
items = 10.times.map { |i| { id: i.to_s, title: "Item #{i}" } }
|
||||
result = tool.call(query: "relevant items", items: items)
|
||||
expect(result[:reranked]).to be true
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
## Resources
|
||||
|
||||
- `references/core-concepts.md` — Signatures, modules, predictors, type system deep-dive
|
||||
- `references/toolsets.md` — Tools::Base, Tools::Toolset DSL, type safety, testing
|
||||
- `references/providers.md` — Provider adapters, RubyLLM, fiber-local LM context, compatibility matrix
|
||||
- `references/optimization.md` — MIPROv2, GEPA, evaluation framework, storage system
|
||||
- `references/observability.md` — Event system, dspy-o11y gems, Langfuse, score reporting
|
||||
- `assets/signature-template.rb` — Signature scaffold with T::Enum, Date/Time, defaults, union types
|
||||
- `assets/module-template.rb` — Module scaffold with .call(), lifecycle callbacks, fiber-local LM
|
||||
- `assets/config-template.rb` — Rails initializer with RubyLLM, observability, feature flags
|
||||
|
||||
## Key URLs
|
||||
|
||||
- Homepage: https://oss.vicente.services/dspy.rb/
|
||||
- GitHub: https://github.com/vicentereig/dspy.rb
|
||||
- Documentation: https://oss.vicente.services/dspy.rb/getting-started/
|
||||
|
||||
## Guidelines for Claude
|
||||
|
||||
When helping users with DSPy.rb:
|
||||
|
||||
1. **Schema over prose** — Define output structure with `T::Struct` and `T::Enum` types, not string descriptions
|
||||
2. **Entities in `app/entities/`** — Extract shared types so signatures stay thin
|
||||
3. **Per-tool model selection** — Use `predictor.configure { |c| c.lm = ... }` to pick the right model per task
|
||||
4. **Short-circuit LLM calls** — Skip the LLM for trivial cases (small data, cached results)
|
||||
5. **Cap input sizes** — Prevent token overflow by limiting array sizes before sending to LLM
|
||||
6. **Test schemas without LLM** — Validate `input_json_schema` and `output_json_schema` in unit tests
|
||||
7. **VCR for integration tests** — Record real HTTP interactions, never mock LLM responses by hand
|
||||
8. **Trace with spans** — Wrap tool calls in `DSPy::Context.with_span` for observability
|
||||
9. **Graceful degradation** — Always rescue LLM errors and return fallback data
|
||||
|
||||
### Signature Best Practices
|
||||
|
||||
**Keep description concise** — The signature `description` should state the goal, not the field details:
|
||||
|
||||
```ruby
|
||||
# Good — concise goal
|
||||
class ParseOutline < DSPy::Signature
|
||||
description 'Extract block-level structure from HTML as a flat list of skeleton sections.'
|
||||
|
||||
input do
|
||||
const :html, String, description: 'Raw HTML to parse'
|
||||
end
|
||||
|
||||
output do
|
||||
const :sections, T::Array[Section], description: 'Block elements: headings, paragraphs, code blocks, lists'
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
**Use defaults over nilable arrays** — For OpenAI structured outputs compatibility:
|
||||
|
||||
```ruby
|
||||
# Good — works with OpenAI structured outputs
|
||||
class ASTNode < T::Struct
|
||||
const :children, T::Array[ASTNode], default: []
|
||||
end
|
||||
```
|
||||
|
||||
### Recursive Types with `$defs`
|
||||
|
||||
DSPy.rb supports recursive types in structured outputs using JSON Schema `$defs`:
|
||||
|
||||
```ruby
|
||||
class TreeNode < T::Struct
|
||||
const :value, String
|
||||
const :children, T::Array[TreeNode], default: [] # Self-reference
|
||||
end
|
||||
```
|
||||
|
||||
The schema generator automatically creates `#/$defs/TreeNode` references for recursive types, compatible with OpenAI and Gemini structured outputs.
|
||||
|
||||
### Field Descriptions for T::Struct
|
||||
|
||||
DSPy.rb extends T::Struct to support field-level `description:` kwargs that flow to JSON Schema:
|
||||
|
||||
```ruby
|
||||
class ASTNode < T::Struct
|
||||
const :node_type, NodeType, description: 'The type of node (heading, paragraph, etc.)'
|
||||
const :text, String, default: "", description: 'Text content of the node'
|
||||
const :level, Integer, default: 0 # No description — field is self-explanatory
|
||||
const :children, T::Array[ASTNode], default: []
|
||||
end
|
||||
```
|
||||
|
||||
**When to use field descriptions**: complex field semantics, enum-like strings, constrained values, nested structs with ambiguous names. **When to skip**: self-explanatory fields like `name`, `id`, `url`, or boolean flags.
|
||||
|
||||
## Version
|
||||
|
||||
Current: 0.34.3
|
||||
@@ -1,187 +0,0 @@
|
||||
# frozen_string_literal: true
|
||||
|
||||
# =============================================================================
|
||||
# DSPy.rb Configuration Template — v0.34.3 API
|
||||
#
|
||||
# Rails initializer patterns for DSPy.rb with RubyLLM, observability,
|
||||
# and feature-flagged model selection.
|
||||
#
|
||||
# Key patterns:
|
||||
# - Use after_initialize for Rails setup
|
||||
# - Use dspy-ruby_llm for multi-provider routing
|
||||
# - Use structured_outputs: true for reliable parsing
|
||||
# - Use dspy-o11y + dspy-o11y-langfuse for observability
|
||||
# - Use ENV-based feature flags for model selection
|
||||
# =============================================================================
|
||||
|
||||
# =============================================================================
|
||||
# Gemfile Dependencies
|
||||
# =============================================================================
|
||||
#
|
||||
# # Core
|
||||
# gem 'dspy'
|
||||
#
|
||||
# # Provider adapter (choose one strategy):
|
||||
#
|
||||
# # Strategy A: Unified adapter via RubyLLM (recommended)
|
||||
# gem 'dspy-ruby_llm'
|
||||
# gem 'ruby_llm'
|
||||
#
|
||||
# # Strategy B: Per-provider adapters (direct SDK access)
|
||||
# gem 'dspy-openai' # OpenAI, OpenRouter, Ollama
|
||||
# gem 'dspy-anthropic' # Claude
|
||||
# gem 'dspy-gemini' # Gemini
|
||||
#
|
||||
# # Observability (optional)
|
||||
# gem 'dspy-o11y'
|
||||
# gem 'dspy-o11y-langfuse'
|
||||
#
|
||||
# # Optimization (optional)
|
||||
# gem 'dspy-miprov2' # MIPROv2 optimizer
|
||||
# gem 'dspy-gepa' # GEPA optimizer
|
||||
#
|
||||
# # Schema formats (optional)
|
||||
# gem 'sorbet-baml' # BAML schema format (84% token reduction)
|
||||
|
||||
# =============================================================================
|
||||
# Rails Initializer — config/initializers/dspy.rb
|
||||
# =============================================================================
|
||||
|
||||
Rails.application.config.after_initialize do
|
||||
# Skip in test unless explicitly enabled
|
||||
next if Rails.env.test? && ENV["DSPY_ENABLE_IN_TEST"].blank?
|
||||
|
||||
# Configure RubyLLM provider credentials
|
||||
RubyLLM.configure do |config|
|
||||
config.gemini_api_key = ENV["GEMINI_API_KEY"] if ENV["GEMINI_API_KEY"].present?
|
||||
config.anthropic_api_key = ENV["ANTHROPIC_API_KEY"] if ENV["ANTHROPIC_API_KEY"].present?
|
||||
config.openai_api_key = ENV["OPENAI_API_KEY"] if ENV["OPENAI_API_KEY"].present?
|
||||
end
|
||||
|
||||
# Configure DSPy with unified RubyLLM adapter
|
||||
model = ENV.fetch("DSPY_MODEL", "ruby_llm/gemini-2.5-flash")
|
||||
DSPy.configure do |config|
|
||||
config.lm = DSPy::LM.new(model, structured_outputs: true)
|
||||
config.logger = Rails.logger
|
||||
end
|
||||
|
||||
# Enable Langfuse observability (optional)
|
||||
if ENV["LANGFUSE_PUBLIC_KEY"].present? && ENV["LANGFUSE_SECRET_KEY"].present?
|
||||
DSPy::Observability.configure!
|
||||
end
|
||||
end
|
||||
|
||||
# =============================================================================
|
||||
# Feature Flags — config/initializers/feature_flags.rb
|
||||
# =============================================================================
|
||||
|
||||
# Use different models for different roles:
|
||||
# - Fast/cheap for classification, routing, simple tasks
|
||||
# - Powerful for synthesis, reasoning, complex analysis
|
||||
|
||||
module FeatureFlags
|
||||
SELECTOR_MODEL = ENV.fetch("DSPY_SELECTOR_MODEL", "ruby_llm/gemini-2.5-flash-lite")
|
||||
SYNTHESIZER_MODEL = ENV.fetch("DSPY_SYNTHESIZER_MODEL", "ruby_llm/gemini-2.5-flash")
|
||||
REASONING_MODEL = ENV.fetch("DSPY_REASONING_MODEL", "ruby_llm/claude-sonnet-4-20250514")
|
||||
end
|
||||
|
||||
# Usage in tools/modules:
|
||||
#
|
||||
# class ClassifyTool < DSPy::Tools::Base
|
||||
# def call(query:)
|
||||
# predictor = DSPy::Predict.new(ClassifySignature)
|
||||
# predictor.configure { |c| c.lm = DSPy::LM.new(FeatureFlags::SELECTOR_MODEL, structured_outputs: true) }
|
||||
# predictor.call(query: query)
|
||||
# end
|
||||
# end
|
||||
|
||||
# =============================================================================
|
||||
# Environment Variables — .env
|
||||
# =============================================================================
|
||||
#
|
||||
# # Provider API keys (set the ones you need)
|
||||
# GEMINI_API_KEY=...
|
||||
# ANTHROPIC_API_KEY=...
|
||||
# OPENAI_API_KEY=...
|
||||
#
|
||||
# # DSPy model configuration
|
||||
# DSPY_MODEL=ruby_llm/gemini-2.5-flash
|
||||
# DSPY_SELECTOR_MODEL=ruby_llm/gemini-2.5-flash-lite
|
||||
# DSPY_SYNTHESIZER_MODEL=ruby_llm/gemini-2.5-flash
|
||||
# DSPY_REASONING_MODEL=ruby_llm/claude-sonnet-4-20250514
|
||||
#
|
||||
# # Langfuse observability (optional)
|
||||
# LANGFUSE_PUBLIC_KEY=pk-...
|
||||
# LANGFUSE_SECRET_KEY=sk-...
|
||||
# DSPY_TELEMETRY_BATCH_SIZE=5
|
||||
#
|
||||
# # Test environment
|
||||
# DSPY_ENABLE_IN_TEST=1 # Set to enable DSPy in test env
|
||||
|
||||
# =============================================================================
|
||||
# Per-Provider Configuration (without RubyLLM)
|
||||
# =============================================================================
|
||||
|
||||
# OpenAI (dspy-openai gem)
|
||||
# DSPy.configure do |c|
|
||||
# c.lm = DSPy::LM.new('openai/gpt-4o-mini', api_key: ENV['OPENAI_API_KEY'])
|
||||
# end
|
||||
|
||||
# Anthropic (dspy-anthropic gem)
|
||||
# DSPy.configure do |c|
|
||||
# c.lm = DSPy::LM.new('anthropic/claude-sonnet-4-20250514', api_key: ENV['ANTHROPIC_API_KEY'])
|
||||
# end
|
||||
|
||||
# Gemini (dspy-gemini gem)
|
||||
# DSPy.configure do |c|
|
||||
# c.lm = DSPy::LM.new('gemini/gemini-2.5-flash', api_key: ENV['GEMINI_API_KEY'])
|
||||
# end
|
||||
|
||||
# Ollama (dspy-openai gem, local models)
|
||||
# DSPy.configure do |c|
|
||||
# c.lm = DSPy::LM.new('ollama/llama3.2', base_url: 'http://localhost:11434')
|
||||
# end
|
||||
|
||||
# OpenRouter (dspy-openai gem, 200+ models)
|
||||
# DSPy.configure do |c|
|
||||
# c.lm = DSPy::LM.new('openrouter/anthropic/claude-3.5-sonnet',
|
||||
# api_key: ENV['OPENROUTER_API_KEY'],
|
||||
# base_url: 'https://openrouter.ai/api/v1')
|
||||
# end
|
||||
|
||||
# =============================================================================
|
||||
# VCR Test Configuration — spec/support/dspy.rb
|
||||
# =============================================================================
|
||||
|
||||
# VCR.configure do |config|
|
||||
# config.cassette_library_dir = "spec/vcr_cassettes"
|
||||
# config.hook_into :webmock
|
||||
# config.configure_rspec_metadata!
|
||||
# config.filter_sensitive_data('<GEMINI_API_KEY>') { ENV['GEMINI_API_KEY'] }
|
||||
# config.filter_sensitive_data('<OPENAI_API_KEY>') { ENV['OPENAI_API_KEY'] }
|
||||
# config.filter_sensitive_data('<ANTHROPIC_API_KEY>') { ENV['ANTHROPIC_API_KEY'] }
|
||||
# end
|
||||
|
||||
# =============================================================================
|
||||
# Schema Format Configuration (optional)
|
||||
# =============================================================================
|
||||
|
||||
# BAML schema format — 84% token reduction for Enhanced Prompting mode
|
||||
# DSPy.configure do |c|
|
||||
# c.lm = DSPy::LM.new('openai/gpt-4o-mini',
|
||||
# api_key: ENV['OPENAI_API_KEY'],
|
||||
# schema_format: :baml # Requires sorbet-baml gem
|
||||
# )
|
||||
# end
|
||||
|
||||
# TOON schema + data format — table-oriented format
|
||||
# DSPy.configure do |c|
|
||||
# c.lm = DSPy::LM.new('openai/gpt-4o-mini',
|
||||
# api_key: ENV['OPENAI_API_KEY'],
|
||||
# schema_format: :toon, # How DSPy describes the signature
|
||||
# data_format: :toon # How inputs/outputs are rendered in prompts
|
||||
# )
|
||||
# end
|
||||
#
|
||||
# Note: BAML and TOON apply only when structured_outputs: false.
|
||||
# With structured_outputs: true, the provider receives JSON Schema directly.
|
||||
@@ -1,300 +0,0 @@
|
||||
# frozen_string_literal: true
|
||||
|
||||
# =============================================================================
|
||||
# DSPy.rb Module Template — v0.34.3 API
|
||||
#
|
||||
# Modules orchestrate predictors, tools, and business logic.
|
||||
#
|
||||
# Key patterns:
|
||||
# - Use .call() to invoke (not .forward())
|
||||
# - Access results with result.field (not result[:field])
|
||||
# - Use DSPy::Tools::Base for tools (not DSPy::Tool)
|
||||
# - Use lifecycle callbacks (before/around/after) for cross-cutting concerns
|
||||
# - Use DSPy.with_lm for temporary model overrides
|
||||
# - Use configure_predictor for fine-grained agent control
|
||||
# =============================================================================
|
||||
|
||||
# --- Basic Module ---
|
||||
|
||||
class BasicClassifier < DSPy::Module
|
||||
def initialize
|
||||
super
|
||||
@predictor = DSPy::Predict.new(ClassificationSignature)
|
||||
end
|
||||
|
||||
def forward(text:)
|
||||
@predictor.call(text: text)
|
||||
end
|
||||
end
|
||||
|
||||
# Usage:
|
||||
# classifier = BasicClassifier.new
|
||||
# result = classifier.call(text: "This is a test")
|
||||
# result.category # => "technical"
|
||||
# result.confidence # => 0.95
|
||||
|
||||
# --- Module with Chain of Thought ---
|
||||
|
||||
class ReasoningClassifier < DSPy::Module
|
||||
def initialize
|
||||
super
|
||||
@predictor = DSPy::ChainOfThought.new(ClassificationSignature)
|
||||
end
|
||||
|
||||
def forward(text:)
|
||||
result = @predictor.call(text: text)
|
||||
# ChainOfThought adds result.reasoning automatically
|
||||
result
|
||||
end
|
||||
end
|
||||
|
||||
# --- Module with Lifecycle Callbacks ---
|
||||
|
||||
class InstrumentedModule < DSPy::Module
|
||||
before :setup_metrics
|
||||
around :manage_context
|
||||
after :log_completion
|
||||
|
||||
def initialize
|
||||
super
|
||||
@predictor = DSPy::Predict.new(AnalysisSignature)
|
||||
@start_time = nil
|
||||
end
|
||||
|
||||
def forward(query:)
|
||||
@predictor.call(query: query)
|
||||
end
|
||||
|
||||
private
|
||||
|
||||
# Runs before forward
|
||||
def setup_metrics
|
||||
@start_time = Time.now
|
||||
Rails.logger.info "Starting prediction"
|
||||
end
|
||||
|
||||
# Wraps forward — must call yield
|
||||
def manage_context
|
||||
load_user_context
|
||||
result = yield
|
||||
save_updated_context(result)
|
||||
result
|
||||
end
|
||||
|
||||
# Runs after forward completes
|
||||
def log_completion
|
||||
duration = Time.now - @start_time
|
||||
Rails.logger.info "Prediction completed in #{duration}s"
|
||||
end
|
||||
|
||||
def load_user_context = nil
|
||||
def save_updated_context(_result) = nil
|
||||
end
|
||||
|
||||
# Execution order: before → around (before yield) → forward → around (after yield) → after
|
||||
# Callbacks are inherited from parent classes and execute in registration order.
|
||||
|
||||
# --- Module with Tools ---
|
||||
|
||||
class SearchTool < DSPy::Tools::Base
|
||||
tool_name "search"
|
||||
tool_description "Search for information by query"
|
||||
|
||||
sig { params(query: String, max_results: Integer).returns(T::Array[T::Hash[Symbol, String]]) }
|
||||
def call(query:, max_results: 5)
|
||||
# Implementation here
|
||||
[{ title: "Result 1", url: "https://example.com" }]
|
||||
end
|
||||
end
|
||||
|
||||
class FinishTool < DSPy::Tools::Base
|
||||
tool_name "finish"
|
||||
tool_description "Submit the final answer"
|
||||
|
||||
sig { params(answer: String).returns(String) }
|
||||
def call(answer:)
|
||||
answer
|
||||
end
|
||||
end
|
||||
|
||||
class ResearchAgent < DSPy::Module
|
||||
def initialize
|
||||
super
|
||||
tools = [SearchTool.new, FinishTool.new]
|
||||
@agent = DSPy::ReAct.new(
|
||||
ResearchSignature,
|
||||
tools: tools,
|
||||
max_iterations: 5
|
||||
)
|
||||
end
|
||||
|
||||
def forward(question:)
|
||||
@agent.call(question: question)
|
||||
end
|
||||
end
|
||||
|
||||
# --- Module with Per-Task Model Selection ---
|
||||
|
||||
class SmartRouter < DSPy::Module
|
||||
def initialize
|
||||
super
|
||||
@classifier = DSPy::Predict.new(RouteSignature)
|
||||
@analyzer = DSPy::ChainOfThought.new(AnalysisSignature)
|
||||
end
|
||||
|
||||
def forward(text:)
|
||||
# Use fast model for classification
|
||||
DSPy.with_lm(fast_model) do
|
||||
route = @classifier.call(text: text)
|
||||
|
||||
if route.requires_deep_analysis
|
||||
# Switch to powerful model for analysis
|
||||
DSPy.with_lm(powerful_model) do
|
||||
@analyzer.call(text: text)
|
||||
end
|
||||
else
|
||||
route
|
||||
end
|
||||
end
|
||||
end
|
||||
|
||||
private
|
||||
|
||||
def fast_model
|
||||
@fast_model ||= DSPy::LM.new(
|
||||
ENV.fetch("DSPY_SELECTOR_MODEL", "ruby_llm/gemini-2.5-flash-lite"),
|
||||
structured_outputs: true
|
||||
)
|
||||
end
|
||||
|
||||
def powerful_model
|
||||
@powerful_model ||= DSPy::LM.new(
|
||||
ENV.fetch("DSPY_SYNTHESIZER_MODEL", "ruby_llm/gemini-2.5-flash"),
|
||||
structured_outputs: true
|
||||
)
|
||||
end
|
||||
end
|
||||
|
||||
# --- Module with configure_predictor ---
|
||||
|
||||
class ConfiguredAgent < DSPy::Module
|
||||
def initialize
|
||||
super
|
||||
tools = [SearchTool.new, FinishTool.new]
|
||||
@agent = DSPy::ReAct.new(ResearchSignature, tools: tools)
|
||||
|
||||
# Set default model for all internal predictors
|
||||
@agent.configure { |c| c.lm = DSPy::LM.new('ruby_llm/gemini-2.5-flash', structured_outputs: true) }
|
||||
|
||||
# Override specific predictor with a more capable model
|
||||
@agent.configure_predictor('thought_generator') do |c|
|
||||
c.lm = DSPy::LM.new('ruby_llm/claude-sonnet-4-20250514', structured_outputs: true)
|
||||
end
|
||||
end
|
||||
|
||||
def forward(question:)
|
||||
@agent.call(question: question)
|
||||
end
|
||||
end
|
||||
|
||||
# Available internal predictors by agent type:
|
||||
# DSPy::ReAct → thought_generator, observation_processor
|
||||
# DSPy::CodeAct → code_generator, observation_processor
|
||||
# DSPy::DeepSearch → seed_predictor, search_predictor, reader_predictor, reason_predictor
|
||||
|
||||
# --- Module with Event Subscriptions ---
|
||||
|
||||
class TokenTrackingModule < DSPy::Module
|
||||
subscribe 'lm.tokens', :track_tokens, scope: :descendants
|
||||
|
||||
def initialize
|
||||
super
|
||||
@predictor = DSPy::Predict.new(AnalysisSignature)
|
||||
@total_tokens = 0
|
||||
end
|
||||
|
||||
def forward(query:)
|
||||
@predictor.call(query: query)
|
||||
end
|
||||
|
||||
def track_tokens(_event, attrs)
|
||||
@total_tokens += attrs.fetch(:total_tokens, 0)
|
||||
end
|
||||
|
||||
def token_usage
|
||||
@total_tokens
|
||||
end
|
||||
end
|
||||
|
||||
# Module-scoped subscriptions automatically scope to the module instance and descendants.
|
||||
# Use scope: :self_only to restrict delivery to the module itself (ignoring children).
|
||||
|
||||
# --- Tool That Wraps a Prediction ---
|
||||
|
||||
class RerankTool < DSPy::Tools::Base
|
||||
tool_name "rerank"
|
||||
tool_description "Score and rank search results by relevance"
|
||||
|
||||
MAX_ITEMS = 200
|
||||
MIN_ITEMS_FOR_LLM = 5
|
||||
|
||||
sig { params(query: String, items: T::Array[T::Hash[Symbol, T.untyped]]).returns(T::Hash[Symbol, T.untyped]) }
|
||||
def call(query:, items: [])
|
||||
# Short-circuit: skip LLM for small sets
|
||||
return { scored_items: items, reranked: false } if items.size < MIN_ITEMS_FOR_LLM
|
||||
|
||||
# Cap to prevent token overflow
|
||||
capped_items = items.first(MAX_ITEMS)
|
||||
|
||||
predictor = DSPy::Predict.new(RerankSignature)
|
||||
predictor.configure { |c| c.lm = DSPy::LM.new("ruby_llm/gemini-2.5-flash", structured_outputs: true) }
|
||||
|
||||
result = predictor.call(query: query, items: capped_items)
|
||||
{ scored_items: result.scored_items, reranked: true }
|
||||
rescue => e
|
||||
Rails.logger.warn "[RerankTool] LLM rerank failed: #{e.message}"
|
||||
{ error: "Rerank failed: #{e.message}", scored_items: items, reranked: false }
|
||||
end
|
||||
end
|
||||
|
||||
# Key patterns for tools wrapping predictions:
|
||||
# - Short-circuit LLM calls when unnecessary (small data, trivial cases)
|
||||
# - Cap input size to prevent token overflow
|
||||
# - Per-tool model selection via configure
|
||||
# - Graceful error handling with fallback data
|
||||
|
||||
# --- Multi-Step Pipeline ---
|
||||
|
||||
class AnalysisPipeline < DSPy::Module
|
||||
def initialize
|
||||
super
|
||||
@classifier = DSPy::Predict.new(ClassifySignature)
|
||||
@analyzer = DSPy::ChainOfThought.new(AnalyzeSignature)
|
||||
@summarizer = DSPy::Predict.new(SummarizeSignature)
|
||||
end
|
||||
|
||||
def forward(text:)
|
||||
classification = @classifier.call(text: text)
|
||||
analysis = @analyzer.call(text: text, category: classification.category)
|
||||
@summarizer.call(analysis: analysis.reasoning, category: classification.category)
|
||||
end
|
||||
end
|
||||
|
||||
# --- Observability with Spans ---
|
||||
|
||||
class TracedModule < DSPy::Module
|
||||
def initialize
|
||||
super
|
||||
@predictor = DSPy::Predict.new(AnalysisSignature)
|
||||
end
|
||||
|
||||
def forward(query:)
|
||||
DSPy::Context.with_span(
|
||||
operation: "traced_module.analyze",
|
||||
"dspy.module" => self.class.name,
|
||||
"query.length" => query.length.to_s
|
||||
) do
|
||||
@predictor.call(query: query)
|
||||
end
|
||||
end
|
||||
end
|
||||
@@ -1,221 +0,0 @@
|
||||
# frozen_string_literal: true
|
||||
|
||||
# =============================================================================
|
||||
# DSPy.rb Signature Template — v0.34.3 API
|
||||
#
|
||||
# Signatures define the interface between your application and LLMs.
|
||||
# They specify inputs, outputs, and task descriptions using Sorbet types.
|
||||
#
|
||||
# Key patterns:
|
||||
# - Use T::Enum classes for controlled outputs (not inline T.enum([...]))
|
||||
# - Use description: kwarg on fields to guide the LLM
|
||||
# - Use default values for optional fields
|
||||
# - Use Date/DateTime/Time for temporal data (auto-converted)
|
||||
# - Access results with result.field (not result[:field])
|
||||
# - Invoke with predictor.call() (not predictor.forward())
|
||||
# =============================================================================
|
||||
|
||||
# --- Basic Signature ---
|
||||
|
||||
class SentimentAnalysis < DSPy::Signature
|
||||
description "Analyze sentiment of text"
|
||||
|
||||
class Sentiment < T::Enum
|
||||
enums do
|
||||
Positive = new('positive')
|
||||
Negative = new('negative')
|
||||
Neutral = new('neutral')
|
||||
end
|
||||
end
|
||||
|
||||
input do
|
||||
const :text, String
|
||||
end
|
||||
|
||||
output do
|
||||
const :sentiment, Sentiment
|
||||
const :score, Float, description: "Confidence score from 0.0 to 1.0"
|
||||
end
|
||||
end
|
||||
|
||||
# Usage:
|
||||
# predictor = DSPy::Predict.new(SentimentAnalysis)
|
||||
# result = predictor.call(text: "This product is amazing!")
|
||||
# result.sentiment # => Sentiment::Positive
|
||||
# result.score # => 0.92
|
||||
|
||||
# --- Signature with Date/Time Types ---
|
||||
|
||||
class EventScheduler < DSPy::Signature
|
||||
description "Schedule events based on requirements"
|
||||
|
||||
input do
|
||||
const :event_name, String
|
||||
const :start_date, Date # ISO 8601: YYYY-MM-DD
|
||||
const :end_date, T.nilable(Date) # Optional date
|
||||
const :preferred_time, DateTime # ISO 8601 with timezone
|
||||
const :deadline, Time # Stored as UTC
|
||||
end
|
||||
|
||||
output do
|
||||
const :scheduled_date, Date # LLM returns ISO string, auto-converted
|
||||
const :event_datetime, DateTime # Preserves timezone
|
||||
const :created_at, Time # Converted to UTC
|
||||
end
|
||||
end
|
||||
|
||||
# Date/Time format handling:
|
||||
# Date → ISO 8601 (YYYY-MM-DD)
|
||||
# DateTime → ISO 8601 with timezone (YYYY-MM-DDTHH:MM:SS+00:00)
|
||||
# Time → ISO 8601, automatically converted to UTC
|
||||
|
||||
# --- Signature with Default Values ---
|
||||
|
||||
class SmartSearch < DSPy::Signature
|
||||
description "Search with intelligent defaults"
|
||||
|
||||
input do
|
||||
const :query, String
|
||||
const :max_results, Integer, default: 10
|
||||
const :language, String, default: "English"
|
||||
const :include_metadata, T::Boolean, default: false
|
||||
end
|
||||
|
||||
output do
|
||||
const :results, T::Array[String]
|
||||
const :total_found, Integer
|
||||
const :search_time_ms, Float, default: 0.0 # Fallback if LLM omits
|
||||
const :cached, T::Boolean, default: false
|
||||
end
|
||||
end
|
||||
|
||||
# Input defaults reduce boilerplate:
|
||||
# search = DSPy::Predict.new(SmartSearch)
|
||||
# result = search.call(query: "Ruby programming")
|
||||
# # max_results=10, language="English", include_metadata=false are applied
|
||||
|
||||
# --- Signature with Nested Structs and Field Descriptions ---
|
||||
|
||||
class EntityExtraction < DSPy::Signature
|
||||
description "Extract named entities from text"
|
||||
|
||||
class EntityType < T::Enum
|
||||
enums do
|
||||
Person = new('person')
|
||||
Organization = new('organization')
|
||||
Location = new('location')
|
||||
DateEntity = new('date')
|
||||
end
|
||||
end
|
||||
|
||||
class Entity < T::Struct
|
||||
const :name, String, description: "The entity text as it appears in the source"
|
||||
const :type, EntityType
|
||||
const :confidence, Float, description: "Extraction confidence from 0.0 to 1.0"
|
||||
const :start_offset, Integer, default: 0
|
||||
end
|
||||
|
||||
input do
|
||||
const :text, String
|
||||
const :entity_types, T::Array[EntityType], default: [],
|
||||
description: "Filter to these entity types; empty means all types"
|
||||
end
|
||||
|
||||
output do
|
||||
const :entities, T::Array[Entity]
|
||||
const :total_found, Integer
|
||||
end
|
||||
end
|
||||
|
||||
# --- Signature with Union Types ---
|
||||
|
||||
class FlexibleClassification < DSPy::Signature
|
||||
description "Classify input with flexible result type"
|
||||
|
||||
class Category < T::Enum
|
||||
enums do
|
||||
Technical = new('technical')
|
||||
Business = new('business')
|
||||
Personal = new('personal')
|
||||
end
|
||||
end
|
||||
|
||||
input do
|
||||
const :text, String
|
||||
end
|
||||
|
||||
output do
|
||||
const :category, Category
|
||||
const :result, T.any(Float, String),
|
||||
description: "Numeric score or text explanation depending on classification"
|
||||
const :confidence, Float
|
||||
end
|
||||
end
|
||||
|
||||
# --- Signature with Recursive Types ---
|
||||
|
||||
class DocumentParser < DSPy::Signature
|
||||
description "Parse document into tree structure"
|
||||
|
||||
class NodeType < T::Enum
|
||||
enums do
|
||||
Heading = new('heading')
|
||||
Paragraph = new('paragraph')
|
||||
List = new('list')
|
||||
CodeBlock = new('code_block')
|
||||
end
|
||||
end
|
||||
|
||||
class TreeNode < T::Struct
|
||||
const :node_type, NodeType, description: "The type of document element"
|
||||
const :text, String, default: "", description: "Text content of the node"
|
||||
const :level, Integer, default: 0
|
||||
const :children, T::Array[TreeNode], default: [] # Self-reference → $defs in JSON Schema
|
||||
end
|
||||
|
||||
input do
|
||||
const :html, String, description: "Raw HTML to parse"
|
||||
end
|
||||
|
||||
output do
|
||||
const :root, TreeNode
|
||||
const :word_count, Integer
|
||||
end
|
||||
end
|
||||
|
||||
# The schema generator creates #/$defs/TreeNode references for recursive types,
|
||||
# compatible with OpenAI and Gemini structured outputs.
|
||||
# Use `default: []` instead of `T.nilable(T::Array[...])` for OpenAI compatibility.
|
||||
|
||||
# --- Vision Signature ---
|
||||
|
||||
class ImageAnalysis < DSPy::Signature
|
||||
description "Analyze an image and answer questions about its content"
|
||||
|
||||
input do
|
||||
const :image, DSPy::Image, description: "The image to analyze"
|
||||
const :question, String, description: "Question about the image content"
|
||||
end
|
||||
|
||||
output do
|
||||
const :answer, String
|
||||
const :confidence, Float, description: "Confidence in the answer (0.0-1.0)"
|
||||
end
|
||||
end
|
||||
|
||||
# Vision usage:
|
||||
# predictor = DSPy::Predict.new(ImageAnalysis)
|
||||
# result = predictor.call(
|
||||
# image: DSPy::Image.from_file("path/to/image.jpg"),
|
||||
# question: "What objects are visible?"
|
||||
# )
|
||||
# result.answer # => "The image shows..."
|
||||
|
||||
# --- Accessing Schemas Programmatically ---
|
||||
#
|
||||
# SentimentAnalysis.input_json_schema # => { type: "object", properties: { ... } }
|
||||
# SentimentAnalysis.output_json_schema # => { type: "object", properties: { ... } }
|
||||
#
|
||||
# # Field descriptions propagate to JSON Schema
|
||||
# Entity.field_descriptions[:name] # => "The entity text as it appears in the source"
|
||||
# Entity.field_descriptions[:confidence] # => "Extraction confidence from 0.0 to 1.0"
|
||||
@@ -1,674 +0,0 @@
|
||||
# DSPy.rb Core Concepts
|
||||
|
||||
## Signatures
|
||||
|
||||
Signatures define the interface between application code and language models. They specify inputs, outputs, and a task description using Sorbet types for compile-time and runtime type safety.
|
||||
|
||||
### Structure
|
||||
|
||||
```ruby
|
||||
class ClassifyEmail < DSPy::Signature
|
||||
description "Classify customer support emails by urgency and category"
|
||||
|
||||
input do
|
||||
const :subject, String
|
||||
const :body, String
|
||||
end
|
||||
|
||||
output do
|
||||
const :category, String
|
||||
const :urgency, String
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
### Supported Types
|
||||
|
||||
| Type | JSON Schema | Notes |
|
||||
|------|-------------|-------|
|
||||
| `String` | `string` | Required string |
|
||||
| `Integer` | `integer` | Whole numbers |
|
||||
| `Float` | `number` | Decimal numbers |
|
||||
| `T::Boolean` | `boolean` | true/false |
|
||||
| `T::Array[X]` | `array` | Typed arrays |
|
||||
| `T::Hash[K, V]` | `object` | Typed key-value maps |
|
||||
| `T.nilable(X)` | nullable | Optional fields |
|
||||
| `Date` | `string` (ISO 8601) | Auto-converted |
|
||||
| `DateTime` | `string` (ISO 8601) | Preserves timezone |
|
||||
| `Time` | `string` (ISO 8601) | Converted to UTC |
|
||||
|
||||
### Date and Time Types
|
||||
|
||||
Date, DateTime, and Time fields serialize to ISO 8601 strings and auto-convert back to Ruby objects on output.
|
||||
|
||||
```ruby
|
||||
class EventScheduler < DSPy::Signature
|
||||
description "Schedule events based on requirements"
|
||||
|
||||
input do
|
||||
const :start_date, Date # ISO 8601: YYYY-MM-DD
|
||||
const :preferred_time, DateTime # ISO 8601 with timezone
|
||||
const :deadline, Time # Converted to UTC
|
||||
const :end_date, T.nilable(Date) # Optional date
|
||||
end
|
||||
|
||||
output do
|
||||
const :scheduled_date, Date # String from LLM, auto-converted to Date
|
||||
const :event_datetime, DateTime # Preserves timezone info
|
||||
const :created_at, Time # Converted to UTC
|
||||
end
|
||||
end
|
||||
|
||||
predictor = DSPy::Predict.new(EventScheduler)
|
||||
result = predictor.call(
|
||||
start_date: "2024-01-15",
|
||||
preferred_time: "2024-01-15T10:30:45Z",
|
||||
deadline: Time.now,
|
||||
end_date: nil
|
||||
)
|
||||
|
||||
result.scheduled_date.class # => Date
|
||||
result.event_datetime.class # => DateTime
|
||||
```
|
||||
|
||||
Timezone conventions follow ActiveRecord: Time objects convert to UTC, DateTime objects preserve timezone, Date objects are timezone-agnostic.
|
||||
|
||||
### Enums with T::Enum
|
||||
|
||||
Define constrained output values using `T::Enum` classes. Do not use inline `T.enum([...])` syntax.
|
||||
|
||||
```ruby
|
||||
class SentimentAnalysis < DSPy::Signature
|
||||
description "Analyze sentiment of text"
|
||||
|
||||
class Sentiment < T::Enum
|
||||
enums do
|
||||
Positive = new('positive')
|
||||
Negative = new('negative')
|
||||
Neutral = new('neutral')
|
||||
end
|
||||
end
|
||||
|
||||
input do
|
||||
const :text, String
|
||||
end
|
||||
|
||||
output do
|
||||
const :sentiment, Sentiment
|
||||
const :confidence, Float
|
||||
end
|
||||
end
|
||||
|
||||
predictor = DSPy::Predict.new(SentimentAnalysis)
|
||||
result = predictor.call(text: "This product is amazing!")
|
||||
|
||||
result.sentiment # => #<Sentiment::Positive>
|
||||
result.sentiment.serialize # => "positive"
|
||||
result.confidence # => 0.92
|
||||
```
|
||||
|
||||
Enum matching is case-insensitive. The LLM returning `"POSITIVE"` matches `new('positive')`.
|
||||
|
||||
### Default Values
|
||||
|
||||
Default values work on both inputs and outputs. Input defaults reduce caller boilerplate. Output defaults provide fallbacks when the LLM omits optional fields.
|
||||
|
||||
```ruby
|
||||
class SmartSearch < DSPy::Signature
|
||||
description "Search with intelligent defaults"
|
||||
|
||||
input do
|
||||
const :query, String
|
||||
const :max_results, Integer, default: 10
|
||||
const :language, String, default: "English"
|
||||
end
|
||||
|
||||
output do
|
||||
const :results, T::Array[String]
|
||||
const :total_found, Integer
|
||||
const :cached, T::Boolean, default: false
|
||||
end
|
||||
end
|
||||
|
||||
search = DSPy::Predict.new(SmartSearch)
|
||||
result = search.call(query: "Ruby programming")
|
||||
# max_results defaults to 10, language defaults to "English"
|
||||
# If LLM omits `cached`, it defaults to false
|
||||
```
|
||||
|
||||
### Field Descriptions
|
||||
|
||||
Add `description:` to any field to guide the LLM on expected content. These descriptions appear in the generated JSON schema sent to the model.
|
||||
|
||||
```ruby
|
||||
class ASTNode < T::Struct
|
||||
const :node_type, String, description: "The type of AST node (heading, paragraph, code_block)"
|
||||
const :text, String, default: "", description: "Text content of the node"
|
||||
const :level, Integer, default: 0, description: "Heading level 1-6, only for heading nodes"
|
||||
const :children, T::Array[ASTNode], default: []
|
||||
end
|
||||
|
||||
ASTNode.field_descriptions[:node_type] # => "The type of AST node ..."
|
||||
ASTNode.field_descriptions[:children] # => nil (no description set)
|
||||
```
|
||||
|
||||
Field descriptions also work inside signature `input` and `output` blocks:
|
||||
|
||||
```ruby
|
||||
class ExtractEntities < DSPy::Signature
|
||||
description "Extract named entities from text"
|
||||
|
||||
input do
|
||||
const :text, String, description: "Raw text to analyze"
|
||||
const :language, String, default: "en", description: "ISO 639-1 language code"
|
||||
end
|
||||
|
||||
output do
|
||||
const :entities, T::Array[String], description: "List of extracted entity names"
|
||||
const :count, Integer, description: "Total number of unique entities found"
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
### Schema Formats
|
||||
|
||||
DSPy.rb supports three schema formats for communicating type structure to LLMs.
|
||||
|
||||
#### JSON Schema (default)
|
||||
|
||||
Verbose but universally supported. Access via `YourSignature.output_json_schema`.
|
||||
|
||||
#### BAML Schema
|
||||
|
||||
Compact format that reduces schema tokens by 80-85%. Requires the `sorbet-baml` gem.
|
||||
|
||||
```ruby
|
||||
DSPy.configure do |c|
|
||||
c.lm = DSPy::LM.new('openai/gpt-4o-mini',
|
||||
api_key: ENV['OPENAI_API_KEY'],
|
||||
schema_format: :baml
|
||||
)
|
||||
end
|
||||
```
|
||||
|
||||
BAML applies only in Enhanced Prompting mode (`structured_outputs: false`). When `structured_outputs: true`, the provider receives JSON Schema directly.
|
||||
|
||||
#### TOON Schema + Data Format
|
||||
|
||||
Table-oriented text format that shrinks both schema definitions and prompt values.
|
||||
|
||||
```ruby
|
||||
DSPy.configure do |c|
|
||||
c.lm = DSPy::LM.new('openai/gpt-4o-mini',
|
||||
api_key: ENV['OPENAI_API_KEY'],
|
||||
schema_format: :toon,
|
||||
data_format: :toon
|
||||
)
|
||||
end
|
||||
```
|
||||
|
||||
`schema_format: :toon` replaces the schema block in the system prompt. `data_format: :toon` renders input values and output templates inside `toon` fences. Only works with Enhanced Prompting mode. The `sorbet-toon` gem is included automatically as a dependency.
|
||||
|
||||
### Recursive Types
|
||||
|
||||
Structs that reference themselves produce `$defs` entries in the generated JSON schema, using `$ref` pointers to avoid infinite recursion.
|
||||
|
||||
```ruby
|
||||
class ASTNode < T::Struct
|
||||
const :node_type, String
|
||||
const :text, String, default: ""
|
||||
const :children, T::Array[ASTNode], default: []
|
||||
end
|
||||
```
|
||||
|
||||
The schema generator detects the self-reference in `T::Array[ASTNode]` and emits:
|
||||
|
||||
```json
|
||||
{
|
||||
"$defs": {
|
||||
"ASTNode": { "type": "object", "properties": { ... } }
|
||||
},
|
||||
"properties": {
|
||||
"children": {
|
||||
"type": "array",
|
||||
"items": { "$ref": "#/$defs/ASTNode" }
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Access the schema with accumulated definitions via `YourSignature.output_json_schema_with_defs`.
|
||||
|
||||
### Union Types with T.any()
|
||||
|
||||
Specify fields that accept multiple types:
|
||||
|
||||
```ruby
|
||||
output do
|
||||
const :result, T.any(Float, String)
|
||||
end
|
||||
```
|
||||
|
||||
For struct unions, DSPy.rb automatically adds a `_type` discriminator field to each struct's JSON schema. The LLM returns `_type` in its response, and DSPy converts the hash to the correct struct instance.
|
||||
|
||||
```ruby
|
||||
class CreateTask < T::Struct
|
||||
const :title, String
|
||||
const :priority, String
|
||||
end
|
||||
|
||||
class DeleteTask < T::Struct
|
||||
const :task_id, String
|
||||
const :reason, T.nilable(String)
|
||||
end
|
||||
|
||||
class TaskRouter < DSPy::Signature
|
||||
description "Route user request to the appropriate task action"
|
||||
|
||||
input do
|
||||
const :request, String
|
||||
end
|
||||
|
||||
output do
|
||||
const :action, T.any(CreateTask, DeleteTask)
|
||||
end
|
||||
end
|
||||
|
||||
result = DSPy::Predict.new(TaskRouter).call(request: "Create a task for Q4 review")
|
||||
result.action.class # => CreateTask
|
||||
result.action.title # => "Q4 Review"
|
||||
```
|
||||
|
||||
Pattern matching works on the result:
|
||||
|
||||
```ruby
|
||||
case result.action
|
||||
when CreateTask then puts "Creating: #{result.action.title}"
|
||||
when DeleteTask then puts "Deleting: #{result.action.task_id}"
|
||||
end
|
||||
```
|
||||
|
||||
Union types also work inside arrays for heterogeneous collections:
|
||||
|
||||
```ruby
|
||||
output do
|
||||
const :events, T::Array[T.any(LoginEvent, PurchaseEvent)]
|
||||
end
|
||||
```
|
||||
|
||||
Limit unions to 2-4 types for reliable LLM comprehension. Use clear struct names since they become the `_type` discriminator values.
|
||||
|
||||
---
|
||||
|
||||
## Modules
|
||||
|
||||
Modules are composable building blocks that wrap predictors. Define a `forward` method; invoke the module with `.call()`.
|
||||
|
||||
### Basic Structure
|
||||
|
||||
```ruby
|
||||
class SentimentAnalyzer < DSPy::Module
|
||||
def initialize
|
||||
super
|
||||
@predictor = DSPy::Predict.new(SentimentSignature)
|
||||
end
|
||||
|
||||
def forward(text:)
|
||||
@predictor.call(text: text)
|
||||
end
|
||||
end
|
||||
|
||||
analyzer = SentimentAnalyzer.new
|
||||
result = analyzer.call(text: "I love this product!")
|
||||
|
||||
result.sentiment # => "positive"
|
||||
result.confidence # => 0.9
|
||||
```
|
||||
|
||||
**API rules:**
|
||||
- Invoke modules and predictors with `.call()`, not `.forward()`.
|
||||
- Access result fields with `result.field`, not `result[:field]`.
|
||||
|
||||
### Module Composition
|
||||
|
||||
Combine multiple modules through explicit method calls in `forward`:
|
||||
|
||||
```ruby
|
||||
class DocumentProcessor < DSPy::Module
|
||||
def initialize
|
||||
super
|
||||
@classifier = DocumentClassifier.new
|
||||
@summarizer = DocumentSummarizer.new
|
||||
end
|
||||
|
||||
def forward(document:)
|
||||
classification = @classifier.call(content: document)
|
||||
summary = @summarizer.call(content: document)
|
||||
|
||||
{
|
||||
document_type: classification.document_type,
|
||||
summary: summary.summary
|
||||
}
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
### Lifecycle Callbacks
|
||||
|
||||
Modules support `before`, `after`, and `around` callbacks on `forward`. Declare them as class-level macros referencing private methods.
|
||||
|
||||
#### Execution order
|
||||
|
||||
1. `before` callbacks (in registration order)
|
||||
2. `around` callbacks (before `yield`)
|
||||
3. `forward` method
|
||||
4. `around` callbacks (after `yield`)
|
||||
5. `after` callbacks (in registration order)
|
||||
|
||||
```ruby
|
||||
class InstrumentedModule < DSPy::Module
|
||||
before :setup_metrics
|
||||
after :log_metrics
|
||||
around :manage_context
|
||||
|
||||
def initialize
|
||||
super
|
||||
@predictor = DSPy::Predict.new(MySignature)
|
||||
@metrics = {}
|
||||
end
|
||||
|
||||
def forward(question:)
|
||||
@predictor.call(question: question)
|
||||
end
|
||||
|
||||
private
|
||||
|
||||
def setup_metrics
|
||||
@metrics[:start_time] = Time.now
|
||||
end
|
||||
|
||||
def manage_context
|
||||
load_context
|
||||
result = yield
|
||||
save_context
|
||||
result
|
||||
end
|
||||
|
||||
def log_metrics
|
||||
@metrics[:duration] = Time.now - @metrics[:start_time]
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
Multiple callbacks of the same type execute in registration order. Callbacks inherit from parent classes; parent callbacks run first.
|
||||
|
||||
#### Around callbacks
|
||||
|
||||
Around callbacks must call `yield` to execute the wrapped method and return the result:
|
||||
|
||||
```ruby
|
||||
def with_retry
|
||||
retries = 0
|
||||
begin
|
||||
yield
|
||||
rescue StandardError => e
|
||||
retries += 1
|
||||
retry if retries < 3
|
||||
raise e
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
### Instruction Update Contract
|
||||
|
||||
Teleprompters (GEPA, MIPROv2) require modules to expose immutable update hooks. Include `DSPy::Mixins::InstructionUpdatable` and implement `with_instruction` and `with_examples`, each returning a new instance:
|
||||
|
||||
```ruby
|
||||
class SentimentPredictor < DSPy::Module
|
||||
include DSPy::Mixins::InstructionUpdatable
|
||||
|
||||
def initialize
|
||||
super
|
||||
@predictor = DSPy::Predict.new(SentimentSignature)
|
||||
end
|
||||
|
||||
def with_instruction(instruction)
|
||||
clone = self.class.new
|
||||
clone.instance_variable_set(:@predictor, @predictor.with_instruction(instruction))
|
||||
clone
|
||||
end
|
||||
|
||||
def with_examples(examples)
|
||||
clone = self.class.new
|
||||
clone.instance_variable_set(:@predictor, @predictor.with_examples(examples))
|
||||
clone
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
If a module omits these hooks, teleprompters raise `DSPy::InstructionUpdateError` instead of silently mutating state.
|
||||
|
||||
---
|
||||
|
||||
## Predictors
|
||||
|
||||
Predictors are execution engines that take a signature and produce structured results from a language model. DSPy.rb provides four predictor types.
|
||||
|
||||
### Predict
|
||||
|
||||
Direct LLM call with typed input/output. Fastest option, lowest token usage.
|
||||
|
||||
```ruby
|
||||
classifier = DSPy::Predict.new(ClassifyText)
|
||||
result = classifier.call(text: "Technical document about APIs")
|
||||
|
||||
result.sentiment # => #<Sentiment::Positive>
|
||||
result.topics # => ["APIs", "technical"]
|
||||
result.confidence # => 0.92
|
||||
```
|
||||
|
||||
### ChainOfThought
|
||||
|
||||
Adds a `reasoning` field to the output automatically. The model generates step-by-step reasoning before the final answer. Do not define a `:reasoning` field in the signature output when using ChainOfThought.
|
||||
|
||||
```ruby
|
||||
class SolveMathProblem < DSPy::Signature
|
||||
description "Solve mathematical word problems step by step"
|
||||
|
||||
input do
|
||||
const :problem, String
|
||||
end
|
||||
|
||||
output do
|
||||
const :answer, String
|
||||
# :reasoning is added automatically by ChainOfThought
|
||||
end
|
||||
end
|
||||
|
||||
solver = DSPy::ChainOfThought.new(SolveMathProblem)
|
||||
result = solver.call(problem: "Sarah has 15 apples. She gives 7 away and buys 12 more.")
|
||||
|
||||
result.reasoning # => "Step by step: 15 - 7 = 8, then 8 + 12 = 20"
|
||||
result.answer # => "20 apples"
|
||||
```
|
||||
|
||||
Use ChainOfThought for complex analysis, multi-step reasoning, or when explainability matters.
|
||||
|
||||
### ReAct
|
||||
|
||||
Reasoning + Action agent that uses tools in an iterative loop. Define tools by subclassing `DSPy::Tools::Base`. Group related tools with `DSPy::Tools::Toolset`.
|
||||
|
||||
```ruby
|
||||
class WeatherTool < DSPy::Tools::Base
|
||||
extend T::Sig
|
||||
|
||||
tool_name "weather"
|
||||
tool_description "Get weather information for a location"
|
||||
|
||||
sig { params(location: String).returns(String) }
|
||||
def call(location:)
|
||||
{ location: location, temperature: 72, condition: "sunny" }.to_json
|
||||
end
|
||||
end
|
||||
|
||||
class TravelSignature < DSPy::Signature
|
||||
description "Help users plan travel"
|
||||
|
||||
input do
|
||||
const :destination, String
|
||||
end
|
||||
|
||||
output do
|
||||
const :recommendations, String
|
||||
end
|
||||
end
|
||||
|
||||
agent = DSPy::ReAct.new(
|
||||
TravelSignature,
|
||||
tools: [WeatherTool.new],
|
||||
max_iterations: 5
|
||||
)
|
||||
|
||||
result = agent.call(destination: "Tokyo, Japan")
|
||||
result.recommendations # => "Visit Senso-ji Temple early morning..."
|
||||
result.history # => Array of reasoning steps, actions, observations
|
||||
result.iterations # => 3
|
||||
result.tools_used # => ["weather"]
|
||||
```
|
||||
|
||||
Use toolsets to expose multiple tool methods from a single class:
|
||||
|
||||
```ruby
|
||||
text_tools = DSPy::Tools::TextProcessingToolset.to_tools
|
||||
agent = DSPy::ReAct.new(MySignature, tools: text_tools)
|
||||
```
|
||||
|
||||
### CodeAct
|
||||
|
||||
Think-Code-Observe agent that synthesizes and executes Ruby code. Ships as a separate gem.
|
||||
|
||||
```ruby
|
||||
# Gemfile
|
||||
gem 'dspy-code_act', '~> 0.29'
|
||||
```
|
||||
|
||||
```ruby
|
||||
programmer = DSPy::CodeAct.new(ProgrammingSignature, max_iterations: 10)
|
||||
result = programmer.call(task: "Calculate the factorial of 20")
|
||||
```
|
||||
|
||||
### Predictor Comparison
|
||||
|
||||
| Predictor | Speed | Token Usage | Best For |
|
||||
|-----------|-------|-------------|----------|
|
||||
| Predict | Fastest | Low | Classification, extraction |
|
||||
| ChainOfThought | Moderate | Medium-High | Complex reasoning, analysis |
|
||||
| ReAct | Slower | High | Multi-step tasks with tools |
|
||||
| CodeAct | Slowest | Very High | Dynamic programming, calculations |
|
||||
|
||||
### Concurrent Predictions
|
||||
|
||||
Process multiple independent predictions simultaneously using `Async::Barrier`:
|
||||
|
||||
```ruby
|
||||
require 'async'
|
||||
require 'async/barrier'
|
||||
|
||||
analyzer = DSPy::Predict.new(ContentAnalyzer)
|
||||
documents = ["Text one", "Text two", "Text three"]
|
||||
|
||||
Async do
|
||||
barrier = Async::Barrier.new
|
||||
|
||||
tasks = documents.map do |doc|
|
||||
barrier.async { analyzer.call(content: doc) }
|
||||
end
|
||||
|
||||
barrier.wait
|
||||
predictions = tasks.map(&:wait)
|
||||
|
||||
predictions.each { |p| puts p.sentiment }
|
||||
end
|
||||
```
|
||||
|
||||
Add `gem 'async', '~> 2.29'` to the Gemfile. Handle errors within each `barrier.async` block to prevent one failure from cancelling others:
|
||||
|
||||
```ruby
|
||||
barrier.async do
|
||||
begin
|
||||
analyzer.call(content: doc)
|
||||
rescue StandardError => e
|
||||
nil
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
### Few-Shot Examples and Instruction Tuning
|
||||
|
||||
```ruby
|
||||
classifier = DSPy::Predict.new(SentimentAnalysis)
|
||||
|
||||
examples = [
|
||||
DSPy::FewShotExample.new(
|
||||
input: { text: "Love it!" },
|
||||
output: { sentiment: "positive", confidence: 0.95 }
|
||||
)
|
||||
]
|
||||
|
||||
optimized = classifier.with_examples(examples)
|
||||
tuned = classifier.with_instruction("Be precise and confident.")
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Type System
|
||||
|
||||
### Automatic Type Conversion
|
||||
|
||||
DSPy.rb v0.9.0+ automatically converts LLM JSON responses to typed Ruby objects:
|
||||
|
||||
- **Enums**: String values become `T::Enum` instances (case-insensitive)
|
||||
- **Structs**: Nested hashes become `T::Struct` objects
|
||||
- **Arrays**: Elements convert recursively
|
||||
- **Defaults**: Missing fields use declared defaults
|
||||
|
||||
### Discriminators for Union Types
|
||||
|
||||
When a field uses `T.any()` with struct types, DSPy adds a `_type` field to each struct's schema. On deserialization, `_type` selects the correct struct class:
|
||||
|
||||
```json
|
||||
{
|
||||
"action": {
|
||||
"_type": "CreateTask",
|
||||
"title": "Review Q4 Report"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
DSPy matches `"CreateTask"` against the union members and instantiates the correct struct. No manual discriminator field is needed.
|
||||
|
||||
### Recursive Types
|
||||
|
||||
Structs referencing themselves are supported. The schema generator tracks visited types and produces `$ref` pointers under `$defs`:
|
||||
|
||||
```ruby
|
||||
class TreeNode < T::Struct
|
||||
const :label, String
|
||||
const :children, T::Array[TreeNode], default: []
|
||||
end
|
||||
```
|
||||
|
||||
The generated schema uses `"$ref": "#/$defs/TreeNode"` for the children array items, preventing infinite schema expansion.
|
||||
|
||||
### Nesting Depth
|
||||
|
||||
- 1-2 levels: reliable across all providers.
|
||||
- 3-4 levels: works but increases schema complexity.
|
||||
- 5+ levels: may trigger OpenAI depth validation warnings and reduce LLM accuracy. Flatten deeply nested structures or split into multiple signatures.
|
||||
|
||||
### Tips
|
||||
|
||||
- Prefer `T::Array[X], default: []` over `T.nilable(T::Array[X])` -- the nilable form causes schema issues with OpenAI structured outputs.
|
||||
- Use clear struct names for union types since they become `_type` discriminator values.
|
||||
- Limit union types to 2-4 members for reliable model comprehension.
|
||||
- Check schema compatibility with `DSPy::OpenAI::LM::SchemaConverter.validate_compatibility(schema)`.
|
||||
@@ -1,366 +0,0 @@
|
||||
# DSPy.rb Observability
|
||||
|
||||
DSPy.rb provides an event-driven observability system built on OpenTelemetry. The system replaces monkey-patching with structured event emission, pluggable listeners, automatic span creation, and non-blocking Langfuse export.
|
||||
|
||||
## Event System
|
||||
|
||||
### Emitting Events
|
||||
|
||||
Emit structured events with `DSPy.event`:
|
||||
|
||||
```ruby
|
||||
DSPy.event('lm.tokens', {
|
||||
'gen_ai.system' => 'openai',
|
||||
'gen_ai.request.model' => 'gpt-4',
|
||||
input_tokens: 150,
|
||||
output_tokens: 50,
|
||||
total_tokens: 200
|
||||
})
|
||||
```
|
||||
|
||||
Event names are **strings** with dot-separated namespaces (e.g., `'llm.generate'`, `'react.iteration_complete'`, `'chain_of_thought.reasoning_complete'`). Do not use symbols for event names.
|
||||
|
||||
Attributes must be JSON-serializable. DSPy automatically merges context (trace ID, module stack) and creates OpenTelemetry spans.
|
||||
|
||||
### Global Subscriptions
|
||||
|
||||
Subscribe to events across the entire application with `DSPy.events.subscribe`:
|
||||
|
||||
```ruby
|
||||
# Exact event name
|
||||
subscription_id = DSPy.events.subscribe('lm.tokens') do |event_name, attrs|
|
||||
puts "Tokens used: #{attrs[:total_tokens]}"
|
||||
end
|
||||
|
||||
# Wildcard pattern -- matches llm.generate, llm.stream, etc.
|
||||
DSPy.events.subscribe('llm.*') do |event_name, attrs|
|
||||
track_llm_usage(attrs)
|
||||
end
|
||||
|
||||
# Catch-all wildcard
|
||||
DSPy.events.subscribe('*') do |event_name, attrs|
|
||||
log_everything(event_name, attrs)
|
||||
end
|
||||
```
|
||||
|
||||
Use global subscriptions for cross-cutting concerns: observability exporters (Langfuse, Datadog), centralized logging, metrics collection.
|
||||
|
||||
### Module-Scoped Subscriptions
|
||||
|
||||
Declare listeners inside a `DSPy::Module` subclass. Subscriptions automatically scope to the module instance and its descendants:
|
||||
|
||||
```ruby
|
||||
class ResearchReport < DSPy::Module
|
||||
subscribe 'lm.tokens', :track_tokens, scope: :descendants
|
||||
|
||||
def initialize
|
||||
super
|
||||
@outliner = DSPy::Predict.new(OutlineSignature)
|
||||
@writer = DSPy::Predict.new(SectionWriterSignature)
|
||||
@token_count = 0
|
||||
end
|
||||
|
||||
def forward(question:)
|
||||
outline = @outliner.call(question: question)
|
||||
outline.sections.map do |title|
|
||||
draft = @writer.call(question: question, section_title: title)
|
||||
{ title: title, body: draft.paragraph }
|
||||
end
|
||||
end
|
||||
|
||||
def track_tokens(_event, attrs)
|
||||
@token_count += attrs.fetch(:total_tokens, 0)
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
The `scope:` parameter accepts:
|
||||
- `:descendants` (default) -- receives events from the module **and** every nested module invoked inside it.
|
||||
- `DSPy::Module::SubcriptionScope::SelfOnly` -- restricts delivery to events emitted by the module instance itself; ignores descendants.
|
||||
|
||||
Inspect active subscriptions with `registered_module_subscriptions`. Tear down with `unsubscribe_module_events`.
|
||||
|
||||
### Unsubscribe and Cleanup
|
||||
|
||||
Remove a global listener by subscription ID:
|
||||
|
||||
```ruby
|
||||
id = DSPy.events.subscribe('llm.*') { |name, attrs| }
|
||||
DSPy.events.unsubscribe(id)
|
||||
```
|
||||
|
||||
Build tracker classes that manage their own subscription lifecycle:
|
||||
|
||||
```ruby
|
||||
class TokenBudgetTracker
|
||||
def initialize(budget:)
|
||||
@budget = budget
|
||||
@usage = 0
|
||||
@subscriptions = []
|
||||
@subscriptions << DSPy.events.subscribe('lm.tokens') do |_event, attrs|
|
||||
@usage += attrs.fetch(:total_tokens, 0)
|
||||
warn("Budget hit") if @usage >= @budget
|
||||
end
|
||||
end
|
||||
|
||||
def unsubscribe
|
||||
@subscriptions.each { |id| DSPy.events.unsubscribe(id) }
|
||||
@subscriptions.clear
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
### Clearing Listeners in Tests
|
||||
|
||||
Call `DSPy.events.clear_listeners` in `before`/`after` blocks to prevent cross-contamination between test cases:
|
||||
|
||||
```ruby
|
||||
RSpec.configure do |config|
|
||||
config.after(:each) { DSPy.events.clear_listeners }
|
||||
end
|
||||
```
|
||||
|
||||
## dspy-o11y Gems
|
||||
|
||||
Three gems compose the observability stack:
|
||||
|
||||
| Gem | Purpose |
|
||||
|---|---|
|
||||
| `dspy` | Core event bus (`DSPy.event`, `DSPy.events`) -- always available |
|
||||
| `dspy-o11y` | OpenTelemetry spans, `AsyncSpanProcessor`, `DSPy::Context.with_span` helpers |
|
||||
| `dspy-o11y-langfuse` | Langfuse adapter -- configures OTLP exporter targeting Langfuse endpoints |
|
||||
|
||||
### Installation
|
||||
|
||||
```ruby
|
||||
# Gemfile
|
||||
gem 'dspy'
|
||||
gem 'dspy-o11y' # core spans + helpers
|
||||
gem 'dspy-o11y-langfuse' # Langfuse/OpenTelemetry adapter (optional)
|
||||
```
|
||||
|
||||
If the optional gems are absent, DSPy falls back to logging-only mode with no errors.
|
||||
|
||||
## Langfuse Integration
|
||||
|
||||
### Environment Variables
|
||||
|
||||
```bash
|
||||
# Required
|
||||
export LANGFUSE_PUBLIC_KEY=pk-lf-your-public-key
|
||||
export LANGFUSE_SECRET_KEY=sk-lf-your-secret-key
|
||||
|
||||
# Optional (defaults to https://cloud.langfuse.com)
|
||||
export LANGFUSE_HOST=https://us.cloud.langfuse.com
|
||||
|
||||
# Tuning (optional)
|
||||
export DSPY_TELEMETRY_BATCH_SIZE=100 # spans per export batch (default 100)
|
||||
export DSPY_TELEMETRY_QUEUE_SIZE=1000 # max queued spans (default 1000)
|
||||
export DSPY_TELEMETRY_EXPORT_INTERVAL=60 # seconds between timed exports (default 60)
|
||||
export DSPY_TELEMETRY_SHUTDOWN_TIMEOUT=10 # seconds to drain on shutdown (default 10)
|
||||
```
|
||||
|
||||
### Automatic Configuration
|
||||
|
||||
Call `DSPy::Observability.configure!` once at boot (it is already called automatically when `require 'dspy'` runs and Langfuse env vars are present):
|
||||
|
||||
```ruby
|
||||
require 'dspy'
|
||||
# If LANGFUSE_PUBLIC_KEY and LANGFUSE_SECRET_KEY are set,
|
||||
# DSPy::Observability.configure! runs automatically and:
|
||||
# 1. Configures the OpenTelemetry SDK with an OTLP exporter
|
||||
# 2. Creates dual output: structured logs AND OpenTelemetry spans
|
||||
# 3. Exports spans to Langfuse using proper authentication
|
||||
# 4. Falls back gracefully if gems are missing
|
||||
```
|
||||
|
||||
Verify status with `DSPy::Observability.enabled?`.
|
||||
|
||||
### Automatic Tracing
|
||||
|
||||
With observability enabled, every `DSPy::Module#forward` call, LM request, and tool invocation creates properly nested spans. Langfuse receives hierarchical traces:
|
||||
|
||||
```
|
||||
Trace: abc-123-def
|
||||
+-- ChainOfThought.forward [2000ms] (observation type: chain)
|
||||
+-- llm.generate [1000ms] (observation type: generation)
|
||||
Model: gpt-4-0613
|
||||
Tokens: 100 in / 50 out / 150 total
|
||||
```
|
||||
|
||||
DSPy maps module classes to Langfuse observation types automatically via `DSPy::ObservationType.for_module_class`:
|
||||
|
||||
| Module | Observation Type |
|
||||
|---|---|
|
||||
| `DSPy::LM` (raw chat) | `generation` |
|
||||
| `DSPy::ChainOfThought` | `chain` |
|
||||
| `DSPy::ReAct` | `agent` |
|
||||
| Tool invocations | `tool` |
|
||||
| Memory/retrieval | `retriever` |
|
||||
| Embedding engines | `embedding` |
|
||||
| Evaluation modules | `evaluator` |
|
||||
| Generic operations | `span` |
|
||||
|
||||
## Score Reporting
|
||||
|
||||
### DSPy.score API
|
||||
|
||||
Report evaluation scores with `DSPy.score`:
|
||||
|
||||
```ruby
|
||||
# Numeric (default)
|
||||
DSPy.score('accuracy', 0.95)
|
||||
|
||||
# With comment
|
||||
DSPy.score('relevance', 0.87, comment: 'High semantic similarity')
|
||||
|
||||
# Boolean
|
||||
DSPy.score('is_valid', 1, data_type: DSPy::Scores::DataType::Boolean)
|
||||
|
||||
# Categorical
|
||||
DSPy.score('sentiment', 'positive', data_type: DSPy::Scores::DataType::Categorical)
|
||||
|
||||
# Explicit trace binding
|
||||
DSPy.score('accuracy', 0.95, trace_id: 'custom-trace-id')
|
||||
```
|
||||
|
||||
Available data types: `DSPy::Scores::DataType::Numeric`, `::Boolean`, `::Categorical`.
|
||||
|
||||
### score.create Events
|
||||
|
||||
Every `DSPy.score` call emits a `'score.create'` event. Subscribe to react:
|
||||
|
||||
```ruby
|
||||
DSPy.events.subscribe('score.create') do |event_name, attrs|
|
||||
puts "#{attrs[:score_name]} = #{attrs[:score_value]}"
|
||||
# Also available: attrs[:score_id], attrs[:score_data_type],
|
||||
# attrs[:score_comment], attrs[:trace_id], attrs[:observation_id],
|
||||
# attrs[:timestamp]
|
||||
end
|
||||
```
|
||||
|
||||
### Async Langfuse Export with DSPy::Scores::Exporter
|
||||
|
||||
Configure the exporter to send scores to Langfuse in the background:
|
||||
|
||||
```ruby
|
||||
exporter = DSPy::Scores::Exporter.configure(
|
||||
public_key: ENV['LANGFUSE_PUBLIC_KEY'],
|
||||
secret_key: ENV['LANGFUSE_SECRET_KEY'],
|
||||
host: 'https://cloud.langfuse.com'
|
||||
)
|
||||
|
||||
# Scores are now exported automatically via a background Thread::Queue
|
||||
DSPy.score('accuracy', 0.95)
|
||||
|
||||
# Shut down gracefully (waits up to 5 seconds by default)
|
||||
exporter.shutdown
|
||||
```
|
||||
|
||||
The exporter subscribes to `'score.create'` events internally, queues them for async processing, and retries with exponential backoff on failure.
|
||||
|
||||
### Automatic Export with DSPy::Evals
|
||||
|
||||
Pass `export_scores: true` to `DSPy::Evals` to export per-example scores and an aggregate batch score automatically:
|
||||
|
||||
```ruby
|
||||
evaluator = DSPy::Evals.new(
|
||||
program,
|
||||
metric: my_metric,
|
||||
export_scores: true,
|
||||
score_name: 'qa_accuracy'
|
||||
)
|
||||
|
||||
result = evaluator.evaluate(test_examples)
|
||||
```
|
||||
|
||||
## DSPy::Context.with_span
|
||||
|
||||
Create manual spans for custom operations. Requires `dspy-o11y`.
|
||||
|
||||
```ruby
|
||||
DSPy::Context.with_span(operation: 'custom.retrieval', 'retrieval.source' => 'pinecone') do |span|
|
||||
results = pinecone_client.query(embedding)
|
||||
span&.set_attribute('retrieval.count', results.size) if span
|
||||
results
|
||||
end
|
||||
```
|
||||
|
||||
Pass semantic attributes as keyword arguments alongside `operation:`. The block receives an OpenTelemetry span object (or `nil` when observability is disabled). The span automatically nests under the current parent span and records `duration.ms`, `langfuse.observation.startTime`, and `langfuse.observation.endTime`.
|
||||
|
||||
Assign a Langfuse observation type to custom spans:
|
||||
|
||||
```ruby
|
||||
DSPy::Context.with_span(
|
||||
operation: 'evaluate.batch',
|
||||
**DSPy::ObservationType::Evaluator.langfuse_attributes,
|
||||
'batch.size' => examples.length
|
||||
) do |span|
|
||||
run_evaluation(examples)
|
||||
end
|
||||
```
|
||||
|
||||
Scores reported inside a `with_span` block automatically inherit the current trace context.
|
||||
|
||||
## Module Stack Metadata
|
||||
|
||||
When `DSPy::Module#forward` runs, the context layer maintains a module stack. Every event includes:
|
||||
|
||||
```ruby
|
||||
{
|
||||
module_path: [
|
||||
{ id: "root_uuid", class: "DeepSearch", label: nil },
|
||||
{ id: "planner_uuid", class: "DSPy::Predict", label: "planner" }
|
||||
],
|
||||
module_root: { id: "root_uuid", class: "DeepSearch", label: nil },
|
||||
module_leaf: { id: "planner_uuid", class: "DSPy::Predict", label: "planner" },
|
||||
module_scope: {
|
||||
ancestry_token: "root_uuid>planner_uuid",
|
||||
depth: 2
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| Key | Meaning |
|
||||
|---|---|
|
||||
| `module_path` | Ordered array of `{id, class, label}` entries from root to leaf |
|
||||
| `module_root` | The outermost module in the current call chain |
|
||||
| `module_leaf` | The innermost (currently executing) module |
|
||||
| `module_scope.ancestry_token` | Stable string of joined UUIDs representing the nesting path |
|
||||
| `module_scope.depth` | Integer depth of the current module in the stack |
|
||||
|
||||
Labels are set via `module_scope_label=` on a module instance or derived automatically from named predictors. Use this metadata to power Langfuse filters, scoped metrics, or custom event routing.
|
||||
|
||||
## Dedicated Export Worker
|
||||
|
||||
The `DSPy::Observability::AsyncSpanProcessor` (from `dspy-o11y`) keeps telemetry export off the hot path:
|
||||
|
||||
- Runs on a `Concurrent::SingleThreadExecutor` -- LLM workflows never compete with OTLP networking.
|
||||
- Buffers finished spans in a `Thread::Queue` (max size configurable via `DSPY_TELEMETRY_QUEUE_SIZE`).
|
||||
- Drains spans in batches of `DSPY_TELEMETRY_BATCH_SIZE` (default 100). When the queue reaches batch size, an immediate async export fires.
|
||||
- A background timer thread triggers periodic export every `DSPY_TELEMETRY_EXPORT_INTERVAL` seconds (default 60).
|
||||
- Applies exponential backoff (`0.1 * 2^attempt` seconds) on export failures, up to `DEFAULT_MAX_RETRIES` (3).
|
||||
- On shutdown, flushes all remaining spans within `DSPY_TELEMETRY_SHUTDOWN_TIMEOUT` seconds, then terminates the executor.
|
||||
- Drops the oldest span when the queue is full, logging `'observability.span_dropped'`.
|
||||
|
||||
No application code interacts with the processor directly. Configure it entirely through environment variables.
|
||||
|
||||
## Built-in Events Reference
|
||||
|
||||
| Event Name | Emitted By | Key Attributes |
|
||||
|---|---|---|
|
||||
| `lm.tokens` | `DSPy::LM` | `gen_ai.system`, `gen_ai.request.model`, `input_tokens`, `output_tokens`, `total_tokens` |
|
||||
| `chain_of_thought.reasoning_complete` | `DSPy::ChainOfThought` | `dspy.signature`, `cot.reasoning_steps`, `cot.reasoning_length`, `cot.has_reasoning` |
|
||||
| `react.iteration_complete` | `DSPy::ReAct` | `iteration`, `thought`, `action`, `observation` |
|
||||
| `codeact.iteration_complete` | `dspy-code_act` gem | `iteration`, `code_executed`, `execution_result` |
|
||||
| `optimization.trial_complete` | Teleprompters (MIPROv2) | `trial_number`, `score` |
|
||||
| `score.create` | `DSPy.score` | `score_name`, `score_value`, `score_data_type`, `trace_id` |
|
||||
| `span.start` | `DSPy::Context.with_span` | `trace_id`, `span_id`, `parent_span_id`, `operation` |
|
||||
|
||||
## Best Practices
|
||||
|
||||
- Use dot-separated string names for events. Follow OpenTelemetry `gen_ai.*` conventions for LLM attributes.
|
||||
- Always call `unsubscribe` (or `unsubscribe_module_events` for scoped subscriptions) when a tracker is no longer needed to prevent memory leaks.
|
||||
- Call `DSPy.events.clear_listeners` in test teardown to avoid cross-contamination.
|
||||
- Wrap risky listener logic in a rescue block. The event system isolates listener failures, but explicit rescue prevents silent swallowing of domain errors.
|
||||
- Prefer module-scoped `subscribe` for agent internals. Reserve global `DSPy.events.subscribe` for infrastructure-level concerns.
|
||||
@@ -1,603 +0,0 @@
|
||||
# DSPy.rb Optimization
|
||||
|
||||
## MIPROv2
|
||||
|
||||
MIPROv2 (Multi-prompt Instruction Proposal with Retrieval Optimization) is the primary instruction tuner in DSPy.rb. It proposes new instructions and few-shot demonstrations per predictor, evaluates them on mini-batches, and retains candidates that improve the metric. It ships as a separate gem to keep the Gaussian Process dependency tree out of apps that do not need it.
|
||||
|
||||
### Installation
|
||||
|
||||
```ruby
|
||||
# Gemfile
|
||||
gem "dspy"
|
||||
gem "dspy-miprov2"
|
||||
```
|
||||
|
||||
Bundler auto-requires `dspy/miprov2`. No additional `require` statement is needed.
|
||||
|
||||
### AutoMode presets
|
||||
|
||||
Use `DSPy::Teleprompt::MIPROv2::AutoMode` for preconfigured optimizers:
|
||||
|
||||
```ruby
|
||||
light = DSPy::Teleprompt::MIPROv2::AutoMode.light(metric: metric) # 6 trials, greedy
|
||||
medium = DSPy::Teleprompt::MIPROv2::AutoMode.medium(metric: metric) # 12 trials, adaptive
|
||||
heavy = DSPy::Teleprompt::MIPROv2::AutoMode.heavy(metric: metric) # 18 trials, Bayesian
|
||||
```
|
||||
|
||||
| Preset | Trials | Strategy | Use case |
|
||||
|----------|--------|------------|-----------------------------------------------------|
|
||||
| `light` | 6 | `:greedy` | Quick wins on small datasets or during prototyping. |
|
||||
| `medium` | 12 | `:adaptive`| Balanced exploration vs. runtime for most pilots. |
|
||||
| `heavy` | 18 | `:bayesian`| Highest accuracy targets or multi-stage programs. |
|
||||
|
||||
### Manual configuration with dry-configurable
|
||||
|
||||
`DSPy::Teleprompt::MIPROv2` includes `Dry::Configurable`. Configure at the class level (defaults for all instances) or instance level (overrides class defaults).
|
||||
|
||||
**Class-level defaults:**
|
||||
|
||||
```ruby
|
||||
DSPy::Teleprompt::MIPROv2.configure do |config|
|
||||
config.optimization_strategy = :bayesian
|
||||
config.num_trials = 30
|
||||
config.bootstrap_sets = 10
|
||||
end
|
||||
```
|
||||
|
||||
**Instance-level overrides:**
|
||||
|
||||
```ruby
|
||||
optimizer = DSPy::Teleprompt::MIPROv2.new(metric: metric)
|
||||
optimizer.configure do |config|
|
||||
config.num_trials = 15
|
||||
config.num_instruction_candidates = 6
|
||||
config.bootstrap_sets = 5
|
||||
config.max_bootstrapped_examples = 4
|
||||
config.max_labeled_examples = 16
|
||||
config.optimization_strategy = :adaptive # :greedy, :adaptive, :bayesian
|
||||
config.early_stopping_patience = 3
|
||||
config.init_temperature = 1.0
|
||||
config.final_temperature = 0.1
|
||||
config.minibatch_size = nil # nil = auto
|
||||
config.auto_seed = 42
|
||||
end
|
||||
```
|
||||
|
||||
The `optimization_strategy` setting accepts symbols (`:greedy`, `:adaptive`, `:bayesian`) and coerces them internally to `DSPy::Teleprompt::OptimizationStrategy` T::Enum values.
|
||||
|
||||
The old `config:` constructor parameter is removed. Passing `config:` raises `ArgumentError`.
|
||||
|
||||
### Auto presets via configure
|
||||
|
||||
Instead of `AutoMode`, set the preset through the configure block:
|
||||
|
||||
```ruby
|
||||
optimizer = DSPy::Teleprompt::MIPROv2.new(metric: metric)
|
||||
optimizer.configure do |config|
|
||||
config.auto_preset = DSPy::Teleprompt::AutoPreset.deserialize("medium")
|
||||
end
|
||||
```
|
||||
|
||||
### Compile and inspect
|
||||
|
||||
```ruby
|
||||
program = DSPy::Predict.new(MySignature)
|
||||
|
||||
result = optimizer.compile(
|
||||
program,
|
||||
trainset: train_examples,
|
||||
valset: val_examples
|
||||
)
|
||||
|
||||
optimized_program = result.optimized_program
|
||||
puts "Best score: #{result.best_score_value}"
|
||||
```
|
||||
|
||||
The `result` object exposes:
|
||||
- `optimized_program` -- ready-to-use predictor with updated instruction and demos.
|
||||
- `optimization_trace[:trial_logs]` -- per-trial record of instructions, demos, and scores.
|
||||
- `metadata[:optimizer]` -- `"MIPROv2"`, useful when persisting experiments from multiple optimizers.
|
||||
|
||||
### Multi-stage programs
|
||||
|
||||
MIPROv2 generates dataset summaries for each predictor and proposes per-stage instructions. For a ReAct agent with `thought_generator` and `observation_processor` predictors, the optimizer handles credit assignment internally. The metric only needs to evaluate the final output.
|
||||
|
||||
### Bootstrap sampling
|
||||
|
||||
During the bootstrap phase MIPROv2:
|
||||
1. Generates dataset summaries from the training set.
|
||||
2. Bootstraps few-shot demonstrations by running the baseline program.
|
||||
3. Proposes candidate instructions grounded in the summaries and bootstrapped examples.
|
||||
4. Evaluates each candidate on mini-batches drawn from the validation set.
|
||||
|
||||
Control the bootstrap phase with `bootstrap_sets`, `max_bootstrapped_examples`, and `max_labeled_examples`.
|
||||
|
||||
### Bayesian optimization
|
||||
|
||||
When `optimization_strategy` is `:bayesian` (or when using the `heavy` preset), MIPROv2 fits a Gaussian Process surrogate over past trial scores to select the next candidate. This replaces random search with informed exploration, reducing the number of trials needed to find high-scoring instructions.
|
||||
|
||||
---
|
||||
|
||||
## GEPA
|
||||
|
||||
GEPA (Genetic-Pareto Reflective Prompt Evolution) is a feedback-driven optimizer. It runs the program on a small batch, collects scores and textual feedback, and asks a reflection LM to rewrite the instruction. Improved candidates are retained on a Pareto frontier.
|
||||
|
||||
### Installation
|
||||
|
||||
```ruby
|
||||
# Gemfile
|
||||
gem "dspy"
|
||||
gem "dspy-gepa"
|
||||
```
|
||||
|
||||
The `dspy-gepa` gem depends on the `gepa` core optimizer gem automatically.
|
||||
|
||||
### Metric contract
|
||||
|
||||
GEPA metrics return `DSPy::Prediction` with both a numeric score and a feedback string. Do not return a plain boolean.
|
||||
|
||||
```ruby
|
||||
metric = lambda do |example, prediction|
|
||||
expected = example.expected_values[:label]
|
||||
predicted = prediction.label
|
||||
|
||||
score = predicted == expected ? 1.0 : 0.0
|
||||
feedback = if score == 1.0
|
||||
"Correct (#{expected}) for: \"#{example.input_values[:text][0..60]}\""
|
||||
else
|
||||
"Misclassified (expected #{expected}, got #{predicted}) for: \"#{example.input_values[:text][0..60]}\""
|
||||
end
|
||||
|
||||
DSPy::Prediction.new(score: score, feedback: feedback)
|
||||
end
|
||||
```
|
||||
|
||||
Keep the score in `[0, 1]`. Always include a short feedback message explaining what happened -- GEPA hands this text to the reflection model so it can reason about failures.
|
||||
|
||||
### Feedback maps
|
||||
|
||||
`feedback_map` targets individual predictors inside a composite module. Each entry receives keyword arguments and returns a `DSPy::Prediction`:
|
||||
|
||||
```ruby
|
||||
feedback_map = {
|
||||
'self' => lambda do |predictor_output:, predictor_inputs:, module_inputs:, module_outputs:, captured_trace:|
|
||||
expected = module_inputs.expected_values[:label]
|
||||
predicted = predictor_output.label
|
||||
|
||||
DSPy::Prediction.new(
|
||||
score: predicted == expected ? 1.0 : 0.0,
|
||||
feedback: "Classifier saw \"#{predictor_inputs[:text][0..80]}\" -> #{predicted} (expected #{expected})"
|
||||
)
|
||||
end
|
||||
}
|
||||
```
|
||||
|
||||
For single-predictor programs, key the map with `'self'`. For multi-predictor chains, add entries per component so the reflection LM sees localized context at each step. Omit `feedback_map` entirely if the top-level metric already covers the basics.
|
||||
|
||||
### Configuring the teleprompter
|
||||
|
||||
```ruby
|
||||
teleprompter = DSPy::Teleprompt::GEPA.new(
|
||||
metric: metric,
|
||||
reflection_lm: DSPy::ReflectionLM.new('openai/gpt-4o-mini', api_key: ENV['OPENAI_API_KEY']),
|
||||
feedback_map: feedback_map,
|
||||
config: {
|
||||
max_metric_calls: 600,
|
||||
minibatch_size: 6,
|
||||
skip_perfect_score: false
|
||||
}
|
||||
)
|
||||
```
|
||||
|
||||
Key configuration knobs:
|
||||
|
||||
| Knob | Purpose |
|
||||
|----------------------|-------------------------------------------------------------------------------------------|
|
||||
| `max_metric_calls` | Hard budget on evaluation calls. Set to at least the validation set size plus a few minibatches. |
|
||||
| `minibatch_size` | Examples per reflective replay batch. Smaller = cheaper iterations, noisier scores. |
|
||||
| `skip_perfect_score` | Set `true` to stop early when a candidate reaches score `1.0`. |
|
||||
|
||||
### Minibatch sizing
|
||||
|
||||
| Goal | Suggested size | Rationale |
|
||||
|-------------------------------------------------|----------------|------------------------------------------------------------|
|
||||
| Explore many candidates within a tight budget | 3--6 | Cheap iterations, more prompt variants, noisier metrics. |
|
||||
| Stable metrics when each rollout is costly | 8--12 | Smoother scores, fewer candidates unless budget is raised. |
|
||||
| Investigate specific failure modes | 3--4 then 8+ | Start with breadth, increase once patterns emerge. |
|
||||
|
||||
### Compile and evaluate
|
||||
|
||||
```ruby
|
||||
program = DSPy::Predict.new(MySignature)
|
||||
|
||||
result = teleprompter.compile(program, trainset: train, valset: val)
|
||||
optimized_program = result.optimized_program
|
||||
|
||||
test_metrics = evaluate(optimized_program, test)
|
||||
```
|
||||
|
||||
The `result` object exposes:
|
||||
- `optimized_program` -- predictor with updated instruction and few-shot examples.
|
||||
- `best_score_value` -- validation score for the best candidate.
|
||||
- `metadata` -- candidate counts, trace hashes, and telemetry IDs.
|
||||
|
||||
### Reflection LM
|
||||
|
||||
Swap `DSPy::ReflectionLM` for any callable object that accepts the reflection prompt hash and returns a string. The default reflection signature extracts the new instruction from triple backticks in the response.
|
||||
|
||||
### Experiment tracking
|
||||
|
||||
Plug `GEPA::Logging::ExperimentTracker` into a persistence layer:
|
||||
|
||||
```ruby
|
||||
tracker = GEPA::Logging::ExperimentTracker.new
|
||||
tracker.with_subscriber { |event| MyModel.create!(payload: event) }
|
||||
|
||||
teleprompter = DSPy::Teleprompt::GEPA.new(
|
||||
metric: metric,
|
||||
reflection_lm: reflection_lm,
|
||||
experiment_tracker: tracker,
|
||||
config: { max_metric_calls: 900 }
|
||||
)
|
||||
```
|
||||
|
||||
The tracker emits Pareto update events, merge decisions, and candidate evolution records as JSONL.
|
||||
|
||||
### Pareto frontier
|
||||
|
||||
GEPA maintains a diverse candidate pool and samples from the Pareto frontier instead of mutating only the top-scoring program. This balances exploration and prevents the search from collapsing onto a single lineage.
|
||||
|
||||
Enable the merge proposer after multiple strong lineages emerge:
|
||||
|
||||
```ruby
|
||||
config: {
|
||||
max_metric_calls: 900,
|
||||
enable_merge_proposer: true
|
||||
}
|
||||
```
|
||||
|
||||
Premature merges eat budget without meaningful gains. Gate merge on having several validated candidates first.
|
||||
|
||||
### Advanced options
|
||||
|
||||
- `acceptance_strategy:` -- plug in bespoke Pareto filters or early-stop heuristics.
|
||||
- Telemetry spans emit via `GEPA::Telemetry`. Enable global observability with `DSPy.configure { |c| c.observability = true }` to stream spans to an OpenTelemetry exporter.
|
||||
|
||||
---
|
||||
|
||||
## Evaluation Framework
|
||||
|
||||
`DSPy::Evals` provides batch evaluation of predictors against test datasets with built-in and custom metrics.
|
||||
|
||||
### Basic usage
|
||||
|
||||
```ruby
|
||||
metric = proc do |example, prediction|
|
||||
prediction.answer == example.expected_values[:answer]
|
||||
end
|
||||
|
||||
evaluator = DSPy::Evals.new(predictor, metric: metric)
|
||||
|
||||
result = evaluator.evaluate(
|
||||
test_examples,
|
||||
display_table: true,
|
||||
display_progress: true
|
||||
)
|
||||
|
||||
puts "Pass rate: #{(result.pass_rate * 100).round(1)}%"
|
||||
puts "Passed: #{result.passed_examples}/#{result.total_examples}"
|
||||
```
|
||||
|
||||
### DSPy::Example
|
||||
|
||||
Convert raw data into `DSPy::Example` instances before passing to optimizers or evaluators. Each example carries `input_values` and `expected_values`:
|
||||
|
||||
```ruby
|
||||
examples = rows.map do |row|
|
||||
DSPy::Example.new(
|
||||
input_values: { text: row[:text] },
|
||||
expected_values: { label: row[:label] }
|
||||
)
|
||||
end
|
||||
|
||||
train, val, test = split_examples(examples, train_ratio: 0.6, val_ratio: 0.2, seed: 42)
|
||||
```
|
||||
|
||||
Hold back a test set from the optimization loop. Optimizers work on train/val; only the test set proves generalization.
|
||||
|
||||
### Built-in metrics
|
||||
|
||||
```ruby
|
||||
# Exact match -- prediction must exactly equal expected value
|
||||
metric = DSPy::Metrics.exact_match(field: :answer, case_sensitive: true)
|
||||
|
||||
# Contains -- prediction must contain expected substring
|
||||
metric = DSPy::Metrics.contains(field: :answer, case_sensitive: false)
|
||||
|
||||
# Numeric difference -- numeric output within tolerance
|
||||
metric = DSPy::Metrics.numeric_difference(field: :answer, tolerance: 0.01)
|
||||
|
||||
# Composite AND -- all sub-metrics must pass
|
||||
metric = DSPy::Metrics.composite_and(
|
||||
DSPy::Metrics.exact_match(field: :answer),
|
||||
DSPy::Metrics.contains(field: :reasoning)
|
||||
)
|
||||
```
|
||||
|
||||
### Custom metrics
|
||||
|
||||
```ruby
|
||||
quality_metric = lambda do |example, prediction|
|
||||
return false unless prediction
|
||||
|
||||
score = 0.0
|
||||
score += 0.5 if prediction.answer == example.expected_values[:answer]
|
||||
score += 0.3 if prediction.explanation && prediction.explanation.length > 50
|
||||
score += 0.2 if prediction.confidence && prediction.confidence > 0.8
|
||||
score >= 0.7
|
||||
end
|
||||
|
||||
evaluator = DSPy::Evals.new(predictor, metric: quality_metric)
|
||||
```
|
||||
|
||||
Access prediction fields with dot notation (`prediction.answer`), not hash notation.
|
||||
|
||||
### Observability hooks
|
||||
|
||||
Register callbacks without editing the evaluator:
|
||||
|
||||
```ruby
|
||||
DSPy::Evals.before_example do |payload|
|
||||
example = payload[:example]
|
||||
DSPy.logger.info("Evaluating example #{example.id}") if example.respond_to?(:id)
|
||||
end
|
||||
|
||||
DSPy::Evals.after_batch do |payload|
|
||||
result = payload[:result]
|
||||
Langfuse.event(
|
||||
name: 'eval.batch',
|
||||
metadata: {
|
||||
total: result.total_examples,
|
||||
passed: result.passed_examples,
|
||||
score: result.score
|
||||
}
|
||||
)
|
||||
end
|
||||
```
|
||||
|
||||
Available hooks: `before_example`, `after_example`, `before_batch`, `after_batch`.
|
||||
|
||||
### Langfuse score export
|
||||
|
||||
Enable `export_scores: true` to emit `score.create` events for each evaluated example and a batch score at the end:
|
||||
|
||||
```ruby
|
||||
evaluator = DSPy::Evals.new(
|
||||
predictor,
|
||||
metric: metric,
|
||||
export_scores: true,
|
||||
score_name: 'qa_accuracy' # default: 'evaluation'
|
||||
)
|
||||
|
||||
result = evaluator.evaluate(test_examples)
|
||||
# Emits per-example scores + overall batch score via DSPy::Scores::Exporter
|
||||
```
|
||||
|
||||
Scores attach to the current trace context automatically and flow to Langfuse asynchronously.
|
||||
|
||||
### Evaluation results
|
||||
|
||||
```ruby
|
||||
result = evaluator.evaluate(test_examples)
|
||||
|
||||
result.score # Overall score (0.0 to 1.0)
|
||||
result.passed_count # Examples that passed
|
||||
result.failed_count # Examples that failed
|
||||
result.error_count # Examples that errored
|
||||
|
||||
result.results.each do |r|
|
||||
r.passed # Boolean
|
||||
r.score # Numeric score
|
||||
r.error # Error message if the example errored
|
||||
end
|
||||
```
|
||||
|
||||
### Integration with optimizers
|
||||
|
||||
```ruby
|
||||
metric = proc do |example, prediction|
|
||||
expected = example.expected_values[:answer].to_s.strip.downcase
|
||||
predicted = prediction.answer.to_s.strip.downcase
|
||||
!expected.empty? && predicted.include?(expected)
|
||||
end
|
||||
|
||||
optimizer = DSPy::Teleprompt::MIPROv2::AutoMode.medium(metric: metric)
|
||||
|
||||
result = optimizer.compile(
|
||||
DSPy::Predict.new(QASignature),
|
||||
trainset: train_examples,
|
||||
valset: val_examples
|
||||
)
|
||||
|
||||
evaluator = DSPy::Evals.new(result.optimized_program, metric: metric)
|
||||
test_result = evaluator.evaluate(test_examples, display_table: true)
|
||||
puts "Test accuracy: #{(test_result.pass_rate * 100).round(2)}%"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Storage System
|
||||
|
||||
`DSPy::Storage` persists optimization results, tracks history, and manages multiple versions of optimized programs.
|
||||
|
||||
### ProgramStorage (low-level)
|
||||
|
||||
```ruby
|
||||
storage = DSPy::Storage::ProgramStorage.new(storage_path: "./dspy_storage")
|
||||
|
||||
# Save
|
||||
saved = storage.save_program(
|
||||
result.optimized_program,
|
||||
result,
|
||||
metadata: {
|
||||
signature_class: 'ClassifyText',
|
||||
optimizer: 'MIPROv2',
|
||||
examples_count: examples.size
|
||||
}
|
||||
)
|
||||
puts "Stored with ID: #{saved.program_id}"
|
||||
|
||||
# Load
|
||||
saved = storage.load_program(program_id)
|
||||
predictor = saved.program
|
||||
score = saved.optimization_result[:best_score_value]
|
||||
|
||||
# List
|
||||
storage.list_programs.each do |p|
|
||||
puts "#{p[:program_id]} -- score: #{p[:best_score]} -- saved: #{p[:saved_at]}"
|
||||
end
|
||||
```
|
||||
|
||||
### StorageManager (recommended)
|
||||
|
||||
```ruby
|
||||
manager = DSPy::Storage::StorageManager.new
|
||||
|
||||
# Save with tags
|
||||
saved = manager.save_optimization_result(
|
||||
result,
|
||||
tags: ['production', 'sentiment-analysis'],
|
||||
description: 'Optimized sentiment classifier v2'
|
||||
)
|
||||
|
||||
# Find programs
|
||||
programs = manager.find_programs(
|
||||
optimizer: 'MIPROv2',
|
||||
min_score: 0.85,
|
||||
tags: ['production']
|
||||
)
|
||||
|
||||
recent = manager.find_programs(
|
||||
max_age_days: 7,
|
||||
signature_class: 'ClassifyText'
|
||||
)
|
||||
|
||||
# Get best program for a signature
|
||||
best = manager.get_best_program('ClassifyText')
|
||||
predictor = best.program
|
||||
```
|
||||
|
||||
Global shorthand:
|
||||
|
||||
```ruby
|
||||
DSPy::Storage::StorageManager.save(result, metadata: { version: '2.0' })
|
||||
DSPy::Storage::StorageManager.load(program_id)
|
||||
DSPy::Storage::StorageManager.best('ClassifyText')
|
||||
```
|
||||
|
||||
### Checkpoints
|
||||
|
||||
Create and restore checkpoints during long-running optimizations:
|
||||
|
||||
```ruby
|
||||
# Save a checkpoint
|
||||
manager.create_checkpoint(
|
||||
current_result,
|
||||
'iteration_50',
|
||||
metadata: { iteration: 50, current_score: 0.87 }
|
||||
)
|
||||
|
||||
# Restore
|
||||
restored = manager.restore_checkpoint('iteration_50')
|
||||
program = restored.program
|
||||
|
||||
# Auto-checkpoint every N iterations
|
||||
if iteration % 10 == 0
|
||||
manager.create_checkpoint(current_result, "auto_checkpoint_#{iteration}")
|
||||
end
|
||||
```
|
||||
|
||||
### Import and export
|
||||
|
||||
Share programs between environments:
|
||||
|
||||
```ruby
|
||||
storage = DSPy::Storage::ProgramStorage.new
|
||||
|
||||
# Export
|
||||
storage.export_programs(['abc123', 'def456'], './export_backup.json')
|
||||
|
||||
# Import
|
||||
imported = storage.import_programs('./export_backup.json')
|
||||
puts "Imported #{imported.size} programs"
|
||||
```
|
||||
|
||||
### Optimization history
|
||||
|
||||
```ruby
|
||||
history = manager.get_optimization_history
|
||||
|
||||
history[:summary][:total_programs]
|
||||
history[:summary][:avg_score]
|
||||
|
||||
history[:optimizer_stats].each do |optimizer, stats|
|
||||
puts "#{optimizer}: #{stats[:count]} programs, best: #{stats[:best_score]}"
|
||||
end
|
||||
|
||||
history[:trends][:improvement_percentage]
|
||||
```
|
||||
|
||||
### Program comparison
|
||||
|
||||
```ruby
|
||||
comparison = manager.compare_programs(id_a, id_b)
|
||||
comparison[:comparison][:score_difference]
|
||||
comparison[:comparison][:better_program]
|
||||
comparison[:comparison][:age_difference_hours]
|
||||
```
|
||||
|
||||
### Storage configuration
|
||||
|
||||
```ruby
|
||||
config = DSPy::Storage::StorageManager::StorageConfig.new
|
||||
config.storage_path = Rails.root.join('dspy_storage')
|
||||
config.auto_save = true
|
||||
config.save_intermediate_results = false
|
||||
config.max_stored_programs = 100
|
||||
|
||||
manager = DSPy::Storage::StorageManager.new(config: config)
|
||||
```
|
||||
|
||||
### Cleanup
|
||||
|
||||
Remove old programs. Cleanup retains the best performing and most recent programs using a weighted score (70% performance, 30% recency):
|
||||
|
||||
```ruby
|
||||
deleted_count = manager.cleanup_old_programs
|
||||
```
|
||||
|
||||
### Storage events
|
||||
|
||||
The storage system emits structured log events for monitoring:
|
||||
- `dspy.storage.save_start`, `dspy.storage.save_complete`, `dspy.storage.save_error`
|
||||
- `dspy.storage.load_start`, `dspy.storage.load_complete`, `dspy.storage.load_error`
|
||||
- `dspy.storage.delete`, `dspy.storage.export`, `dspy.storage.import`, `dspy.storage.cleanup`
|
||||
|
||||
### File layout
|
||||
|
||||
```
|
||||
dspy_storage/
|
||||
programs/
|
||||
abc123def456.json
|
||||
789xyz012345.json
|
||||
history.json
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## API rules
|
||||
|
||||
- Call predictors with `.call()`, not `.forward()`.
|
||||
- Access prediction fields with dot notation (`result.answer`), not hash notation (`result[:answer]`).
|
||||
- GEPA metrics return `DSPy::Prediction.new(score:, feedback:)`, not a boolean.
|
||||
- MIPROv2 metrics may return `true`/`false`, a numeric score, or `DSPy::Prediction`.
|
||||
@@ -1,418 +0,0 @@
|
||||
# DSPy.rb LLM Providers
|
||||
|
||||
## Adapter Architecture
|
||||
|
||||
DSPy.rb ships provider SDKs as separate adapter gems. Install only the adapters the project needs. Each adapter gem depends on the official SDK for its provider and auto-loads when present -- no explicit `require` necessary.
|
||||
|
||||
```ruby
|
||||
# Gemfile
|
||||
gem 'dspy' # core framework (no provider SDKs)
|
||||
gem 'dspy-openai' # OpenAI, OpenRouter, Ollama
|
||||
gem 'dspy-anthropic' # Claude
|
||||
gem 'dspy-gemini' # Gemini
|
||||
gem 'dspy-ruby_llm' # RubyLLM unified adapter (12+ providers)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Per-Provider Adapters
|
||||
|
||||
### dspy-openai
|
||||
|
||||
Covers any endpoint that speaks the OpenAI chat-completions protocol: OpenAI itself, OpenRouter, and Ollama.
|
||||
|
||||
**SDK dependency:** `openai ~> 0.17`
|
||||
|
||||
```ruby
|
||||
# OpenAI
|
||||
lm = DSPy::LM.new('openai/gpt-4o-mini', api_key: ENV['OPENAI_API_KEY'])
|
||||
|
||||
# OpenRouter -- access 200+ models behind a single key
|
||||
lm = DSPy::LM.new('openrouter/x-ai/grok-4-fast:free',
|
||||
api_key: ENV['OPENROUTER_API_KEY']
|
||||
)
|
||||
|
||||
# Ollama -- local models, no API key required
|
||||
lm = DSPy::LM.new('ollama/llama3.2')
|
||||
|
||||
# Remote Ollama instance
|
||||
lm = DSPy::LM.new('ollama/llama3.2',
|
||||
base_url: 'https://my-ollama.example.com/v1',
|
||||
api_key: 'optional-auth-token'
|
||||
)
|
||||
```
|
||||
|
||||
All three sub-adapters share the same request handling, structured-output support, and error reporting. Swap providers without changing higher-level DSPy code.
|
||||
|
||||
For OpenRouter models that lack native structured-output support, disable it explicitly:
|
||||
|
||||
```ruby
|
||||
lm = DSPy::LM.new('openrouter/deepseek/deepseek-chat-v3.1:free',
|
||||
api_key: ENV['OPENROUTER_API_KEY'],
|
||||
structured_outputs: false
|
||||
)
|
||||
```
|
||||
|
||||
### dspy-anthropic
|
||||
|
||||
Provides the Claude adapter. Install it for any `anthropic/*` model id.
|
||||
|
||||
**SDK dependency:** `anthropic ~> 1.12`
|
||||
|
||||
```ruby
|
||||
lm = DSPy::LM.new('anthropic/claude-sonnet-4-20250514',
|
||||
api_key: ENV['ANTHROPIC_API_KEY']
|
||||
)
|
||||
```
|
||||
|
||||
Structured outputs default to tool-based JSON extraction (`structured_outputs: true`). Set `structured_outputs: false` to use enhanced-prompting extraction instead.
|
||||
|
||||
```ruby
|
||||
# Tool-based extraction (default, most reliable)
|
||||
lm = DSPy::LM.new('anthropic/claude-sonnet-4-20250514',
|
||||
api_key: ENV['ANTHROPIC_API_KEY'],
|
||||
structured_outputs: true
|
||||
)
|
||||
|
||||
# Enhanced prompting extraction
|
||||
lm = DSPy::LM.new('anthropic/claude-sonnet-4-20250514',
|
||||
api_key: ENV['ANTHROPIC_API_KEY'],
|
||||
structured_outputs: false
|
||||
)
|
||||
```
|
||||
|
||||
### dspy-gemini
|
||||
|
||||
Provides the Gemini adapter. Install it for any `gemini/*` model id.
|
||||
|
||||
**SDK dependency:** `gemini-ai ~> 4.3`
|
||||
|
||||
```ruby
|
||||
lm = DSPy::LM.new('gemini/gemini-2.5-flash',
|
||||
api_key: ENV['GEMINI_API_KEY']
|
||||
)
|
||||
```
|
||||
|
||||
**Environment variable:** `GEMINI_API_KEY` (also accepts `GOOGLE_API_KEY`).
|
||||
|
||||
---
|
||||
|
||||
## RubyLLM Unified Adapter
|
||||
|
||||
The `dspy-ruby_llm` gem provides a single adapter that routes to 12+ providers through [RubyLLM](https://rubyllm.com). Use it when a project talks to multiple providers or needs access to Bedrock, VertexAI, DeepSeek, or Mistral without dedicated adapter gems.
|
||||
|
||||
**SDK dependency:** `ruby_llm ~> 1.3`
|
||||
|
||||
### Model ID Format
|
||||
|
||||
Prefix every model id with `ruby_llm/`:
|
||||
|
||||
```ruby
|
||||
lm = DSPy::LM.new('ruby_llm/gpt-4o-mini')
|
||||
lm = DSPy::LM.new('ruby_llm/claude-sonnet-4-20250514')
|
||||
lm = DSPy::LM.new('ruby_llm/gemini-2.5-flash')
|
||||
```
|
||||
|
||||
The adapter detects the provider from RubyLLM's model registry automatically. For models not in the registry, pass `provider:` explicitly:
|
||||
|
||||
```ruby
|
||||
lm = DSPy::LM.new('ruby_llm/llama3.2', provider: 'ollama')
|
||||
lm = DSPy::LM.new('ruby_llm/anthropic/claude-3-opus',
|
||||
api_key: ENV['OPENROUTER_API_KEY'],
|
||||
provider: 'openrouter'
|
||||
)
|
||||
```
|
||||
|
||||
### Using Existing RubyLLM Configuration
|
||||
|
||||
When RubyLLM is already configured globally, omit the `api_key:` argument. DSPy reuses the global config automatically:
|
||||
|
||||
```ruby
|
||||
RubyLLM.configure do |config|
|
||||
config.openai_api_key = ENV['OPENAI_API_KEY']
|
||||
config.anthropic_api_key = ENV['ANTHROPIC_API_KEY']
|
||||
end
|
||||
|
||||
# No api_key needed -- picks up the global config
|
||||
DSPy.configure do |c|
|
||||
c.lm = DSPy::LM.new('ruby_llm/gpt-4o-mini')
|
||||
end
|
||||
```
|
||||
|
||||
When an `api_key:` (or any of `base_url:`, `timeout:`, `max_retries:`) is passed, DSPy creates a **scoped context** instead of reusing the global config.
|
||||
|
||||
### Cloud-Hosted Providers (Bedrock, VertexAI)
|
||||
|
||||
Configure RubyLLM globally first, then reference the model:
|
||||
|
||||
```ruby
|
||||
# AWS Bedrock
|
||||
RubyLLM.configure do |c|
|
||||
c.bedrock_api_key = ENV['AWS_ACCESS_KEY_ID']
|
||||
c.bedrock_secret_key = ENV['AWS_SECRET_ACCESS_KEY']
|
||||
c.bedrock_region = 'us-east-1'
|
||||
end
|
||||
lm = DSPy::LM.new('ruby_llm/anthropic.claude-3-5-sonnet', provider: 'bedrock')
|
||||
|
||||
# Google VertexAI
|
||||
RubyLLM.configure do |c|
|
||||
c.vertexai_project_id = 'your-project-id'
|
||||
c.vertexai_location = 'us-central1'
|
||||
end
|
||||
lm = DSPy::LM.new('ruby_llm/gemini-pro', provider: 'vertexai')
|
||||
```
|
||||
|
||||
### Supported Providers Table
|
||||
|
||||
| Provider | Example Model ID | Notes |
|
||||
|-------------|--------------------------------------------|---------------------------------|
|
||||
| OpenAI | `ruby_llm/gpt-4o-mini` | Auto-detected from registry |
|
||||
| Anthropic | `ruby_llm/claude-sonnet-4-20250514` | Auto-detected from registry |
|
||||
| Gemini | `ruby_llm/gemini-2.5-flash` | Auto-detected from registry |
|
||||
| DeepSeek | `ruby_llm/deepseek-chat` | Auto-detected from registry |
|
||||
| Mistral | `ruby_llm/mistral-large` | Auto-detected from registry |
|
||||
| Ollama | `ruby_llm/llama3.2` | Use `provider: 'ollama'` |
|
||||
| AWS Bedrock | `ruby_llm/anthropic.claude-3-5-sonnet` | Configure RubyLLM globally |
|
||||
| VertexAI | `ruby_llm/gemini-pro` | Configure RubyLLM globally |
|
||||
| OpenRouter | `ruby_llm/anthropic/claude-3-opus` | Use `provider: 'openrouter'` |
|
||||
| Perplexity | `ruby_llm/llama-3.1-sonar-large` | Use `provider: 'perplexity'` |
|
||||
| GPUStack | `ruby_llm/model-name` | Use `provider: 'gpustack'` |
|
||||
|
||||
---
|
||||
|
||||
## Rails Initializer Pattern
|
||||
|
||||
Configure DSPy inside an `after_initialize` block so Rails credentials and environment are fully loaded:
|
||||
|
||||
```ruby
|
||||
# config/initializers/dspy.rb
|
||||
Rails.application.config.after_initialize do
|
||||
return if Rails.env.test? # skip in test -- use VCR cassettes instead
|
||||
|
||||
DSPy.configure do |config|
|
||||
config.lm = DSPy::LM.new(
|
||||
'openai/gpt-4o-mini',
|
||||
api_key: Rails.application.credentials.openai_api_key,
|
||||
structured_outputs: true
|
||||
)
|
||||
|
||||
config.logger = if Rails.env.production?
|
||||
Dry.Logger(:dspy, formatter: :json) do |logger|
|
||||
logger.add_backend(stream: Rails.root.join("log/dspy.log"))
|
||||
end
|
||||
else
|
||||
Dry.Logger(:dspy) do |logger|
|
||||
logger.add_backend(level: :debug, stream: $stdout)
|
||||
end
|
||||
end
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
Key points:
|
||||
|
||||
- Wrap in `after_initialize` so `Rails.application.credentials` is available.
|
||||
- Return early in the test environment. Rely on VCR cassettes for deterministic LLM responses.
|
||||
- Set `structured_outputs: true` (the default) for provider-native JSON extraction.
|
||||
- Use `Dry.Logger` with `:json` formatter in production for structured log parsing.
|
||||
|
||||
---
|
||||
|
||||
## Fiber-Local LM Context
|
||||
|
||||
`DSPy.with_lm` sets a temporary language-model override scoped to the current Fiber. Every predictor call inside the block uses the override; outside the block the previous LM takes effect again.
|
||||
|
||||
```ruby
|
||||
fast = DSPy::LM.new('openai/gpt-4o-mini', api_key: ENV['OPENAI_API_KEY'])
|
||||
powerful = DSPy::LM.new('anthropic/claude-sonnet-4-20250514', api_key: ENV['ANTHROPIC_API_KEY'])
|
||||
|
||||
classifier = Classifier.new
|
||||
|
||||
# Uses the global LM
|
||||
result = classifier.call(text: "Hello")
|
||||
|
||||
# Temporarily switch to the fast model
|
||||
DSPy.with_lm(fast) do
|
||||
result = classifier.call(text: "Hello") # uses gpt-4o-mini
|
||||
end
|
||||
|
||||
# Temporarily switch to the powerful model
|
||||
DSPy.with_lm(powerful) do
|
||||
result = classifier.call(text: "Hello") # uses claude-sonnet-4
|
||||
end
|
||||
```
|
||||
|
||||
### LM Resolution Hierarchy
|
||||
|
||||
DSPy resolves the active language model in this order:
|
||||
|
||||
1. **Instance-level LM** -- set directly on a module instance via `configure`
|
||||
2. **Fiber-local LM** -- set via `DSPy.with_lm`
|
||||
3. **Global LM** -- set via `DSPy.configure`
|
||||
|
||||
Instance-level configuration always wins, even inside a `DSPy.with_lm` block:
|
||||
|
||||
```ruby
|
||||
classifier = Classifier.new
|
||||
classifier.configure { |c| c.lm = DSPy::LM.new('anthropic/claude-sonnet-4-20250514', api_key: ENV['ANTHROPIC_API_KEY']) }
|
||||
|
||||
fast = DSPy::LM.new('openai/gpt-4o-mini', api_key: ENV['OPENAI_API_KEY'])
|
||||
|
||||
DSPy.with_lm(fast) do
|
||||
classifier.call(text: "Test") # still uses claude-sonnet-4 (instance-level wins)
|
||||
end
|
||||
```
|
||||
|
||||
### configure_predictor for Fine-Grained Agent Control
|
||||
|
||||
Complex agents (`ReAct`, `CodeAct`, `DeepResearch`, `DeepSearch`) contain internal predictors. Use `configure` for a blanket override and `configure_predictor` to target a specific sub-predictor:
|
||||
|
||||
```ruby
|
||||
agent = DSPy::ReAct.new(MySignature, tools: tools)
|
||||
|
||||
# Set a default LM for the agent and all its children
|
||||
agent.configure { |c| c.lm = DSPy::LM.new('openai/gpt-4o-mini', api_key: ENV['OPENAI_API_KEY']) }
|
||||
|
||||
# Override just the reasoning predictor with a more capable model
|
||||
agent.configure_predictor('thought_generator') do |c|
|
||||
c.lm = DSPy::LM.new('anthropic/claude-sonnet-4-20250514', api_key: ENV['ANTHROPIC_API_KEY'])
|
||||
end
|
||||
|
||||
result = agent.call(question: "Summarize the report")
|
||||
```
|
||||
|
||||
Both methods support chaining:
|
||||
|
||||
```ruby
|
||||
agent
|
||||
.configure { |c| c.lm = cheap_model }
|
||||
.configure_predictor('thought_generator') { |c| c.lm = expensive_model }
|
||||
```
|
||||
|
||||
#### Available Predictors by Agent Type
|
||||
|
||||
| Agent | Internal Predictors |
|
||||
|----------------------|------------------------------------------------------------------|
|
||||
| `DSPy::ReAct` | `thought_generator`, `observation_processor` |
|
||||
| `DSPy::CodeAct` | `code_generator`, `observation_processor` |
|
||||
| `DSPy::DeepResearch` | `planner`, `synthesizer`, `qa_reviewer`, `reporter` |
|
||||
| `DSPy::DeepSearch` | `seed_predictor`, `search_predictor`, `reader_predictor`, `reason_predictor` |
|
||||
|
||||
#### Propagation Rules
|
||||
|
||||
- Configuration propagates recursively to children and grandchildren.
|
||||
- Children with an already-configured LM are **not** overwritten by a later parent `configure` call.
|
||||
- Configure the parent first, then override specific children.
|
||||
|
||||
---
|
||||
|
||||
## Feature-Flagged Model Selection
|
||||
|
||||
Use a `FeatureFlags` module backed by ENV vars to centralize model selection. Each tool or agent reads its model from the flags, falling back to a global default.
|
||||
|
||||
```ruby
|
||||
module FeatureFlags
|
||||
module_function
|
||||
|
||||
def default_model
|
||||
ENV.fetch('DSPY_DEFAULT_MODEL', 'openai/gpt-4o-mini')
|
||||
end
|
||||
|
||||
def default_api_key
|
||||
ENV.fetch('DSPY_DEFAULT_API_KEY') { ENV.fetch('OPENAI_API_KEY', nil) }
|
||||
end
|
||||
|
||||
def model_for(tool_name)
|
||||
env_key = "DSPY_MODEL_#{tool_name.upcase}"
|
||||
ENV.fetch(env_key, default_model)
|
||||
end
|
||||
|
||||
def api_key_for(tool_name)
|
||||
env_key = "DSPY_API_KEY_#{tool_name.upcase}"
|
||||
ENV.fetch(env_key, default_api_key)
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
### Per-Tool Model Override
|
||||
|
||||
Override an individual tool's model without touching application code:
|
||||
|
||||
```bash
|
||||
# .env
|
||||
DSPY_DEFAULT_MODEL=openai/gpt-4o-mini
|
||||
DSPY_DEFAULT_API_KEY=sk-...
|
||||
|
||||
# Override the classifier to use Claude
|
||||
DSPY_MODEL_CLASSIFIER=anthropic/claude-sonnet-4-20250514
|
||||
DSPY_API_KEY_CLASSIFIER=sk-ant-...
|
||||
|
||||
# Override the summarizer to use Gemini
|
||||
DSPY_MODEL_SUMMARIZER=gemini/gemini-2.5-flash
|
||||
DSPY_API_KEY_SUMMARIZER=...
|
||||
```
|
||||
|
||||
Wire each agent to its flag at initialization:
|
||||
|
||||
```ruby
|
||||
class ClassifierAgent < DSPy::Module
|
||||
def initialize
|
||||
super
|
||||
model = FeatureFlags.model_for('classifier')
|
||||
api_key = FeatureFlags.api_key_for('classifier')
|
||||
|
||||
@predictor = DSPy::Predict.new(ClassifySignature)
|
||||
configure { |c| c.lm = DSPy::LM.new(model, api_key: api_key) }
|
||||
end
|
||||
|
||||
def forward(text:)
|
||||
@predictor.call(text: text)
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
This pattern keeps model routing declarative and avoids scattering `DSPy::LM.new` calls across the codebase.
|
||||
|
||||
---
|
||||
|
||||
## Compatibility Matrix
|
||||
|
||||
Feature support across direct adapter gems. All features listed assume `structured_outputs: true` (the default).
|
||||
|
||||
| Feature | OpenAI | Anthropic | Gemini | Ollama | OpenRouter | RubyLLM |
|
||||
|----------------------|--------|-----------|--------|----------|------------|-------------|
|
||||
| Structured Output | Native JSON mode | Tool-based extraction | Native JSON schema | OpenAI-compatible JSON | Varies by model | Via `with_schema` |
|
||||
| Vision (Images) | File + URL | File + Base64 | File + Base64 | Limited | Varies | Delegates to underlying provider |
|
||||
| Image URLs | Yes | No | No | No | Varies | Depends on provider |
|
||||
| Tool Calling | Yes | Yes | Yes | Varies | Varies | Yes |
|
||||
| Streaming | Yes | Yes | Yes | Yes | Yes | Yes |
|
||||
|
||||
**Notes:**
|
||||
|
||||
- **Structured Output** is enabled by default on every adapter. Set `structured_outputs: false` to fall back to enhanced-prompting extraction.
|
||||
- **Vision / Image URLs:** Only OpenAI supports passing a URL directly. For Anthropic and Gemini, load images from file or Base64:
|
||||
```ruby
|
||||
DSPy::Image.from_url("https://example.com/img.jpg") # OpenAI only
|
||||
DSPy::Image.from_file("path/to/image.jpg") # all providers
|
||||
DSPy::Image.from_base64(data, mime_type: "image/jpeg") # all providers
|
||||
```
|
||||
- **RubyLLM** delegates to the underlying provider, so feature support matches the provider column in the table.
|
||||
|
||||
### Choosing an Adapter Strategy
|
||||
|
||||
| Scenario | Recommended Adapter |
|
||||
|-------------------------------------------|--------------------------------|
|
||||
| Single provider (OpenAI, Claude, or Gemini) | Dedicated gem (`dspy-openai`, `dspy-anthropic`, `dspy-gemini`) |
|
||||
| Multi-provider with per-agent model routing | `dspy-ruby_llm` |
|
||||
| AWS Bedrock or Google VertexAI | `dspy-ruby_llm` |
|
||||
| Local development with Ollama | `dspy-openai` (Ollama sub-adapter) or `dspy-ruby_llm` |
|
||||
| OpenRouter for cost optimization | `dspy-openai` (OpenRouter sub-adapter) |
|
||||
|
||||
### Current Recommended Models
|
||||
|
||||
| Provider | Model ID | Use Case |
|
||||
|-----------|---------------------------------------|-----------------------|
|
||||
| OpenAI | `openai/gpt-4o-mini` | Fast, cost-effective |
|
||||
| Anthropic | `anthropic/claude-sonnet-4-20250514` | Balanced reasoning |
|
||||
| Gemini | `gemini/gemini-2.5-flash` | Fast, cost-effective |
|
||||
| Ollama | `ollama/llama3.2` | Local, zero API cost |
|
||||
@@ -1,502 +0,0 @@
|
||||
# DSPy.rb Toolsets
|
||||
|
||||
## Tools::Base
|
||||
|
||||
`DSPy::Tools::Base` is the base class for single-purpose tools. Each subclass exposes one operation to an LLM agent through a `call` method.
|
||||
|
||||
### Defining a Tool
|
||||
|
||||
Set the tool's identity with the `tool_name` and `tool_description` class-level DSL methods. Define the `call` instance method with a Sorbet `sig` declaration so DSPy.rb can generate the JSON schema the LLM uses to invoke the tool.
|
||||
|
||||
```ruby
|
||||
class WeatherLookup < DSPy::Tools::Base
|
||||
extend T::Sig
|
||||
|
||||
tool_name "weather_lookup"
|
||||
tool_description "Look up current weather for a given city"
|
||||
|
||||
sig { params(city: String, units: T.nilable(String)).returns(String) }
|
||||
def call(city:, units: nil)
|
||||
# Fetch weather data and return a string summary
|
||||
"72F and sunny in #{city}"
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
Key points:
|
||||
|
||||
- Inherit from `DSPy::Tools::Base`, not `DSPy::Tool`.
|
||||
- Use `tool_name` (class method) to set the name the LLM sees. Without it, the class name is lowercased as a fallback.
|
||||
- Use `tool_description` (class method) to set the human-readable description surfaced in the tool schema.
|
||||
- The `call` method must use **keyword arguments**. Positional arguments are supported but keyword arguments produce better schemas.
|
||||
- Always attach a Sorbet `sig` to `call`. Without a signature, the generated schema has empty properties and the LLM cannot determine parameter types.
|
||||
|
||||
### Schema Generation
|
||||
|
||||
`call_schema_object` introspects the Sorbet signature on `call` and returns a hash representing the JSON Schema `parameters` object:
|
||||
|
||||
```ruby
|
||||
WeatherLookup.call_schema_object
|
||||
# => {
|
||||
# type: "object",
|
||||
# properties: {
|
||||
# city: { type: "string", description: "Parameter city" },
|
||||
# units: { type: "string", description: "Parameter units (optional)" }
|
||||
# },
|
||||
# required: ["city"]
|
||||
# }
|
||||
```
|
||||
|
||||
`call_schema` wraps this in the full LLM tool-calling format:
|
||||
|
||||
```ruby
|
||||
WeatherLookup.call_schema
|
||||
# => {
|
||||
# type: "function",
|
||||
# function: {
|
||||
# name: "call",
|
||||
# description: "Call the WeatherLookup tool",
|
||||
# parameters: { ... }
|
||||
# }
|
||||
# }
|
||||
```
|
||||
|
||||
### Using Tools with ReAct
|
||||
|
||||
Pass tool instances in an array to `DSPy::ReAct`:
|
||||
|
||||
```ruby
|
||||
agent = DSPy::ReAct.new(
|
||||
MySignature,
|
||||
tools: [WeatherLookup.new, AnotherTool.new]
|
||||
)
|
||||
|
||||
result = agent.call(question: "What is the weather in Berlin?")
|
||||
puts result.answer
|
||||
```
|
||||
|
||||
Access output fields with dot notation (`result.answer`), not hash access (`result[:answer]`).
|
||||
|
||||
---
|
||||
|
||||
## Tools::Toolset
|
||||
|
||||
`DSPy::Tools::Toolset` groups multiple related methods into a single class. Each exposed method becomes an independent tool from the LLM's perspective.
|
||||
|
||||
### Defining a Toolset
|
||||
|
||||
```ruby
|
||||
class DatabaseToolset < DSPy::Tools::Toolset
|
||||
extend T::Sig
|
||||
|
||||
toolset_name "db"
|
||||
|
||||
tool :query, description: "Run a read-only SQL query"
|
||||
tool :insert, description: "Insert a record into a table"
|
||||
tool :delete, description: "Delete a record by ID"
|
||||
|
||||
sig { params(sql: String).returns(String) }
|
||||
def query(sql:)
|
||||
# Execute read query
|
||||
end
|
||||
|
||||
sig { params(table: String, data: T::Hash[String, String]).returns(String) }
|
||||
def insert(table:, data:)
|
||||
# Insert record
|
||||
end
|
||||
|
||||
sig { params(table: String, id: Integer).returns(String) }
|
||||
def delete(table:, id:)
|
||||
# Delete record
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
### DSL Methods
|
||||
|
||||
**`toolset_name(name)`** -- Set the prefix for all generated tool names. If omitted, the class name minus `Toolset` suffix is lowercased (e.g., `DatabaseToolset` becomes `database`).
|
||||
|
||||
```ruby
|
||||
toolset_name "db"
|
||||
# tool :query produces a tool named "db_query"
|
||||
```
|
||||
|
||||
**`tool(method_name, tool_name:, description:)`** -- Expose a method as a tool.
|
||||
|
||||
- `method_name` (Symbol, required) -- the instance method to expose.
|
||||
- `tool_name:` (String, optional) -- override the default `<toolset_name>_<method_name>` naming.
|
||||
- `description:` (String, optional) -- description shown to the LLM. Defaults to a humanized version of the method name.
|
||||
|
||||
```ruby
|
||||
tool :word_count, tool_name: "text_wc", description: "Count lines, words, and characters"
|
||||
# Produces a tool named "text_wc" instead of "text_word_count"
|
||||
```
|
||||
|
||||
### Converting to a Tool Array
|
||||
|
||||
Call `to_tools` on the class (not an instance) to get an array of `ToolProxy` objects compatible with `DSPy::Tools::Base`:
|
||||
|
||||
```ruby
|
||||
agent = DSPy::ReAct.new(
|
||||
AnalyzeText,
|
||||
tools: DatabaseToolset.to_tools
|
||||
)
|
||||
```
|
||||
|
||||
Each `ToolProxy` wraps one method, delegates `call` to the underlying toolset instance, and generates its own JSON schema from the method's Sorbet signature.
|
||||
|
||||
### Shared State
|
||||
|
||||
All tool proxies from a single `to_tools` call share one toolset instance. Store shared state (connections, caches, configuration) in the toolset's `initialize`:
|
||||
|
||||
```ruby
|
||||
class ApiToolset < DSPy::Tools::Toolset
|
||||
extend T::Sig
|
||||
|
||||
toolset_name "api"
|
||||
|
||||
tool :get, description: "Make a GET request"
|
||||
tool :post, description: "Make a POST request"
|
||||
|
||||
sig { params(base_url: String).void }
|
||||
def initialize(base_url:)
|
||||
@base_url = base_url
|
||||
@client = HTTP.persistent(base_url)
|
||||
end
|
||||
|
||||
sig { params(path: String).returns(String) }
|
||||
def get(path:)
|
||||
@client.get("#{@base_url}#{path}").body.to_s
|
||||
end
|
||||
|
||||
sig { params(path: String, body: String).returns(String) }
|
||||
def post(path:, body:)
|
||||
@client.post("#{@base_url}#{path}", body: body).body.to_s
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Type Safety
|
||||
|
||||
Sorbet signatures on tool methods drive both JSON schema generation and automatic type coercion of LLM responses.
|
||||
|
||||
### Basic Types
|
||||
|
||||
```ruby
|
||||
sig { params(
|
||||
text: String,
|
||||
count: Integer,
|
||||
score: Float,
|
||||
enabled: T::Boolean,
|
||||
threshold: Numeric
|
||||
).returns(String) }
|
||||
def analyze(text:, count:, score:, enabled:, threshold:)
|
||||
# ...
|
||||
end
|
||||
```
|
||||
|
||||
| Sorbet Type | JSON Schema |
|
||||
|------------------|----------------------------------------------------|
|
||||
| `String` | `{"type": "string"}` |
|
||||
| `Integer` | `{"type": "integer"}` |
|
||||
| `Float` | `{"type": "number"}` |
|
||||
| `Numeric` | `{"type": "number"}` |
|
||||
| `T::Boolean` | `{"type": "boolean"}` |
|
||||
| `T::Enum` | `{"type": "string", "enum": [...]}` |
|
||||
| `T::Struct` | `{"type": "object", "properties": {...}}` |
|
||||
| `T::Array[Type]` | `{"type": "array", "items": {...}}` |
|
||||
| `T::Hash[K, V]` | `{"type": "object", "additionalProperties": {...}}`|
|
||||
| `T.nilable(Type)`| `{"type": [original, "null"]}` |
|
||||
| `T.any(T1, T2)` | `{"oneOf": [{...}, {...}]}` |
|
||||
| `T.class_of(X)` | `{"type": "string"}` |
|
||||
|
||||
### T::Enum Parameters
|
||||
|
||||
Define a `T::Enum` and reference it in a tool signature. DSPy.rb generates a JSON Schema `enum` constraint and automatically deserializes the LLM's string response into the correct enum instance.
|
||||
|
||||
```ruby
|
||||
class Priority < T::Enum
|
||||
enums do
|
||||
Low = new('low')
|
||||
Medium = new('medium')
|
||||
High = new('high')
|
||||
Critical = new('critical')
|
||||
end
|
||||
end
|
||||
|
||||
class Status < T::Enum
|
||||
enums do
|
||||
Pending = new('pending')
|
||||
InProgress = new('in-progress')
|
||||
Completed = new('completed')
|
||||
end
|
||||
end
|
||||
|
||||
sig { params(priority: Priority, status: Status).returns(String) }
|
||||
def update_task(priority:, status:)
|
||||
"Updated to #{priority.serialize} / #{status.serialize}"
|
||||
end
|
||||
```
|
||||
|
||||
The generated schema constrains the parameter to valid values:
|
||||
|
||||
```json
|
||||
{
|
||||
"priority": {
|
||||
"type": "string",
|
||||
"enum": ["low", "medium", "high", "critical"]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Case-insensitive matching**: When the LLM returns `"HIGH"` or `"High"` instead of `"high"`, DSPy.rb first tries an exact `try_deserialize`, then falls back to a case-insensitive lookup. This prevents failures caused by LLM casing variations.
|
||||
|
||||
### T::Struct Parameters
|
||||
|
||||
Use `T::Struct` for complex nested objects. DSPy.rb generates nested JSON Schema properties and recursively coerces the LLM's hash response into struct instances.
|
||||
|
||||
```ruby
|
||||
class TaskMetadata < T::Struct
|
||||
prop :id, String
|
||||
prop :priority, Priority
|
||||
prop :tags, T::Array[String]
|
||||
prop :estimated_hours, T.nilable(Float), default: nil
|
||||
end
|
||||
|
||||
class TaskRequest < T::Struct
|
||||
prop :title, String
|
||||
prop :description, String
|
||||
prop :status, Status
|
||||
prop :metadata, TaskMetadata
|
||||
prop :assignees, T::Array[String]
|
||||
end
|
||||
|
||||
sig { params(task: TaskRequest).returns(String) }
|
||||
def create_task(task:)
|
||||
"Created: #{task.title} (#{task.status.serialize})"
|
||||
end
|
||||
```
|
||||
|
||||
The LLM sees the full nested object schema and DSPy.rb reconstructs the struct tree from the JSON response, including enum fields inside nested structs.
|
||||
|
||||
### Nilable Parameters
|
||||
|
||||
Mark optional parameters with `T.nilable(...)` and provide a default value of `nil` in the method signature. These parameters are excluded from the JSON Schema `required` array.
|
||||
|
||||
```ruby
|
||||
sig { params(
|
||||
query: String,
|
||||
max_results: T.nilable(Integer),
|
||||
filter: T.nilable(String)
|
||||
).returns(String) }
|
||||
def search(query:, max_results: nil, filter: nil)
|
||||
# query is required; max_results and filter are optional
|
||||
end
|
||||
```
|
||||
|
||||
### Collections
|
||||
|
||||
Typed arrays and hashes generate precise item/value schemas:
|
||||
|
||||
```ruby
|
||||
sig { params(
|
||||
tags: T::Array[String],
|
||||
priorities: T::Array[Priority],
|
||||
config: T::Hash[String, T.any(String, Integer, Float)]
|
||||
).returns(String) }
|
||||
def configure(tags:, priorities:, config:)
|
||||
# Array elements and hash values are validated and coerced
|
||||
end
|
||||
```
|
||||
|
||||
### Union Types
|
||||
|
||||
`T.any(...)` generates a `oneOf` JSON Schema. When one of the union members is a `T::Struct`, DSPy.rb uses the `_type` discriminator field to select the correct struct class during coercion.
|
||||
|
||||
```ruby
|
||||
sig { params(value: T.any(String, Integer, Float)).returns(String) }
|
||||
def handle_flexible(value:)
|
||||
# Accepts multiple types
|
||||
end
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Built-in Toolsets
|
||||
|
||||
### TextProcessingToolset
|
||||
|
||||
`DSPy::Tools::TextProcessingToolset` provides Unix-style text analysis and manipulation operations. Toolset name prefix: `text`.
|
||||
|
||||
| Tool Name | Method | Description |
|
||||
|-----------------------------------|-------------------|--------------------------------------------|
|
||||
| `text_grep` | `grep` | Search for patterns with optional case-insensitive and count-only modes |
|
||||
| `text_wc` | `word_count` | Count lines, words, and characters |
|
||||
| `text_rg` | `ripgrep` | Fast pattern search with context lines |
|
||||
| `text_extract_lines` | `extract_lines` | Extract a range of lines by number |
|
||||
| `text_filter_lines` | `filter_lines` | Keep or reject lines matching a regex |
|
||||
| `text_unique_lines` | `unique_lines` | Deduplicate lines, optionally preserving order |
|
||||
| `text_sort_lines` | `sort_lines` | Sort lines alphabetically or numerically |
|
||||
| `text_summarize_text` | `summarize_text` | Produce a statistical summary (counts, averages, frequent words) |
|
||||
|
||||
Usage:
|
||||
|
||||
```ruby
|
||||
agent = DSPy::ReAct.new(
|
||||
AnalyzeText,
|
||||
tools: DSPy::Tools::TextProcessingToolset.to_tools
|
||||
)
|
||||
|
||||
result = agent.call(text: log_contents, question: "How many error lines are there?")
|
||||
puts result.answer
|
||||
```
|
||||
|
||||
### GitHubCLIToolset
|
||||
|
||||
`DSPy::Tools::GitHubCLIToolset` wraps the `gh` CLI for read-oriented GitHub operations. Toolset name prefix: `github`.
|
||||
|
||||
| Tool Name | Method | Description |
|
||||
|------------------------|-------------------|---------------------------------------------------|
|
||||
| `github_list_issues` | `list_issues` | List issues filtered by state, labels, assignee |
|
||||
| `github_list_prs` | `list_prs` | List pull requests filtered by state, author, base|
|
||||
| `github_get_issue` | `get_issue` | Retrieve details of a single issue |
|
||||
| `github_get_pr` | `get_pr` | Retrieve details of a single pull request |
|
||||
| `github_api_request` | `api_request` | Make an arbitrary GET request to the GitHub API |
|
||||
| `github_traffic_views` | `traffic_views` | Fetch repository traffic view counts |
|
||||
| `github_traffic_clones`| `traffic_clones` | Fetch repository traffic clone counts |
|
||||
|
||||
This toolset uses `T::Enum` parameters (`IssueState`, `PRState`, `ReviewState`) for state filters, demonstrating enum-based tool signatures in practice.
|
||||
|
||||
```ruby
|
||||
agent = DSPy::ReAct.new(
|
||||
RepoAnalysis,
|
||||
tools: DSPy::Tools::GitHubCLIToolset.to_tools
|
||||
)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Testing
|
||||
|
||||
### Unit Testing Individual Tools
|
||||
|
||||
Test `DSPy::Tools::Base` subclasses by instantiating and calling `call` directly:
|
||||
|
||||
```ruby
|
||||
RSpec.describe WeatherLookup do
|
||||
subject(:tool) { described_class.new }
|
||||
|
||||
it "returns weather for a city" do
|
||||
result = tool.call(city: "Berlin")
|
||||
expect(result).to include("Berlin")
|
||||
end
|
||||
|
||||
it "exposes the correct tool name" do
|
||||
expect(tool.name).to eq("weather_lookup")
|
||||
end
|
||||
|
||||
it "generates a valid schema" do
|
||||
schema = described_class.call_schema_object
|
||||
expect(schema[:required]).to include("city")
|
||||
expect(schema[:properties]).to have_key(:city)
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
### Unit Testing Toolsets
|
||||
|
||||
Test toolset methods directly on an instance. Verify tool generation with `to_tools`:
|
||||
|
||||
```ruby
|
||||
RSpec.describe DatabaseToolset do
|
||||
subject(:toolset) { described_class.new }
|
||||
|
||||
it "executes a query" do
|
||||
result = toolset.query(sql: "SELECT 1")
|
||||
expect(result).to be_a(String)
|
||||
end
|
||||
|
||||
it "generates tools with correct names" do
|
||||
tools = described_class.to_tools
|
||||
names = tools.map(&:name)
|
||||
expect(names).to contain_exactly("db_query", "db_insert", "db_delete")
|
||||
end
|
||||
|
||||
it "generates tool descriptions" do
|
||||
tools = described_class.to_tools
|
||||
query_tool = tools.find { |t| t.name == "db_query" }
|
||||
expect(query_tool.description).to eq("Run a read-only SQL query")
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
### Mocking Predictions Inside Tools
|
||||
|
||||
When a tool calls a DSPy predictor internally, stub the predictor to isolate tool logic from LLM calls:
|
||||
|
||||
```ruby
|
||||
class SmartSearchTool < DSPy::Tools::Base
|
||||
extend T::Sig
|
||||
|
||||
tool_name "smart_search"
|
||||
tool_description "Search with query expansion"
|
||||
|
||||
sig { void }
|
||||
def initialize
|
||||
@expander = DSPy::Predict.new(QueryExpansionSignature)
|
||||
end
|
||||
|
||||
sig { params(query: String).returns(String) }
|
||||
def call(query:)
|
||||
expanded = @expander.call(query: query)
|
||||
perform_search(expanded.expanded_query)
|
||||
end
|
||||
|
||||
private
|
||||
|
||||
def perform_search(query)
|
||||
# actual search logic
|
||||
end
|
||||
end
|
||||
|
||||
RSpec.describe SmartSearchTool do
|
||||
subject(:tool) { described_class.new }
|
||||
|
||||
before do
|
||||
expansion_result = double("result", expanded_query: "expanded test query")
|
||||
allow_any_instance_of(DSPy::Predict).to receive(:call).and_return(expansion_result)
|
||||
end
|
||||
|
||||
it "expands the query before searching" do
|
||||
allow(tool).to receive(:perform_search).with("expanded test query").and_return("found 3 results")
|
||||
result = tool.call(query: "test")
|
||||
expect(result).to eq("found 3 results")
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
### Testing Enum Coercion
|
||||
|
||||
Verify that string values from LLM responses deserialize into the correct enum instances:
|
||||
|
||||
```ruby
|
||||
RSpec.describe "enum coercion" do
|
||||
it "handles case-insensitive enum values" do
|
||||
toolset = GitHubCLIToolset.new
|
||||
# The LLM may return "OPEN" instead of "open"
|
||||
result = toolset.list_issues(state: IssueState::Open)
|
||||
expect(result).to be_a(String)
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Constraints
|
||||
|
||||
- All exposed tool methods must use **keyword arguments**. Positional-only parameters generate schemas but keyword arguments produce more reliable LLM interactions.
|
||||
- Each exposed method becomes a **separate, independent tool**. Method chaining or multi-step sequences within a single tool call are not supported.
|
||||
- Shared state across tool proxies is scoped to a single `to_tools` call. Separate `to_tools` invocations create separate toolset instances.
|
||||
- Methods without a Sorbet `sig` produce an empty parameter schema. The LLM will not know what arguments to pass.
|
||||
@@ -1,135 +0,0 @@
|
||||
---
|
||||
name: ce-every-style-editor
|
||||
description: This skill should be used when reviewing or editing copy to ensure adherence to Every's style guide. It provides a systematic line-by-line review process for grammar, punctuation, mechanics, and style guide compliance.
|
||||
---
|
||||
|
||||
# Every Style Editor
|
||||
|
||||
This skill provides a systematic approach to reviewing copy against Every's comprehensive style guide. It transforms Claude into a meticulous line editor and proofreader specializing in grammar, mechanics, and style guide compliance.
|
||||
|
||||
## When to Use This Skill
|
||||
|
||||
Use this skill when:
|
||||
- Reviewing articles, blog posts, newsletters, or any written content
|
||||
- Ensuring copy follows Every's specific style conventions
|
||||
- Providing feedback on grammar, punctuation, and mechanics
|
||||
- Flagging deviations from the Every style guide
|
||||
- Preparing clean copy for human editorial review
|
||||
|
||||
## Skill Overview
|
||||
|
||||
This skill enables performing a comprehensive review of written content in four phases:
|
||||
|
||||
1. **Initial Assessment** - Understanding context and document type
|
||||
2. **Detailed Line Edit** - Checking every sentence for compliance
|
||||
3. **Mechanical Review** - Verifying formatting and consistency
|
||||
4. **Recommendations** - Providing actionable improvement suggestions
|
||||
|
||||
## How to Use This Skill
|
||||
|
||||
### Step 1: Initial Assessment
|
||||
|
||||
Begin by reading the entire piece to understand:
|
||||
- Document type (article, knowledge base entry, social post, etc.)
|
||||
- Target audience
|
||||
- Overall tone and voice
|
||||
- Content context
|
||||
|
||||
### Step 2: Detailed Line Edit
|
||||
|
||||
Review each paragraph systematically, checking for:
|
||||
- Sentence structure and grammar correctness
|
||||
- Punctuation usage (commas, semicolons, em dashes, etc.)
|
||||
- Capitalization rules (especially job titles, headlines)
|
||||
- Word choice and usage (overused words, passive voice)
|
||||
- Adherence to Every style guide rules
|
||||
|
||||
Reference the complete style guide at `references/EVERY_WRITE_STYLE.md` for specific rules when in doubt.
|
||||
|
||||
### Step 3: Mechanical Review
|
||||
|
||||
Verify:
|
||||
- Spacing and formatting consistency
|
||||
- Style choices applied uniformly throughout
|
||||
- Special elements (lists, quotes, citations)
|
||||
- Proper use of italics and formatting
|
||||
- Number formatting (numerals vs. spelled out)
|
||||
- Link formatting and descriptions
|
||||
|
||||
### Step 4: Output Results
|
||||
|
||||
Present findings using this structure:
|
||||
|
||||
```
|
||||
DOCUMENT REVIEW SUMMARY
|
||||
=====================
|
||||
Document Type: [type]
|
||||
Word Count: [approximate]
|
||||
Overall Assessment: [brief overview]
|
||||
|
||||
ERRORS FOUND: [total number]
|
||||
|
||||
DETAILED CORRECTIONS
|
||||
===================
|
||||
|
||||
[For each error found:]
|
||||
|
||||
**Location**: [Paragraph #, Sentence #]
|
||||
**Issue Type**: [Grammar/Punctuation/Mechanics/Style Guide]
|
||||
**Original**: "[exact text with error]"
|
||||
**Correction**: "[corrected text]"
|
||||
**Rule Reference**: [Specific style guide rule violated]
|
||||
**Explanation**: [Brief explanation of why this is an error]
|
||||
|
||||
---
|
||||
|
||||
RECURRING ISSUES
|
||||
===============
|
||||
[List patterns of errors that appear multiple times]
|
||||
|
||||
STYLE GUIDE COMPLIANCE CHECKLIST
|
||||
==============================
|
||||
✓ [Rule followed correctly]
|
||||
✗ [Rule violated - with count of violations]
|
||||
|
||||
FINAL RECOMMENDATIONS
|
||||
===================
|
||||
[2-3 actionable suggestions for improving the draft]
|
||||
```
|
||||
|
||||
## Style Guide Reference
|
||||
|
||||
The complete Every style guide is at `references/EVERY_WRITE_STYLE.md`. Key areas to focus on:
|
||||
|
||||
- **Quick Rules**: Title case for headlines, sentence case elsewhere
|
||||
- **Tone**: Active voice, avoid overused words (actually, very, just), be specific
|
||||
- **Numbers**: Spell out one through nine; use numerals for 10+
|
||||
- **Punctuation**: Oxford commas, em dashes without spaces, proper quotation mark usage
|
||||
- **Capitalization**: Lowercase job titles, company as singular (it), teams as plural (they)
|
||||
- **Emphasis**: Italics only (no bold for emphasis)
|
||||
- **Links**: 2-4 words, don't say "click here"
|
||||
|
||||
## Key Principles
|
||||
|
||||
- **Be specific**: Always quote the exact text with the error
|
||||
- **Reference rules**: Cite the specific style guide rule for each correction
|
||||
- **Maintain voice**: Preserve the author's voice while correcting errors
|
||||
- **Prioritize clarity**: Focus on changes that improve readability
|
||||
- **Be constructive**: Frame feedback to help writers improve
|
||||
- **Flag ambiguous cases**: When style guide doesn't address an issue, explain options and recommend the clearest choice
|
||||
|
||||
## Common Areas to Focus On
|
||||
|
||||
Based on Every's style guide, pay special attention to:
|
||||
|
||||
- Punctuation (comma usage, semicolons, apostrophes, quotation marks)
|
||||
- Capitalization (proper nouns, titles, sentence starts)
|
||||
- Numbers (when to spell out vs. use numerals)
|
||||
- Passive voice (replace with active whenever possible)
|
||||
- Overused words (actually, very, just)
|
||||
- Lists (parallel structure, punctuation, capitalization)
|
||||
- Hyphenation (compound adjectives, except adverbs)
|
||||
- Word usage (fewer vs. less, they vs. them)
|
||||
- Company references (singular "it", teams as plural "they")
|
||||
- Job title capitalization
|
||||
|
||||
@@ -1,529 +0,0 @@
|
||||
# Every Style Guide
|
||||
|
||||
## Quick-and-dirty Every style guide
|
||||
|
||||
Always use the following style guide, go though the items one by one and suggest edits.
|
||||
|
||||
- **Title case** for headlines, **sentence case** for everything else.
|
||||
- Refer to **companies as singular** ("it" instead of "they" or "them") and teams or people within companies as plural ("they").
|
||||
- Don't overuse "**actually**," "**very**," or "**just**" (they can almost always be deleted).
|
||||
- When linking to another source, **hyperlink** between 2-4 words.
|
||||
- You can generally **cut adverbs**.
|
||||
- Watch out for **passive voice**—use active whenever possible.
|
||||
- Spell out **numbers** one through nine. Spell out a number if it is the first word of a sentence, unless it's a year. Use numerals for numbers 10 and greater.
|
||||
- You may use _italics_ for emphasis, but never **bold** or underline.
|
||||
- **Image credits** in captions are italicized, like this: _Source: X/Name_ (if Twitter), _Source: Website name._
|
||||
- Don't capitalize **job titles**.
|
||||
- **Colons** determine capitalization rules. When a colon introduces an independent clause, the first word of that clause should be capitalized. When a colon introduces a dependent clause, the first word of the clause should not be capitalized.
|
||||
- Use an **Oxford comma** for serialization (x, y, and z).
|
||||
- Use a comma to separate **independent clauses** but not dependent clauses.
|
||||
- Do not use a space after an **ellipsis**.
|
||||
- Use an **em dash** (—) to set off a parenthetical statement. Do not put spaces around an em dash. Generally, don't use em dashes more than twice in a paragraph.
|
||||
- Use **hyphens** in compound adjectives, with the exception of adverbs (i.e., words ending in "ly"). Example: fine-tuned vs. finely tuned.
|
||||
- **Italicize titles** of books, newspapers, periodicals, movies, TV shows, and video games. Do not italicize "the" before _New York Times_ or "magazine" after _New York_.
|
||||
- Identify people by their full names on first mention, last name thereafter. In newsletter and social media communications, use first names rather than last names.
|
||||
- **Percentages** always use numerals, and spell out percent: 7 percent.
|
||||
- **Numbers over three digits** take a comma: 1,000.
|
||||
- Punctuation goes outside of a **parentheses** unless the text in parentheses is a full sentence, or there's a question or exclamation within the parenthetical.
|
||||
- Place periods and commas inside **quotation marks**.
|
||||
- Quotes within quotations should be placed in **single quotation marks** (' ').
|
||||
- If the text preceding a quote **introduces the quote**, include a comma before the quote. If the text before the quote leads directly into the quote, don't include a comma. Capitalize the first letter in the quote when it's a full sentence or when following "said," "says," or other introductory language.
|
||||
- Rather than "above" or "below," use terms like **"earlier," "later," "previously,"** etc.
|
||||
- Rather than "over" or "under," use **"more" or "less"/"fewer"** when referring to numbers or quantities.
|
||||
- Try to avoid slashes (like and/or), and use **hyphens** instead when needed.
|
||||
- **Avoid starting sentences with "This,"** and be specific with what you're referring to.
|
||||
- **Avoid starting sentences with "We have" or "We get,"** and instead, say directly what is happening.
|
||||
- **Avoid cliches or jargon.**
|
||||
- **Write out "times"** when referring to more powerful software: "two times faster." You can write "10x" in reference to the common trope.
|
||||
- Use a **dollar sign** instead of writing out "dollars": $1 billion.
|
||||
- **Identify most people** by company and/or job title: Stripe's Patrick McKenzie. (Exception: Mark Zuckerberg)
|
||||
|
||||
## Our grammar and mechanics
|
||||
|
||||
Every generally follows Merriam-Webster and the AP Stylebook.
|
||||
|
||||
### Abbreviations and acronyms
|
||||
|
||||
#### First Usage Rule
|
||||
|
||||
If there's a chance a reader won't recognize an abbreviation or acronym, then spell it out the first time. When you write out an entity's full name the first time, include an abbreviation in brackets if you plan to use it again: United States Air Force (USAAF). If the abbreviation is more common than the long form, then just use the short form (CMS, DVD, FTP).
|
||||
|
||||
#### Common Abbreviations
|
||||
|
||||
Abbreviate words, phrases, and titles that are almost always abbreviated in English: a.m., p.m., et al., i.e. and e.g. (both of which are followed by a comma), vs., etc.
|
||||
|
||||
#### Established Acronyms
|
||||
|
||||
Abbreviate firmly established shortened forms, acronyms, and similar abbreviations: AI, TV, UK, UN
|
||||
|
||||
#### Punctuation in Abbreviations
|
||||
|
||||
Set most abbreviations without points, though there are some exceptions: U.S.A., U.S., L.A., N.Y.C., D.C.
|
||||
|
||||
#### Plural Abbreviations
|
||||
|
||||
When forming plurals of abbreviations, add an s to those without points, an apostrophe and s to those with points: LLMs, TVs, Ph.D.'s, M.B.A.'s
|
||||
|
||||
#### Specific Abbreviations
|
||||
|
||||
Specific abbreviations: LGBTQIA+
|
||||
|
||||
#### Geography
|
||||
|
||||
Spell out cities and states in full. Include the state when referring to non-major cities or for specificity. Offset the state with commas: They were born in Paris, Texas, and moved to San Francisco in 1995.
|
||||
|
||||
#### Time Format
|
||||
|
||||
Spell out the day and the month, and separate them with a comma: Sunday, January 21
|
||||
|
||||
### Ampersands
|
||||
|
||||
#### Usage Rule
|
||||
|
||||
Avoid using them unless they're part of a proper noun or company name. Write out "and" instead. In the event of a joint byline, the same rule applies: She interned for the law firm of Wilson Sonsini Goodrich & Rosati. By Dan Shipper and Evan Armstrong
|
||||
|
||||
### Bold, italics, underline
|
||||
|
||||
#### Emphasis Guidelines
|
||||
|
||||
Italics may be used in rare cases for emphasis, especially if doing so will increase clarity. Bold and underline should not be used for emphasis: Hosting a meeting with all 20 team members *seemed* like a good idea, but the conversation quickly got out of hand.
|
||||
|
||||
### Buttons
|
||||
|
||||
#### Button Text
|
||||
|
||||
Use the sentence case in CTA buttons: Register for the course
|
||||
|
||||
### Bylines
|
||||
|
||||
#### Guest Author Biography
|
||||
|
||||
Pieces written by guest authors include a biography for the author at the bottom of the piece. If a piece was previously published, cite and link to the original source. Use italics: *Leo Polovets is a general partner at [Humba Ventures](https://humbaventures.com/), an early-stage deep tech fund in the Susa Ventures fund family. Before cofounding Susa and Humba, Leo spent 10 years as a software engineer. Previously, he was the second engineering hire at LinkedIn, among other roles. This piece was originally published [in his newsletter](https://www.codingvc.com/p/betting-on-deep-tech).*
|
||||
|
||||
#### Guest Author Introduction
|
||||
|
||||
Pieces written by guest authors also include an introduction from an Every staff member that identifies the author, their background, the subject of the piece, and why we recommend it. The introduction is signed by the staff member who wrote it. Use italics: *When I was coming up in tech, the conventional wisdom was that working at or investing in software companies was a great way to make money, while doing so with companies that took on scientific risk or produced hardware components were a wonderful way to lose every cent to your name. This has always struck me as, you know, wrong, which is why this piece by venture capitalist Leo Polovets resonated with me. He takes a data-driven approach to understanding how deep tech companies can produce superior financial returns. If you're on the fence with your career—perhaps facing temptation to do something relatively safe in B2B SaaS—take this piece as a rational encouragement to dream bigger. —[Evan](https://twitter.com/itsurboyevan)*
|
||||
|
||||
### Capitalization
|
||||
|
||||
#### General Rule
|
||||
|
||||
Use common sense. When in doubt, don't capitalize. Do not capitalize these words: website, internet, online, email, web3, custom instructions
|
||||
|
||||
#### Job Titles
|
||||
|
||||
Do not capitalize job titles, whether on their own or preceding names, unless they're very unusual: He accepted the position of director of business operations. Director of business operations Lucas Crespo manages Every's ad sales. Lucas Crespo, director of business operations, manages Every's ad sales. Chief Happiness Officer
|
||||
|
||||
#### Colons
|
||||
|
||||
Colons (:) determine capitalization rules. When a colon introduces: An independent clause, the first word of that clause should be capitalized. A dependent clause, the first word of the clause should not be capitalized.
|
||||
|
||||
#### Civic Titles
|
||||
|
||||
Capitalize civic titles only when they precede a name and function as a proper title: Secretary of State Antony Blinken. Lowercase such titles when they appear as a common noun: a senator (common noun), Senator Schumer (title preceding name), Chuck Schumer, senator from New York (common noun), New York senator Schumer (common noun used in apposition), the president, President Biden, former president Obama, the mayor, Mayor Adams, New York mayor Eric Adams
|
||||
|
||||
#### Academia
|
||||
|
||||
Capitalize course titles mentioned in text, and don't enclose them in quotation marks: She took Computer Science and Maximize Your Mind With ChatGPT. Lowercase the names of academic disciplines: One job requirement is a master's in computer science.
|
||||
|
||||
#### Geography Names
|
||||
|
||||
Lowercase the initial the in place names and in the names of bands, bars, restaurants, hotels, products, and the like: the Netherlands, the Pixies, the Pentagon
|
||||
|
||||
### Captions
|
||||
|
||||
#### Caption Format
|
||||
|
||||
Capitalize the first word of a caption, and end with a period, whether or not the body of the caption is a full sentence.
|
||||
|
||||
#### Identifying Names
|
||||
|
||||
When a caption consists of nothing but an identifying name, however, omit the end punctuation. If the identifying caption includes any language beyond just a name, though, use the final punctuation: Dan Shipper. Dan Shipper, Every CEO.
|
||||
|
||||
#### Image Credits
|
||||
|
||||
When a caption includes an image credit, the credit should be formatted as DALL-E/Every illustration.
|
||||
|
||||
### Commas
|
||||
|
||||
#### Serial Comma
|
||||
|
||||
Use the serial or Oxford comma before the conjunction in a series: x, y, and z
|
||||
|
||||
#### Independent vs Dependent Clauses
|
||||
|
||||
Use a comma to separate independent clauses but not dependent clauses: He helped trouble-shoot an issue, and she wrote code. She signed up for Every and became a subscriber.
|
||||
|
||||
#### Restrictive Elements
|
||||
|
||||
Set off nonrestrictive elements with commas; don't set off restrictive elements. The most frequent example is the that/which difference: The piece, which garnered 15,000 readers, is one of Every's most successful. The piece that garnered 15,000 readers is one of Every's most successful.
|
||||
|
||||
#### Too Usage
|
||||
|
||||
Include a comma before "too" when used to mean "in addition." Don't use a comma when "too" refers to the subject of the sentence: I ate a bowl of ice cream. I had a cookie, too. You're a cat person? I am too.
|
||||
|
||||
#### Names
|
||||
|
||||
Don't include commas before "Jr." or "Sr.": Hank Aaron Jr.
|
||||
|
||||
#### Repetition
|
||||
|
||||
Don't include commas before words repeated for emphasis: It's what makes you you.
|
||||
|
||||
#### General Comma Usage
|
||||
|
||||
Otherwise, follow common sense with commas. Read the sentence out loud. If you need to take a breath, use a comma.
|
||||
|
||||
### Dates
|
||||
|
||||
#### Date Formats
|
||||
|
||||
Write dates as follows: April 13, 2018, The 19th of April was a nice day, March 2020, Thanksgiving 2023, summer 1999, the years 1980–85
|
||||
|
||||
#### Decades
|
||||
|
||||
When referring to a decade, write out the full year numerically at first mention and abbreviate on the second: She was born in the 1980s. The '80s was a wild decade.
|
||||
|
||||
### Ellipses
|
||||
|
||||
#### Usage
|
||||
|
||||
Use ellipses (…) to show that you're omitting words or trailing off before the end of a thought. Don't use an ellipsis for emphasis or drama. Don't use ellipses in titles or headers, nor when you should be using a colon (a list is to follow). There is no space before an ellipsis, and one space after… like this.
|
||||
|
||||
### Em dashes
|
||||
|
||||
#### Usage and Spacing
|
||||
|
||||
Use an em dash ( — ) for a true break or to set off a parenthetical statement. Do not put spaces around them. Try not to use em dashes more than twice in a paragraph. Don't use hyphens in place of an em dash: It's an anxious time to be an independent bookseller—but a recent upswing in sales is cause for optimism.
|
||||
|
||||
### En dash
|
||||
|
||||
#### Usage
|
||||
|
||||
Use them in compound adjectives, compound noun constructions, or when indicating spans or ranges: 5°C–10°C, from 10 a.m.–2 p.m., January 2019–November 2020, Texas–Mexico border, then–VP of engineering
|
||||
|
||||
### Filenames
|
||||
|
||||
#### File Types
|
||||
|
||||
When referring to a file type, use the appropriate acronym in all caps: GIF, PDF
|
||||
|
||||
#### Specific Files
|
||||
|
||||
When referring to a specific file, specify the filename followed by a period and the file type, all lowercase: important-graph.jpg
|
||||
|
||||
### Headlines
|
||||
|
||||
#### Title Case
|
||||
|
||||
Use title case for headlines. Use sentence case for subtitles and subheadings. Capitalize important words — everything but articles, conjunctions (for, and, nor, but, or, yet, so), and prepositions under four letters — in headings. Capitalize the first word only in subtitles and subheadings.
|
||||
|
||||
#### Prepositions
|
||||
|
||||
Capitalize short prepositions that form an integral part of a verb: Growing Up in China
|
||||
|
||||
#### Internal Punctuation
|
||||
|
||||
Capitalize all words following an internal punctuation mark: My Company Died — Learn From My Mistakes
|
||||
|
||||
#### First and Last Words
|
||||
|
||||
The first and last words of a headline are capitalized, no matter their parts of speech. Don't use punctuation in a title unless it's a question or exclamatory sentence.
|
||||
|
||||
#### Handwritten Letters
|
||||
|
||||
Headlines include one handwritten letter: The Secret [F]ather of Modern Computing
|
||||
|
||||
#### Subheadings
|
||||
|
||||
In general, start with h2 heading size and go smaller as needed for subheads. Some things to keep in mind: make sure that the hed doesn't run on too long (or onto a second line), or look out of place on the page. If it does, go smaller. For interview questions, use h5 heading size.
|
||||
|
||||
### Hyphens
|
||||
|
||||
#### Compound Adjectives
|
||||
|
||||
Use hyphens in compound adjectives, with the exception of adverbs (words ending in "-ly" or modifying a verb). A compound adjective that contains another compound adjective calls for an en dash: first-time founder, state-of-the-art design, open-source project, Pulitzer Prize–winning novelist, newly released program
|
||||
|
||||
#### Post-Noun Usage
|
||||
|
||||
Don't use hyphens when the compound adjective is placed after the noun it modifies or when the adjective is made up of nouns: The team is world class. video game console, The feature is first of its kind. toilet paper roll
|
||||
|
||||
#### Suspended Hyphens
|
||||
|
||||
Use a suspended hyphen for multiple hyphenated compounds or words: NewYork- and San Francisco-based company, university-owned and -operated bookstore
|
||||
|
||||
#### Percentages and Amounts
|
||||
|
||||
Hyphenation is usually unnecessary when expressing percentage, degree, or dollar amounts in figures: a 50 percent decline, $50 billion investment. But: a 50- to 60-percent decline, a $1-million-a-month burn rate
|
||||
|
||||
#### Fractions
|
||||
|
||||
Use hyphens in fractions, no matter their part of speech: three-fourths of the team, a share of one-third, one-third the size, a three-fourths share, one-third slower
|
||||
|
||||
### Italics
|
||||
|
||||
#### Titles
|
||||
|
||||
Italicize titles of books, newspapers, periodicals, movies, TV shows, and video games, with the following rules: If a magazine title must be followed by "magazine" to distinguish it from other publications, do not italicize "magazine" unless it is formally included in the title: *New York* magazine vs. *The New York Times Magazine*. For magazine titles, italicize the article if it is a formal part of the title: *The New Yorker*. For newspapers, do not italicize the article: the *New York Times*
|
||||
|
||||
#### Short Works
|
||||
|
||||
Titles of short works (poems, songs, TV episodes, book chapters) take quotation marks.
|
||||
|
||||
#### Punctuation After Italics
|
||||
|
||||
Do not italicize punctuation that follows an italicized term: Stewart Brand published the first issue of his seminal magazine, the *Whole Earth Catalogue*, in 1968. Which earned more at the box office, *Barbie* or *Oppenheimer*?
|
||||
|
||||
#### Websites
|
||||
|
||||
Italicize a website's title if it is also the name of a print newspaper or magazine. Otherwise, leave it unitalicized.
|
||||
|
||||
### Linking
|
||||
|
||||
#### Link Guidelines
|
||||
|
||||
Provide a link when referring to a website. Don't capitalize links or words within links, and don't say things like "Click here!" or "Click for more information." Write the sentence as you normally would, and link relevant keywords.
|
||||
|
||||
#### Link Text Length
|
||||
|
||||
Include only links you need and make the links as useful as possible. Keep the link text short, ideally two to four words. But not too short: Just one word can be difficult to click or tap on, especially if you're reading on a phone.
|
||||
|
||||
#### URL Format
|
||||
|
||||
URLs included in print should appear as is (i.e., not shortened by a URL shortener). The URL should be all lowercase, unless adding camel caps would increase readability. Don't include "www." or anything preceding it: You can read more on every.to. She's the founder of GetOutTheVoteNewYork.com.
|
||||
|
||||
### Lists
|
||||
|
||||
#### Usage
|
||||
|
||||
Use lists to present groups of information. Only number lists when order is important (describing steps of a process).
|
||||
|
||||
#### Numbering Format
|
||||
|
||||
Preferred format of lists is: 1., not 1)
|
||||
|
||||
#### Punctuation in Lists
|
||||
|
||||
If one of the list items is a complete sentence, use punctuation on all of the items. Otherwise, don't use punctuation in lists: 1. Enter your email. 2. Input your credit card information.
|
||||
|
||||
#### Numbered Lists
|
||||
|
||||
If the items are numbered, a period follows the numeral and each item begins with a capital letter.
|
||||
|
||||
#### Bulleted Lists
|
||||
|
||||
Don't use numbers when the list's order doesn't matter: Here are some chatbots that we created for the course: Hidden Premise Finder, Reflective Coach, Motivational Interviewing
|
||||
|
||||
### Naming
|
||||
|
||||
#### Name References
|
||||
|
||||
Identify people by their full names on first mention, last name thereafter. In newsletter and social media communications, use first names rather than last names.
|
||||
|
||||
#### Special Titles
|
||||
|
||||
By convention, the sitting U.S. president, active senior religious leaders, and living royalty should be referred to as Title (Last)Name: Pope Francis, John Paul II, King Charles, Elizabeth II, President Biden (but Donald Trump), Rishi Sunak, Dr. Jill Biden (not First Lady Biden), Mike Johnson (not Speaker Johnson or Congressman Johnson), Madonna, Andre the Giant
|
||||
|
||||
### Numbers
|
||||
|
||||
#### Spelling Out Numbers
|
||||
|
||||
Spell out one through nine and first through ninth, and spell out a number if it's the first word of a sentence. Use numerals below 10 only if decimal accuracy is required (5.6 miles) or for currency ($8), or when writing whole numbers greater than a million (4 million). Figures are also used when an abbreviation or symbol is used as the unit of measure: 75 mph, 15 km, 6'3", -40º Celsius
|
||||
|
||||
#### Percentages
|
||||
|
||||
Percentages always use numerals and spell out "percent": 7 percent
|
||||
|
||||
#### Ages
|
||||
|
||||
Ages always use numerals: He had a 5-year-old daughter.
|
||||
|
||||
#### Bitcoin
|
||||
|
||||
Write "bitcoin" for the generic currency but "bitcoins" for quantities of them: Since the company began accepting bitcoin, it has raked in over 1,000 bitcoins.
|
||||
|
||||
#### Other Figure Usage
|
||||
|
||||
There are a few more exceptions. Use figures for the following: the 1990s or the '90s, 70 degrees, chapter 16
|
||||
|
||||
#### Time of Day
|
||||
|
||||
Expressions of the time of day — even, half, and quarter hours, for example — may be spelled out. If you want to indicate the hour more specifically or to emphasize exactness, figures are used: ten o'clock, Eight-thirty, quarter past nine, 11:37 p.m., the 10:15 standup, Dan scheduled the meeting for 9:00 a.m. sharp.
|
||||
|
||||
#### Starting Sentences
|
||||
|
||||
Spell out any number that starts a sentence, unless it's a year. (Alternatively, revise the sentence so it doesn't start with a number.) Hyphens should be used in spelled-out numbers to join parts of a two-digit number: Twenty-five engineers joined the company in January. Ten thousand five hundred people signed up in a single day. 2020 was a tough year.
|
||||
|
||||
#### Commas in Numbers
|
||||
|
||||
Except in years, use a comma to separate 000's: 1,440,434. Numbers over three digits take commas: 1,000
|
||||
|
||||
#### Charts and Tables
|
||||
|
||||
Use figures for all numbers in charts and tables.
|
||||
|
||||
#### Ratios
|
||||
|
||||
Ratios are spelled out without hyphens: one in five, or one in 20.
|
||||
|
||||
### Parentheses
|
||||
|
||||
#### Usage
|
||||
|
||||
Use them only when the clause or phrase is non-essential, or when used for clarification or as an editorial aside: The investigation revealed groundbreaking information (though it has yet to be widely publicized). Please include the following information (if available)
|
||||
|
||||
#### Punctuation Placement
|
||||
|
||||
Punctuation goes outside of the parentheses unless the text in parentheses is a full sentence, or there's a question or exclamation within the parenthetical: How many hours per week do your developers spend on maintenance (i.e., debugging, refactoring, modifying)? She wondered if the world was out to get her. (Don't we all?)
|
||||
|
||||
### Plurals
|
||||
|
||||
#### Names Ending in S
|
||||
|
||||
For singular names and words that end in s, add 's, not just an apostrophe: Leo Polovets's fund, Paris's bridges
|
||||
|
||||
#### Entities Ending in S
|
||||
|
||||
For entities that end in s, add an 's as well: the New York Times's readers
|
||||
|
||||
#### Plural Names
|
||||
|
||||
For plural names and words, add just an apostrophe: the Williamses' farm, the Joneses' printer
|
||||
|
||||
#### Plural Words Not Ending in S
|
||||
|
||||
For plural words that don't end in s, treat them like singular nouns: men's, women's, children's
|
||||
|
||||
#### Figures and Characters
|
||||
|
||||
Use an apostrophe and s to form the plural of figures, lowercase characters, and symbols: two o's, two k's, and two e's in bookkeeper (but the three Rs; the five Ws), five @'s, a fleet of 747B's, stolen .22's
|
||||
|
||||
#### Exceptions
|
||||
|
||||
There are some exceptions: the 2000s, a woman in her 20s, temperature in the 70s, a fleet of 747s
|
||||
|
||||
### Pronouns
|
||||
|
||||
#### Singular They
|
||||
|
||||
Use the singular "they" (not "he or she") when making a gender-neutral statement. Use "it" for companies and brands: If a team member is feeling burnt out, consider how you can help support them. The company released its new product on Monday.
|
||||
|
||||
#### Pronoun References
|
||||
|
||||
Use the terms "he/him pronouns" and "she/her pronouns" when referring to a person's pronouns, not "male pronouns" and "female pronouns." Avoid the term "preferred pronouns."
|
||||
|
||||
### Proper nouns and names
|
||||
|
||||
#### Every Capitalization
|
||||
|
||||
"Every" is always capitalized. The only times Every appears in lowercase are in social media handles and URLs.
|
||||
|
||||
#### Geography
|
||||
|
||||
Capitalize place names, but use lowercase for general directions or regions: the East (world and U.S.), the West (world and U.S.), the South, the North, Western United States, Southeast Asia, Northern Hemisphere, eastern Long Island, the Bay Area, Westerner, Easterner, Northerner, Southerner, the Midwest, Midwestern, Southwestern (referring to style of art), southwestern (all other uses), Western Europe, Eastern Europe, southern California, northern California, west Texas, east Tennessee, south Florida, the South of France, Continental Europe, Washington State
|
||||
|
||||
#### Neighborhoods
|
||||
|
||||
Neighborhood nicknames are also capitalized: Midtown, Soho, Tribeca, the Tenderloin
|
||||
|
||||
#### Earth
|
||||
|
||||
Capitalize Earth when writing about it as a planet ("Venus, Mars, and Earth"), but lowercase in phrases like "salt of the earth."
|
||||
|
||||
#### Initials in Names
|
||||
|
||||
For proper names written with initials, use periods and no spaces: E.L. James, J.K. Simmons, J.Crew. But when the initials comprise the whole name, no periods are used (FDR, DFW).
|
||||
|
||||
### Punctuation
|
||||
|
||||
#### Exclamation Points
|
||||
|
||||
Use exclamation points sparingly. Seriously! (Unless you're quoting someone.) Use emojis with discretion.
|
||||
|
||||
### Quotation marks
|
||||
|
||||
#### Basic Usage
|
||||
|
||||
Spoken text should be placed in double quotation marks (" "). Quotes within quotations should be placed in single quotation marks (' '): "He told me, 'That's a fantastic idea.'" "You may find it hard to prioritize the 'I got problems' meeting at first."
|
||||
|
||||
#### Tense Usage
|
||||
|
||||
Use the present tense when the quote was spoken directly to the author. Use the past tense when the quote is a recollection or happened at a specific time in the past. Treat thoughts the same way: "That was a long day," she recalls. She remembers the frustrations of that day well. It began when her manager said, "I'm afraid we've got trouble." I thought, "What's next?"
|
||||
|
||||
#### Punctuation Placement
|
||||
|
||||
Place periods and commas inside quotation marks. If a question mark or exclamation mark is part of the quote, place it within the quotation marks. If the question or exclamation refers to the quote itself, place the punctuation outside of the quote: She asked, "Who else is taking the week of Christmas off?" Who said, "To thine own self be true"?
|
||||
|
||||
#### Introducing Quotes
|
||||
|
||||
If the text preceding a quote introduces the quote, include a comma before the quote. If the text before the quote leads directly into the quote, don't include a comma. Capitalize the first letter in the quote when it's a full sentence or when following "said," "says," or other introductory language. Generally avoid using a colon to introduce a quote unless it's more than two sentences long: When doing strategic planning for the year, "it's important to carve out time to solicit everyone' feedback," she says. Every's mission is "to feed the minds and hearts of the people who build the internet," says Shipper. He recalls, "We had no choice but to start from scratch."
|
||||
|
||||
#### Multi-Paragraph Quotes
|
||||
|
||||
When a quote continues across multiple paragraphs, the quote is left open at the end of each paragraph. A new open-quote mark is to start the next paragraph, only closing the quote when the full quote is finished: Guillermo has noticed developers at Vercel becoming more full stack. "I think it's an important asset to have. They can bring context, data, copywriting into their creations that otherwise would have required chatting with other people and crowdsourcing ideas. "The trend has been away from the implementation detail, which is the code, and toward the end goal, which is to deliver a great product or a great experience."
|
||||
|
||||
#### Edited Text
|
||||
|
||||
Use square brackets to indicate edited text in a quote. Keep text in square brackets to a minimum—use only when the edit would increase clarity and comprehension or add necessary context. If you need to place an entire sentence in square brackets, it's probably better to paraphrase: "It was difficult [to prioritize addressing tech debt] because we had so many features to work on."
|
||||
|
||||
#### Block Quotes
|
||||
|
||||
Use block quotes when a quotation is more than four lines long. Introduce it with a colon, and include quotation marks.
|
||||
|
||||
### References to other parts of the text
|
||||
|
||||
#### Directional References
|
||||
|
||||
Rather than "above" or "below," use terms like "earlier," "later," "previously," etc.: As I mentioned earlier,
|
||||
|
||||
### Semicolons
|
||||
|
||||
#### Usage Guidelines
|
||||
|
||||
Go easy on semicolons. When appropriate, use an em dash ( — ) instead, or simply start a new sentence. Never use a semicolon in site or email copy.
|
||||
|
||||
### Slashes
|
||||
|
||||
#### Usage
|
||||
|
||||
Try to avoid them, and minimize constructions like "and/or." Use hyphens instead when needed. However, slashes should always be used when referring to an individual's pronouns: We needed all of our designers and illustrators to sign the contract. She's an accomplished singer-songwriter. they/them pronouns, We had a team of 20 engineers and developers.
|
||||
|
||||
### Spelling
|
||||
|
||||
#### American Spelling
|
||||
|
||||
Use American spellings (i.e., color, not colour).
|
||||
|
||||
#### Unconventional Spellings
|
||||
|
||||
Do not follow unconventional or artistic spellings of names, products, and corporations: Questlove (not ?uestlove), Kesha (not Ke$ha), India Arie (not India.Arie), E.E. Cummings (not e e cummings), Kiss (not KISS), Adidas (not adidas), Yahoo (not Yahoo!)
|
||||
|
||||
#### Common Exceptions
|
||||
|
||||
The common exceptions are: ChatGPT, WhatsApp, iPod, iPhone, iMac, etc., TikTok, eBay, PayPal, BuzzFeed
|
||||
|
||||
### Time zones
|
||||
|
||||
#### Abbreviations
|
||||
|
||||
Abbreviate time zones within the continental United States, and spell out the rest: Eastern Time (ET), Central Time (CT), Mountain Time (MT), Pacific Time (PT)
|
||||
|
||||
### Usage
|
||||
|
||||
#### Collective Nouns
|
||||
|
||||
Collective nouns can be construed as plural if you want to emphasize the individuals forming the group, but most often they should be treated as singular. Subsequent pronouns should agree with the verb tense chosen. The Every trivia squad is considered one of the league's strongest teams. But: The lucky trio are collecting their Amazon gift cards. The Grammys are coming to Los Angeles.
|
||||
|
||||
#### Fewer vs Less
|
||||
|
||||
Use "fewer" instead of "less" with nouns for countable objects and concepts. Don't use "over" or "under" when referring to numbers or quantities: Fewer than seven days remain until the quarter ends. In less than an hour, more than an inch of rain fell.
|
||||
|
||||
#### Overused Words
|
||||
|
||||
Don't overuse "actually," "very," or "just" (they can almost always be deleted).
|
||||
|
||||
### Word and phrase bank
|
||||
|
||||
#### Standard Terms
|
||||
|
||||
add on (verb), add-on (noun, adjective), back end (noun), back-end (adjective), beta (lowercase unless it's part of a proper noun), cofounder, Covid-19, coworker, double-click, drop-down, e-commerce, front end (noun), front-end (adjective), geolocation, hashtag, homepage, large language model, login (noun, adjective), log in (verb), millennial, nonprofit, Online, open source, open-source software, opt in (verb), opt-in (noun, adjective), pop-up (noun, adjective), pop up (verb), signup (noun, adjective), sign up (verb), startup, sync, username, URL (always uppercase), web3, well-being, WiFi, workspace
|
||||
Reference in New Issue
Block a user