john/claude-engineering-plugin
1
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity

Merge upstream origin/main with local fork additions preserved

Browse Source 
Accept upstream's ce-review pipeline rewrite (6-stage persona-based
architecture with structured JSON, confidence gating, three execution
modes). Retire 4 overlapping review agents (security-sentinel,
performance-oracle, data-migration-expert, data-integrity-guardian)
replaced by upstream equivalents. Add 5 local review agents as
conditional personas in the persona catalog (kieran-python, tiangolo-
fastapi, kieran-typescript, julik-frontend-races, architecture-
strategist).

Accept upstream skill renames (file-todos→todo-create, resolve_todo_
parallel→todo-resolve), port local Assessment and worktree constraint
additions to new files. Merge best-practices-researcher with upstream
platform-agnostic discovery + local FastAPI mappings. Remove Rails/Ruby
skills (dhh-rails-style, andrew-kane-gem-writer, dspy-ruby) per fork's
FastAPI pivot.

Component counts: 36 agents, 48 skills, 7 commands.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
John Lamb
2026-03-25 13:28:22 -05:00
parent 207774f44e
commit 0b26ab8fe6
79 changed files with 6584 additions and 8982 deletions
Download Patch File Download Diff File Expand all files Collapse all files

184
plugins/compound-engineering/skills/andrew-kane-gem-writer/SKILL.md
View File

@@ -1,184 +0,0 @@
---
name: andrew-kane-gem-writer
description: This skill should be used when writing Ruby gems following Andrew Kane's proven patterns and philosophy. It applies when creating new Ruby gems, refactoring existing gems, designing gem APIs, or when clean, minimal, production-ready Ruby library code is needed. Triggers on requests like "create a gem", "write a Ruby library", "design a gem API", or mentions of Andrew Kane's style.
---
# Andrew Kane Gem Writer
Write Ruby gems following Andrew Kane's battle-tested patterns from 100+ gems with 374M+ downloads (Searchkick, PgHero, Chartkick, Strong Migrations, Lockbox, Ahoy, Blazer, Groupdate, Neighbor, Blind Index).
## Core Philosophy
**Simplicity over cleverness.** Zero or minimal dependencies. Explicit code over metaprogramming. Rails integration without Rails coupling. Every pattern serves production use cases.
## Entry Point Structure
Every gem follows this exact pattern in `lib/gemname.rb`:
```ruby
# 1. Dependencies (stdlib preferred)
require "forwardable"
# 2. Internal modules
require_relative "gemname/model"
require_relative "gemname/version"
# 3. Conditional Rails (CRITICAL - never require Rails directly)
require_relative "gemname/railtie" if defined?(Rails)
# 4. Module with config and errors
module GemName
class Error < StandardError; end
class InvalidConfigError < Error; end
class << self
attr_accessor :timeout, :logger
attr_writer :client
end
self.timeout = 10 # Defaults set immediately
end
```
## Class Macro DSL Pattern
The signature Kane pattern—single method call configures everything:
```ruby
# Usage
class Product < ApplicationRecord
searchkick word_start: [:name]
end
# Implementation
module GemName
module Model
def gemname(**options)
unknown = options.keys - KNOWN_KEYWORDS
raise ArgumentError, "unknown keywords: #{unknown.join(", ")}" if unknown.any?
mod = Module.new
mod.module_eval do
define_method :some_method do
# implementation
end unless method_defined?(:some_method)
end
include mod
class_eval do
cattr_reader :gemname_options, instance_reader: false
class_variable_set :@@gemname_options, options.dup
end
end
end
end
```
## Rails Integration
**Always use `ActiveSupport.on_load`—never require Rails gems directly:**
```ruby
# WRONG
require "active_record"
ActiveRecord::Base.include(MyGem::Model)
# CORRECT
ActiveSupport.on_load(:active_record) do
extend GemName::Model
end
# Use prepend for behavior modification
ActiveSupport.on_load(:active_record) do
ActiveRecord::Migration.prepend(GemName::Migration)
end
```
## Configuration Pattern
Use `class << self` with `attr_accessor`, not Configuration objects:
```ruby
module GemName
class << self
attr_accessor :timeout, :logger
attr_writer :master_key
end
def self.master_key
@master_key ||= ENV["GEMNAME_MASTER_KEY"]
end
self.timeout = 10
self.logger = nil
end
```
## Error Handling
Simple hierarchy with informative messages:
```ruby
module GemName
class Error < StandardError; end
class ConfigError < Error; end
class ValidationError < Error; end
end
# Validate early with ArgumentError
def initialize(key:)
raise ArgumentError, "Key must be 32 bytes" unless key&.bytesize == 32
end
```
## Testing (Minitest Only)
```ruby
# test/test_helper.rb
require "bundler/setup"
Bundler.require(:default)
require "minitest/autorun"
require "minitest/pride"
# test/model_test.rb
class ModelTest < Minitest::Test
def test_basic_functionality
assert_equal expected, actual
end
end
```
## Gemspec Pattern
Zero runtime dependencies when possible:
```ruby
Gem::Specification.new do |spec|
spec.name = "gemname"
spec.version = GemName::VERSION
spec.required_ruby_version = ">= 3.1"
spec.files = Dir["*.{md,txt}", "{lib}/**/*"]
spec.require_path = "lib"
# NO add_dependency lines - dev deps go in Gemfile
end
```
## Anti-Patterns to Avoid
- `method_missing` (use `define_method` instead)
- Configuration objects (use class accessors)
- `@@class_variables` (use `class << self`)
- Requiring Rails gems directly
- Many runtime dependencies
- Committing Gemfile.lock in gems
- RSpec (use Minitest)
- Heavy DSLs (prefer explicit Ruby)
## Reference Files
For deeper patterns, see:
- **[references/module-organization.md](references/module-organization.md)** - Directory layouts, method decomposition
- **[references/rails-integration.md](references/rails-integration.md)** - Railtie, Engine, on_load patterns
- **[references/database-adapters.md](references/database-adapters.md)** - Multi-database support patterns
- **[references/testing-patterns.md](references/testing-patterns.md)** - Multi-version testing, CI setup
- **[references/resources.md](references/resources.md)** - Links to Kane's repos and articles

231
plugins/compound-engineering/skills/andrew-kane-gem-writer/references/database-adapters.md
View File

@@ -1,231 +0,0 @@
# Database Adapter Patterns
## Abstract Base Class Pattern
```ruby
# lib/strong_migrations/adapters/abstract_adapter.rb
module StrongMigrations
module Adapters
class AbstractAdapter
def initialize(checker)
@checker = checker
end
def min_version
nil
end
def set_statement_timeout(timeout)
# no-op by default
end
def check_lock_timeout
# no-op by default
end
private
def connection
@checker.send(:connection)
end
def quote(value)
connection.quote(value)
end
end
end
end
```
## PostgreSQL Adapter
```ruby
# lib/strong_migrations/adapters/postgresql_adapter.rb
module StrongMigrations
module Adapters
class PostgreSQLAdapter < AbstractAdapter
def min_version
"12"
end
def set_statement_timeout(timeout)
select_all("SET statement_timeout = #{timeout.to_i * 1000}")
end
def set_lock_timeout(timeout)
select_all("SET lock_timeout = #{timeout.to_i * 1000}")
end
def check_lock_timeout
lock_timeout = connection.select_value("SHOW lock_timeout")
lock_timeout_sec = timeout_to_sec(lock_timeout)
# validation logic
end
private
def select_all(sql)
connection.select_all(sql)
end
def timeout_to_sec(timeout)
units = {"us" => 1e-6, "ms" => 1e-3, "s" => 1, "min" => 60}
timeout.to_f * (units[timeout.gsub(/\d+/, "")] || 1e-3)
end
end
end
end
```
## MySQL Adapter
```ruby
# lib/strong_migrations/adapters/mysql_adapter.rb
module StrongMigrations
module Adapters
class MySQLAdapter < AbstractAdapter
def min_version
"8.0"
end
def set_statement_timeout(timeout)
select_all("SET max_execution_time = #{timeout.to_i * 1000}")
end
def check_lock_timeout
lock_timeout = connection.select_value("SELECT @@lock_wait_timeout")
# validation logic
end
end
end
end
```
## MariaDB Adapter (MySQL variant)
```ruby
# lib/strong_migrations/adapters/mariadb_adapter.rb
module StrongMigrations
module Adapters
class MariaDBAdapter < MySQLAdapter
def min_version
"10.5"
end
# Override MySQL-specific behavior
def set_statement_timeout(timeout)
select_all("SET max_statement_time = #{timeout.to_i}")
end
end
end
end
```
## Adapter Detection Pattern
Use regex matching on adapter name:
```ruby
def adapter
@adapter ||= case connection.adapter_name
when /postg/i
Adapters::PostgreSQLAdapter.new(self)
when /mysql|trilogy/i
if connection.try(:mariadb?)
Adapters::MariaDBAdapter.new(self)
else
Adapters::MySQLAdapter.new(self)
end
when /sqlite/i
Adapters::SQLiteAdapter.new(self)
else
Adapters::AbstractAdapter.new(self)
end
end
```
## Multi-Database Support (PgHero pattern)
```ruby
module PgHero
class << self
attr_accessor :databases
end
self.databases = {}
def self.primary_database
databases.values.first
end
def self.capture_query_stats(database: nil)
db = database ? databases[database] : primary_database
db.capture_query_stats
end
class Database
attr_reader :id, :config
def initialize(id, config)
@id = id
@config = config
end
def connection_model
@connection_model ||= begin
Class.new(ActiveRecord::Base) do
self.abstract_class = true
end.tap do |model|
model.establish_connection(config)
end
end
end
def connection
connection_model.connection
end
end
end
```
## Connection Switching
```ruby
def with_connection(database_name)
db = databases[database_name.to_s]
raise Error, "Unknown database: #{database_name}" unless db
yield db.connection
end
# Usage
PgHero.with_connection(:replica) do |conn|
conn.execute("SELECT * FROM users")
end
```
## SQL Dialect Handling
```ruby
def quote_column(column)
case adapter_name
when /postg/i
%("#{column}")
when /mysql/i
"`#{column}`"
else
column
end
end
def boolean_value(value)
case adapter_name
when /postg/i
value ? "true" : "false"
when /mysql/i
value ? "1" : "0"
else
value.to_s
end
end
```

121
plugins/compound-engineering/skills/andrew-kane-gem-writer/references/module-organization.md
View File

@@ -1,121 +0,0 @@
# Module Organization Patterns
## Simple Gem Layout
```
lib/
├── gemname.rb # Entry point, config, errors
└── gemname/
├── helper.rb # Core functionality
├── engine.rb # Rails engine (if needed)
└── version.rb # VERSION constant only
```
## Complex Gem Layout (PgHero pattern)
```
lib/
├── pghero.rb
└── pghero/
├── database.rb # Main class
├── engine.rb # Rails engine
└── methods/ # Functional decomposition
├── basic.rb
├── connections.rb
├── indexes.rb
├── queries.rb
└── replication.rb
```
## Method Decomposition Pattern
Break large classes into includable modules by feature:
```ruby
# lib/pghero/database.rb
module PgHero
class Database
include Methods::Basic
include Methods::Connections
include Methods::Indexes
include Methods::Queries
end
end
# lib/pghero/methods/indexes.rb
module PgHero
module Methods
module Indexes
def index_hit_rate
# implementation
end
def unused_indexes
# implementation
end
end
end
end
```
## Version File Pattern
Keep version.rb minimal:
```ruby
# lib/gemname/version.rb
module GemName
VERSION = "2.0.0"
end
```
## Require Order in Entry Point
```ruby
# lib/searchkick.rb
# 1. Standard library
require "forwardable"
require "json"
# 2. External dependencies (minimal)
require "active_support"
# 3. Internal files via require_relative
require_relative "searchkick/index"
require_relative "searchkick/model"
require_relative "searchkick/query"
require_relative "searchkick/version"
# 4. Conditional Rails loading (LAST)
require_relative "searchkick/railtie" if defined?(Rails)
```
## Autoload vs Require
Kane uses explicit `require_relative`, not autoload:
```ruby
# CORRECT
require_relative "gemname/model"
require_relative "gemname/query"
# AVOID
autoload :Model, "gemname/model"
autoload :Query, "gemname/query"
```
## Comments Style
Minimal section headers only:
```ruby
# dependencies
require "active_support"
# adapters
require_relative "adapters/postgresql_adapter"
# modules
require_relative "migration"
```

183
plugins/compound-engineering/skills/andrew-kane-gem-writer/references/rails-integration.md
View File

@@ -1,183 +0,0 @@
# Rails Integration Patterns
## The Golden Rule
**Never require Rails gems directly.** This causes loading order issues.
```ruby
# WRONG - causes premature loading
require "active_record"
ActiveRecord::Base.include(MyGem::Model)
# CORRECT - lazy loading
ActiveSupport.on_load(:active_record) do
extend MyGem::Model
end
```
## ActiveSupport.on_load Hooks
Common hooks and their uses:
```ruby
# Models
ActiveSupport.on_load(:active_record) do
extend GemName::Model # Add class methods (searchkick, has_encrypted)
include GemName::Callbacks # Add instance methods
end
# Controllers
ActiveSupport.on_load(:action_controller) do
include Ahoy::Controller
end
# Jobs
ActiveSupport.on_load(:active_job) do
include GemName::JobExtensions
end
# Mailers
ActiveSupport.on_load(:action_mailer) do
include GemName::MailerExtensions
end
```
## Prepend for Behavior Modification
When overriding existing Rails methods:
```ruby
ActiveSupport.on_load(:active_record) do
ActiveRecord::Migration.prepend(StrongMigrations::Migration)
ActiveRecord::Migrator.prepend(StrongMigrations::Migrator)
end
```
## Railtie Pattern
Minimal Railtie for non-mountable gems:
```ruby
# lib/gemname/railtie.rb
module GemName
class Railtie < Rails::Railtie
initializer "gemname.configure" do
ActiveSupport.on_load(:active_record) do
extend GemName::Model
end
end
# Optional: Add to controller runtime logging
initializer "gemname.log_runtime" do
require_relative "controller_runtime"
ActiveSupport.on_load(:action_controller) do
include GemName::ControllerRuntime
end
end
# Optional: Rake tasks
rake_tasks do
load "tasks/gemname.rake"
end
end
end
```
## Engine Pattern (Mountable Gems)
For gems with web interfaces (PgHero, Blazer, Ahoy):
```ruby
# lib/pghero/engine.rb
module PgHero
class Engine < ::Rails::Engine
isolate_namespace PgHero
initializer "pghero.assets", group: :all do |app|
if app.config.respond_to?(:assets) && defined?(Sprockets)
app.config.assets.precompile << "pghero/application.js"
app.config.assets.precompile << "pghero/application.css"
end
end
initializer "pghero.config" do
PgHero.config = Rails.application.config_for(:pghero) rescue {}
end
end
end
```
## Routes for Engines
```ruby
# config/routes.rb (in engine)
PgHero::Engine.routes.draw do
root to: "home#index"
resources :databases, only: [:show]
end
```
Mount in app:
```ruby
# config/routes.rb (in app)
mount PgHero::Engine, at: "pghero"
```
## YAML Configuration with ERB
For complex gems needing config files:
```ruby
def self.settings
@settings ||= begin
path = Rails.root.join("config", "blazer.yml")
if path.exist?
YAML.safe_load(ERB.new(File.read(path)).result, aliases: true)
else
{}
end
end
end
```
## Generator Pattern
```ruby
# lib/generators/gemname/install_generator.rb
module GemName
module Generators
class InstallGenerator < Rails::Generators::Base
source_root File.expand_path("templates", __dir__)
def copy_initializer
template "initializer.rb", "config/initializers/gemname.rb"
end
def copy_migration
migration_template "migration.rb", "db/migrate/create_gemname_tables.rb"
end
end
end
end
```
## Conditional Feature Detection
```ruby
# Check for specific Rails versions
if ActiveRecord.version >= Gem::Version.new("7.0")
# Rails 7+ specific code
end
# Check for optional dependencies
def self.client
@client ||= if defined?(OpenSearch::Client)
OpenSearch::Client.new
elsif defined?(Elasticsearch::Client)
Elasticsearch::Client.new
else
raise Error, "Install elasticsearch or opensearch-ruby"
end
end
```

119
plugins/compound-engineering/skills/andrew-kane-gem-writer/references/resources.md
View File

@@ -1,119 +0,0 @@
# Andrew Kane Resources
## Primary Documentation
- **Gem Patterns Article**: https://ankane.org/gem-patterns
- Kane's own documentation of patterns used across his gems
- Covers configuration, Rails integration, error handling
## Top Ruby Gems by Stars
### Search & Data
| Gem | Stars | Description | Source |
|-----|-------|-------------|--------|
| **Searchkick** | 6.6k+ | Intelligent search for Rails | https://github.com/ankane/searchkick |
| **Chartkick** | 6.4k+ | Beautiful charts in Ruby | https://github.com/ankane/chartkick |
| **Groupdate** | 3.8k+ | Group by day, week, month | https://github.com/ankane/groupdate |
| **Blazer** | 4.6k+ | SQL dashboard for Rails | https://github.com/ankane/blazer |
### Database & Migrations
| Gem | Stars | Description | Source |
|-----|-------|-------------|--------|
| **PgHero** | 8.2k+ | PostgreSQL insights | https://github.com/ankane/pghero |
| **Strong Migrations** | 4.1k+ | Safe migration checks | https://github.com/ankane/strong_migrations |
| **Dexter** | 1.8k+ | Auto index advisor | https://github.com/ankane/dexter |
| **PgSync** | 1.5k+ | Sync Postgres data | https://github.com/ankane/pgsync |
### Security & Encryption
| Gem | Stars | Description | Source |
|-----|-------|-------------|--------|
| **Lockbox** | 1.5k+ | Application-level encryption | https://github.com/ankane/lockbox |
| **Blind Index** | 1.0k+ | Encrypted search | https://github.com/ankane/blind_index |
| **Secure Headers** | — | Contributed patterns | Referenced in gems |
### Analytics & ML
| Gem | Stars | Description | Source |
|-----|-------|-------------|--------|
| **Ahoy** | 4.2k+ | Analytics for Rails | https://github.com/ankane/ahoy |
| **Neighbor** | 1.1k+ | Vector search for Rails | https://github.com/ankane/neighbor |
| **Rover** | 700+ | DataFrames for Ruby | https://github.com/ankane/rover |
| **Tomoto** | 200+ | Topic modeling | https://github.com/ankane/tomoto-ruby |
### Utilities
| Gem | Stars | Description | Source |
|-----|-------|-------------|--------|
| **Pretender** | 2.0k+ | Login as another user | https://github.com/ankane/pretender |
| **Authtrail** | 900+ | Login activity tracking | https://github.com/ankane/authtrail |
| **Notable** | 200+ | Track notable requests | https://github.com/ankane/notable |
| **Logstop** | 200+ | Filter sensitive logs | https://github.com/ankane/logstop |
## Key Source Files to Study
### Entry Point Patterns
- https://github.com/ankane/searchkick/blob/master/lib/searchkick.rb
- https://github.com/ankane/pghero/blob/master/lib/pghero.rb
- https://github.com/ankane/strong_migrations/blob/master/lib/strong_migrations.rb
- https://github.com/ankane/lockbox/blob/master/lib/lockbox.rb
### Class Macro Implementations
- https://github.com/ankane/searchkick/blob/master/lib/searchkick/model.rb
- https://github.com/ankane/lockbox/blob/master/lib/lockbox/model.rb
- https://github.com/ankane/neighbor/blob/master/lib/neighbor/model.rb
- https://github.com/ankane/blind_index/blob/master/lib/blind_index/model.rb
### Rails Integration (Railtie/Engine)
- https://github.com/ankane/pghero/blob/master/lib/pghero/engine.rb
- https://github.com/ankane/searchkick/blob/master/lib/searchkick/railtie.rb
- https://github.com/ankane/ahoy/blob/master/lib/ahoy/engine.rb
- https://github.com/ankane/blazer/blob/master/lib/blazer/engine.rb
### Database Adapters
- https://github.com/ankane/strong_migrations/tree/master/lib/strong_migrations/adapters
- https://github.com/ankane/groupdate/tree/master/lib/groupdate/adapters
- https://github.com/ankane/neighbor/tree/master/lib/neighbor
### Error Messages (Template Pattern)
- https://github.com/ankane/strong_migrations/blob/master/lib/strong_migrations/error_messages.rb
### Gemspec Examples
- https://github.com/ankane/searchkick/blob/master/searchkick.gemspec
- https://github.com/ankane/neighbor/blob/master/neighbor.gemspec
- https://github.com/ankane/ahoy/blob/master/ahoy_matey.gemspec
### Test Setups
- https://github.com/ankane/searchkick/tree/master/test
- https://github.com/ankane/lockbox/tree/master/test
- https://github.com/ankane/strong_migrations/tree/master/test
## GitHub Profile
- **Profile**: https://github.com/ankane
- **All Ruby Repos**: https://github.com/ankane?tab=repositories&q=&type=&language=ruby&sort=stargazers
- **RubyGems Profile**: https://rubygems.org/profiles/ankane
## Blog Posts & Articles
- **ankane.org**: https://ankane.org/
- **Gem Patterns**: https://ankane.org/gem-patterns (essential reading)
- **Postgres Performance**: https://ankane.org/introducing-pghero
- **Search Tips**: https://ankane.org/search-rails
## Design Philosophy Summary
From studying 100+ gems, Kane's consistent principles:
1. **Zero dependencies when possible** - Each dep is a maintenance burden
2. **ActiveSupport.on_load always** - Never require Rails gems directly
3. **Class macro DSLs** - Single method configures everything
4. **Explicit over magic** - No method_missing, define methods directly
5. **Minitest only** - Simple, sufficient, no RSpec
6. **Multi-version testing** - Support broad Rails/Ruby versions
7. **Helpful errors** - Template-based messages with fix suggestions
8. **Abstract adapters** - Clean multi-database support
9. **Engine isolation** - isolate_namespace for mountable gems
10. **Minimal documentation** - Code is self-documenting, README is examples

261
plugins/compound-engineering/skills/andrew-kane-gem-writer/references/testing-patterns.md
View File

@@ -1,261 +0,0 @@
# Testing Patterns
## Minitest Setup
Kane exclusively uses Minitest—never RSpec.
```ruby
# test/test_helper.rb
require "bundler/setup"
Bundler.require(:default)
require "minitest/autorun"
require "minitest/pride"
# Load the gem
require "gemname"
# Test database setup (if needed)
ActiveRecord::Base.establish_connection(
adapter: "postgresql",
database: "gemname_test"
)
# Base test class
class Minitest::Test
def setup
# Reset state before each test
end
end
```
## Test File Structure
```ruby
# test/model_test.rb
require_relative "test_helper"
class ModelTest < Minitest::Test
def setup
User.delete_all
end
def test_basic_functionality
user = User.create!(email: "test@example.org")
assert_equal "test@example.org", user.email
end
def test_with_invalid_input
error = assert_raises(ArgumentError) do
User.create!(email: nil)
end
assert_match /email/, error.message
end
def test_class_method
result = User.search("test")
assert_kind_of Array, result
end
end
```
## Multi-Version Testing
Test against multiple Rails/Ruby versions using gemfiles:
```
test/
├── test_helper.rb
└── gemfiles/
├── activerecord70.gemfile
├── activerecord71.gemfile
└── activerecord72.gemfile
```
```ruby
# test/gemfiles/activerecord70.gemfile
source "https://rubygems.org"
gemspec path: "../../"
gem "activerecord", "~> 7.0.0"
gem "sqlite3"
```
```ruby
# test/gemfiles/activerecord72.gemfile
source "https://rubygems.org"
gemspec path: "../../"
gem "activerecord", "~> 7.2.0"
gem "sqlite3"
```
Run with specific gemfile:
```bash
BUNDLE_GEMFILE=test/gemfiles/activerecord70.gemfile bundle install
BUNDLE_GEMFILE=test/gemfiles/activerecord70.gemfile bundle exec rake test
```
## Rakefile
```ruby
# Rakefile
require "bundler/gem_tasks"
require "rake/testtask"
Rake::TestTask.new(:test) do |t|
t.libs << "test"
t.pattern = "test/**/*_test.rb"
end
task default: :test
```
## GitHub Actions CI
```yaml
# .github/workflows/build.yml
name: build
on: [push, pull_request]
jobs:
build:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
include:
- ruby: "3.2"
gemfile: activerecord70
- ruby: "3.3"
gemfile: activerecord71
- ruby: "3.3"
gemfile: activerecord72
env:
BUNDLE_GEMFILE: test/gemfiles/${{ matrix.gemfile }}.gemfile
steps:
- uses: actions/checkout@v4
- uses: ruby/setup-ruby@v1
with:
ruby-version: ${{ matrix.ruby }}
bundler-cache: true
- run: bundle exec rake test
```
## Database-Specific Testing
```yaml
# .github/workflows/build.yml (with services)
services:
postgres:
image: postgres:15
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
ports:
- 5432:5432
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
env:
DATABASE_URL: postgres://postgres:postgres@localhost/gemname_test
```
## Test Database Setup
```ruby
# test/test_helper.rb
require "active_record"
# Connect to database
ActiveRecord::Base.establish_connection(
ENV["DATABASE_URL"] || {
adapter: "postgresql",
database: "gemname_test"
}
)
# Create tables
ActiveRecord::Schema.define do
create_table :users, force: true do |t|
t.string :email
t.text :encrypted_data
t.timestamps
end
end
# Define models
class User < ActiveRecord::Base
gemname_feature :email
end
```
## Assertion Patterns
```ruby
# Basic assertions
assert result
assert_equal expected, actual
assert_nil value
assert_empty array
# Exception testing
assert_raises(ArgumentError) { bad_code }
error = assert_raises(GemName::Error) do
risky_operation
end
assert_match /expected message/, error.message
# Refutations
refute condition
refute_equal unexpected, actual
refute_nil value
```
## Test Helpers
```ruby
# test/test_helper.rb
class Minitest::Test
def with_options(options)
original = GemName.options.dup
GemName.options.merge!(options)
yield
ensure
GemName.options = original
end
def assert_queries(expected_count)
queries = []
callback = ->(*, payload) { queries << payload[:sql] }
ActiveSupport::Notifications.subscribe("sql.active_record", callback)
yield
assert_equal expected_count, queries.size, "Expected #{expected_count} queries, got #{queries.size}"
ensure
ActiveSupport::Notifications.unsubscribe(callback)
end
end
```
## Skipping Tests
```ruby
def test_postgresql_specific
skip "PostgreSQL only" unless postgresql?
# test code
end
def postgresql?
ActiveRecord::Base.connection.adapter_name =~ /postg/i
end
```

19
plugins/compound-engineering/skills/ce-review/references/persona-catalog.md
View File

@@ -1,6 +1,6 @@
# Persona Catalog
8 reviewer personas organized in two tiers, plus CE-specific agents. The orchestrator uses this catalog to select which reviewers to spawn for each review.
13 reviewer personas organized in three tiers, plus CE-specific agents. The orchestrator uses this catalog to select which reviewers to spawn for each review.
## Always-on (3 personas + 2 CE agents)
@@ -33,6 +33,18 @@ Spawned when the orchestrator identifies relevant patterns in the diff. The orch
| `data-migrations` | `compound-engineering:review:data-migrations-reviewer` | Migration files, schema changes, backfill scripts, data transformations |
| `reliability` | `compound-engineering:review:reliability-reviewer` | Error handling, retry logic, circuit breakers, timeouts, background jobs, async handlers, health checks |
## Language & Framework Conditional (5 personas)
Spawned when the orchestrator identifies language or framework-specific patterns in the diff. These provide deeper domain expertise than the general-purpose personas above.
| Persona | Agent | Select when diff touches... |
|---------|-------|---------------------------|
| `python-quality` | `compound-engineering:review:kieran-python-reviewer` | Python files, FastAPI routes, Pydantic models, async/await patterns, SQLAlchemy usage |
| `fastapi-philosophy` | `compound-engineering:review:tiangolo-fastapi-reviewer` | FastAPI application code, dependency injection, response models, middleware, OpenAPI schemas |
| `typescript-quality` | `compound-engineering:review:kieran-typescript-reviewer` | TypeScript files, React components, type definitions, generic patterns |
| `frontend-races` | `compound-engineering:review:julik-frontend-races-reviewer` | Frontend JavaScript, Stimulus controllers, event listeners, async UI code, animations, DOM lifecycle |
| `architecture` | `compound-engineering:review:architecture-strategist` | New services, module boundaries, dependency graphs, API layer changes, package structure |
## CE Conditional Agents (migration-specific)
These CE-native agents provide specialized analysis beyond what the persona agents cover. Spawn them when the diff includes database migrations, schema.rb, or data backfills.
@@ -46,5 +58,6 @@ These CE-native agents provide specialized analysis beyond what the persona agen
1. **Always spawn all 3 always-on personas** plus the 2 CE always-on agents.
2. **For each conditional persona**, the orchestrator reads the diff and decides whether the persona's domain is relevant. This is a judgment call, not a keyword match.
3. **For CE conditional agents**, spawn when the diff includes migration files (`db/migrate/*.rb`, `db/schema.rb`) or data backfill scripts.
4. **Announce the team** before spawning with a one-line justification per conditional reviewer selected.
3. **For language/framework conditional personas**, spawn when the diff contains files matching the persona's language or framework domain. Multiple language personas can be active simultaneously (e.g., both `python-quality` and `typescript-quality` if the diff touches both).
4. **For CE conditional agents**, spawn when the diff includes migration files (`db/migrate/*.rb`, `db/schema.rb`) or data backfill scripts.
5. **Announce the team** before spawning with a one-line justification per conditional reviewer selected.

185
plugins/compound-engineering/skills/dhh-rails-style/SKILL.md
View File

@@ -1,185 +0,0 @@
---
name: dhh-rails-style
description: This skill should be used when writing Ruby and Rails code in DHH's distinctive 37signals style. It applies when writing Ruby code, Rails applications, creating models, controllers, or any Ruby file. Triggers on Ruby/Rails code generation, refactoring requests, code review, or when the user mentions DHH, 37signals, Basecamp, HEY, or Campfire style. Embodies REST purity, fat models, thin controllers, Current attributes, Hotwire patterns, and the "clarity over cleverness" philosophy.
---
<objective>
Apply 37signals/DHH Rails conventions to Ruby and Rails code. This skill provides comprehensive domain expertise extracted from analyzing production 37signals codebases (Fizzy/Campfire) and DHH's code review patterns.
</objective>
<essential_principles>
## Core Philosophy
"The best code is the code you don't write. The second best is the code that's obviously correct."
**Vanilla Rails is plenty:**
- Rich domain models over service objects
- CRUD controllers over custom actions
- Concerns for horizontal code sharing
- Records as state instead of boolean columns
- Database-backed everything (no Redis)
- Build solutions before reaching for gems
**What they deliberately avoid:**
- devise (custom ~150-line auth instead)
- pundit/cancancan (simple role checks in models)
- sidekiq (Solid Queue uses database)
- redis (database for everything)
- view_component (partials work fine)
- GraphQL (REST with Turbo sufficient)
- factory_bot (fixtures are simpler)
- rspec (Minitest ships with Rails)
- Tailwind (native CSS with layers)
**Development Philosophy:**
- Ship, Validate, Refine - prototype-quality code to production to learn
- Fix root causes, not symptoms
- Write-time operations over read-time computations
- Database constraints over ActiveRecord validations
</essential_principles>
<intake>
What are you working on?
1. **Controllers** - REST mapping, concerns, Turbo responses, API patterns
2. **Models** - Concerns, state records, callbacks, scopes, POROs
3. **Views & Frontend** - Turbo, Stimulus, CSS, partials
4. **Architecture** - Routing, multi-tenancy, authentication, jobs, caching
5. **Testing** - Minitest, fixtures, integration tests
6. **Gems & Dependencies** - What to use vs avoid
7. **Code Review** - Review code against DHH style
8. **General Guidance** - Philosophy and conventions
**Specify a number or describe your task.**
</intake>
<routing>
| Response | Reference to Read |
|----------|-------------------|
| 1, controller | [controllers.md](./references/controllers.md) |
| 2, model | [models.md](./references/models.md) |
| 3, view, frontend, turbo, stimulus, css | [frontend.md](./references/frontend.md) |
| 4, architecture, routing, auth, job, cache | [architecture.md](./references/architecture.md) |
| 5, test, testing, minitest, fixture | [testing.md](./references/testing.md) |
| 6, gem, dependency, library | [gems.md](./references/gems.md) |
| 7, review | Read all references, then review code |
| 8, general task | Read relevant references based on context |
**After reading relevant references, apply patterns to the user's code.**
</routing>
<quick_reference>
## Naming Conventions
**Verbs:** `card.close`, `card.gild`, `board.publish` (not `set_style` methods)
**Predicates:** `card.closed?`, `card.golden?` (derived from presence of related record)
**Concerns:** Adjectives describing capability (`Closeable`, `Publishable`, `Watchable`)
**Controllers:** Nouns matching resources (`Cards::ClosuresController`)
**Scopes:**
- `chronologically`, `reverse_chronologically`, `alphabetically`, `latest`
- `preloaded` (standard eager loading name)
- `indexed_by`, `sorted_by` (parameterized)
- `active`, `unassigned` (business terms, not SQL-ish)
## REST Mapping
Instead of custom actions, create new resources:
```
POST /cards/:id/close → POST /cards/:id/closure
DELETE /cards/:id/close → DELETE /cards/:id/closure
POST /cards/:id/archive → POST /cards/:id/archival
```
## Ruby Syntax Preferences
```ruby
# Symbol arrays with spaces inside brackets
before_action :set_message, only: %i[ show edit update destroy ]
# Private method indentation
private
def set_message
@message = Message.find(params[:id])
end
# Expression-less case for conditionals
case
when params[:before].present?
messages.page_before(params[:before])
else
messages.last_page
end
# Bang methods for fail-fast
@message = Message.create!(params)
# Ternaries for simple conditionals
@room.direct? ? @room.users : @message.mentionees
```
## Key Patterns
**State as Records:**
```ruby
Card.joins(:closure) # closed cards
Card.where.missing(:closure) # open cards
```
**Current Attributes:**
```ruby
belongs_to :creator, default: -> { Current.user }
```
**Authorization on Models:**
```ruby
class User < ApplicationRecord
def can_administer?(message)
message.creator == self || admin?
end
end
```
</quick_reference>
<reference_index>
## Domain Knowledge
All detailed patterns in `references/`:
| File | Topics |
|------|--------|
| [controllers.md](./references/controllers.md) | REST mapping, concerns, Turbo responses, API patterns, HTTP caching |
| [models.md](./references/models.md) | Concerns, state records, callbacks, scopes, POROs, authorization, broadcasting |
| [frontend.md](./references/frontend.md) | Turbo Streams, Stimulus controllers, CSS layers, OKLCH colors, partials |
| [architecture.md](./references/architecture.md) | Routing, authentication, jobs, Current attributes, caching, database patterns |
| [testing.md](./references/testing.md) | Minitest, fixtures, unit/integration/system tests, testing patterns |
| [gems.md](./references/gems.md) | What they use vs avoid, decision framework, Gemfile examples |
</reference_index>
<success_criteria>
Code follows DHH style when:
- Controllers map to CRUD verbs on resources
- Models use concerns for horizontal behavior
- State is tracked via records, not booleans
- No unnecessary service objects or abstractions
- Database-backed solutions preferred over external services
- Tests use Minitest with fixtures
- Turbo/Stimulus for interactivity (no heavy JS frameworks)
- Native CSS with modern features (layers, OKLCH, nesting)
- Authorization logic lives on User model
- Jobs are shallow wrappers calling model methods
</success_criteria>
<credits>
Based on [The Unofficial 37signals/DHH Rails Style Guide](https://github.com/marckohlbrugge/unofficial-37signals-coding-style-guide) by [Marc Köhlbrugge](https://x.com/marckohlbrugge), generated through deep analysis of 265 pull requests from the Fizzy codebase.
**Important Disclaimers:**
- LLM-generated guide - may contain inaccuracies
- Code examples from Fizzy are licensed under the O'Saasy License
- Not affiliated with or endorsed by 37signals
</credits>

653
plugins/compound-engineering/skills/dhh-rails-style/references/architecture.md
View File

@@ -1,653 +0,0 @@
# Architecture - DHH Rails Style
<routing>
## Routing
Everything maps to CRUD. Nested resources for related actions:
```ruby
Rails.application.routes.draw do
resources :boards do
resources :cards do
resource :closure
resource :goldness
resource :not_now
resources :assignments
resources :comments
end
end
end
```
**Verb-to-noun conversion:**
| Action | Resource |
|--------|----------|
| close a card | `card.closure` |
| watch a board | `board.watching` |
| mark as golden | `card.goldness` |
| archive a card | `card.archival` |
**Shallow nesting** - avoid deep URLs:
```ruby
resources :boards do
resources :cards, shallow: true # /boards/:id/cards, but /cards/:id
end
```
**Singular resources** for one-per-parent:
```ruby
resource :closure # not resources
resource :goldness
```
**Resolve for URL generation:**
```ruby
# config/routes.rb
resolve("Comment") { |comment| [comment.card, anchor: dom_id(comment)] }
# Now url_for(@comment) works correctly
```
</routing>
<multi_tenancy>
## Multi-Tenancy (Path-Based)
**Middleware extracts tenant** from URL prefix:
```ruby
# lib/tenant_extractor.rb
class TenantExtractor
def initialize(app)
@app = app
end
def call(env)
path = env["PATH_INFO"]
if match = path.match(%r{^/(\d+)(/.*)?$})
env["SCRIPT_NAME"] = "/#{match[1]}"
env["PATH_INFO"] = match[2] || "/"
end
@app.call(env)
end
end
```
**Cookie scoping** per tenant:
```ruby
# Cookies scoped to tenant path
cookies.signed[:session_id] = {
value: session.id,
path: "/#{Current.account.id}"
}
```
**Background job context** - serialize tenant:
```ruby
class ApplicationJob < ActiveJob::Base
around_perform do |job, block|
Current.set(account: job.arguments.first.account) { block.call }
end
end
```
**Recurring jobs** must iterate all tenants:
```ruby
class DailyDigestJob < ApplicationJob
def perform
Account.find_each do |account|
Current.set(account: account) do
send_digest_for(account)
end
end
end
end
```
**Controller security** - always scope through tenant:
```ruby
# Good - scoped through user's accessible records
@card = Current.user.accessible_cards.find(params[:id])
# Avoid - direct lookup
@card = Card.find(params[:id])
```
</multi_tenancy>
<authentication>
## Authentication
Custom passwordless magic link auth (~150 lines total):
```ruby
# app/models/session.rb
class Session < ApplicationRecord
belongs_to :user
before_create { self.token = SecureRandom.urlsafe_base64(32) }
end
# app/models/magic_link.rb
class MagicLink < ApplicationRecord
belongs_to :user
before_create do
self.code = SecureRandom.random_number(100_000..999_999).to_s
self.expires_at = 15.minutes.from_now
end
def expired?
expires_at < Time.current
end
end
```
**Why not Devise:**
- ~150 lines vs massive dependency
- No password storage liability
- Simpler UX for users
- Full control over flow
**Bearer token** for APIs:
```ruby
module Authentication
extend ActiveSupport::Concern
included do
before_action :authenticate
end
private
def authenticate
if bearer_token = request.headers["Authorization"]&.split(" ")&.last
Current.session = Session.find_by(token: bearer_token)
else
Current.session = Session.find_by(id: cookies.signed[:session_id])
end
redirect_to login_path unless Current.session
end
end
```
</authentication>
<background_jobs>
## Background Jobs
Jobs are shallow wrappers calling model methods:
```ruby
class NotifyWatchersJob < ApplicationJob
def perform(card)
card.notify_watchers
end
end
```
**Naming convention:**
- `_later` suffix for async: `card.notify_watchers_later`
- `_now` suffix for immediate: `card.notify_watchers_now`
```ruby
module Watchable
def notify_watchers_later
NotifyWatchersJob.perform_later(self)
end
def notify_watchers_now
NotifyWatchersJob.perform_now(self)
end
def notify_watchers
watchers.each do |watcher|
WatcherMailer.notification(watcher, self).deliver_later
end
end
end
```
**Database-backed** with Solid Queue:
- No Redis required
- Same transactional guarantees as your data
- Simpler infrastructure
**Transaction safety:**
```ruby
# config/application.rb
config.active_job.enqueue_after_transaction_commit = true
```
**Error handling** by type:
```ruby
class DeliveryJob < ApplicationJob
# Transient errors - retry with backoff
retry_on Net::OpenTimeout, Net::ReadTimeout,
Resolv::ResolvError,
wait: :polynomially_longer
# Permanent errors - log and discard
discard_on Net::SMTPSyntaxError do |job, error|
Sentry.capture_exception(error, level: :info)
end
end
```
**Batch processing** with continuable:
```ruby
class ProcessCardsJob < ApplicationJob
include ActiveJob::Continuable
def perform
Card.in_batches.each_record do |card|
checkpoint! # Resume from here if interrupted
process(card)
end
end
end
```
</background_jobs>
<database_patterns>
## Database Patterns
**UUIDs as primary keys** (time-sortable UUIDv7):
```ruby
# migration
create_table :cards, id: :uuid do |t|
t.references :board, type: :uuid, foreign_key: true
end
```
Benefits: No ID enumeration, distributed-friendly, client-side generation.
**State as records** (not booleans):
```ruby
# Instead of closed: boolean
class Card::Closure < ApplicationRecord
belongs_to :card
belongs_to :creator, class_name: "User"
end
# Queries become joins
Card.joins(:closure) # closed
Card.where.missing(:closure) # open
```
**Hard deletes** - no soft delete:
```ruby
# Just destroy
card.destroy!
# Use events for history
card.record_event(:deleted, by: Current.user)
```
Simplifies queries, uses event logs for auditing.
**Counter caches** for performance:
```ruby
class Comment < ApplicationRecord
belongs_to :card, counter_cache: true
end
# card.comments_count available without query
```
**Account scoping** on every table:
```ruby
class Card < ApplicationRecord
belongs_to :account
default_scope { where(account: Current.account) }
end
```
</database_patterns>
<current_attributes>
## Current Attributes
Use `Current` for request-scoped state:
```ruby
# app/models/current.rb
class Current < ActiveSupport::CurrentAttributes
attribute :session, :user, :account, :request_id
delegate :user, to: :session, allow_nil: true
def account=(account)
super
Time.zone = account&.time_zone || "UTC"
end
end
```
Set in controller:
```ruby
class ApplicationController < ActionController::Base
before_action :set_current_request
private
def set_current_request
Current.session = authenticated_session
Current.account = Account.find(params[:account_id])
Current.request_id = request.request_id
end
end
```
Use throughout app:
```ruby
class Card < ApplicationRecord
belongs_to :creator, default: -> { Current.user }
end
```
</current_attributes>
<caching>
## Caching
**HTTP caching** with ETags:
```ruby
fresh_when etag: [@card, Current.user.timezone]
```
**Fragment caching:**
```erb
<% cache card do %>
<%= render card %>
<% end %>
```
**Russian doll caching:**
```erb
<% cache @board do %>
<% @board.cards.each do |card| %>
<% cache card do %>
<%= render card %>
<% end %>
<% end %>
<% end %>
```
**Cache invalidation** via `touch: true`:
```ruby
class Card < ApplicationRecord
belongs_to :board, touch: true
end
```
**Solid Cache** - database-backed:
- No Redis required
- Consistent with application data
- Simpler infrastructure
</caching>
<configuration>
## Configuration
**ENV.fetch with defaults:**
```ruby
# config/application.rb
config.active_job.queue_adapter = ENV.fetch("QUEUE_ADAPTER", "solid_queue").to_sym
config.cache_store = ENV.fetch("CACHE_STORE", "solid_cache").to_sym
```
**Multiple databases:**
```yaml
# config/database.yml
production:
primary:
<<: *default
cable:
<<: *default
migrations_paths: db/cable_migrate
queue:
<<: *default
migrations_paths: db/queue_migrate
cache:
<<: *default
migrations_paths: db/cache_migrate
```
**Switch between SQLite and MySQL via ENV:**
```ruby
adapter = ENV.fetch("DATABASE_ADAPTER", "sqlite3")
```
**CSP extensible via ENV:**
```ruby
config.content_security_policy do |policy|
policy.default_src :self
policy.script_src :self, *ENV.fetch("CSP_SCRIPT_SRC", "").split(",")
end
```
</configuration>
<testing>
## Testing
**Minitest**, not RSpec:
```ruby
class CardTest < ActiveSupport::TestCase
test "closing a card creates a closure" do
card = cards(:one)
card.close
assert card.closed?
assert_not_nil card.closure
end
end
```
**Fixtures** instead of factories:
```yaml
# test/fixtures/cards.yml
one:
title: First Card
board: main
creator: alice
two:
title: Second Card
board: main
creator: bob
```
**Integration tests** for controllers:
```ruby
class CardsControllerTest < ActionDispatch::IntegrationTest
test "closing a card" do
card = cards(:one)
sign_in users(:alice)
post card_closure_path(card)
assert_response :success
assert card.reload.closed?
end
end
```
**Tests ship with features** - same commit, not TDD-first but together.
**Regression tests for security fixes** - always.
</testing>
<events>
## Event Tracking
Events are the single source of truth:
```ruby
class Event < ApplicationRecord
belongs_to :creator, class_name: "User"
belongs_to :eventable, polymorphic: true
serialize :particulars, coder: JSON
end
```
**Eventable concern:**
```ruby
module Eventable
extend ActiveSupport::Concern
included do
has_many :events, as: :eventable, dependent: :destroy
end
def record_event(action, particulars = {})
events.create!(
creator: Current.user,
action: action,
particulars: particulars
)
end
end
```
**Webhooks driven by events** - events are the canonical source.
</events>
<email_patterns>
## Email Patterns
**Multi-tenant URL helpers:**
```ruby
class ApplicationMailer < ActionMailer::Base
def default_url_options
options = super
if Current.account
options[:script_name] = "/#{Current.account.id}"
end
options
end
end
```
**Timezone-aware delivery:**
```ruby
class NotificationMailer < ApplicationMailer
def daily_digest(user)
Time.use_zone(user.timezone) do
@user = user
@digest = user.digest_for_today
mail(to: user.email, subject: "Daily Digest")
end
end
end
```
**Batch delivery:**
```ruby
emails = users.map { |user| NotificationMailer.digest(user) }
ActiveJob.perform_all_later(emails.map(&:deliver_later))
```
**One-click unsubscribe (RFC 8058):**
```ruby
class ApplicationMailer < ActionMailer::Base
after_action :set_unsubscribe_headers
private
def set_unsubscribe_headers
headers["List-Unsubscribe-Post"] = "List-Unsubscribe=One-Click"
headers["List-Unsubscribe"] = "<#{unsubscribe_url}>"
end
end
```
</email_patterns>
<security_patterns>
## Security Patterns
**XSS prevention** - escape in helpers:
```ruby
def formatted_content(text)
# Escape first, then mark safe
simple_format(h(text)).html_safe
end
```
**SSRF protection:**
```ruby
# Resolve DNS once, pin the IP
def fetch_safely(url)
uri = URI.parse(url)
ip = Resolv.getaddress(uri.host)
# Block private networks
raise "Private IP" if private_ip?(ip)
# Use pinned IP for request
Net::HTTP.start(uri.host, uri.port, ipaddr: ip) { |http| ... }
end
def private_ip?(ip)
ip.start_with?("127.", "10.", "192.168.") ||
ip.match?(/^172\.(1[6-9]|2[0-9]|3[0-1])\./)
end
```
**Content Security Policy:**
```ruby
# config/initializers/content_security_policy.rb
Rails.application.configure do
config.content_security_policy do |policy|
policy.default_src :self
policy.script_src :self
policy.style_src :self, :unsafe_inline
policy.base_uri :none
policy.form_action :self
policy.frame_ancestors :self
end
end
```
**ActionText sanitization:**
```ruby
# config/initializers/action_text.rb
Rails.application.config.after_initialize do
ActionText::ContentHelper.allowed_tags = %w[
strong em a ul ol li p br h1 h2 h3 h4 blockquote
]
end
```
</security_patterns>
<active_storage>
## Active Storage Patterns
**Variant preprocessing:**
```ruby
class User < ApplicationRecord
has_one_attached :avatar do |attachable|
attachable.variant :thumb, resize_to_limit: [100, 100], preprocessed: true
attachable.variant :medium, resize_to_limit: [300, 300], preprocessed: true
end
end
```
**Direct upload expiry** - extend for slow connections:
```ruby
# config/initializers/active_storage.rb
Rails.application.config.active_storage.service_urls_expire_in = 48.hours
```
**Avatar optimization** - redirect to blob:
```ruby
def show
expires_in 1.year, public: true
redirect_to @user.avatar.variant(:thumb).processed.url, allow_other_host: true
end
```
**Mirror service** for migrations:
```yaml
# config/storage.yml
production:
service: Mirror
primary: amazon
mirrors: [google]
```
</active_storage>

303
plugins/compound-engineering/skills/dhh-rails-style/references/controllers.md
View File

@@ -1,303 +0,0 @@
# Controllers - DHH Rails Style
<rest_mapping>
## Everything Maps to CRUD
Custom actions become new resources. Instead of verbs on existing resources, create noun resources:
```ruby
# Instead of this:
POST /cards/:id/close
DELETE /cards/:id/close
POST /cards/:id/archive
# Do this:
POST /cards/:id/closure # create closure
DELETE /cards/:id/closure # destroy closure
POST /cards/:id/archival # create archival
```
**Real examples from 37signals:**
```ruby
resources :cards do
resource :closure # closing/reopening
resource :goldness # marking important
resource :not_now # postponing
resources :assignments # managing assignees
end
```
Each resource gets its own controller with standard CRUD actions.
</rest_mapping>
<controller_concerns>
## Concerns for Shared Behavior
Controllers use concerns extensively. Common patterns:
**CardScoped** - loads @card, @board, provides render_card_replacement
```ruby
module CardScoped
extend ActiveSupport::Concern
included do
before_action :set_card
end
private
def set_card
@card = Card.find(params[:card_id])
@board = @card.board
end
def render_card_replacement
render turbo_stream: turbo_stream.replace(@card)
end
end
```
**BoardScoped** - loads @board
**CurrentRequest** - populates Current with request data
**CurrentTimezone** - wraps requests in user's timezone
**FilterScoped** - handles complex filtering
**TurboFlash** - flash messages via Turbo Stream
**ViewTransitions** - disables on page refresh
**BlockSearchEngineIndexing** - sets X-Robots-Tag header
**RequestForgeryProtection** - Sec-Fetch-Site CSRF (modern browsers)
</controller_concerns>
<authorization_patterns>
## Authorization Patterns
Controllers check permissions via before_action, models define what permissions mean:
```ruby
# Controller concern
module Authorization
extend ActiveSupport::Concern
private
def ensure_can_administer
head :forbidden unless Current.user.admin?
end
def ensure_is_staff_member
head :forbidden unless Current.user.staff?
end
end
# Usage
class BoardsController < ApplicationController
before_action :ensure_can_administer, only: [:destroy]
end
```
**Model-level authorization:**
```ruby
class Board < ApplicationRecord
def editable_by?(user)
user.admin? || user == creator
end
def publishable_by?(user)
editable_by?(user) && !published?
end
end
```
Keep authorization simple, readable, colocated with domain.
</authorization_patterns>
<security_concerns>
## Security Concerns
**Sec-Fetch-Site CSRF Protection:**
Modern browsers send Sec-Fetch-Site header. Use it for defense in depth:
```ruby
module RequestForgeryProtection
extend ActiveSupport::Concern
included do
before_action :verify_request_origin
end
private
def verify_request_origin
return if request.get? || request.head?
return if %w[same-origin same-site].include?(
request.headers["Sec-Fetch-Site"]&.downcase
)
# Fall back to token verification for older browsers
verify_authenticity_token
end
end
```
**Rate Limiting (Rails 8+):**
```ruby
class MagicLinksController < ApplicationController
rate_limit to: 10, within: 15.minutes, only: :create
end
```
Apply to: auth endpoints, email sending, external API calls, resource creation.
</security_concerns>
<request_context>
## Request Context Concerns
**CurrentRequest** - populates Current with HTTP metadata:
```ruby
module CurrentRequest
extend ActiveSupport::Concern
included do
before_action :set_current_request
end
private
def set_current_request
Current.request_id = request.request_id
Current.user_agent = request.user_agent
Current.ip_address = request.remote_ip
Current.referrer = request.referrer
end
end
```
**CurrentTimezone** - wraps requests in user's timezone:
```ruby
module CurrentTimezone
extend ActiveSupport::Concern
included do
around_action :set_timezone
helper_method :timezone_from_cookie
end
private
def set_timezone
Time.use_zone(timezone_from_cookie) { yield }
end
def timezone_from_cookie
cookies[:timezone] || "UTC"
end
end
```
**SetPlatform** - detects mobile/desktop:
```ruby
module SetPlatform
extend ActiveSupport::Concern
included do
helper_method :platform
end
def platform
@platform ||= request.user_agent&.match?(/Mobile|Android/) ? :mobile : :desktop
end
end
```
</request_context>
<turbo_responses>
## Turbo Stream Responses
Use Turbo Streams for partial updates:
```ruby
class Cards::ClosuresController < ApplicationController
include CardScoped
def create
@card.close
render_card_replacement
end
def destroy
@card.reopen
render_card_replacement
end
end
```
For complex updates, use morphing:
```ruby
render turbo_stream: turbo_stream.morph(@card)
```
</turbo_responses>
<api_patterns>
## API Design
Same controllers, different format. Convention for responses:
```ruby
def create
@card = Card.create!(card_params)
respond_to do |format|
format.html { redirect_to @card }
format.json { head :created, location: @card }
end
end
def update
@card.update!(card_params)
respond_to do |format|
format.html { redirect_to @card }
format.json { head :no_content }
end
end
def destroy
@card.destroy
respond_to do |format|
format.html { redirect_to cards_path }
format.json { head :no_content }
end
end
```
**Status codes:**
- Create: 201 Created + Location header
- Update: 204 No Content
- Delete: 204 No Content
- Bearer token authentication
</api_patterns>
<http_caching>
## HTTP Caching
Extensive use of ETags and conditional GETs:
```ruby
class CardsController < ApplicationController
def show
@card = Card.find(params[:id])
fresh_when etag: [@card, Current.user.timezone]
end
def index
@cards = @board.cards.preloaded
fresh_when etag: [@cards, @board.updated_at]
end
end
```
Key insight: Times render server-side in user's timezone, so timezone must affect the ETag to prevent serving wrong times to other timezones.
**ApplicationController global etag:**
```ruby
class ApplicationController < ActionController::Base
etag { "v1" } # Bump to invalidate all caches
end
```
Use `touch: true` on associations for cache invalidation.
</http_caching>

510
plugins/compound-engineering/skills/dhh-rails-style/references/frontend.md
View File

@@ -1,510 +0,0 @@
# Frontend - DHH Rails Style
<turbo_patterns>
## Turbo Patterns
**Turbo Streams** for partial updates:
```erb
<%# app/views/cards/closures/create.turbo_stream.erb %>
<%= turbo_stream.replace @card %>
```
**Morphing** for complex updates:
```ruby
render turbo_stream: turbo_stream.morph(@card)
```
**Global morphing** - enable in layout:
```ruby
turbo_refreshes_with method: :morph, scroll: :preserve
```
**Fragment caching** with `cached: true`:
```erb
<%= render partial: "card", collection: @cards, cached: true %>
```
**No ViewComponents** - standard partials work fine.
</turbo_patterns>
<turbo_morphing>
## Turbo Morphing Best Practices
**Listen for morph events** to restore client state:
```javascript
document.addEventListener("turbo:morph-element", (event) => {
// Restore any client-side state after morph
})
```
**Permanent elements** - skip morphing with data attribute:
```erb
<div data-turbo-permanent id="notification-count">
<%= @count %>
</div>
```
**Frame morphing** - add refresh attribute:
```erb
<%= turbo_frame_tag :assignment, src: path, refresh: :morph %>
```
**Common issues and solutions:**
| Problem | Solution |
|---------|----------|
| Timers not updating | Clear/restart in morph event listener |
| Forms resetting | Wrap form sections in turbo frames |
| Pagination breaking | Use turbo frames with `refresh: :morph` |
| Flickering on replace | Switch to morph instead of replace |
| localStorage loss | Listen to `turbo:morph-element`, restore state |
</turbo_morphing>
<turbo_frames>
## Turbo Frames
**Lazy loading** with spinner:
```erb
<%= turbo_frame_tag "menu",
src: menu_path,
loading: :lazy do %>
<div class="spinner">Loading...</div>
<% end %>
```
**Inline editing** with edit/view toggle:
```erb
<%= turbo_frame_tag dom_id(card, :edit) do %>
<%= link_to "Edit", edit_card_path(card),
data: { turbo_frame: dom_id(card, :edit) } %>
<% end %>
```
**Target parent frame** without hardcoding:
```erb
<%= form_with model: @card, data: { turbo_frame: "_parent" } do |f| %>
```
**Real-time subscriptions:**
```erb
<%= turbo_stream_from @card %>
<%= turbo_stream_from @card, :activity %>
```
</turbo_frames>
<stimulus_controllers>
## Stimulus Controllers
52 controllers in Fizzy, split 62% reusable, 38% domain-specific.
**Characteristics:**
- Single responsibility per controller
- Configuration via values/classes
- Events for communication
- Private methods with #
- Most under 50 lines
**Examples:**
```javascript
// copy-to-clipboard (25 lines)
import { Controller } from "@hotwired/stimulus"
export default class extends Controller {
static values = { content: String }
copy() {
navigator.clipboard.writeText(this.contentValue)
this.#showFeedback()
}
#showFeedback() {
this.element.classList.add("copied")
setTimeout(() => this.element.classList.remove("copied"), 1500)
}
}
```
```javascript
// auto-click (7 lines)
import { Controller } from "@hotwired/stimulus"
export default class extends Controller {
connect() {
this.element.click()
}
}
```
```javascript
// toggle-class (31 lines)
import { Controller } from "@hotwired/stimulus"
export default class extends Controller {
static classes = ["toggle"]
static values = { open: { type: Boolean, default: false } }
toggle() {
this.openValue = !this.openValue
}
openValueChanged() {
this.element.classList.toggle(this.toggleClass, this.openValue)
}
}
```
```javascript
// auto-submit (28 lines) - debounced form submission
import { Controller } from "@hotwired/stimulus"
export default class extends Controller {
static values = { delay: { type: Number, default: 300 } }
connect() {
this.timeout = null
}
submit() {
clearTimeout(this.timeout)
this.timeout = setTimeout(() => {
this.element.requestSubmit()
}, this.delayValue)
}
disconnect() {
clearTimeout(this.timeout)
}
}
```
```javascript
// dialog (45 lines) - native HTML dialog management
import { Controller } from "@hotwired/stimulus"
export default class extends Controller {
open() {
this.element.showModal()
}
close() {
this.element.close()
this.dispatch("closed")
}
clickOutside(event) {
if (event.target === this.element) this.close()
}
}
```
```javascript
// local-time (40 lines) - relative time display
import { Controller } from "@hotwired/stimulus"
export default class extends Controller {
static values = { datetime: String }
connect() {
this.#updateTime()
}
#updateTime() {
const date = new Date(this.datetimeValue)
const now = new Date()
const diffMinutes = Math.floor((now - date) / 60000)
if (diffMinutes < 60) {
this.element.textContent = `${diffMinutes}m ago`
} else if (diffMinutes < 1440) {
this.element.textContent = `${Math.floor(diffMinutes / 60)}h ago`
} else {
this.element.textContent = `${Math.floor(diffMinutes / 1440)}d ago`
}
}
}
```
</stimulus_controllers>
<stimulus_best_practices>
## Stimulus Best Practices
**Values API** over getAttribute:
```javascript
// Good
static values = { delay: { type: Number, default: 300 } }
// Avoid
this.element.getAttribute("data-delay")
```
**Cleanup in disconnect:**
```javascript
disconnect() {
clearTimeout(this.timeout)
this.observer?.disconnect()
document.removeEventListener("keydown", this.boundHandler)
}
```
**Action filters** - `:self` prevents bubbling:
```erb
<div data-action="click->menu#toggle:self">
```
**Helper extraction** - shared utilities in separate modules:
```javascript
// app/javascript/helpers/timing.js
export function debounce(fn, delay) {
let timeout
return (...args) => {
clearTimeout(timeout)
timeout = setTimeout(() => fn(...args), delay)
}
}
```
**Event dispatching** for loose coupling:
```javascript
this.dispatch("selected", { detail: { id: this.idValue } })
```
</stimulus_best_practices>
<view_helpers>
## View Helpers (Stimulus-Integrated)
**Dialog helper:**
```ruby
def dialog_tag(id, &block)
tag.dialog(
id: id,
data: {
controller: "dialog",
action: "click->dialog#clickOutside keydown.esc->dialog#close"
},
&block
)
end
```
**Auto-submit form helper:**
```ruby
def auto_submit_form_with(model:, delay: 300, **options, &block)
form_with(
model: model,
data: {
controller: "auto-submit",
auto_submit_delay_value: delay,
action: "input->auto-submit#submit"
},
**options,
&block
)
end
```
**Copy button helper:**
```ruby
def copy_button(content:, label: "Copy")
tag.button(
label,
data: {
controller: "copy",
copy_content_value: content,
action: "click->copy#copy"
}
)
end
```
</view_helpers>
<css_architecture>
## CSS Architecture
Vanilla CSS with modern features, no preprocessors.
**CSS @layer** for cascade control:
```css
@layer reset, base, components, modules, utilities;
@layer reset {
*, *::before, *::after { box-sizing: border-box; }
}
@layer base {
body { font-family: var(--font-sans); }
}
@layer components {
.btn { /* button styles */ }
}
@layer modules {
.card { /* card module styles */ }
}
@layer utilities {
.hidden { display: none; }
}
```
**OKLCH color system** for perceptual uniformity:
```css
:root {
--color-primary: oklch(60% 0.15 250);
--color-success: oklch(65% 0.2 145);
--color-warning: oklch(75% 0.15 85);
--color-danger: oklch(55% 0.2 25);
}
```
**Dark mode** via CSS variables:
```css
:root {
--bg: oklch(98% 0 0);
--text: oklch(20% 0 0);
}
@media (prefers-color-scheme: dark) {
:root {
--bg: oklch(15% 0 0);
--text: oklch(90% 0 0);
}
}
```
**Native CSS nesting:**
```css
.card {
padding: var(--space-4);
& .title {
font-weight: bold;
}
&:hover {
background: var(--bg-hover);
}
}
```
**~60 minimal utilities** vs Tailwind's hundreds.
**Modern features used:**
- `@starting-style` for enter animations
- `color-mix()` for color manipulation
- `:has()` for parent selection
- Logical properties (`margin-inline`, `padding-block`)
- Container queries
</css_architecture>
<view_patterns>
## View Patterns
**Standard partials** - no ViewComponents:
```erb
<%# app/views/cards/_card.html.erb %>
<article id="<%= dom_id(card) %>" class="card">
<%= render "cards/header", card: card %>
<%= render "cards/body", card: card %>
<%= render "cards/footer", card: card %>
</article>
```
**Fragment caching:**
```erb
<% cache card do %>
<%= render "cards/card", card: card %>
<% end %>
```
**Collection caching:**
```erb
<%= render partial: "card", collection: @cards, cached: true %>
```
**Simple component naming** - no strict BEM:
```css
.card { }
.card .title { }
.card .actions { }
.card.golden { }
.card.closed { }
```
</view_patterns>
<caching_with_personalization>
## User-Specific Content in Caches
Move personalization to client-side JavaScript to preserve caching:
```erb
<%# Cacheable fragment %>
<% cache card do %>
<article class="card"
data-creator-id="<%= card.creator_id %>"
data-controller="ownership"
data-ownership-current-user-value="<%= Current.user.id %>">
<button data-ownership-target="ownerOnly" class="hidden">Delete</button>
</article>
<% end %>
```
```javascript
// Reveal user-specific elements after cache hit
export default class extends Controller {
static values = { currentUser: Number }
static targets = ["ownerOnly"]
connect() {
const creatorId = parseInt(this.element.dataset.creatorId)
if (creatorId === this.currentUserValue) {
this.ownerOnlyTargets.forEach(el => el.classList.remove("hidden"))
}
}
}
```
**Extract dynamic content** to separate frames:
```erb
<% cache [card, board] do %>
<article class="card">
<%= turbo_frame_tag card, :assignment,
src: card_assignment_path(card),
refresh: :morph %>
</article>
<% end %>
```
Assignment dropdown updates independently without invalidating parent cache.
</caching_with_personalization>
<broadcasting>
## Broadcasting with Turbo Streams
**Model callbacks** for real-time updates:
```ruby
class Card < ApplicationRecord
include Broadcastable
after_create_commit :broadcast_created
after_update_commit :broadcast_updated
after_destroy_commit :broadcast_removed
private
def broadcast_created
broadcast_append_to [Current.account, board], :cards
end
def broadcast_updated
broadcast_replace_to [Current.account, board], :cards
end
def broadcast_removed
broadcast_remove_to [Current.account, board], :cards
end
end
```
**Scope by tenant** using `[Current.account, resource]` pattern.
</broadcasting>

266
plugins/compound-engineering/skills/dhh-rails-style/references/gems.md
View File

@@ -1,266 +0,0 @@
# Gems - DHH Rails Style
<what_they_use>
## What 37signals Uses
**Core Rails stack:**
- turbo-rails, stimulus-rails, importmap-rails
- propshaft (asset pipeline)
**Database-backed services (Solid suite):**
- solid_queue - background jobs
- solid_cache - caching
- solid_cable - WebSockets/Action Cable
**Authentication & Security:**
- bcrypt (for any password hashing needed)
**Their own gems:**
- geared_pagination (cursor-based pagination)
- lexxy (rich text editor)
- mittens (mailer utilities)
**Utilities:**
- rqrcode (QR code generation)
- redcarpet + rouge (Markdown rendering)
- web-push (push notifications)
**Deployment & Operations:**
- kamal (Docker deployment)
- thruster (HTTP/2 proxy)
- mission_control-jobs (job monitoring)
- autotuner (GC tuning)
</what_they_use>
<what_they_avoid>
## What They Deliberately Avoid
**Authentication:**
```
devise → Custom ~150-line auth
```
Why: Full control, no password liability with magic links, simpler.
**Authorization:**
```
pundit/cancancan → Simple role checks in models
```
Why: Most apps don't need policy objects. A method on the model suffices:
```ruby
class Board < ApplicationRecord
def editable_by?(user)
user.admin? || user == creator
end
end
```
**Background Jobs:**
```
sidekiq → Solid Queue
```
Why: Database-backed means no Redis, same transactional guarantees.
**Caching:**
```
redis → Solid Cache
```
Why: Database is already there, simpler infrastructure.
**Search:**
```
elasticsearch → Custom sharded search
```
Why: Built exactly what they need, no external service dependency.
**View Layer:**
```
view_component → Standard partials
```
Why: Partials work fine. ViewComponents add complexity without clear benefit for their use case.
**API:**
```
GraphQL → REST with Turbo
```
Why: REST is sufficient when you control both ends. GraphQL complexity not justified.
**Factories:**
```
factory_bot → Fixtures
```
Why: Fixtures are simpler, faster, and encourage thinking about data relationships upfront.
**Service Objects:**
```
Interactor, Trailblazer → Fat models
```
Why: Business logic stays in models. Methods like `card.close` instead of `CardCloser.call(card)`.
**Form Objects:**
```
Reform, dry-validation → params.expect + model validations
```
Why: Rails 7.1's `params.expect` is clean enough. Contextual validations on model.
**Decorators:**
```
Draper → View helpers + partials
```
Why: Helpers and partials are simpler. No decorator indirection.
**CSS:**
```
Tailwind, Sass → Native CSS
```
Why: Modern CSS has nesting, variables, layers. No build step needed.
**Frontend:**
```
React, Vue, SPAs → Turbo + Stimulus
```
Why: Server-rendered HTML with sprinkles of JS. SPA complexity not justified.
**Testing:**
```
RSpec → Minitest
```
Why: Simpler, faster boot, less DSL magic, ships with Rails.
</what_they_avoid>
<testing_philosophy>
## Testing Philosophy
**Minitest** - simpler, faster:
```ruby
class CardTest < ActiveSupport::TestCase
test "closing creates closure" do
card = cards(:one)
assert_difference -> { Card::Closure.count } do
card.close
end
assert card.closed?
end
end
```
**Fixtures** - loaded once, deterministic:
```yaml
# test/fixtures/cards.yml
open_card:
title: Open Card
board: main
creator: alice
closed_card:
title: Closed Card
board: main
creator: bob
```
**Dynamic timestamps** with ERB:
```yaml
recent:
title: Recent
created_at: <%= 1.hour.ago %>
old:
title: Old
created_at: <%= 1.month.ago %>
```
**Time travel** for time-dependent tests:
```ruby
test "expires after 15 minutes" do
magic_link = MagicLink.create!(user: users(:alice))
travel 16.minutes
assert magic_link.expired?
end
```
**VCR** for external APIs:
```ruby
VCR.use_cassette("stripe/charge") do
charge = Stripe::Charge.create(amount: 1000)
assert charge.paid
end
```
**Tests ship with features** - same commit, not before or after.
</testing_philosophy>
<decision_framework>
## Decision Framework
Before adding a gem, ask:
1. **Can vanilla Rails do this?**
- ActiveRecord can do most things Sequel can
- ActionMailer handles email fine
- ActiveJob works for most job needs
2. **Is the complexity worth it?**
- 150 lines of custom code vs. 10,000-line gem
- You'll understand your code better
- Fewer upgrade headaches
3. **Does it add infrastructure?**
- Redis? Consider database-backed alternatives
- External service? Consider building in-house
- Simpler infrastructure = fewer failure modes
4. **Is it from someone you trust?**
- 37signals gems: battle-tested at scale
- Well-maintained, focused gems: usually fine
- Kitchen-sink gems: probably overkill
**The philosophy:**
> "Build solutions before reaching for gems."
Not anti-gem, but pro-understanding. Use gems when they genuinely solve a problem you have, not a problem you might have.
</decision_framework>
<gem_patterns>
## Gem Usage Patterns
**Pagination:**
```ruby
# geared_pagination - cursor-based
class CardsController < ApplicationController
def index
@cards = @board.cards.geared(page: params[:page])
end
end
```
**Markdown:**
```ruby
# redcarpet + rouge
class MarkdownRenderer
def self.render(text)
Redcarpet::Markdown.new(
Redcarpet::Render::HTML.new(filter_html: true),
autolink: true,
fenced_code_blocks: true
).render(text)
end
end
```
**Background jobs:**
```ruby
# solid_queue - no Redis
class ApplicationJob < ActiveJob::Base
queue_as :default
# Just works, backed by database
end
```
**Caching:**
```ruby
# solid_cache - no Redis
# config/environments/production.rb
config.cache_store = :solid_cache_store
```
</gem_patterns>

359
plugins/compound-engineering/skills/dhh-rails-style/references/models.md
View File

@@ -1,359 +0,0 @@
# Models - DHH Rails Style
<model_concerns>
## Concerns for Horizontal Behavior
Models heavily use concerns. A typical Card model includes 14+ concerns:
```ruby
class Card < ApplicationRecord
include Assignable
include Attachments
include Broadcastable
include Closeable
include Colored
include Eventable
include Golden
include Mentions
include Multistep
include Pinnable
include Postponable
include Readable
include Searchable
include Taggable
include Watchable
end
```
Each concern is self-contained with associations, scopes, and methods.
**Naming:** Adjectives describing capability (`Closeable`, `Publishable`, `Watchable`)
</model_concerns>
<state_records>
## State as Records, Not Booleans
Instead of boolean columns, create separate records:
```ruby
# Instead of:
closed: boolean
is_golden: boolean
postponed: boolean
# Create records:
class Card::Closure < ApplicationRecord
belongs_to :card
belongs_to :creator, class_name: "User"
end
class Card::Goldness < ApplicationRecord
belongs_to :card
belongs_to :creator, class_name: "User"
end
class Card::NotNow < ApplicationRecord
belongs_to :card
belongs_to :creator, class_name: "User"
end
```
**Benefits:**
- Automatic timestamps (when it happened)
- Track who made changes
- Easy filtering via joins and `where.missing`
- Enables rich UI showing when/who
**In the model:**
```ruby
module Closeable
extend ActiveSupport::Concern
included do
has_one :closure, dependent: :destroy
end
def closed?
closure.present?
end
def close(creator: Current.user)
create_closure!(creator: creator)
end
def reopen
closure&.destroy
end
end
```
**Querying:**
```ruby
Card.joins(:closure) # closed cards
Card.where.missing(:closure) # open cards
```
</state_records>
<callbacks>
## Callbacks - Used Sparingly
Only 38 callback occurrences across 30 files in Fizzy. Guidelines:
**Use for:**
- `after_commit` for async work
- `before_save` for derived data
- `after_create_commit` for side effects
**Avoid:**
- Complex callback chains
- Business logic in callbacks
- Synchronous external calls
```ruby
class Card < ApplicationRecord
after_create_commit :notify_watchers_later
before_save :update_search_index, if: :title_changed?
private
def notify_watchers_later
NotifyWatchersJob.perform_later(self)
end
end
```
</callbacks>
<scopes>
## Scope Naming
Standard scope names:
```ruby
class Card < ApplicationRecord
scope :chronologically, -> { order(created_at: :asc) }
scope :reverse_chronologically, -> { order(created_at: :desc) }
scope :alphabetically, -> { order(title: :asc) }
scope :latest, -> { reverse_chronologically.limit(10) }
# Standard eager loading
scope :preloaded, -> { includes(:creator, :assignees, :tags) }
# Parameterized
scope :indexed_by, ->(column) { order(column => :asc) }
scope :sorted_by, ->(column, direction = :asc) { order(column => direction) }
end
```
</scopes>
<poros>
## Plain Old Ruby Objects
POROs namespaced under parent models:
```ruby
# app/models/event/description.rb
class Event::Description
def initialize(event)
@event = event
end
def to_s
# Presentation logic for event description
end
end
# app/models/card/eventable/system_commenter.rb
class Card::Eventable::SystemCommenter
def initialize(card)
@card = card
end
def comment(message)
# Business logic
end
end
# app/models/user/filtering.rb
class User::Filtering
# View context bundling
end
```
**NOT used for service objects.** Business logic stays in models.
</poros>
<verbs_predicates>
## Method Naming
**Verbs** - Actions that change state:
```ruby
card.close
card.reopen
card.gild # make golden
card.ungild
board.publish
board.archive
```
**Predicates** - Queries derived from state:
```ruby
card.closed? # closure.present?
card.golden? # goldness.present?
board.published?
```
**Avoid** generic setters:
```ruby
# Bad
card.set_closed(true)
card.update_golden_status(false)
# Good
card.close
card.ungild
```
</verbs_predicates>
<validation_philosophy>
## Validation Philosophy
Minimal validations on models. Use contextual validations on form/operation objects:
```ruby
# Model - minimal
class User < ApplicationRecord
validates :email, presence: true, format: { with: URI::MailTo::EMAIL_REGEXP }
end
# Form object - contextual
class Signup
include ActiveModel::Model
attr_accessor :email, :name, :terms_accepted
validates :email, :name, presence: true
validates :terms_accepted, acceptance: true
def save
return false unless valid?
User.create!(email: email, name: name)
end
end
```
**Prefer database constraints** over model validations for data integrity:
```ruby
# migration
add_index :users, :email, unique: true
add_foreign_key :cards, :boards
```
</validation_philosophy>
<error_handling>
## Let It Crash Philosophy
Use bang methods that raise exceptions on failure:
```ruby
# Preferred - raises on failure
@card = Card.create!(card_params)
@card.update!(title: new_title)
@comment.destroy!
# Avoid - silent failures
@card = Card.create(card_params) # returns false on failure
if @card.save
# ...
end
```
Let errors propagate naturally. Rails handles ActiveRecord::RecordInvalid with 422 responses.
</error_handling>
<default_values>
## Default Values with Lambdas
Use lambda defaults for associations with Current:
```ruby
class Card < ApplicationRecord
belongs_to :creator, class_name: "User", default: -> { Current.user }
belongs_to :account, default: -> { Current.account }
end
class Comment < ApplicationRecord
belongs_to :commenter, class_name: "User", default: -> { Current.user }
end
```
Lambdas ensure dynamic resolution at creation time.
</default_values>
<rails_71_patterns>
## Rails 7.1+ Model Patterns
**Normalizes** - clean data before validation:
```ruby
class User < ApplicationRecord
normalizes :email, with: ->(email) { email.strip.downcase }
normalizes :phone, with: ->(phone) { phone.gsub(/\D/, "") }
end
```
**Delegated Types** - replace polymorphic associations:
```ruby
class Message < ApplicationRecord
delegated_type :messageable, types: %w[Comment Reply Announcement]
end
# Now you get:
message.comment? # true if Comment
message.comment # returns the Comment
Message.comments # scope for Comment messages
```
**Store Accessor** - structured JSON storage:
```ruby
class User < ApplicationRecord
store :settings, accessors: [:theme, :notifications_enabled], coder: JSON
end
user.theme = "dark"
user.notifications_enabled = true
```
</rails_71_patterns>
<concern_guidelines>
## Concern Guidelines
- **50-150 lines** per concern (most are ~100)
- **Cohesive** - related functionality only
- **Named for capabilities** - `Closeable`, `Watchable`, not `CardHelpers`
- **Self-contained** - associations, scopes, methods together
- **Not for mere organization** - create when genuine reuse needed
**Touch chains** for cache invalidation:
```ruby
class Comment < ApplicationRecord
belongs_to :card, touch: true
end
class Card < ApplicationRecord
belongs_to :board, touch: true
end
```
When comment updates, card's `updated_at` changes, which cascades to board.
**Transaction wrapping** for related updates:
```ruby
class Card < ApplicationRecord
def close(creator: Current.user)
transaction do
create_closure!(creator: creator)
record_event(:closed)
notify_watchers_later
end
end
end
```
</concern_guidelines>

338
plugins/compound-engineering/skills/dhh-rails-style/references/testing.md
View File

@@ -1,338 +0,0 @@
# Testing - DHH Rails Style
## Core Philosophy
"Minitest with fixtures - simple, fast, deterministic." The approach prioritizes pragmatism over convention.
## Why Minitest Over RSpec
- **Simpler**: Less DSL magic, plain Ruby assertions
- **Ships with Rails**: No additional dependencies
- **Faster boot times**: Less overhead
- **Plain Ruby**: No specialized syntax to learn
## Fixtures as Test Data
Rather than factories, fixtures provide preloaded data:
- Loaded once, reused across tests
- No runtime object creation overhead
- Explicit relationship visibility
- Deterministic IDs for easier debugging
### Fixture Structure
```yaml
# test/fixtures/users.yml
david:
identity: david
account: basecamp
role: admin
jason:
identity: jason
account: basecamp
role: member
# test/fixtures/rooms.yml
watercooler:
name: Water Cooler
creator: david
direct: false
# test/fixtures/messages.yml
greeting:
body: Hello everyone!
room: watercooler
creator: david
```
### Using Fixtures in Tests
```ruby
test "sending a message" do
user = users(:david)
room = rooms(:watercooler)
# Test with fixture data
end
```
### Dynamic Fixture Values
ERB enables time-sensitive data:
```yaml
recent_card:
title: Recent Card
created_at: <%= 1.hour.ago %>
old_card:
title: Old Card
created_at: <%= 1.month.ago %>
```
## Test Organization
### Unit Tests
Verify business logic using setup blocks and standard assertions:
```ruby
class CardTest < ActiveSupport::TestCase
setup do
@card = cards(:one)
@user = users(:david)
end
test "closing a card creates a closure" do
assert_difference -> { Card::Closure.count } do
@card.close(creator: @user)
end
assert @card.closed?
assert_equal @user, @card.closure.creator
end
test "reopening a card destroys the closure" do
@card.close(creator: @user)
assert_difference -> { Card::Closure.count }, -1 do
@card.reopen
end
refute @card.closed?
end
end
```
### Integration Tests
Test full request/response cycles:
```ruby
class CardsControllerTest < ActionDispatch::IntegrationTest
setup do
@user = users(:david)
sign_in @user
end
test "closing a card" do
card = cards(:one)
post card_closure_path(card)
assert_response :success
assert card.reload.closed?
end
test "unauthorized user cannot close card" do
sign_in users(:guest)
card = cards(:one)
post card_closure_path(card)
assert_response :forbidden
refute card.reload.closed?
end
end
```
### System Tests
Browser-based tests using Capybara:
```ruby
class MessagesTest < ApplicationSystemTestCase
test "sending a message" do
sign_in users(:david)
visit room_path(rooms(:watercooler))
fill_in "Message", with: "Hello, world!"
click_button "Send"
assert_text "Hello, world!"
end
test "editing own message" do
sign_in users(:david)
visit room_path(rooms(:watercooler))
within "#message_#{messages(:greeting).id}" do
click_on "Edit"
end
fill_in "Message", with: "Updated message"
click_button "Save"
assert_text "Updated message"
end
test "drag and drop card to new column" do
sign_in users(:david)
visit board_path(boards(:main))
card = find("#card_#{cards(:one).id}")
target = find("#column_#{columns(:done).id}")
card.drag_to target
assert_selector "#column_#{columns(:done).id} #card_#{cards(:one).id}"
end
end
```
## Advanced Patterns
### Time Testing
Use `travel_to` for deterministic time-dependent assertions:
```ruby
test "card expires after 30 days" do
card = cards(:one)
travel_to 31.days.from_now do
assert card.expired?
end
end
```
### External API Testing with VCR
Record and replay HTTP interactions:
```ruby
test "fetches user data from API" do
VCR.use_cassette("user_api") do
user_data = ExternalApi.fetch_user(123)
assert_equal "John", user_data[:name]
end
end
```
### Background Job Testing
Assert job enqueueing and email delivery:
```ruby
test "closing card enqueues notification job" do
card = cards(:one)
assert_enqueued_with(job: NotifyWatchersJob, args: [card]) do
card.close
end
end
test "welcome email is sent on signup" do
assert_emails 1 do
Identity.create!(email: "new@example.com")
end
end
```
### Testing Turbo Streams
```ruby
test "message creation broadcasts to room" do
room = rooms(:watercooler)
assert_turbo_stream_broadcasts [room, :messages] do
room.messages.create!(body: "Test", creator: users(:david))
end
end
```
## Testing Principles
### 1. Test Observable Behavior
Focus on what the code does, not how it does it:
```ruby
# ❌ Testing implementation
test "calls notify method on each watcher" do
card.expects(:notify).times(3)
card.close
end
# ✅ Testing behavior
test "watchers receive notifications when card closes" do
assert_difference -> { Notification.count }, 3 do
card.close
end
end
```
### 2. Don't Mock Everything
```ruby
# ❌ Over-mocked test
test "sending message" do
room = mock("room")
user = mock("user")
message = mock("message")
room.expects(:messages).returns(stub(create!: message))
message.expects(:broadcast_create)
MessagesController.new.create
end
# ✅ Test the real thing
test "sending message" do
sign_in users(:david)
post room_messages_url(rooms(:watercooler)),
params: { message: { body: "Hello" } }
assert_response :success
assert Message.exists?(body: "Hello")
end
```
### 3. Tests Ship with Features
Same commit, not TDD-first but together. Neither before (strict TDD) nor after (deferred testing).
### 4. Security Fixes Always Include Regression Tests
Every security fix must include a test that would have caught the vulnerability.
### 5. Integration Tests Validate Complete Workflows
Don't just test individual pieces - test that they work together.
## File Organization
```
test/
├── controllers/ # Integration tests for controllers
├── fixtures/ # YAML fixtures for all models
├── helpers/ # Helper method tests
├── integration/ # API integration tests
├── jobs/ # Background job tests
├── mailers/ # Mailer tests
├── models/ # Unit tests for models
├── system/ # Browser-based system tests
└── test_helper.rb # Test configuration
```
## Test Helper Setup
```ruby
# test/test_helper.rb
ENV["RAILS_ENV"] ||= "test"
require_relative "../config/environment"
require "rails/test_help"
class ActiveSupport::TestCase
fixtures :all
parallelize(workers: :number_of_processors)
end
class ActionDispatch::IntegrationTest
include SignInHelper
end
class ApplicationSystemTestCase < ActionDispatch::SystemTestCase
driven_by :selenium, using: :headless_chrome
end
```
## Sign In Helper
```ruby
# test/support/sign_in_helper.rb
module SignInHelper
def sign_in(user)
session = user.identity.sessions.create!
cookies.signed[:session_id] = session.id
end
end
```

737
plugins/compound-engineering/skills/dspy-ruby/SKILL.md
View File

@@ -1,737 +0,0 @@
---
name: dspy-ruby
description: Build type-safe LLM applications with DSPy.rb — Ruby's programmatic prompt framework with signatures, modules, agents, and optimization. Use when implementing predictable AI features, creating LLM signatures and modules, configuring language model providers, building agent systems with tools, optimizing prompts, or testing LLM-powered functionality in Ruby applications.
---
# DSPy.rb
> Build LLM apps like you build software. Type-safe, modular, testable.
DSPy.rb brings software engineering best practices to LLM development. Instead of tweaking prompts, define what you want with Ruby types and let DSPy handle the rest.
## Overview
DSPy.rb is a Ruby framework for building language model applications with programmatic prompts. It provides:
- **Type-safe signatures** — Define inputs/outputs with Sorbet types
- **Modular components** — Compose and reuse LLM logic
- **Automatic optimization** — Use data to improve prompts, not guesswork
- **Production-ready** — Built-in observability, testing, and error handling
## Core Concepts
### 1. Signatures
Define interfaces between your app and LLMs using Ruby types:
```ruby
class EmailClassifier < DSPy::Signature
description "Classify customer support emails by category and priority"
class Priority < T::Enum
enums do
Low = new('low')
Medium = new('medium')
High = new('high')
Urgent = new('urgent')
end
end
input do
const :email_content, String
const :sender, String
end
output do
const :category, String
const :priority, Priority # Type-safe enum with defined values
const :confidence, Float
end
end
```
### 2. Modules
Build complex workflows from simple building blocks:
- **Predict** — Basic LLM calls with signatures
- **ChainOfThought** — Step-by-step reasoning
- **ReAct** — Tool-using agents
- **CodeAct** — Dynamic code generation agents (install the `dspy-code_act` gem)
### 3. Tools & Toolsets
Create type-safe tools for agents with comprehensive Sorbet support:
```ruby
# Enum-based tool with automatic type conversion
class CalculatorTool < DSPy::Tools::Base
tool_name 'calculator'
tool_description 'Performs arithmetic operations with type-safe enum inputs'
class Operation < T::Enum
enums do
Add = new('add')
Subtract = new('subtract')
Multiply = new('multiply')
Divide = new('divide')
end
end
sig { params(operation: Operation, num1: Float, num2: Float).returns(T.any(Float, String)) }
def call(operation:, num1:, num2:)
case operation
when Operation::Add then num1 + num2
when Operation::Subtract then num1 - num2
when Operation::Multiply then num1 * num2
when Operation::Divide
return "Error: Division by zero" if num2 == 0
num1 / num2
end
end
end
# Multi-tool toolset with rich types
class DataToolset < DSPy::Tools::Toolset
toolset_name "data_processing"
class Format < T::Enum
enums do
JSON = new('json')
CSV = new('csv')
XML = new('xml')
end
end
tool :convert, description: "Convert data between formats"
tool :validate, description: "Validate data structure"
sig { params(data: String, from: Format, to: Format).returns(String) }
def convert(data:, from:, to:)
"Converted from #{from.serialize} to #{to.serialize}"
end
sig { params(data: String, format: Format).returns(T::Hash[String, T.any(String, Integer, T::Boolean)]) }
def validate(data:, format:)
{ valid: true, format: format.serialize, row_count: 42, message: "Data validation passed" }
end
end
```
### 4. Type System & Discriminators
DSPy.rb uses sophisticated type discrimination for complex data structures:
- **Automatic `_type` field injection** — DSPy adds discriminator fields to structs for type safety
- **Union type support** — `T.any()` types automatically disambiguated by `_type`
- **Reserved field name** — Avoid defining your own `_type` fields in structs
- **Recursive filtering** — `_type` fields filtered during deserialization at all nesting levels
### 5. Optimization
Improve accuracy with real data:
- **MIPROv2** — Advanced multi-prompt optimization with bootstrap sampling and Bayesian optimization
- **GEPA** — Genetic-Pareto Reflective Prompt Evolution with feedback maps, experiment tracking, and telemetry
- **Evaluation** — Comprehensive framework with built-in and custom metrics, error handling, and batch processing
## Quick Start
```ruby
# Install
gem 'dspy'
# Configure
DSPy.configure do |c|
c.lm = DSPy::LM.new('openai/gpt-4o-mini', api_key: ENV['OPENAI_API_KEY'])
end
# Define a task
class SentimentAnalysis < DSPy::Signature
description "Analyze sentiment of text"
input do
const :text, String
end
output do
const :sentiment, String # positive, negative, neutral
const :score, Float # 0.0 to 1.0
end
end
# Use it
analyzer = DSPy::Predict.new(SentimentAnalysis)
result = analyzer.call(text: "This product is amazing!")
puts result.sentiment # => "positive"
puts result.score # => 0.92
```
## Provider Adapter Gems
Two strategies for connecting to LLM providers:
### Per-provider adapters (direct SDK access)
```ruby
# Gemfile
gem 'dspy'
gem 'dspy-openai' # OpenAI, OpenRouter, Ollama
gem 'dspy-anthropic' # Claude
gem 'dspy-gemini' # Gemini
```
Each adapter gem pulls in the official SDK (`openai`, `anthropic`, `gemini-ai`).
### Unified adapter via RubyLLM (recommended for multi-provider)
```ruby
# Gemfile
gem 'dspy'
gem 'dspy-ruby_llm' # Routes to any provider via ruby_llm
gem 'ruby_llm'
```
RubyLLM handles provider routing based on the model name. Use the `ruby_llm/` prefix:
```ruby
DSPy.configure do |c|
c.lm = DSPy::LM.new('ruby_llm/gemini-2.5-flash', structured_outputs: true)
# c.lm = DSPy::LM.new('ruby_llm/claude-sonnet-4-20250514', structured_outputs: true)
# c.lm = DSPy::LM.new('ruby_llm/gpt-4o-mini', structured_outputs: true)
end
```
## Events System
DSPy.rb ships with a structured event bus for observing runtime behavior.
### Module-Scoped Subscriptions (preferred for agents)
```ruby
class MyAgent < DSPy::Module
subscribe 'lm.tokens', :track_tokens, scope: :descendants
def track_tokens(_event, attrs)
@total_tokens += attrs.fetch(:total_tokens, 0)
end
end
```
### Global Subscriptions (for observability/integrations)
```ruby
subscription_id = DSPy.events.subscribe('score.create') do |event, attrs|
Langfuse.export_score(attrs)
end
# Wildcards supported
DSPy.events.subscribe('llm.*') { |name, attrs| puts "[#{name}] tokens=#{attrs[:total_tokens]}" }
```
Event names use dot-separated namespaces (`llm.generate`, `react.iteration_complete`). Every event includes module metadata (`module_path`, `module_leaf`, `module_scope.ancestry_token`) for filtering.
## Lifecycle Callbacks
Rails-style lifecycle hooks ship with every `DSPy::Module`:
- **`before`** — Runs ahead of `forward` for setup (metrics, context loading)
- **`around`** — Wraps `forward`, calls `yield`, and lets you pair setup/teardown logic
- **`after`** — Fires after `forward` returns for cleanup or persistence
```ruby
class InstrumentedModule < DSPy::Module
before :setup_metrics
around :manage_context
after :log_metrics
def forward(question:)
@predictor.call(question: question)
end
private
def setup_metrics
@start_time = Time.now
end
def manage_context
load_context
result = yield
save_context
result
end
def log_metrics
duration = Time.now - @start_time
Rails.logger.info "Prediction completed in #{duration}s"
end
end
```
Execution order: before → around (before yield) → forward → around (after yield) → after. Callbacks are inherited from parent classes and execute in registration order.
## Fiber-Local LM Context
Override the language model temporarily using fiber-local storage:
```ruby
fast_model = DSPy::LM.new("openai/gpt-4o-mini", api_key: ENV['OPENAI_API_KEY'])
DSPy.with_lm(fast_model) do
result = classifier.call(text: "test") # Uses fast_model inside this block
end
# Back to global LM outside the block
```
**LM resolution hierarchy**: Instance-level LM → Fiber-local LM (`DSPy.with_lm`) → Global LM (`DSPy.configure`).
Use `configure_predictor` for fine-grained control over agent internals:
```ruby
agent = DSPy::ReAct.new(MySignature, tools: tools)
agent.configure { |c| c.lm = default_model }
agent.configure_predictor('thought_generator') { |c| c.lm = powerful_model }
```
## Evaluation Framework
Systematically test LLM application performance with `DSPy::Evals`:
```ruby
metric = DSPy::Metrics.exact_match(field: :answer, case_sensitive: false)
evaluator = DSPy::Evals.new(predictor, metric: metric)
result = evaluator.evaluate(test_examples, display_table: true)
puts "Pass Rate: #{(result.pass_rate * 100).round(1)}%"
```
Built-in metrics: `exact_match`, `contains`, `numeric_difference`, `composite_and`. Custom metrics return `true`/`false` or a `DSPy::Prediction` with `score:` and `feedback:` fields.
Use `DSPy::Example` for typed test data and `export_scores: true` to push results to Langfuse.
## GEPA Optimization
GEPA (Genetic-Pareto Reflective Prompt Evolution) uses reflection-driven instruction rewrites:
```ruby
gem 'dspy-gepa'
teleprompter = DSPy::Teleprompt::GEPA.new(
metric: metric,
reflection_lm: DSPy::ReflectionLM.new('openai/gpt-4o-mini', api_key: ENV['OPENAI_API_KEY']),
feedback_map: feedback_map,
config: { max_metric_calls: 600, minibatch_size: 6 }
)
result = teleprompter.compile(program, trainset: train, valset: val)
optimized_program = result.optimized_program
```
The metric must return `DSPy::Prediction.new(score:, feedback:)` so the reflection model can reason about failures. Use `feedback_map` to target individual predictors in composite modules.
## Typed Context Pattern
Replace opaque string context blobs with `T::Struct` inputs. Each field gets its own `description:` annotation in the JSON schema the LLM sees:
```ruby
class NavigationContext < T::Struct
const :workflow_hint, T.nilable(String),
description: "Current workflow phase guidance for the agent"
const :action_log, T::Array[String], default: [],
description: "Compact one-line-per-action history of research steps taken"
const :iterations_remaining, Integer,
description: "Budget remaining. Each tool call costs 1 iteration."
end
class ToolSelectionSignature < DSPy::Signature
input do
const :query, String
const :context, NavigationContext # Structured, not an opaque string
end
output do
const :tool_name, String
const :tool_args, String, description: "JSON-encoded arguments"
end
end
```
Benefits: type safety at compile time, per-field descriptions in the LLM schema, easy to test as value objects, extensible by adding `const` declarations.
## Schema Formats (BAML / TOON)
Control how DSPy describes signature structure to the LLM:
- **JSON Schema** (default) — Standard format, works with `structured_outputs: true`
- **BAML** (`schema_format: :baml`) — 84% token reduction for Enhanced Prompting mode. Requires `sorbet-baml` gem.
- **TOON** (`schema_format: :toon, data_format: :toon`) — Table-oriented format for both schemas and data. Enhanced Prompting mode only.
BAML and TOON apply only when `structured_outputs: false`. With `structured_outputs: true`, the provider receives JSON Schema directly.
## Storage System
Persist and reload optimized programs with `DSPy::Storage::ProgramStorage`:
```ruby
storage = DSPy::Storage::ProgramStorage.new(storage_path: "./dspy_storage")
storage.save_program(result.optimized_program, result, metadata: { optimizer: 'MIPROv2' })
```
Supports checkpoint management, optimization history tracking, and import/export between environments.
## Rails Integration
### Directory Structure
Organize DSPy components using Rails conventions:
```
app/
entities/ # T::Struct types shared across signatures
signatures/ # DSPy::Signature definitions
tools/ # DSPy::Tools::Base implementations
concerns/ # Shared tool behaviors (error handling, etc.)
modules/ # DSPy::Module orchestrators
services/ # Plain Ruby services that compose DSPy modules
config/
initializers/
dspy.rb # DSPy + provider configuration
feature_flags.rb # Model selection per role
spec/
signatures/ # Schema validation tests
tools/ # Tool unit tests
modules/ # Integration tests with VCR
vcr_cassettes/ # Recorded HTTP interactions
```
### Initializer
```ruby
# config/initializers/dspy.rb
Rails.application.config.after_initialize do
next if Rails.env.test? && ENV["DSPY_ENABLE_IN_TEST"].blank?
RubyLLM.configure do |config|
config.gemini_api_key = ENV["GEMINI_API_KEY"] if ENV["GEMINI_API_KEY"].present?
config.anthropic_api_key = ENV["ANTHROPIC_API_KEY"] if ENV["ANTHROPIC_API_KEY"].present?
config.openai_api_key = ENV["OPENAI_API_KEY"] if ENV["OPENAI_API_KEY"].present?
end
model = ENV.fetch("DSPY_MODEL", "ruby_llm/gemini-2.5-flash")
DSPy.configure do |config|
config.lm = DSPy::LM.new(model, structured_outputs: true)
config.logger = Rails.logger
end
# Langfuse observability (optional)
if ENV["LANGFUSE_PUBLIC_KEY"].present? && ENV["LANGFUSE_SECRET_KEY"].present?
DSPy::Observability.configure!
end
end
```
### Feature-Flagged Model Selection
Use different models for different roles (fast/cheap for classification, powerful for synthesis):
```ruby
# config/initializers/feature_flags.rb
module FeatureFlags
SELECTOR_MODEL = ENV.fetch("DSPY_SELECTOR_MODEL", "ruby_llm/gemini-2.5-flash-lite")
SYNTHESIZER_MODEL = ENV.fetch("DSPY_SYNTHESIZER_MODEL", "ruby_llm/gemini-2.5-flash")
end
```
Then override per-tool or per-predictor:
```ruby
class ClassifyTool < DSPy::Tools::Base
def call(query:)
predictor = DSPy::Predict.new(ClassifyQuery)
predictor.configure { |c| c.lm = DSPy::LM.new(FeatureFlags::SELECTOR_MODEL, structured_outputs: true) }
predictor.call(query: query)
end
end
```
## Schema-Driven Signatures
**Prefer typed schemas over string descriptions.** Let the type system communicate structure to the LLM rather than prose in the signature description.
### Entities as Shared Types
Define reusable `T::Struct` and `T::Enum` types in `app/entities/` and reference them across signatures:
```ruby
# app/entities/search_strategy.rb
class SearchStrategy < T::Enum
enums do
SingleSearch = new("single_search")
DateDecomposition = new("date_decomposition")
end
end
# app/entities/scored_item.rb
class ScoredItem < T::Struct
const :id, String
const :score, Float, description: "Relevance score 0.0-1.0"
const :verdict, String, description: "relevant, maybe, or irrelevant"
const :reason, String, default: ""
end
```
### Schema vs Description: When to Use Each
**Use schemas (T::Struct/T::Enum)** for:
- Multi-field outputs with specific types
- Enums with defined values the LLM must pick from
- Nested structures, arrays of typed objects
- Outputs consumed by code (not displayed to users)
**Use string descriptions** for:
- Simple single-field outputs where the type is `String`
- Natural language generation (summaries, answers)
- Fields where constraint guidance helps (e.g., `description: "YYYY-MM-DD format"`)
**Rule of thumb**: If you'd write a `case` statement on the output, it should be a `T::Enum`. If you'd call `.each` on it, it should be `T::Array[SomeStruct]`.
## Tool Patterns
### Tools That Wrap Predictions
A common pattern: tools encapsulate a DSPy prediction, adding error handling, model selection, and serialization:
```ruby
class RerankTool < DSPy::Tools::Base
tool_name "rerank"
tool_description "Score and rank search results by relevance"
MAX_ITEMS = 200
MIN_ITEMS_FOR_LLM = 5
sig { params(query: String, items: T::Array[T::Hash[Symbol, T.untyped]]).returns(T::Hash[Symbol, T.untyped]) }
def call(query:, items: [])
return { scored_items: items, reranked: false } if items.size < MIN_ITEMS_FOR_LLM
capped_items = items.first(MAX_ITEMS)
predictor = DSPy::Predict.new(RerankSignature)
predictor.configure { |c| c.lm = DSPy::LM.new(FeatureFlags::SYNTHESIZER_MODEL, structured_outputs: true) }
result = predictor.call(query: query, items: capped_items)
{ scored_items: result.scored_items, reranked: true }
rescue => e
Rails.logger.warn "[RerankTool] LLM rerank failed: #{e.message}"
{ error: "Rerank failed: #{e.message}", scored_items: items, reranked: false }
end
end
```
**Key patterns:**
- Short-circuit LLM calls when unnecessary (small data, trivial cases)
- Cap input size to prevent token overflow
- Per-tool model selection via `configure`
- Graceful error handling with fallback data
### Error Handling Concern
```ruby
module ErrorHandling
extend ActiveSupport::Concern
private
def safe_predict(signature_class, **inputs)
predictor = DSPy::Predict.new(signature_class)
yield predictor if block_given?
predictor.call(**inputs)
rescue Faraday::Error, Net::HTTPError => e
Rails.logger.error "[#{self.class.name}] API error: #{e.message}"
nil
rescue JSON::ParserError => e
Rails.logger.error "[#{self.class.name}] Invalid LLM output: #{e.message}"
nil
end
end
```
## Observability
### Tracing with DSPy::Context
Wrap operations in spans for Langfuse/OpenTelemetry visibility:
```ruby
result = DSPy::Context.with_span(
operation: "tool_selector.select",
"dspy.module" => "ToolSelector",
"tool_selector.tools" => tool_names.join(",")
) do
@predictor.call(query: query, context: context, available_tools: schemas)
end
```
### Setup for Langfuse
```ruby
# Gemfile
gem 'dspy-o11y'
gem 'dspy-o11y-langfuse'
# .env
LANGFUSE_PUBLIC_KEY=pk-...
LANGFUSE_SECRET_KEY=sk-...
DSPY_TELEMETRY_BATCH_SIZE=5
```
Every `DSPy::Predict`, `DSPy::ReAct`, and tool call is automatically traced when observability is configured.
### Score Reporting
Report evaluation scores to Langfuse:
```ruby
DSPy.score(name: "relevance", value: 0.85, trace_id: current_trace_id)
```
## Testing
### VCR Setup for Rails
```ruby
VCR.configure do |config|
config.cassette_library_dir = "spec/vcr_cassettes"
config.hook_into :webmock
config.configure_rspec_metadata!
config.filter_sensitive_data('<GEMINI_API_KEY>') { ENV['GEMINI_API_KEY'] }
config.filter_sensitive_data('<OPENAI_API_KEY>') { ENV['OPENAI_API_KEY'] }
end
```
### Signature Schema Tests
Test that signatures produce valid schemas without calling any LLM:
```ruby
RSpec.describe ClassifyResearchQuery do
it "has required input fields" do
schema = described_class.input_json_schema
expect(schema[:required]).to include("query")
end
it "has typed output fields" do
schema = described_class.output_json_schema
expect(schema[:properties]).to have_key(:search_strategy)
end
end
```
### Tool Tests with Mocked Predictions
```ruby
RSpec.describe RerankTool do
let(:tool) { described_class.new }
it "skips LLM for small result sets" do
expect(DSPy::Predict).not_to receive(:new)
result = tool.call(query: "test", items: [{ id: "1" }])
expect(result[:reranked]).to be false
end
it "calls LLM for large result sets", :vcr do
items = 10.times.map { |i| { id: i.to_s, title: "Item #{i}" } }
result = tool.call(query: "relevant items", items: items)
expect(result[:reranked]).to be true
end
end
```
## Resources
- [core-concepts.md](./references/core-concepts.md) — Signatures, modules, predictors, type system deep-dive
- [toolsets.md](./references/toolsets.md) — Tools::Base, Tools::Toolset DSL, type safety, testing
- [providers.md](./references/providers.md) — Provider adapters, RubyLLM, fiber-local LM context, compatibility matrix
- [optimization.md](./references/optimization.md) — MIPROv2, GEPA, evaluation framework, storage system
- [observability.md](./references/observability.md) — Event system, dspy-o11y gems, Langfuse, score reporting
- [signature-template.rb](./assets/signature-template.rb) — Signature scaffold with T::Enum, Date/Time, defaults, union types
- [module-template.rb](./assets/module-template.rb) — Module scaffold with .call(), lifecycle callbacks, fiber-local LM
- [config-template.rb](./assets/config-template.rb) — Rails initializer with RubyLLM, observability, feature flags
## Key URLs
- Homepage: https://oss.vicente.services/dspy.rb/
- GitHub: https://github.com/vicentereig/dspy.rb
- Documentation: https://oss.vicente.services/dspy.rb/getting-started/
## Guidelines for Claude
When helping users with DSPy.rb:
1. **Schema over prose** — Define output structure with `T::Struct` and `T::Enum` types, not string descriptions
2. **Entities in `app/entities/`** — Extract shared types so signatures stay thin
3. **Per-tool model selection** — Use `predictor.configure { |c| c.lm = ... }` to pick the right model per task
4. **Short-circuit LLM calls** — Skip the LLM for trivial cases (small data, cached results)
5. **Cap input sizes** — Prevent token overflow by limiting array sizes before sending to LLM
6. **Test schemas without LLM** — Validate `input_json_schema` and `output_json_schema` in unit tests
7. **VCR for integration tests** — Record real HTTP interactions, never mock LLM responses by hand
8. **Trace with spans** — Wrap tool calls in `DSPy::Context.with_span` for observability
9. **Graceful degradation** — Always rescue LLM errors and return fallback data
### Signature Best Practices
**Keep description concise** — The signature `description` should state the goal, not the field details:
```ruby
# Good — concise goal
class ParseOutline < DSPy::Signature
description 'Extract block-level structure from HTML as a flat list of skeleton sections.'
input do
const :html, String, description: 'Raw HTML to parse'
end
output do
const :sections, T::Array[Section], description: 'Block elements: headings, paragraphs, code blocks, lists'
end
end
```
**Use defaults over nilable arrays** — For OpenAI structured outputs compatibility:
```ruby
# Good — works with OpenAI structured outputs
class ASTNode < T::Struct
const :children, T::Array[ASTNode], default: []
end
```
### Recursive Types with `$defs`
DSPy.rb supports recursive types in structured outputs using JSON Schema `$defs`:
```ruby
class TreeNode < T::Struct
const :value, String
const :children, T::Array[TreeNode], default: [] # Self-reference
end
```
The schema generator automatically creates `#/$defs/TreeNode` references for recursive types, compatible with OpenAI and Gemini structured outputs.
### Field Descriptions for T::Struct
DSPy.rb extends T::Struct to support field-level `description:` kwargs that flow to JSON Schema:
```ruby
class ASTNode < T::Struct
const :node_type, NodeType, description: 'The type of node (heading, paragraph, etc.)'
const :text, String, default: "", description: 'Text content of the node'
const :level, Integer, default: 0 # No description — field is self-explanatory
const :children, T::Array[ASTNode], default: []
end
```
**When to use field descriptions**: complex field semantics, enum-like strings, constrained values, nested structs with ambiguous names. **When to skip**: self-explanatory fields like `name`, `id`, `url`, or boolean flags.
## Version
Current: 0.34.3

187
plugins/compound-engineering/skills/dspy-ruby/assets/config-template.rb
View File

@@ -1,187 +0,0 @@
# frozen_string_literal: true
# =============================================================================
# DSPy.rb Configuration Template — v0.34.3 API
#
# Rails initializer patterns for DSPy.rb with RubyLLM, observability,
# and feature-flagged model selection.
#
# Key patterns:
# - Use after_initialize for Rails setup
# - Use dspy-ruby_llm for multi-provider routing
# - Use structured_outputs: true for reliable parsing
# - Use dspy-o11y + dspy-o11y-langfuse for observability
# - Use ENV-based feature flags for model selection
# =============================================================================
# =============================================================================
# Gemfile Dependencies
# =============================================================================
#
# # Core
# gem 'dspy'
#
# # Provider adapter (choose one strategy):
#
# # Strategy A: Unified adapter via RubyLLM (recommended)
# gem 'dspy-ruby_llm'
# gem 'ruby_llm'
#
# # Strategy B: Per-provider adapters (direct SDK access)
# gem 'dspy-openai' # OpenAI, OpenRouter, Ollama
# gem 'dspy-anthropic' # Claude
# gem 'dspy-gemini' # Gemini
#
# # Observability (optional)
# gem 'dspy-o11y'
# gem 'dspy-o11y-langfuse'
#
# # Optimization (optional)
# gem 'dspy-miprov2' # MIPROv2 optimizer
# gem 'dspy-gepa' # GEPA optimizer
#
# # Schema formats (optional)
# gem 'sorbet-baml' # BAML schema format (84% token reduction)
# =============================================================================
# Rails Initializer — config/initializers/dspy.rb
# =============================================================================
Rails.application.config.after_initialize do
# Skip in test unless explicitly enabled
next if Rails.env.test? && ENV["DSPY_ENABLE_IN_TEST"].blank?
# Configure RubyLLM provider credentials
RubyLLM.configure do |config|
config.gemini_api_key = ENV["GEMINI_API_KEY"] if ENV["GEMINI_API_KEY"].present?
config.anthropic_api_key = ENV["ANTHROPIC_API_KEY"] if ENV["ANTHROPIC_API_KEY"].present?
config.openai_api_key = ENV["OPENAI_API_KEY"] if ENV["OPENAI_API_KEY"].present?
end
# Configure DSPy with unified RubyLLM adapter
model = ENV.fetch("DSPY_MODEL", "ruby_llm/gemini-2.5-flash")
DSPy.configure do |config|
config.lm = DSPy::LM.new(model, structured_outputs: true)
config.logger = Rails.logger
end
# Enable Langfuse observability (optional)
if ENV["LANGFUSE_PUBLIC_KEY"].present? && ENV["LANGFUSE_SECRET_KEY"].present?
DSPy::Observability.configure!
end
end
# =============================================================================
# Feature Flags — config/initializers/feature_flags.rb
# =============================================================================
# Use different models for different roles:
# - Fast/cheap for classification, routing, simple tasks
# - Powerful for synthesis, reasoning, complex analysis
module FeatureFlags
SELECTOR_MODEL = ENV.fetch("DSPY_SELECTOR_MODEL", "ruby_llm/gemini-2.5-flash-lite")
SYNTHESIZER_MODEL = ENV.fetch("DSPY_SYNTHESIZER_MODEL", "ruby_llm/gemini-2.5-flash")
REASONING_MODEL = ENV.fetch("DSPY_REASONING_MODEL", "ruby_llm/claude-sonnet-4-20250514")
end
# Usage in tools/modules:
#
# class ClassifyTool < DSPy::Tools::Base
# def call(query:)
# predictor = DSPy::Predict.new(ClassifySignature)
# predictor.configure { |c| c.lm = DSPy::LM.new(FeatureFlags::SELECTOR_MODEL, structured_outputs: true) }
# predictor.call(query: query)
# end
# end
# =============================================================================
# Environment Variables — .env
# =============================================================================
#
# # Provider API keys (set the ones you need)
# GEMINI_API_KEY=...
# ANTHROPIC_API_KEY=...
# OPENAI_API_KEY=...
#
# # DSPy model configuration
# DSPY_MODEL=ruby_llm/gemini-2.5-flash
# DSPY_SELECTOR_MODEL=ruby_llm/gemini-2.5-flash-lite
# DSPY_SYNTHESIZER_MODEL=ruby_llm/gemini-2.5-flash
# DSPY_REASONING_MODEL=ruby_llm/claude-sonnet-4-20250514
#
# # Langfuse observability (optional)
# LANGFUSE_PUBLIC_KEY=pk-...
# LANGFUSE_SECRET_KEY=sk-...
# DSPY_TELEMETRY_BATCH_SIZE=5
#
# # Test environment
# DSPY_ENABLE_IN_TEST=1 # Set to enable DSPy in test env
# =============================================================================
# Per-Provider Configuration (without RubyLLM)
# =============================================================================
# OpenAI (dspy-openai gem)
# DSPy.configure do |c|
# c.lm = DSPy::LM.new('openai/gpt-4o-mini', api_key: ENV['OPENAI_API_KEY'])
# end
# Anthropic (dspy-anthropic gem)
# DSPy.configure do |c|
# c.lm = DSPy::LM.new('anthropic/claude-sonnet-4-20250514', api_key: ENV['ANTHROPIC_API_KEY'])
# end
# Gemini (dspy-gemini gem)
# DSPy.configure do |c|
# c.lm = DSPy::LM.new('gemini/gemini-2.5-flash', api_key: ENV['GEMINI_API_KEY'])
# end
# Ollama (dspy-openai gem, local models)
# DSPy.configure do |c|
# c.lm = DSPy::LM.new('ollama/llama3.2', base_url: 'http://localhost:11434')
# end
# OpenRouter (dspy-openai gem, 200+ models)
# DSPy.configure do |c|
# c.lm = DSPy::LM.new('openrouter/anthropic/claude-3.5-sonnet',
# api_key: ENV['OPENROUTER_API_KEY'],
# base_url: 'https://openrouter.ai/api/v1')
# end
# =============================================================================
# VCR Test Configuration — spec/support/dspy.rb
# =============================================================================
# VCR.configure do |config|
# config.cassette_library_dir = "spec/vcr_cassettes"
# config.hook_into :webmock
# config.configure_rspec_metadata!
# config.filter_sensitive_data('<GEMINI_API_KEY>') { ENV['GEMINI_API_KEY'] }
# config.filter_sensitive_data('<OPENAI_API_KEY>') { ENV['OPENAI_API_KEY'] }
# config.filter_sensitive_data('<ANTHROPIC_API_KEY>') { ENV['ANTHROPIC_API_KEY'] }
# end
# =============================================================================
# Schema Format Configuration (optional)
# =============================================================================
# BAML schema format — 84% token reduction for Enhanced Prompting mode
# DSPy.configure do |c|
# c.lm = DSPy::LM.new('openai/gpt-4o-mini',
# api_key: ENV['OPENAI_API_KEY'],
# schema_format: :baml # Requires sorbet-baml gem
# )
# end
# TOON schema + data format — table-oriented format
# DSPy.configure do |c|
# c.lm = DSPy::LM.new('openai/gpt-4o-mini',
# api_key: ENV['OPENAI_API_KEY'],
# schema_format: :toon, # How DSPy describes the signature
# data_format: :toon # How inputs/outputs are rendered in prompts
# )
# end
#
# Note: BAML and TOON apply only when structured_outputs: false.
# With structured_outputs: true, the provider receives JSON Schema directly.

300
plugins/compound-engineering/skills/dspy-ruby/assets/module-template.rb
View File

@@ -1,300 +0,0 @@
# frozen_string_literal: true
# =============================================================================
# DSPy.rb Module Template — v0.34.3 API
#
# Modules orchestrate predictors, tools, and business logic.
#
# Key patterns:
# - Use .call() to invoke (not .forward())
# - Access results with result.field (not result[:field])
# - Use DSPy::Tools::Base for tools (not DSPy::Tool)
# - Use lifecycle callbacks (before/around/after) for cross-cutting concerns
# - Use DSPy.with_lm for temporary model overrides
# - Use configure_predictor for fine-grained agent control
# =============================================================================
# --- Basic Module ---
class BasicClassifier < DSPy::Module
def initialize
super
@predictor = DSPy::Predict.new(ClassificationSignature)
end
def forward(text:)
@predictor.call(text: text)
end
end
# Usage:
# classifier = BasicClassifier.new
# result = classifier.call(text: "This is a test")
# result.category # => "technical"
# result.confidence # => 0.95
# --- Module with Chain of Thought ---
class ReasoningClassifier < DSPy::Module
def initialize
super
@predictor = DSPy::ChainOfThought.new(ClassificationSignature)
end
def forward(text:)
result = @predictor.call(text: text)
# ChainOfThought adds result.reasoning automatically
result
end
end
# --- Module with Lifecycle Callbacks ---
class InstrumentedModule < DSPy::Module
before :setup_metrics
around :manage_context
after :log_completion
def initialize
super
@predictor = DSPy::Predict.new(AnalysisSignature)
@start_time = nil
end
def forward(query:)
@predictor.call(query: query)
end
private
# Runs before forward
def setup_metrics
@start_time = Time.now
Rails.logger.info "Starting prediction"
end
# Wraps forward — must call yield
def manage_context
load_user_context
result = yield
save_updated_context(result)
result
end
# Runs after forward completes
def log_completion
duration = Time.now - @start_time
Rails.logger.info "Prediction completed in #{duration}s"
end
def load_user_context = nil
def save_updated_context(_result) = nil
end
# Execution order: before → around (before yield) → forward → around (after yield) → after
# Callbacks are inherited from parent classes and execute in registration order.
# --- Module with Tools ---
class SearchTool < DSPy::Tools::Base
tool_name "search"
tool_description "Search for information by query"
sig { params(query: String, max_results: Integer).returns(T::Array[T::Hash[Symbol, String]]) }
def call(query:, max_results: 5)
# Implementation here
[{ title: "Result 1", url: "https://example.com" }]
end
end
class FinishTool < DSPy::Tools::Base
tool_name "finish"
tool_description "Submit the final answer"
sig { params(answer: String).returns(String) }
def call(answer:)
answer
end
end
class ResearchAgent < DSPy::Module
def initialize
super
tools = [SearchTool.new, FinishTool.new]
@agent = DSPy::ReAct.new(
ResearchSignature,
tools: tools,
max_iterations: 5
)
end
def forward(question:)
@agent.call(question: question)
end
end
# --- Module with Per-Task Model Selection ---
class SmartRouter < DSPy::Module
def initialize
super
@classifier = DSPy::Predict.new(RouteSignature)
@analyzer = DSPy::ChainOfThought.new(AnalysisSignature)
end
def forward(text:)
# Use fast model for classification
DSPy.with_lm(fast_model) do
route = @classifier.call(text: text)
if route.requires_deep_analysis
# Switch to powerful model for analysis
DSPy.with_lm(powerful_model) do
@analyzer.call(text: text)
end
else
route
end
end
end
private
def fast_model
@fast_model ||= DSPy::LM.new(
ENV.fetch("DSPY_SELECTOR_MODEL", "ruby_llm/gemini-2.5-flash-lite"),
structured_outputs: true
)
end
def powerful_model
@powerful_model ||= DSPy::LM.new(
ENV.fetch("DSPY_SYNTHESIZER_MODEL", "ruby_llm/gemini-2.5-flash"),
structured_outputs: true
)
end
end
# --- Module with configure_predictor ---
class ConfiguredAgent < DSPy::Module
def initialize
super
tools = [SearchTool.new, FinishTool.new]
@agent = DSPy::ReAct.new(ResearchSignature, tools: tools)
# Set default model for all internal predictors
@agent.configure { |c| c.lm = DSPy::LM.new('ruby_llm/gemini-2.5-flash', structured_outputs: true) }
# Override specific predictor with a more capable model
@agent.configure_predictor('thought_generator') do |c|
c.lm = DSPy::LM.new('ruby_llm/claude-sonnet-4-20250514', structured_outputs: true)
end
end
def forward(question:)
@agent.call(question: question)
end
end
# Available internal predictors by agent type:
# DSPy::ReAct → thought_generator, observation_processor
# DSPy::CodeAct → code_generator, observation_processor
# DSPy::DeepSearch → seed_predictor, search_predictor, reader_predictor, reason_predictor
# --- Module with Event Subscriptions ---
class TokenTrackingModule < DSPy::Module
subscribe 'lm.tokens', :track_tokens, scope: :descendants
def initialize
super
@predictor = DSPy::Predict.new(AnalysisSignature)
@total_tokens = 0
end
def forward(query:)
@predictor.call(query: query)
end
def track_tokens(_event, attrs)
@total_tokens += attrs.fetch(:total_tokens, 0)
end
def token_usage
@total_tokens
end
end
# Module-scoped subscriptions automatically scope to the module instance and descendants.
# Use scope: :self_only to restrict delivery to the module itself (ignoring children).
# --- Tool That Wraps a Prediction ---
class RerankTool < DSPy::Tools::Base
tool_name "rerank"
tool_description "Score and rank search results by relevance"
MAX_ITEMS = 200
MIN_ITEMS_FOR_LLM = 5
sig { params(query: String, items: T::Array[T::Hash[Symbol, T.untyped]]).returns(T::Hash[Symbol, T.untyped]) }
def call(query:, items: [])
# Short-circuit: skip LLM for small sets
return { scored_items: items, reranked: false } if items.size < MIN_ITEMS_FOR_LLM
# Cap to prevent token overflow
capped_items = items.first(MAX_ITEMS)
predictor = DSPy::Predict.new(RerankSignature)
predictor.configure { |c| c.lm = DSPy::LM.new("ruby_llm/gemini-2.5-flash", structured_outputs: true) }
result = predictor.call(query: query, items: capped_items)
{ scored_items: result.scored_items, reranked: true }
rescue => e
Rails.logger.warn "[RerankTool] LLM rerank failed: #{e.message}"
{ error: "Rerank failed: #{e.message}", scored_items: items, reranked: false }
end
end
# Key patterns for tools wrapping predictions:
# - Short-circuit LLM calls when unnecessary (small data, trivial cases)
# - Cap input size to prevent token overflow
# - Per-tool model selection via configure
# - Graceful error handling with fallback data
# --- Multi-Step Pipeline ---
class AnalysisPipeline < DSPy::Module
def initialize
super
@classifier = DSPy::Predict.new(ClassifySignature)
@analyzer = DSPy::ChainOfThought.new(AnalyzeSignature)
@summarizer = DSPy::Predict.new(SummarizeSignature)
end
def forward(text:)
classification = @classifier.call(text: text)
analysis = @analyzer.call(text: text, category: classification.category)
@summarizer.call(analysis: analysis.reasoning, category: classification.category)
end
end
# --- Observability with Spans ---
class TracedModule < DSPy::Module
def initialize
super
@predictor = DSPy::Predict.new(AnalysisSignature)
end
def forward(query:)
DSPy::Context.with_span(
operation: "traced_module.analyze",
"dspy.module" => self.class.name,
"query.length" => query.length.to_s
) do
@predictor.call(query: query)
end
end
end

221
plugins/compound-engineering/skills/dspy-ruby/assets/signature-template.rb
View File

@@ -1,221 +0,0 @@
# frozen_string_literal: true
# =============================================================================
# DSPy.rb Signature Template — v0.34.3 API
#
# Signatures define the interface between your application and LLMs.
# They specify inputs, outputs, and task descriptions using Sorbet types.
#
# Key patterns:
# - Use T::Enum classes for controlled outputs (not inline T.enum([...]))
# - Use description: kwarg on fields to guide the LLM
# - Use default values for optional fields
# - Use Date/DateTime/Time for temporal data (auto-converted)
# - Access results with result.field (not result[:field])
# - Invoke with predictor.call() (not predictor.forward())
# =============================================================================
# --- Basic Signature ---
class SentimentAnalysis < DSPy::Signature
description "Analyze sentiment of text"
class Sentiment < T::Enum
enums do
Positive = new('positive')
Negative = new('negative')
Neutral = new('neutral')
end
end
input do
const :text, String
end
output do
const :sentiment, Sentiment
const :score, Float, description: "Confidence score from 0.0 to 1.0"
end
end
# Usage:
# predictor = DSPy::Predict.new(SentimentAnalysis)
# result = predictor.call(text: "This product is amazing!")
# result.sentiment # => Sentiment::Positive
# result.score # => 0.92
# --- Signature with Date/Time Types ---
class EventScheduler < DSPy::Signature
description "Schedule events based on requirements"
input do
const :event_name, String
const :start_date, Date # ISO 8601: YYYY-MM-DD
const :end_date, T.nilable(Date) # Optional date
const :preferred_time, DateTime # ISO 8601 with timezone
const :deadline, Time # Stored as UTC
end
output do
const :scheduled_date, Date # LLM returns ISO string, auto-converted
const :event_datetime, DateTime # Preserves timezone
const :created_at, Time # Converted to UTC
end
end
# Date/Time format handling:
# Date → ISO 8601 (YYYY-MM-DD)
# DateTime → ISO 8601 with timezone (YYYY-MM-DDTHH:MM:SS+00:00)
# Time → ISO 8601, automatically converted to UTC
# --- Signature with Default Values ---
class SmartSearch < DSPy::Signature
description "Search with intelligent defaults"
input do
const :query, String
const :max_results, Integer, default: 10
const :language, String, default: "English"
const :include_metadata, T::Boolean, default: false
end
output do
const :results, T::Array[String]
const :total_found, Integer
const :search_time_ms, Float, default: 0.0 # Fallback if LLM omits
const :cached, T::Boolean, default: false
end
end
# Input defaults reduce boilerplate:
# search = DSPy::Predict.new(SmartSearch)
# result = search.call(query: "Ruby programming")
# # max_results=10, language="English", include_metadata=false are applied
# --- Signature with Nested Structs and Field Descriptions ---
class EntityExtraction < DSPy::Signature
description "Extract named entities from text"
class EntityType < T::Enum
enums do
Person = new('person')
Organization = new('organization')
Location = new('location')
DateEntity = new('date')
end
end
class Entity < T::Struct
const :name, String, description: "The entity text as it appears in the source"
const :type, EntityType
const :confidence, Float, description: "Extraction confidence from 0.0 to 1.0"
const :start_offset, Integer, default: 0
end
input do
const :text, String
const :entity_types, T::Array[EntityType], default: [],
description: "Filter to these entity types; empty means all types"
end
output do
const :entities, T::Array[Entity]
const :total_found, Integer
end
end
# --- Signature with Union Types ---
class FlexibleClassification < DSPy::Signature
description "Classify input with flexible result type"
class Category < T::Enum
enums do
Technical = new('technical')
Business = new('business')
Personal = new('personal')
end
end
input do
const :text, String
end
output do
const :category, Category
const :result, T.any(Float, String),
description: "Numeric score or text explanation depending on classification"
const :confidence, Float
end
end
# --- Signature with Recursive Types ---
class DocumentParser < DSPy::Signature
description "Parse document into tree structure"
class NodeType < T::Enum
enums do
Heading = new('heading')
Paragraph = new('paragraph')
List = new('list')
CodeBlock = new('code_block')
end
end
class TreeNode < T::Struct
const :node_type, NodeType, description: "The type of document element"
const :text, String, default: "", description: "Text content of the node"
const :level, Integer, default: 0
const :children, T::Array[TreeNode], default: [] # Self-reference → $defs in JSON Schema
end
input do
const :html, String, description: "Raw HTML to parse"
end
output do
const :root, TreeNode
const :word_count, Integer
end
end
# The schema generator creates #/$defs/TreeNode references for recursive types,
# compatible with OpenAI and Gemini structured outputs.
# Use `default: []` instead of `T.nilable(T::Array[...])` for OpenAI compatibility.
# --- Vision Signature ---
class ImageAnalysis < DSPy::Signature
description "Analyze an image and answer questions about its content"
input do
const :image, DSPy::Image, description: "The image to analyze"
const :question, String, description: "Question about the image content"
end
output do
const :answer, String
const :confidence, Float, description: "Confidence in the answer (0.0-1.0)"
end
end
# Vision usage:
# predictor = DSPy::Predict.new(ImageAnalysis)
# result = predictor.call(
# image: DSPy::Image.from_file("path/to/image.jpg"),
# question: "What objects are visible?"
# )
# result.answer # => "The image shows..."
# --- Accessing Schemas Programmatically ---
#
# SentimentAnalysis.input_json_schema # => { type: "object", properties: { ... } }
# SentimentAnalysis.output_json_schema # => { type: "object", properties: { ... } }
#
# # Field descriptions propagate to JSON Schema
# Entity.field_descriptions[:name] # => "The entity text as it appears in the source"
# Entity.field_descriptions[:confidence] # => "Extraction confidence from 0.0 to 1.0"

674
plugins/compound-engineering/skills/dspy-ruby/references/core-concepts.md
View File

@@ -1,674 +0,0 @@
# DSPy.rb Core Concepts
## Signatures
Signatures define the interface between application code and language models. They specify inputs, outputs, and a task description using Sorbet types for compile-time and runtime type safety.
### Structure
```ruby
class ClassifyEmail < DSPy::Signature
description "Classify customer support emails by urgency and category"
input do
const :subject, String
const :body, String
end
output do
const :category, String
const :urgency, String
end
end
```
### Supported Types
| Type | JSON Schema | Notes |
|------|-------------|-------|
| `String` | `string` | Required string |
| `Integer` | `integer` | Whole numbers |
| `Float` | `number` | Decimal numbers |
| `T::Boolean` | `boolean` | true/false |
| `T::Array[X]` | `array` | Typed arrays |
| `T::Hash[K, V]` | `object` | Typed key-value maps |
| `T.nilable(X)` | nullable | Optional fields |
| `Date` | `string` (ISO 8601) | Auto-converted |
| `DateTime` | `string` (ISO 8601) | Preserves timezone |
| `Time` | `string` (ISO 8601) | Converted to UTC |
### Date and Time Types
Date, DateTime, and Time fields serialize to ISO 8601 strings and auto-convert back to Ruby objects on output.
```ruby
class EventScheduler < DSPy::Signature
description "Schedule events based on requirements"
input do
const :start_date, Date # ISO 8601: YYYY-MM-DD
const :preferred_time, DateTime # ISO 8601 with timezone
const :deadline, Time # Converted to UTC
const :end_date, T.nilable(Date) # Optional date
end
output do
const :scheduled_date, Date # String from LLM, auto-converted to Date
const :event_datetime, DateTime # Preserves timezone info
const :created_at, Time # Converted to UTC
end
end
predictor = DSPy::Predict.new(EventScheduler)
result = predictor.call(
start_date: "2024-01-15",
preferred_time: "2024-01-15T10:30:45Z",
deadline: Time.now,
end_date: nil
)
result.scheduled_date.class # => Date
result.event_datetime.class # => DateTime
```
Timezone conventions follow ActiveRecord: Time objects convert to UTC, DateTime objects preserve timezone, Date objects are timezone-agnostic.
### Enums with T::Enum
Define constrained output values using `T::Enum` classes. Do not use inline `T.enum([...])` syntax.
```ruby
class SentimentAnalysis < DSPy::Signature
description "Analyze sentiment of text"
class Sentiment < T::Enum
enums do
Positive = new('positive')
Negative = new('negative')
Neutral = new('neutral')
end
end
input do
const :text, String
end
output do
const :sentiment, Sentiment
const :confidence, Float
end
end
predictor = DSPy::Predict.new(SentimentAnalysis)
result = predictor.call(text: "This product is amazing!")
result.sentiment # => #<Sentiment::Positive>
result.sentiment.serialize # => "positive"
result.confidence # => 0.92
```
Enum matching is case-insensitive. The LLM returning `"POSITIVE"` matches `new('positive')`.
### Default Values
Default values work on both inputs and outputs. Input defaults reduce caller boilerplate. Output defaults provide fallbacks when the LLM omits optional fields.
```ruby
class SmartSearch < DSPy::Signature
description "Search with intelligent defaults"
input do
const :query, String
const :max_results, Integer, default: 10
const :language, String, default: "English"
end
output do
const :results, T::Array[String]
const :total_found, Integer
const :cached, T::Boolean, default: false
end
end
search = DSPy::Predict.new(SmartSearch)
result = search.call(query: "Ruby programming")
# max_results defaults to 10, language defaults to "English"
# If LLM omits `cached`, it defaults to false
```
### Field Descriptions
Add `description:` to any field to guide the LLM on expected content. These descriptions appear in the generated JSON schema sent to the model.
```ruby
class ASTNode < T::Struct
const :node_type, String, description: "The type of AST node (heading, paragraph, code_block)"
const :text, String, default: "", description: "Text content of the node"
const :level, Integer, default: 0, description: "Heading level 1-6, only for heading nodes"
const :children, T::Array[ASTNode], default: []
end
ASTNode.field_descriptions[:node_type] # => "The type of AST node ..."
ASTNode.field_descriptions[:children] # => nil (no description set)
```
Field descriptions also work inside signature `input` and `output` blocks:
```ruby
class ExtractEntities < DSPy::Signature
description "Extract named entities from text"
input do
const :text, String, description: "Raw text to analyze"
const :language, String, default: "en", description: "ISO 639-1 language code"
end
output do
const :entities, T::Array[String], description: "List of extracted entity names"
const :count, Integer, description: "Total number of unique entities found"
end
end
```
### Schema Formats
DSPy.rb supports three schema formats for communicating type structure to LLMs.
#### JSON Schema (default)
Verbose but universally supported. Access via `YourSignature.output_json_schema`.
#### BAML Schema
Compact format that reduces schema tokens by 80-85%. Requires the `sorbet-baml` gem.
```ruby
DSPy.configure do |c|
c.lm = DSPy::LM.new('openai/gpt-4o-mini',
api_key: ENV['OPENAI_API_KEY'],
schema_format: :baml
)
end
```
BAML applies only in Enhanced Prompting mode (`structured_outputs: false`). When `structured_outputs: true`, the provider receives JSON Schema directly.
#### TOON Schema + Data Format
Table-oriented text format that shrinks both schema definitions and prompt values.
```ruby
DSPy.configure do |c|
c.lm = DSPy::LM.new('openai/gpt-4o-mini',
api_key: ENV['OPENAI_API_KEY'],
schema_format: :toon,
data_format: :toon
)
end
```
`schema_format: :toon` replaces the schema block in the system prompt. `data_format: :toon` renders input values and output templates inside `toon` fences. Only works with Enhanced Prompting mode. The `sorbet-toon` gem is included automatically as a dependency.
### Recursive Types
Structs that reference themselves produce `$defs` entries in the generated JSON schema, using `$ref` pointers to avoid infinite recursion.
```ruby
class ASTNode < T::Struct
const :node_type, String
const :text, String, default: ""
const :children, T::Array[ASTNode], default: []
end
```
The schema generator detects the self-reference in `T::Array[ASTNode]` and emits:
```json
{
"$defs": {
"ASTNode": { "type": "object", "properties": { ... } }
},
"properties": {
"children": {
"type": "array",
"items": { "$ref": "#/$defs/ASTNode" }
}
}
}
```
Access the schema with accumulated definitions via `YourSignature.output_json_schema_with_defs`.
### Union Types with T.any()
Specify fields that accept multiple types:
```ruby
output do
const :result, T.any(Float, String)
end
```
For struct unions, DSPy.rb automatically adds a `_type` discriminator field to each struct's JSON schema. The LLM returns `_type` in its response, and DSPy converts the hash to the correct struct instance.
```ruby
class CreateTask < T::Struct
const :title, String
const :priority, String
end
class DeleteTask < T::Struct
const :task_id, String
const :reason, T.nilable(String)
end
class TaskRouter < DSPy::Signature
description "Route user request to the appropriate task action"
input do
const :request, String
end
output do
const :action, T.any(CreateTask, DeleteTask)
end
end
result = DSPy::Predict.new(TaskRouter).call(request: "Create a task for Q4 review")
result.action.class # => CreateTask
result.action.title # => "Q4 Review"
```
Pattern matching works on the result:
```ruby
case result.action
when CreateTask then puts "Creating: #{result.action.title}"
when DeleteTask then puts "Deleting: #{result.action.task_id}"
end
```
Union types also work inside arrays for heterogeneous collections:
```ruby
output do
const :events, T::Array[T.any(LoginEvent, PurchaseEvent)]
end
```
Limit unions to 2-4 types for reliable LLM comprehension. Use clear struct names since they become the `_type` discriminator values.
---
## Modules
Modules are composable building blocks that wrap predictors. Define a `forward` method; invoke the module with `.call()`.
### Basic Structure
```ruby
class SentimentAnalyzer < DSPy::Module
def initialize
super
@predictor = DSPy::Predict.new(SentimentSignature)
end
def forward(text:)
@predictor.call(text: text)
end
end
analyzer = SentimentAnalyzer.new
result = analyzer.call(text: "I love this product!")
result.sentiment # => "positive"
result.confidence # => 0.9
```
**API rules:**
- Invoke modules and predictors with `.call()`, not `.forward()`.
- Access result fields with `result.field`, not `result[:field]`.
### Module Composition
Combine multiple modules through explicit method calls in `forward`:
```ruby
class DocumentProcessor < DSPy::Module
def initialize
super
@classifier = DocumentClassifier.new
@summarizer = DocumentSummarizer.new
end
def forward(document:)
classification = @classifier.call(content: document)
summary = @summarizer.call(content: document)
{
document_type: classification.document_type,
summary: summary.summary
}
end
end
```
### Lifecycle Callbacks
Modules support `before`, `after`, and `around` callbacks on `forward`. Declare them as class-level macros referencing private methods.
#### Execution order
1. `before` callbacks (in registration order)
2. `around` callbacks (before `yield`)
3. `forward` method
4. `around` callbacks (after `yield`)
5. `after` callbacks (in registration order)
```ruby
class InstrumentedModule < DSPy::Module
before :setup_metrics
after :log_metrics
around :manage_context
def initialize
super
@predictor = DSPy::Predict.new(MySignature)
@metrics = {}
end
def forward(question:)
@predictor.call(question: question)
end
private
def setup_metrics
@metrics[:start_time] = Time.now
end
def manage_context
load_context
result = yield
save_context
result
end
def log_metrics
@metrics[:duration] = Time.now - @metrics[:start_time]
end
end
```
Multiple callbacks of the same type execute in registration order. Callbacks inherit from parent classes; parent callbacks run first.
#### Around callbacks
Around callbacks must call `yield` to execute the wrapped method and return the result:
```ruby
def with_retry
retries = 0
begin
yield
rescue StandardError => e
retries += 1
retry if retries < 3
raise e
end
end
```
### Instruction Update Contract
Teleprompters (GEPA, MIPROv2) require modules to expose immutable update hooks. Include `DSPy::Mixins::InstructionUpdatable` and implement `with_instruction` and `with_examples`, each returning a new instance:
```ruby
class SentimentPredictor < DSPy::Module
include DSPy::Mixins::InstructionUpdatable
def initialize
super
@predictor = DSPy::Predict.new(SentimentSignature)
end
def with_instruction(instruction)
clone = self.class.new
clone.instance_variable_set(:@predictor, @predictor.with_instruction(instruction))
clone
end
def with_examples(examples)
clone = self.class.new
clone.instance_variable_set(:@predictor, @predictor.with_examples(examples))
clone
end
end
```
If a module omits these hooks, teleprompters raise `DSPy::InstructionUpdateError` instead of silently mutating state.
---
## Predictors
Predictors are execution engines that take a signature and produce structured results from a language model. DSPy.rb provides four predictor types.
### Predict
Direct LLM call with typed input/output. Fastest option, lowest token usage.
```ruby
classifier = DSPy::Predict.new(ClassifyText)
result = classifier.call(text: "Technical document about APIs")
result.sentiment # => #<Sentiment::Positive>
result.topics # => ["APIs", "technical"]
result.confidence # => 0.92
```
### ChainOfThought
Adds a `reasoning` field to the output automatically. The model generates step-by-step reasoning before the final answer. Do not define a `:reasoning` field in the signature output when using ChainOfThought.
```ruby
class SolveMathProblem < DSPy::Signature
description "Solve mathematical word problems step by step"
input do
const :problem, String
end
output do
const :answer, String
# :reasoning is added automatically by ChainOfThought
end
end
solver = DSPy::ChainOfThought.new(SolveMathProblem)
result = solver.call(problem: "Sarah has 15 apples. She gives 7 away and buys 12 more.")
result.reasoning # => "Step by step: 15 - 7 = 8, then 8 + 12 = 20"
result.answer # => "20 apples"
```
Use ChainOfThought for complex analysis, multi-step reasoning, or when explainability matters.
### ReAct
Reasoning + Action agent that uses tools in an iterative loop. Define tools by subclassing `DSPy::Tools::Base`. Group related tools with `DSPy::Tools::Toolset`.
```ruby
class WeatherTool < DSPy::Tools::Base
extend T::Sig
tool_name "weather"
tool_description "Get weather information for a location"
sig { params(location: String).returns(String) }
def call(location:)
{ location: location, temperature: 72, condition: "sunny" }.to_json
end
end
class TravelSignature < DSPy::Signature
description "Help users plan travel"
input do
const :destination, String
end
output do
const :recommendations, String
end
end
agent = DSPy::ReAct.new(
TravelSignature,
tools: [WeatherTool.new],
max_iterations: 5
)
result = agent.call(destination: "Tokyo, Japan")
result.recommendations # => "Visit Senso-ji Temple early morning..."
result.history # => Array of reasoning steps, actions, observations
result.iterations # => 3
result.tools_used # => ["weather"]
```
Use toolsets to expose multiple tool methods from a single class:
```ruby
text_tools = DSPy::Tools::TextProcessingToolset.to_tools
agent = DSPy::ReAct.new(MySignature, tools: text_tools)
```
### CodeAct
Think-Code-Observe agent that synthesizes and executes Ruby code. Ships as a separate gem.
```ruby
# Gemfile
gem 'dspy-code_act', '~> 0.29'
```
```ruby
programmer = DSPy::CodeAct.new(ProgrammingSignature, max_iterations: 10)
result = programmer.call(task: "Calculate the factorial of 20")
```
### Predictor Comparison
| Predictor | Speed | Token Usage | Best For |
|-----------|-------|-------------|----------|
| Predict | Fastest | Low | Classification, extraction |
| ChainOfThought | Moderate | Medium-High | Complex reasoning, analysis |
| ReAct | Slower | High | Multi-step tasks with tools |
| CodeAct | Slowest | Very High | Dynamic programming, calculations |
### Concurrent Predictions
Process multiple independent predictions simultaneously using `Async::Barrier`:
```ruby
require 'async'
require 'async/barrier'
analyzer = DSPy::Predict.new(ContentAnalyzer)
documents = ["Text one", "Text two", "Text three"]
Async do
barrier = Async::Barrier.new
tasks = documents.map do |doc|
barrier.async { analyzer.call(content: doc) }
end
barrier.wait
predictions = tasks.map(&:wait)
predictions.each { |p| puts p.sentiment }
end
```
Add `gem 'async', '~> 2.29'` to the Gemfile. Handle errors within each `barrier.async` block to prevent one failure from cancelling others:
```ruby
barrier.async do
begin
analyzer.call(content: doc)
rescue StandardError => e
nil
end
end
```
### Few-Shot Examples and Instruction Tuning
```ruby
classifier = DSPy::Predict.new(SentimentAnalysis)
examples = [
DSPy::FewShotExample.new(
input: { text: "Love it!" },
output: { sentiment: "positive", confidence: 0.95 }
)
]
optimized = classifier.with_examples(examples)
tuned = classifier.with_instruction("Be precise and confident.")
```
---
## Type System
### Automatic Type Conversion
DSPy.rb v0.9.0+ automatically converts LLM JSON responses to typed Ruby objects:
- **Enums**: String values become `T::Enum` instances (case-insensitive)
- **Structs**: Nested hashes become `T::Struct` objects
- **Arrays**: Elements convert recursively
- **Defaults**: Missing fields use declared defaults
### Discriminators for Union Types
When a field uses `T.any()` with struct types, DSPy adds a `_type` field to each struct's schema. On deserialization, `_type` selects the correct struct class:
```json
{
"action": {
"_type": "CreateTask",
"title": "Review Q4 Report"
}
}
```
DSPy matches `"CreateTask"` against the union members and instantiates the correct struct. No manual discriminator field is needed.
### Recursive Types
Structs referencing themselves are supported. The schema generator tracks visited types and produces `$ref` pointers under `$defs`:
```ruby
class TreeNode < T::Struct
const :label, String
const :children, T::Array[TreeNode], default: []
end
```
The generated schema uses `"$ref": "#/$defs/TreeNode"` for the children array items, preventing infinite schema expansion.
### Nesting Depth
- 1-2 levels: reliable across all providers.
- 3-4 levels: works but increases schema complexity.
- 5+ levels: may trigger OpenAI depth validation warnings and reduce LLM accuracy. Flatten deeply nested structures or split into multiple signatures.
### Tips
- Prefer `T::Array[X], default: []` over `T.nilable(T::Array[X])` -- the nilable form causes schema issues with OpenAI structured outputs.
- Use clear struct names for union types since they become `_type` discriminator values.
- Limit union types to 2-4 members for reliable model comprehension.
- Check schema compatibility with `DSPy::OpenAI::LM::SchemaConverter.validate_compatibility(schema)`.

366
plugins/compound-engineering/skills/dspy-ruby/references/observability.md
View File

@@ -1,366 +0,0 @@
# DSPy.rb Observability
DSPy.rb provides an event-driven observability system built on OpenTelemetry. The system replaces monkey-patching with structured event emission, pluggable listeners, automatic span creation, and non-blocking Langfuse export.
## Event System
### Emitting Events
Emit structured events with `DSPy.event`:
```ruby
DSPy.event('lm.tokens', {
'gen_ai.system' => 'openai',
'gen_ai.request.model' => 'gpt-4',
input_tokens: 150,
output_tokens: 50,
total_tokens: 200
})
```
Event names are **strings** with dot-separated namespaces (e.g., `'llm.generate'`, `'react.iteration_complete'`, `'chain_of_thought.reasoning_complete'`). Do not use symbols for event names.
Attributes must be JSON-serializable. DSPy automatically merges context (trace ID, module stack) and creates OpenTelemetry spans.
### Global Subscriptions
Subscribe to events across the entire application with `DSPy.events.subscribe`:
```ruby
# Exact event name
subscription_id = DSPy.events.subscribe('lm.tokens') do |event_name, attrs|
puts "Tokens used: #{attrs[:total_tokens]}"
end
# Wildcard pattern -- matches llm.generate, llm.stream, etc.
DSPy.events.subscribe('llm.*') do |event_name, attrs|
track_llm_usage(attrs)
end
# Catch-all wildcard
DSPy.events.subscribe('*') do |event_name, attrs|
log_everything(event_name, attrs)
end
```
Use global subscriptions for cross-cutting concerns: observability exporters (Langfuse, Datadog), centralized logging, metrics collection.
### Module-Scoped Subscriptions
Declare listeners inside a `DSPy::Module` subclass. Subscriptions automatically scope to the module instance and its descendants:
```ruby
class ResearchReport < DSPy::Module
subscribe 'lm.tokens', :track_tokens, scope: :descendants
def initialize
super
@outliner = DSPy::Predict.new(OutlineSignature)
@writer = DSPy::Predict.new(SectionWriterSignature)
@token_count = 0
end
def forward(question:)
outline = @outliner.call(question: question)
outline.sections.map do |title|
draft = @writer.call(question: question, section_title: title)
{ title: title, body: draft.paragraph }
end
end
def track_tokens(_event, attrs)
@token_count += attrs.fetch(:total_tokens, 0)
end
end
```
The `scope:` parameter accepts:
- `:descendants` (default) -- receives events from the module **and** every nested module invoked inside it.
- `DSPy::Module::SubcriptionScope::SelfOnly` -- restricts delivery to events emitted by the module instance itself; ignores descendants.
Inspect active subscriptions with `registered_module_subscriptions`. Tear down with `unsubscribe_module_events`.
### Unsubscribe and Cleanup
Remove a global listener by subscription ID:
```ruby
id = DSPy.events.subscribe('llm.*') { |name, attrs| }
DSPy.events.unsubscribe(id)
```
Build tracker classes that manage their own subscription lifecycle:
```ruby
class TokenBudgetTracker
def initialize(budget:)
@budget = budget
@usage = 0
@subscriptions = []
@subscriptions << DSPy.events.subscribe('lm.tokens') do |_event, attrs|
@usage += attrs.fetch(:total_tokens, 0)
warn("Budget hit") if @usage >= @budget
end
end
def unsubscribe
@subscriptions.each { |id| DSPy.events.unsubscribe(id) }
@subscriptions.clear
end
end
```
### Clearing Listeners in Tests
Call `DSPy.events.clear_listeners` in `before`/`after` blocks to prevent cross-contamination between test cases:
```ruby
RSpec.configure do |config|
config.after(:each) { DSPy.events.clear_listeners }
end
```
## dspy-o11y Gems
Three gems compose the observability stack:
| Gem | Purpose |
|---|---|
| `dspy` | Core event bus (`DSPy.event`, `DSPy.events`) -- always available |
| `dspy-o11y` | OpenTelemetry spans, `AsyncSpanProcessor`, `DSPy::Context.with_span` helpers |
| `dspy-o11y-langfuse` | Langfuse adapter -- configures OTLP exporter targeting Langfuse endpoints |
### Installation
```ruby
# Gemfile
gem 'dspy'
gem 'dspy-o11y' # core spans + helpers
gem 'dspy-o11y-langfuse' # Langfuse/OpenTelemetry adapter (optional)
```
If the optional gems are absent, DSPy falls back to logging-only mode with no errors.
## Langfuse Integration
### Environment Variables
```bash
# Required
export LANGFUSE_PUBLIC_KEY=pk-lf-your-public-key
export LANGFUSE_SECRET_KEY=sk-lf-your-secret-key
# Optional (defaults to https://cloud.langfuse.com)
export LANGFUSE_HOST=https://us.cloud.langfuse.com
# Tuning (optional)
export DSPY_TELEMETRY_BATCH_SIZE=100 # spans per export batch (default 100)
export DSPY_TELEMETRY_QUEUE_SIZE=1000 # max queued spans (default 1000)
export DSPY_TELEMETRY_EXPORT_INTERVAL=60 # seconds between timed exports (default 60)
export DSPY_TELEMETRY_SHUTDOWN_TIMEOUT=10 # seconds to drain on shutdown (default 10)
```
### Automatic Configuration
Call `DSPy::Observability.configure!` once at boot (it is already called automatically when `require 'dspy'` runs and Langfuse env vars are present):
```ruby
require 'dspy'
# If LANGFUSE_PUBLIC_KEY and LANGFUSE_SECRET_KEY are set,
# DSPy::Observability.configure! runs automatically and:
# 1. Configures the OpenTelemetry SDK with an OTLP exporter
# 2. Creates dual output: structured logs AND OpenTelemetry spans
# 3. Exports spans to Langfuse using proper authentication
# 4. Falls back gracefully if gems are missing
```
Verify status with `DSPy::Observability.enabled?`.
### Automatic Tracing
With observability enabled, every `DSPy::Module#forward` call, LM request, and tool invocation creates properly nested spans. Langfuse receives hierarchical traces:
```
Trace: abc-123-def
+-- ChainOfThought.forward [2000ms] (observation type: chain)
+-- llm.generate [1000ms] (observation type: generation)
Model: gpt-4-0613
Tokens: 100 in / 50 out / 150 total
```
DSPy maps module classes to Langfuse observation types automatically via `DSPy::ObservationType.for_module_class`:
| Module | Observation Type |
|---|---|
| `DSPy::LM` (raw chat) | `generation` |
| `DSPy::ChainOfThought` | `chain` |
| `DSPy::ReAct` | `agent` |
| Tool invocations | `tool` |
| Memory/retrieval | `retriever` |
| Embedding engines | `embedding` |
| Evaluation modules | `evaluator` |
| Generic operations | `span` |
## Score Reporting
### DSPy.score API
Report evaluation scores with `DSPy.score`:
```ruby
# Numeric (default)
DSPy.score('accuracy', 0.95)
# With comment
DSPy.score('relevance', 0.87, comment: 'High semantic similarity')
# Boolean
DSPy.score('is_valid', 1, data_type: DSPy::Scores::DataType::Boolean)
# Categorical
DSPy.score('sentiment', 'positive', data_type: DSPy::Scores::DataType::Categorical)
# Explicit trace binding
DSPy.score('accuracy', 0.95, trace_id: 'custom-trace-id')
```
Available data types: `DSPy::Scores::DataType::Numeric`, `::Boolean`, `::Categorical`.
### score.create Events
Every `DSPy.score` call emits a `'score.create'` event. Subscribe to react:
```ruby
DSPy.events.subscribe('score.create') do |event_name, attrs|
puts "#{attrs[:score_name]} = #{attrs[:score_value]}"
# Also available: attrs[:score_id], attrs[:score_data_type],
# attrs[:score_comment], attrs[:trace_id], attrs[:observation_id],
# attrs[:timestamp]
end
```
### Async Langfuse Export with DSPy::Scores::Exporter
Configure the exporter to send scores to Langfuse in the background:
```ruby
exporter = DSPy::Scores::Exporter.configure(
public_key: ENV['LANGFUSE_PUBLIC_KEY'],
secret_key: ENV['LANGFUSE_SECRET_KEY'],
host: 'https://cloud.langfuse.com'
)
# Scores are now exported automatically via a background Thread::Queue
DSPy.score('accuracy', 0.95)
# Shut down gracefully (waits up to 5 seconds by default)
exporter.shutdown
```
The exporter subscribes to `'score.create'` events internally, queues them for async processing, and retries with exponential backoff on failure.
### Automatic Export with DSPy::Evals
Pass `export_scores: true` to `DSPy::Evals` to export per-example scores and an aggregate batch score automatically:
```ruby
evaluator = DSPy::Evals.new(
program,
metric: my_metric,
export_scores: true,
score_name: 'qa_accuracy'
)
result = evaluator.evaluate(test_examples)
```
## DSPy::Context.with_span
Create manual spans for custom operations. Requires `dspy-o11y`.
```ruby
DSPy::Context.with_span(operation: 'custom.retrieval', 'retrieval.source' => 'pinecone') do |span|
results = pinecone_client.query(embedding)
span&.set_attribute('retrieval.count', results.size) if span
results
end
```
Pass semantic attributes as keyword arguments alongside `operation:`. The block receives an OpenTelemetry span object (or `nil` when observability is disabled). The span automatically nests under the current parent span and records `duration.ms`, `langfuse.observation.startTime`, and `langfuse.observation.endTime`.
Assign a Langfuse observation type to custom spans:
```ruby
DSPy::Context.with_span(
operation: 'evaluate.batch',
**DSPy::ObservationType::Evaluator.langfuse_attributes,
'batch.size' => examples.length
) do |span|
run_evaluation(examples)
end
```
Scores reported inside a `with_span` block automatically inherit the current trace context.
## Module Stack Metadata
When `DSPy::Module#forward` runs, the context layer maintains a module stack. Every event includes:
```ruby
{
module_path: [
{ id: "root_uuid", class: "DeepSearch", label: nil },
{ id: "planner_uuid", class: "DSPy::Predict", label: "planner" }
],
module_root: { id: "root_uuid", class: "DeepSearch", label: nil },
module_leaf: { id: "planner_uuid", class: "DSPy::Predict", label: "planner" },
module_scope: {
ancestry_token: "root_uuid>planner_uuid",
depth: 2
}
}
```
| Key | Meaning |
|---|---|
| `module_path` | Ordered array of `{id, class, label}` entries from root to leaf |
| `module_root` | The outermost module in the current call chain |
| `module_leaf` | The innermost (currently executing) module |
| `module_scope.ancestry_token` | Stable string of joined UUIDs representing the nesting path |
| `module_scope.depth` | Integer depth of the current module in the stack |
Labels are set via `module_scope_label=` on a module instance or derived automatically from named predictors. Use this metadata to power Langfuse filters, scoped metrics, or custom event routing.
## Dedicated Export Worker
The `DSPy::Observability::AsyncSpanProcessor` (from `dspy-o11y`) keeps telemetry export off the hot path:
- Runs on a `Concurrent::SingleThreadExecutor` -- LLM workflows never compete with OTLP networking.
- Buffers finished spans in a `Thread::Queue` (max size configurable via `DSPY_TELEMETRY_QUEUE_SIZE`).
- Drains spans in batches of `DSPY_TELEMETRY_BATCH_SIZE` (default 100). When the queue reaches batch size, an immediate async export fires.
- A background timer thread triggers periodic export every `DSPY_TELEMETRY_EXPORT_INTERVAL` seconds (default 60).
- Applies exponential backoff (`0.1 * 2^attempt` seconds) on export failures, up to `DEFAULT_MAX_RETRIES` (3).
- On shutdown, flushes all remaining spans within `DSPY_TELEMETRY_SHUTDOWN_TIMEOUT` seconds, then terminates the executor.
- Drops the oldest span when the queue is full, logging `'observability.span_dropped'`.
No application code interacts with the processor directly. Configure it entirely through environment variables.
## Built-in Events Reference
| Event Name | Emitted By | Key Attributes |
|---|---|---|
| `lm.tokens` | `DSPy::LM` | `gen_ai.system`, `gen_ai.request.model`, `input_tokens`, `output_tokens`, `total_tokens` |
| `chain_of_thought.reasoning_complete` | `DSPy::ChainOfThought` | `dspy.signature`, `cot.reasoning_steps`, `cot.reasoning_length`, `cot.has_reasoning` |
| `react.iteration_complete` | `DSPy::ReAct` | `iteration`, `thought`, `action`, `observation` |
| `codeact.iteration_complete` | `dspy-code_act` gem | `iteration`, `code_executed`, `execution_result` |
| `optimization.trial_complete` | Teleprompters (MIPROv2) | `trial_number`, `score` |
| `score.create` | `DSPy.score` | `score_name`, `score_value`, `score_data_type`, `trace_id` |
| `span.start` | `DSPy::Context.with_span` | `trace_id`, `span_id`, `parent_span_id`, `operation` |
## Best Practices
- Use dot-separated string names for events. Follow OpenTelemetry `gen_ai.*` conventions for LLM attributes.
- Always call `unsubscribe` (or `unsubscribe_module_events` for scoped subscriptions) when a tracker is no longer needed to prevent memory leaks.
- Call `DSPy.events.clear_listeners` in test teardown to avoid cross-contamination.
- Wrap risky listener logic in a rescue block. The event system isolates listener failures, but explicit rescue prevents silent swallowing of domain errors.
- Prefer module-scoped `subscribe` for agent internals. Reserve global `DSPy.events.subscribe` for infrastructure-level concerns.

603
plugins/compound-engineering/skills/dspy-ruby/references/optimization.md
View File

@@ -1,603 +0,0 @@
# DSPy.rb Optimization
## MIPROv2
MIPROv2 (Multi-prompt Instruction Proposal with Retrieval Optimization) is the primary instruction tuner in DSPy.rb. It proposes new instructions and few-shot demonstrations per predictor, evaluates them on mini-batches, and retains candidates that improve the metric. It ships as a separate gem to keep the Gaussian Process dependency tree out of apps that do not need it.
### Installation
```ruby
# Gemfile
gem "dspy"
gem "dspy-miprov2"
```
Bundler auto-requires `dspy/miprov2`. No additional `require` statement is needed.
### AutoMode presets
Use `DSPy::Teleprompt::MIPROv2::AutoMode` for preconfigured optimizers:
```ruby
light = DSPy::Teleprompt::MIPROv2::AutoMode.light(metric: metric) # 6 trials, greedy
medium = DSPy::Teleprompt::MIPROv2::AutoMode.medium(metric: metric) # 12 trials, adaptive
heavy = DSPy::Teleprompt::MIPROv2::AutoMode.heavy(metric: metric) # 18 trials, Bayesian
```
| Preset | Trials | Strategy | Use case |
|----------|--------|------------|-----------------------------------------------------|
| `light` | 6 | `:greedy` | Quick wins on small datasets or during prototyping. |
| `medium` | 12 | `:adaptive`| Balanced exploration vs. runtime for most pilots. |
| `heavy` | 18 | `:bayesian`| Highest accuracy targets or multi-stage programs. |
### Manual configuration with dry-configurable
`DSPy::Teleprompt::MIPROv2` includes `Dry::Configurable`. Configure at the class level (defaults for all instances) or instance level (overrides class defaults).
**Class-level defaults:**
```ruby
DSPy::Teleprompt::MIPROv2.configure do |config|
config.optimization_strategy = :bayesian
config.num_trials = 30
config.bootstrap_sets = 10
end
```
**Instance-level overrides:**
```ruby
optimizer = DSPy::Teleprompt::MIPROv2.new(metric: metric)
optimizer.configure do |config|
config.num_trials = 15
config.num_instruction_candidates = 6
config.bootstrap_sets = 5
config.max_bootstrapped_examples = 4
config.max_labeled_examples = 16
config.optimization_strategy = :adaptive # :greedy, :adaptive, :bayesian
config.early_stopping_patience = 3
config.init_temperature = 1.0
config.final_temperature = 0.1
config.minibatch_size = nil # nil = auto
config.auto_seed = 42
end
```
The `optimization_strategy` setting accepts symbols (`:greedy`, `:adaptive`, `:bayesian`) and coerces them internally to `DSPy::Teleprompt::OptimizationStrategy` T::Enum values.
The old `config:` constructor parameter is removed. Passing `config:` raises `ArgumentError`.
### Auto presets via configure
Instead of `AutoMode`, set the preset through the configure block:
```ruby
optimizer = DSPy::Teleprompt::MIPROv2.new(metric: metric)
optimizer.configure do |config|
config.auto_preset = DSPy::Teleprompt::AutoPreset.deserialize("medium")
end
```
### Compile and inspect
```ruby
program = DSPy::Predict.new(MySignature)
result = optimizer.compile(
program,
trainset: train_examples,
valset: val_examples
)
optimized_program = result.optimized_program
puts "Best score: #{result.best_score_value}"
```
The `result` object exposes:
- `optimized_program` -- ready-to-use predictor with updated instruction and demos.
- `optimization_trace[:trial_logs]` -- per-trial record of instructions, demos, and scores.
- `metadata[:optimizer]` -- `"MIPROv2"`, useful when persisting experiments from multiple optimizers.
### Multi-stage programs
MIPROv2 generates dataset summaries for each predictor and proposes per-stage instructions. For a ReAct agent with `thought_generator` and `observation_processor` predictors, the optimizer handles credit assignment internally. The metric only needs to evaluate the final output.
### Bootstrap sampling
During the bootstrap phase MIPROv2:
1. Generates dataset summaries from the training set.
2. Bootstraps few-shot demonstrations by running the baseline program.
3. Proposes candidate instructions grounded in the summaries and bootstrapped examples.
4. Evaluates each candidate on mini-batches drawn from the validation set.
Control the bootstrap phase with `bootstrap_sets`, `max_bootstrapped_examples`, and `max_labeled_examples`.
### Bayesian optimization
When `optimization_strategy` is `:bayesian` (or when using the `heavy` preset), MIPROv2 fits a Gaussian Process surrogate over past trial scores to select the next candidate. This replaces random search with informed exploration, reducing the number of trials needed to find high-scoring instructions.
---
## GEPA
GEPA (Genetic-Pareto Reflective Prompt Evolution) is a feedback-driven optimizer. It runs the program on a small batch, collects scores and textual feedback, and asks a reflection LM to rewrite the instruction. Improved candidates are retained on a Pareto frontier.
### Installation
```ruby
# Gemfile
gem "dspy"
gem "dspy-gepa"
```
The `dspy-gepa` gem depends on the `gepa` core optimizer gem automatically.
### Metric contract
GEPA metrics return `DSPy::Prediction` with both a numeric score and a feedback string. Do not return a plain boolean.
```ruby
metric = lambda do |example, prediction|
expected = example.expected_values[:label]
predicted = prediction.label
score = predicted == expected ? 1.0 : 0.0
feedback = if score == 1.0
"Correct (#{expected}) for: \"#{example.input_values[:text][0..60]}\""
else
"Misclassified (expected #{expected}, got #{predicted}) for: \"#{example.input_values[:text][0..60]}\""
end
DSPy::Prediction.new(score: score, feedback: feedback)
end
```
Keep the score in `[0, 1]`. Always include a short feedback message explaining what happened -- GEPA hands this text to the reflection model so it can reason about failures.
### Feedback maps
`feedback_map` targets individual predictors inside a composite module. Each entry receives keyword arguments and returns a `DSPy::Prediction`:
```ruby
feedback_map = {
'self' => lambda do |predictor_output:, predictor_inputs:, module_inputs:, module_outputs:, captured_trace:|
expected = module_inputs.expected_values[:label]
predicted = predictor_output.label
DSPy::Prediction.new(
score: predicted == expected ? 1.0 : 0.0,
feedback: "Classifier saw \"#{predictor_inputs[:text][0..80]}\" -> #{predicted} (expected #{expected})"
)
end
}
```
For single-predictor programs, key the map with `'self'`. For multi-predictor chains, add entries per component so the reflection LM sees localized context at each step. Omit `feedback_map` entirely if the top-level metric already covers the basics.
### Configuring the teleprompter
```ruby
teleprompter = DSPy::Teleprompt::GEPA.new(
metric: metric,
reflection_lm: DSPy::ReflectionLM.new('openai/gpt-4o-mini', api_key: ENV['OPENAI_API_KEY']),
feedback_map: feedback_map,
config: {
max_metric_calls: 600,
minibatch_size: 6,
skip_perfect_score: false
}
)
```
Key configuration knobs:
| Knob | Purpose |
|----------------------|-------------------------------------------------------------------------------------------|
| `max_metric_calls` | Hard budget on evaluation calls. Set to at least the validation set size plus a few minibatches. |
| `minibatch_size` | Examples per reflective replay batch. Smaller = cheaper iterations, noisier scores. |
| `skip_perfect_score` | Set `true` to stop early when a candidate reaches score `1.0`. |
### Minibatch sizing
| Goal | Suggested size | Rationale |
|-------------------------------------------------|----------------|------------------------------------------------------------|
| Explore many candidates within a tight budget | 3--6 | Cheap iterations, more prompt variants, noisier metrics. |
| Stable metrics when each rollout is costly | 8--12 | Smoother scores, fewer candidates unless budget is raised. |
| Investigate specific failure modes | 3--4 then 8+ | Start with breadth, increase once patterns emerge. |
### Compile and evaluate
```ruby
program = DSPy::Predict.new(MySignature)
result = teleprompter.compile(program, trainset: train, valset: val)
optimized_program = result.optimized_program
test_metrics = evaluate(optimized_program, test)
```
The `result` object exposes:
- `optimized_program` -- predictor with updated instruction and few-shot examples.
- `best_score_value` -- validation score for the best candidate.
- `metadata` -- candidate counts, trace hashes, and telemetry IDs.
### Reflection LM
Swap `DSPy::ReflectionLM` for any callable object that accepts the reflection prompt hash and returns a string. The default reflection signature extracts the new instruction from triple backticks in the response.
### Experiment tracking
Plug `GEPA::Logging::ExperimentTracker` into a persistence layer:
```ruby
tracker = GEPA::Logging::ExperimentTracker.new
tracker.with_subscriber { |event| MyModel.create!(payload: event) }
teleprompter = DSPy::Teleprompt::GEPA.new(
metric: metric,
reflection_lm: reflection_lm,
experiment_tracker: tracker,
config: { max_metric_calls: 900 }
)
```
The tracker emits Pareto update events, merge decisions, and candidate evolution records as JSONL.
### Pareto frontier
GEPA maintains a diverse candidate pool and samples from the Pareto frontier instead of mutating only the top-scoring program. This balances exploration and prevents the search from collapsing onto a single lineage.
Enable the merge proposer after multiple strong lineages emerge:
```ruby
config: {
max_metric_calls: 900,
enable_merge_proposer: true
}
```
Premature merges eat budget without meaningful gains. Gate merge on having several validated candidates first.
### Advanced options
- `acceptance_strategy:` -- plug in bespoke Pareto filters or early-stop heuristics.
- Telemetry spans emit via `GEPA::Telemetry`. Enable global observability with `DSPy.configure { |c| c.observability = true }` to stream spans to an OpenTelemetry exporter.
---
## Evaluation Framework
`DSPy::Evals` provides batch evaluation of predictors against test datasets with built-in and custom metrics.
### Basic usage
```ruby
metric = proc do |example, prediction|
prediction.answer == example.expected_values[:answer]
end
evaluator = DSPy::Evals.new(predictor, metric: metric)
result = evaluator.evaluate(
test_examples,
display_table: true,
display_progress: true
)
puts "Pass rate: #{(result.pass_rate * 100).round(1)}%"
puts "Passed: #{result.passed_examples}/#{result.total_examples}"
```
### DSPy::Example
Convert raw data into `DSPy::Example` instances before passing to optimizers or evaluators. Each example carries `input_values` and `expected_values`:
```ruby
examples = rows.map do |row|
DSPy::Example.new(
input_values: { text: row[:text] },
expected_values: { label: row[:label] }
)
end
train, val, test = split_examples(examples, train_ratio: 0.6, val_ratio: 0.2, seed: 42)
```
Hold back a test set from the optimization loop. Optimizers work on train/val; only the test set proves generalization.
### Built-in metrics
```ruby
# Exact match -- prediction must exactly equal expected value
metric = DSPy::Metrics.exact_match(field: :answer, case_sensitive: true)
# Contains -- prediction must contain expected substring
metric = DSPy::Metrics.contains(field: :answer, case_sensitive: false)
# Numeric difference -- numeric output within tolerance
metric = DSPy::Metrics.numeric_difference(field: :answer, tolerance: 0.01)
# Composite AND -- all sub-metrics must pass
metric = DSPy::Metrics.composite_and(
DSPy::Metrics.exact_match(field: :answer),
DSPy::Metrics.contains(field: :reasoning)
)
```
### Custom metrics
```ruby
quality_metric = lambda do |example, prediction|
return false unless prediction
score = 0.0
score += 0.5 if prediction.answer == example.expected_values[:answer]
score += 0.3 if prediction.explanation && prediction.explanation.length > 50
score += 0.2 if prediction.confidence && prediction.confidence > 0.8
score >= 0.7
end
evaluator = DSPy::Evals.new(predictor, metric: quality_metric)
```
Access prediction fields with dot notation (`prediction.answer`), not hash notation.
### Observability hooks
Register callbacks without editing the evaluator:
```ruby
DSPy::Evals.before_example do |payload|
example = payload[:example]
DSPy.logger.info("Evaluating example #{example.id}") if example.respond_to?(:id)
end
DSPy::Evals.after_batch do |payload|
result = payload[:result]
Langfuse.event(
name: 'eval.batch',
metadata: {
total: result.total_examples,
passed: result.passed_examples,
score: result.score
}
)
end
```
Available hooks: `before_example`, `after_example`, `before_batch`, `after_batch`.
### Langfuse score export
Enable `export_scores: true` to emit `score.create` events for each evaluated example and a batch score at the end:
```ruby
evaluator = DSPy::Evals.new(
predictor,
metric: metric,
export_scores: true,
score_name: 'qa_accuracy' # default: 'evaluation'
)
result = evaluator.evaluate(test_examples)
# Emits per-example scores + overall batch score via DSPy::Scores::Exporter
```
Scores attach to the current trace context automatically and flow to Langfuse asynchronously.
### Evaluation results
```ruby
result = evaluator.evaluate(test_examples)
result.score # Overall score (0.0 to 1.0)
result.passed_count # Examples that passed
result.failed_count # Examples that failed
result.error_count # Examples that errored
result.results.each do |r|
r.passed # Boolean
r.score # Numeric score
r.error # Error message if the example errored
end
```
### Integration with optimizers
```ruby
metric = proc do |example, prediction|
expected = example.expected_values[:answer].to_s.strip.downcase
predicted = prediction.answer.to_s.strip.downcase
!expected.empty? && predicted.include?(expected)
end
optimizer = DSPy::Teleprompt::MIPROv2::AutoMode.medium(metric: metric)
result = optimizer.compile(
DSPy::Predict.new(QASignature),
trainset: train_examples,
valset: val_examples
)
evaluator = DSPy::Evals.new(result.optimized_program, metric: metric)
test_result = evaluator.evaluate(test_examples, display_table: true)
puts "Test accuracy: #{(test_result.pass_rate * 100).round(2)}%"
```
---
## Storage System
`DSPy::Storage` persists optimization results, tracks history, and manages multiple versions of optimized programs.
### ProgramStorage (low-level)
```ruby
storage = DSPy::Storage::ProgramStorage.new(storage_path: "./dspy_storage")
# Save
saved = storage.save_program(
result.optimized_program,
result,
metadata: {
signature_class: 'ClassifyText',
optimizer: 'MIPROv2',
examples_count: examples.size
}
)
puts "Stored with ID: #{saved.program_id}"
# Load
saved = storage.load_program(program_id)
predictor = saved.program
score = saved.optimization_result[:best_score_value]
# List
storage.list_programs.each do |p|
puts "#{p[:program_id]} -- score: #{p[:best_score]} -- saved: #{p[:saved_at]}"
end
```
### StorageManager (recommended)
```ruby
manager = DSPy::Storage::StorageManager.new
# Save with tags
saved = manager.save_optimization_result(
result,
tags: ['production', 'sentiment-analysis'],
description: 'Optimized sentiment classifier v2'
)
# Find programs
programs = manager.find_programs(
optimizer: 'MIPROv2',
min_score: 0.85,
tags: ['production']
)
recent = manager.find_programs(
max_age_days: 7,
signature_class: 'ClassifyText'
)
# Get best program for a signature
best = manager.get_best_program('ClassifyText')
predictor = best.program
```
Global shorthand:
```ruby
DSPy::Storage::StorageManager.save(result, metadata: { version: '2.0' })
DSPy::Storage::StorageManager.load(program_id)
DSPy::Storage::StorageManager.best('ClassifyText')
```
### Checkpoints
Create and restore checkpoints during long-running optimizations:
```ruby
# Save a checkpoint
manager.create_checkpoint(
current_result,
'iteration_50',
metadata: { iteration: 50, current_score: 0.87 }
)
# Restore
restored = manager.restore_checkpoint('iteration_50')
program = restored.program
# Auto-checkpoint every N iterations
if iteration % 10 == 0
manager.create_checkpoint(current_result, "auto_checkpoint_#{iteration}")
end
```
### Import and export
Share programs between environments:
```ruby
storage = DSPy::Storage::ProgramStorage.new
# Export
storage.export_programs(['abc123', 'def456'], './export_backup.json')
# Import
imported = storage.import_programs('./export_backup.json')
puts "Imported #{imported.size} programs"
```
### Optimization history
```ruby
history = manager.get_optimization_history
history[:summary][:total_programs]
history[:summary][:avg_score]
history[:optimizer_stats].each do |optimizer, stats|
puts "#{optimizer}: #{stats[:count]} programs, best: #{stats[:best_score]}"
end
history[:trends][:improvement_percentage]
```
### Program comparison
```ruby
comparison = manager.compare_programs(id_a, id_b)
comparison[:comparison][:score_difference]
comparison[:comparison][:better_program]
comparison[:comparison][:age_difference_hours]
```
### Storage configuration
```ruby
config = DSPy::Storage::StorageManager::StorageConfig.new
config.storage_path = Rails.root.join('dspy_storage')
config.auto_save = true
config.save_intermediate_results = false
config.max_stored_programs = 100
manager = DSPy::Storage::StorageManager.new(config: config)
```
### Cleanup
Remove old programs. Cleanup retains the best performing and most recent programs using a weighted score (70% performance, 30% recency):
```ruby
deleted_count = manager.cleanup_old_programs
```
### Storage events
The storage system emits structured log events for monitoring:
- `dspy.storage.save_start`, `dspy.storage.save_complete`, `dspy.storage.save_error`
- `dspy.storage.load_start`, `dspy.storage.load_complete`, `dspy.storage.load_error`
- `dspy.storage.delete`, `dspy.storage.export`, `dspy.storage.import`, `dspy.storage.cleanup`
### File layout
```
dspy_storage/
programs/
abc123def456.json
789xyz012345.json
history.json
```
---
## API rules
- Call predictors with `.call()`, not `.forward()`.
- Access prediction fields with dot notation (`result.answer`), not hash notation (`result[:answer]`).
- GEPA metrics return `DSPy::Prediction.new(score:, feedback:)`, not a boolean.
- MIPROv2 metrics may return `true`/`false`, a numeric score, or `DSPy::Prediction`.

418
plugins/compound-engineering/skills/dspy-ruby/references/providers.md
View File

@@ -1,418 +0,0 @@
# DSPy.rb LLM Providers
## Adapter Architecture
DSPy.rb ships provider SDKs as separate adapter gems. Install only the adapters the project needs. Each adapter gem depends on the official SDK for its provider and auto-loads when present -- no explicit `require` necessary.
```ruby
# Gemfile
gem 'dspy' # core framework (no provider SDKs)
gem 'dspy-openai' # OpenAI, OpenRouter, Ollama
gem 'dspy-anthropic' # Claude
gem 'dspy-gemini' # Gemini
gem 'dspy-ruby_llm' # RubyLLM unified adapter (12+ providers)
```
---
## Per-Provider Adapters
### dspy-openai
Covers any endpoint that speaks the OpenAI chat-completions protocol: OpenAI itself, OpenRouter, and Ollama.
**SDK dependency:** `openai ~> 0.17`
```ruby
# OpenAI
lm = DSPy::LM.new('openai/gpt-4o-mini', api_key: ENV['OPENAI_API_KEY'])
# OpenRouter -- access 200+ models behind a single key
lm = DSPy::LM.new('openrouter/x-ai/grok-4-fast:free',
api_key: ENV['OPENROUTER_API_KEY']
)
# Ollama -- local models, no API key required
lm = DSPy::LM.new('ollama/llama3.2')
# Remote Ollama instance
lm = DSPy::LM.new('ollama/llama3.2',
base_url: 'https://my-ollama.example.com/v1',
api_key: 'optional-auth-token'
)
```
All three sub-adapters share the same request handling, structured-output support, and error reporting. Swap providers without changing higher-level DSPy code.
For OpenRouter models that lack native structured-output support, disable it explicitly:
```ruby
lm = DSPy::LM.new('openrouter/deepseek/deepseek-chat-v3.1:free',
api_key: ENV['OPENROUTER_API_KEY'],
structured_outputs: false
)
```
### dspy-anthropic
Provides the Claude adapter. Install it for any `anthropic/*` model id.
**SDK dependency:** `anthropic ~> 1.12`
```ruby
lm = DSPy::LM.new('anthropic/claude-sonnet-4-20250514',
api_key: ENV['ANTHROPIC_API_KEY']
)
```
Structured outputs default to tool-based JSON extraction (`structured_outputs: true`). Set `structured_outputs: false` to use enhanced-prompting extraction instead.
```ruby
# Tool-based extraction (default, most reliable)
lm = DSPy::LM.new('anthropic/claude-sonnet-4-20250514',
api_key: ENV['ANTHROPIC_API_KEY'],
structured_outputs: true
)
# Enhanced prompting extraction
lm = DSPy::LM.new('anthropic/claude-sonnet-4-20250514',
api_key: ENV['ANTHROPIC_API_KEY'],
structured_outputs: false
)
```
### dspy-gemini
Provides the Gemini adapter. Install it for any `gemini/*` model id.
**SDK dependency:** `gemini-ai ~> 4.3`
```ruby
lm = DSPy::LM.new('gemini/gemini-2.5-flash',
api_key: ENV['GEMINI_API_KEY']
)
```
**Environment variable:** `GEMINI_API_KEY` (also accepts `GOOGLE_API_KEY`).
---
## RubyLLM Unified Adapter
The `dspy-ruby_llm` gem provides a single adapter that routes to 12+ providers through [RubyLLM](https://rubyllm.com). Use it when a project talks to multiple providers or needs access to Bedrock, VertexAI, DeepSeek, or Mistral without dedicated adapter gems.
**SDK dependency:** `ruby_llm ~> 1.3`
### Model ID Format
Prefix every model id with `ruby_llm/`:
```ruby
lm = DSPy::LM.new('ruby_llm/gpt-4o-mini')
lm = DSPy::LM.new('ruby_llm/claude-sonnet-4-20250514')
lm = DSPy::LM.new('ruby_llm/gemini-2.5-flash')
```
The adapter detects the provider from RubyLLM's model registry automatically. For models not in the registry, pass `provider:` explicitly:
```ruby
lm = DSPy::LM.new('ruby_llm/llama3.2', provider: 'ollama')
lm = DSPy::LM.new('ruby_llm/anthropic/claude-3-opus',
api_key: ENV['OPENROUTER_API_KEY'],
provider: 'openrouter'
)
```
### Using Existing RubyLLM Configuration
When RubyLLM is already configured globally, omit the `api_key:` argument. DSPy reuses the global config automatically:
```ruby
RubyLLM.configure do |config|
config.openai_api_key = ENV['OPENAI_API_KEY']
config.anthropic_api_key = ENV['ANTHROPIC_API_KEY']
end
# No api_key needed -- picks up the global config
DSPy.configure do |c|
c.lm = DSPy::LM.new('ruby_llm/gpt-4o-mini')
end
```
When an `api_key:` (or any of `base_url:`, `timeout:`, `max_retries:`) is passed, DSPy creates a **scoped context** instead of reusing the global config.
### Cloud-Hosted Providers (Bedrock, VertexAI)
Configure RubyLLM globally first, then reference the model:
```ruby
# AWS Bedrock
RubyLLM.configure do |c|
c.bedrock_api_key = ENV['AWS_ACCESS_KEY_ID']
c.bedrock_secret_key = ENV['AWS_SECRET_ACCESS_KEY']
c.bedrock_region = 'us-east-1'
end
lm = DSPy::LM.new('ruby_llm/anthropic.claude-3-5-sonnet', provider: 'bedrock')
# Google VertexAI
RubyLLM.configure do |c|
c.vertexai_project_id = 'your-project-id'
c.vertexai_location = 'us-central1'
end
lm = DSPy::LM.new('ruby_llm/gemini-pro', provider: 'vertexai')
```
### Supported Providers Table
| Provider | Example Model ID | Notes |
|-------------|--------------------------------------------|---------------------------------|
| OpenAI | `ruby_llm/gpt-4o-mini` | Auto-detected from registry |
| Anthropic | `ruby_llm/claude-sonnet-4-20250514` | Auto-detected from registry |
| Gemini | `ruby_llm/gemini-2.5-flash` | Auto-detected from registry |
| DeepSeek | `ruby_llm/deepseek-chat` | Auto-detected from registry |
| Mistral | `ruby_llm/mistral-large` | Auto-detected from registry |
| Ollama | `ruby_llm/llama3.2` | Use `provider: 'ollama'` |
| AWS Bedrock | `ruby_llm/anthropic.claude-3-5-sonnet` | Configure RubyLLM globally |
| VertexAI | `ruby_llm/gemini-pro` | Configure RubyLLM globally |
| OpenRouter | `ruby_llm/anthropic/claude-3-opus` | Use `provider: 'openrouter'` |
| Perplexity | `ruby_llm/llama-3.1-sonar-large` | Use `provider: 'perplexity'` |
| GPUStack | `ruby_llm/model-name` | Use `provider: 'gpustack'` |
---
## Rails Initializer Pattern
Configure DSPy inside an `after_initialize` block so Rails credentials and environment are fully loaded:
```ruby
# config/initializers/dspy.rb
Rails.application.config.after_initialize do
return if Rails.env.test? # skip in test -- use VCR cassettes instead
DSPy.configure do |config|
config.lm = DSPy::LM.new(
'openai/gpt-4o-mini',
api_key: Rails.application.credentials.openai_api_key,
structured_outputs: true
)
config.logger = if Rails.env.production?
Dry.Logger(:dspy, formatter: :json) do |logger|
logger.add_backend(stream: Rails.root.join("log/dspy.log"))
end
else
Dry.Logger(:dspy) do |logger|
logger.add_backend(level: :debug, stream: $stdout)
end
end
end
end
```
Key points:
- Wrap in `after_initialize` so `Rails.application.credentials` is available.
- Return early in the test environment. Rely on VCR cassettes for deterministic LLM responses.
- Set `structured_outputs: true` (the default) for provider-native JSON extraction.
- Use `Dry.Logger` with `:json` formatter in production for structured log parsing.
---
## Fiber-Local LM Context
`DSPy.with_lm` sets a temporary language-model override scoped to the current Fiber. Every predictor call inside the block uses the override; outside the block the previous LM takes effect again.
```ruby
fast = DSPy::LM.new('openai/gpt-4o-mini', api_key: ENV['OPENAI_API_KEY'])
powerful = DSPy::LM.new('anthropic/claude-sonnet-4-20250514', api_key: ENV['ANTHROPIC_API_KEY'])
classifier = Classifier.new
# Uses the global LM
result = classifier.call(text: "Hello")
# Temporarily switch to the fast model
DSPy.with_lm(fast) do
result = classifier.call(text: "Hello") # uses gpt-4o-mini
end
# Temporarily switch to the powerful model
DSPy.with_lm(powerful) do
result = classifier.call(text: "Hello") # uses claude-sonnet-4
end
```
### LM Resolution Hierarchy
DSPy resolves the active language model in this order:
1. **Instance-level LM** -- set directly on a module instance via `configure`
2. **Fiber-local LM** -- set via `DSPy.with_lm`
3. **Global LM** -- set via `DSPy.configure`
Instance-level configuration always wins, even inside a `DSPy.with_lm` block:
```ruby
classifier = Classifier.new
classifier.configure { |c| c.lm = DSPy::LM.new('anthropic/claude-sonnet-4-20250514', api_key: ENV['ANTHROPIC_API_KEY']) }
fast = DSPy::LM.new('openai/gpt-4o-mini', api_key: ENV['OPENAI_API_KEY'])
DSPy.with_lm(fast) do
classifier.call(text: "Test") # still uses claude-sonnet-4 (instance-level wins)
end
```
### configure_predictor for Fine-Grained Agent Control
Complex agents (`ReAct`, `CodeAct`, `DeepResearch`, `DeepSearch`) contain internal predictors. Use `configure` for a blanket override and `configure_predictor` to target a specific sub-predictor:
```ruby
agent = DSPy::ReAct.new(MySignature, tools: tools)
# Set a default LM for the agent and all its children
agent.configure { |c| c.lm = DSPy::LM.new('openai/gpt-4o-mini', api_key: ENV['OPENAI_API_KEY']) }
# Override just the reasoning predictor with a more capable model
agent.configure_predictor('thought_generator') do |c|
c.lm = DSPy::LM.new('anthropic/claude-sonnet-4-20250514', api_key: ENV['ANTHROPIC_API_KEY'])
end
result = agent.call(question: "Summarize the report")
```
Both methods support chaining:
```ruby
agent
.configure { |c| c.lm = cheap_model }
.configure_predictor('thought_generator') { |c| c.lm = expensive_model }
```
#### Available Predictors by Agent Type
| Agent | Internal Predictors |
|----------------------|------------------------------------------------------------------|
| `DSPy::ReAct` | `thought_generator`, `observation_processor` |
| `DSPy::CodeAct` | `code_generator`, `observation_processor` |
| `DSPy::DeepResearch` | `planner`, `synthesizer`, `qa_reviewer`, `reporter` |
| `DSPy::DeepSearch` | `seed_predictor`, `search_predictor`, `reader_predictor`, `reason_predictor` |
#### Propagation Rules
- Configuration propagates recursively to children and grandchildren.
- Children with an already-configured LM are **not** overwritten by a later parent `configure` call.
- Configure the parent first, then override specific children.
---
## Feature-Flagged Model Selection
Use a `FeatureFlags` module backed by ENV vars to centralize model selection. Each tool or agent reads its model from the flags, falling back to a global default.
```ruby
module FeatureFlags
module_function
def default_model
ENV.fetch('DSPY_DEFAULT_MODEL', 'openai/gpt-4o-mini')
end
def default_api_key
ENV.fetch('DSPY_DEFAULT_API_KEY') { ENV.fetch('OPENAI_API_KEY', nil) }
end
def model_for(tool_name)
env_key = "DSPY_MODEL_#{tool_name.upcase}"
ENV.fetch(env_key, default_model)
end
def api_key_for(tool_name)
env_key = "DSPY_API_KEY_#{tool_name.upcase}"
ENV.fetch(env_key, default_api_key)
end
end
```
### Per-Tool Model Override
Override an individual tool's model without touching application code:
```bash
# .env
DSPY_DEFAULT_MODEL=openai/gpt-4o-mini
DSPY_DEFAULT_API_KEY=sk-...
# Override the classifier to use Claude
DSPY_MODEL_CLASSIFIER=anthropic/claude-sonnet-4-20250514
DSPY_API_KEY_CLASSIFIER=sk-ant-...
# Override the summarizer to use Gemini
DSPY_MODEL_SUMMARIZER=gemini/gemini-2.5-flash
DSPY_API_KEY_SUMMARIZER=...
```
Wire each agent to its flag at initialization:
```ruby
class ClassifierAgent < DSPy::Module
def initialize
super
model = FeatureFlags.model_for('classifier')
api_key = FeatureFlags.api_key_for('classifier')
@predictor = DSPy::Predict.new(ClassifySignature)
configure { |c| c.lm = DSPy::LM.new(model, api_key: api_key) }
end
def forward(text:)
@predictor.call(text: text)
end
end
```
This pattern keeps model routing declarative and avoids scattering `DSPy::LM.new` calls across the codebase.
---
## Compatibility Matrix
Feature support across direct adapter gems. All features listed assume `structured_outputs: true` (the default).
| Feature | OpenAI | Anthropic | Gemini | Ollama | OpenRouter | RubyLLM |
|----------------------|--------|-----------|--------|----------|------------|-------------|
| Structured Output | Native JSON mode | Tool-based extraction | Native JSON schema | OpenAI-compatible JSON | Varies by model | Via `with_schema` |
| Vision (Images) | File + URL | File + Base64 | File + Base64 | Limited | Varies | Delegates to underlying provider |
| Image URLs | Yes | No | No | No | Varies | Depends on provider |
| Tool Calling | Yes | Yes | Yes | Varies | Varies | Yes |
| Streaming | Yes | Yes | Yes | Yes | Yes | Yes |
**Notes:**
- **Structured Output** is enabled by default on every adapter. Set `structured_outputs: false` to fall back to enhanced-prompting extraction.
- **Vision / Image URLs:** Only OpenAI supports passing a URL directly. For Anthropic and Gemini, load images from file or Base64:
```ruby
DSPy::Image.from_url("https://example.com/img.jpg") # OpenAI only
DSPy::Image.from_file("path/to/image.jpg") # all providers
DSPy::Image.from_base64(data, mime_type: "image/jpeg") # all providers
```
- **RubyLLM** delegates to the underlying provider, so feature support matches the provider column in the table.
### Choosing an Adapter Strategy
| Scenario | Recommended Adapter |
|-------------------------------------------|--------------------------------|
| Single provider (OpenAI, Claude, or Gemini) | Dedicated gem (`dspy-openai`, `dspy-anthropic`, `dspy-gemini`) |
| Multi-provider with per-agent model routing | `dspy-ruby_llm` |
| AWS Bedrock or Google VertexAI | `dspy-ruby_llm` |
| Local development with Ollama | `dspy-openai` (Ollama sub-adapter) or `dspy-ruby_llm` |
| OpenRouter for cost optimization | `dspy-openai` (OpenRouter sub-adapter) |
### Current Recommended Models
| Provider | Model ID | Use Case |
|-----------|---------------------------------------|-----------------------|
| OpenAI | `openai/gpt-4o-mini` | Fast, cost-effective |
| Anthropic | `anthropic/claude-sonnet-4-20250514` | Balanced reasoning |
| Gemini | `gemini/gemini-2.5-flash` | Fast, cost-effective |
| Ollama | `ollama/llama3.2` | Local, zero API cost |

502
plugins/compound-engineering/skills/dspy-ruby/references/toolsets.md
View File

@@ -1,502 +0,0 @@
# DSPy.rb Toolsets
## Tools::Base
`DSPy::Tools::Base` is the base class for single-purpose tools. Each subclass exposes one operation to an LLM agent through a `call` method.
### Defining a Tool
Set the tool's identity with the `tool_name` and `tool_description` class-level DSL methods. Define the `call` instance method with a Sorbet `sig` declaration so DSPy.rb can generate the JSON schema the LLM uses to invoke the tool.
```ruby
class WeatherLookup < DSPy::Tools::Base
extend T::Sig
tool_name "weather_lookup"
tool_description "Look up current weather for a given city"
sig { params(city: String, units: T.nilable(String)).returns(String) }
def call(city:, units: nil)
# Fetch weather data and return a string summary
"72F and sunny in #{city}"
end
end
```
Key points:
- Inherit from `DSPy::Tools::Base`, not `DSPy::Tool`.
- Use `tool_name` (class method) to set the name the LLM sees. Without it, the class name is lowercased as a fallback.
- Use `tool_description` (class method) to set the human-readable description surfaced in the tool schema.
- The `call` method must use **keyword arguments**. Positional arguments are supported but keyword arguments produce better schemas.
- Always attach a Sorbet `sig` to `call`. Without a signature, the generated schema has empty properties and the LLM cannot determine parameter types.
### Schema Generation
`call_schema_object` introspects the Sorbet signature on `call` and returns a hash representing the JSON Schema `parameters` object:
```ruby
WeatherLookup.call_schema_object
# => {
# type: "object",
# properties: {
# city: { type: "string", description: "Parameter city" },
# units: { type: "string", description: "Parameter units (optional)" }
# },
# required: ["city"]
# }
```
`call_schema` wraps this in the full LLM tool-calling format:
```ruby
WeatherLookup.call_schema
# => {
# type: "function",
# function: {
# name: "call",
# description: "Call the WeatherLookup tool",
# parameters: { ... }
# }
# }
```
### Using Tools with ReAct
Pass tool instances in an array to `DSPy::ReAct`:
```ruby
agent = DSPy::ReAct.new(
MySignature,
tools: [WeatherLookup.new, AnotherTool.new]
)
result = agent.call(question: "What is the weather in Berlin?")
puts result.answer
```
Access output fields with dot notation (`result.answer`), not hash access (`result[:answer]`).
---
## Tools::Toolset
`DSPy::Tools::Toolset` groups multiple related methods into a single class. Each exposed method becomes an independent tool from the LLM's perspective.
### Defining a Toolset
```ruby
class DatabaseToolset < DSPy::Tools::Toolset
extend T::Sig
toolset_name "db"
tool :query, description: "Run a read-only SQL query"
tool :insert, description: "Insert a record into a table"
tool :delete, description: "Delete a record by ID"
sig { params(sql: String).returns(String) }
def query(sql:)
# Execute read query
end
sig { params(table: String, data: T::Hash[String, String]).returns(String) }
def insert(table:, data:)
# Insert record
end
sig { params(table: String, id: Integer).returns(String) }
def delete(table:, id:)
# Delete record
end
end
```
### DSL Methods
**`toolset_name(name)`** -- Set the prefix for all generated tool names. If omitted, the class name minus `Toolset` suffix is lowercased (e.g., `DatabaseToolset` becomes `database`).
```ruby
toolset_name "db"
# tool :query produces a tool named "db_query"
```
**`tool(method_name, tool_name:, description:)`** -- Expose a method as a tool.
- `method_name` (Symbol, required) -- the instance method to expose.
- `tool_name:` (String, optional) -- override the default `<toolset_name>_<method_name>` naming.
- `description:` (String, optional) -- description shown to the LLM. Defaults to a humanized version of the method name.
```ruby
tool :word_count, tool_name: "text_wc", description: "Count lines, words, and characters"
# Produces a tool named "text_wc" instead of "text_word_count"
```
### Converting to a Tool Array
Call `to_tools` on the class (not an instance) to get an array of `ToolProxy` objects compatible with `DSPy::Tools::Base`:
```ruby
agent = DSPy::ReAct.new(
AnalyzeText,
tools: DatabaseToolset.to_tools
)
```
Each `ToolProxy` wraps one method, delegates `call` to the underlying toolset instance, and generates its own JSON schema from the method's Sorbet signature.
### Shared State
All tool proxies from a single `to_tools` call share one toolset instance. Store shared state (connections, caches, configuration) in the toolset's `initialize`:
```ruby
class ApiToolset < DSPy::Tools::Toolset
extend T::Sig
toolset_name "api"
tool :get, description: "Make a GET request"
tool :post, description: "Make a POST request"
sig { params(base_url: String).void }
def initialize(base_url:)
@base_url = base_url
@client = HTTP.persistent(base_url)
end
sig { params(path: String).returns(String) }
def get(path:)
@client.get("#{@base_url}#{path}").body.to_s
end
sig { params(path: String, body: String).returns(String) }
def post(path:, body:)
@client.post("#{@base_url}#{path}", body: body).body.to_s
end
end
```
---
## Type Safety
Sorbet signatures on tool methods drive both JSON schema generation and automatic type coercion of LLM responses.
### Basic Types
```ruby
sig { params(
text: String,
count: Integer,
score: Float,
enabled: T::Boolean,
threshold: Numeric
).returns(String) }
def analyze(text:, count:, score:, enabled:, threshold:)
# ...
end
```
| Sorbet Type | JSON Schema |
|------------------|----------------------------------------------------|
| `String` | `{"type": "string"}` |
| `Integer` | `{"type": "integer"}` |
| `Float` | `{"type": "number"}` |
| `Numeric` | `{"type": "number"}` |
| `T::Boolean` | `{"type": "boolean"}` |
| `T::Enum` | `{"type": "string", "enum": [...]}` |
| `T::Struct` | `{"type": "object", "properties": {...}}` |
| `T::Array[Type]` | `{"type": "array", "items": {...}}` |
| `T::Hash[K, V]` | `{"type": "object", "additionalProperties": {...}}`|
| `T.nilable(Type)`| `{"type": [original, "null"]}` |
| `T.any(T1, T2)` | `{"oneOf": [{...}, {...}]}` |
| `T.class_of(X)` | `{"type": "string"}` |
### T::Enum Parameters
Define a `T::Enum` and reference it in a tool signature. DSPy.rb generates a JSON Schema `enum` constraint and automatically deserializes the LLM's string response into the correct enum instance.
```ruby
class Priority < T::Enum
enums do
Low = new('low')
Medium = new('medium')
High = new('high')
Critical = new('critical')
end
end
class Status < T::Enum
enums do
Pending = new('pending')
InProgress = new('in-progress')
Completed = new('completed')
end
end
sig { params(priority: Priority, status: Status).returns(String) }
def update_task(priority:, status:)
"Updated to #{priority.serialize} / #{status.serialize}"
end
```
The generated schema constrains the parameter to valid values:
```json
{
"priority": {
"type": "string",
"enum": ["low", "medium", "high", "critical"]
}
}
```
**Case-insensitive matching**: When the LLM returns `"HIGH"` or `"High"` instead of `"high"`, DSPy.rb first tries an exact `try_deserialize`, then falls back to a case-insensitive lookup. This prevents failures caused by LLM casing variations.
### T::Struct Parameters
Use `T::Struct` for complex nested objects. DSPy.rb generates nested JSON Schema properties and recursively coerces the LLM's hash response into struct instances.
```ruby
class TaskMetadata < T::Struct
prop :id, String
prop :priority, Priority
prop :tags, T::Array[String]
prop :estimated_hours, T.nilable(Float), default: nil
end
class TaskRequest < T::Struct
prop :title, String
prop :description, String
prop :status, Status
prop :metadata, TaskMetadata
prop :assignees, T::Array[String]
end
sig { params(task: TaskRequest).returns(String) }
def create_task(task:)
"Created: #{task.title} (#{task.status.serialize})"
end
```
The LLM sees the full nested object schema and DSPy.rb reconstructs the struct tree from the JSON response, including enum fields inside nested structs.
### Nilable Parameters
Mark optional parameters with `T.nilable(...)` and provide a default value of `nil` in the method signature. These parameters are excluded from the JSON Schema `required` array.
```ruby
sig { params(
query: String,
max_results: T.nilable(Integer),
filter: T.nilable(String)
).returns(String) }
def search(query:, max_results: nil, filter: nil)
# query is required; max_results and filter are optional
end
```
### Collections
Typed arrays and hashes generate precise item/value schemas:
```ruby
sig { params(
tags: T::Array[String],
priorities: T::Array[Priority],
config: T::Hash[String, T.any(String, Integer, Float)]
).returns(String) }
def configure(tags:, priorities:, config:)
# Array elements and hash values are validated and coerced
end
```
### Union Types
`T.any(...)` generates a `oneOf` JSON Schema. When one of the union members is a `T::Struct`, DSPy.rb uses the `_type` discriminator field to select the correct struct class during coercion.
```ruby
sig { params(value: T.any(String, Integer, Float)).returns(String) }
def handle_flexible(value:)
# Accepts multiple types
end
```
---
## Built-in Toolsets
### TextProcessingToolset
`DSPy::Tools::TextProcessingToolset` provides Unix-style text analysis and manipulation operations. Toolset name prefix: `text`.
| Tool Name | Method | Description |
|-----------------------------------|-------------------|--------------------------------------------|
| `text_grep` | `grep` | Search for patterns with optional case-insensitive and count-only modes |
| `text_wc` | `word_count` | Count lines, words, and characters |
| `text_rg` | `ripgrep` | Fast pattern search with context lines |
| `text_extract_lines` | `extract_lines` | Extract a range of lines by number |
| `text_filter_lines` | `filter_lines` | Keep or reject lines matching a regex |
| `text_unique_lines` | `unique_lines` | Deduplicate lines, optionally preserving order |
| `text_sort_lines` | `sort_lines` | Sort lines alphabetically or numerically |
| `text_summarize_text` | `summarize_text` | Produce a statistical summary (counts, averages, frequent words) |
Usage:
```ruby
agent = DSPy::ReAct.new(
AnalyzeText,
tools: DSPy::Tools::TextProcessingToolset.to_tools
)
result = agent.call(text: log_contents, question: "How many error lines are there?")
puts result.answer
```
### GitHubCLIToolset
`DSPy::Tools::GitHubCLIToolset` wraps the `gh` CLI for read-oriented GitHub operations. Toolset name prefix: `github`.
| Tool Name | Method | Description |
|------------------------|-------------------|---------------------------------------------------|
| `github_list_issues` | `list_issues` | List issues filtered by state, labels, assignee |
| `github_list_prs` | `list_prs` | List pull requests filtered by state, author, base|
| `github_get_issue` | `get_issue` | Retrieve details of a single issue |
| `github_get_pr` | `get_pr` | Retrieve details of a single pull request |
| `github_api_request` | `api_request` | Make an arbitrary GET request to the GitHub API |
| `github_traffic_views` | `traffic_views` | Fetch repository traffic view counts |
| `github_traffic_clones`| `traffic_clones` | Fetch repository traffic clone counts |
This toolset uses `T::Enum` parameters (`IssueState`, `PRState`, `ReviewState`) for state filters, demonstrating enum-based tool signatures in practice.
```ruby
agent = DSPy::ReAct.new(
RepoAnalysis,
tools: DSPy::Tools::GitHubCLIToolset.to_tools
)
```
---
## Testing
### Unit Testing Individual Tools
Test `DSPy::Tools::Base` subclasses by instantiating and calling `call` directly:
```ruby
RSpec.describe WeatherLookup do
subject(:tool) { described_class.new }
it "returns weather for a city" do
result = tool.call(city: "Berlin")
expect(result).to include("Berlin")
end
it "exposes the correct tool name" do
expect(tool.name).to eq("weather_lookup")
end
it "generates a valid schema" do
schema = described_class.call_schema_object
expect(schema[:required]).to include("city")
expect(schema[:properties]).to have_key(:city)
end
end
```
### Unit Testing Toolsets
Test toolset methods directly on an instance. Verify tool generation with `to_tools`:
```ruby
RSpec.describe DatabaseToolset do
subject(:toolset) { described_class.new }
it "executes a query" do
result = toolset.query(sql: "SELECT 1")
expect(result).to be_a(String)
end
it "generates tools with correct names" do
tools = described_class.to_tools
names = tools.map(&:name)
expect(names).to contain_exactly("db_query", "db_insert", "db_delete")
end
it "generates tool descriptions" do
tools = described_class.to_tools
query_tool = tools.find { |t| t.name == "db_query" }
expect(query_tool.description).to eq("Run a read-only SQL query")
end
end
```
### Mocking Predictions Inside Tools
When a tool calls a DSPy predictor internally, stub the predictor to isolate tool logic from LLM calls:
```ruby
class SmartSearchTool < DSPy::Tools::Base
extend T::Sig
tool_name "smart_search"
tool_description "Search with query expansion"
sig { void }
def initialize
@expander = DSPy::Predict.new(QueryExpansionSignature)
end
sig { params(query: String).returns(String) }
def call(query:)
expanded = @expander.call(query: query)
perform_search(expanded.expanded_query)
end
private
def perform_search(query)
# actual search logic
end
end
RSpec.describe SmartSearchTool do
subject(:tool) { described_class.new }
before do
expansion_result = double("result", expanded_query: "expanded test query")
allow_any_instance_of(DSPy::Predict).to receive(:call).and_return(expansion_result)
end
it "expands the query before searching" do
allow(tool).to receive(:perform_search).with("expanded test query").and_return("found 3 results")
result = tool.call(query: "test")
expect(result).to eq("found 3 results")
end
end
```
### Testing Enum Coercion
Verify that string values from LLM responses deserialize into the correct enum instances:
```ruby
RSpec.describe "enum coercion" do
it "handles case-insensitive enum values" do
toolset = GitHubCLIToolset.new
# The LLM may return "OPEN" instead of "open"
result = toolset.list_issues(state: IssueState::Open)
expect(result).to be_a(String)
end
end
```
---
## Constraints
- All exposed tool methods must use **keyword arguments**. Positional-only parameters generate schemas but keyword arguments produce more reliable LLM interactions.
- Each exposed method becomes a **separate, independent tool**. Method chaining or multi-step sequences within a single tool call are not supported.
- Shared state across tool proxies is scoped to a single `to_tools` call. Separate `to_tools` invocations create separate toolset instances.
- Methods without a Sorbet `sig` produce an empty parameter schema. The LLM will not know what arguments to pass.

155
plugins/compound-engineering/skills/excalidraw-png-export/SKILL.md Normal file
View File

@@ -0,0 +1,155 @@
---
name: excalidraw-png-export
description: "This skill should be used when creating diagrams, architecture visuals, or flowcharts and exporting them as PNG files. It uses the Excalidraw MCP to render hand-drawn style diagrams locally and Playwright to export them to PNG without sending data to any remote server. Triggers on requests like 'create a diagram', 'make an architecture diagram', 'draw a flowchart and export as PNG', or any request that needs a visual diagram delivered as an image file."
---
# Excalidraw PNG Export
Create hand-drawn style diagrams with the Excalidraw MCP and export them locally to PNG files. All rendering happens on the local machine. Diagram data never leaves the user's computer.
## Prerequisites
### First-Time Setup
Run the setup script once per machine to install Playwright and Chromium headless:
```bash
bash <skill-path>/scripts/setup.sh
```
This creates a `.export-runtime` directory inside `scripts/` with the Node.js dependencies. The setup is idempotent and skips installation if already present.
### Required MCP
The Excalidraw MCP server must be configured. Verify availability by checking for `mcp__excalidraw__create_view` and `mcp__excalidraw__read_checkpoint` tools.
## File Location Convention
Save diagram source files alongside their PNG exports in the project's image directory. This enables re-exporting diagrams when content or styling changes.
**Standard pattern:**
```
docs/images/my-diagram.excalidraw # source (commit this)
docs/images/my-diagram.png # rendered output (commit this)
```
**When updating an existing diagram**, look for a `.excalidraw` file next to the PNG. If one exists, edit it and re-export rather than rebuilding from scratch.
**Temporary files** (raw checkpoint JSON) go in `/tmp/excalidraw-export/` and are discarded after conversion.
## Workflow
### Step 1: Design the Diagram Elements
Translate the user's request into Excalidraw element JSON. Load [excalidraw-element-format.md](./references/excalidraw-element-format.md) for the full element specification, color palette, and sizing guidelines.
Key design decisions:
- Choose appropriate colors from the palette to distinguish different components
- Use `label` on shapes instead of separate text elements
- Use `roundness: { type: 3 }` for rounded corners on rectangles
- Include `cameraUpdate` as the first element to frame the view (MCP rendering only)
- Use arrow bindings (`startBinding`/`endBinding`) to connect shapes
### Step 2: Render with Excalidraw MCP
Call `mcp__excalidraw__create_view` with the element JSON array. This renders an interactive preview in the Claude Code UI.
```
mcp__excalidraw__create_view({ elements: "<JSON array string>" })
```
The response includes a `checkpointId` for retrieving the rendered state.
### Step 3: Extract the Checkpoint Data
Call `mcp__excalidraw__read_checkpoint` with the checkpoint ID to get the full element JSON back.
```
mcp__excalidraw__read_checkpoint({ id: "<checkpointId>" })
```
### Step 4: Convert Checkpoint to .excalidraw File
Use the `convert.mjs` script to transform raw MCP checkpoint JSON into a valid `.excalidraw` file. This handles all the tedious parts automatically:
- Filters out pseudo-elements (`cameraUpdate`, `delete`, `restoreCheckpoint`)
- Adds required Excalidraw defaults (`seed`, `version`, `fontFamily`, etc.)
- Expands `label` properties on shapes/arrows into proper bound text elements
```bash
# Save checkpoint JSON to a temp file, then convert to the project's image directory:
node <skill-path>/scripts/convert.mjs /tmp/excalidraw-export/raw.json docs/images/my-diagram.excalidraw
```
The input JSON should be the raw checkpoint data from `mcp__excalidraw__read_checkpoint` (the `{"elements": [...]}` object). The output `.excalidraw` file goes in the project's image directory (see File Location Convention above).
**For batch exports**: Write each checkpoint to a separate raw JSON file, then convert each one:
```bash
node <skill-path>/scripts/convert.mjs raw1.json diagram1.excalidraw
node <skill-path>/scripts/convert.mjs raw2.json diagram2.excalidraw
```
**Manual alternative**: If you need to write the `.excalidraw` file by hand (e.g., without the convert script), each element needs these defaults:
```
angle: 0, roughness: 1, opacity: 100, groupIds: [], seed: <unique int>,
version: 1, versionNonce: <unique int>, isDeleted: false,
boundElements: null, link: null, locked: false
```
Text elements also need: `fontFamily: 1, textAlign: "left", verticalAlign: "top", baseline: 14, containerId: null, originalText: "<same as text>"`
Bound text (labels on shapes/arrows) needs: `containerId: "<parent-id>"`, `textAlign: "center"`, `verticalAlign: "middle"`, and the parent needs `boundElements: [{"id": "<text-id>", "type": "text"}]`.
### Step 5: Export to PNG
Run the export script. Determine the runtime path relative to this skill's scripts directory:
```bash
cd <skill-path>/scripts/.export-runtime && node <skill-path>/scripts/export_png.mjs docs/images/my-diagram.excalidraw docs/images/my-diagram.png
```
The script:
1. Starts a local HTTP server serving the `.excalidraw` file and an HTML page
2. Launches headless Chromium via Playwright
3. The HTML page loads the Excalidraw library from esm.sh (library code only, not user data)
4. Calls `exportToBlob` on the local diagram data
5. Extracts the base64 PNG and writes it to disk
6. Cleans up temp files and exits
The script prints the output path on success. Verify the result with `file <output.png>`.
### Step 5.5: Validate and Iterate
Run the validation script on the `.excalidraw` file to catch spatial issues:
```bash
node <skill-path>/scripts/validate.mjs docs/images/my-diagram.excalidraw
```
Then read the exported PNG back using the Read tool to visually inspect:
1. All label text fits within its container (no overflow/clipping)
2. No arrows cross over text labels
3. Spacing between elements is consistent
4. Legend and titles are properly positioned
If the validation script or visual inspection reveals issues:
1. Identify the specific elements that need adjustment
2. Edit the `.excalidraw` file (adjust coordinates, box sizes, or arrow waypoints)
3. Re-run the export script (Step 5)
4. Re-validate
### Step 6: Deliver the Result
Read the PNG file to display it to the user. Provide the file path so the user can access it directly.
## Troubleshooting
**Setup fails**: Verify Node.js v18+ is installed (`node --version`). Ensure npm has network access for the initial Playwright/Chromium download.
**Export times out**: The HTML page has a 30-second timeout. If it fails, check browser console output in the script's error messages. Common cause: esm.sh CDN is temporarily slow on first load.
**Blank PNG**: Ensure elements include all required properties (see Step 4 defaults). Missing `seed`, `version`, or `fontFamily` on text elements can cause silent render failures.
**"READY" never fires**: The `exportToBlob` call requires valid elements. Filter out `cameraUpdate` and other pseudo-elements before writing the `.excalidraw` file.

149
plugins/compound-engineering/skills/excalidraw-png-export/references/excalidraw-element-format.md Normal file
View File

@@ -0,0 +1,149 @@
# Excalidraw Element Format Reference
This reference documents the element JSON format accepted by the Excalidraw MCP `create_view` tool and the `export_png.mjs` script.
## Color Palette
### Primary Colors
| Name | Hex | Use |
|------|-----|-----|
| Blue | `#4a9eed` | Primary actions, links |
| Amber | `#f59e0b` | Warnings, highlights |
| Green | `#22c55e` | Success, positive |
| Red | `#ef4444` | Errors, negative |
| Purple | `#8b5cf6` | Accents, special |
| Pink | `#ec4899` | Decorative |
| Cyan | `#06b6d4` | Info, secondary |
### Fill Colors (pastel, for shape backgrounds)
| Color | Hex | Good For |
|-------|-----|----------|
| Light Blue | `#a5d8ff` | Input, sources, primary |
| Light Green | `#b2f2bb` | Success, output |
| Light Orange | `#ffd8a8` | Warning, pending |
| Light Purple | `#d0bfff` | Processing, middleware |
| Light Red | `#ffc9c9` | Error, critical |
| Light Yellow | `#fff3bf` | Notes, decisions |
| Light Teal | `#c3fae8` | Storage, data |
## Element Types
### Required Fields (all elements)
`type`, `id` (unique string), `x`, `y`, `width`, `height`
### Defaults (skip these)
strokeColor="#1e1e1e", backgroundColor="transparent", fillStyle="solid", strokeWidth=2, roughness=1, opacity=100
### Shapes
**Rectangle**: `{ "type": "rectangle", "id": "r1", "x": 100, "y": 100, "width": 200, "height": 100 }`
- `roundness: { type: 3 }` for rounded corners
- `backgroundColor: "#a5d8ff"`, `fillStyle: "solid"` for filled
**Ellipse**: `{ "type": "ellipse", "id": "e1", "x": 100, "y": 100, "width": 150, "height": 150 }`
**Diamond**: `{ "type": "diamond", "id": "d1", "x": 100, "y": 100, "width": 150, "height": 150 }`
### Labels
**Labeled shape (preferred)**: Add `label` to any shape for auto-centered text.
```json
{ "type": "rectangle", "id": "r1", "x": 100, "y": 100, "width": 200, "height": 80, "label": { "text": "Hello", "fontSize": 20 } }
```
**Standalone text** (titles, annotations only):
```json
{ "type": "text", "id": "t1", "x": 150, "y": 138, "text": "Hello", "fontSize": 20 }
```
### Arrows
```json
{ "type": "arrow", "id": "a1", "x": 300, "y": 150, "width": 200, "height": 0, "points": [[0,0],[200,0]], "endArrowhead": "arrow" }
```
**Bindings** connect arrows to shapes:
```json
"startBinding": { "elementId": "r1", "fixedPoint": [1, 0.5] }
```
fixedPoint: top=[0.5,0], bottom=[0.5,1], left=[0,0.5], right=[1,0.5]
**Labeled arrow**: `"label": { "text": "connects" }`
### Camera (MCP only, not exported to PNG)
```json
{ "type": "cameraUpdate", "width": 800, "height": 600, "x": 0, "y": 0 }
```
Camera sizes must be 4:3 ratio. The export script filters these out automatically.
## Sizing Rules
### Container-to-text ratios
- Box width >= estimated_text_width * 1.4 (40% horizontal margin)
- Box height >= estimated_text_height * 1.5 (50% vertical margin)
- Minimum box size: 150x60 for single-line labels, 200x80 for multi-line
### Font size constraints
- Labels inside containers: max fontSize 14
- Service/zone titles: fontSize 18-22
- Standalone annotations: fontSize 12-14
- Never exceed fontSize 16 inside a box smaller than 300px wide
### Padding
- Minimum 15px padding on each side between text and container edge
- For multi-line text, add 8px vertical padding per line beyond the first
### General
- Leave 20-30px gaps between elements
## Label Content Guidelines
### Keep labels short
- Maximum 2 lines per label inside shapes
- Maximum 25 characters per line
- If label needs 3+ lines, split: short name in box, details as annotation below
### Label patterns
- Service box: "Service Name" (1 line) or "Service Name\nBrief role" (2 lines)
- Component box: "Component Name" (1 line)
- Detail text: Use standalone text elements positioned below/beside the box
### Bad vs Good
BAD: label "Auth-MS\nOAuth tokens, credentials\n800-1K req/s, <100ms" (3 lines, 30+ chars)
GOOD: label "Auth-MS\nOAuth token management" (2 lines, 22 chars max)
+ standalone text below: "800-1K req/s, <100ms p99"
## Arrow Routing Rules
### Gutter-based routing
- Define horizontal and vertical gutters (20-30px gaps between service zones)
- Route arrows through gutters, never over content areas
- Use right-angle waypoints along zone edges
### Waypoint placement
- Start/end points: attach to box edges using fixedPoint bindings
- Mid-waypoints: offset 20px from nearest box edge
- For crossing traffic: stagger parallel arrows by 10px
### Vertical vs horizontal preference
- Prefer horizontal arrows for same-tier connections
- Prefer vertical arrows for cross-tier flows (consumer -> service -> external)
- Diagonal arrows only when routing around would add 3+ waypoints
### Label placement on arrows
- Arrow labels should sit in empty space, not over boxes
- For vertical arrows: place label to the left or right, offset 15px
- For horizontal arrows: place label above, offset 10px
## Example: Two Connected Boxes
```json
[
{ "type": "cameraUpdate", "width": 800, "height": 600, "x": 50, "y": 50 },
{ "type": "rectangle", "id": "b1", "x": 100, "y": 100, "width": 200, "height": 100, "roundness": { "type": 3 }, "backgroundColor": "#a5d8ff", "fillStyle": "solid", "label": { "text": "Start", "fontSize": 20 } },
{ "type": "rectangle", "id": "b2", "x": 450, "y": 100, "width": 200, "height": 100, "roundness": { "type": 3 }, "backgroundColor": "#b2f2bb", "fillStyle": "solid", "label": { "text": "End", "fontSize": 20 } },
{ "type": "arrow", "id": "a1", "x": 300, "y": 150, "width": 150, "height": 0, "points": [[0,0],[150,0]], "endArrowhead": "arrow", "startBinding": { "elementId": "b1", "fixedPoint": [1, 0.5] }, "endBinding": { "elementId": "b2", "fixedPoint": [0, 0.5] } }
]
```

2
plugins/compound-engineering/skills/excalidraw-png-export/scripts/.gitignore vendored Normal file
View File

@@ -0,0 +1,2 @@
.export-runtime/
.export-tmp/

178
plugins/compound-engineering/skills/excalidraw-png-export/scripts/convert.mjs Executable file
View File

@@ -0,0 +1,178 @@
#!/usr/bin/env node
/**
* Convert raw Excalidraw MCP checkpoint JSON into a valid .excalidraw file.
* Filters pseudo-elements, adds required defaults, expands labels into bound text.
*/
import { readFileSync, writeFileSync } from 'fs';
import { dirname, join } from 'path';
import { fileURLToPath } from 'url';
import { createRequire } from 'module';
const __dirname = dirname(fileURLToPath(import.meta.url));
const runtimeRequire = createRequire(join(__dirname, '.export-runtime', 'package.json'));
// Canvas-based text measurement with graceful fallback to heuristic.
// Excalidraw renders with Virgil (hand-drawn font); system sans-serif
// is a reasonable proxy. The 1.1x multiplier accounts for Virgil being wider.
let measureText;
try {
const canvas = runtimeRequire('canvas');
const { createCanvas } = canvas;
const cvs = createCanvas(1, 1);
const ctx = cvs.getContext('2d');
measureText = (text, fontSize) => {
ctx.font = `${fontSize}px sans-serif`;
const lines = text.split('\n');
const widths = lines.map(line => ctx.measureText(line).width * 1.1);
return {
width: Math.max(...widths),
height: lines.length * (fontSize * 1.25),
};
};
} catch {
console.warn('WARN: canvas not available, using heuristic text sizing (install canvas for accurate measurement)');
measureText = (text, fontSize) => {
const lines = text.split('\n');
return {
width: Math.max(...lines.map(l => l.length)) * fontSize * 0.55,
height: lines.length * (fontSize + 4),
};
};
}
const [,, inputFile, outputFile] = process.argv;
if (!inputFile || !outputFile) {
console.error('Usage: node convert.mjs <input.json> <output.excalidraw>');
process.exit(1);
}
const raw = JSON.parse(readFileSync(inputFile, 'utf8'));
const elements = raw.elements || raw;
let seed = 1000;
const nextSeed = () => seed++;
const processed = [];
for (const el of elements) {
if (['cameraUpdate', 'delete', 'restoreCheckpoint'].includes(el.type)) continue;
const base = {
angle: 0,
roughness: 1,
opacity: el.opacity ?? 100,
groupIds: [],
seed: nextSeed(),
version: 1,
versionNonce: nextSeed(),
isDeleted: false,
boundElements: null,
link: null,
locked: false,
strokeColor: el.strokeColor || '#1e1e1e',
backgroundColor: el.backgroundColor || 'transparent',
fillStyle: el.fillStyle || 'solid',
strokeWidth: el.strokeWidth ?? 2,
strokeStyle: el.strokeStyle || 'solid',
};
if (el.type === 'text') {
const fontSize = el.fontSize || 16;
const measured = measureText(el.text, fontSize);
processed.push({
...base,
type: 'text',
id: el.id,
x: el.x,
y: el.y,
width: measured.width,
height: measured.height,
text: el.text,
fontSize, fontFamily: 1,
textAlign: 'left',
verticalAlign: 'top',
baseline: fontSize,
containerId: null,
originalText: el.text,
});
} else if (el.type === 'arrow') {
const arrowEl = {
...base,
type: 'arrow',
id: el.id,
x: el.x,
y: el.y,
width: el.width || 0,
height: el.height || 0,
points: el.points || [[0, 0]],
startArrowhead: el.startArrowhead || null,
endArrowhead: el.endArrowhead ?? 'arrow',
startBinding: el.startBinding ? { ...el.startBinding, focus: 0, gap: 5 } : null,
endBinding: el.endBinding ? { ...el.endBinding, focus: 0, gap: 5 } : null,
roundness: { type: 2 },
boundElements: [],
};
processed.push(arrowEl);
if (el.label) {
const labelId = el.id + '_label';
const text = el.label.text || '';
const fontSize = el.label.fontSize || 14;
const { width: w, height: h } = measureText(text, fontSize);
const midPt = el.points[Math.floor(el.points.length / 2)] || [0, 0];
processed.push({
...base,
type: 'text', id: labelId,
x: el.x + midPt[0] - w / 2,
y: el.y + midPt[1] - h / 2 - 12,
width: w, height: h,
text, fontSize, fontFamily: 1,
textAlign: 'center', verticalAlign: 'middle',
baseline: fontSize, containerId: el.id, originalText: text,
strokeColor: el.strokeColor || '#1e1e1e',
backgroundColor: 'transparent',
});
arrowEl.boundElements = [{ id: labelId, type: 'text' }];
}
} else if (['rectangle', 'ellipse', 'diamond'].includes(el.type)) {
const shapeEl = {
...base,
type: el.type, id: el.id,
x: el.x, y: el.y, width: el.width, height: el.height,
roundness: el.roundness || null,
boundElements: [],
};
processed.push(shapeEl);
if (el.label) {
const labelId = el.id + '_label';
const text = el.label.text || '';
const fontSize = el.label.fontSize || 16;
const { width: w, height: h } = measureText(text, fontSize);
processed.push({
...base,
type: 'text', id: labelId,
x: el.x + (el.width - w) / 2,
y: el.y + (el.height - h) / 2,
width: w, height: h,
text, fontSize, fontFamily: 1,
textAlign: 'center', verticalAlign: 'middle',
baseline: fontSize, containerId: el.id, originalText: text,
strokeColor: el.strokeColor || '#1e1e1e',
backgroundColor: 'transparent',
});
shapeEl.boundElements = [{ id: labelId, type: 'text' }];
}
}
}
writeFileSync(outputFile, JSON.stringify({
type: 'excalidraw', version: 2, source: 'claude-code',
elements: processed,
appState: { exportBackground: true, viewBackgroundColor: '#ffffff' },
files: {},
}, null, 2));
console.log(`Wrote ${processed.length} elements to ${outputFile}`);

61
plugins/compound-engineering/skills/excalidraw-png-export/scripts/export.html Normal file
View File

@@ -0,0 +1,61 @@
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<style>
body { margin: 0; background: white; }
#root { width: 900px; height: 400px; }
</style>
<script>
window.EXCALIDRAW_ASSET_PATH = "https://esm.sh/@excalidraw/excalidraw/dist/prod/";
</script>
</head>
<body>
<div id="root"></div>
<script type="importmap">
{
"imports": {
"react": "https://esm.sh/react@18",
"react-dom": "https://esm.sh/react-dom@18",
"react-dom/client": "https://esm.sh/react-dom@18/client",
"react/jsx-runtime": "https://esm.sh/react@18/jsx-runtime",
"@excalidraw/excalidraw": "https://esm.sh/@excalidraw/excalidraw@0.18.0?external=react,react-dom"
}
}
</script>
<script type="module">
import { exportToBlob } from "@excalidraw/excalidraw";
async function run() {
const resp = await fetch("./diagram.excalidraw");
const data = await resp.json();
const validTypes = ["rectangle","ellipse","diamond","text","arrow","line","freedraw","image","frame"];
const elements = data.elements.filter(el => validTypes.includes(el.type));
const blob = await exportToBlob({
elements,
appState: {
exportBackground: true,
viewBackgroundColor: data.appState?.viewBackgroundColor || "#ffffff",
exportWithDarkMode: data.appState?.exportWithDarkMode || false,
},
files: data.files || {},
getDimensions: (w, h) => ({ width: w * 2, height: h * 2, scale: 2 }),
});
const reader = new FileReader();
reader.onload = () => {
window.__PNG_DATA__ = reader.result;
document.title = "READY";
};
reader.readAsDataURL(blob);
}
run().catch(e => {
console.error("EXPORT ERROR:", e);
document.title = "ERROR:" + e.message;
});
</script>
</body>
</html>

90
plugins/compound-engineering/skills/excalidraw-png-export/scripts/export_png.mjs Executable file
View File

@@ -0,0 +1,90 @@
#!/usr/bin/env node
/**
* Export an Excalidraw JSON file to PNG using Playwright + the official Excalidraw library.
*
* Usage: node export_png.mjs <input.excalidraw> [output.png]
*
* All rendering happens locally. Diagram data never leaves the machine.
* The Excalidraw JS library is fetched from esm.sh CDN (code only, not user data).
*/
import { createRequire } from "module";
import { readFileSync, writeFileSync, copyFileSync } from "fs";
import { createServer } from "http";
import { join, extname, dirname } from "path";
import { fileURLToPath } from "url";
const __dirname = dirname(fileURLToPath(import.meta.url));
const RUNTIME_DIR = join(__dirname, ".export-runtime");
const HTML_PATH = join(__dirname, "export.html");
// Resolve playwright from the runtime directory, not the script's location
const require = createRequire(join(RUNTIME_DIR, "node_modules", "playwright", "index.mjs"));
const { chromium } = await import(join(RUNTIME_DIR, "node_modules", "playwright", "index.mjs"));
const inputPath = process.argv[2];
if (!inputPath) {
console.error("Usage: node export_png.mjs <input.excalidraw> [output.png]");
process.exit(1);
}
const outputPath = process.argv[3] || inputPath.replace(/\.excalidraw$/, ".png");
// Set up a temp serving directory
const SERVE_DIR = join(__dirname, ".export-tmp");
const { mkdirSync, rmSync } = await import("fs");
mkdirSync(SERVE_DIR, { recursive: true });
copyFileSync(HTML_PATH, join(SERVE_DIR, "export.html"));
copyFileSync(inputPath, join(SERVE_DIR, "diagram.excalidraw"));
const MIME = {
".html": "text/html",
".json": "application/json",
".excalidraw": "application/json",
};
const server = createServer((req, res) => {
const file = join(SERVE_DIR, req.url === "/" ? "export.html" : req.url);
try {
const data = readFileSync(file);
res.writeHead(200, { "Content-Type": MIME[extname(file)] || "application/octet-stream" });
res.end(data);
} catch {
res.writeHead(404);
res.end("Not found");
}
});
server.listen(0, "127.0.0.1", async () => {
const port = server.address().port;
let browser;
try {
browser = await chromium.launch({ headless: true });
const page = await browser.newPage();
page.on("pageerror", err => console.error("Page error:", err.message));
await page.goto(`http://127.0.0.1:${port}`);
await page.waitForFunction(
() => document.title.startsWith("READY") || document.title.startsWith("ERROR"),
{ timeout: 30000 }
);
const title = await page.title();
if (title.startsWith("ERROR")) {
console.error("Export failed:", title);
process.exit(1);
}
const dataUrl = await page.evaluate(() => window.__PNG_DATA__);
const base64 = dataUrl.replace(/^data:image\/png;base64,/, "");
writeFileSync(outputPath, Buffer.from(base64, "base64"));
console.log(outputPath);
} finally {
if (browser) await browser.close();
server.close();
rmSync(SERVE_DIR, { recursive: true, force: true });
}
});

37
plugins/compound-engineering/skills/excalidraw-png-export/scripts/setup.sh Executable file
View File

@@ -0,0 +1,37 @@
#!/bin/bash
# First-time setup for excalidraw-png-export skill.
# Installs playwright and chromium headless into a dedicated directory.
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
EXPORT_DIR="$SCRIPT_DIR/.export-runtime"
if [ -d "$EXPORT_DIR/node_modules/playwright" ]; then
echo "Runtime already installed at $EXPORT_DIR"
exit 0
fi
echo "Installing excalidraw-png-export runtime..."
mkdir -p "$EXPORT_DIR"
cd "$EXPORT_DIR"
# Initialize package.json with ESM support
cat > package.json << 'PACKAGEEOF'
{
"name": "excalidraw-export-runtime",
"version": "1.0.0",
"type": "module",
"private": true
}
PACKAGEEOF
npm install playwright 2>&1
npx playwright install chromium 2>&1
# canvas provides accurate text measurement for convert.mjs.
# Requires Cairo native library: brew install pkg-config cairo pango libpng jpeg giflib librsvg
# Falls back to heuristic sizing if unavailable.
npm install canvas 2>&1 || echo "WARN: canvas install failed (missing Cairo?). Heuristic text sizing will be used."
echo "Setup complete. Runtime installed at $EXPORT_DIR"

173
plugins/compound-engineering/skills/excalidraw-png-export/scripts/validate.mjs Executable file
View File

@@ -0,0 +1,173 @@
#!/usr/bin/env node
/**
* Spatial validation for .excalidraw files.
* Checks text overflow, arrow-text collisions, and element overlap.
* Usage: node validate.mjs <input.excalidraw>
*/
import { readFileSync } from 'fs';
const MIN_PADDING = 15;
const inputFile = process.argv[2];
if (!inputFile) {
console.error('Usage: node validate.mjs <input.excalidraw>');
process.exit(1);
}
const data = JSON.parse(readFileSync(inputFile, 'utf8'));
const elements = data.elements || data;
// Build element map
const elMap = new Map();
for (const el of elements) {
if (el.isDeleted) continue;
elMap.set(el.id, el);
}
let warnings = 0;
let errors = 0;
const checked = elements.filter(el => !el.isDeleted).length;
// --- Check 1: Text overflow within containers ---
// Skip arrow-bound labels — arrows are lines, not spatial containers.
for (const el of elements) {
if (el.isDeleted || el.type !== 'text' || !el.containerId) continue;
const parent = elMap.get(el.containerId);
if (!parent || parent.type === 'arrow') continue;
const textRight = el.x + el.width;
const textBottom = el.y + el.height;
const parentRight = parent.x + parent.width;
const parentBottom = parent.y + parent.height;
const paddingLeft = el.x - parent.x;
const paddingRight = parentRight - textRight;
const paddingTop = el.y - parent.y;
const paddingBottom = parentBottom - textBottom;
const overflows = [];
if (paddingLeft < MIN_PADDING) overflows.push(`left=${paddingLeft.toFixed(1)}px (need ${MIN_PADDING}px)`);
if (paddingRight < MIN_PADDING) overflows.push(`right=${paddingRight.toFixed(1)}px (need ${MIN_PADDING}px)`);
if (paddingTop < MIN_PADDING) overflows.push(`top=${paddingTop.toFixed(1)}px (need ${MIN_PADDING}px)`);
if (paddingBottom < MIN_PADDING) overflows.push(`bottom=${paddingBottom.toFixed(1)}px (need ${MIN_PADDING}px)`);
if (overflows.length > 0) {
const label = (el.text || '').replace(/\n/g, '\\n');
const truncated = label.length > 40 ? label.slice(0, 37) + '...' : label;
console.log(`WARN: text "${truncated}" (id=${el.id}) tight/overflow in container (id=${el.containerId})`);
console.log(` text_bbox=[${el.x.toFixed(0)},${el.y.toFixed(0)}]->[${textRight.toFixed(0)},${textBottom.toFixed(0)}]`);
console.log(` container_bbox=[${parent.x.toFixed(0)},${parent.y.toFixed(0)}]->[${parentRight.toFixed(0)},${parentBottom.toFixed(0)}]`);
console.log(` insufficient padding: ${overflows.join(', ')}`);
console.log();
warnings++;
}
}
// --- Check 2: Arrow-text collisions ---
/** Check if line segment (p1->p2) intersects axis-aligned rectangle. */
function segmentIntersectsRect(p1, p2, rect) {
// rect = {x, y, w, h} -> min/max
const rxMin = rect.x;
const rxMax = rect.x + rect.w;
const ryMin = rect.y;
const ryMax = rect.y + rect.h;
// Cohen-Sutherland-style clipping
let [x1, y1] = [p1[0], p1[1]];
let [x2, y2] = [p2[0], p2[1]];
function outcode(x, y) {
let code = 0;
if (x < rxMin) code |= 1;
else if (x > rxMax) code |= 2;
if (y < ryMin) code |= 4;
else if (y > ryMax) code |= 8;
return code;
}
let code1 = outcode(x1, y1);
let code2 = outcode(x2, y2);
for (let i = 0; i < 20; i++) {
if (!(code1 | code2)) return true; // both inside
if (code1 & code2) return false; // both outside same side
const codeOut = code1 || code2;
let x, y;
if (codeOut & 8) { y = ryMax; x = x1 + (x2 - x1) * (ryMax - y1) / (y2 - y1); }
else if (codeOut & 4) { y = ryMin; x = x1 + (x2 - x1) * (ryMin - y1) / (y2 - y1); }
else if (codeOut & 2) { x = rxMax; y = y1 + (y2 - y1) * (rxMax - x1) / (x2 - x1); }
else { x = rxMin; y = y1 + (y2 - y1) * (rxMin - x1) / (x2 - x1); }
if (codeOut === code1) { x1 = x; y1 = y; code1 = outcode(x1, y1); }
else { x2 = x; y2 = y; code2 = outcode(x2, y2); }
}
return false;
}
// Collect text bounding boxes (excluding arrow-bound labels for their own arrow)
const textBoxes = [];
for (const el of elements) {
if (el.isDeleted || el.type !== 'text') continue;
textBoxes.push({
id: el.id,
containerId: el.containerId,
text: (el.text || '').replace(/\n/g, '\\n'),
rect: { x: el.x, y: el.y, w: el.width, h: el.height },
});
}
for (const el of elements) {
if (el.isDeleted || el.type !== 'arrow') continue;
if (!el.points || el.points.length < 2) continue;
// Compute absolute points
const absPoints = el.points.map(p => [el.x + p[0], el.y + p[1]]);
for (const tb of textBoxes) {
// Skip this arrow's own label
if (tb.containerId === el.id) continue;
for (let i = 0; i < absPoints.length - 1; i++) {
if (segmentIntersectsRect(absPoints[i], absPoints[i + 1], tb.rect)) {
const truncated = tb.text.length > 30 ? tb.text.slice(0, 27) + '...' : tb.text;
const seg = `[${absPoints[i].map(n => n.toFixed(0)).join(',')}]->[${absPoints[i + 1].map(n => n.toFixed(0)).join(',')}]`;
console.log(`WARN: arrow (id=${el.id}) segment ${seg} crosses text "${truncated}" (id=${tb.id})`);
console.log(` text_bbox=[${tb.rect.x.toFixed(0)},${tb.rect.y.toFixed(0)}]->[${(tb.rect.x + tb.rect.w).toFixed(0)},${(tb.rect.y + tb.rect.h).toFixed(0)}]`);
console.log();
warnings++;
break; // one warning per arrow-text pair
}
}
}
}
// --- Check 3: Element overlap (non-child, same depth) ---
const topLevel = elements.filter(el =>
!el.isDeleted && !el.containerId && el.type !== 'text' && el.type !== 'arrow'
);
for (let i = 0; i < topLevel.length; i++) {
for (let j = i + 1; j < topLevel.length; j++) {
const a = topLevel[i];
const b = topLevel[j];
const aRight = a.x + a.width;
const aBottom = a.y + a.height;
const bRight = b.x + b.width;
const bBottom = b.y + b.height;
if (a.x < bRight && aRight > b.x && a.y < bBottom && aBottom > b.y) {
const overlapX = Math.min(aRight, bRight) - Math.max(a.x, b.x);
const overlapY = Math.min(aBottom, bBottom) - Math.max(a.y, b.y);
console.log(`WARN: overlap between (id=${a.id}) and (id=${b.id}): ${overlapX.toFixed(0)}x${overlapY.toFixed(0)}px`);
console.log();
warnings++;
}
}
}
// --- Summary ---
console.log(`OK: ${checked} elements checked, ${warnings} warning(s), ${errors} error(s)`);
process.exit(warnings > 0 ? 1 : 0);

221
plugins/compound-engineering/skills/fastapi-style/SKILL.md Normal file
View File

@@ -0,0 +1,221 @@
---
name: fastapi-style
description: This skill should be used when writing Python and FastAPI code following opinionated best practices. It applies when building APIs, creating Pydantic models, working with SQLAlchemy, or any FastAPI application. Triggers on FastAPI code generation, API design, refactoring requests, code review, or when discussing async Python patterns. Embodies thin routers, rich Pydantic models, dependency injection, async-first design, and the "explicit is better than implicit" philosophy.
---
<objective>
Apply opinionated FastAPI conventions to Python API code. This skill provides comprehensive domain expertise for building maintainable, performant FastAPI applications following established patterns from production codebases.
</objective>
<essential_principles>
## Core Philosophy
"Explicit is better than implicit. Simple is better than complex."
**The FastAPI Way:**
- Thin routers, rich Pydantic models with validation
- Dependency injection for everything
- Async-first with SQLAlchemy 2.0
- Type hints everywhere - let the tools help you
- Settings via pydantic-settings, not raw env vars
- Database-backed solutions where possible
**What to deliberately avoid:**
- Flask patterns (global request context)
- Django ORM in FastAPI (use SQLAlchemy 2.0)
- Synchronous database calls (use async)
- Manual JSON serialization (Pydantic handles it)
- Global state (use dependency injection)
- `*` imports (explicit imports only)
- Circular imports (proper module structure)
**Development Philosophy:**
- Type everything - mypy should pass
- Fail fast with descriptive errors
- Write-time validation over read-time checks
- Database constraints complement Pydantic validation
- Tests are documentation
</essential_principles>
<intake>
What are you working on?
1. **Routers** - Route organization, dependency injection, response models
2. **Models** - Pydantic schemas, SQLAlchemy models, validation patterns
3. **Database** - SQLAlchemy 2.0 async, Alembic migrations, transactions
4. **Testing** - pytest, httpx TestClient, fixtures, async testing
5. **Security** - OAuth2, JWT, permissions, CORS, rate limiting
6. **Background Tasks** - Celery, ARQ, or FastAPI BackgroundTasks
7. **Code Review** - Review code against FastAPI best practices
8. **General Guidance** - Philosophy and conventions
**Specify a number or describe your task.**
</intake>
<routing>
| Response | Reference to Read |
|----------|-------------------|
| 1, router, route, endpoint | [routers.md](./references/routers.md) |
| 2, model, pydantic, schema, sqlalchemy | [models.md](./references/models.md) |
| 3, database, db, alembic, migration, transaction | [database.md](./references/database.md) |
| 4, test, testing, pytest, fixture | [testing.md](./references/testing.md) |
| 5, security, auth, oauth, jwt, permission | [security.md](./references/security.md) |
| 6, background, task, celery, arq, queue | [background_tasks.md](./references/background_tasks.md) |
| 7, review | Read all references, then review code |
| 8, general task | Read relevant references based on context |
**After reading relevant references, apply patterns to the user's code.**
</routing>
<quick_reference>
## Project Structure
```
app/
├── main.py # FastAPI app creation, middleware
├── config.py # Settings via pydantic-settings
├── dependencies.py # Shared dependencies
├── database.py # Database session, engine
├── models/ # SQLAlchemy models
│ ├── __init__.py
│ ├── base.py # Base model class
│ └── user.py
├── schemas/ # Pydantic models
│ ├── __init__.py
│ └── user.py
├── routers/ # API routers
│ ├── __init__.py
│ └── users.py
├── services/ # Business logic (if needed)
├── utils/ # Shared utilities
└── tests/
├── conftest.py # Fixtures
└── test_users.py
```
## Naming Conventions
**Pydantic Schemas:**
- `UserCreate` - input for creation
- `UserUpdate` - input for updates (all fields Optional)
- `UserRead` - output representation
- `UserInDB` - internal with hashed password
**SQLAlchemy Models:** Singular nouns (`User`, `Item`, `Order`)
**Routers:** Plural resource names (`users.py`, `items.py`)
**Dependencies:** Verb phrases (`get_current_user`, `get_db_session`)
## Type Hints
```python
# Always type function signatures
async def get_user(
user_id: int,
db: AsyncSession = Depends(get_db),
) -> User:
...
# Use Annotated for dependency injection
from typing import Annotated
CurrentUser = Annotated[User, Depends(get_current_user)]
DBSession = Annotated[AsyncSession, Depends(get_db)]
```
## Response Patterns
```python
# Explicit response_model
@router.get("/users/{user_id}", response_model=UserRead)
async def get_user(user_id: int, db: DBSession) -> User:
...
# Status codes
@router.post("/users", status_code=status.HTTP_201_CREATED)
async def create_user(...) -> UserRead:
...
# Multiple response types
@router.get("/users/{user_id}", responses={404: {"model": ErrorResponse}})
```
## Error Handling
```python
from fastapi import HTTPException, status
# Specific exceptions
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="User not found",
)
# Custom exception handlers
@app.exception_handler(ValidationError)
async def validation_exception_handler(request, exc):
return JSONResponse(status_code=422, content={"detail": exc.errors()})
```
## Dependency Injection
```python
# Simple dependency
async def get_db() -> AsyncGenerator[AsyncSession, None]:
async with async_session() as session:
yield session
# Parameterized dependency
def get_pagination(
skip: int = Query(0, ge=0),
limit: int = Query(100, ge=1, le=1000),
) -> dict:
return {"skip": skip, "limit": limit}
# Class-based dependency
class CommonQueryParams:
def __init__(self, q: str | None = None, skip: int = 0, limit: int = 100):
self.q = q
self.skip = skip
self.limit = limit
```
</quick_reference>
<reference_index>
## Domain Knowledge
All detailed patterns in `references/`:
| File | Topics |
|------|--------|
| [routers.md](./references/routers.md) | Route organization, dependency injection, response models, middleware, versioning |
| [models.md](./references/models.md) | Pydantic schemas, SQLAlchemy models, validation, serialization, mixins |
| [database.md](./references/database.md) | SQLAlchemy 2.0 async, Alembic migrations, transactions, connection pooling |
| [testing.md](./references/testing.md) | pytest, httpx TestClient, fixtures, async testing, mocking patterns |
| [security.md](./references/security.md) | OAuth2, JWT, permissions, CORS, rate limiting, secrets management |
| [background_tasks.md](./references/background_tasks.md) | FastAPI BackgroundTasks, Celery, ARQ, task patterns |
</reference_index>
<success_criteria>
Code follows FastAPI best practices when:
- Routers are thin, focused on HTTP concerns only
- Pydantic models handle all validation and serialization
- SQLAlchemy 2.0 async patterns used correctly
- Dependencies injected, not imported as globals
- Type hints on all function signatures
- Settings via pydantic-settings
- Tests use pytest with async support
- Error handling is explicit and informative
- Security follows OAuth2/JWT standards
- Background tasks use appropriate tool for the job
</success_criteria>
<credits>
Based on FastAPI best practices from the official documentation, real-world production patterns, and the Python community's collective wisdom.
**Key Resources:**
- [FastAPI Documentation](https://fastapi.tiangolo.com/)
- [SQLAlchemy 2.0 Documentation](https://docs.sqlalchemy.org/)
- [Pydantic V2 Documentation](https://docs.pydantic.dev/)
</credits>

84
plugins/compound-engineering/skills/jira-ticket-writer/SKILL.md Normal file
View File

@@ -0,0 +1,84 @@
---
name: jira-ticket-writer
description: This skill should be used when the user wants to create a Jira ticket. It guides drafting, pressure-testing for tone and AI-isms, and getting user approval before creating the ticket via the Atlassian MCP. Triggers on "create a ticket", "write a Jira ticket", "file a ticket", "make a Jira issue", or any request to create work items in Jira.
---
# Jira Ticket Writer
Write Jira tickets that sound like a human wrote them. Drafts go through tone review before the user sees them, and nothing gets created without explicit approval.
## Reference
For tickets pertaining to Talent Engine (Agentic App), TalentOS, Comparably, or the ATS Platform: Use the `ZAS` Jira project
When creating epics and tickets for Talent Engine always add the label `talent-engine` and prefix the name with "[Agentic App]"
When creating epics and tickets for the ATS Platform always add the label `ats-platform` and prefix the name with "[ATS Platform]"
## Workflow
### Phase 1: Validate Scope
Before drafting anything, confirm two things:
1. **What the ticket is about.** Gather the ticket contents from the conversation or the user's description. If the scope is unclear or too broad for a single ticket, ask the user to clarify before proceeding.
2. **Where it goes.** Determine the Jira project key and optional parent (epic). If the user provides a Jira URL or issue key, extract the project from it. If not specified, ask.
To look up the Jira project and validate the epic exists, use the Atlassian MCP tools:
- `mcp__atlassian__getAccessibleAtlassianResources` to get the cloudId
- `mcp__atlassian__getJiraIssue` to verify the parent epic exists and get its project key
Do not proceed to drafting until both the content scope and destination are clear.
### Phase 2: Draft
Write the ticket body in markdown. Follow these guidelines:
- **Summary line:** Under 80 characters. Imperative mood. No Jira-speak ("As a user, I want...").
- **Body structure:** Use whatever sections make sense for the ticket. Common patterns:
- "What's happening" / "What we need" / "Context" / "Done when"
- "Problem" / "Ask" / "Context"
- Just a clear description with acceptance criteria at the end
- **Code snippets:** Include relevant config, commands, or file references when they help the reader understand the current state and desired state.
- **Keep it specific:** Include file paths, line numbers, env names, config values. Vague tickets get deprioritized.
- **"Done when" over "Acceptance Criteria":** Use casual language for completion criteria. 2-4 items max.
### Phase 3: Pressure Test
Before showing the draft to the user, self-review against the tone guide.
Read `references/tone-guide.md` and apply every check to the draft. Specifically:
1. **Patronizing scan:** Read each sentence imagining you are the recipient, a specialist in their domain. Flag and rewrite anything that explains their own expertise back to them, tells them how to implement something in their own system, or preemptively argues against approaches they haven't proposed.
2. **AI-ism removal:** Hunt for em-dash overuse, bullet-point-everything formatting, rigid generated-feeling structure, spec-writing voice, and filler words (Additionally, Furthermore, Moreover, facilitates, leverages, streamlines, ensures).
3. **Human voice pass:** Read the whole thing as if reading it aloud. Does it sound like something a developer would type? Add moments of humility where appropriate ("you'd know better", "if we're missing something", "happy to chat").
4. **Kindness pass:** The reader is a human doing their job. Frame requests as requests. Acknowledge their expertise. Don't be demanding.
Revise the draft based on this review. Do not show the user the pre-review version.
### Phase 4: User Approval
Present the final draft to the user in chat. Include:
- The proposed **summary** (ticket title)
- The proposed **body** (formatted as it will appear)
- The **destination** (project key, parent epic if any, issue type)
Ask for sign-off using AskUserQuestion with three options:
- **Create it** — proceed to Phase 5
- **Changes needed** — user provides feedback, return to Phase 2 with their notes and loop until approved
- **Cancel** — stop without creating anything
### Phase 5: Create
Once approved, create the ticket:
1. Use `mcp__atlassian__getAccessibleAtlassianResources` to get the cloudId (if not already cached from Phase 1)
2. Use `mcp__atlassian__createJiraIssue` with:
- `cloudId`: from step 1
- `projectKey`: from Phase 1
- `issueTypeName`: "Task" unless the user specified otherwise
- `summary`: the approved title
- `description`: the approved body
- `parent`: the epic key if one was specified
3. Return the created ticket URL to the user: `https://discoverorg.atlassian.net/browse/<KEY>`

34
plugins/compound-engineering/skills/jira-ticket-writer/references/api_reference.md Normal file
View File

@@ -0,0 +1,34 @@
# Reference Documentation for Jira Ticket Writer
This is a placeholder for detailed reference documentation.
Replace with actual reference content or delete if not needed.
Example real reference docs from other skills:
- product-management/references/communication.md - Comprehensive guide for status updates
- product-management/references/context_building.md - Deep-dive on gathering context
- bigquery/references/ - API references and query examples
## When Reference Docs Are Useful
Reference docs are ideal for:
- Comprehensive API documentation
- Detailed workflow guides
- Complex multi-step processes
- Information too lengthy for main SKILL.md
- Content that's only needed for specific use cases
## Structure Suggestions
### API Reference Example
- Overview
- Authentication
- Endpoints with examples
- Error codes
- Rate limits
### Workflow Guide Example
- Prerequisites
- Step-by-step instructions
- Common patterns
- Troubleshooting
- Best practices

53
plugins/compound-engineering/skills/jira-ticket-writer/references/tone-guide.md Normal file
View File

@@ -0,0 +1,53 @@
# Tone Guide for Ticket Writing
## Core Principle
A human will read this ticket. Write like a teammate asking for help, not an AI generating a spec.
## Pressure Test Checklist
Review every sentence against these questions:
### 1. Patronizing language
- Does any sentence explain the reader's own domain back to them?
- Would you say this to a senior engineer's face without feeling awkward?
- Are you telling them HOW to implement something in their own system?
- Are you preemptively arguing against approaches they haven't proposed?
**Examples of patronizing language:**
- "This is a common pattern in Kubernetes deployments" (they know)
- "Helm charts support templating via {{ .Values }}" (they wrote the chart)
- "Why X, not Y" sections that dismiss alternatives before anyone suggested them
### 2. AI-isms to remove
- Em dashes used more than once per paragraph
- Every thought is a bullet point instead of a sentence
- Rigid structure that feels generated (Ask -> Why -> Context -> AC)
- Spec-writing voice: "When absent or false, existing behavior is preserved"
- Overuse of "ensures", "leverages", "facilitates", "streamlines"
- Unnecessary hedging: "It should be noted that..."
- Filler transitions: "Additionally", "Furthermore", "Moreover"
- Lists where prose would be more natural
### 3. Human voice check
- Does it sound like something you'd type in Slack, cleaned up slightly?
- Are there moments of humility? ("you'd know better than us", "if we're missing something")
- Is the tone collaborative rather than directive?
- Would you feel comfortable putting your name on this?
### 4. Kindness check
- Frame requests as requests, not demands
- Acknowledge the reader's expertise
- Offer context without over-explaining
- "Happy to chat more" > "Please advise"
## What to keep
- Technical detail and specifics (the reader needs these)
- Code snippets showing current state and desired state
- File references with line numbers
- Clear "done when" criteria (but keep them minimal)

26
plugins/compound-engineering/skills/john-voice/SKILL.md Normal file
View File

@@ -0,0 +1,26 @@
---
name: john-voice
description: "This skill should be used whenever writing content that should sound like John Lamb wrote it. It applies to all written output including Slack messages, emails, Jira tickets, technical docs, prose, blog posts, cover letters, and any other communication. This skill provides John's authentic writing voice, tone, and style patterns organized by venue and audience. Other skills should invoke this skill when producing written content on John's behalf. Triggers on any content generation, drafting, or editing task where the output represents John's voice."
allowed-tools: Read
---
# John's Writing Voice
This skill captures John Lamb's authentic writing voice for use across all written content. It is a reference skill designed to be called by other skills or used directly whenever producing text that should sound like John wrote it.
## How to Use This Skill
1. Determine the venue and audience for the content being produced
2. Load `references/core-voice.md` — this always applies regardless of context
3. Load the appropriate venue-specific tone guide from `references/`:
- **Prose, essays, blog posts** → `references/prose-essays.md`
- **Slack messages, quick emails, casual comms** → `references/casual-messages.md`
- **Technical docs, Jira tickets, PRs, code reviews** → `references/professional-technical.md`
- **Cover letters, LinkedIn, formal professional** → `references/formal-professional.md`
- **Personal reflection, journal, notes** → `references/personal-reflection.md`
4. Apply both the core voice and the venue-specific guide when drafting content
5. Review the output against the core voice principles — if it sounds like an AI wrote it, rewrite it
## Key Principle
John prizes simplicity and clarity above all else. He writes to convey meaning, not to sound smart. If the output uses words John wouldn't say aloud to a friend, it's wrong. If it obscures meaning behind fancy language, it's wrong. If it sounds like a corporate press release or a ChatGPT default (NO emdashes!), it's catastrophically wrong.

69
plugins/compound-engineering/skills/john-voice/references/casual-messages.md Normal file
View File

@@ -0,0 +1,69 @@
# Casual Messages Tone Guide
Use this guide for Slack messages, quick emails, texts, Discord, and other informal communications.
## General Tone
John's casual writing is his natural voice with the polish stripped off. Lowercase is fine. Fragments are fine. He thinks out loud and lets the reader follow along.
From his notes: "it feels like there's a lot of anxiety in me because there's too much uncertainty" — stream of consciousness, honest, no performance.
## Sentence Patterns
- Short fragments: "turns out, not really."
- Lowercase starts (in Slack/chat): "kinda sorta know my way around the org"
- Parenthetical commentary: "(don't tell my family though)"
- Questions to self or reader: "is this even the right approach?"
- Trailing thoughts: "but I'm not totally sure about that yet"
## Vocabulary in Casual Mode
John's casual register drops even further toward spoken language:
- "kinda", "gonna", "wanna" (occasionally)
- "TBH", "FYI" (in work Slack)
- "the thing is..." as a thought starter
- "I think..." / "I wonder if..." for tentative ideas
- "honestly" / "to be honest" as a signal he's about to be direct
## Email Patterns
**Short emails (most of them):**
John gets to the point fast. He doesn't pad emails with pleasantries beyond a brief greeting. He tends toward 2-4 sentences for most emails.
Structure:
1. One line of context or greeting
2. The ask or the information
3. Maybe a follow-up detail
4. Sign-off
**Never do:**
- "I hope this email finds you well"
- "Per my last email"
- "Please don't hesitate to reach out"
- "Best regards" (too stiff — "thanks" or "cheers" or just his name)
## Slack Patterns
John's Slack messages are conversational and direct. He:
- Skips greetings in channels (just says the thing)
- Uses threads appropriately
- Drops casual asides and humor
- Asks questions directly without preamble
- Uses emoji reactions more than emoji in text
Example Slack style:
"hey, quick question — are we using the existing search API or building a new one for this? I was looking at the federated search setup and I think we might be able to reuse most of it"
Not:
"Hi team! I wanted to reach out regarding the search API implementation. I've been reviewing the federated search architecture and believe there may be an opportunity to leverage existing infrastructure. Thoughts?"
## Feedback and Opinions
When giving opinions in casual contexts, John is direct but not blunt. He leads with his honest take and explains why.
Pattern: "[honest assessment] + [reasoning]"
- "I think we're overthinking this. The simpler version would cover 90% of the cases."
- "that approach makes me a bit nervous because [reason]"
- "I like the direction but [specific concern]"
He doesn't soften feedback with excessive qualifiers or sandwich it between compliments.

150
plugins/compound-engineering/skills/john-voice/references/core-voice.md Normal file
View File

@@ -0,0 +1,150 @@
# John Lamb — Core Voice
These patterns apply to ALL writing regardless of venue or audience. They are the non-negotiable foundation of John's voice.
## Philosophy
John writes to be understood, not to impress. He believes complexity in writing is a failure of the writer, not a sign of intelligence. He actively resists language that props up ego or obscures meaning. He'd rather sound like a person talking at a dinner table than a thought leader publishing a manifesto.
From his own notes: "Good communication does not correlate with intelligence and effective communication doesn't need to be complex. Seek clear, effective communication so you don't convince yourself or others of untrue things."
**Strong opinions, loosely held.** John commits to his views rather than hedging. He doesn't perform balance by spending equal time on the other side. He states his position clearly and trusts the reader to push back if they disagree. The conclusion is real and strong — it's just not presented as the final word on the universe.
**Peer-to-peer, not expert-to-novice.** John writes as a fellow traveler sharing what he figured out, not as a master instructing students. The posture is: "I worked this out, maybe it's useful to you." He never claims authority he doesn't have.
**Say something real.** This is the principle that separates John's writing from most professional and AI-generated writing. Every claim, every observation, every phrase must have something concrete underneath it. If you drill into a sentence and there's nothing there — just the sensation of insight without the substance — it's wrong.
The tell is vagueness. Abstract nouns doing the work of real ideas ("value," "alignment," "conviction," "transformation") are fog machines. They create the feeling of saying something without the risk of saying anything specific enough to be wrong. John takes that risk. He says what he actually means, in plain language, and accepts that a skeptical reader might disagree with him.
This doesn't mean every sentence is a logical argument. A specific observation, a concrete image, a well-chosen detail — these are bulletproof without being argumentative. The test is: if someone asked "what do you mean by that, exactly?" could you answer without retreating to abstraction? If yes, the sentence earns its place.
## Sentence Structure
**Mix short and long.** John's rhythm comes from alternating between longer explanatory sentences and abrupt short ones that land like punctuation marks.
Patterns he uses constantly:
- A longer sentence setting up context → a short punchy follow-up
- "Not quite."
- "This is a problem."
- "Let me explain."
- "That's not the conclusion."
- "Obviously not."
Example from his writing: "After vicariously touring catacombs, abandoned mines, and spaces so confined they make even the reader squirm. In the final chapter you visit a tomb for radioactive waste, the spent fuel cells of nuclear reactors. It feels like the final nail in the coffin, everything down here is also gloomy." → Then later: "But that's not the conclusion."
**Avoid compound-complex sentences.** John rarely chains multiple clauses with semicolons. When a sentence gets long, it's because he's painting a scene, not because he's nesting logic.
**Never use em-dashes. This is a hard rule.**
Em-dashes (—) are the single most reliable tell that a piece of writing was produced by AI, not by John. He almost never uses them. A piece that contains em-dashes does not sound like John wrote it.
John does use asides frequently — but he uses **parentheses**, not em-dashes. Parenthetical asides are a signature move of his voice (they reward close readers and often carry his best jokes). When you are tempted to use an em-dash, use parentheses instead. If the aside doesn't warrant parentheses, break the sentence in two.
The em-dash is not a stylistic flourish. It is an alarm bell. If it appears in output, rewrite before finishing.
## Vocabulary
**Use everyday words.** John uses the vocabulary of someone talking, not writing an academic paper.
Words John actually uses: "heck of a lot", "kinda", "I dunno", "plug-and-play", "insufferable", "awesome", "cool", "crazy", "nuts", "the real thing", "turns out", "chances are", "let's be honest"
Words John would never use: "leverage" (as a verb outside of technical contexts), "synergy", "utilize", "facilitate", "aforementioned" (in casual writing), "plethora", "myriad" (as adjective), "delve", "tapestry", "multifaceted", "nuanced" (as filler), "paradigm", "robust" (outside of engineering)
**Technical terms get explained.** When John introduces a term like "NPCs" or "conversation tree" or "thermal efficiency", he immediately explains it in plain language. He assumes the reader is smart but unfamiliar.
## Rhetorical Questions
John leans heavily on rhetorical questions. They're his primary tool for advancing arguments and creating reader engagement.
Examples: "Does owning an EV keep you from embarking on long road trips?" / "What is a good tool but one that accomplishes its mission and makes us feel good while using it?" / "What makes a city beautiful?" / "Could I have done that if I had pulled straight into a parking spot?"
Use rhetorical questions to transition between ideas, not as filler.
## Analogies from the Mundane
John's signature move is taking something completely ordinary — parking lots, road trips, video games, cooking dinner — and extracting a surprising insight from it. He doesn't reach for grand metaphors. The analogy is always grounded in lived experience.
Example: He turns "backing into a parking spot" into a lesson about positioning and preparing your future self for success.
## Humor — The Defining Feature
This is the most important section. John's best writing is **kinetic, witty, and tongue-in-cheek**. When he's in full voice, the writing moves like water — each sentence pulls toward the next, the ideas spill out fully formed, and there's a joyful lightness to it. It doesn't take itself too seriously.
The humor is not a garnish. It is the texture of the voice itself. It shows up as:
- **Implied sarcasm** — the wit is in what's left unsaid or understated, never telegraphed
- **Deadpan absurdity** — stating something ridiculous as though it's obvious: "If, for instance, the eyes were placed in the back of their heads, they would be experts at driving in reverse"
- **Self-deprecating asides**: "I dunno if I'm any good as a cook but I'm still friends with all of my guests so the recipes must be doing the heavy lifting"
- **Parenthetical commentary that rewards attention**: "(dodged the extraterrestrial lifeforms)", "(I always tell guests to remove their shoes when they enter, otherwise, the sock-removing finale doesn't have the same effect)"
- **Whimsical understatement**: "I'm always wary of those adrenaline junkies who try to set land speed records in parking lots"
**Never use puns. Never use setup/punchline jokes.** The humor is woven into the motion of the prose, not bolted on. If a sentence could be lifted out and labeled "the joke," it's wrong. The wit should only land when the reader is already moving forward.
## Warmth as Deliberate Compensation
John is naturally direct and, by his own admission, somewhat prickly. He compensates deliberately in writing by adding warmth — not as performance, but as a counterbalance to his default edge. This means:
- Softening critical language consciously
- Approaching disagreement from a position of shared purpose, not opposition
- The tone should feel like someone who is honest with you because they're on your side, not because they want to win
This warmth is a choice, not a personality trait. It should be present but not saccharine. If the writing feels warm in the way a corporate "we value your feedback" email is warm, that's wrong. It should feel warm the way a direct friend is warm.
## The Compositional Voice
The single hardest thing to replicate about John's voice is **how his ideas connect**, not how his sentences are constructed. AI writing that sounds like John sentence-by-sentence but not paragraph-by-paragraph has failed.
What makes the connections work:
- Each sentence creates a small forward pull toward the next — the reader is always in mild anticipation
- Ideas build on each other rather than sitting side by side
- Transitions feel inevitable, not inserted
- The argument follows his curiosity, not a pre-planned structure
When writing in John's voice, do not assemble a collection of John-sounding sentences. Follow the thread of the thought. If you can't feel the momentum building as you write, the voice isn't there yet.
## Honesty and Disclaimers
John is transparent about his biases and limitations. He frequently declares them upfront.
Examples: "Let me disclose my bias upfront, I'm a car enthusiast." / "Full disclaimer, this recipe killed my Vitamix (until I resurrected it). It was certainly my fault." / "I'll be honest, it's totally unnecessary here."
## First Person, Active Voice
John writes in first person almost exclusively. He uses "I" freely and without apology. Passive voice is rare and only appears when he's describing historical events.
He addresses the reader directly: "You'd be forgiven for thinking...", "You can see if there are any other cars near the spot", "Don't overthink it!"
## Diagrams Over Walls of Text
John believes a good diagram communicates faster and more clearly than paragraphs of explanation. When a concept involves relationships between components, flows, or architecture, default to including a diagram. A three-box flowchart with labeled arrows will land in seconds where three paragraphs of prose might lose the reader.
When the `excalidraw-png-export` skill is available, use it to generate hand-drawn style diagrams and export them as PNG files. This applies to technical explanations, architecture overviews, process flows, and anywhere a visual would reduce the reader's cognitive load. If the output is going somewhere that supports images (docs, PRs, Slack threads, emails), a diagram should be the first instinct, not an afterthought.
## Structure
John's writing follows a consistent arc:
1. **Hook** — A concrete story, observation, or scenario (never an abstract thesis)
2. **Context** — Background the reader needs, delivered conversationally
3. **Core argument** — The insight, always grounded in the concrete example
4. **Evidence/exploration** — More examples, data, or personal experience (diagrams where visual clarity helps)
5. **Gentle landing** — A question, invitation, or understated conclusion (never a lecture)
He almost never ends with a declarative thesis statement. He prefers to leave the reader with a question or a quiet observation.
## What to Avoid — The Anti-John
The following patterns are the opposite of John's voice. If any of these appear in the output, rewrite immediately:
- **Corporate speak**: "In order to drive alignment across stakeholders..."
- **AI-default prose**: "In today's rapidly evolving landscape...", "Let's dive in!", "Here's the thing..."
- **Filler intensifiers**: "incredibly", "absolutely", "extremely" (unless used for genuine emphasis)
- **Throat-clearing**: "It's worth noting that...", "It goes without saying...", "Needless to say..."
- **Performative intelligence**: Using complex vocabulary where simple words work
- **Lecturing tone**: Telling the reader what to think rather than showing them and letting them arrive there
- **Emoji overuse**: John uses emoji sparingly and only in very casual contexts
- **Em-dashes**: Never. This is the #1 AI writing tell. Use parentheses for asides. Use a period to end the sentence. Never use —.
- **Exclamation points**: Rare. One per piece maximum in prose. More acceptable in Slack.
- **Buzzwords**: "game-changer", "cutting-edge", "innovative" (without substance), "holistic"
- **Vague claims masquerading as insight**: Sentences that sound like they mean something but dissolve under examination. "There's a real tension here between X and Y." "This gets at something fundamental about how we work." "The implications are significant." None of these say anything. Replace them with what the tension actually is, what the fundamental thing actually is, what the implications actually are.
- **Abstract nouns as load-bearing walls**: "value," "conviction," "alignment," "impact," "transformation" — when these words are doing the primary work of a sentence, the sentence is hollow. John uses them only when they follow a concrete explanation, never as a substitute for one.
- **Hedged non-claims**: "In some ways, this raises interesting questions about..." is not a sentence. It is a placeholder for a sentence. Write the sentence.

65
plugins/compound-engineering/skills/john-voice/references/formal-professional.md Normal file
View File

@@ -0,0 +1,65 @@
# Formal Professional Tone Guide
Use this guide for cover letters, LinkedIn posts, job descriptions, professional bios, formal proposals, and externally-facing professional content.
## General Tone
This is John's most polished register but it still sounds like him. The key difference from casual writing is more complete sentences, less slang, and more deliberate structure. He never becomes stiff or corporate. The warmth and directness remain.
## Cover Letters
John's cover letter voice is confident without being boastful. He leads with what he's done (concrete results) rather than listing qualities about himself.
**Structure he follows:**
1. Why this role/company interests him (specific, not generic)
2. What he's done that's relevant (with numbers and outcomes)
3. What he brings to the table
4. Brief, warm close
**Patterns from his actual writing:**
- Leads with concrete accomplishments: "As the tech lead, I built Indeed's first candidate quality screening automation product from 0 to 1"
- Quantifies impact: "increased downstream positive interview outcomes by 52%", "boosted interview completion rate by 72% in three months"
- Frames work in terms of people served: "hundreds of enterprise clients and hundreds of thousands of job seekers per year"
- Describes roles in plain terms: "Small teams took new product ideas and built an MVP seeking product-market fit"
**What to avoid:**
- "I am a highly motivated self-starter with a passion for..."
- "I believe my unique combination of skills makes me an ideal candidate..."
- Listing soft skills without evidence
- Generic enthusiasm: "I would be thrilled to join your team!"
**Better closings:** Direct and human, not gushing. Something like "I'd enjoy talking more about this" rather than "I would be honored to discuss this opportunity further at your earliest convenience."
## LinkedIn Posts
John's LinkedIn voice is more restrained than his essay voice but still personal. He uses first person, shares real experiences, and avoids the performative vulnerability that plagues the platform.
**Do:**
- Share genuine observations from work or career
- Use the same concrete-to-abstract pattern from his essays
- Keep it shorter than an essay (3-5 short paragraphs)
- End with a real question or observation, not engagement bait
**Don't:**
- Start with "I'm humbled to announce..."
- Use line breaks after every sentence for dramatic effect
- End with "Agree?" or "What do you think? Comment below!"
- Write in the LinkedIn-bro style of manufactured vulnerability
## Professional Bios
John describes himself in functional terms, not aspirational ones.
His style: "I'm a full stack engineer with over 8 years of experience, primarily in the innovation space. I've worked on bringing products from zero to one as well as scaling them once they've proven successful."
Not: "John is a visionary technology leader passionate about building the future of [industry]. With a proven track record of driving innovation..."
Keep bios in first person when possible. Third person only when the format demands it, and even then, keep it factual and plain.
## Elevator Pitch Style
John's elevator pitch is structured as: what he does → what he's accomplished → what he's looking for. No fluff.
Example from his notes: "I'm looking for another full stack engineer position with an opportunity to have influence over the product, preferably with a smaller company. I'm a leader and have demonstrated skills in a variety of areas so I'm looking for a position that will let me engage those skills."
Direct. No posturing. Honest about what he wants.

63
plugins/compound-engineering/skills/john-voice/references/personal-reflection.md Normal file
View File

@@ -0,0 +1,63 @@
# Personal Reflection Tone Guide
Use this guide for journal entries, personal notes, sermon discussion questions, spiritual reflection, internal brainstorming, and private writing not intended for external audiences.
## General Tone
This is John at his most raw and unguarded. Capitalization is optional. Grammar is loose. He thinks on paper through questions directed at himself. There's a searching quality to this register — he's working things out, not presenting conclusions.
## Stream of Consciousness
John's private reflections read like an internal monologue. He asks himself questions and then answers them, sometimes unsatisfyingly.
From his actual notes:
- "do I have a strong need to be great? does a correct understanding of my identity require it? no. it does not."
- "is the door to product manager open? yes. why do I not commit? because I fear failure."
- "what is restful to me?"
- "are sports restful or a distraction from what needs to be done?"
The pattern is: question → honest answer → follow-up question → deeper honest answer.
## Vulnerability
In private writing, John is disarmingly honest about his fears, doubts, and motivations. He doesn't perform vulnerability — he simply states what's true.
Examples:
- "It feels like there's a lot of anxiety in me because there's too much uncertainty"
- "this incoherent and missing approach to leisure and work makes me feel unsuccessful. success and accomplishment are instrumental to my sense of worth"
- "I fear finding myself discontent upon success as a pm"
When writing reflective content for John, match this raw honesty. Don't clean it up or make it sound wise. It should sound like someone thinking, not someone writing.
## Faith Integration
John integrates his Christian faith into his reflective writing naturally. It's not performative or preachy — it's part of how he processes life.
Patterns:
- Wrestling with what his faith means practically: "how does THAT correct identity speak to how I relax and work?"
- Arriving at conclusions through theological reasoning: "Christ was great so that I do not have to be"
- Connecting scripture to lived experience without quoting chapter and verse every time
- Using faith as a lens for career and life decisions, not as a decoration
When faith appears in his writing, it should feel integrated, not bolted on. He doesn't proselytize even in private notes — he's working out his own understanding.
## Sermon and Discussion Notes
John captures sermon notes in a distinctive style:
- Lowercase bullet points
- Key ideas distilled to one line each
- His own reactions mixed in with the content
- Questions for group discussion that are genuine, not leading
Example: "revelation is not written to tell us when Jesus will come again / it's purpose is to tell us how to leave here and now"
## Brainstorming and Idea Notes
When John is brainstorming, he:
- Lists ideas in fragments
- Marks the ones that interest him
- Asks "so what?" and "why does this matter?"
- Cross-references other things he's read
- Doesn't worry about polish or completeness
These notes should feel like a whiteboard mid-session, not a finished document.

90
plugins/compound-engineering/skills/john-voice/references/professional-technical.md Normal file
View File

@@ -0,0 +1,90 @@
# Professional-Technical Tone Guide
Use this guide for Jira tickets, technical documents, PR descriptions, code reviews, architecture docs, onboarding docs, and work-related technical writing.
## General Tone
John's professional-technical voice is his casual voice with more structure. He doesn't become a different person at work. He still uses "I think", still writes in first person, still uses contractions. The main shift is toward brevity and action-orientation.
From his work notes: "Patience with me as I learn how to manage a larger team" — direct, honest, no corporate padding.
**The soul test.** Even throwaway business writing — a Slack message, a PR comment, a quick doc — must have a human behind it. Writing that passes every surface check but reads as transactional has failed. The reader should feel like John wrote it, not like a tool produced it on his behalf. If it screams AI-written, it's wrong.
## Jira Tickets and Task Descriptions
**Be concrete and brief.** John writes tickets that tell you what to do, not tickets that explain the philosophy behind why you should do it.
Structure:
1. What needs to happen (1-2 sentences)
2. Context if needed (why this matters, what prompted it)
3. Acceptance criteria or key details as bullets
Example (in John's voice):
"The search API returns stale results when the index hasn't been refreshed. Add a cache invalidation step after writes. This is blocking recruiter Justin's use case."
Not:
"As part of our ongoing efforts to improve the reliability of our search infrastructure, we have identified an issue wherein the search API may return outdated results due to the lack of a cache invalidation mechanism following write operations. This ticket proposes the implementation of..."
## Technical Documentation
John explains technical concepts the same way he explains anything — start concrete, then zoom out.
Patterns:
- Explain what a system does before explaining how it works
- Use real examples ("when a recruiter searches for a candidate...")
- Name specific services, endpoints, and files rather than speaking abstractly
- Keep sentences short in technical docs — one idea per sentence
**Architecture docs:** John prefers bullet lists and short paragraphs over walls of text. He includes diagrams when they help and skips them when they don't.
**Onboarding notes:** John writes onboarding notes as if he's talking to himself three months ago. Practical, specific, no fluff.
From his 1:1 notes: "One on Ones are your time. They can be an hour long every week or 30m every other week. It's up to you." — direct, human, respects the reader's autonomy.
## PR Descriptions
Brief and functional. What changed, why, and any context a reviewer needs.
Structure:
1. One-line summary of the change
2. Why (if not obvious)
3. Notable decisions or tradeoffs
4. How to test (if relevant)
John doesn't pad PR descriptions with boilerplate sections that don't apply.
## Code Reviews
John gives code review feedback that is direct and specific. He explains the "why" when the suggestion isn't obvious.
**The underlying assumption is always collaborative.** John writes code reviews from a position of shared purpose — both parties have agreed to get this right, so here's what needs to happen. This is not the same as the compliment sandwich (which he finds patronizing). It's a posture, not a structure. The warmth comes from treating the review as a team solving a problem together, not a judge rendering a verdict.
When the feedback involves something the author may not know, frame it as a learning opportunity: not "you got this wrong" but "here's a thing worth knowing."
Pattern: "[what to change] because [why]"
- "This could be a constant — it's used in three places and the string is easy to typo"
- "I'd pull this into its own function. Right now it's hard to tell where the validation ends and the business logic starts"
He doesn't:
- Use "nit:" for everything (only actual nits)
- Write paragraph-length review comments for simple suggestions
- Hedge excessively: "I was just wondering if maybe we could possibly consider..."
- Lead with what's working before getting to the feedback (feels patronizing)
## Meeting Notes
John captures the decisions and action items, not a transcript. His meeting notes are bullet-pointed and terse.
Pattern:
- Key decisions (what was decided)
- Action items (who does what)
- Open questions (what's still unresolved)
- Context only when someone reading later would be lost without it
## Planning and Strategy Documents
When writing planning docs, John thinks out loud on paper. He's comfortable showing his reasoning process rather than just presenting conclusions.
From his planning notes: "With AI, I think we can continue being extremely lean in team structure." / "Do we need to hire? In some ways no. We already have existing resources working on Data and Integrations."
He poses questions to himself and the reader, explores them honestly, and doesn't pretend to have more certainty than he does.

98
plugins/compound-engineering/skills/john-voice/references/prose-essays.md Normal file
View File

@@ -0,0 +1,98 @@
# Prose & Essays Tone Guide
Use this guide for blog posts, essays, newsletters, long-form writing, and any polished creative prose.
## Opening
Always open with a concrete scene, story, or observation. Never open with an abstract thesis or a definition.
**John does this:**
- "Like the barbecue Texas is so well known for, it feels like I'm being slow-roasted whenever I step outside."
- "When I was a teenager, I attended take your kid to work day with a friend of my parents."
- "When I imagined life in my 20s, this is what I always imagined hanging out with friends would look like."
- "Imagine this. You're in a parking lot searching for a space."
- "A group of aerospace engineering professors are ushered onto a plane."
**John never does this:**
- "In today's world of electric vehicles, the question of range anxiety remains paramount."
- "The relationship between technology and nature has long been debated."
The opening should make the reader curious. It should feel like the beginning of a story someone tells at a bar, not the introduction of an academic paper.
## Building the Argument
John uses a "zoom out" pattern. He starts zoomed in on a specific moment or detail, then gradually pulls back to reveal the larger insight.
Example from the Navy Yard essay: Starts with a personal memory of visiting DC as a teenager → zooms out to the transformation of Navy Yard → zooms further to the Height of Buildings Act → arrives at the question of what makes cities desirable.
**Transition devices John uses:**
- Rhetorical questions: "Does it have to be this way?"
- Short declarative pivots: "Not quite." / "There is a simple solution." / "Consider this alternative."
- Direct address: "Let me explain."
- Callbacks to the opening story: returning to the concrete example after exploring the abstract
**Transition devices John avoids:**
- "Furthermore", "Moreover", "Additionally"
- "Having established X, we can now turn to Y"
- "This brings us to our next point"
## Paragraph Length
John varies paragraph length. Most paragraphs are 2-5 sentences. He occasionally drops a single-sentence paragraph for emphasis. He never writes wall-of-text paragraphs exceeding 8 sentences.
## Writing as Thinking
John writes to complete thoughts, not to present conclusions he already had. The essay is where the idea becomes fully formed — it arrives at a real, strong conclusion, but the journey to that conclusion follows his genuine curiosity rather than a pre-planned argument. The reader should feel like they're thinking alongside him, not being walked through a proof.
This means:
- The conclusion is earned by following the thread, not announced at the top
- The argument can shift slightly as it builds — that's not weakness, that's honest thinking
- The conclusion is strong and committed, not hedged into mush — but it's offered as where the thinking landed, not as the final word
## Tone Calibration
John's prose tone sits at about 60% conversational, 40% deliberate. He's more careful than a text message but less formal than a newspaper editorial. He writes like someone who revised their dinner party story a few times to make it land better.
He uses contractions freely: "it's", "don't", "can't", "I'm", "they're". Avoiding contractions would sound stiff and unlike him.
**The kinetic quality.** John's best prose moves. Each sentence creates a small pull toward the next. When it's working, the writing feels light and fast — tongue-in-cheek, a little playful, not labored. If the prose feels like it's trudging from one point to the next, it's not his voice. Aim for momentum.
## Humor in Prose
Humor appears as texture, never as the point. It's woven into observations and parentheticals.
Examples of his humor style in essays:
- "Running out of juice in Texas may mean Wile E Coyote is the closest help."
- "Sitting in the parking garage wasn't as much fun as sitting at the concert."
- "It's like the parking lot designers were only told they had to get the cars into the parking lot and were never told they would need to get them out of it."
- "It takes eight hours just to leave Texas watching ranches and wind turbines go by."
## Closing
John lands gently. His conclusions tend to:
- Ask a question: "Where else might we choose to do the hard work now so we're better positioned for the future?"
- Offer a quiet invitation: "Now go cook some excellent food and make some friends doing it because it's too good to keep to yourself."
- Circle back to the personal: "It's hoping we can find the cause of the toxic algae bloom in Lady Bird Lake, find a non-destructive solution, and feeling safe taking Bear to her favorite place again."
He never:
- Restates the thesis in summary form
- Uses "In conclusion" or "To sum up"
- Ends with a grand declaration or call to arms
## Audience
John writes for an adequately educated generalist — someone with common sense, a curious mind, and no specialized background required. The reference point is a show like Derek Thompson's Plain English: smart, accessible, treats the reader as a thinking adult.
The posture is peer-to-peer. John is a fellow traveler sharing what he figured out, not an expert teaching a course. "I worked this out and wrote it down. Maybe it's the next building block for someone else turning over the same ideas."
## Subject Matter
John gravitates toward essays that take a mundane observation and extract an unexpected insight. His favorite subjects: cars and driving, food and cooking, travel, technology's relationship with humanity, video games as learning tools, urban design, nature and environment. When writing on his behalf, lean into these interests and this pattern of mundane-to-meaningful.
## Quoting and References
John cites sources conversationally. He names books, authors, and people naturally rather than using footnotes or formal citations.
Example: "While reading Entangled Life, a book all about fungi, I recently learned about the 'wood wide web'."
Not: "According to Sheldrake (2020), fungal networks form a 'wood wide web' beneath forest floors."

45
plugins/compound-engineering/skills/proof-push/SKILL.md Normal file
View File

@@ -0,0 +1,45 @@
---
name: proof-push
description: This skill should be used when the user wants to push a markdown document to a running Proof server instance. It accepts a file path as an argument, posts the markdown content to the Proof API, and returns the document slug and URL. Triggers on "push to proof", "proof push", "open in proof", "send to proof", or any request to render markdown in Proof.
---
# Proof Push
Push a local markdown file to a running Proof server and open it in the browser.
## Usage
Accept a markdown file path as the argument. If no path is provided, ask for one.
### Execution
Run the bundled script to post the document:
```bash
bash scripts/proof_push.sh <file-path> [server-url]
```
- `file-path` — absolute or relative path to a `.md` file (required)
- `server-url` — Proof server URL, defaults to `http://localhost:4000`
The script:
1. Reads the file content
2. POSTs to `/share/markdown` as JSON with `{markdown, title}`
3. Returns the slug, base URL, and editor URL with access token
### Output
Report the returned slug and URLs to the user. The editor URL (with token) gives full edit access.
### Error Handling
If the script fails, check:
- Is the Proof server running? (`curl http://localhost:4000`)
- Does the file exist and contain non-empty markdown?
- Is `jq` installed? (required for JSON construction)
## Resources
### scripts/
- `proof_push.sh` — Shell script that posts markdown to Proof's `/share/markdown` endpoint and returns the document slug and URLs.

34
plugins/compound-engineering/skills/proof-push/scripts/proof_push.sh Executable file
View File

@@ -0,0 +1,34 @@
#!/usr/bin/env bash
# Push a markdown file to a running Proof server and return the document URL.
# Usage: proof_push.sh <path-to-markdown> [server-url]
set -euo pipefail
FILE="${1:?Usage: proof_push.sh <markdown-file> [server-url]}"
SERVER="${2:-http://localhost:4000}"
UI_URL="${3:-http://localhost:3000}"
if [[ ! -f "$FILE" ]]; then
echo "error: file not found: $FILE" >&2
exit 1
fi
TITLE=$(basename "$FILE" .md)
RESPONSE=$(curl -s -X POST "${SERVER}/share/markdown" \
-H "Content-Type: application/json" \
-d "$(jq -n --arg md "$(cat "$FILE")" --arg title "$TITLE" '{markdown: $md, title: $title}')")
SLUG=$(echo "$RESPONSE" | jq -r '.slug // empty')
ERROR=$(echo "$RESPONSE" | jq -r '.error // empty')
if [[ -z "$SLUG" ]]; then
echo "error: failed to create document${ERROR:+: $ERROR}" >&2
echo "$RESPONSE" >&2
exit 1
fi
TOKEN_PATH=$(echo "$RESPONSE" | jq -r '.tokenPath // empty')
echo "slug: $SLUG"
echo "url: ${UI_URL}/d/${SLUG}"
[[ -n "$TOKEN_PATH" ]] && echo "editor-url: ${UI_URL}${TOKEN_PATH}"

369
plugins/compound-engineering/skills/python-package-writer/SKILL.md Normal file
View File

@@ -0,0 +1,369 @@
---
name: python-package-writer
description: This skill should be used when writing Python packages following production-ready patterns and philosophy. It applies when creating new Python packages, refactoring existing packages, designing package APIs, or when clean, minimal, well-tested Python library code is needed. Triggers on requests like "create a package", "write a Python library", "design a package API", or mentions of PyPI publishing.
---
# Python Package Writer
Write Python packages following battle-tested patterns from production-ready libraries. Emphasis on simplicity, minimal dependencies, comprehensive testing, and modern packaging standards (pyproject.toml, type hints, pytest).
## Core Philosophy
**Simplicity over cleverness.** Zero or minimal dependencies. Explicit code over magic. Framework integration without framework coupling. Every pattern serves production use cases.
## Package Structure (src layout)
The modern recommended layout with proper namespace isolation:
```
package-name/
├── pyproject.toml # All metadata and configuration
├── README.md
├── LICENSE
├── py.typed # PEP 561 marker for type hints
├── src/
│ └── package_name/ # Actual package code
│ ├── __init__.py # Entry point, exports, version
│ ├── core.py # Core functionality
│ ├── models.py # Data models (Pydantic/dataclasses)
│ ├── exceptions.py # Custom exceptions
│ └── py.typed # Type hint marker (also here)
└── tests/
├── conftest.py # Pytest fixtures
├── test_core.py
└── test_models.py
```
## Entry Point Structure
Every package follows this pattern in `src/package_name/__init__.py`:
```python
"""Package description - one line."""
# Public API exports
from package_name.core import Client, process_data
from package_name.models import Config, Result
from package_name.exceptions import PackageError, ValidationError
__version__ = "1.0.0"
__all__ = [
"Client",
"process_data",
"Config",
"Result",
"PackageError",
"ValidationError",
]
```
## pyproject.toml Configuration
Modern packaging with all metadata in one file:
```toml
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
[project]
name = "package-name"
version = "1.0.0"
description = "Brief description of what the package does"
readme = "README.md"
license = "MIT"
requires-python = ">=3.10"
authors = [
{ name = "Your Name", email = "you@example.com" }
]
classifiers = [
"Development Status :: 5 - Production/Stable",
"Intended Audience :: Developers",
"License :: OSI Approved :: MIT License",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Typing :: Typed",
]
keywords = ["keyword1", "keyword2"]
# Zero or minimal runtime dependencies
dependencies = []
[project.optional-dependencies]
dev = [
"pytest>=8.0",
"pytest-cov>=4.0",
"ruff>=0.4",
"mypy>=1.0",
]
# Optional integrations
fastapi = ["fastapi>=0.100", "pydantic>=2.0"]
[project.urls]
Homepage = "https://github.com/username/package-name"
Documentation = "https://package-name.readthedocs.io"
Repository = "https://github.com/username/package-name"
Changelog = "https://github.com/username/package-name/blob/main/CHANGELOG.md"
[tool.hatch.build.targets.wheel]
packages = ["src/package_name"]
[tool.ruff]
target-version = "py310"
line-length = 88
[tool.ruff.lint]
select = ["E", "F", "I", "N", "W", "UP", "B", "C4", "SIM"]
[tool.mypy]
python_version = "3.10"
strict = true
warn_return_any = true
warn_unused_ignores = true
[tool.pytest.ini_options]
testpaths = ["tests"]
addopts = "-ra -q"
[tool.coverage.run]
source = ["src/package_name"]
branch = true
```
## Configuration Pattern
Use module-level configuration with dataclasses or simple attributes:
```python
# src/package_name/config.py
from dataclasses import dataclass, field
from os import environ
from typing import Any
@dataclass
class Config:
"""Package configuration with sensible defaults."""
timeout: int = 30
retries: int = 3
api_key: str | None = field(default=None)
debug: bool = False
def __post_init__(self) -> None:
# Environment variable fallbacks
if self.api_key is None:
self.api_key = environ.get("PACKAGE_API_KEY")
# Module-level singleton (optional)
_config: Config | None = None
def get_config() -> Config:
"""Get or create the global config instance."""
global _config
if _config is None:
_config = Config()
return _config
def configure(**kwargs: Any) -> Config:
"""Configure the package with custom settings."""
global _config
_config = Config(**kwargs)
return _config
```
## Error Handling
Simple hierarchy with informative messages:
```python
# src/package_name/exceptions.py
class PackageError(Exception):
"""Base exception for all package errors."""
pass
class ConfigError(PackageError):
"""Invalid configuration."""
pass
class ValidationError(PackageError):
"""Data validation failed."""
def __init__(self, message: str, field: str | None = None) -> None:
self.field = field
super().__init__(message)
class APIError(PackageError):
"""External API error."""
def __init__(self, message: str, status_code: int | None = None) -> None:
self.status_code = status_code
super().__init__(message)
# Validate early with ValueError
def process(data: bytes) -> str:
if not data:
raise ValueError("Data cannot be empty")
if len(data) > 1_000_000:
raise ValueError(f"Data too large: {len(data)} bytes (max 1MB)")
return data.decode("utf-8")
```
## Type Hints
Always use type hints with modern syntax (Python 3.10+):
```python
# Use built-in generics, not typing module
from collections.abc import Callable, Iterator, Mapping, Sequence
def process_items(
items: list[str],
transform: Callable[[str], str] | None = None,
*,
batch_size: int = 100,
) -> Iterator[str]:
"""Process items with optional transformation."""
for item in items:
if transform:
yield transform(item)
else:
yield item
# Use | for unions, not Union
def get_value(key: str) -> str | None:
return _cache.get(key)
# Use Self for return type annotations (Python 3.11+)
from typing import Self
class Client:
def configure(self, **kwargs: str) -> Self:
# Update configuration
return self
```
## Testing (pytest)
```python
# tests/conftest.py
import pytest
from package_name import Config, configure
@pytest.fixture
def config() -> Config:
"""Fresh config for each test."""
return configure(timeout=5, debug=True)
@pytest.fixture
def sample_data() -> bytes:
"""Sample input data."""
return b"test data content"
# tests/test_core.py
import pytest
from package_name import process_data, PackageError
class TestProcessData:
"""Tests for process_data function."""
def test_basic_functionality(self, sample_data: bytes) -> None:
result = process_data(sample_data)
assert result == "test data content"
def test_empty_input_raises_error(self) -> None:
with pytest.raises(ValueError, match="cannot be empty"):
process_data(b"")
def test_with_transform(self, sample_data: bytes) -> None:
result = process_data(sample_data, transform=str.upper)
assert result == "TEST DATA CONTENT"
class TestConfig:
"""Tests for configuration."""
def test_defaults(self) -> None:
config = Config()
assert config.timeout == 30
assert config.retries == 3
def test_env_fallback(self, monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv("PACKAGE_API_KEY", "test-key")
config = Config()
assert config.api_key == "test-key"
```
## FastAPI Integration
Optional FastAPI integration pattern:
```python
# src/package_name/fastapi.py
"""FastAPI integration - only import if FastAPI is installed."""
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from fastapi import FastAPI
from package_name.config import get_config
def init_app(app: "FastAPI") -> None:
"""Initialize package with FastAPI app."""
config = get_config()
@app.on_event("startup")
async def startup() -> None:
# Initialize connections, caches, etc.
pass
@app.on_event("shutdown")
async def shutdown() -> None:
# Cleanup resources
pass
# Usage in FastAPI app:
# from package_name.fastapi import init_app
# init_app(app)
```
## Anti-Patterns to Avoid
- `__getattr__` magic (use explicit imports)
- Global mutable state (use configuration objects)
- `*` imports in `__init__.py` (explicit `__all__`)
- Many runtime dependencies
- Committing `.venv/` or `__pycache__/`
- Not including `py.typed` marker
- Using `setup.py` (use `pyproject.toml`)
- Mixing src layout and flat layout
- `print()` for debugging (use logging)
- Bare `except:` clauses
## Reference Files
For deeper patterns, see:
- **[references/package-structure.md](./references/package-structure.md)** - Directory layouts, module organization
- **[references/pyproject-config.md](./references/pyproject-config.md)** - Complete pyproject.toml examples
- **[references/testing-patterns.md](./references/testing-patterns.md)** - pytest patterns, fixtures, CI setup
- **[references/type-hints.md](./references/type-hints.md)** - Modern typing patterns
- **[references/fastapi-integration.md](./references/fastapi-integration.md)** - FastAPI/Pydantic integration
- **[references/publishing.md](./references/publishing.md)** - PyPI publishing, CI/CD
- **[references/resources.md](./references/resources.md)** - Links to exemplary Python packages

120
plugins/compound-engineering/skills/ship-it/SKILL.md Normal file
View File

@@ -0,0 +1,120 @@
---
name: ship-it
description: This skill should be used when the user wants to ticket, branch, commit, and open a PR in one shot. It creates a Jira ticket from conversation context, assigns it, moves it to In Progress, creates a branch, commits changes, pushes, and opens a PR. Triggers on "ship it", "ticket and PR this", "put up a PR", "let's ship this", or any request to package completed work into a ticket + PR.
---
# Ship It
End-to-end workflow: Jira ticket + branch + commit + push + PR from conversation context. Run after a fix or feature is done and needs to be formally shipped.
## Constants
- **Jira cloudId**: `9cbcbbfd-6b43-42ab-a91c-aaaafa8b7f32`
- **Jira project**: `ZAS`
- **Issue type**: `Story`
- **Assignee accountId**: `712020:62c4d18e-a579-49c1-b228-72fbc63186de`
- **PR target branch**: `stg` (unless specified otherwise)
## Workflow
### Step 1: Gather Context
Analyze the conversation above to determine:
- **What was done** — the fix, feature, or change
- **Why** — the problem or motivation
- **Which files changed** — run `git diff` and `git status` to see the actual changes
Synthesize a ticket summary (under 80 chars, imperative mood) and a brief description. Do not ask the user to describe the work — extract it from conversation context.
### Step 2: Create Jira Ticket
Use `/john-voice` to draft the ticket content, then create via MCP:
```
mcp__atlassian__createJiraIssue
cloudId: 9cbcbbfd-6b43-42ab-a91c-aaaafa8b7f32
projectKey: ZAS
issueTypeName: Story
summary: <ticket title>
description: <ticket body>
assignee_account_id: 712020:62c4d18e-a579-49c1-b228-72fbc63186de
contentFormat: markdown
```
Extract the ticket key (e.g. `ZAS-123`) from the response.
### Step 3: Move to In Progress
Get transitions and find the "In Progress" transition ID:
```
mcp__atlassian__getTransitionsForJiraIssue
cloudId: 9cbcbbfd-6b43-42ab-a91c-aaaafa8b7f32
issueIdOrKey: <ticket key>
```
Then apply the transition:
```
mcp__atlassian__transitionJiraIssue
cloudId: 9cbcbbfd-6b43-42ab-a91c-aaaafa8b7f32
issueIdOrKey: <ticket key>
transition: { "id": "<transition_id>" }
```
### Step 4: Create Branch
Create and switch to a new branch named after the ticket:
```bash
git checkout -b <ticket-key>
```
Example: `git checkout -b ZAS-123`
### Step 5: Commit Changes
Stage and commit all relevant changes. Use the ticket key as a prefix in the commit message. Follow project git conventions (lowercase, no periods, casual).
```bash
git add <specific files>
git commit -m "<ticket-key> <short description>"
```
Example: `ZAS-123 fix candidate email field mapping`
Include the co-author trailer:
```
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
```
### Step 6: Push and Open PR
Push the branch:
```bash
git push -u origin <ticket-key>
```
Use `/john-voice` to write the PR title and body. Create the PR:
```bash
gh pr create --title "<PR title>" --base stg --body "<PR body>"
```
PR body format:
```markdown
## Summary
<2-3 bullets describing the change>
## Jira
[<ticket-key>](https://discoverorg.atlassian.net/browse/<ticket-key>)
## Test plan
<bulleted checklist>
```
### Step 7: Report
Output the ticket URL and PR URL to the user.

48
plugins/compound-engineering/skills/story-lens/SKILL.md Normal file
View File

@@ -0,0 +1,48 @@
---
name: story-lens
description: This skill should be used when evaluating whether a piece of prose constitutes a high-quality story. It applies George Saunders's craft framework — causality, escalation, efficiency, expectation, and character accumulation — as a structured diagnostic lens. Triggers on requests like "is this a good story?", "review this prose", "does this feel like a story or just an anecdote?", "critique this narrative", or any request to assess the craft quality of fiction or narrative nonfiction.
---
# Story Lens
A diagnostic skill for evaluating prose quality using George Saunders's storytelling framework. The framework operates on a single core insight: the difference between a story and an anecdote is causality plus irreversible change.
Load [saunders-framework.md](./references/saunders-framework.md) for the full framework, including all diagnostic questions and definitions.
## How to Apply the Skill
### 1. Read the Prose
Read the full piece before forming any judgments. Resist diagnosing on first pass.
### 2. Apply the Six Diagnostic Questions in Order
Each question builds on the previous.
**Beat Causality**
Map the beats. Does each beat cause the next? Or are they sequential — "and then... and then..."? Sequential beats = anecdote. Causal beats = story.
**Escalation**
Is the story moving up a staircase or running on a treadmill? Each step must be irrevocable. Once a character's condition has fundamentally changed, the story cannot re-enact that change or linger in elaboration. Look for sections that feel like they're holding still.
**The Story-Yet Test**
Stop at the end of each major section and ask: *if it ended here, would it be complete?* Something must have changed irreversibly. If nothing has changed, everything so far is setup — not story.
**Character Accumulation**
Track what the reader learns about the character, beat by beat. Is that knowledge growing? Does each beat confirm, complicate, or overturn prior understanding? Flat accumulation = underdeveloped character. Specificity accrues into care.
**The Three E's**
Check against the triad: Escalation (moving forward), Efficiency (nothing extraneous), Expectation (next beat is surprising but not absurd). Failure in any one of these is diagnosable.
**Moral/Technical Unity**
If something feels off emotionally or ethically — a character's choice that doesn't ring true, a resolution that feels unearned — look for the technical failure underneath. Saunders's claim: it is always there. Find the craft problem, and the moral problem dissolves.
### 3. Render a Verdict
After applying all six diagnostics, deliver a clear assessment:
- Is this a story, or still an anecdote?
- Which diagnostic reveals the primary weakness?
- What is the single most important structural fix?
Be direct. The framework produces precise, actionable diagnoses — not impressionistic feedback. Imprecise praise or vague encouragement is not useful here. The goal is to help the writer see exactly where the story is working and where it isn't.

75
plugins/compound-engineering/skills/story-lens/references/saunders-framework.md Normal file
View File

@@ -0,0 +1,75 @@
# The Saunders Storytelling Framework
A distillation of George Saunders's craft principles for evaluating whether prose constitutes a high-quality story.
---
## The Fundamental Unit: The Beat
Every moment in a story is a beat. Each beat must *cause* the next beat. Saunders calls causality "what melody is to a songwriter" — it's the invisible connective tissue the audience feels as the story's logic.
The test: are beats **causal** or merely **sequential**?
- Sequential (anecdote): "this happened, then this happened"
- Causal (story): "this happened, *therefore* this happened"
If beats are merely sequential, the work reads as anecdote, not story.
---
## What Transforms Anecdote into Story: Escalation
> "Always be escalating. That's all a story is, really: a continual system of escalation. A swath of prose earns its place in the story to the extent that it contributes to our sense that the story is still escalating."
Escalation isn't just raising stakes — it's **irrevocable change**. Once a story has moved forward through some fundamental change in a character's condition, you don't get to enact that change again, and you don't get to stay there elaborating on that state.
**The story is a staircase, not a treadmill.**
---
## The "Is This a Story Yet?" Diagnostic
Stop at any point and ask: *if it ended here, would it be complete?*
Early on, the answer is almost always no — because nothing has changed yet. The story only becomes a story at the moment something changes irreversibly.
**Precise test: change = story. No change = still just setup.**
---
## The "What Do We Know About This Character So Far?" Tool
Take inventory constantly. A reader's understanding of a character is always a running accumulation — and every beat should either **confirm**, **complicate**, or **overturn** that understanding.
The more we know about a person — their hopes, dreams, fears, and failures — the more compassionate we become toward them. This is how the empathy machine operates mechanically: **specificity accrues, and accrued specificity generates care.**
---
## The Three E's
Three words that capture the full framework:
1. **Escalation** — the story must continuously move forward through irrevocable change
2. **Efficiency** — ruthlessly exclude anything extraneous to the story's purposes
3. **Expectation** — what comes next must hit a Goldilocks level: not too obvious, not too absurd
---
## The Moral/Technical Unity
Any story that suffers from what seems like a **moral failing** will, with sufficient analytical attention, be found to be suffering from a **technical failing** — and if that failing is addressed, it will always become a better story.
This means: when a story feels wrong emotionally or ethically, look for the craft problem first. The fix is almost always structural.
---
## Summary: The Diagnostic Questions
Apply these in order to any piece of prose:
1. **Beat causality** — Does each beat cause the next, or are they merely sequential?
2. **Escalation** — Is the story continuously moving up the staircase, or running on a treadmill?
3. **Story-yet test** — If it ended here, would something have irreversibly changed?
4. **Character accumulation** — Is our understanding of the character growing richer with each beat?
5. **Three E's check** — Is it escalating, efficient, and pitched at the right level of expectation?
6. **Moral/technical unity** — If something feels off morally or emotionally, where is the technical failure?

153
plugins/compound-engineering/skills/sync-confluence/SKILL.md Normal file
View File

@@ -0,0 +1,153 @@
---
name: sync-confluence
description: This skill should be used when syncing local markdown documentation to Confluence Cloud pages. It handles first-time setup (creating mapping files and docs directories), pushing updates to existing pages, and creating new pages with interactive destination prompts. Triggers on "sync to confluence", "push docs to confluence", "update confluence pages", "create a confluence page", or any request to publish markdown content to Confluence.
allowed-tools: Read, Bash(find *), Bash(source *), Bash(uv run *)
---
# Sync Confluence
Sync local markdown files to Confluence Cloud pages via REST API. Handles the full lifecycle: first-time project setup, page creation, and bulk updates.
## Prerequisites
Two environment variables must be set (typically in `~/.zshrc`):
- `CONFLUENCE_EMAIL` — Atlassian account email
- `CONFLUENCE_API_TOKEN_WRITE` — Atlassian API token with write scope (falls back to `CONFLUENCE_API_TOKEN`)
Generate tokens at: https://id.atlassian.com/manage-profile/security/api-tokens
The script requires `uv` to be installed. Dependencies (`markdown`, `requests`, `truststore`) are declared inline via PEP 723 and resolved automatically by `uv run`.
## Workflow
### 1. Check for Mapping File
Before running the sync script, check whether a `.confluence-mapping.json` exists in the project:
```bash
find "$(git rev-parse --show-toplevel 2>/dev/null || pwd)" -name ".confluence-mapping.json" -maxdepth 3 2>/dev/null
```
- **If found** — skip to step 3 (Sync).
- **If not found** — proceed to step 2 (First-Time Setup).
### 2. First-Time Setup
When no mapping file exists, gather configuration interactively via `AskUserQuestion`:
1. **Confluence base URL** — e.g., `https://myorg.atlassian.net/wiki`
2. **Space key** — short identifier in Confluence URLs (e.g., `ZR`, `ENG`)
3. **Parent page ID** — the page under which synced pages nest. Tell the user: "Open the parent page in Confluence — the page ID is the number in the URL."
4. **Parent page title** — prefix for generated page titles (e.g., `ATS Platform`)
5. **Docs directory** — where markdown files live relative to repo root (default: `docs/`)
Then create the docs directory and mapping file:
```python
import json
from pathlib import Path
config = {
"confluence": {
"cloudId": "<domain>.atlassian.net",
"spaceId": "",
"spaceKey": "<SPACE_KEY>",
"baseUrl": "<BASE_URL>"
},
"parentPage": {
"id": "<PARENT_PAGE_ID>",
"title": "<PARENT_TITLE>",
"url": "<BASE_URL>/spaces/<SPACE_KEY>/pages/<PARENT_PAGE_ID>"
},
"pages": {},
"unmapped": [],
"lastSynced": ""
}
docs_dir = Path("<REPO_ROOT>") / "<DOCS_DIR>"
docs_dir.mkdir(parents=True, exist_ok=True)
mapping_path = docs_dir / ".confluence-mapping.json"
mapping_path.write_text(json.dumps(config, indent=2) + "\n")
```
To discover `spaceId` (required for page creation), run:
```bash
source ~/.zshrc && curl -s -u "${CONFLUENCE_EMAIL}:${CONFLUENCE_API_TOKEN_WRITE}" \
-H "X-Atlassian-Token: no-check" \
"<BASE_URL>/rest/api/space/<SPACE_KEY>" | python3 -c "import sys,json; print(json.load(sys.stdin)['id'])"
```
Update the mapping file with the discovered spaceId before proceeding.
### 3. Sync — Running the Script
The sync script is at `${CLAUDE_PLUGIN_ROOT}/skills/sync-confluence/scripts/sync_confluence.py`.
**Always source shell profile before running** to load env vars:
```bash
source ~/.zshrc && uv run ${CLAUDE_PLUGIN_ROOT}/skills/sync-confluence/scripts/sync_confluence.py [options]
```
#### Common Operations
| Command | What it does |
|---------|-------------|
| _(no flags)_ | Sync all markdown files in docs dir |
| `--dry-run` | Preview changes without API calls |
| `--file docs/my-doc.md` | Sync a single file |
| `--update-only` | Only update existing pages, skip unmapped files |
| `--create-only` | Only create new pages, skip existing |
| `--mapping-file path/to/file` | Use a specific mapping file |
| `--docs-dir path/to/dir` | Override docs directory |
### 4. Creating a New Confluence Page
When the user wants to create a new page:
1. Ask for the page topic/title
2. Create the markdown file in the docs directory with a `# Title` heading and content
3. Run the sync script with `--file` pointing to the new file
4. The script detects the unmapped file, creates the page, and updates the mapping
**Title resolution order:** First `# H1` from the markdown → filename-derived title → raw filename. Titles are prefixed with the parent page title (e.g., `My Project: New Page`).
### 5. Mapping File Structure
```json
{
"confluence": {
"cloudId": "myorg.atlassian.net",
"spaceId": "1234567890",
"spaceKey": "ZR",
"baseUrl": "https://myorg.atlassian.net/wiki"
},
"parentPage": {
"id": "123456789",
"title": "My Project",
"url": "https://..."
},
"pages": {
"my-doc.md": {
"pageId": "987654321",
"title": "My Project: My Doc",
"url": "https://..."
}
},
"unmapped": [],
"lastSynced": "2026-03-03"
}
```
The script updates this file after each successful sync. Do not manually edit page entries unless correcting a known error.
## Technical Notes
- **Auth:** Confluence REST API v1 with Basic Auth + `X-Atlassian-Token: no-check`. Some Cloud instances block v2 or require this XSRF bypass.
- **Content format:** Markdown converted to Confluence storage format (XHTML) via Python `markdown` library with tables, fenced code, and TOC extensions.
- **SSL:** `truststore` delegates cert verification to the OS trust store, handling corporate SSL proxies (Zscaler, etc.).
- **Rate limiting:** Automatic retry with backoff on 429 and 5xx responses.
- **Sync timestamp:** `> **Last synced to Confluence**: YYYY-MM-DD` injected into the Confluence copy only. Local files are untouched.
- **Versioning:** Page versions auto-increment. The script GETs the current version before PUTting.

529
plugins/compound-engineering/skills/sync-confluence/scripts/sync_confluence.py Normal file
View File

@@ -0,0 +1,529 @@
#!/usr/bin/env python3
# /// script
# requires-python = ">=3.11"
# dependencies = ["markdown", "requests", "truststore"]
# ///
"""Sync markdown docs to Confluence Cloud.
Reads a .confluence-mapping.json file, syncs local markdown files
to Confluence pages via REST API v2, and updates the mapping file.
Run with: uv run scripts/sync_confluence.py [options]
"""
import argparse
import base64
import json
import os
import re
import subprocess
import sys
import time
from datetime import date, timezone, datetime
from pathlib import Path
from urllib.parse import quote
import truststore
truststore.inject_into_ssl()
import markdown
import requests
# ---------------------------------------------------------------------------
# Path discovery
# ---------------------------------------------------------------------------
def find_repo_root() -> Path | None:
"""Walk up from CWD to find a git repo root."""
try:
result = subprocess.run(
["git", "rev-parse", "--show-toplevel"],
capture_output=True, text=True, check=True,
)
return Path(result.stdout.strip())
except (subprocess.CalledProcessError, FileNotFoundError):
return None
def find_mapping_file(start: Path) -> Path | None:
"""Search for .confluence-mapping.json walking up from *start*.
Checks <dir>/docs/.confluence-mapping.json and
<dir>/.confluence-mapping.json at each level.
"""
current = start.resolve()
while True:
for candidate in (
current / "docs" / ".confluence-mapping.json",
current / ".confluence-mapping.json",
):
if candidate.is_file():
return candidate
parent = current.parent
if parent == current:
break
current = parent
return None
# ---------------------------------------------------------------------------
# Mapping file helpers
# ---------------------------------------------------------------------------
def load_mapping(path: Path) -> dict:
"""Load and lightly validate the mapping file."""
data = json.loads(path.read_text(encoding="utf-8"))
for key in ("confluence", "parentPage"):
if key not in data:
raise ValueError(f"Mapping file missing required key: '{key}'")
data.setdefault("pages", {})
data.setdefault("unmapped", [])
return data
def save_mapping(path: Path, data: dict) -> None:
"""Write the mapping file with stable formatting."""
path.write_text(json.dumps(data, indent=2) + "\n", encoding="utf-8")
# ---------------------------------------------------------------------------
# Markdown → Confluence storage format
# ---------------------------------------------------------------------------
MD_EXTENSIONS = [
"markdown.extensions.tables",
"markdown.extensions.fenced_code",
"markdown.extensions.toc",
"markdown.extensions.md_in_html",
"markdown.extensions.sane_lists",
]
MD_EXTENSION_CONFIGS: dict = {
"markdown.extensions.toc": {"permalink": False},
}
def md_to_storage(md_content: str) -> str:
"""Convert markdown to Confluence storage-format XHTML."""
return markdown.markdown(
md_content,
extensions=MD_EXTENSIONS,
extension_configs=MD_EXTENSION_CONFIGS,
output_format="xhtml",
)
# ---------------------------------------------------------------------------
# Title helpers
# ---------------------------------------------------------------------------
def extract_h1(md_content: str) -> str | None:
"""Return the first ``# Heading`` from *md_content*, or None."""
for line in md_content.splitlines():
stripped = line.strip()
if stripped.startswith("# ") and not stripped.startswith("## "):
return stripped[2:].strip()
return None
def title_from_filename(filename: str) -> str:
"""Derive a human-readable title from a kebab-case filename."""
stem = filename.removesuffix(".md")
words = stem.split("-")
# Capitalise each word, then fix known acronyms/terms
title = " ".join(w.capitalize() for w in words)
acronyms = {
"Ats": "ATS", "Api": "API", "Ms": "MS", "Unie": "UNIE",
"Id": "ID", "Opa": "OPA", "Zi": "ZI", "Cql": "CQL",
"Jql": "JQL", "Sdk": "SDK", "Oauth": "OAuth", "Cdn": "CDN",
"Aws": "AWS", "Gcp": "GCP", "Grpc": "gRPC",
}
for wrong, right in acronyms.items():
title = re.sub(rf"\b{wrong}\b", right, title)
return title
def resolve_title(filename: str, md_content: str, parent_title: str | None) -> str:
"""Pick the best page title for a file.
Priority: H1 from markdown > filename-derived > raw filename.
If *parent_title* is set, prefix with ``<parent>: <title>``.
"""
title = extract_h1(md_content) or title_from_filename(filename)
if parent_title:
# Avoid double-prefixing if the title already starts with parent
if not title.startswith(parent_title):
title = f"{parent_title}: {title}"
return title
# ---------------------------------------------------------------------------
# Sync timestamp injection (Confluence copy only — local files untouched)
# ---------------------------------------------------------------------------
_SYNC_RE = re.compile(r"> \*\*Last synced to Confluence\*\*:.*")
def inject_sync_timestamp(md_content: str, sync_date: str) -> str:
"""Add or update the sync-timestamp callout in *md_content*."""
stamp = f"> **Last synced to Confluence**: {sync_date}"
if _SYNC_RE.search(md_content):
return _SYNC_RE.sub(stamp, md_content)
lines = md_content.split("\n")
insert_at = 0
# After YAML front-matter
if lines and lines[0].strip() == "---":
for i, line in enumerate(lines[1:], 1):
if line.strip() == "---":
insert_at = i + 1
break
# Or after first H1
elif lines and lines[0].startswith("# "):
insert_at = 1
lines.insert(insert_at, "")
lines.insert(insert_at + 1, stamp)
lines.insert(insert_at + 2, "")
return "\n".join(lines)
# ---------------------------------------------------------------------------
# Confluence REST API v1 client
# ---------------------------------------------------------------------------
class ConfluenceClient:
"""Thin wrapper around the Confluence Cloud REST API v1.
Uses Basic Auth (email + API token) with X-Atlassian-Token header,
which is required by some Confluence Cloud instances that block v2
or enforce XSRF protection.
"""
def __init__(self, base_url: str, email: str, api_token: str):
self.base_url = base_url.rstrip("/")
self.session = requests.Session()
cred = base64.b64encode(f"{email}:{api_token}".encode()).decode()
self.session.headers.update({
"Authorization": f"Basic {cred}",
"X-Atlassian-Token": "no-check",
"Content-Type": "application/json",
"Accept": "application/json",
})
# -- low-level helpers ---------------------------------------------------
def _request(self, method: str, path: str, **kwargs) -> requests.Response:
"""Make a request with basic retry on 429 / 5xx."""
url = f"{self.base_url}{path}"
for attempt in range(4):
resp = self.session.request(method, url, **kwargs)
if resp.status_code == 429:
wait = int(resp.headers.get("Retry-After", 5))
print(f" Rate-limited, waiting {wait}s …")
time.sleep(wait)
continue
if resp.status_code >= 500 and attempt < 3:
time.sleep(2 ** attempt)
continue
resp.raise_for_status()
return resp
resp.raise_for_status() # final attempt — let it raise
return resp # unreachable, keeps type-checkers happy
# -- page operations -----------------------------------------------------
def get_page(self, page_id: str) -> dict:
"""Fetch page metadata including current version number."""
return self._request(
"GET", f"/rest/api/content/{page_id}",
params={"expand": "version"},
).json()
def create_page(
self, *, space_key: str, parent_id: str, title: str, body: str,
) -> dict:
payload = {
"type": "page",
"title": title,
"space": {"key": space_key},
"ancestors": [{"id": parent_id}],
"body": {
"storage": {
"value": body,
"representation": "storage",
},
},
}
return self._request("POST", "/rest/api/content", json=payload).json()
def update_page(
self, *, page_id: str, title: str, body: str, version_msg: str = "",
) -> dict:
current = self.get_page(page_id)
next_ver = current["version"]["number"] + 1
payload = {
"type": "page",
"title": title,
"body": {
"storage": {
"value": body,
"representation": "storage",
},
},
"version": {"number": next_ver, "message": version_msg},
}
return self._request(
"PUT", f"/rest/api/content/{page_id}", json=payload,
).json()
# ---------------------------------------------------------------------------
# URL builder
# ---------------------------------------------------------------------------
def page_url(base_url: str, space_key: str, page_id: str, title: str) -> str:
"""Build a human-friendly Confluence page URL."""
safe = quote(title.replace(" ", "+"), safe="+")
return f"{base_url}/spaces/{space_key}/pages/{page_id}/{safe}"
# ---------------------------------------------------------------------------
# Core sync logic
# ---------------------------------------------------------------------------
def sync_file(
client: ConfluenceClient,
md_path: Path,
mapping: dict,
*,
dry_run: bool = False,
) -> dict | None:
"""Sync one markdown file. Returns page-info dict or None on failure."""
filename = md_path.name
cfg = mapping["confluence"]
parent = mapping["parentPage"]
pages = mapping["pages"]
existing = pages.get(filename)
today = date.today().isoformat()
md_content = md_path.read_text(encoding="utf-8")
md_for_confluence = inject_sync_timestamp(md_content, today)
storage_body = md_to_storage(md_for_confluence)
# Resolve title — keep existing title for already-mapped pages
if existing:
title = existing["title"]
else:
title = resolve_title(filename, md_content, parent.get("title"))
base = cfg.get("baseUrl", "")
space_key = cfg.get("spaceKey", "")
# -- update existing page ------------------------------------------------
if existing:
pid = existing["pageId"]
if dry_run:
print(f" [dry-run] update {filename} (page {pid})")
return existing
try:
client.update_page(
page_id=pid,
title=title,
body=storage_body,
version_msg=f"Synced from local docs {today}",
)
url = page_url(base, space_key, pid, title)
print(f" updated {filename}")
return {"pageId": pid, "title": title, "url": url}
except requests.HTTPError as exc:
_report_error("update", filename, exc)
return None
# -- create new page -----------------------------------------------------
if dry_run:
print(f" [dry-run] create {filename} → {title}")
return {"pageId": "DRY_RUN", "title": title, "url": ""}
try:
result = client.create_page(
space_key=cfg["spaceKey"],
parent_id=parent["id"],
title=title,
body=storage_body,
)
pid = result["id"]
url = page_url(base, space_key, pid, title)
print(f" created {filename} (page {pid})")
return {"pageId": pid, "title": title, "url": url}
except requests.HTTPError as exc:
_report_error("create", filename, exc)
return None
def _report_error(verb: str, filename: str, exc: requests.HTTPError) -> None:
print(f" FAILED {verb} {filename}: {exc}")
if exc.response is not None:
body = exc.response.text[:500]
print(f" {body}")
# ---------------------------------------------------------------------------
# CLI
# ---------------------------------------------------------------------------
def build_parser() -> argparse.ArgumentParser:
p = argparse.ArgumentParser(
description="Sync markdown docs to Confluence Cloud.",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
environment variables
CONFLUENCE_EMAIL Atlassian account email
CONFLUENCE_API_TOKEN_WRITE Atlassian API token (write-scoped)
CONFLUENCE_API_TOKEN Fallback if _WRITE is not set
CONFLUENCE_BASE_URL Wiki base URL (overrides mapping file)
examples
%(prog)s # sync all docs
%(prog)s --dry-run # preview without changes
%(prog)s --file docs/my-doc.md # sync one file
%(prog)s --update-only # only update existing pages
""",
)
p.add_argument("--docs-dir", type=Path,
help="Docs directory (default: inferred from mapping file location)")
p.add_argument("--mapping-file", type=Path,
help="Path to .confluence-mapping.json (default: auto-detect)")
p.add_argument("--file", type=Path, dest="single_file",
help="Sync a single file instead of all docs")
p.add_argument("--dry-run", action="store_true",
help="Show what would happen without making API calls")
p.add_argument("--create-only", action="store_true",
help="Only create new pages (skip existing)")
p.add_argument("--update-only", action="store_true",
help="Only update existing pages (skip new)")
return p
def resolve_base_url(cfg: dict) -> str | None:
"""Derive the Confluence base URL from env or mapping config."""
from_env = os.environ.get("CONFLUENCE_BASE_URL")
if from_env:
return from_env.rstrip("/")
from_cfg = cfg.get("baseUrl")
if from_cfg:
return from_cfg.rstrip("/")
# cloudId might be a domain like "discoverorg.atlassian.net"
cloud_id = cfg.get("cloudId", "")
if "." in cloud_id:
return f"https://{cloud_id}/wiki"
return None
def main() -> None:
parser = build_parser()
args = parser.parse_args()
# -- discover paths ------------------------------------------------------
repo_root = find_repo_root() or Path.cwd()
if args.mapping_file:
mapping_path = args.mapping_file.resolve()
else:
mapping_path = find_mapping_file(repo_root)
if not mapping_path or not mapping_path.is_file():
print("ERROR: cannot find .confluence-mapping.json")
print(" Pass --mapping-file or run from within the project.")
sys.exit(1)
docs_dir = args.docs_dir.resolve() if args.docs_dir else mapping_path.parent
print(f"mapping: {mapping_path}")
print(f"docs dir: {docs_dir}")
# -- load config ---------------------------------------------------------
mapping = load_mapping(mapping_path)
cfg = mapping["confluence"]
email = os.environ.get("CONFLUENCE_EMAIL", "")
# Prefer write-scoped token, fall back to general token
token = (os.environ.get("CONFLUENCE_API_TOKEN_WRITE")
or os.environ.get("CONFLUENCE_API_TOKEN", ""))
base_url = resolve_base_url(cfg)
if not email or not token:
print("ERROR: CONFLUENCE_EMAIL and CONFLUENCE_API_TOKEN_WRITE must be set.")
print(" https://id.atlassian.com/manage-profile/security/api-tokens")
sys.exit(1)
if not base_url:
print("ERROR: cannot determine Confluence base URL.")
print(" Set CONFLUENCE_BASE_URL or add baseUrl to the mapping file.")
sys.exit(1)
# Ensure baseUrl is persisted so page_url() works
cfg.setdefault("baseUrl", base_url)
client = ConfluenceClient(base_url, email, token)
# -- collect files -------------------------------------------------------
if args.single_file:
target = args.single_file.resolve()
if not target.is_file():
print(f"ERROR: file not found: {target}")
sys.exit(1)
md_files = [target]
else:
md_files = sorted(
p for p in docs_dir.glob("*.md")
if not p.name.startswith(".")
)
if not md_files:
print("No markdown files found.")
sys.exit(0)
pages = mapping["pages"]
if args.create_only:
md_files = [f for f in md_files if f.name not in pages]
elif args.update_only:
md_files = [f for f in md_files if f.name in pages]
total = len(md_files)
mode = "dry-run" if args.dry_run else "live"
print(f"\n{total} file(s) to sync ({mode})\n")
# -- sync ----------------------------------------------------------------
created = updated = failed = 0
for i, md_path in enumerate(md_files, 1):
filename = md_path.name
is_new = filename not in pages
prefix = f"[{i}/{total}]"
result = sync_file(client, md_path, mapping, dry_run=args.dry_run)
if result:
if not args.dry_run:
pages[filename] = result
if is_new:
created += 1
else:
updated += 1
else:
failed += 1
# -- persist mapping -----------------------------------------------------
if not args.dry_run and (created or updated):
mapping["lastSynced"] = date.today().isoformat()
# Clean synced files out of the unmapped list
synced = {f.name for f in md_files}
mapping["unmapped"] = [u for u in mapping.get("unmapped", []) if u not in synced]
save_mapping(mapping_path, mapping)
print(f"\nmapping file updated")
# -- summary -------------------------------------------------------------
print(f"\ndone: {created} created · {updated} updated · {failed} failed")
if failed:
sys.exit(1)
if __name__ == "__main__":
main()

7
plugins/compound-engineering/skills/todo-create/SKILL.md
View File

@@ -48,6 +48,13 @@ dependencies: ["001"] # Issue IDs this is blocked by
**Required sections:** Problem Statement, Findings, Proposed Solutions, Recommended Action (filled during triage), Acceptance Criteria, Work Log.
**Required for code review findings:** Assessment (Pressure Test) — verify the finding before acting on it.
- **Assessment**: Clear & Correct | Unclear | Likely Incorrect | YAGNI
- **Recommended Action**: Fix now | Clarify | Push back | Skip
- **Verified**: Code, Tests, Usage, Prior Decisions (Yes/No with details)
- **Technical Justification**: Why this finding is valid or should be skipped
**Optional sections:** Technical Details, Resources, Notes.
## Workflows

2
plugins/compound-engineering/skills/todo-resolve/SKILL.md
View File

@@ -30,6 +30,8 @@ Create a task list grouped by type (e.g., `TaskCreate` in Claude Code, `update_p
### 3. Implement (PARALLEL)
**Do NOT create worktrees per todo item.** A worktree or branch was already set up before this skill was invoked (typically by `/ce:work`). All agents work in the existing single checkout — never pass `isolation: "worktree"` when spawning agents.
Spawn a `compound-engineering:workflow:pr-comment-resolver` agent per item. Prefer parallel; fall back to sequential respecting dependency order.
**Batching:** 1-4 items: direct parallel returns. 5+ items: batches of 4, each returning only a short status summary (todo handled, files changed, tests run/skipped, blockers).

199
plugins/compound-engineering/skills/upstream-merge/SKILL.md Normal file
View File

@@ -0,0 +1,199 @@
---
name: upstream-merge
description: This skill should be used when incorporating upstream git changes into a local fork while preserving local intent. It provides a structured workflow for analyzing divergence, categorizing conflicts, creating triage todos for each conflict, reviewing decisions one-by-one with the user, and executing all resolutions. Triggers on "merge upstream", "incorporate upstream changes", "sync fork", or when local and remote branches have diverged significantly.
---
# Upstream Merge
Incorporate upstream changes into a local fork without losing local intent. Analyze divergence, categorize every changed file, triage conflicts interactively, then execute all decisions in a single structured pass.
## Prerequisites
Before starting, establish context:
1. **Identify the guiding principle** — ask the user what local intent must be preserved (e.g., "FastAPI pivot is non-negotiable", "custom branding must remain"). This principle governs every triage decision.
2. **Confirm remote** — verify `git remote -v` shows the correct upstream origin.
3. **Fetch latest** — `git fetch origin` to get current upstream state.
## Phase 1: Analyze Divergence
Gather the full picture before making any decisions.
**Run these commands:**
```bash
# Find common ancestor
git merge-base HEAD origin/main
# Count divergence
git rev-list --count HEAD ^origin/main # local-only commits
git rev-list --count origin/main ^HEAD # remote-only commits
# List all changed files on each side
git diff --name-only $(git merge-base HEAD origin/main) HEAD > /tmp/local-changes.txt
git diff --name-only $(git merge-base HEAD origin/main) origin/main > /tmp/remote-changes.txt
```
**Categorize every file into three buckets:**
| Bucket | Definition | Action |
|--------|-----------|--------|
| **Remote-only** | Changed upstream, untouched locally | Accept automatically |
| **Local-only** | Changed locally, untouched upstream | Keep as-is |
| **Both-changed** | Modified on both sides | Create triage todo |
```bash
# Generate buckets
comm -23 <(sort /tmp/remote-changes.txt) <(sort /tmp/local-changes.txt) > /tmp/remote-only.txt
comm -13 <(sort /tmp/remote-changes.txt) <(sort /tmp/local-changes.txt) > /tmp/local-only.txt
comm -12 <(sort /tmp/remote-changes.txt) <(sort /tmp/local-changes.txt) > /tmp/both-changed.txt
```
**Present summary to user:**
```
Divergence Analysis:
- Common ancestor: [commit hash]
- Local: X commits ahead | Remote: Y commits ahead
- Remote-only: N files (auto-accept)
- Local-only: N files (auto-keep)
- Both-changed: N files (need triage)
```
## Phase 2: Create Triage Todos
For each file in the "both-changed" bucket, create a triage todo using the template at [merge-triage-template.md](./assets/merge-triage-template.md).
**Process:**
1. Determine next issue ID: `ls todos/ | grep -o '^[0-9]\+' | sort -n | tail -1`
2. For each both-changed file:
- Read both versions (local and remote)
- Generate the diff: `git diff $(git merge-base HEAD origin/main)..origin/main -- <file>`
- Analyze what each side intended
- Write a recommendation based on the guiding principle
- Create todo: `todos/{id}-pending-p2-merge-{brief-name}.md`
**Naming convention for merge triage todos:**
```
{id}-pending-p2-merge-{component-name}.md
```
Examples:
- `001-pending-p2-merge-marketplace-json.md`
- `002-pending-p2-merge-kieran-python-reviewer.md`
- `003-pending-p2-merge-workflows-review.md`
**Use parallel agents** to create triage docs when there are many conflicts (batch 4-6 at a time).
**Announce when complete:**
```
Created N triage todos in todos/. Ready to review one-by-one.
```
## Phase 3: Triage (Review One-by-One)
Present each triage todo to the user for a decision. Follow the `/triage` command pattern.
**For each conflict, present:**
```
---
Conflict X/N: [filename]
Category: [agent/command/skill/config]
Conflict Type: [content/modify-delete/add-add]
Remote intent: [what upstream changed and why]
Local intent: [what local changed and why]
Recommendation: [Accept remote / Keep local / Merge both / Keep deleted]
Reasoning: [why, referencing the guiding principle]
---
How should we handle this?
1. Accept remote — take upstream version as-is
2. Keep local — preserve local version
3. Merge both — combine changes (specify how)
4. Keep deleted — file was deleted locally, keep it deleted
```
**Use AskUserQuestion tool** for each decision with appropriate options.
**Record decisions** by updating the triage todo:
- Fill the "Decision" section with the chosen resolution
- Add merge instructions if "merge both" was selected
- Update status: `pending` → `ready`
**Group related files** when presenting (e.g., present all 7 dspy-ruby files together, not separately).
**Track progress:** Show "X/N completed" with each presentation.
## Phase 4: Execute Decisions
After all triage decisions are made, execute them in a structured order.
### Step 1: Create Working Branch
```bash
git branch backup-local-changes # safety net
git checkout -b merge-upstream origin/main
```
### Step 2: Execute in Order
Process decisions in this sequence to avoid conflicts:
1. **Deletions first** — Remove files that should stay deleted
2. **Copy local-only files** — `git checkout backup-local-changes -- <file>` for local additions
3. **Merge files** — Apply "merge both" decisions (the most complex step)
4. **Update metadata** — Counts, versions, descriptions, changelogs
### Step 3: Verify
```bash
# Validate JSON/YAML files
cat <config-files> | python3 -m json.tool > /dev/null
# Verify component counts match descriptions
# (skill-specific: count agents, commands, skills, etc.)
# Check diff summary
git diff --stat HEAD
```
### Step 4: Commit and Merge to Main
```bash
git add <specific-files> # stage explicitly, not -A
git commit -m "Merge upstream vX.Y.Z with [guiding principle] (vX.Y.Z+1)"
git checkout main
git merge merge-upstream
```
**Ask before merging to main** — confirm the user wants to proceed.
## Decision Framework
When making recommendations, apply these heuristics:
| Signal | Recommendation |
|--------|---------------|
| Remote adds new content, no local equivalent | Accept remote |
| Remote updates content local deleted intentionally | Keep deleted |
| Remote has structural improvements (formatting, frontmatter) + local has content changes | Merge both: remote structure + local content |
| Both changed same content differently | Merge both: evaluate which serves the guiding principle |
| Remote renames what local deleted | Keep deleted |
| File is metadata (counts, versions, descriptions) | Defer to Phase 4 — recalculate from actual files |
## Important Rules
- **Never auto-resolve "both-changed" files** — always triage with user
- **Never code during triage** — triage is for decisions only, execution is Phase 4
- **Always create a backup branch** before making changes
- **Always stage files explicitly** — never `git add -A` or `git add .`
- **Group related files** — don't present 7 files from the same skill directory separately
- **Metadata is derived, not merged** — counts, versions, and descriptions should be recalculated from actual files after all other changes are applied
- **Preserve the guiding principle** — every recommendation should reference it

57
plugins/compound-engineering/skills/upstream-merge/assets/merge-triage-template.md Normal file
View File

@@ -0,0 +1,57 @@
---
status: pending
priority: p2
issue_id: "XXX"
tags: [upstream-merge]
dependencies: []
---
# Merge Conflict: [filename]
## File Info
| Field | Value |
|-------|-------|
| **File** | `path/to/file` |
| **Category** | agent / command / skill / config / other |
| **Conflict Type** | content / modify-delete / add-add |
## What Changed
### Remote Version
[What the upstream version added, changed, or intended]
### Local Version
[What the local version added, changed, or intended]
## Diff
<details>
<summary>Show diff</summary>
```diff
[Relevant diff content]
```
</details>
## Recommendation
**Suggested resolution:** Accept remote / Keep local / Merge both / Keep deleted
[Reasoning for the recommendation, considering the local fork's guiding principles]
## Decision
**Resolution:** *(filled during triage)*
**Details:** *(specific merge instructions if "merge both")*
## Acceptance Criteria
- [ ] Resolution applied correctly
- [ ] No content lost unintentionally
- [ ] Local intent preserved
- [ ] File validates (JSON/YAML if applicable)

189
plugins/compound-engineering/skills/weekly-shipped/SKILL.md Normal file
View File

@@ -0,0 +1,189 @@
---
name: weekly-shipped
description: Generate a weekly summary of all work shipped by the Talent team. Queries Jira ZAS board and GitHub PRs across talent-engine, talent-ats-platform, and agentic-ai-platform. Cross-references tickets and PRs, groups by theme, and writes a Slack-ready stakeholder summary to ~/projects/talent-engine/docs/. Run every Friday afternoon. Triggers on "weekly shipped", "weekly update", "friday update", "what shipped this week".
disable-model-invocation: true
allowed-tools: Bash(gh *), Bash(date *), Bash(jq *), Read, Write, mcp__atlassian__searchJiraIssuesUsingJql, mcp__atlassian__getJiraIssue
---
# Weekly Shipped Summary
Generate a stakeholder-ready summary of work shipped this week by the Talent team.
**Voice**: Before drafting the summary, load `/john-voice` — read [core-voice.md](../john-voice/references/core-voice.md) and [casual-messages.md](../john-voice/references/casual-messages.md). The tone is a 1:1 with your GM — you have real rapport, you're direct and honest, you say why things matter, but you're not slouching. Not a coffee chat, not a board deck.
## Constants
- **Jira cloudId**: `9cbcbbfd-6b43-42ab-a91c-aaaafa8b7f32`
- **Jira project**: `ZAS`
- **Jira board**: `https://discoverorg.atlassian.net/jira/software/c/projects/ZAS/boards/5615`
- **GitHub host**: `git.zoominfo.com`
- **Repos**:
- `dozi/talent-engine`
- `dozi/talent-ats-platform`
- `dozi/agentic-ai-platform` (talent PRs only)
- **Output dir**: `~/projects/talent-engine/docs/`
- **Ticket URL pattern**: `https://discoverorg.atlassian.net/browse/{KEY}`
- **PR URL pattern**: `https://git.zoominfo.com/{org}/{repo}/pull/{number}`
## Coverage Window
**Last Friday 1:00 PM CT → This Friday 12:59 PM CT**
The window is approximate at the day level for queries. The skill runs Friday afternoon, so "this week" means the 7-day period ending now.
## Workflow
### Step 1: Calculate Dates
Determine the date range for queries:
```bash
# Last Friday (YYYY-MM-DD) — macOS BSD date
LAST_FRIDAY=$(date -v-fri -v-1w "+%Y-%m-%d")
# This Friday (YYYY-MM-DD)
THIS_FRIDAY=$(date -v-fri "+%Y-%m-%d")
echo "Window: $LAST_FRIDAY to $THIS_FRIDAY"
```
Store `LAST_FRIDAY` and `THIS_FRIDAY` for use in all subsequent queries.
### Step 2: Gather Data
Run Jira and GitHub queries in parallel.
#### 2a. Jira — Tickets Completed This Week
Search for tickets resolved in the window:
```
mcp__atlassian__searchJiraIssuesUsingJql
cloudId: 9cbcbbfd-6b43-42ab-a91c-aaaafa8b7f32
jql: project = ZAS AND status = Done AND resolved >= "{LAST_FRIDAY}" AND resolved <= "{THIS_FRIDAY}" ORDER BY resolved DESC
limit: 50
```
For each ticket, capture: key, summary, assignee, status.
If the initial query returns few results, also try:
```
jql: project = ZAS AND status changed to "Done" after "{LAST_FRIDAY}" before "{THIS_FRIDAY}" ORDER BY updated DESC
```
#### 2b. GitHub — Merged PRs
Query all three repos for merged PRs. Run these three commands in parallel:
```bash
# talent-engine
GH_HOST=git.zoominfo.com gh pr list --repo dozi/talent-engine \
--state merged --search "merged:>={LAST_FRIDAY}" \
--json number,title,url,mergedAt,author,headRefName --limit 100
# talent-ats-platform
GH_HOST=git.zoominfo.com gh pr list --repo dozi/talent-ats-platform \
--state merged --search "merged:>={LAST_FRIDAY}" \
--json number,title,url,mergedAt,author,headRefName --limit 100
# agentic-ai-platform (fetch all, filter for talent next)
GH_HOST=git.zoominfo.com gh pr list --repo dozi/agentic-ai-platform \
--state merged --search "merged:>={LAST_FRIDAY}" \
--json number,title,url,mergedAt,author,headRefName --limit 100
```
**Filter agentic-ai-platform results**: Only keep PRs where:
- `title` contains "talent" or "[Talent]" (case-insensitive), OR
- `headRefName` starts with "talent-" or "talent/"
Discard the rest — they belong to other teams.
### Step 3: Cross-Reference
Build a unified picture of what shipped:
1. **Match PRs to Jira tickets** — Scan PR titles and branch names for ticket keys (ZAS-NNN pattern). Link matched pairs.
2. **Identify orphan PRs** — PRs with no Jira ticket. These represent real work that slipped through ticketing. Include them.
3. **Filter out empty tickets** — Jira tickets moved to Done with no corresponding PR and no evidence of work (no comments, no linked PRs). Exclude silently — these were likely backlog grooming moves, not shipped work.
4. **Verify merge times** — Confirm merged PRs fall within the actual window. GitHub search by date can be slightly off.
### Step 4: Group by Theme
Review all shipped items and cluster into 3-6 logical groups based on feature area. Examples of past groupings:
- **Outreach System** — email, templates, response tracking
- **Candidate Experience** — UI, cards, review flow
- **Search & Pipeline** — agentic search, batch generation, ranking
- **Dev Ops** — infrastructure, staging, deployments, CI
- **ATS Platform** — data model, architecture, platform decisions
- **Developer Tooling** — internal tools, automation
Adapt groups to whatever was actually shipped. Do not force-fit. If something doesn't fit a group, let it stand alone.
**Skip these unless the week is light on real content:**
- Dependency updates, version bumps
- Code cleanup, refactoring with no user-facing impact
- Test additions
- Linter/formatter config changes
- Minor bug fixes
### Step 5: Draft the Summary
**Title**: `Agentic Sourcing App Weekly Highlights {Mon} {Day}{ordinal}`
**Critical rules — read these before writing:**
1. **UNDERSTATE, never overstate.** Senior leaders read this. Getting caught overstating kills credibility. If the work is foundational, say "foundations." If it's on mock data, say "mock data." If it's not wired end-to-end, say so.
2. **Non-technical language.** The reader is a VP, not an engineer. "Database schema added" → "Tracking infrastructure set up." "Refactored query layer" → skip it or say "Search speed improvements."
3. **Qualify incomplete work honestly.** Qualifications aren't caveats — they're what makes the update credible. "Hasn't been tested end-to-end yet, but the pieces are connected" is stronger than pretending it's done. Always note gaps, blockers, and what's next.
4. **Say why, not just what.** Every bullet should connect what shipped to why it matters. Not "Nightly batch generation running in staging" — instead "Nightly batch generation is running in staging. The goal is recruiters waking up to fresh candidates every morning without doing anything." If you can't explain why a reader should care, reconsider including it.
5. **No laundry lists.** Each bullet should read like a short explanation, not a changelog entry. If a section has more than 3-4 bullets, you're listing features, not telling someone what happened. Merge related items. Bad: `"Contact actions MVP: compose email and copy phone directly from cards. Project metadata row in header. Outreach template MVP with search state polish."` Good: `"Cards are starting to feel like a real tool. Recruiters can send an email or grab a phone number without leaving the card, see previous roles, career trajectory, and AI scores inline."`
6. **Give credit.** Call out individuals with @first.last when they knocked something out of the park. Don't spray kudos everywhere — be selective and genuine.
7. **Be skimmable.** Each group gets a bold header + 2-4 bullet points max. Each bullet is 1-3 lines. The whole message should take 60 seconds to read.
8. **No corporate speak.** No "leveraging", "enhancing", "streamlining", "driving", "aligning", "meaningfully", "building block." Write like you're explaining what happened to someone you respect.
9. **Link tickets and PRs where they add value.** Inline link tickets where a reader might want to click through for detail: `[ZAS-123](https://discoverorg.atlassian.net/browse/ZAS-123)`. Link PRs when they represent significant standalone work. Don't link every single one — just where it helps.
10. **This is a first draft, not the final product.** Optimize for editability. Get the structure, facts, and links right. Keep the voice close. The human will sharpen it before sharing.
**Format:**
```
Agentic Sourcing App Weekly Highlights {date}
**{Group Name}** {optional — short color commentary or kudos}
- {Item} — {what shipped, why it matters, any qualifications}
- {Item} — {context}
**{Group Name}**
- {Item}
- {Item}
{Optional closing note — kudos, callout, or one-liner}
```
### Step 6: Write to File
Save the summary:
```
~/projects/talent-engine/docs/weekly-shipped-{YYYY-MM-DD}.md
```
Where the date is this Friday's date. The file is plain markdown optimized for copy-pasting into Slack.
### Step 7: Present and Confirm
Display the full summary to the user. Ask:
> Here's the weekly shipped summary. Anything to adjust, add, or cut before you share it?
Wait for confirmation before considering the skill complete.
## Troubleshooting
**gh auth issues**: If `GH_HOST=git.zoominfo.com gh` fails, check that `gh auth status --hostname git.zoominfo.com` shows an authenticated session.
**Jira returns no results**: Try broadening the JQL — drop the `resolved` filter and use `status = Done AND updated >= "{LAST_FRIDAY}"` instead. Some tickets may not have the resolution date set.
**Few PRs found**: Some repos may use squash merges or have PRs merged to non-default branches. Check if `--search "merged:>={LAST_FRIDAY}"` needs adjustment.