john/claude-engineering-plugin
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[2.18.0] Add Dynamic Capability Discovery and iCloud sync patterns (#62)

Browse Source 
* [2.17.0] Expand agent-native skill with mobile app learnings

Major expansion of agent-native-architecture skill based on real-world
learnings from building the Every Reader iOS app.

New reference documents:
- dynamic-context-injection.md: Runtime app state in system prompts
- action-parity-discipline.md: Ensuring agents can do what users can
- shared-workspace-architecture.md: Agents and users in same data space
- agent-native-testing.md: Testing patterns for agent-native apps
- mobile-patterns.md: Background execution, permissions, cost awareness

Updated references:
- architecture-patterns.md: Added Unified Agent Architecture, Agent-to-UI
  Communication, and Model Tier Selection patterns

Enhanced agent-native-reviewer with comprehensive review process covering
all new patterns, including mobile-specific verification.

Key insight: "The agent should be able to do anything the user can do,
through tools that mirror UI capabilities, with full context about the
app state."

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* [2.18.0] Add Dynamic Capability Discovery and iCloud sync patterns

New patterns in agent-native-architecture skill:

- **Dynamic Capability Discovery** - For agent-native apps integrating with
  external APIs (HealthKit, HomeKit, GraphQL), use a discovery tool (list_*)
  plus a generic access tool instead of individual tools per endpoint.
  (Note: Static mapping is fine for constrained agents with limited scope.)

- **CRUD Completeness** - Every entity needs create, read, update, AND delete.

- **iCloud File Storage** - Use iCloud Documents for shared workspace to get
  free, automatic multi-device sync without building a sync layer.

- **Architecture Review Checklist** - Pushes reviewer findings earlier into
  design phase. Covers tool design, action parity, UI integration, context.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
Dan Shipper
2025-12-25 13:03:07 -05:00
committed by GitHub
parent 5a79f97374
commit 1bc6bd9164
12 changed files with 3523 additions and 60 deletions
Download Patch File Download Diff File Expand all files Collapse all files

658
plugins/compound-engineering/skills/agent-native-architecture/references/mobile-patterns.md Normal file
View File

@@ -0,0 +1,658 @@
<overview>
Mobile agent-native apps face unique challenges: background execution limits, system permissions, network constraints, and cost sensitivity. This guide covers patterns for building robust agent experiences on iOS and Android.
</overview>
<background_execution>
## Background Execution & Resumption
Mobile apps can be suspended or terminated at any time. Agents must handle this gracefully.
### The Challenge
```
User starts research agent
↓
Agent begins web search
↓
User switches to another app
↓
iOS suspends your app
↓
Agent is mid-execution... what happens?
```
### Checkpoint/Resume Pattern
Save agent state before backgrounding, restore on foreground:
```swift
class AgentOrchestrator: ObservableObject {
@Published var activeSessions: [AgentSession] = []
// Called when app is about to background
func handleAppWillBackground() {
for session in activeSessions {
saveCheckpoint(session)
session.transition(to: .backgrounded)
}
}
// Called when app returns to foreground
func handleAppDidForeground() {
for session in activeSessions where session.state == .backgrounded {
if let checkpoint = loadCheckpoint(session.id) {
resumeFromCheckpoint(session, checkpoint)
}
}
}
private func saveCheckpoint(_ session: AgentSession) {
let checkpoint = AgentCheckpoint(
sessionId: session.id,
conversationHistory: session.messages,
pendingToolCalls: session.pendingToolCalls,
partialResults: session.partialResults,
timestamp: Date()
)
storage.save(checkpoint, for: session.id)
}
private func resumeFromCheckpoint(_ session: AgentSession, _ checkpoint: AgentCheckpoint) {
session.messages = checkpoint.conversationHistory
session.pendingToolCalls = checkpoint.pendingToolCalls
// Resume execution if there were pending tool calls
if !checkpoint.pendingToolCalls.isEmpty {
session.transition(to: .running)
Task { await executeNextTool(session) }
}
}
}
```
### State Machine for Agent Lifecycle
```swift
enum AgentState {
case idle // Not running
case running // Actively executing
case waitingForUser // Paused, waiting for user input
case backgrounded // App backgrounded, state saved
case completed // Finished successfully
case failed(Error) // Finished with error
}
class AgentSession: ObservableObject {
@Published var state: AgentState = .idle
func transition(to newState: AgentState) {
let validTransitions: [AgentState: Set<AgentState>] = [
.idle: [.running],
.running: [.waitingForUser, .backgrounded, .completed, .failed],
.waitingForUser: [.running, .backgrounded],
.backgrounded: [.running, .completed],
]
guard validTransitions[state]?.contains(newState) == true else {
logger.warning("Invalid transition: \(state) → \(newState)")
return
}
state = newState
}
}
```
### Background Task Extension (iOS)
Request extra time when backgrounded during critical operations:
```swift
class AgentOrchestrator {
private var backgroundTask: UIBackgroundTaskIdentifier = .invalid
func handleAppWillBackground() {
// Request extra time for saving state
backgroundTask = UIApplication.shared.beginBackgroundTask { [weak self] in
self?.endBackgroundTask()
}
// Save all checkpoints
Task {
for session in activeSessions {
await saveCheckpoint(session)
}
endBackgroundTask()
}
}
private func endBackgroundTask() {
if backgroundTask != .invalid {
UIApplication.shared.endBackgroundTask(backgroundTask)
backgroundTask = .invalid
}
}
}
```
### User Communication
Let users know what's happening:
```swift
struct AgentStatusView: View {
@ObservedObject var session: AgentSession
var body: some View {
switch session.state {
case .backgrounded:
Label("Paused (app in background)", systemImage: "pause.circle")
.foregroundColor(.orange)
case .running:
Label("Working...", systemImage: "ellipsis.circle")
.foregroundColor(.blue)
case .waitingForUser:
Label("Waiting for your input", systemImage: "person.circle")
.foregroundColor(.green)
// ...
}
}
}
```
</background_execution>
<permissions>
## Permission Handling
Mobile agents may need access to system resources. Handle permission requests gracefully.
### Common Permissions
| Resource | iOS Permission | Use Case |
|----------|---------------|----------|
| Photo Library | PHPhotoLibrary | Profile generation from photos |
| Files | Document picker | Reading user documents |
| Camera | AVCaptureDevice | Scanning book covers |
| Location | CLLocationManager | Location-aware recommendations |
| Network | (automatic) | Web search, API calls |
### Permission-Aware Tools
Check permissions before executing:
```swift
struct PhotoTools {
static func readPhotos() -> AgentTool {
tool(
name: "read_photos",
description: "Read photos from the user's photo library",
parameters: [
"limit": .number("Maximum photos to read"),
"dateRange": .string("Date range filter").optional()
],
execute: { params, context in
// Check permission first
let status = await PHPhotoLibrary.requestAuthorization(for: .readWrite)
switch status {
case .authorized, .limited:
// Proceed with reading photos
let photos = await fetchPhotos(params)
return ToolResult(text: "Found \(photos.count) photos", images: photos)
case .denied, .restricted:
return ToolResult(
text: "Photo access needed. Please grant permission in Settings → Privacy → Photos.",
isError: true
)
case .notDetermined:
return ToolResult(
text: "Photo permission required. Please try again.",
isError: true
)
@unknown default:
return ToolResult(text: "Unknown permission status", isError: true)
}
}
)
}
}
```
### Graceful Degradation
When permissions aren't granted, offer alternatives:
```swift
func readPhotos() async -> ToolResult {
let status = PHPhotoLibrary.authorizationStatus(for: .readWrite)
switch status {
case .denied, .restricted:
// Suggest alternative
return ToolResult(
text: """
I don't have access to your photos. You can either:
1. Grant access in Settings → Privacy → Photos
2. Share specific photos directly in our chat
Would you like me to help with something else instead?
""",
isError: false // Not a hard error, just a limitation
)
// ...
}
}
```
### Permission Request Timing
Don't request permissions until needed:
```swift
// BAD: Request all permissions at launch
func applicationDidFinishLaunching() {
requestPhotoAccess()
requestCameraAccess()
requestLocationAccess()
// User is overwhelmed with permission dialogs
}
// GOOD: Request when the feature is used
tool("analyze_book_cover", async ({ image }) => {
// Only request camera access when user tries to scan a cover
let status = await AVCaptureDevice.requestAccess(for: .video)
if status {
return await scanCover(image)
} else {
return ToolResult(text: "Camera access needed for book scanning")
}
})
```
</permissions>
<cost_awareness>
## Cost-Aware Design
Mobile users may be on cellular data or concerned about API costs. Design agents to be efficient.
### Model Tier Selection
Use the cheapest model that achieves the outcome:
```swift
enum ModelTier {
case fast // claude-3-haiku: ~$0.25/1M tokens
case balanced // claude-3-sonnet: ~$3/1M tokens
case powerful // claude-3-opus: ~$15/1M tokens
var modelId: String {
switch self {
case .fast: return "claude-3-haiku-20240307"
case .balanced: return "claude-3-sonnet-20240229"
case .powerful: return "claude-3-opus-20240229"
}
}
}
// Match model to task complexity
let agentConfigs: [AgentType: ModelTier] = [
.quickLookup: .fast, // "What's in my library?"
.chatAssistant: .balanced, // General conversation
.researchAgent: .balanced, // Web search + synthesis
.profileGenerator: .powerful, // Complex photo analysis
.introductionWriter: .balanced,
]
```
### Token Budgets
Limit tokens per agent session:
```swift
struct AgentConfig {
let modelTier: ModelTier
let maxInputTokens: Int
let maxOutputTokens: Int
let maxTurns: Int
static let research = AgentConfig(
modelTier: .balanced,
maxInputTokens: 50_000,
maxOutputTokens: 4_000,
maxTurns: 20
)
static let quickChat = AgentConfig(
modelTier: .fast,
maxInputTokens: 10_000,
maxOutputTokens: 1_000,
maxTurns: 5
)
}
class AgentSession {
var totalTokensUsed: Int = 0
func checkBudget() -> Bool {
if totalTokensUsed > config.maxInputTokens {
transition(to: .failed(AgentError.budgetExceeded))
return false
}
return true
}
}
```
### Network-Aware Execution
Defer heavy operations to WiFi:
```swift
class NetworkMonitor: ObservableObject {
@Published var isOnWiFi: Bool = false
@Published var isExpensive: Bool = false // Cellular or hotspot
private let monitor = NWPathMonitor()
func startMonitoring() {
monitor.pathUpdateHandler = { [weak self] path in
DispatchQueue.main.async {
self?.isOnWiFi = path.usesInterfaceType(.wifi)
self?.isExpensive = path.isExpensive
}
}
monitor.start(queue: .global())
}
}
class AgentOrchestrator {
@ObservedObject var network = NetworkMonitor()
func startResearchAgent(for book: Book) async {
if network.isExpensive {
// Warn user or defer
let proceed = await showAlert(
"Research uses data",
message: "This will use approximately 1-2 MB of cellular data. Continue?"
)
if !proceed { return }
}
// Proceed with research
await runAgent(ResearchAgent.create(book: book))
}
}
```
### Batch API Calls
Combine multiple small requests:
```swift
// BAD: Many small API calls
for book in books {
await agent.chat("Summarize \(book.title)")
}
// GOOD: Batch into one request
let bookList = books.map { $0.title }.joined(separator: ", ")
await agent.chat("Summarize each of these books briefly: \(bookList)")
```
### Caching
Cache expensive operations:
```swift
class ResearchCache {
private var cache: [String: CachedResearch] = [:]
func getCachedResearch(for bookId: String) -> CachedResearch? {
guard let cached = cache[bookId] else { return nil }
// Expire after 24 hours
if Date().timeIntervalSince(cached.timestamp) > 86400 {
cache.removeValue(forKey: bookId)
return nil
}
return cached
}
func cacheResearch(_ research: Research, for bookId: String) {
cache[bookId] = CachedResearch(
research: research,
timestamp: Date()
)
}
}
// In research tool
tool("web_search", async ({ query, bookId }) => {
// Check cache first
if let cached = cache.getCachedResearch(for: bookId) {
return ToolResult(text: cached.research.summary, cached: true)
}
// Otherwise, perform search
let results = await webSearch(query)
cache.cacheResearch(results, for: bookId)
return ToolResult(text: results.summary)
})
```
### Cost Visibility
Show users what they're spending:
```swift
struct AgentCostView: View {
@ObservedObject var session: AgentSession
var body: some View {
VStack(alignment: .leading) {
Text("Session Stats")
.font(.headline)
HStack {
Label("\(session.turnCount) turns", systemImage: "arrow.2.squarepath")
Spacer()
Label(formatTokens(session.totalTokensUsed), systemImage: "text.word.spacing")
}
if let estimatedCost = session.estimatedCost {
Text("Est. cost: \(estimatedCost, format: .currency(code: "USD"))")
.font(.caption)
.foregroundColor(.secondary)
}
}
}
}
```
</cost_awareness>
<offline_handling>
## Offline Graceful Degradation
Handle offline scenarios gracefully:
```swift
class ConnectivityAwareAgent {
@ObservedObject var network = NetworkMonitor()
func executeToolCall(_ toolCall: ToolCall) async -> ToolResult {
// Check if tool requires network
let requiresNetwork = ["web_search", "web_fetch", "call_api"]
.contains(toolCall.name)
if requiresNetwork && !network.isConnected {
return ToolResult(
text: """
I can't access the internet right now. Here's what I can do offline:
- Read your library and existing research
- Answer questions from cached data
- Write notes and drafts for later
Would you like me to try something that works offline?
""",
isError: false
)
}
return await executeOnline(toolCall)
}
}
```
### Offline-First Tools
Some tools should work entirely offline:
```swift
let offlineTools: Set<String> = [
"read_file",
"write_file",
"list_files",
"read_library", // Local database
"search_local", // Local search
]
let onlineTools: Set<String> = [
"web_search",
"web_fetch",
"publish_to_cloud",
]
let hybridTools: Set<String> = [
"publish_to_feed", // Works offline, syncs later
]
```
### Queued Actions
Queue actions that require connectivity:
```swift
class OfflineQueue: ObservableObject {
@Published var pendingActions: [QueuedAction] = []
func queue(_ action: QueuedAction) {
pendingActions.append(action)
persist()
}
func processWhenOnline() {
network.$isConnected
.filter { $0 }
.sink { [weak self] _ in
self?.processPendingActions()
}
}
private func processPendingActions() {
for action in pendingActions {
Task {
try await execute(action)
remove(action)
}
}
}
}
```
</offline_handling>
<battery_awareness>
## Battery-Aware Execution
Respect device battery state:
```swift
class BatteryMonitor: ObservableObject {
@Published var batteryLevel: Float = 1.0
@Published var isCharging: Bool = false
@Published var isLowPowerMode: Bool = false
var shouldDeferHeavyWork: Bool {
return batteryLevel < 0.2 && !isCharging
}
func startMonitoring() {
UIDevice.current.isBatteryMonitoringEnabled = true
NotificationCenter.default.addObserver(
forName: UIDevice.batteryLevelDidChangeNotification,
object: nil,
queue: .main
) { [weak self] _ in
self?.batteryLevel = UIDevice.current.batteryLevel
}
NotificationCenter.default.addObserver(
forName: NSNotification.Name.NSProcessInfoPowerStateDidChange,
object: nil,
queue: .main
) { [weak self] _ in
self?.isLowPowerMode = ProcessInfo.processInfo.isLowPowerModeEnabled
}
}
}
class AgentOrchestrator {
@ObservedObject var battery = BatteryMonitor()
func startAgent(_ config: AgentConfig) async {
if battery.shouldDeferHeavyWork && config.isHeavy {
let proceed = await showAlert(
"Low Battery",
message: "This task uses significant battery. Continue or defer until charging?"
)
if !proceed { return }
}
// Adjust model tier based on battery
let adjustedConfig = battery.isLowPowerMode
? config.withModelTier(.fast)
: config
await runAgent(adjustedConfig)
}
}
```
</battery_awareness>
<checklist>
## Mobile Agent-Native Checklist
**Background Execution:**
- [ ] Checkpoint/resume implemented for all agent sessions
- [ ] State machine for agent lifecycle (idle, running, backgrounded, etc.)
- [ ] Background task extension for critical saves
- [ ] User-visible status for backgrounded agents
**Permissions:**
- [ ] Permissions requested only when needed, not at launch
- [ ] Graceful degradation when permissions denied
- [ ] Clear error messages with Settings deep links
- [ ] Alternative paths when permissions unavailable
**Cost Awareness:**
- [ ] Model tier matched to task complexity
- [ ] Token budgets per session
- [ ] Network-aware (defer heavy work to WiFi)
- [ ] Caching for expensive operations
- [ ] Cost visibility to users
**Offline Handling:**
- [ ] Offline-capable tools identified
- [ ] Graceful degradation for online-only features
- [ ] Action queue for sync when online
- [ ] Clear user communication about offline state
**Battery Awareness:**
- [ ] Battery monitoring for heavy operations
- [ ] Low power mode detection
- [ ] Defer or downgrade based on battery state
</checklist>