162 lines
6.0 KiB
Markdown
162 lines
6.0 KiB
Markdown
# Investigation Techniques
|
|
|
|
Techniques for deeper investigation when standard code tracing is not enough. Load this when a bug does not reproduce reliably, involves timing or concurrency, or requires framework-specific tracing.
|
|
|
|
---
|
|
|
|
## Root-Cause Tracing
|
|
|
|
When a bug manifests deep in the call stack, the instinct is to fix where the error appears. That treats a symptom. Instead, trace backward through the call chain to find where the bad state originated.
|
|
|
|
**Backward tracing:**
|
|
|
|
- Start at the error
|
|
- At each level, ask: where did this value come from? Who called this function? What state was passed in?
|
|
- Keep going upstream until finding the point where valid state first became invalid — that is the root cause
|
|
|
|
**Worked example:**
|
|
|
|
```
|
|
Symptom: API returns 500 with "Cannot read property 'email' of undefined"
|
|
Where it crashes: sendWelcomeEmail(user.email) in NotificationService
|
|
Who called this? UserController.create() after saving the user record
|
|
What was passed? user = await UserRepo.create(params) — but create() returns undefined on duplicate key
|
|
Original cause: UserRepo.create() silently swallows duplicate key errors and returns undefined instead of throwing
|
|
```
|
|
|
|
The fix belongs at the origin (UserRepo.create should throw on duplicate key), not where the error appeared (NotificationService).
|
|
|
|
**When manual tracing stalls**, add instrumentation:
|
|
|
|
```
|
|
// Before the problematic operation
|
|
const stack = new Error().stack;
|
|
console.error('DEBUG [operation]:', { value, cwd: process.cwd(), stack });
|
|
```
|
|
|
|
Use `console.error()` in tests — logger output may be suppressed. Log before the dangerous operation, not after it fails.
|
|
|
|
---
|
|
|
|
## Git Bisect for Regressions
|
|
|
|
When a bug is a regression ("it worked before"), use binary search to find the breaking commit:
|
|
|
|
```bash
|
|
git bisect start
|
|
git bisect bad # current commit is broken
|
|
git bisect good <known-good-ref> # a commit where it worked
|
|
# git bisect will checkout a middle commit — test it
|
|
# mark as good or bad, repeat until the breaking commit is found
|
|
git bisect reset # return to original branch when done
|
|
```
|
|
|
|
For automated bisection with a test script:
|
|
|
|
```bash
|
|
git bisect start HEAD <known-good-ref>
|
|
git bisect run <test-command>
|
|
```
|
|
|
|
The test command should exit 0 for good, non-zero for bad.
|
|
|
|
---
|
|
|
|
## Intermittent Bug Techniques
|
|
|
|
When a bug does not reproduce reliably after 2-3 attempts:
|
|
|
|
**Logging traps.** Add targeted logging at the suspected failure point and run the scenario repeatedly. Capture the state that differs between passing and failing runs.
|
|
|
|
**Statistical reproduction.** Run the failing scenario in a loop to establish a reproduction rate:
|
|
|
|
```bash
|
|
for i in $(seq 1 20); do echo "Run $i:"; <test-command> && echo "PASS" || echo "FAIL"; done
|
|
```
|
|
|
|
A 5% reproduction rate confirms the bug exists but suggests timing or data sensitivity.
|
|
|
|
**Environment isolation.** Systematically eliminate variables:
|
|
- Same test, different machine?
|
|
- Same test, different data seed?
|
|
- Same test, serial vs parallel execution?
|
|
- Same test, with vs without network access?
|
|
|
|
**Data-dependent triggers.** If the bug only appears with certain data, identify the trigger condition:
|
|
- What is unique about the failing input?
|
|
- Does the input size, encoding, or edge value matter?
|
|
- Is the data order significant (sorted vs random)?
|
|
|
|
---
|
|
|
|
## Framework-Specific Debugging
|
|
|
|
### Rails
|
|
- Check callbacks: `before_save`, `after_commit`, `around_action` — these execute implicitly and can alter state
|
|
- Check middleware chain: `rake middleware` lists the full stack
|
|
- Check Active Record query generation: `.to_sql` on any relation
|
|
- Use `Rails.logger.debug` with tagged logging for request tracing
|
|
|
|
### Node.js
|
|
- Async stack traces: run with `--async-stack-traces` flag for full async call chains
|
|
- Unhandled rejections: check for missing `.catch()` or `await` on promises
|
|
- Event loop delays: `process.hrtime()` before and after suspect operations
|
|
- Memory leaks: `--inspect` flag + Chrome DevTools heap snapshots
|
|
|
|
### Python
|
|
- Traceback enrichment: `traceback.print_exc()` in except blocks
|
|
- `pdb.set_trace()` or `breakpoint()` for interactive debugging
|
|
- `sys.settrace()` for execution tracing
|
|
- `logging.basicConfig(level=logging.DEBUG)` for verbose output
|
|
|
|
---
|
|
|
|
## Race Condition Investigation
|
|
|
|
When timing or concurrency is suspected:
|
|
|
|
**Timing isolation.** Add deliberate delays at suspect points to widen the race window and make it reproducible:
|
|
|
|
```
|
|
// Simulate slow operation to expose race
|
|
await new Promise(r => setTimeout(r, 100));
|
|
```
|
|
|
|
**Shared mutable state.** Search for variables, caches, or database rows accessed by multiple threads or processes without synchronization. Common patterns:
|
|
- Global or module-level mutable state
|
|
- Cache reads without locks
|
|
- Database rows read then updated without optimistic locking
|
|
|
|
**Async ordering.** Check whether operations assume a specific execution order that is not guaranteed:
|
|
- Promise.all with dependent operations
|
|
- Event handlers that assume emission order
|
|
- Database writes that assume read consistency
|
|
|
|
---
|
|
|
|
## Browser Debugging
|
|
|
|
When investigating UI bugs with `agent-browser` or equivalent tools:
|
|
|
|
```bash
|
|
# Open the affected page
|
|
agent-browser open http://localhost:${PORT:-3000}/affected/route
|
|
|
|
# Capture current state
|
|
agent-browser snapshot -i
|
|
|
|
# Interact with the page
|
|
agent-browser click @ref # click an element
|
|
agent-browser fill @ref "text" # fill a form field
|
|
agent-browser snapshot -i # capture state after interaction
|
|
|
|
# Save visual evidence
|
|
agent-browser screenshot bug-evidence.png
|
|
```
|
|
|
|
**Port detection:** Check project instruction files (`AGENTS.md`, `CLAUDE.md`) for port references, then `package.json` dev scripts, then `.env` files, falling back to `3000`.
|
|
|
|
**Console errors:** Check browser console output for JavaScript errors, failed network requests, and CORS issues. These often reveal the root cause of UI bugs before any code tracing is needed.
|
|
|
|
**Network tab:** Check for failed API requests, unexpected response codes, or missing CORS headers. A 422 or 500 response from the backend narrows the investigation immediately.
|