# 2026-06-10 (later) — Paperclip free models + auto-fallback chain

## What I built

### Problem
All 13 Paperclip agents in error: `claude-3-5-haiku-20241022` was no longer in OpenCode's available model list. The configured model had been retired. No free replacements had been assigned.

### Solution (3 pieces)

**1. Free model assignment per agent**
Single SQL UPDATE set each of the 13 agents' `adapter_config.model` to a free `opencode/*` model that routes through the opencode CLI broker (which is auth'd by your OpenRouter + Ollama Cloud + Groq env keys).

**2. Auto-fallback wrapper script**
`/usr/local/bin/opencode-with-fallback.sh` (also copied to `/paperclip/` inside the container):
- Discovery mode (`models` arg): passthrough to `opencode` so the cache works
- Run mode (`run --model X ...`): tries X, then tries each model in `$PAPERCLIP_FALLBACK_MODELS` env var, until one succeeds

**3. Per-agent fallback chain**
Each agent's `adapter_config.env.PAPERCLIP_FALLBACK_MODELS` = 3-model CSV chosen by role:
- Strategy (CEO/COO/SalesLead/CFO): deepseek-v4-flash-free, mimo, north-mini-code
- Creative (CMO/AdCopy/AdCreative/Content): deepseek, north-mini-code, nemotron
- Engineering (CTO/FoundingEngineer/DataAnalyst/AISpecialist): deepseek, mimo, nemotron

## Fixes I had to apply

1. **OPENROUTER_API_KEY missing from paperclip container** — added to `docker-compose.yml` and `.env`, recreated container with `docker compose up -d server`
2. **opencode CLI permission auto-rejection** — wrote `/root/.config/opencode/opencode.json` with `{"permission": "allow"}` so non-interactive runs don't get blocked
3. **Adapter discovery cache stale** — for 2 stuck agents (AdCopy, AdCreative), forced re-schedule by nulling `last_heartbeat_at`
4. **Heartbeat interval is 30 min** — paperclip only re-enqueues if `now - last_heartbeat_at >= 30min`, so a stale failed run blocks retries
5. **CTO was using north-mini-code-free** which gave "Bad Request" errors — swapped to deepseek-v4-flash-free

## Files changed

- `/usr/local/bin/opencode-with-fallback.sh` (host) — the wrapper script
- `/paperclip/opencode-with-fallback.sh` (container) — same file, inside the persistent volume
- `/paperclip/opencode-with-fallback-v2.sh`, `v3.sh` — additional copies used as cache-busters during debugging (can delete)
- `/root/.config/opencode/opencode.json` (container) — permission config
- `/root/paperclip/.env` — added `OPENROUTER_API_KEY`
- `/root/paperclip/docker-compose.yml` — added `OPENROUTER_API_KEY: "${OPENROUTER_API_KEY:-}"` to server env
- Postgres: 13 agents' `adapter_config` (model + command + env)

## Result

Before: 13/13 error. After: 0/13 error. All using free models with 3-fallback chains. Total cost: $0.

## Lesson

- The opencode_local adapter in Paperclip has no built-in fallback mechanism. If a model is slow or down, the heartbeat just times out. Building a wrapper at the `adapter_config.command` level is the cleanest fix because paperclip passes through arbitrary command paths.
- Paperclip's heartbeat scheduler uses `last_heartbeat_at` as the schedule baseline. A failed run still updates this timestamp, which can cause 30-min delays before retry. Forcing `last_heartbeat_at = NULL` is the workaround.
- The opencode CLI's `permission: "allow"` config (in `$HOME/.config/opencode/opencode.json`) is required for any non-interactive use. Without it, every tool call is auto-rejected and the agent fails immediately.