Insight Run, one model per request type
Insight Run calls language models for tasks of very different sizes, from a few sentences of advice to a 16,000-token training plan block. The Cloudflare Worker maps each route to a request type and picks the model from a configuration stored in KV, which can change without shipping the app.
Insight Run is an iOS running analysis app. This article describes how its backend picks the language model that handles each request, and what happens when that model does not answer.
Requests that look nothing alike
Several features of the app go through a language model, with different needs. Daily readiness asks for a short piece of advice, capped at 600 tokens and awaited for 12 seconds at most. A generated workout is a JSON object of at most 4,000 tokens. A training plan block can reach 16,000 tokens of JSON. The coach answers as a stream, up to 2,000 tokens, and can create a workout along the way.
No single model offers the best balance of cost, latency and quality for all of these requests. The choice also has to change when a better-suited model comes out, without shipping a new version of the app.
The app holds no model provider key. It identifies itself to a Cloudflare Worker with the X-App-Key and X-User-ID headers, and the Worker, which keeps the OpenRouter key as a secret, calls the models.
One request type per route
The backend defines the request types in the RequestType enum, including SIMPLE, MODERATE, COMPLEX and WORKOUT_GENERATION. Each AI route picks its model from a type, and the mapping from type to model is data stored in KV. The app does not handle model names: its workout generation and history analysis calls explicitly pass model: nil.
In November 2026, the production configuration is as follows.
| Route | Type | Set by | Model | OpenRouter fallback |
|---|---|---|---|---|
/api/agent/chat (coach) |
COMPLEX |
server | GPT-5.6 Terra | Gemini 2.5 Flash |
/api/training-plan-jobs (plan) |
COMPLEX |
server | GPT-5.6 Terra | GPT 5.4 nano |
/api/adapt-training-plan |
COMPLEX |
server | GPT-5.6 Terra | GPT 5.4 nano |
/api/generate-workout |
WORKOUT_GENERATION |
route, single type | GPT-5.6 Terra | Gemini 2.5 Flash Lite |
/api/workout/smart-suggestion |
SMART_SUGGESTION |
route, single type | GPT-5.6 Luna | Gemini 2.5 Flash Lite |
/api/daily-readiness |
SIMPLE |
server | GPT-5.6 Luna | Gemini 2.5 Flash Lite |
/api/analyze-history/batch |
BATCH_PROCESSING |
route, single type | GPT-5.6 Luna | none |
/api/analyze-history/consolidate |
MODERATE |
route, single type | GPT-5.6 Terra | none |
/api/chat (weekly review) |
MODERATE |
app, allowlist | GPT-5.6 Terra | none |
Two models cover every use today: GPT-5.6 Luna for suggestions, daily readiness and batch history analysis, GPT-5.6 Terra for the rest. The fallback models are set in the code of each route.
Who picks the type
On the routes the app calls, a type sent by the client is not taken as is: a modified app could ask for COMPLEX on a route meant for a cheaper type. The selectModelFromRequest function therefore receives an allowlist of types for each route. A type missing from the list is ignored, and the route uses its default type. Without a list, only the route’s default type is accepted.
In practice, the server sets the type almost everywhere. The coach, plan generation and plan adaptation hardcode COMPLEX, daily readiness SIMPLE. The workout, suggestion and history routes read the type sent by the app, but accept only one, their own. /api/chat accepts several, including SIMPLE and MODERATE, but not COMPLEX, the type meant for the most expensive model, because it also receives the client’s system prompt. With the November 2026 configuration, MODERATE and COMPLEX point to the same model; the list protects the route if the configuration changes. The app uses it for the weekly review, as MODERATE.
A few routes still accept an explicit model field. It is honored only when no type is sent, if the model is in the catalog and, for a model flagged as premium in the catalog, if the user’s quota allows it. Otherwise the route falls back to its default type.
The configuration in KV
The configuration lives in two keys of the KV namespace bound to the Worker as RATE_LIMITER. admin:config:models maps each type to a catalog key. admin:config:custom_models holds the models added from the admin: OpenRouter identifier, display name, description, and a flag saying whether the model is premium. GPT-5.6 Luna and GPT-5.6 Terra are not in the code: they were added to the catalog this way.
The admin writes these keys through three routes protected by the X-Admin-Key header: PUT /api/admin/config/models for the mapping, PUT and DELETE /api/admin/models/:key for the catalog. Models defined in the code cannot be deleted.
The configuration is read on every AI request and goes through two caches. The first is a module variable, local to the Worker isolate, valid for 30 seconds. Once it has expired, the Worker reads both keys in parallel with the cacheTtl option set to 60 seconds, which keeps the value cached in the Cloudflare data center. The mapping starts from the code defaults, and each type present in KV replaces its own.
If admin:config:models is missing or its JSON is invalid, the code defaults apply as they are: Grok 4.1 Fast for SIMPLE, MODERATE and suggestions, Gemini 2.5 Flash for workouts, Gemini 2.5 Flash Lite for history batches, Gemini 3 Pro Preview for COMPLEX. A type that points to a key missing from the catalog, a deleted model for instance, gets Claude Haiku 4.5.
getModelConfigCached and selectModelForRequestType. The merged result goes back into the isolate cache for 30 seconds. Another isolate sees a change at most 90 seconds later (30 + 60), a delay derived from the code, not measured.After a write, invalidateCache() clears the in-memory cache, but only in the isolate that handled the admin request. Other isolates keep their copy for up to 30 seconds, then read a value the KV cache may hold for another 60 seconds. This delay suits a configuration that rarely changes, and it avoids a KV read on every request.
The path of a question to the coach
The coach is the route that involves the most parts.
agentChat.ts and the Worker's quota middleware, with the November 2026 production models. If GPT-5.6 Terra returns an error, OpenRouter moves to Gemini 2.5 Flash within the same call. A function_result event, when there is one, comes before [DONE].The app sends the question (5,000 characters at most), the language and the training data in structured form. The Worker checks the app key, rejects users and IP addresses blocked from the admin, then checks the quotas. It then asks for the model mapped to COMPLEX, which also reads the user’s premium counter, and builds the system prompt itself from the data it received.
The call to OpenRouter is streamed, with two models in the models field and one tool, generate_workout, which the model can call to suggest a workout. It forwards each chunk of text to the app as Server-Sent Events, in a content event. If the model called the tool, the Worker builds the workout steps itself from the arguments (type, duration, target pace) and sends them in a function_result event, right before the [DONE] marker.
Both sides check that the answer is complete. The Worker ends with an error event an empty answer, an answer cut off by the token limit, or a stream that ends without the end marker. The app rejects a [DONE] that no content preceded, as well as a stream that stops without [DONE]: a truncated answer ends in an error instead of being taken for a complete one.
The quota counter is incremented in waitUntil as soon as the route returns its response, so at the start of the stream for the coach, so that a counting error does not interrupt the answer. At the end of the stream, the Worker sends the analytics event and, if the primary model is premium and it is the one that answered, increments the premium counter.
When the model does not answer
There are three levels of fallback.
The first is delegated to OpenRouter. On the coach, workout, suggestion, daily readiness and plan routes, the call sends a models array: the configured model, then a fallback model. If the first returns an error, such as a rate limit or an outage, OpenRouter moves to the second within the same call, and the response says which model answered (documentation). The fallback model is written in the code of each route, as the table shows. A mistake in the admin configuration therefore cannot remove the fallback; on the other hand, changing it requires redeploying the Worker. /api/chat and history analysis have no fallback.
The second level is decided by the Worker. For workout generation, a network error, a timeout, a 408, 429 or 5xx status, or a response without choices lead to a second, identical call. Workout generation also runs again when the JSON is invalid or truncated, adding the reason for the rejection to the prompt. For plans and plan adaptation, any failed attempt, after a network error, a timeout or invalid JSON, is followed by a second one on GPT 5.4 nano, with the rejection reason when the answer was invalid. History analysis, batches and consolidation alike, retries the same model once, two seconds after a transient error. Plan generation, described in an earlier article, adds Cloudflare Workflows step retries on top of this.
The third level does without a model. If generating the daily readiness advice fails, the route returns predefined text, chosen from the day’s status, and the app still shows advice.
Quotas
Every /api/* route goes through a quota middleware, except the admin, Strava webhooks and plan job polling (GET and DELETE); calls without a valid app key are not counted either. Two counters are kept in KV: ratelimit:ip:<ip> and ratelimit:user:<id>. By default the code allows 100 requests per hour per IP, and 1,000 requests per user over 30 days. The admin:config:rate_limits key overrides these values: in November 2026, production keeps 100 requests per hour per IP and allows 2,000 requests per user, over the same 30-day window. Each window is fixed and opens with the first request. Past the limit, the Worker answers 429 with the X-RateLimit-* and Retry-After headers.
The quota counts HTTP requests, whatever model is called: creating a 16-week plan counts as one request, like a question to the coach. The counter is read then rewritten without an atomic operation, which KV does not offer: with its eventual consistency, two simultaneous requests may be counted once. The gap is accepted, because this quota only caps costs.
A separate quota limits models flagged as premium to 20 requests per calendar month, per user and per category (plans, coach, everything else); once used up, it moves COMPLEX to Gemini 2.5 Flash and the other types to Grok 4.1 Fast. In November 2026, no type uses a premium model, and this quota does not kick in.
Measuring without reading prompts
Every AI request that completes produces a PostHog $ai_generation event: model, input and output tokens, latency, cost in dollars as returned by OpenRouter, trace identifier and route. The coach also records its failures there; plans and history analysis report theirs with separate events. The prompts contain health data (age, weight, heart rate variability, sleep). The event only keeps their length and a SHA-256 fingerprint truncated to 8 bytes, for the user message as well as the system prompt. Of the answer, it only keeps the length.
On the coach, workout, suggestion, daily readiness and history routes, $ai_model is the model that actually answered, as reported by OpenRouter: a fallback shows up by comparing this value with the configured model. On plans, it is the requested model. The event carries neither the request type nor a fallback flag.