Insight Run, generating a training plan without paying for the LLM twice

Insight Run is an iOS running analysis app, available on the App Store. It generates training plans leading up to a race, from 5K to ultra. This post describes how that generation is built.

The constraints

A 16-week plan with detailed sessions is a lot of text for a model to produce. In production, generation took 40 seconds at the median and 99 seconds at the 90th percentile.

Three constraints come on top of these durations. iOS suspends the app shortly after it moves to the background, and a request that lasts more than a minute does not complete. When the client disconnects, Cloudflare gives the Worker only 30 seconds (waitUntil) to finish its work. Finally, the model call is not streamed: if it is interrupted, it is still billed, because the provider has already generated the tokens.

So generation has to continue without the app, and nothing that was already generated should be requested from the model again.

A skeleton computed by the code

The code first computes a deterministic skeleton: the phases (base, build, peak, taper) and the volume of each week. During the build-up, volume grows by at most 10% from one loading week to the next, and one week in four is lighter. The taper lasts one to three weeks depending on the distance.

The model then fills in the sessions, in blocks of at most four weeks. The prompt gives it the phase and the target volume of each week. When the answer comes back, the code puts the skeleton phases back, and uses the skeleton volume when the model does not return a valid one. Phases and volumes stay consistent from one block to the next, even though the blocks are generated separately.

Since the blocks do not depend on each other, they can be generated in parallel.

The pipeline

The app creates a job on the Worker, which starts a workflow. The workflow generates the blocks, records progress in KV, then assembles the plan. The app does not keep a connection open: it polls the Worker until the plan is ready.

Four calls in parallel

A Cloudflare Worker can open six simultaneous outgoing connections. Generation therefore runs at most four tasks in parallel, and leaves the other two connections for KV reads and writes. Each task is a loop: it takes the next block that has not started yet, calls the model, waits for the answer, then moves on to the next block. There are never more than four model calls at the same time.

After a first failure, queued blocks no longer start. Those already in flight run to completion: they are already paid for, and their result is kept for the retry.

One Workflows step per block

waitUntil only allows 30 seconds after a disconnection, and does not resume anything after an error. Generation therefore runs in Cloudflare Workflows. A workflow is a sequence of steps: the result of each step is saved, and a failed step can be retried on its own. Each block becomes a step, named after its weeks (weeks 5-8), with a five-minute timeout and a single retry, ten seconds after the failure. At the end of the step, the block is recorded as finished in KV.

A last step assembles the plan and delivers it. A finished block is never generated again, even if the workflow restarts, and the blocks recorded in KV give the app real progress.

The plan is computed from the job’s creation date, not from the date it runs: a resumed run finds the same number of weeks and the same skeleton.

Each generated block is also cached in KV for one hour. The key is a hash of the user and the content of the request: goal, dates, level, skeleton. If a job fails and the same generation is started again, the blocks already produced are reused instead of being paid for a second time. On delivery, these entries are deleted, so that a new request for the same race produces a new plan.

When a block fails

Each answer is validated before it is accepted: number and numbering of the weeks, session types, race placed in the last week. If it is rejected, or if the call fails, the block is requested again from a second model. When the answer was invalid, the reason for the rejection is added to the prompt, so that the model does not return the same output. If both attempts fail, Workflows retries the step once: a block can therefore cost up to four calls.

Idempotency, from the iPhone to the server

The app can be suspended right after it starts generation, or lose the response to its request. So it creates a job identifier (a UUID) and saves it on the phone before the call. This identifier goes in the Idempotency-Key header; without a valid UUID, the server returns a 400.

The Worker derives the workflow instance identifier from it: plan-<hash of the user>-<UUID>, where the user is the identifier the app sends in the X-User-ID header. A resent request lands on the existing instance instead of creating a second one, and a job identifier alone does not give access to another user’s plan.

The app then polls the server every three seconds. It ignores transient network errors and gives up after 20 minutes. If it is suspended, the workflow continues without it. On the next launch or when it comes back to the foreground, it picks up the saved jobs and fetches the plan. If generation is started again for the same goal and the same dates, the app joins the running job instead of creating another one.

Once the plan is saved on the phone, the app deletes the job on the server. A job that is never collected expires after 30 days, according to the Workflows retention period.

On iOS 26 and later, the app asks iOS for execution time with BGContinuedProcessingTask, and the system shows a progress bar. It follows the finished blocks and, between two blocks, the elapsed time, capped at 95%: iOS stops a task whose progress stalls.