On-device, scale is bounded by indexing the report subset, not the genome — thousands of chunks, a few MB index, low-ms retrieval. The slow step is generation.
Performance levers: Cache embeddings (don't re-embed unchanged content) Debounce query embedding Run the embedder on the Neural Engine/NNAPI Stream the LLM response for perceived latency Pre-warm the model Set explicit budgets (first-launch index time, query latency, peak memory, index size) and measure on a named device class.
For a multi-tenant backend variant, scale the LLM tier and use batching/caching there.