Interviewers love concrete budgets.
A realistic on-device query: Embed query — tens of ms ANN search — low ms for thousands of vectors Generation — the slow part: local LLM is seconds, or a network call to a hosted LLM Memory: keep peak under a few hundred MB; stream/quantize to avoid OOM kills.
Battery/heat: avoid re-embedding on every keystroke (debounce), avoid full re-index loops, and prefer the Neural Engine/NNAPI over CPU.
Always cite the device class you measured on ("mid-tier Android, 6GB RAM").