This is the question that separates a strong candidate.
Options: (a) on-device LLM — best privacy, weakest quality, heaviest battery; good when answers are simple/templated. (b) hosted frontier LLM — best quality, but retrieved snippets leave the device, so you need a BAA, zero-retention, and to send only non-identifying minimum-necessary text. (c) your backend bridge — proxy to an LLM so you enforce the guard, redaction, and audit in one controlled place. There is no single right answer; the reasoning is the answer: choose per the sensitivity of what must be in the prompt, the quality bar, and the compliance posture, and keep retrieval on-device regardless.