Repaso
Español
LLMs, RAG & agentsAdvancedLLM-39

Design an AI assistant that answers customers' questions from a company's help center. Walk through your design.

Answer

  • Scope first: which questions it should answer, which it must refuse or pass to a person, and what a wrong answer costs.
  • Knowledge: index the help center with RAG (structure-aware chunks, metadata, hybrid search, reranking) and refresh it when articles change.
  • Answering: answer only from the retrieved text, cite the articles, and say "I don't know" with an easy handoff to a human.
  • Safety: treat user and retrieved text as untrusted, give the assistant no write access, and apply guardrails for personal data and off-topic requests.
  • Quality: build an eval set from real tickets, measure retrieval and faithfulness separately, and run it before every prompt, model or index change.
  • Operations: stream the answer, cache stable prefixes, route simple questions to a small model, set budgets and rate limits, and monitor cost, latency, handoffs and user feedback.
  • Roll out gradually (shadow mode, then a share of traffic) and feed failures back into the eval set.

Key terms

Practice this question in Repaso