Professional domain 3 · Integration
Capability bloat and progressive discovery
- Every tool or agent you add costs context, adds cost and widens the attack surface. Anthropic's docs say tool selection degrades once more than about 30 to 50 tools are available.
- The tool search tool keeps large tool sets usable: tools marked
defer_loadingstay out of context until a search finds them, typically cutting tool-definition tokens by over 85%. Keep the 3 to 5 most-used tools always loaded. It's worth it from about 10 tools or 10k tokens of definitions. - Progressive discovery (tool search, Skills, retrieval, subagents) loads capabilities when they're needed. A monolithic context (everything up front) is simpler, predictable and easy to cache, and fine while it's small and stable. Choose by size, growth and how often each part is used.
Authentication and authorisation
- Typical gaps: a shared service account or long-lived key behind every user, an agent that can do more than the user it acts for, secrets in prompts or logs, and no audit trail per user.
- Act with the user's identity and permissions, grant the least privilege each tool needs, and keep secrets in configuration, never in the context.
- Authorisation is optional in MCP. When an HTTP server uses it, it follows OAuth 2.1; a local stdio server takes credentials from its environment. A server must accept only tokens issued for it, and must not pass a client's token through to downstream APIs. Ask for minimal scopes and step up when a privileged action needs it.
Accuracy, latency and cost
- Levers: model size, effort and thinking, how much is retrieved, reranking, and output length. Streaming improves perceived latency without changing total time.
- Work nobody waits for can go to the Message Batches API: half the price, results within 24 hours (most in under an hour), matched by
custom_id, not by order. - Justify a setting with measurements against the SLA: accuracy on the eval set, p95 latency, cost per task.
See also: Foundations domain 4 · Prompt engineering and structured output and Professional domain 4 · Evaluation, testing and optimisation.
Observability at scale
- Trace each request end to end: prompt version, model, tokens, latency, tool calls, retrieved documents and outcome. Without that you can't tell a retrieval miss from a prompt failure.
- Sample outputs for quality review, alert on cost, latency and error spikes, and feed failures back into the evaluation set.
RAG: chunking, indexing and retrieval
- Chunk along the document's structure (sections, headings), with a size that holds one idea and some overlap; keep metadata (source, date, owner, access level) for filtering.
- Contextual retrieval: Claude writes 50 to 100 tokens of context for each chunk before it's embedded and indexed for BM25. Anthropic measured 49% fewer failed retrievals, and 67% with reranking. Prompt caching keeps generating that context cheap.
- If the whole knowledge base is small (Anthropic's post suggests under about 200,000 tokens), put it in the prompt with caching instead of building RAG.
See also: Professional domain 2 · Prompt reuse: caching, modular prompts and Skills.
| Data and queries | Retrieval that fits |
|---|---|
| Exact identifiers: codes, part numbers, names | Keyword search (BM25) or an exact lookup |
| Conceptual questions over prose | Semantic search with embeddings |
| A mix of both | Hybrid search (embeddings + BM25), then rerank |
| Structured records and figures | Query the database or API through a tool; embeddings are poor at totals and filters |
| Time-sensitive or permissioned content | Filter on metadata (date, access level) before ranking |
| Questions needing several hops | Agentic retrieval: search, read, search again |
Choosing an integration mechanism
| Mechanism | Fits when |
|---|---|
| MCP server | The same tools or data should serve several clients or agents, with standard discovery and authorisation |
| Direct API or CLI call as a tool | One application needs one integration; the fewest moving parts |
| Agent-to-agent | Independent agents, often owned by different teams or vendors, need to hand tasks to each other |
See also: Foundations domain 2 · MCP primitives: who controls what.
Docs: Tool search toolDefining toolsMCP authorizationMCP security best practicesContextual retrievalBatch processingReducing latency
Practise Domain 3