Agentic RAG Chatbot — Multi-Step, Tool-Calling Retrieval

Overview

The site's chatbot ships with two retrieval modes side by side: a Simple RAG pipeline (one fixed retrieval pass, described in the companion Portfolio RAG Chatbot project) and this Agentic RAG mode, where the model itself decides what to search for, how many times, and which part of the knowledge base to look in - rather than one blind top-k pull across everything.

How it differs from Simple RAG

Simple RAG always does the same thing: embed the question, pull the top matches across all content, answer once. That works well for direct questions but breaks down on compound ones - "compare your career experience with your project work on RAG systems" genuinely needs two different searches (career history, then project work) to answer well.

Agentic RAG hands the model a tool, search_knowledge_base(query, source_type), via OpenAI function calling, and lets it call that tool as many times as it needs (capped at 3 rounds) with different queries and different source_type filters (profile, career, project, blog, or any) before composing a final answer.

Flow

flowchart TD
    A[Visitor asks a question] --> B[Agent LLM with search tool]
    B -->|decides to search| C[Call search_knowledge_base with query + source_type]
    C --> D[Metadata-filtered Pinecone search]
    D --> E{Relevant passages found?}
    E -->|Yes| F[Return passages to the agent as a tool result]
    E -->|No| G[Return: no relevant results]
    F --> B
    G --> B
    B -->|has enough info, or hit the round limit| H{Any relevant sources retrieved overall?}
    H -->|No| I[Refuse - out of scope]
    H -->|Yes| J[Synthesize final answer from retrieved passages]
    J --> K[Attach citations built from retrieval metadata, not the model]
    K --> L[Return answer + sources + search trail to the visitor]

Guardrails carry over unchanged

Agentic mode keeps every guardrail from the simple pipeline - it does not trade safety for capability:

• A code-level check, not just a prompt instruction: if nothing relevant was retrieved across any of the search rounds, the final answer is overridden with the same refusal message regardless of what the model produced.
• Citations are still built from retrieval metadata after the fact, never asked of the model, so a source link can't be hallucinated.
• The same anti-prompt-injection instructions apply to every round of the loop, not just the first message.
• A hard cap of 3 search rounds bounds both cost and latency per question.

Visible reasoning trail

Because the searches are real, discrete tool calls, the exact queries and filters the agent chose are returned alongside the answer and shown in the chat widget as a "Searched: career (...), project (...)" line - turning an otherwise invisible reasoning process into something a visitor can actually see and evaluate.

Result

Asking "Compare your career experience with your project work on RAG systems" through Agentic mode produces two independent searches - one filtered to career, one filtered to project - and a final answer citing both, where Simple mode would have run a single undifferentiated search across everything. Off-topic questions and prompt-injection attempts are refused in both modes.