Progressive Tool Discovery & The Evolution of MCP

1. Executive Summary: The "Many-Tools" Problem

Before late 2024, the Model Context Protocol (MCP) standardized how Large Language Models (LLMs) connect to data but failed to solve how much they could connect.

• The Constraint: LLMs have limited "Attention" and "Context Windows." Feeding a model 500 tool definitions (JSON Schemas) dilutes its ability to reason, increases hallucination rates, and drastically spikes token costs.
• The Solution: Progressive Tool Discovery (often implemented via the Tool Search Tool). This pattern decouples "Available Tools" from "Context Window Size," allowing an agent to access thousands of capabilities while keeping its working memory nearly empty.

2. MCP Foundations (The "Old" Way)

To understand the new pattern, we must first look at the foundational MCP architecture.

The Protocol Structure:

• Server: Provides the tools (e.g., a "GitHub" server provides get_issue, create_pr).
• Host: The application (e.g., Claude Desktop, Cursor, or a custom Agent) that connects the LLM to the Server.
• Client: The LLM itself.

The "Static Loading" Anti-Pattern: In standard MCP, the Host fetches every tool from the Server and dumps them all into the System Prompt at the start of the session.

Code Snippet: The "Static" Prompt Injection

(This is what happens behind the scenes in a standard setup. Note the heavy token load.)

// SYSTEM PROMPT (Standard MCP)
// The model is forced to "memorize" all of this before you even ask a question.
[
  { "name": "github_get_issue", "description": "Fetches issue details...", "parameters": {...} },
  { "name": "github_create_pr", "description": "Creates a new PR...", "parameters": {...} },
  // ... 48 more tools ...
  { "name": "jira_get_ticket", "description": "Retrieves Jira ticket...", "parameters": {...} }
]

Result: High latency, high cost, and "Context Bloat."

3. The "New" Concept: Progressive Tool Discovery

This architecture introduces a "Just-in-Time" loading mechanism. It fundamentally changes the relationship between the Registry (all available tools) and the Context (what the LLM sees).

3.1 The Architecture

1. The Registry (Off-Context): All 1,000+ tool definitions live here. This is a database or index on the Host/Client side. The LLM cannot see this.
2. The Context (In-Context): The LLM sees only one tool initially: tool_search.
3. The Trigger: When the LLM realizes it can't fulfill a request with its current tools, it uses tool_search to find what it needs.

3.2 The Core Mechanic: "Defer Loading"

Anthropic introduced a specific flag to support this. Tools are marked conceptually as defer_loading: true.

Code Snippet: The "Progressive" Prompt Injection

(This is what the LLM actually sees at the start of the chat. Note how clean it is.)

// SYSTEM PROMPT (Progressive Loading)
[
  {
    "name": "tool_search",
    "description": "Search for tools to perform tasks. Use this when you lack the capability to answer.",
    "parameters": {
      "query": { "type": "string", "description": "Keywords to find relevant tools." }
    }
  }
  // Note: All GitHub/Jira tools are MISSING from this list.
]

4. The Workflow: Search vs. Execution (The "Crux")

This section addresses the confusion regarding "redundancy." The architecture involves two distinct steps that use the same tool description for different purposes.

Step A: The "Blind" Search (Finding)

• User: "Check the status of ticket #123."
• LLM State: It has no idea what a "ticket" is. It only has tool_search.
• Action: LLM calls tool_search(query="ticket status").
• System Action (The Magic):
  1. The Host receives the text "ticket status".
  2. It runs a search algorithm (BM25 or Vector) against the External Registry.
  3. It finds jira_get_ticket because the description in the registry contains "ticket".

Step B: The Injection (Loading)

The Host takes the full schema of jira_get_ticket (which was previously hidden) and dynamically injects it into the current conversation history.

Step C: The Execution (Reasoning)

• LLM State: The context now contains jira_get_ticket.
• Action: The LLM reads the description of the just-loaded tool.
  • Why? To check parameter constraints (e.g., "Does this tool take an ID or a URL?" or "Use 'metric' for Celsius").
• Result: LLM calls jira_get_ticket(id="123").

Visual Flow

sequenceDiagram
    participant User
    participant LLM (Context)
    participant Host (Registry)
    
    Note over LLM (Context): Only knows "tool_search"
    User->>LLM (Context): "Check bug #123"
    LLM (Context)->>Host (Registry): Call tool_search("bug")
    Host (Registry)->>Host (Registry): Scans 1000 descriptions...
    Host (Registry)->>LLM (Context): Returns "jira_get_ticket" Schema
    
    Note over LLM (Context): Now knows "jira_get_ticket"
    LLM (Context)->>LLM (Context): Reads description to format params
    LLM (Context)->>Host (Registry): Call jira_get_ticket(id="123")

5. Addressing the "Redundancy" Confusion

Question: "Doesn't the search make the tool description redundant? If the search found it, why do we need to load the description again?"

Answer: No.

• Search uses the description to LOCATE the tool among thousands (Information Retrieval).
• Execution uses the description to INSTRUCT the model on how to use that specific tool correctly (Reasoning & Nuance).

The description is the "bridge" that exists in both the Search Index (for the machine to find) and the Context Window (for the intelligence to read). Without reading the description after loading, the LLM would know which function to call, but not how to fill out its complex parameters.

6. Technical Summary

Footnotes & References

1. "Advanced Tool Use": Terminology typically associated with Anthropic's late 2024/2025 engineering updates regarding agentic workflows.
2. "Defer Loading": The specific mechanism allowing schemas to be held back until requested.
3. Model Context Protocol (MCP): The open standard for connecting AI assistants to systems.